CN104574439A - Kalman filtering and TLD (tracking-learning-detection) algorithm integrated target tracking method - Google Patents

Kalman filtering and TLD (tracking-learning-detection) algorithm integrated target tracking method Download PDF

Info

Publication number
CN104574439A
CN104574439A CN201410819971.9A CN201410819971A CN104574439A CN 104574439 A CN104574439 A CN 104574439A CN 201410819971 A CN201410819971 A CN 201410819971A CN 104574439 A CN104574439 A CN 104574439A
Authority
CN
China
Prior art keywords
target
frame
state
tracking
kalman filtering
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201410819971.9A
Other languages
Chinese (zh)
Inventor
朱松豪
刘佳伟
胡荣林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Post and Telecommunication University
Nanjing University of Posts and Telecommunications
Original Assignee
Nanjing Post and Telecommunication University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Post and Telecommunication University filed Critical Nanjing Post and Telecommunication University
Priority to CN201410819971.9A priority Critical patent/CN104574439A/en
Publication of CN104574439A publication Critical patent/CN104574439A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24147—Distances to closest patterns, e.g. nearest neighbour classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory

Abstract

The invention discloses a Kalman filtering and TLD (tracking-learning-detection) algorithm integrated target tracking method. The method comprises steps as follows: firstly, the principle of a Kalman filter is analyzed, and the Kalman filter is improved; then a TLD algorithm is enhanced by the aid of the improved Kalman filter, and the reliability of a system is improved; finally, a detection module is improved by the aid of a prediction result obtained through the improved Kalman filter, the detection area of the TLD algorithm is reduced, and the tracking instantaneity is further improved.

Description

The method for tracking target of a kind of fusion card Kalman Filtering and TLD algorithm
Technical field
The present invention relates to the method for tracking target of a kind of fusion card Kalman Filtering and TLD algorithm, belong to technical field of image processing.
Background technology
The detection and tracking of moving target always is hot issue in computer vision research field, can arrive apply widely, as aspects such as intelligent monitoring, space flight and aviation, artificial intelligence, navigational guidances in national defence and civilian every field.In object detecting and tracking process, the challenge of many aspects can be faced, the movement velocity as the change of the change of the blocking of target, light, target trajectory, target is too fast, the appearance of similar purpose, complicated background etc.
Along with computing power ground constantly promotes, the ability of the large-scale datas such as process video image also with constantly promoting thereupon, thus makes object detecting and tracking technology obtain tremendous expansion.Nineteen fifty, first Gibson proposes to adopt optical flow approach, realizes by static state to dynamic image procossing.Along with going deep into of research, optical flow method has been difficult to meet system to real-time requirement, and thus, other methods many are arisen at the historic moment.As: Meanshift algorithm is paid attention to its printenv, advantage that calculated amount is little; Condesation algorithm make use of the thought of particle filter; VSAM algorithm realization carries out omni-directional video monitoring to complicated city.
In general, object detecting and tracking common at present is mainly divided into following a few class: (1) Knowledge based engineering method: although this type of algorithm solves the limitation of statistical model method, but have also been introduced certain problem simultaneously, as: checking difficulty, cost is comparatively large, needs to redefine and organization knowledge etc. in new scene.(2) based on the method for model: first such algorithm extracts target signature; Then, the spatial model of target is set up; Next, the feature in conjunction with these other parameters on target of characteristic sum is screened, and sets up effective original hypothesis; Finally, according to characteristic, target is predicted, after mark reaches specific threshold, then think that coupling effectively.(3) method of Corpus--based Method pattern: first this type of algorithm trains on a large scale to system, counts the distribution of clarification of objective; Then, space length is standard in mode, to characteristic matching differential count.Such algorithm be mainly used in target and periphery background single when.(4) based on the method for artificial neural network and expert system: this type of algorithm, by means of neural network, solves the indeterminable problem of traditional algorithm, but simultaneously because the real-time of neural network is not good, so the real-time of this type of algorithm is relatively poor.And the present invention can solve problem above well.
Summary of the invention
The object of the invention is to solve the deficiencies in the prior art, provide the method for tracking target of a kind of fusion card Kalman Filtering and TLD algorithm, the method is applied to Kalman filtering, the semi-supervised learning mechanism utilizing a kind of improvement, constantly detection and tracking module is upgraded, thus improve stability, robustness, the reliability of system.
The method for tracking target of fusion card Kalman Filtering of the present invention and TLD algorithm, is characterized in that, comprises following step:
Step 1: the principle analyzing and utilize Kalman filter, and it is improved.
The system that Kalman filter is used for modeling is linear system; Prediction and the noise measured are all white noises, noise Gaussian distributed, and Article 1 condition is the system state being obtained the k moment by the system state in k-1 moment and the product of a parameter matrix; All the other two conditions represent that noise does not change in time and changes, and can be just amplitude modeling exactly by covariance and average;
In real-time follow-up process, the time interval of two interframe is very little, and the target travel of consecutive frame can be considered linear; So just meet first condition in the large condition of Kalman filtering three, utilize the motion state of target in Kalman Filter Estimation video sequence, motion estimation process comprises the steps:
(1) motion-projection stage;
Suppose that the coordinate of moving target is for (x, y), movement velocity is (v x, v y), then the state s in k moment kfor (x, y, v x, v y) t, measuring state is z kfor (x k, y k), A kfor the transfer matrix in system state predictive equation is:
A k = 1 0 dt 0 0 1 0 dt 0 0 1 0 0 0 0 1 - - - ( 1 )
Owing to there is no input control variable in this system, therefore B kbe 0, measure equation H kin parameter matrix be:
H k = 1 0 0 0 0 1 0 0 - - - ( 2 )
Second and third two conditional request noises of Kalman filtering must be white noises, and Gaussian distributed, so the noise measured in equation and predictive equation is the white noise being Gaussian distributed, their covariance matrix P k, Q kdistribution is respectively:
P k = 1 0 0 1 , Q k = 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 - - - ( 3 )
In sum, the measurement equation of system is:
x k y k = 1 0 0 0 0 1 0 0 x k - 1 y k - 1 + 1 0 0 1 - - - ( 4 )
The predictive equation of system is:
x k y k v xk v yk = 1 0 dt 0 0 1 0 dt 0 0 1 0 0 0 0 1 x k - 1 y k - 1 v k - 1 v yk - 1 + 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 - - - ( 5 )
(2) move the more new stage;
The system estimation P utilizing forecast period to obtain kwith system state x k, constantly repeat prediction steps, until system finishing;
Kalman filtering, in metastable system, can obtain prediction effect; But when target travel uses Kalman filter with during randomness, often there is larger error in the prediction effect obtained; Need be improved appropriately Kalman filtering;
After the state in Kalman filtering moment on obtaining, determined the predicted value of current time state by state-transition matrix, state-transition matrix is also relevant with the state estimation in k moment simultaneously, and relatively accurate measured value can ensure the accuracy of predicted value;
If the k sometime from k to k+1 the moment tthere is state mutation, then from k to k 1the state-transition matrix of time period is the filter value in k moment, but k tit is the state of k+1 time observation to the transition matrix in the k+1 moment; Suppose that in k to the k+1 time period, the k+1/2 moment is intermediate time, then have following formula:
X ( k | k - 1 ) = A ( k , k - 1 2 ) * [ A ( k - 1 2 ) * X ( k - 1 ) ] - - - ( 6 )
Step 2: utilize the Kalman filter improved to strengthen TLD algorithm, improve the reliability of system.
Strengthen TLD algorithm by the Kalman filtering improved, namely according to the time step of target location, obtain the Parameter transfer matrix of synchronized update.Main process is described below: (1) initialization system: give ptcurrent and ptpredict by the center of initial frame, revise initial value and the initial predicted of Kalman filter system state simultaneously, and utilize this two numerical value initialized card Thalmann filters.(2) by Kalman filter, subsequent frame is predicted, and the system state that this moment is predicted is assigned to ptpredict; Meanwhile, center point LK optical flow method being followed the tracks of the object boundary frame obtained is assigned to ptcurrent.(3), before upgrading ptcurrent, preserve original system state with ptlast and measure.(4) with the status predication amount of renewal system and state measurement, Kalman filter is corrected, and the center point of target of prediction frame, adopt the ratio of width to height of the target frame originally had to restore frame.(5) confidence level of each frame target frame of Kalman filter prediction is utilized, namely the similarity of To Template in target frame and nearest neighbor classifier is calculated: if this value is greater than 0.85, then replace the detection frame result of LK optical flow method with this target frame, and this frame testing result is passed to follow-up study module and detection module; On the contrary, then the target frame of LK optical flow method gained is retained.If confidence level is less than 0.6 or after meeting certain frame number, then reinitialize Kalman filter by the frame testing result of Lk optical flow method.
Step 3: what utilize the Kalman filter improved to obtain predicts the outcome, and improves, reduce the surveyed area of TLD algorithm to detection module, improves the real-time of following the tracks of further.
According to the function of Kalman filtering, estimate the region that in present frame, target exists, using the object detection area of expected zone as TLD algorithm.
Arranged the position range of subwindow by the estimation results of improved Kalman filter device, detailed process is as described below: (1) estimates the center of moving target in present frame by the Kalman filter improved; (2) if the length breadth ratio of a certain rectangular area is consistent with the length breadth ratio of object boundary frame in previous frame, size is 4 times of previous frame bounding box, namely thinks that this rectangular area comprises detection target; (3) ask and all subwindows having common factor with this delimitation rectangular area, then this series of subwindow is sent into sorter, whether comprise target by detection of classifier subwindow.
After the state in Kalman filtering moment on obtaining, determined the predicted value of current time state by state-transition matrix, state-transition matrix is also relevant with the state estimation in k moment simultaneously, so relatively accurate measured value can ensure the accuracy of predicted value.
Beneficial effect:
1, Kalman filtering of the present invention adds the sampling rate of system within the time period of system sudden change, ensure that the confidence level predicted the outcome, and only adopts the method when undergoing mutation, and little on the tracking velocity impact of system.
2, invention increases the stability of system, robustness, reliability.
Accompanying drawing explanation
Fig. 1 is TLD algorithm frame figure of the present invention.
Fig. 2 is the process flow diagram of optical flow method of the present invention.
Fig. 3 is that Kalman filter of the present invention strengthens TLD algorithm frame figure.
Fig. 4 is cascade classifier schematic diagram of the present invention.
Embodiment
Below in conjunction with Figure of description, the invention is described in further detail.
As Figure 1-Figure 4, the present invention proposes the method for tracking target of a kind of fusion card Kalman Filtering and TLD algorithm, first the method is analyze and utilize the principle of Kalman filter, and is improved it; Then, utilize the Kalman filter improved to strengthen TLD algorithm, improve the reliability of system; Finally, what utilize the Kalman filter improved to obtain predicts the outcome, and improves, reduce the surveyed area of TLD algorithm to detection module, improves the real-time of following the tracks of further.
Specific embodiment of the invention process comprises the following steps:
Step 1: the principle analyzing and utilize Kalman filter, and it is improved.
The realization of Kalman filtering must meet three important conditions: (1) is linear system for the system of modeling; (2) noise of prediction and measurement is all white noise; (3) noise Gaussian distributed.Article 1, condition is the system state being obtained the k moment by the system state in k-1 moment and the product of a parameter matrix; All the other two conditions represent that noise does not change in time and changes, so can be just amplitude modeling exactly by covariance and average.Although above three conditions are comparatively harsh, Kalman filtering in actual applications widely.
Due in real-time follow-up process, the time interval of two interframe is very little, so the target travel of consecutive frame can be considered linear.So just meet first condition in the large condition of Kalman filtering three, therefore can utilize the motion state of target in Kalman Filter Estimation video sequence.Concrete motion estimation process comprises following two steps:
(1) motion-projection stage
Suppose that the coordinate of moving target is for (x, y), movement velocity is (v x, v y), then the state s in k moment kfor (x, y, v x, v y) t, measuring state is z kfor (x k, y k).A kfor the transfer matrix in system state predictive equation is:
A k = 1 0 dt 0 0 1 0 dt 0 0 1 0 0 0 0 1 - - - ( 1 )
Owing to there is no input control variable in this system, therefore B kbe 0.Measure equation H kin parameter matrix be:
H k = 1 0 0 0 0 1 0 0 - - - ( 2 )
Second and third two conditional request noises of Kalman filtering must be white noises, and Gaussian distributed, so the noise measured in equation and predictive equation is the white noise being Gaussian distributed, their covariance matrix P k, Q kdistribution is respectively:
P k = 1 0 0 1 , Q k = 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 - - - ( 3 )
In sum, the measurement equation of system is:
x k y k = 1 0 0 0 0 1 0 0 x k - 1 y k - 1 + 1 0 0 1 - - - ( 4 )
The predictive equation of system is:
x k y k v xk v yk = 1 0 dt 0 0 1 0 dt 0 0 1 0 0 0 0 1 x k - 1 y k - 1 v xk - 1 v yk - 1 + 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 - - - ( 5 )
(2) move the more new stage
The system estimation P utilizing forecast period to obtain kwith system state x k, constantly repeat prediction steps, until system finishing.
Kalman filtering, in metastable system, can obtain good prediction effect; But when target travel uses Kalman filter with during randomness, often there is larger error in the prediction effect obtained.Therefore, need be improved appropriately Kalman filtering.
After the state in Kalman filtering moment on obtaining, determined the predicted value of current time state by state-transition matrix, state-transition matrix is also relevant with the state estimation in k moment simultaneously, so relatively accurate measured value can ensure the accuracy of predicted value.
Suppose the k sometime from k to k+1 the moment tthere is state mutation, then from k to k 1the state-transition matrix of time period is the filter value in k moment, but k tit is the state of k+1 time observation to the transition matrix in the k+1 moment.The present invention supposes that in k to the k+1 time period, the k+1/2 moment is intermediate time, then have following formula:
X ( k | k - 1 ) = A ( k , k - 1 2 ) * [ A ( k - 1 2 ) * X ( k - 1 ) ] - - - ( 6 )
Kalman filtering after improvement adds the sampling rate of system within the time period of system sudden change, ensure that the confidence level predicted the outcome, and only adopts the method when undergoing mutation.Therefore, little on the tracking velocity impact of system.
Step 2: utilize the Kalman filter improved to strengthen TLD algorithm, improve the reliability of system.
TLD algorithm is a kind of long-time track algorithm of single goal efficiently, the major advantage of this algorithm is only need obtain little prior imformation, just can carry out long on-line tracing to target, and real-time is high, fast operation, stability strong, can also be applicable to the scene such as target occlusion, disappearance occurs.Just as the name implies, TLD algorithm mainly contains three module compositions: tracking module (Tracking), study module (Learning), detection module (Detecting), wherein the function of tracking module is by a short-period adaptive tracing device, when the finiteness of moving between frames, observability, the movement tendency of target of prediction; The function of study module evaluates the performance of tracking module and detection module, and the effective training set utilizing algorithm to generate is to upgrade detection module, eliminates metrical error; The function of detection module detects in real time target, corrects tracker where necessary simultaneously.
The framework of TLD algorithm as shown in Figure 1.First, set position and the size of target in the first frame, complete the initialization of TLD algorithm; Then, in tracing process, detection module and tracking module process each two field picture jointly, wherein tracking module predicts the position of same target in present frame according to the positional information of target in former frame, detection module then carries out whole scan to detect target to present frame, and fusion tracking result and testing result, provide present frame and whether there is the information such as whether moving target, target position and pursuit path effective; Finally, testing result and tracking results are inputed to study module, determine whether tracking module and detection module are upgraded by study module.
The present invention makes improvements on the basis of original TLD algorithm, thus obtains more preferably tracking effect.Improvement involved in the present invention has following two aspects: (1) improves tracking module, carries out certain improvement based on the Kalman filter principle improved to tracker.Simultaneously according to each time step of target location, calculate and undated parameter transfer matrix F; (2) detection module is improved.According to the function of Kalman filtering, estimate the region that in present frame, target exists, using the object detection area of expected zone as TLD algorithm, thus decrease the sensing range of TLD algorithm, to scan for the image subwindow of global image relative to detection module in former TLD algorithm, substantially increase the efficiency of detection.
TLD algorithm keeps track module is based on Lucas-Kanede (LK) method, i.e. optical flow method.Light stream refers to the speed of each pixel motion of image, and it is the projection of three dimensional velocity vectors on imaging plane of visible point in scenery, represents the instantaneous variation of scenery surface point position in the picture.
LK optical flow method has specific three conditions, is respectively: (1) brightness constancy, and namely the brightness of same pixel between different frame remains unchanged; (2) time continuity, namely the time dependent speed of picture material is slowly; (3) Space Consistency, the point namely closed in scene, its projection on image is also neighbor point, and the speed of neighbor point is consistent.As shown in Figure 2, its principle of work is as described below for the flow process of LK optical flow method.First, at image I ttarget frame β tmiddle random selecting 10 × 10 point, uses x t i(i=1...100) represent; Then, by LK optical flow algorithm, these 100 points are followed the tracks of, obtain next frame image I t+1in these 100 corresponding point x t+1 iposition, and obtain the forward direction-backward error of these 100 points according to formula (7), therefrom select part point as credible trace point; Finally, target frame β is obtained t+1at image I t+1in size.
Wherein T k f=(x t, x t+1..., x t+k) be forward direction track, T k b=(x' t, x' t+1..., x' t+k) be back trajectca-rles, and x ' t+k=x t+k.
If the forward direction in t+1 moment-backward error FB (T k f| S t+ 1) forward direction-backward error FB (T of t is less than k f| S t), then this point is that trace point is credible; Otherwise forward direction pursuit path is incorrect, then delete this point.
Formula (8) provides the range rate of front and back two frame:
r = | | x i t + 1 - x j t + 1 | | | | x i t - x j t | | , i , j = 1,2 , . . . , n . i ≠ j - - - ( 8 )
Wherein, n is credible trace point number, and r is an x t iwith an x i t+1range rate in the frame of front and back two.Target frame β t+1size s t+1can by target frame β tsize s tobtain:
s t+1＝s t*r m(9)
Wherein r mfor average range rate.
Following the tracks of the key factor of study, is whether tracking module can correct tracking target.With this simultaneously, confidence level calculating can be carried out to the target location that detection module and tracking module obtain in Fusion Module, thus determine the final position of target.Therefore, the accuracy of tracking module need be improved.
The present invention strengthens TLD algorithm by the Kalman filtering improved, and namely according to the time step of target location, obtains the Parameter transfer matrix of synchronized update.The flow process of algorithm of the present invention as shown in Figure 3, mainly comprise as follows: (1) initialization system: give ptcurrent and ptpredict by the center of initial frame, revise initial value and the initial predicted of Kalman filter system state simultaneously, and utilize this two numerical value initialized card Thalmann filters.(2) by Kalman filter, subsequent frame is predicted, and the system state that this moment is predicted is assigned to ptpredict; Meanwhile, center point LK optical flow method being followed the tracks of the object boundary frame obtained is assigned to ptcurrent.(3), before upgrading ptcurrent, preserve original system state with ptlast and measure.(4) with the status predication amount of renewal system and state measurement, Kalman filter is corrected, and the center point of target of prediction frame, adopt the ratio of width to height of the target frame originally had to restore frame.(5) confidence level of each frame target frame of Kalman filter prediction is utilized, namely the similarity of To Template in target frame and nearest neighbor classifier is calculated: if this value is greater than 0.85, then replace the detection frame result of LK optical flow method with this target frame, and this frame testing result is passed to follow-up study module and detection module; On the contrary, then the target frame of LK optical flow method gained is retained.If confidence level is less than 0.6 or after meeting certain frame number, then reinitialize Kalman filter by the frame testing result of Lk optical flow method.
Step 3: what utilize the Kalman filter improved to obtain predicts the outcome, and improves, reduce the surveyed area of TLD algorithm to detection module, improves the real-time of following the tracks of further.
Whether the detection module of TLD algorithm is by existing target in scanning window detected image.Its scanning window optimum configurations is as follows: the step-size factor of scaling is 1.2, and horizontal step-length is 10% of width, and vertical step-length is 10% of height, and minimum scanning window size is 20 pixels.
For improving accuracy of detection, introduce a three-stage cascade sorter as shown in Figure 4 in detection module, respectively: (1) classification of equation device; (2) random forest sorter; (3) nearest neighbor classifier.The image block do not satisfied condition is removed by each stage, the image block satisfied condition is sent into next sorter simultaneously, which not only improves the efficiency of sorter, also improve the stability of sorter simultaneously.
Although detection module has very effective sorter, detection module needs the subwindow to target likely occurs to scan, to confirm whether target exists in image; And TLD algorithm specify only the magnitude range of subwindow, do not specify the position range of subwindow.So TLD algorithm needs to detect a large amount of windows, and in these subwindows, the overwhelming majority does not comprise object content in fact, therefore wastes a large amount of computational resources.For this reason, the present invention arranges the position range of subwindow by the estimation results of improved Kalman filter device, and detailed process is as described below: (1) estimates the center of moving target in present frame by the Kalman filter improved; (2) if the length breadth ratio of a certain rectangular area is consistent with the length breadth ratio of object boundary frame in previous frame, size is 4 times of previous frame bounding box, namely thinks that this rectangular area comprises detection target; (3) ask and all subwindows having common factor with this delimitation rectangular area, then this series of subwindow is sent into sorter, whether comprise target by detection of classifier subwindow.
The present invention makes improvements on the basis of original TLD algorithm, thus obtains more preferably tracking effect.Improvement of the present invention comprises following two aspects: (1) improves tracking module.Based on the Kalman filter principle improved, certain improvement is carried out to tracker.Simultaneously according to each time step of target location, calculate and undated parameter transfer matrix F; (2) detection module is improved.According to the function of Kalman filtering, estimate the region that in present frame, target exists, using the object detection area of expected zone as TLD algorithm, thus decrease the sensing range of TLD algorithm, to scan for the image subwindow of global image relative to detection module in former TLD algorithm, substantially increase the efficiency of detection.
Following the tracks of the key factor of study, is whether tracking module can correct tracking target.With this simultaneously, confidence level calculating can be carried out to the target location that detection module and tracking module obtain in Fusion Module, thus determine the final position of target.Therefore, the accuracy of tracking module need be improved.
The present invention strengthens TLD algorithm by the Kalman filtering improved, and namely according to the time step of target location, obtains the Parameter transfer matrix of synchronized update.Algorithm flow of the present invention mainly comprises: (1) initialization system: give ptcurrent and ptpredict by the center of initial frame, revise initial value and the initial predicted of Kalman filter system state simultaneously, and utilize this two numerical value initialized card Thalmann filters.(2) by Kalman filter, subsequent frame is predicted, and the system state that this moment is predicted is assigned to ptcurrent.(3), before upgrading ptcurrent, original ptpredict is preserved with ptlast; Meanwhile, center point LK optical flow method being followed the tracks of the object boundary frame obtained is assigned to system state and measures.(4) with the status predication amount of renewal system and state measurement, Kalman filter is corrected, and the center point of target of prediction frame, adopt the ratio of width to height of the target frame originally had to restore frame.(5) confidence level of each frame target frame of Kalman filter prediction is utilized, namely the similarity of To Template in target frame and nearest neighbor classifier is calculated: if this value is greater than 0.85, then replace the detection frame result of LK optical flow method with this target frame, and this frame testing result is passed to follow-up study module and detection module; On the contrary, then the target frame of LK optical flow method gained is retained.If confidence level is less than 0.6 or after meeting certain frame number, then reinitialize Kalman filter by the frame testing result of Lk optical flow method.
According to the function of Kalman filtering, estimate the region that in present frame, target exists, using the object detection area of expected zone as TLD algorithm, thus decrease the sensing range of TLD algorithm, to scan for the image subwindow of global image relative to detection module in former TLD algorithm, substantially increase the efficiency of detection.
Although detection module has very effective sorter, detection module needs the subwindow to target likely occurs to scan, to confirm whether target exists in image; And TLD algorithm specify only the magnitude range of subwindow, do not specify the position range of subwindow.So TLD algorithm needs to detect a large amount of windows, and in these subwindows, the overwhelming majority does not comprise object content in fact, therefore wastes a large amount of computational resources.For this reason, the present invention arranges the position range of subwindow by the estimation results of improved Kalman filter device, and detailed process is as described below: (1) estimates the center of moving target in present frame by the Kalman filter improved; (2) if the length breadth ratio of a certain rectangular area is consistent with the length breadth ratio of object boundary frame in previous frame, size is 4 times of previous frame bounding box, namely thinks that this rectangular area comprises detection target; (3) ask and all subwindows having common factor with this delimitation rectangular area, then this series of subwindow is sent into sorter, whether comprise target by detection of classifier subwindow.

Claims (8)

1. a method for tracking target for fusion card Kalman Filtering and TLD algorithm, is characterized in that, described method comprises the steps:
Step 1: the principle analyzing and utilize Kalman filter, and it is improved;
The system that Kalman filter is used for modeling is linear system; Prediction and the noise measured are all white noises, noise Gaussian distributed;
Step 2: utilize the Kalman filter improved to strengthen TLD algorithm;
Strengthen TLD algorithm by the Kalman filtering improved, namely according to the time step of target location, obtain the Parameter transfer matrix of synchronized update;
Step 3: what utilize the Kalman filter improved to obtain predicts the outcome, and improves, reduce the surveyed area of TLD algorithm to detection module;
According to the function of Kalman filtering, estimate the region that in present frame, target exists, using the object detection area of expected zone as TLD algorithm;
The position range of subwindow is set by the estimation results of improved Kalman filter device.
2. the method for tracking target of a kind of fusion card Kalman Filtering according to claim 1 and TLD algorithm, it is characterized in that, described method step 1 comprises: Article 1 condition is the system state being obtained the k moment by the system state in k-1 moment and the product of a parameter matrix; All the other two conditions represent that noise does not change in time and changes, and can be just amplitude modeling exactly by covariance and average;
In real-time follow-up process, the time interval of two interframe is very little, and the target travel of consecutive frame can be considered linear; So just meet first condition in the large condition of Kalman filtering three, utilize the motion state of target in Kalman Filter Estimation video sequence, motion estimation process comprises the steps:
(1) motion-projection stage;
Suppose that the coordinate of moving target is for (x, y), movement velocity is (v x, v y), then the state s in k moment kfor (x, y, v x, v y) t, measuring state is z kfor (x k, y k), A kfor the transfer matrix in system state predictive equation is:
A k = 1 0 dt 0 0 1 0 dt 0 0 1 0 0 0 0 1 - - - ( 1 )
Owing to there is no input control variable in this system, therefore B kbe 0, measure equation H kin parameter matrix be:
H k = 1 0 0 0 0 1 0 0 - - - ( 2 )
Second and third two conditional request noises of Kalman filtering must be white noises, and Gaussian distributed, so the noise measured in equation and predictive equation is the white noise being Gaussian distributed, their covariance matrix P k, Q kdistribution is respectively:
P k = 1 0 0 1 , Q k = 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 - - - ( 3 )
The measurement equation of system is:
x k y k = 1 0 0 0 0 1 0 0 x k - 1 y k - 1 + 1 0 0 1 - - - ( 4 )
The predictive equation of system is:
x k y k v xk v yk = 1 0 dt 0 0 1 0 dt 0 0 1 0 0 0 0 1 x k - 1 y k - 1 v xk - 1 v yk - 1 + 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 - - - ( 5 )
(2) move the more new stage;
The system estimation P utilizing forecast period to obtain kwith system state x k, constantly repeat prediction steps, until system finishing;
Kalman filtering, in metastable system, can obtain prediction effect; But when target travel uses Kalman filter with during randomness, often there is larger error in the prediction effect obtained; Need be improved appropriately Kalman filtering;
After the state in Kalman filtering moment on obtaining, determined the predicted value of current time state by state-transition matrix, state-transition matrix is also relevant with the state estimation in k moment simultaneously, and relatively accurate measured value can ensure the accuracy of predicted value;
If the k sometime from k to k+1 the moment tthere is state mutation, then from k to k 1the state-transition matrix of time period is the filter value in k moment, but k tit is the state of k+1 time observation to the transition matrix in the k+1 moment; Suppose that in k to the k+1 time period, the k+1/2 moment is intermediate time, then have following formula:
X ( k | k - 1 ) = A ( k , k - 1 2 ) * [ A ( k - 1 2 ) * X ( k - 1 ) ] - - - ( 6 )
。
3. the method for tracking target of a kind of fusion card Kalman Filtering according to claim 1 and TLD algorithm, it is characterized in that, described method step 2 comprises: (1) initialization system, ptcurrent and ptpredict is given by the center of initial frame, revise initial value and the initial predicted of Kalman filter system state simultaneously, and utilize this two numerical value initialized card Thalmann filters; (2) by Kalman filter, subsequent frame is predicted, and the system state that this moment is predicted is assigned to ptpredict; Meanwhile, center point LK optical flow method being followed the tracks of the object boundary frame obtained is assigned to ptcurrent; (3), before upgrading ptcurrent, preserve original system state with ptlast and measure; (4) with the status predication amount of renewal system and state measurement, Kalman filter is corrected, and the center point of target of prediction frame, adopt the ratio of width to height of the target frame originally had to restore frame; (5) confidence level of each frame target frame of Kalman filter prediction is utilized, namely the similarity of To Template in target frame and nearest neighbor classifier is calculated: if this value is greater than 0.85, then replace the detection frame result of LK optical flow method with this target frame, and this frame testing result is passed to follow-up study module and detection module; On the contrary, then the target frame of LK optical flow method gained is retained; If confidence level is less than 0.6 or after meeting certain frame number, then reinitialize Kalman filter by the frame testing result of Lk optical flow method.
4. the method for tracking target of a kind of fusion card Kalman Filtering according to claim 1 and TLD algorithm, it is characterized in that, the TLD algoritic module of described method step 2 comprises: tracking module, study module, detection module, wherein the function of tracking module is by a short-period adaptive tracing device, when the finiteness of moving between frames, observability, the movement tendency of target of prediction; The function of study module evaluates the performance of tracking module and detection module, and the effective training set utilizing algorithm to generate is to upgrade detection module, eliminates metrical error; The function of detection module detects in real time target, corrects tracker where necessary simultaneously.
5. the method for tracking target of a kind of fusion card Kalman Filtering according to claim 1 and TLD algorithm, is characterized in that, the TLD algorithm of described method step 2 comprises: first, sets position and the size of target in the first frame, completes the initialization of TLD algorithm; Then, in tracing process, detection module and tracking module process each two field picture jointly, wherein tracking module predicts the position of same target in present frame according to the positional information of target in former frame, detection module then carries out whole scan to detect target to present frame, and fusion tracking result and testing result, provide present frame and whether there is the information such as whether moving target, target position and pursuit path effective; Finally, testing result and tracking results are inputed to study module, determine whether tracking module and detection module are upgraded by study module.
6. the method for tracking target of a kind of fusion card Kalman Filtering according to claim 1 and TLD algorithm, is characterized in that, described method step 3 comprises: (1) estimates the center of moving target in present frame by the Kalman filter improved; (2) if the length breadth ratio of a certain rectangular area is consistent with the length breadth ratio of object boundary frame in previous frame, size is 4 times of previous frame bounding box, namely thinks that this rectangular area comprises detection target; (3) ask and all subwindows having common factor with this delimitation rectangular area, then this series of subwindow is sent into sorter, whether comprise target by detection of classifier subwindow.
7. the method for tracking target of a kind of fusion card Kalman Filtering according to claim 1 and TLD algorithm, is characterized in that, the detection module described in described method step 3 comprises: classification of equation device; Random forest sorter; Nearest neighbor classifier; The image block do not satisfied condition is removed by each stage, the image block satisfied condition is sent into next sorter simultaneously.
8. the method for tracking target of a kind of fusion card Kalman Filtering according to claim 1 and TLD algorithm, is characterized in that: described method is applied to Kalman filtering.
CN201410819971.9A 2014-12-25 2014-12-25 Kalman filtering and TLD (tracking-learning-detection) algorithm integrated target tracking method Pending CN104574439A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410819971.9A CN104574439A (en) 2014-12-25 2014-12-25 Kalman filtering and TLD (tracking-learning-detection) algorithm integrated target tracking method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410819971.9A CN104574439A (en) 2014-12-25 2014-12-25 Kalman filtering and TLD (tracking-learning-detection) algorithm integrated target tracking method

Publications (1)

Publication Number Publication Date
CN104574439A true CN104574439A (en) 2015-04-29

Family

ID=53090407

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410819971.9A Pending CN104574439A (en) 2014-12-25 2014-12-25 Kalman filtering and TLD (tracking-learning-detection) algorithm integrated target tracking method

Country Status (1)

Country Link
CN (1) CN104574439A (en)

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105844664A (en) * 2016-03-21 2016-08-10 辽宁师范大学 Monitoring video vehicle detection tracking method based on improved TLD
CN106204649A (en) * 2016-07-05 2016-12-07 西安电子科技大学 A kind of method for tracking target based on TLD algorithm
CN106204644A (en) * 2016-07-01 2016-12-07 屈桢深 A kind of target long-term follow method based on video
CN106373146A (en) * 2016-10-27 2017-02-01 南京航空航天大学 Target tracking method based on fuzzy learning
CN106971195A (en) * 2017-02-27 2017-07-21 天津大学 A kind of 3D method for checking object based on detector and tracker
CN107145167A (en) * 2017-04-07 2017-09-08 南京邮电大学 A kind of video target tracking method based on digital image processing techniques
CN107292911A (en) * 2017-05-23 2017-10-24 南京邮电大学 A kind of multi-object tracking method merged based on multi-model with data correlation
CN108022258A (en) * 2017-10-20 2018-05-11 南京邮电大学 Real-time multi-target tracking based on the more frame detectors of single and Kalman filtering
CN108446722A (en) * 2018-03-06 2018-08-24 中国船舶重工集团公司第七二四研究所 A kind of long-acting tenacious tracking method of target based on radar video data distribution characteristics
CN108765450A (en) * 2018-05-21 2018-11-06 南京邮电大学 intelligent robot visual tracking method based on deep learning
CN108805901A (en) * 2018-05-04 2018-11-13 北京航空航天大学 A kind of quick detecting and tracking parallel computation of sensation target based on multi-core DSP and fusion method
CN109118787A (en) * 2018-08-20 2019-01-01 浙江工业大学 A kind of car speed prediction technique based on deep neural network
CN109360163A (en) * 2018-09-26 2019-02-19 深圳积木易搭科技技术有限公司 A kind of fusion method and emerging system of high dynamic range images
CN109636834A (en) * 2018-11-22 2019-04-16 北京工业大学 Video frequency vehicle target tracking algorism based on TLD innovatory algorithm
CN109840919A (en) * 2019-01-21 2019-06-04 长安大学 One kind being based on the improved tracking of TLD
CN110033050A (en) * 2019-04-18 2019-07-19 杭州电子科技大学 A kind of water surface unmanned boat real-time target detection calculation method
CN110827313A (en) * 2019-09-19 2020-02-21 深圳云天励飞技术有限公司 Fast optical flow tracking method and related equipment
CN111160212A (en) * 2019-12-24 2020-05-15 浙江大学 Improved tracking learning detection system and method based on YOLOv3-Tiny
CN111914690A (en) * 2020-07-15 2020-11-10 西安米克斯智能技术有限公司 Method for tracking target object in video recognition for medium and long periods
US11560657B2 (en) 2020-12-02 2023-01-24 Industrial Technology Research Institute Braiding path generating method and device using the same, and dynamic correcting method and braiding system using the same

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102881024A (en) * 2012-08-24 2013-01-16 南京航空航天大学 Tracking-learning-detection (TLD)-based video object tracking method

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102881024A (en) * 2012-08-24 2013-01-16 南京航空航天大学 Tracking-learning-detection (TLD)-based video object tracking method

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
周鑫等: "改进后的 TLD 视频目标跟踪方法", 《中国图象图形学报》 *
张甜等: "一种基于改进卡尔曼滤波的 camshift 目标跟踪算法", 《测试技术学报》 *
江博: "基于Kalman的TLD目标跟踪算法研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 *

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105844664B (en) * 2016-03-21 2019-01-11 辽宁师范大学 Based on the monitor video vehicle detecting and tracking method for improving TLD
CN105844664A (en) * 2016-03-21 2016-08-10 辽宁师范大学 Monitoring video vehicle detection tracking method based on improved TLD
CN106204644A (en) * 2016-07-01 2016-12-07 屈桢深 A kind of target long-term follow method based on video
CN106204649A (en) * 2016-07-05 2016-12-07 西安电子科技大学 A kind of method for tracking target based on TLD algorithm
CN106204649B (en) * 2016-07-05 2019-06-28 西安电子科技大学 A kind of method for tracking target based on TLD algorithm
CN106373146A (en) * 2016-10-27 2017-02-01 南京航空航天大学 Target tracking method based on fuzzy learning
CN106373146B (en) * 2016-10-27 2019-02-15 南京航空航天大学 A kind of method for tracking target based on fuzzy learning
CN106971195A (en) * 2017-02-27 2017-07-21 天津大学 A kind of 3D method for checking object based on detector and tracker
CN107145167A (en) * 2017-04-07 2017-09-08 南京邮电大学 A kind of video target tracking method based on digital image processing techniques
CN107292911A (en) * 2017-05-23 2017-10-24 南京邮电大学 A kind of multi-object tracking method merged based on multi-model with data correlation
CN107292911B (en) * 2017-05-23 2021-03-30 南京邮电大学 Multi-target tracking method based on multi-model fusion and data association
CN108022258B (en) * 2017-10-20 2020-07-03 南京邮电大学 Real-time multi-target tracking method based on single multi-frame detector and Kalman filtering
CN108022258A (en) * 2017-10-20 2018-05-11 南京邮电大学 Real-time multi-target tracking based on the more frame detectors of single and Kalman filtering
CN108446722A (en) * 2018-03-06 2018-08-24 中国船舶重工集团公司第七二四研究所 A kind of long-acting tenacious tracking method of target based on radar video data distribution characteristics
CN108446722B (en) * 2018-03-06 2021-04-30 中国船舶重工集团公司第七二四研究所 Target long-term stable tracking method based on radar video data distribution characteristics
CN108805901A (en) * 2018-05-04 2018-11-13 北京航空航天大学 A kind of quick detecting and tracking parallel computation of sensation target based on multi-core DSP and fusion method
CN108805901B (en) * 2018-05-04 2022-02-22 北京航空航天大学 Visual target rapid detection tracking parallel computing and fusion method based on multi-core DSP
CN108765450A (en) * 2018-05-21 2018-11-06 南京邮电大学 intelligent robot visual tracking method based on deep learning
CN109118787A (en) * 2018-08-20 2019-01-01 浙江工业大学 A kind of car speed prediction technique based on deep neural network
CN109360163A (en) * 2018-09-26 2019-02-19 深圳积木易搭科技技术有限公司 A kind of fusion method and emerging system of high dynamic range images
CN109636834A (en) * 2018-11-22 2019-04-16 北京工业大学 Video frequency vehicle target tracking algorism based on TLD innovatory algorithm
CN109840919A (en) * 2019-01-21 2019-06-04 长安大学 One kind being based on the improved tracking of TLD
CN110033050B (en) * 2019-04-18 2021-06-22 杭州电子科技大学 Real-time target detection and calculation method for unmanned surface vehicle
CN110033050A (en) * 2019-04-18 2019-07-19 杭州电子科技大学 A kind of water surface unmanned boat real-time target detection calculation method
CN110827313A (en) * 2019-09-19 2020-02-21 深圳云天励飞技术有限公司 Fast optical flow tracking method and related equipment
CN110827313B (en) * 2019-09-19 2023-03-03 深圳云天励飞技术股份有限公司 Fast optical flow tracking method and related equipment
CN111160212A (en) * 2019-12-24 2020-05-15 浙江大学 Improved tracking learning detection system and method based on YOLOv3-Tiny
CN111914690A (en) * 2020-07-15 2020-11-10 西安米克斯智能技术有限公司 Method for tracking target object in video recognition for medium and long periods
CN111914690B (en) * 2020-07-15 2023-11-10 西安米克斯智能技术有限公司 Target object medium-long-term tracking method in video identification
US11560657B2 (en) 2020-12-02 2023-01-24 Industrial Technology Research Institute Braiding path generating method and device using the same, and dynamic correcting method and braiding system using the same

Similar Documents

Publication Publication Date Title
CN104574439A (en) Kalman filtering and TLD (tracking-learning-detection) algorithm integrated target tracking method
Wang et al. Automatic laser profile recognition and fast tracking for structured light measurement using deep learning and template matching
CN106846359B (en) Moving target rapid detection method based on video sequence
CN102881022B (en) Concealed-target tracking method based on on-line learning
CN105405151B (en) Anti-shelter target tracking based on particle filter and weighting Surf
CN102542289B (en) Pedestrian volume statistical method based on plurality of Gaussian counting models
CN104820997B (en) A kind of method for tracking target based on piecemeal sparse expression Yu HSV Feature Fusion
CN104091348A (en) Multi-target tracking method integrating obvious characteristics and block division templates
CN104616318A (en) Moving object tracking method in video sequence image
CN103886325B (en) Cyclic matrix video tracking method with partition
CN102142085B (en) Robust tracking method for moving flame target in forest region monitoring video
CN104318263A (en) Real-time high-precision people stream counting method
CN105654516B (en) Satellite image based on target conspicuousness is to ground weak moving target detection method
CN102063625B (en) Improved particle filtering method for multi-target tracking under multiple viewing angles
CN102722702B (en) Multiple feature fusion based particle filter video object tracking method
CN103488993A (en) Crowd abnormal behavior identification method based on FAST
CN104299243A (en) Target tracking method based on Hough forests
CN102903121A (en) Fusion algorithm based on moving target tracking
CN111797785B (en) Multi-aircraft tracking method based on deep learning
Yaghoobi Ershadi et al. Vehicle tracking and counting system in dusty weather with vibrating camera conditions
He et al. A novel multi-source vehicle detection algorithm based on deep learning
Liu et al. Self-correction ship tracking and counting with variable time window based on YOLOv3
CN112164093A (en) Automatic person tracking method based on edge features and related filtering
CN104091352A (en) Visual tracking method based on structural similarity
Qing et al. A novel particle filter implementation for a multiple-vehicle detection and tracking system using tail light segmentation

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20150429

RJ01 Rejection of invention patent application after publication