CN107403439B - Cam-shift-based prediction tracking method - Google Patents

Cam-shift-based prediction tracking method Download PDF

Info

Publication number
CN107403439B
CN107403439B CN201710419084.6A CN201710419084A CN107403439B CN 107403439 B CN107403439 B CN 107403439B CN 201710419084 A CN201710419084 A CN 201710419084A CN 107403439 B CN107403439 B CN 107403439B
Authority
CN
China
Prior art keywords
target
cam
tracking
shift algorithm
algorithm
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710419084.6A
Other languages
Chinese (zh)
Other versions
CN107403439A (en
Inventor
刘振宇
张百颖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenyang University of Technology
Original Assignee
Shenyang University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenyang University of Technology filed Critical Shenyang University of Technology
Priority to CN201710419084.6A priority Critical patent/CN107403439B/en
Publication of CN107403439A publication Critical patent/CN107403439A/en
Application granted granted Critical
Publication of CN107403439B publication Critical patent/CN107403439B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/207—Analysis of motion for motion estimation over a hierarchy of resolutions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20024—Filtering details

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The method provides a method for combining Kalman filtering and a Cam-shift algorithm, and provides a target prediction tracking method combining linear prediction and the Cam-shift algorithm, namely, the linear prediction method replaces the Kalman filtering to complete prediction estimation, and a prediction estimation result is substituted into the Cam-shift algorithm for tracking. In order to well complete the position prediction tracking of a moving target under a complex background interfered by factors such as shielding and the like, the method researches a method combining Kalman filtering and a Cam-shift algorithm. Kalman filtering can accurately complete the position prediction of a moving target, and can well complete tracking by combining with a Cam-shift algorithm. On the basis, a target prediction tracking method combining linear prediction and a Cam-shift algorithm is provided, namely the linear prediction method replaces Kalman filtering to complete prediction estimation, and the prediction estimation result is substituted into the Cam-shift algorithm for tracking. Experiments show that the method can ensure the tracking accuracy in the shielding process, reduce the iteration time and better meet the requirement of real-time performance.

Description

Cam-shift-based prediction tracking method
The technical field is as follows:
the invention relates to a Cam-shift-based prediction tracking method.
Background art:
the tracking and prediction of the moving target locus is an indispensable part in the artificial intelligence research field and is a key technology for machine vision and target tracking detection, and different application backgrounds comprise different tracking methods of L-K (L ucas)&Kanade) optical flow method and H-S (Horn)&Schunck) optical flow method does not need to consider background information, and by endowing a speed vector to a pixel point, a target is tracked according to the difference of the target and the background speed vector, but the brightness between adjacent frames is required to be constant; the graph cut algorithm maps a target tracking problem into an energy function, optimizes and solves a target area, and is complex in calculation; snake model algorithm proposed by Kass[3-4]Is a tracking method based on the edge characteristics of the contour, the energy utilization of which is minimum, but the contour initialization is complexMixing; dongchun et al put forward an adaptive nonlinear filtering algorithm by combining particle filtering and GVF-Snake to track moving and deformed targets, but the efficiency is low; and Yizong Cheng[6]The proposed Mean-shift algorithm is a density gradient non-parameter estimation algorithm based on color features, is simple in form and easy to operate, does not have a function of updating a search window in real time, is quick in scale change of a moving target, and is easy to cause target tracking loss; the Cam-shift algorithm provided by Bradski on the basis of the Mean-shift algorithm can automatically adjust the size of a search window in real time, and retains the advantages of the Mean-shift algorithm, but tracking failure can be caused when the color characteristics of a moving target are not obvious or are interfered by factors such as shielding.
The invention content is as follows:
the purpose of the invention is as follows: the invention relates to a Cam-shift-based prediction tracking method, and aims to solve the problems in the prior art.
The technical scheme is as follows:
a prediction tracking method based on Cam-shift is characterized in that: the method provides a method for combining Kalman filtering and a Cam-shift algorithm, and provides a target prediction tracking method combining linear prediction and the Cam-shift algorithm, namely, the linear prediction method replaces the Kalman filtering to complete prediction estimation, and a prediction estimation result is substituted into the Cam-shift algorithm to track.
The method comprises the following steps:
principle analysis of Cam-shift algorithm:
the Cam-shift algorithm is an improved algorithm provided on the basis of a Mean-shift algorithm, the method carries out Mean-shift algorithm processing on each frame of a continuous image sequence, a processing result of a current frame is used as an initial value of a Mean-shift algorithm search window of a next frame, iterative operation is carried out in sequence, and tracking is completed;
(II) researching a Cam-shift improved algorithm:
(2.1) Kalman in combination with Cam-shift algorithm:
(2.1.1) Kalman filtering:
the Kalman filter is a prediction estimation method for realizing a moving target through a minimum mean square error criterion, and a discrete time linear system is constructed by modeling through the relation of a state equation and an observation equation, wherein the state equation and the observation equation of the system are respectively as follows:
X(k)＝AX(k-1)+BU(k)+W(k) (7)
Z(k)＝HX(k)+V(k) (8)
where x (K) is the state vector of the system at time K, which can be defined as:
X(k)＝[x(k) y(k) v(k) u(k)]T(9)
x (k) and y (k) respectively represent coordinate components of the target center in x and y axes, v (k), and u (k) respectively represent the velocity of the target center in x and y axes; u (K) is the system control quantity at the moment K, and A and B are system parameters; w (k) is process noise.
Z (K) is an observation vector of the system at time K, and can be defined as:
Z(k)＝[xz(k) yz(k)]T(10)
xz(k) and yz(k) Respectively representing coordinate values of the target center on x and y axes, H is an observation matrix, and V (k) is observation noise.
xz(k) And yz(k) Respectively representing coordinate values of the target center on x and y axes,
the Kalman filtering algorithm is as follows:
Figure BDA0001314477590000021
wherein, X (k | k-1) is the prediction result of the next state, X (k-1| k-1) is the optimal estimation of the current state of the system, and U (k) is the control quantity of the system state. P (k | k-1) is the error covariance of X (k | k-1), P (k-1| k-1) is the error covariance of X (k-1| k-1), X (k | k) is the optimal estimate, K (k) is the Kalman gain, I is the identity matrix, Q, R is the motion noise correlation matrix; after the above process is finished, returning to (11), namely predicting the k +1 state, and sequentially circulating to finish the complete process;
in the method, when a state equation and an observation equation of a Kalman system are constructed, according to a random accelerated motion equation, the following are obtained:
Figure BDA0001314477590000022
thus, the Kalman filter modeling is completed;
(2.1.2) the Kalman and Cam-shift algorithm is combined to realize the process:
according to the method, the Kalman algorithm and the Cam-shift algorithm are combined according to the advantages and disadvantages of the Kalman algorithm and the Cam-shift algorithm, and the global search is converted into the local search, so that the calculated amount is reduced, and the accuracy is improved;
(2.2) Linear prediction in combination with Cam-shift algorithm:
(2.2.1) Linear prediction:
from the physics, when a motion state equation of a target object in a description system is a linear equation, the motion state of the target at any moment after the target can be predicted by giving the initial condition of the motion state of the target; from the kinematic analysis, the curvilinear motion is regarded as being formed by constant-speed linear motion in a short time, and when the speed and the motion direction of the moving object are kept to be the same as the previous moment, the curvilinear motion is regarded as being constant-speed linear motion; particularly, when the moving speed of the target is slow, the time interval of each frame is short, so that the object motion between frames can be regarded as uniform linear motion;
the calculation process of the linear prediction method is as follows: knowing the initial position of the target, calculating the coordinate offset difference of the motion speed and the motion direction according to the current position coordinate and the initial position coordinate of the target, and then calculating the expected position of the next frame according to the obtained coordinate offset difference, namely, assuming that the central position coordinate of the target in the previous frame is (x)i-1,yi-1) The coordinate of the center position of the current frame is (x)i,yi) Then the coordinate offset difference is approximated as (Δ x, Δ y) ═ xi-xi-1,yi-yi-1) Since the moving time of the moving target between two frames is short and is approximately regarded as uniform linear motion, the expected position of the target in the next frame is obtained according to a kinematic formula as follows: (x)i+1,yi+1)＝(xi+Δx,yi+Δy)；
(2.2.2) implementation of linear prediction in combination with Cam-shift algorithm:
according to the idea of linear prediction, the linear prediction and a Cam-shift algorithm are combined, the obtained target expected position is used as the center of a search window and is substituted into the Cam-shift algorithm, so that a real moving target mass center is converged in local iteration, a new center position coordinate is updated, and the tracking process is well completed by circulating in sequence; the method comprises the following specific steps:
(1) converting the tracking target image into an HSV color histogram and generating a reverse projection diagram;
(2) determining the position of an initial target, and setting a reasonable initial window size;
(3) searching a window mass center position according to a Cam-shift algorithm, moving a window center to the window mass center position, and outputting an actual mass center position through a convergence condition;
(4) calculating a coordinate offset difference according to the obtained centroid position and the centroid position of the previous frame, and predicting a target expected position of the next frame;
(5) and (4) updating the window size, and returning to the step (3) to sequentially and circularly judge until the last frame is finished.
The method specifically comprises the following steps of analyzing the principle of the Cam-shift algorithm:
(1) establishing a color model:
selecting an HSV color space H component as a characteristic of a tracking target, and establishing a color histogram, so that the calculated amount is reduced greatly compared with that of a complete RGB and HSV model; the conversion formula of the RGB and HSV models is as follows:
V＝max(R,G,B) (1)
S＝(V-min(R,G,B))/V (2)
Figure BDA0001314477590000031
(2) generation of a back projection map:
converting the generated H component information histogram into a probability distribution image of colors, namely correspondingly setting the probability of a point with a pixel value of 255 as 1, and setting the probability of a point with a pixel value of 0 as 0, wherein the obtained color probability projection image is a reverse projection image of the original color histogram;
(3) finding the centroid using the Mean-shift algorithm:
initializing a search window, setting I (x, y) as a corresponding pixel point of a pixel point (x, y) reverse projection graph, obtaining a centroid of the search window by calculating a zero order moment and a first order moment of the search window, moving the center of the search window to the centroid position, and obtaining the centroid position according to the zero order moment M00Updating the window size; the specific formula is as follows:
Figure BDA0001314477590000032
the centroid of the search window is:
Figure BDA0001314477590000041
the update window size is:
Figure BDA0001314477590000042
and when the distance from the window to the centroid is greater than a preset value, recalculating the centroid position of the window until the convergence condition is met, then performing tracking calculation of the next frame, and sequentially circulating to complete the whole tracking process.
The method for realizing the combination of the Kalman algorithm and the Cam-shift algorithm comprises the following specific steps:
(1) firstly, setting an initial value of a tracking target;
(2) calculating the variance of the next frame image, substituting the variance into a formula to obtain the target state at the moment, and estimating the most probable position and motion speed of the tracking target in the next frame by using the predicted covariance and state updating of the formulas (7) to (11) in (2.1.1) Kalman filtering;
(3) importing the result into a Cam-shift algorithm, performing local target search and matching near the predicted and estimated target position, and taking the searched window center position as a Kalman measurement value;
(4) updating a tracking window;
(5) and (5) updating the covariance matrix and the target state equation of the target, and repeating the step (2) until the end.
The advantages and effects are as follows: the invention relates to a Cam-shift-based prediction tracking method, which researches a method for combining Kalman filtering and a Cam-shift algorithm in order to well complete the position prediction tracking of a moving target under a complex background interfered by factors such as shielding and the like. Kalman filtering can accurately complete the position prediction of a moving target, and can well complete tracking by combining with a Cam-shift algorithm. On the basis, a target prediction tracking method combining linear prediction and a Cam-shift algorithm is provided, namely the linear prediction method replaces Kalman filtering to complete prediction estimation, and the prediction estimation result is substituted into the Cam-shift algorithm for tracking. Experiments show that the method can ensure the tracking accuracy in the shielding process, reduce the iteration time and better meet the requirement of real-time performance.
Description of the drawings:
FIG. 1 is a graph of the tracking results of a conventional Cam-shift algorithm; wherein (a) is a tracking result of a traditional Cam-shift algorithm under a normal background; (b) tracking results of the Cam-shift algorithm under the shielding condition;
FIG. 2 is a tracking flow chart;
FIG. 3 is a graph of the improved algorithm tracking results, where (a) is the tracking effect of Kalman combined with the Cam-shift algorithm; (b) the tracking effect of the combination of linear prediction and Cam-shift is predicted.
The specific implementation mode is as follows:
the invention provides a prediction tracking method based on Cam-shift,
1. principle analysis of Cam-shift algorithm:
the Cam-shift algorithm is an improved algorithm provided on the basis of a Mean-shift algorithm, the Mean-shift algorithm processing is carried out on each frame of a continuous image sequence, the processing result of the current frame is used as the initial value of a Mean-shift algorithm searching window of the next frame, iterative operation is carried out in sequence, and tracking is completed. The method comprises the following concrete steps:
(1) establishment of color model
And the Cam-shift algorithm adopts a color model as a characteristic to track the target. The RGB color model generates or displays colors, the HSV color model can reflect the essential characteristics of colors, and can separately process the chroma, saturation and lightness, thereby improving the stability of the algorithm. Therefore, the HSV color space H component is selected as the characteristic of the tracking target, and a color histogram is established, so that the calculation amount is reduced greatly compared with that of the complete RGB and HSV model. The conversion formula of the RGB and HSV models is as follows:
V＝max(R,G,B) (1)
S＝(V-min(R,G,B))/V (2)
Figure BDA0001314477590000051
(2) generation of backprojection maps
And converting the generated H component information histogram into a probability distribution image of colors, namely correspondingly setting the probability of a point with a pixel value of 255 as 1, and setting the probability of a point with a pixel value of 0 as 0, wherein the obtained color probability projection image is a reverse projection image of the original color histogram.
(3) Finding centroids using Mean-shift algorithm
Initializing a search window, setting I (x, y) as a corresponding pixel point of a pixel point (x, y) reverse projection graph, obtaining a centroid of the search window by calculating a zero order moment and a first order moment of the search window, moving the center of the search window to the centroid position, and obtaining the centroid position according to the zero order moment M00The window size is updated. The concrete formula is as follows[9]：
Figure BDA0001314477590000052
The centroid of the search window is:
Figure BDA0001314477590000053
the update window size is:
Figure BDA0001314477590000054
and when the distance from the window to the centroid is greater than a preset value, recalculating the centroid position of the window until the convergence condition is met, then performing tracking calculation of the next frame, and sequentially circulating to complete the whole tracking process.
Fig. 1 shows the tracking result of moving target sequence under complex background by Cam-shift algorithm, wherein a is the tracking result of the 10 th, 20 th, 30 th and 40 th frame sequences under normal non-occlusion condition, and b is the tracking result of the 50 th, 60 th, 70 th and 80 th frame sequences under occlusion condition. It is easy to see that the Cam-shift algorithm can automatically adjust the size of the search window in real time, and can accurately track the target under an interference-free background, but when the influence of factors such as shielding is encountered, the calculation error is easily caused due to unobvious color characteristics and complicated iteration process, and the tracking failure is further caused. Therefore, the position of the moving target is predicted by the estimator aiming at the defect, the iteration number is reduced, and the error rate is reduced.
Study of the 2Cam-shift improved algorithm:
2.1Kalman in combination with Cam-shift algorithm:
2.1.1Kalman Filter:
the Kalman filter is a prediction estimation method for realizing a moving target through a minimum mean square error criterion, and a discrete time linear system is constructed by modeling through the relation of a state equation and an observation equation. The state equation and the observation equation of the system are respectively as follows:
X(k)＝AX(k-1)+BU(k)+W(k) (7)
Z(k)＝HX(k)+V(k) (8)
where x (K) is the state vector of the system at time K, which can be defined as:
X(k)＝[x(k) y(k) v(k) u(k)]T(9)
x (k) and y (k) represent coordinate components of the target center in the x and y axes, respectively; v (k), u (k) represent the speed of the target center on the x and y axes respectively; u (K) Z (K) are respectively the control quantity and observation vector of the system at the moment K, xz(k) And yz(k) And the coordinate values of the target center on the x axis and the y axis respectively.
The Kalman filtering algorithm is as follows:
Figure BDA0001314477590000061
wherein, X (k | k-1) is the prediction result of the next state, X (k-1| k-1) is the optimal estimation of the current state of the system, P (k | k-1) is the error covariance of X (k | k-1), X (k | k) is the optimal estimation value, and K (k) is the Kalman gain. And (5) after the above process is finished, returning to (10), namely predicting the k +1 state, and sequentially circulating to finish the complete process.
In the method, when a state equation and an observation equation of a Kalman system are constructed, according to a random accelerated motion equation, the following can be obtained:
Figure BDA0001314477590000062
thus, the Kalman filter modeling is completed.
2.1.2Kalman and Cam-shift algorithm combined implementation process
Because the global search based on the pixels is carried out when the Kalman filtering algorithm is applied to the prediction tracking, the calculation amount is large, and when the Kalman filtering algorithm is applied to a complex background, the global search has weak anti-interference capability and is easy to lose a tracking target. According to the method, the global search can be converted into the local search by combining the advantages and the disadvantages of the Kalman algorithm and the Cam-shift algorithm, so that the calculated amount is reduced, and the accuracy is improved.
The specific method comprises the following steps:
(1) first, initial values of a tracking target, such as a speed, a position, and the like, are set;
(2) calculating the variance of the next frame image, substituting the variance into a formula to obtain the current target state, predicting the covariance and state updating by using a Kalman correlation formula, and estimating the most probable position and motion speed of the tracking target in the next frame;
(3) importing the result into a Cam-shift algorithm, performing local target search and matching near the predicted and estimated target position, and taking the searched window center position as a Kalman measurement value;
(4) updating a tracking window;
(5) and (5) updating the covariance matrix and the target state equation of the target, and repeating the step (2) until the end.
2.2 Linear prediction in combination with Cam-shift algorithm:
2.2.1 Linear prediction
As can be seen from physics, when the equation describing the motion state of the target object in the system is a linear equation, given the initial condition of the motion state of the target, the motion state of the target at any time after the target can be predicted. From the kinematic analysis, the curvilinear motion can be regarded as a constant linear motion in a short time, and when the speed and the motion direction of the moving object are kept substantially the same as the previous time, the curvilinear motion can be regarded as a constant linear motion. Particularly, when the moving speed of the target is slow, the object motion between frames can be regarded as uniform linear motion due to the short time interval of each frame.
The calculation process of the linear prediction method is as follows: knowing the initial position of the target, calculating the coordinate offset difference of the motion speed and the motion direction according to the current position coordinate and the initial position coordinate of the target, and then calculating the expected position of the next frame according to the obtained coordinate offset difference, namely, assuming that the central position coordinate of the target in the previous frame is (x)i-1,yi-1) The coordinate of the center position of the current frame is (x)i,yi) The coordinate offset difference can be approximated as (Δ x, Δ y) ═ xi-xi-1,yi-yi-1) Since the moving time of the moving target between two frames is short, the moving target can be approximately regarded as uniform linear motion, and therefore the expected position of the target in the next frame can be obtained according to a kinematic formula as follows: (x)i+1,yi+1)＝(xi+Δx,yi+Δy)。
2.2.2 implementation of Linear prediction in combination with Cam-shift Algorithm
According to the idea of linear prediction, the method is combined with a Cam-shift algorithm, the obtained target expected position is used as the center of a search window and is substituted into the Cam-shift algorithm, so that the real moving target mass center is converged in local iteration, a new center position coordinate is updated, and the tracking process is completed well through sequential circulation.
The tracking flow chart is shown in fig. 2:
3 experimental results and analysis:
the method verifies the effectiveness and the real-time performance of the combination of the linear prediction and the Cam-shift algorithm through the comparative analysis of the combination of the Kalman and the Cam-shift algorithm and the combination of the linear prediction and the Cam-shift algorithm. The method adopts a human vertical walking sequence image (with the size of 360 x 640) with a shielding process under a complex background to carry out a tracking experiment, a tracking window is a rectangular window, an initial window is determined by detecting the position of a first frame of moving target by a background difference method, and the tracking sequence images of 30 th, 60 th, 90 th and 130 th frames are respectively selected for illustration.
FIG. 3 is a tracking result of Kalman and Cam-shift algorithm combination and linear prediction and Cam-shift algorithm combination:
as can be seen from FIG. 3, both the method combining Kalman and Cam-shift algorithms and the method combining linear prediction and Cam-shift algorithms can automatically update the window size in real time, and when an occlusion process exists, compared with the traditional Cam-shift method, the two improved methods are slightly influenced by occlusion factors, can ensure the tracking accuracy and well complete tracking.
The method also carries out tracking experiments for 50 times under a complex background, analyzes and compares the tracking accuracy of the three algorithms, and can be intuitively seen from the table 1, and the tracking accuracy of the linear prediction and Cam-shift combined algorithm and the Kalman and Cam-shift combined algorithm provided by the method is much higher than that of the traditional Cam-shift algorithm. By tracking a sequence image of continuous 30 frames without shielding interference, the iteration times and the consumed time of a traditional Cam-shift algorithm and an improved algorithm are analyzed and compared, and it can be seen that the difference between the linear prediction algorithm and the Cam-shift algorithm provided by the method and the total iteration times of the Kalman and Cam-shift algorithm is very small, and the difference is reduced by about half compared with the traditional Cam-shift algorithm; the linear prediction and Cam-shift combined algorithm provided by the method has little difference from the traditional Cam-shift algorithm in single iteration average time consumption, but the total time consumption is much less than that of the traditional Cam-shift algorithm because the iteration times are half of that of the traditional Cam-shift algorithm. And the prediction system constructed by the algorithm combining Kalman and Cam-shift is complex, and the average time consumption of single iteration is long, so the total time consumption is higher than the time consumption of the algorithm proposed by the method, but still lower than the time consumption of the traditional Cam-shift algorithm. Therefore, the algorithm provided by the method can not only meet the requirement of accurate tracking of interference such as shielding and the like, but also ensure better real-time performance.
Tracking accuracy Total number of iterations Total time consumption Average time consumption for a single iteration
Traditional Cam-shift algorithm 84％ 45 68.5ms 1.52ms
Kalman in combination with Cam-shift 100％ 21 41.7ms 1.98ms
Linear prediction combined with Cam-shift 100％ 23 35.4ms 1.54ms
TABLE 1 comparison of iteration number and elapsed time for three algorithms
Tab.1 Iterative number and time comparison of three algorithms
4, summarizing the following steps:
because the traditional Cam-shift algorithm has the possibility of being greatly influenced by shielding factors and inaccurate in tracking under a complex background, the method is inspired from researching a method combining Kalman and Cam-shift algorithms, the linear prediction is used for replacing Kalman filtering, the structure of a linear estimation system is simplified, and the predictive tracking algorithm combining the linear prediction and the Cam-shift algorithm is provided. The tracking experiment shows that the algorithm can not only meet the successful tracking of the moving target in the shielding process, but also reduce the single iteration time, thereby improving the tracking efficiency and better meeting the requirement of real-time property.
The method starts from Cam-shift algorithm analysis, combines Kalman filtering to identify and estimate the moving target, well reduces the probability of losing the target under a complex background, and provides a method for combining linear prediction and the Cam-shift algorithm by using physics motion related knowledge.

Claims (3)

1. A prediction tracking method based on Cam-shift is characterized in that: the method replaces Kalman filtering with a linear prediction method to finish prediction estimation, and substitutes the prediction estimation result into a Cam-shift algorithm for tracking;
the method comprises the following steps:
principle analysis of Cam-shift algorithm:
the Cam-shift algorithm is an improved algorithm provided on the basis of a Mean-shift algorithm, the method carries out Mean-shift algorithm processing on each frame of a continuous image sequence, a processing result of a current frame is used as an initial value of a Mean-shift algorithm search window of a next frame, iterative operation is carried out in sequence, and tracking is completed;
(II) researching a Cam-shift improved algorithm:
(2.1) Kalman in combination with Cam-shift algorithm:
(2.1.1) Kalman filtering:
the Kalman filter is a prediction estimation method for realizing a moving target through a minimum mean square error criterion, and a discrete time linear system is constructed by modeling through the relation of a state equation and an observation equation, wherein the state equation and the observation equation of the system are respectively as follows:
X(k)＝AX(k-1)+BU(k)+W(k) (7)
Z(k)＝HX(k)+V(k) (8)
where x (K) is the state vector of the system at time K, which can be defined as:
X(k)＝[x(k) y(k) v(k) u(k)]T(9)
x (k) and y (k) respectively represent coordinate components of the target center in x and y axes, v (k), and u (k) respectively represent the velocity of the target center in x and y axes; u (K) is the system control quantity at the moment K, and A and B are system parameters; w (k) is process noise;
z (K) is an observation vector of the system at time K, and can be defined as:
Z(k)＝[xz(k) yz(k)]T(10)
xz(k) and yz(k) Respectively representing coordinate values of the target center on x and y axes, H is an observation matrix, and V (k) is observation noise;
xz(k) and yz(k) Respectively representing coordinate values of the target center on x and y axes,
the Kalman filtering algorithm is as follows:
Figure FDA0002505724330000011
wherein, X (k | k-1) is the prediction result of the next state, X (k-1| k-1) is the optimal estimation of the current state of the system, and U (k) is the control quantity of the system state; p (k | k-1) is the error covariance of X (k | k-1), P (k-1| k-1) is the error covariance of X (k-1| k-1), X (k | k) is the optimal estimate, K (k) is the Kalman gain, I is the identity matrix, Q, R is the motion noise correlation matrix; after the above process is finished, returning to (11), namely predicting the k +1 state, and sequentially circulating to finish the complete process;
in the method, when a state equation and an observation equation of a Kalman system are constructed, according to a random accelerated motion equation, the following are obtained:
Figure FDA0002505724330000021
thus, the Kalman filter modeling is completed;
(2.1.2) the Kalman and Cam-shift algorithm is combined to realize the process:
according to the method, the Kalman algorithm and the Cam-shift algorithm are combined according to the advantages and disadvantages of the Kalman algorithm and the Cam-shift algorithm, and the global search is converted into the local search, so that the calculated amount is reduced, and the accuracy is improved;
(2.2) Linear prediction in combination with Cam-shift algorithm:
(2.2.1) Linear prediction:
from the physics, when a motion state equation of a target object in a description system is a linear equation, the motion state of the target at any moment after the target can be predicted by giving the initial condition of the motion state of the target; from the kinematic analysis, the curvilinear motion is regarded as being formed by constant-speed linear motion in a short time, and when the speed and the motion direction of the moving object are kept to be the same as the previous moment, the curvilinear motion is regarded as being constant-speed linear motion; particularly, when the moving speed of the target is slow, the time interval of each frame is short, so that the object motion between frames can be regarded as uniform linear motion;
the calculation process of the linear prediction method is as follows: knowing the initial position of the target, calculating the coordinate offset difference of the motion speed and the motion direction according to the current position coordinate and the initial position coordinate of the target, and then calculating the expected position of the next frame according to the obtained coordinate offset difference, namely, assuming that the central position coordinate of the target in the previous frame is (x)i-1,yi-1) Current frameHas a center position coordinate of (x)i,yi) Then the coordinate offset difference is approximated as (Δ x, Δ y) ═ xi-xi-1,yi-yi-1) Since the moving time of the moving target between two frames is short and is approximately regarded as uniform linear motion, the expected position of the target in the next frame is obtained according to a kinematic formula as follows: (x)i+1,yi+1)＝(xi+Δx,yi+Δy)；
(2.2.2) implementation of linear prediction in combination with Cam-shift algorithm:
according to the idea of linear prediction, the linear prediction and a Cam-shift algorithm are combined, the obtained target expected position is used as the center of a search window and is substituted into the Cam-shift algorithm, so that a real moving target mass center is converged in local iteration, a new center position coordinate is updated, and the tracking process is well completed by circulating in sequence; the method comprises the following specific steps:
(1) converting the tracking target image into an HSV color histogram and generating a reverse projection diagram;
(2) determining the position of an initial target, and setting a reasonable initial window size;
(3) searching a window mass center position according to a Cam-shift algorithm, moving a window center to the window mass center position, and outputting an actual mass center position through a convergence condition;
(4) calculating a coordinate offset difference according to the obtained centroid position and the centroid position of the previous frame, and predicting a target expected position of the next frame;
(5) and (4) updating the window size, and returning to the step (3) to sequentially and circularly judge until the last frame is finished.
2. The Cam-shift-based predictive tracking method of claim 1, wherein: the method specifically comprises the following steps of analyzing the principle of the Cam-shift algorithm:
(1) establishing a color model:
selecting an HSV color space H component as a characteristic of a tracking target, and establishing a color histogram, so that the calculated amount is reduced greatly compared with that of a complete RGB and HSV model; the conversion formula of the RGB and HSV models is as follows:
V＝max(R,G,B) (1)
S＝(V-min(R,G,B))/V (2)
Figure FDA0002505724330000022
(2) generation of a back projection map:
converting the generated H component information histogram into a probability distribution image of colors, namely correspondingly setting the probability of a point with a pixel value of 255 as 1, and setting the probability of a point with a pixel value of 0 as 0, wherein the obtained color probability projection image is a reverse projection image of the original color histogram;
(3) finding the centroid using the Mean-shift algorithm:
initializing a search window, setting I (x, y) as a corresponding pixel point of a pixel point (x, y) reverse projection graph, obtaining a centroid of the search window by calculating a zero order moment and a first order moment of the search window, moving the center of the search window to the centroid position, and obtaining the centroid position according to the zero order moment M00Updating the window size; the specific formula is as follows:
Figure FDA0002505724330000031
the centroid of the search window is:
Figure FDA0002505724330000032
the update window size is:
Figure FDA0002505724330000033
and when the distance from the window to the centroid is greater than a preset value, recalculating the centroid position of the window until the convergence condition is met, then performing tracking calculation of the next frame, and sequentially circulating to complete the whole tracking process.
3. The Cam-shift-based predictive tracking method of claim 1, wherein:
the method for realizing the combination of the Kalman algorithm and the Cam-shift algorithm comprises the following specific steps:
(1) firstly, setting an initial value of a tracking target;
(2) calculating the variance of the next frame image, substituting the variance into a formula to obtain the target state at the moment, and estimating the most probable position and motion speed of the tracking target in the next frame by using the predicted covariance and state updating of the formulas (7) to (11) in (2.1.1) Kalman filtering;
(3) importing the result into a Cam-shift algorithm, performing local target search and matching near the predicted and estimated target position, and taking the searched window center position as a Kalman measurement value;
(4) updating a tracking window;
(5) and (5) updating the covariance matrix and the target state equation of the target, and repeating the step (2) until the end.
CN201710419084.6A 2017-06-06 2017-06-06 Cam-shift-based prediction tracking method Active CN107403439B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710419084.6A CN107403439B (en) 2017-06-06 2017-06-06 Cam-shift-based prediction tracking method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710419084.6A CN107403439B (en) 2017-06-06 2017-06-06 Cam-shift-based prediction tracking method

Publications (2)

Publication Number Publication Date
CN107403439A CN107403439A (en) 2017-11-28
CN107403439B true CN107403439B (en) 2020-07-24

Family

ID=60405075

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710419084.6A Active CN107403439B (en) 2017-06-06 2017-06-06 Cam-shift-based prediction tracking method

Country Status (1)

Country Link
CN (1) CN107403439B (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108257159A (en) * 2017-12-07 2018-07-06 广西师范大学 A kind of audio-video monitoring server-side, system and method
CN108198205A (en) * 2017-12-22 2018-06-22 湖南源信光电科技股份有限公司 A kind of method for tracking target based on Vibe and Camshift algorithms
CN108259703B (en) * 2017-12-31 2021-06-01 深圳市越疆科技有限公司 Pan-tilt and pan-tilt tracking control method and device and pan-tilt
CN108376246A (en) * 2018-02-05 2018-08-07 南京蓝泰交通设施有限责任公司 A kind of identification of plurality of human faces and tracking system and method
CN108898624B (en) * 2018-06-12 2020-12-08 浙江大华技术股份有限公司 Moving object tracking method and device, electronic equipment and storage medium
CN109472812B (en) * 2018-09-29 2021-11-02 深圳市锦润防务科技有限公司 Method, system and storage medium for updating target tracking template
CN110211160B (en) * 2019-05-30 2022-03-25 华南理工大学 Face tracking method based on improved Camshift algorithm
CN110490902B (en) * 2019-08-02 2022-06-14 西安天和防务技术股份有限公司 Target tracking method and device applied to smart city and computer equipment
CN112347955A (en) * 2020-11-12 2021-02-09 上海影卓信息科技有限公司 Method, system and medium for fast object identification based on frame prediction in video

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101610412A (en) * 2009-07-21 2009-12-23 北京大学 A kind of visual tracking method that merges based on multi thread
CN101860729A (en) * 2010-04-16 2010-10-13 天津理工大学 Target tracking method for omnidirectional vision
CN104866823A (en) * 2015-05-11 2015-08-26 重庆邮电大学 Vehicle detection and tracking method based on monocular vision
CN105632170A (en) * 2014-11-26 2016-06-01 安徽中杰信息科技有限公司 Mean shift tracking algorithm-based traffic flow detection method
CN105844664A (en) * 2016-03-21 2016-08-10 辽宁师范大学 Monitoring video vehicle detection tracking method based on improved TLD
CN106096645A (en) * 2016-06-07 2016-11-09 上海瑞孚电子科技有限公司 Resist and repeatedly block and the recognition and tracking method and system of color interference
KR101712136B1 (en) * 2015-04-30 2017-03-03 수원대학교산학협력단 Method and apparatus for detecting a fainting situation of an object by using thermal image camera
KR101741758B1 (en) * 2016-01-21 2017-05-30 광운대학교 산학협력단 A Real-time Face Tracking Method Robust to Occlusion Based on Improved CamShift with Depth Information

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101610412A (en) * 2009-07-21 2009-12-23 北京大学 A kind of visual tracking method that merges based on multi thread
CN101860729A (en) * 2010-04-16 2010-10-13 天津理工大学 Target tracking method for omnidirectional vision
CN105632170A (en) * 2014-11-26 2016-06-01 安徽中杰信息科技有限公司 Mean shift tracking algorithm-based traffic flow detection method
KR101712136B1 (en) * 2015-04-30 2017-03-03 수원대학교산학협력단 Method and apparatus for detecting a fainting situation of an object by using thermal image camera
CN104866823A (en) * 2015-05-11 2015-08-26 重庆邮电大学 Vehicle detection and tracking method based on monocular vision
KR101741758B1 (en) * 2016-01-21 2017-05-30 광운대학교 산학협력단 A Real-time Face Tracking Method Robust to Occlusion Based on Improved CamShift with Depth Information
CN105844664A (en) * 2016-03-21 2016-08-10 辽宁师范大学 Monitoring video vehicle detection tracking method based on improved TLD
CN106096645A (en) * 2016-06-07 2016-11-09 上海瑞孚电子科技有限公司 Resist and repeatedly block and the recognition and tracking method and system of color interference

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
一种基于改进卡尔曼滤波的camshift目标跟踪算法;张甜;《测试技术学报》;20121231;第26卷(第6期);全文 *

Also Published As

Publication number Publication date
CN107403439A (en) 2017-11-28

Similar Documents

Publication Publication Date Title
CN107403439B (en) Cam-shift-based prediction tracking method
Hu et al. Active contour-based visual tracking by integrating colors, shapes, and motions
CN102881024B (en) Tracking-learning-detection (TLD)-based video object tracking method
CN104915969B (en) A kind of stencil matching tracking based on particle group optimizing
CN103927764B (en) A kind of wireless vehicle tracking of combining target information and estimation
CN109816692A (en) A kind of motion target tracking method based on Camshift algorithm
CN110796010A (en) Video image stabilization method combining optical flow method and Kalman filtering
JP2007523429A (en) Method and system for multi-modal component-based tracking of objects using robust information fusion
CN107301657B (en) A kind of video target tracking method considering target movable information
CN110211160B (en) Face tracking method based on improved Camshift algorithm
CN106296732B (en) A kind of method for tracking moving object under complex background
KR20170015299A (en) Method and apparatus for object tracking and segmentation via background tracking
Gibson et al. Robust optical flow estimation based on a sparse motion trajectory set
Bagherpour et al. Upper body tracking using KLT and Kalman filter
CN108764177B (en) Moving target detection method based on low-rank decomposition and representation joint learning
CN111402303A (en) Target tracking architecture based on KFSTRCF
CN113532420A (en) Visual inertial odometer method integrating point-line characteristics
White et al. An iterative pose estimation algorithm based on epipolar geometry with application to multi-target tracking
CN104123733A (en) Motion detection and error rate reduction method based on block matching
Chang et al. Topology-constrained layered tracking with latent flow
Wang et al. Foreground Object Detecting Algorithm based on Mixture of Gaussian and Kalman Filter in Video Surveillance.
Zhao et al. 3D object tracking via boundary constrained region-based model
CN110660081B (en) Target tracking method based on self-adaptive feature selection
Tian et al. Motion segmentation and estimation
Zheng et al. A boosted adaptive particle filter for face detection and tracking

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant