CN102915545A - OpenCV(open source computer vision library)-based video target tracking algorithm - Google Patents

Abstract

The invention discloses an openCV(open source computer vision library)-based video target tracking algorithm. The openCV-based video target tracking algorithm is characterized in that a selected template is matched with a video frame; a subgraph position which is most similar to a template image in the video frame is found out through calculation of correlative coefficients; updating of the template is determined according to a predicted position of a kalman filter and the correlative coefficient values; and the specific algorithm includes template matching, position prediction and template updating. Compared with the prior art, the openCV-based video target tracking algorithm has the advantages that recognition and tracking of a target are not affected by environment change; the target object is accurately recognized and tracked in time; and the updating of the tracked object is available, so that the template can be dynamically updated during the system tracking, and the tracking is more accurate under the condition of environment and object change. In addition, a plurality of parameters are used as tracking evidences, so that the tracking is more reliable; and under the condition of target object moving, continuous background change and shadow influence, the tracking object cannot be lost..

Description

A kind of video frequency object tracking algorithm based on OpenCV

Technical field

The present invention relates to the video data analysis technical field, especially a kind of video frequency object tracking algorithm based on OpenCV for picture catching and video data analysis.

Background technology

Video tracking is a research direction of rising in recent years, and the algorithm of video tracking be take image sequence as input, output be the various attributes of target in image, as the size of target, position etc.Under ideal state, these output be all accurately, real-time, yet, in real world, due to the existence of various interference, often be difficult to reach perfect condition, can so get rid of, to disturb captured target object exactly be the key of weighing video tracking algorithm quality.

At present, video tracking algorithm commonly used has several as follows:

(i), the Meanshift algorithm, be a kind of method of mating the image in fixed size zone with colouring information.The Meanshift algorithm is a kind of nothing ginseng algorithm for estimating that utilizes data-driven, be called again the Density Estimator algorithm, mainly by mean-shift vector, find the posterior probability local optimum, that is to say, the meanshift algorithm is that matching area is determined in the zone the most similar by color, and the meanshift algorithm is not supported the renewal of target.

(ii), background subtraction point-score: be to utilize the difference of present image and background image to identify the method for moving target, obtaining and upgrading of its background model is the gordian technique in the method.A kind of method of simple background extraction image is to catch background image while occurring without any target in scene, and the background image that this artificial non-self-adapting method is obtained is only applicable to the video monitoring in the short time.The most algorithm has been abandoned the background image method of estimation of this non-self-adapting.When the scene environment while not being very complicated, can use statistical filtering to complete the estimation of background image in scene, in most cases can obtain correct background estimating image, but while having the part object to do random motion in scene, can cause that in scene, pixel value constantly changes, thereby cause evaluated error.Background estimating method based on the Gaussian statistics model also can estimate more exactly background model in the scene that has subregion constantly to change, but its calculating more complicated, and can't meet the real-time demand.

(iii), temporal difference: be to adopt the time difference based on pixel between two consecutive frames in video sequence, and thresholding extracts the method for moving target, being about to two adjacent two field pictures does by the pixel additive operation, and setting threshold, if difference is greater than threshold value, it is regarded as to foreground image (moving target), otherwise regard background image as.Temporal difference is stronger for the adaptability of dynamic environment, do not need the background extraction image, when the information such as the texture of adjacent two two field pictures, gray scale relatively approach, can obtain more exactly the peripheral profile of moving object, but the particular content of target usually can not intactly be detected, easily at interior of articles, produce cavity.

In sum, the problem that prior art exists is: the recognition and tracking of target is subject to the impact of environmental change larger, does not support the renewal of tracking object; Only rely on single or less quantity of information not accurate enough as the standard of following the tracks of, follow the tracks of or tracking error in the situation that background and target constantly change to be easier to lose.

Summary of the invention

The objective of the invention is a kind of video frequency object tracking algorithm based on OpenCV designed for the deficiencies in the prior art, with template and the frame of video selected, mated, find out subgraph position the most similar to template image in frame of video by the calculating of related coefficient, can the shade impact be arranged in the situation that target object moves the continuous variation with background simultaneously, accurately and timely identify and the tracking target object.

The object of the present invention is achieved like this: a kind of video frequency object tracking algorithm based on OpenCV, be characterized in that this video frequency object tracking algorithm is mated with template and the frame of video selected, find out subgraph position the most similar to template image in frame of video by the calculating of related coefficient, the renewal of template is definite according to predicted position and the facies relationship numerical value of Kalman filter, and specific algorithm carries out in the steps below:

(1), template matches

The known little image of take is template, template is overlayed to mobile search target in searched large image, the target area covered by template is subgraph, the whole possible position calculate the related coefficient of subgraph and template on each position of search in searched large image, the related coefficient calculated leaves in an array, wherein position corresponding to maximal value is the subgraph position of mating the most with masterplate, its related coefficient is pressed following formula (a) and is calculated, for each subgraph, calculate corresponding normalized correlation coefficient R (i, j) (1≤R (i, j)≤1), and find out the maximal value R of R _max(i _m, j _m), its corresponding subgraph is the coupling target, when template and subgraph when just the same, and coefficient R (i, j)=1,

$R(i,j)=\frac{\underset{x=1}{\overset{n}{\mathrm{\Σ}}}\underset{y=1}{\overset{m}{\mathrm{\Σ}}}{S}_{\mathrm{ij}}(x,y)*T(x,y)}{\sqrt{\underset{x=1}{\overset{n}{\mathrm{\Σ}}}\underset{y=1}{\overset{m}{\mathrm{\Σ}}}{\left[S_{\mathrm{ij}}(x,y)\right]}^2}\sqrt{\underset{x=1}{\overset{n}{\mathrm{\Σ}}}\underset{y=1}{\overset{m}{\mathrm{\Σ}}}{\left[T(x,y)\right]}^2}}---\left(a\right)$

Wherein: n, m mean width and the height of template T, and i, j mean subgraph S _IjCoordinate position in large figure, x, y mean subgraph S _IjThe intrinsic coordinates value, S _Ij(x, y) is illustrated in subgraph S _IjThe pixel value of the point that middle coordinate is (x, y), T(x, y) be illustrated in the pixel value of the point that in template T, coordinate is (x, y);

(2), position prediction

For the two field picture newly obtained, adopt Kalman filter to predict best match position in this frame according to the best match position of former frames; Described Kalman filter is comprised of the time predictive equation upgraded and the correction equation of measuring renewal, and the covariance of reckon error forward that the reckoning state variable forward that described predictive equation is calculated by following formula (b) and following formula (c) calculate is constructed the prior estimate of next time state; The posteriority covariance that posteriority is estimated and following formula (f) calculates that the kalman gain that described correction equation is calculated by following formula (d), following formula (e) calculate is as the prior estimate of calculating next time, each need be current according to former measurand recursive calculation state;

Calculate forward state variable: ${\hat{x}}_k^{-}=A{\hat{x}}_{k-1}+B{u}_{k-1}---\left(b\right)$

Reckon error covariance forward: $P_k^{-}=A{P}_{k-1}{A}^T+Q---\left(c\right)$

Kalman gain: $K_k=P_k^{-}{H}^T{({\mathrm{HP}}_k^{-}{H}^T+R)}^{-1}---\left(d\right)$

Posteriority is estimated: ${{{\hat{x}}_k={\hat{x}}_k^{-}+K_k(z_k-H{\hat{x}}_k^{-})}_{}}_{}---\left(e\right)$

The posteriority covariance: $P_k=(I-K_{k}H)P_K^{-}---\left(f\right)$

Wherein: A is state-transition matrix, and B is gating matrix or unit matrix; Q is the process noise covariance matrix; H is for measuring matrix; R is for measuring noise covariance matrix; P is posteriority mistake covariance matrix; I is unit matrix; u _K-1For observation noise; z _kThe storage actual measured results, x _kFor state variable;

(3), template renewal

According to predictive variable y _kWith observational variable z _kBetween apart from distance and z _kCorresponding facies relationship numerical value R _maxDetermine whether that renewal works as front template, some y _k(x ₁, y ₁) and some z _k(x ₂, y ₂) between press following formula (g) apart from distance and calculate; When distance≤30: R _maxBe greater than at 0.90 o'clock and continue to follow the tracks of; R _maxBe less than 0.90 and be greater than at 0.85 o'clock new template more; R _maxBe less than at 0.85 o'clock and lose tracking; As distance>30 the time: R _maxBe greater than at 0.85 o'clock and use y _kAs trace point; R _maxBe less than at 0.85 o'clock and lose tracking.

$\mathrm{dis}\tan \mathrm{ce}=\sqrt{{(x_1-x_2)}^2+{(y_1-y_2)}^2}---\left(g\right)$

The present invention compared with prior art has the identification of target and follows the tracks of the impact that is not subject to environmental change, can identify timely and accurately and the tracking target object, support the renewal of tracking object, make system can dynamically upgrade template when following the tracks of, tracking is in the situation that environment and object variation are more accurate, using a plurality of parameters as the foundation of following the tracks of, follows the tracks of more reliable, in the situation that target object moves, background constantly changes and have the shade impact to be difficult for losing tracking.

The accompanying drawing explanation

Fig. 1 is the template schematic diagram

Fig. 2 is template searching target schematic diagram on large figure

Fig. 3～Fig. 4 is the Kalman filter operation chart

Embodiment

Embodiment 1

The present invention is mated with template and the frame of video selected, find out subgraph position the most similar to template image in video image and carry out the calculating of correlation coefficient matching method, whether and the position of this frame coupling and the position according to former frame prediction gained are compared, thereby determining to upgrade works as front template.

At first, create an OpenCV engineering under Visual C++6.0 environment, then according to the kind of camera and operating system, to call suitable API and open camera and obtain the picture frame that camera captures and carry out video tracking, specific algorithm and operation are carried out in the steps below:

(1), template matches

Consult accompanying drawing 1～accompanying drawing 2, the little image of known m * n pixel of take is template T, template T is overlayed to the upper translation searching target of searched large image S of W * H pixel, and this target has identical size, direction and image with template T, the target area covered by template T is subgraph S _Ij, i wherein, j is the coordinate of the subgraph lower left corner on searched large image S, i, j moves on rectangle ABCD, so its span is:

0≤i≤W–m

0≤j≤H–n

In searched large image S, search is possible position calculate the related coefficient of subgraph and template on each position all, the related coefficient calculated leaves in an array, wherein position corresponding to maximal value is the subgraph position of mating the most with masterplate, related coefficient is pressed following formula (a) and is calculated, and for each subgraph, calculates corresponding normalized correlation coefficient R (i, and find out the maximal value R of R j), _max(i _m, j _m), the subgraph S that it is corresponding _IjBe the coupling target, as template T and subgraph S _IjWhen just the same, coefficient R (i, j)=1.

$R(i,j)=\frac{\underset{x=1}{\overset{n}{\mathrm{\Σ}}}\underset{y=1}{\overset{m}{\mathrm{\Σ}}}{S}_{\mathrm{ij}}(x,y)*T(x,y)}{\sqrt{\underset{x=1}{\overset{n}{\mathrm{\Σ}}}\underset{y=1}{\overset{m}{\mathrm{\Σ}}}{\left[S_{\mathrm{ij}}(x,y)\right]}^2}\sqrt{\underset{x=1}{\overset{n}{\mathrm{\Σ}}}\underset{y=1}{\overset{m}{\mathrm{\Σ}}}{\left[T(x,y)\right]}^2}}---\left(a\right)$

Wherein: n, m mean width and the height of template T, and i, j mean the coordinate position of subgraph in large figure, and x, y mean subgraph S _IjThe intrinsic coordinates value, S _Ij(x, y) is illustrated in subgraph S _IjThe pixel value of the point that middle coordinate is (x, y), T(x, y) be illustrated in the pixel value of the point that in template T, coordinate is (x, y).

For each two field picture, because the scope of coordinate i, the j of subgraph is respectively 0≤i≤W – m and 0≤j≤H – n, so it is inferior that coefficient R needs to calculate (W – m+1) * (H – n+1), calculate and all carry out according to above formula (a) call by value each time, the result of calculation of each frame is stored into a result array, above-mentioned computation process can realize by the built-in function cvMatchTemplate called in OpenCV, then the value of preserving in the result array is carried out to maximal value and ask for, draw maximal value and coordinate thereof.

For example, the screen resolution caught for camera is 640x480, and template size is the situation of 40x60, its Calculation of correlation factor and peaked ask for as follows:

1., to create type in OpenCV be that IplImage*(or other can be stored the type of numerical value) variable result for the result of calculation of the related coefficient of storing each two field picture, its size is (640-40+1) x(480-60+1).

2., the image caught for each frame camera, calculate the related coefficient of itself and template with the built-in function cvMatchTemplate in OpenCV, image mFrame(that the variable that need to import into is present frame size is 640x480), template image mTarget(size is 40x60), related coefficient array result and a grand CV_TM_CCOEFF_NORMED of related coefficient parameter as a result, wherein

CV_TM_CCOEFF_NORMED means to use normalized Calculation of correlation factor, and this function is inserted the result of Calculation of correlation factor in the result array.

3., to the result array, traveled through or use cvMinMaxLoc to be calculated the result array, obtain wherein maximum numerical value and draw its coordinate position, this coordinate position is the best match position in present frame.

(2), position prediction

For the two field picture newly obtained, adopt Kalman filter to predict the best match position in this frame according to the best match position of former frames; Described Kalman filter is comprised of the time predictive equation upgraded and the correction equation of measuring renewal, and the covariance of reckon error forward that the reckoning state variable forward that described predictive equation is calculated by following formula (b) and following formula (c) calculate is constructed the prior estimate of next time state; The kalman gain correction predictive equation that described correction equation is calculated by following formula (d), the posteriority covariance that posteriority is estimated and following formula (f) calculates that following formula (e) calculates are as the prior estimate of calculating next time, and the recurrence of state estimation that at every turn only need be current according to former measurand recursive calculation is calculated.

Calculate forward state variable: ${\hat{x}}_k^{-}=A{\hat{x}}_{k-1}+B{u}_{k-1}---\left(b\right)$

Reckon error covariance forward: $P_k^{-}=A{P}_{k-1}{A}^T+Q---\left(c\right)$

Kalman gain: $K_k=P_k^{-}{H}^T{({\mathrm{HP}}_k^{-}{H}^T+R)}^{-1}---\left(d\right)$

Posteriority is estimated: ${{{\hat{x}}_k={\hat{x}}_k^{-}+K_k(z_k-H{\hat{x}}_k^{-})}_{}}_{}---\left(e\right)$

The posteriority covariance: $P_k=(I-K_{k}H)P_k^{-}---\left(f\right)$

Wherein: Q is the procedure activation noise covariance matrix; A, B are gain matrix; H means state variable x _kTo measurand z _kGain matrix; I is unit matrix.

Consult accompanying drawing 3～accompanying drawing 4, in the predictive equation of time renewal, Kalman filter calculates the prior estimate of next frame according to the posterior estimate obtained before by above formula (b), then by above formula (c), calculate the priori covariance.At first calculate kalman gain K and measure renewal equation _k, secondly measure output to obtain z _k, then use K _kWith z _kPress the posteriority of above formula (e) computing mode and estimate, the posteriority covariance of finally pressing above formula (f) estimated state.In Kalman filter, the posteriority that the last time calculates estimates to be used as the prior estimate of next time calculating, and each need be according to former measurand recursive calculation current state.

In the specific implementation process of position prediction, can adopt and manually not construct Kalman filter, be encapsulated in the structure CvKalman in OpenCV and directly use, structure CvKalman is used for preserving the Kalman filter status, it is created by function cvCreateKalman, by function cvKalmanPredict and function cvKalmanCorrect, upgraded, by function cvReleaseKalman, discharged, to standard K alman wave filter, in OpenCV, the establishment of Kalman filter and renewal process are as follows:

1) create Kalman filter:

By cvCreateKalman function creation object Kalman filter pointer CvKalman*, deposit in variable kalman, wherein state vector x _kDimension, observation vector z _kDimension all is set to 2, in this algorithm, needn't use control vector, and its dimension is made as 0, and the initial setting method of the member variable in variable kalman is as follows:

I, shift-matrix A: can be set to float F[]={ 1,1,0,1}; In the CvKalman structure, transition matrix leaves kalman-in > in transition_matrix.

II, measure matrix H: be set to element on principal diagonal and be entirely 1 diagonal matrix (arranging with cvSetIdentity), be stored in kalman-> in measurement_matrix.

III, process noise covariance matrix Q: be set to element on principal diagonal entirely for the diagonal matrix of 1e-5 (arranging with cvSetIdentity), be stored in kalman-> in process_noise_cov.

IV, measure noise covariance matrix R: be set to element on principal diagonal entirely for the diagonal matrix of 1e-1 (arranging with cvSetIdentity), be stored in kalman-> in measurement_noise_cov.

V, posteriority mistake covariance matrix P: be set to element on principal diagonal and be entirely 1 diagonal matrix (arranging with cvSetIdentity), be stored in kalman-> in error_cov_post.

The line number of above matrix and columns are set according to state vector dimension, observation vector dimension and control vector dimension when creating Kalman filter.

2) upgrade Kalman filter:

Init state vector kalman-> state_post, can be set to the coordinate figure (320,240) of screen center, the first two field picture obtained for camera, need to do following work:

I, call the cvKalmanPredict function and obtain the system prediction value, deposit temporary variable y in _k.

II, obtain the best match position calculated with correlation coefficient process, deposit observational variable z in _k.

III, call the cvKalmanCorrect function, according to observational variable z _kAdjust the Kalman filter current state.

Often obtain the image that a width is new, the step of above-mentioned renewal Kalman filter that circulates, occur until stop the tracking event.

3) release tab Thalmann filter:

When following the tracks of end, call the cvReleaseKalman function and be released in above-mentioned steps 1) the middle variable kalman created.

(3), template renewal

According to predictive variable y _kWith observational variable z _kBetween apart from distance and z _kCorresponding facies relationship numerical value R _maxDetermine whether that renewal works as front template, some y _k(x ₁, y ₁) and some z _k(x ₂, y ₂) between press following formula (g) apart from distance and calculate, when distance≤30: R _maxBe greater than at 0.90 o'clock and continue to follow the tracks of; R _maxBe less than 0.90 and be greater than at 0.85 o'clock new template more; R _maxBe less than at 0.85 o'clock and lose tracking; As distance>30 the time: R _maxBe greater than at 0.85 o'clock and use y _kAs trace point; R _maxBe less than at 0.85 o'clock and lose tracking.

$\mathrm{dis}\tan \mathrm{ce}=\sqrt{{(x_1-x_2)}^2+{(y_1-y_2)}^2}---\left(g\right)$

More than just the present invention is further illustrated, and not in order to limit this patent, all is the present invention's equivalence enforcement, within all should being contained in the claim scope of this patent.

Claims (1)

1. the video frequency object tracking algorithm based on OpenCV, it is characterized in that this video frequency object tracking algorithm is mated with template and the frame of video selected, find out subgraph position the most similar to template image in frame of video by the calculating of related coefficient, the renewal of template is definite according to predicted position and the facies relationship numerical value of Kalman filter, and specific algorithm carries out in the steps below:

(1), template matches

The known little image of take is template, template is overlayed to mobile search target in searched large image, the target area covered by template is subgraph, the whole possible position calculate the related coefficient of subgraph and template on each position of search in searched large image, the related coefficient calculated leaves in an array, wherein position corresponding to maximal value is the subgraph position of mating the most with masterplate, its related coefficient is pressed following formula (a) and is calculated, for each subgraph, calculate corresponding normalized correlation coefficient R (i, j) (1≤R (i, j)≤1), and find out the maximal value R of R _max(i _m, j _m), its corresponding subgraph is the coupling target, when template and subgraph when just the same, and coefficient R (i, j)=1,

$R(i,j)=\frac{\underset{x=1}{\overset{n}{\mathrm{\Σ}}}\underset{y=1}{\overset{m}{\mathrm{\Σ}}}{S}_{\mathrm{ij}}(x,y)*T(x,y)}{\sqrt{\underset{x=1}{\overset{n}{\mathrm{\Σ}}}\underset{y=1}{\overset{m}{\mathrm{\Σ}}}{\left[S_{\mathrm{ij}}(x,y)\right]}^2}\sqrt{\underset{x=1}{\overset{n}{\mathrm{\Σ}}}\underset{y=1}{\overset{m}{\mathrm{\Σ}}}{\left[T(x,y)\right]}^2}}---\left(a\right)$

Wherein: n, m mean width and the height of template T, and i, j mean subgraph S _IjCoordinate position in large figure, x, y mean subgraph S _IjThe intrinsic coordinates value, S _Ij(x, y) is illustrated in subgraph S _IjThe pixel value of the point that middle coordinate is (x, y), T(x, y) be illustrated in the pixel value of the point that in template T, coordinate is (x, y);

(2), position prediction

For the two field picture newly obtained, adopt Kalman filter to predict best match position in this frame according to the best match position of former frames; Described Kalman filter is comprised of the time predictive equation upgraded and the correction equation of measuring renewal, and the covariance of reckon error forward that the reckoning state variable forward that described predictive equation is calculated by following formula (b) and following formula (c) calculate is constructed the prior estimate of next time state; The posteriority covariance that posteriority is estimated and following formula (f) calculates that the kalman gain that described correction equation is calculated by following formula (d), following formula (e) calculate is as the prior estimate of calculating next time, each need be current according to former measurand recursive calculation state;

Calculate forward state variable: ${\hat{x}}_k^{-}=A{\hat{x}}_{k-1}+B{u}_{k-1}---\left(b\right)$

Reckon error covariance forward: $P_k^{-}=A{P}_{k-1}{A}^T+Q---\left(c\right)$

Kalman gain: $K_k=P_k^{-}{H}^T{({\mathrm{HP}}_k^{-}{H}^T+R)}^{-1}---\left(d\right)$

Posteriority is estimated: ${{{\hat{x}}_k={\hat{x}}_k^{-}+K_k(z_k-H{\hat{x}}_k^{-})}_{}}_{}---\left(e\right)$

The posteriority covariance: $P_k=(I-K_{k}H)P_K^{-}---\left(f\right)$

Wherein: A is state-transition matrix, and B is gating matrix or unit matrix; Q is the process noise covariance matrix; H is for measuring matrix; R is for measuring noise covariance matrix; P is posteriority mistake covariance matrix; I is unit matrix; u _K-1For observation noise; z _kThe storage actual measured results, x _kFor state variable;

(3), template renewal

According to predictive variable y _kWith observational variable z _kBetween apart from distance and z _kCorresponding facies relationship numerical value R _maxDetermine whether that renewal works as front template, some y _k(x ₁, y ₁) and some z _k(x ₂, y ₂) between press following formula (g) apart from distance and calculate; When distance≤30: R _maxBe greater than at 0.90 o'clock and continue to follow the tracks of; R _maxBe less than 0.90 and be greater than at 0.85 o'clock new template more; R _maxBe less than at 0.85 o'clock and lose tracking; As distance>30 the time: R _maxBe greater than at 0.85 o'clock and use y _kAs trace point; R _maxBe less than at 0.85 o'clock and lose tracking;

$\mathrm{dis}\tan \mathrm{ce}=\sqrt{{(x_1-x_2)}^2+{(y_1-y_2)}^2}---\left(g\right).$

Images