CN101894378B - Moving target visual tracking method and system based on double ROI (Region of Interest) - Google Patents

Abstract

The invention provides a moving target visual tracking method based on a double ROI (Region of Interest) and a system based on the method, belonging to the technical field of an information technology and being particularly suitable for intelligent video monitoring. The method of the invention acquires the fusing state estimation of double ROI based color information and motion state information according to the motion information of a target and the color probability distribution of the double ROI of a current tracking target, the system comprises a double ROI color characteristic-acquiring module, a Kalman prediction module, a characteristic-fusing and state-estimating module and a target position-confirming module. Compared with the prior art, the invention is simple and effective, has improved tracking rapidness and robustness while keeping the efficiency of an original algorithm and can realize the reentry positioning of the lost target.

Description

Moving target visual tracking method and system based on two area-of-interests

Technical field:

The present invention relates to a kind of vision track technology, be specifically related to a kind of motion target tracking method and system, belong to areas of information technology, be specially adapted to intelligent video monitoring, the tracking quick and precisely of interesting target based on two area-of-interests.

Background technology:

Computer vision is that intelligence machine obtains external information and understands the main path in the world, motion target detection with follow the tracks of one of core topic as computer vision, become the hot research problem of computer vision field in recent years.Motion target detection is a kind of the sequence image that comprises moving target to be carried out analysis and synthesis and to the technology of target localization with following the tracks of; This technological incorporation the advanced technology in fields such as Flame Image Process, pattern-recognition, artificial intelligence, control automatically, computing machine, at aspects such as vision guided navigation, video monitoring, medical diagnosis, industrial detection significant values and wide prospect are arranged.Therefore it is studied and be very important.

In existing vision track technology; The Camshift algorithm is approved by numerous scholars with the simplicity and the high-level efficiency of its calculating, but the Camshift algorithm only utilizes the colouring information of target, can cause the mistake of tracking to the false judgment of color; As the variation of illumination with receive the similar look of large tracts of land with target and disturb just to be easy to cause follow the tracks of and disperse and lose; In the process that target moves,, follow the tracks of the static state of losing previous moment that remains on of window in addition owing to block when perhaps target shifts out video window; When target got into once more, the Camshift algorithm can't continue target caught and continue to follow the tracks of.Though many trackings adopted the method for many Feature Fusion go to compensate Camshift (as with the combining of particle filter) strengthen the robustness of tracking, the calculated amount and the computing time that can increase system like this, sacrificed the performance index of real-time performance of tracking.

Summary of the invention

Goal of the invention:

Technical matters to be solved by this invention is the deficiency to prior art; A kind of motion target tracking method based on two area-of-interests has been proposed; Under the situation of not obvious increase calculated amount, make when computing machine is followed the tracks of moving target, satisfy the requirement of reliability and rapidity and can realize the continuation tracking that target reenters.

Technical scheme:

The present invention adopts following technical scheme for realizing the foregoing invention purpose:

A kind of moving target visual tracking method based on two area-of-interests; This method obtains the state estimation based on two ROI field color information and motion state information fusion according to the movable information of target and the regional color probability distribution of ROI of tracking target; And then the quick real-time tracking and the reentry positioning of realization target, concrete steps comprise:

Step 21 is obtained two ROI field color information of moving target;

Step 22 adopts the Kalman wave filter to obtain the discreet value information of moving target;

Step 23, state merge, obtain the init state estimation of the search window or the direction of search;

Step 24 according to the described state estimation result of step 23, calculates the target centroid position;

Step 25 according to two ROI regional aim centroid positions that step 24 obtains, is judged the state of moving target, and court verdict is delivered to step 23;

Step 26, thus circulation execution in step 21-25 realizes the tracking of dynamic object in the successive frame, until withdrawing from.

Further, the detailed process of obtaining two ROI field color information of the moving target visual tracking method based on two area-of-interests of the present invention is following:

Steps A is confirmed two and follows the tracks of window that said tracking window comprises the target area in first frame of one section video sequence, said target area comprises the object of tracking, and the tracking window shape is provided with according to the shape of target;

Selected ROI#0 as Camshift#0 search matched zone after, with the zone of loseing interest in that ROI#0 is made as Camshift#1, set the tracking of Camshift#1 simultaneously, and ROI#1 be made as the zone of loseing interest in of Camshift#0 ROI#1;

Step B for each frame from second frame, obtains the color characteristic probability distribution graph of two tracing areas of former frame.

Further, it is following to obtain the implementation method of discreet value information in the step 22 of the moving target visual tracking method based on two area-of-interests of the present invention:

Step C, state equation and the observation equation of setting up moving target are following:

X(k)＝AX(k-1)+W(k) (1)

Z(k)＝HX(k)+V(k) (2)

Wherein X (k) is a system in k state vector constantly, and it is vectorial that Z (k) be that k system state is constantly measured, and k is a natural number; The state-noise that W (k) and V (k) are respectively normal distribution with measure noise, and W (k) and V (k) all obey P (W (k))～N (0, Q), P (V (k))～N (0, R) distribute, wherein Q is the covariance matrix of state-noise, R is the covariance matrix of measurement noise; The definition status vector sum is measured vector:

$X\left(k\right)={[x\left(k\right),y\left(k\right),\stackrel{\·}{x}\left(k\right),\stackrel{\·}{y}\left(k\right)]}^T$

Z(k)＝[x(k)，y(k)] ^T

Wherein x (k) and y (k) represent the position of target center of gravity on vision window X axle and Y axle;

and

representes corresponding speed; System state matrix A and measurement matrix H have with reference to X (k) and Z (k):

$A=\left[\begin{array}{cccc}1& 0& \mathrm{<figure-callout\; id="0"\; label="T"\; filenames="HSA00000156647000032.png"\; state="\{\{state\}\}">T</figure-callout>}& 0\\ 0& 1& 0& \mathrm{<figure-callout\; id="0"\; label="T"\; filenames="HSA00000156647000032.png"\; state="\{\{state\}\}">T</figure-callout>}\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right],$ $H=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\end{array}\right],$ T is the sampling time;

Step D, adopt Kalman filter equation predicted position information (x (k|k-1), y (k|k-1)):

Update stage:

X(k|k-1)＝AX(k-1|k-1) (3)

P(k|k-1)＝AP(k-1|k-1)A ^T+Q (4)

Calibration phase:

K _g(k)＝P(k|k-1)H ^T/(HP(k|k-1)H ^T+R) (5)

X(k|k)＝X(k|k-1)+K _g(k)(Z(k)-HX(k|k-1)) (6)

P(k|k)＝(I-K _g(k)H)P(k|k-1) (7)

In above formula (3)-(7), X (k|k-1) utilizes the laststate prediction result, and X (k-1|k-1) is a k-1 optimal value constantly, and P (k|k-1) is the prior uncertainty correlation matrix, and P (k-1|k-1) is the posteriority error correlation matrix of back, K _g(k) be the Kalman gain matrix, Z (k) is given measurement parameter;

With the positional information (x (k|k-1), y (k|k-1)) among the predicted state vector X (k|k-1) is that the discreet value information of moving target is sent to step 23.

Further, it is following to obtain the implementation method that the init state of the search window or the direction of search estimates in the step 23 of the moving target visual tracking method based on two area-of-interests of the present invention:

The target state judged result that obtains according to last round of step 25 place is not if target loses with separating and then adopt method for estimating state, according to the size and the position of the search window of the initial present frame of size of the search window of discreet value and former frame;

If target is lost in video window and is separated then size according to the search window of the initial present frame of size and position of the search window of former frame with the position and give the position that the window certain speed information of separating comes the linear prediction target in present frame, possibly occur;

If shifting out search window, target then comes the initialization direction of search by the velocity information of regulation.

Further, in the moving target visual tracking method based on two area-of-interests of the present invention:

E, said in video window, losing when target given the separator window certain speed position that information comes the linear prediction target in present frame, possibly occur when separating, and its velocity information is confirmed as follows:

$\stackrel{\&CenterDot;}{x}\left(k\right)=\frac{x\left(k\right|k-1)-x(k-1)}{T}---\left(8\right)$

$\stackrel{\&CenterDot;}{y}\left(k\right)=\frac{y\left(k\right|k-1)-y(k-1)}{T}---\left(9\right)$

Wherein, T is the time interval of two image frame grabbers, and (x (k|k-1), y (k|k-1)) is search window location estimation information, and (x (k-1), y (k-1)) is the previous frame target position information;

F, if said target shifts out search window and then comes the initialization direction of search by the velocity information of regulation, and wherein the velocity information regulation is as follows:

$\stackrel{\&CenterDot;}{x}\left(k\right)=\frac{x_m\left(k\right)-x_l}{T}---\left(10\right)$

$\stackrel{\&CenterDot;}{y}\left(k\right)=\frac{y_m\left(k\right)-y_l}{T}---\left(11\right)$

(x wherein _m(k), y _mThe centre coordinate of search window when (k)) bumping for search window and video window, (x _l, y _l) coordinate of the point that will shift to for window collision rear hatch, (x _l, y _l) be the random point in the form, or preestablish.

Further, of the present invention following based on calculating the implementation method of obtaining the target centroid position in the moving target visual tracking method of two area-of-interests:

G is not if target has to depart from or lose the centroid position that then obtains target by the Camshift algorithm;

H, if target has taken place to lose in video window and departed from and then can use the position that formula (8)-(9) linear prediction target possibly occur in present frame, concrete centroid position computing formula is following:

$x(k+1)=x\left(k\right)+T\stackrel{\&CenterDot;}{x}\left(k\right)---\left(12\right)$

Wherein x (k+1) is the target centroid position of k+1 frame; X (k) is the target centroid position of k frame;

is the target systemic velocity of k frame, and T is the time interval of two image frame grabbers;

I if target has shifted out video window, then constantly searches for by the direction of search that formula (10)-give (11), till searching target.

Further, the implementation method based on the state that obtains moving target in the step 25 in the moving target visual tracking method of two area-of-interests of the present invention is following:

J, the position of following the tracks of window center through two ROI models two concern to judge whether follow the tracks of windows for two takes place to separate or lose:

$\sqrt{{(x_{\mathrm{dc}}-x_{\mathrm{uc}})}^2+{(y_{\mathrm{dc}}-y_{\mathrm{uc}})}^2}<E---\left(13\right)$

(x wherein _Dc, y _Dc), (x _Uc, y _Uc) being respectively the centre coordinate of Camshift#0 and Camshift#1 tracing area, E is a threshold value;

The size of K, the centroid position through two ROI model following windows and video window is confirmed to follow the tracks of window and whether is shifted out video window.

A kind of moving target vision track system that uses of the present invention based on two area-of-interests, this system comprise that two ROI field color characteristic acquisition modules, Kalman estimate module, Feature Fusion and state estimation module, target location determination module, wherein:

Two ROI field color characteristic acquisition modules are used to obtain the color characteristic information in two ROI zone, and send Feature Fusion and state estimation module to;

Kalman estimates module, is used for obtaining according to the position detection value of centroid position predicted value and current time the discreet value of next frame, and to Feature Fusion and the state estimation module of sending;

Feature Fusion state estimation module, the discreet value that color characteristic information that is used for two ROI field color characteristic acquisition modules are obtained and Kalman estimate module, the size of initialization search window, centroid position or the direction of search;

The target location determination module is used for obtaining through interative computation according to the estimated value of Feature Fusion state estimation module the target location of present frame.

Further; Moving target vision track system based on two area-of-interests of the present invention also comprises moving target state determination module; Be used to judge the state of target following window; If follow the tracks of window in form and the centroid position between two windows greater than certain threshold value, judge that then it departs from and loses.

Beneficial effect:

Compared with prior art, the present invention is simply effective, when not sacrificing original efficiency of algorithm, has increased the rapidity of tracking and the location again that robustness also can realize target.Its key is: adopted two ROI models; Realized the window of tracing area is separated differentiation; Be adapted to different scene, obtained good tracking effect, be particularly useful for target by the situation that has the similar look of large tracts of land to disturb in partial occlusion or the targeted environment; Introduce the speed that Kalman prediction device model has improved the CAMSHIFT algorithm, block in the window and orientation problem again through separating effectively to the utilization of Kalman prediction device velocity information; Target is passed through the definition velocity information after shifting out video window, makes search window obtain the certain direction of search and speed, thereby realizes the quick location after target reenters window.This paper has provided the tracking effect figure when the reentry positioning figure behind the track rejection and target receive similar look and disturb, and is as shown in Figure 5.

Utilize the present invention to carry out vision track, promptly can also can be used as the intermediate result that vision is understood and analyzed as tracking results.The present invention has application fields and prospect, can be applied in various fields such as traffic monitoring, industrial detection, medical research, military investigation, navigational guidance, virtual reality, visual servos.Video monitoring with security fields is an example, if just can realize the existing video monitoring system of this technical application the active of target is followed the tracks of, thereby saves safety monitoring personnel's plenty of time and alleviates their work load intensity.

Description of drawings:

Fig. 1 is the vision track structural representation of the embodiment of the invention based on two ROI models;

Fig. 2 is the principle process flow diagram of the embodiment of the invention based on the visual tracking method of two ROI models;

Fig. 3 carries out the method flow diagram of a specific embodiment of target following for utilization principle shown in Figure 2;

Fig. 4 is the two ROI model synoptic diagram of the present invention;

Fig. 5 a-f reentry positioning design sketch that to be two ROI models proposed by the invention shift out video window to tracking effect figure and the target of target when disturbed by the similar look of large tracts of land.

Specific embodiments:

Below in conjunction with accompanying drawing the enforcement of technical scheme is done further to describe in detail:

Referring to Fig. 1 is the vision track synoptic diagram of the embodiment of the invention based on two ROI models, and this system comprises two ROI field color information acquisition modules 11, Feature Fusion and state estimation module 12, Kalman estimates module 13 and target location determination module 14.

Two ROI field color information acquisition modules 11 obtain the colouring information of area-of-interest, send Feature Fusion and state estimation module 12 to, and wherein the selection rule of two ROI models sees step 21. for details

Kalman estimates module 13, obtains the predicted position and the velocity information of target, sends Feature Fusion and state estimation module 12 to, and the characteristic information of its transmission receives the influence of Window state, specifically sees step 22.

Feature Fusion and state estimation module 12 are according to size, centroid position or the direction of search and the power of the characteristic information initialization search window of the state of window and acquisition.Wherein the judgement of the state of window sees the description at step 25 place for details.

Target location determination module 14, through the determined target position information in two ROI zone of interative computation acquisition present frame, and according to two states of following the tracks of windows of two ROI window center position relation judgement.These status informations comprise whether separate, whether track rejection takes place if following the tracks of window for two, concrete criterion sees step 25. for details

Referring to Fig. 2, be the principle process flow diagram of the embodiment of the invention based on the visual tracking method of two ROI models, this method may further comprise the steps:

Step 21 is obtained two ROI field color information.Its detailed process is following:

At first, in first frame of one section video sequence, confirm two and follow the tracks of window that said tracking window comprises the target area, said target area comprises the object of tracking, and the tracking window shape is provided with according to the shape of target.This paper modelling is as shown in Figure 4.After the selected ROI#0 of Camshift#0 is as the search matched zone; ROI#0 is made as the zone of loseing interest in of Camshift#1; Set the tracking of Camshift#1 simultaneously to ROI#1; And ROI#1 is made as the zone of loseing interest in of Camshift#0, the purpose of more than setting is the regional mutual exclusion that lets a tracker that another tracker is concerned about.

Then, for each frame, obtain the color characteristic probability distribution graph of two tracing areas of former frame from second frame.

Step 22 is obtained Kalman and is estimated position, velocity information.Implementation method is following when concrete:

The state equation of system and observation equation can be expressed as:

X(k)＝AX(k-1)+W(k) (1)

Z(k)＝HX(k)+V(k) (2)

Wherein X (k) is a system in k state vector constantly, and Z (k) be that k system state is constantly measured vector, and W (k) and V (k) are respectively the state-noise and the measurement noise of normal distribution; And obey P (W)～N (0; Q), and P (V)～N (0, R) distribute; Wherein Q is the covariance matrix of state-noise, and R is for measuring the covariance matrix of noise.In the vision track of Camshift method, because the interval time of consecutive frame image is shorter, target travel changes less, and be approximately target and between two frames, move with uniform velocity, uniform motion modelings such as this paper employing, the definition status vector sum is measured vector and is:

$X\left(k\right)={[x\left(k\right),y\left(k\right),\stackrel{\&CenterDot;}{x}\left(k\right),\stackrel{\&CenterDot;}{y}\left(k\right)]}^T$

Z(k)＝[x(k)，y(k)] ^T

Wherein x (k) and y (k) represent the position of target center of gravity on vision window X axle and Y axle, and and

representes corresponding speed.System state matrix A and measurement matrix H have with reference to X (k) and Z (k):

$A=\left[\begin{array}{cccc}1& 0& \mathrm{<figure-callout\; id="0"\; label="T"\; filenames="HSA00000156647000032.png"\; state="\{\{state\}\}">T</figure-callout>}& 0\\ 0& 1& 0& \mathrm{<figure-callout\; id="0"\; label="T"\; filenames="HSA00000156647000032.png"\; state="\{\{state\}\}">T</figure-callout>}\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right],$ $H=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\end{array}\right],$ T is the sampling time.

The Kalman wave filter this prediction and proofread and correct two stages and be:

Update stage:

X(k|k-1)＝AX(k-1|k-1) (3)

P(k|k-1)＝AP(k-1|k-1)A ^T+Q (4)

Calibration phase:

K _g(k)＝P(k|k-1)H ^T/(HP(k|k-1)H ^T+R) (5)

X(k|k)＝X(k|k-1)+K _g(k)(Z(k)-HX(k|k-1)) (6)

P(k|k)＝(I-K _g(k)H)P(k|k-1) (7)

In above formula (3)-(7), X (k|k-1) utilizes the laststate prediction result, and X (k-1|k-1) is a k-1 optimal value constantly, and P (k|k-1) is the prior uncertainty correlation matrix, and P (k-1|k-1) is the posteriority error correlation matrix of back, K _g(k) be the Kalman gain matrix, Z (k) is given measurement parameter.This step will be sent to step 23 place to positional information (x (k|k-1), y (k|k-1)) and the corresponding velocity information among the predicted state vector X (k|k-1).

Step 23, the state estimation of state fusion, search window size and centroid position or the direction of search, the size and the position of predicting the search window of present frame in principle according to the positioning result of former frame.

Judged result according to step 25 place; If target is still separated (the similar look of large tracts of land disturbs) and track rejection (blocking situation) in video window and not; At this moment the positional information that transmits according to step 22 place (x (k|k-1); Y (k|k-1)) center of initialization Camshift search window, and proofread and correct Kalman prediction device model to the centre of gravity place of Camshift algorithm output as measured value, and then obtain Kalman optimization estimated value X (k|k); Be applied in the formula (3), circulation finishes up to algorithm again; If separation or track rejection situation have still still taken place in target in video window; Because the interval time of consecutive frame image is shorter; Target travel changes less; Be approximately target and between two frames, move with uniform velocity, the position that can use corresponding velocity information linear prediction target in present frame, possibly occur, step 34 places are seen in concrete realization; If target is moved out to outside the video window, when not searching target yet after search window and the form collision, at this moment give the motion window new velocity information:

$\stackrel{\&CenterDot;}{x}\left(k\right)=\frac{x_m\left(k\right)-x_l}{T}---\left(8\right)$

$\stackrel{\&CenterDot;}{y}\left(k\right)=\frac{y_m\left(k\right)-y_l}{T}---\left(9\right)$

(x wherein _m(k), y _mThe centre coordinate of search window when (k)) bumping for search window and video window, (x _l, y _l) coordinate of the point that will shift to for window collision rear hatch, (x _l, y _l) can be the random point in the form, also can preestablish.

The velocity of target gives track rejection or receives similar look to disturb " direction " and " power " of search (tracking) window in back to target search; Make search window with direction of motion and speeds before the target, lose or receive the target of similar look interference with search.

Step 24 calculates the target centroid position according to state estimation.Based on the result of step 25,, obtain the actual centroid position of target as follows if target is not lost or followed the tracks of window and do not separate.If target is lost or the seizure that then should at first accomplish target by corresponding velocity information appears separating in window.

Transmit search window location estimation information (x (k|k-1), y (k|k-1)) obtains present frame through the Camshift algorithm two center point coordinates of following the tracks of window according to probability distribution graph and step 23 place.The core process of Camshift algorithm comprises: calculate zeroth order square (formula 10) and first moment (the formula 11 and 12) through type (13) of following the tracks of window, formula (14) iterative computation (x _c, y _c) coordinate, the coordinate when this coordinate does not have obvious displacement or iterates to maximum times is exactly the tracking window center of present frame.

$M_{00}=\underset{x}{\mathrm{\&Sigma;}}\underset{y}{\mathrm{\&Sigma;}}I(x,y)---\left(10\right)$

$M_{10}=\underset{x}{\mathrm{\&Sigma;}}\underset{y}{\mathrm{\&Sigma;}}\mathrm{xI}(x,y)---\left(11\right)$

$M_{01}=\underset{x}{\mathrm{\&Sigma;}}\underset{y}{\mathrm{\&Sigma;}}\mathrm{yI}(x,y)---\left(12\right)$

x _c＝M ₁₀/M ₀₀ (13)

y _c＝M ₀₁/M ₀₀ (14)

Step 25, the state of judgement search window.This video window size is certain, can judge whether it bumps with video window according to the centre coordinate of search window.If the centre coordinate (x of search window when colliding then collision _m(k), y _m(k)) be sent to step 23 place; If target is appointed so in video window; But the situation that two ROI follow the tracks of window separation or track rejection has taken place; Then deliver to step 23 places to the velocity information at step 22 place (specifically seeing step 34); If the separation of window and the situation of track rejection do not take place to follow the tracks of, then deliver to step 23 places to the positional information at step 22 place.Provide the criterion that window separates or loses below:

Adopt four-dimensional vector that ROI#0 and ROI#1 are expressed as (x respectively _d, y _d, w _d, h _d) and (x _u, y _u, w _u, h _u), (x wherein _d, y _d), (x _u, y _u) be a left side, the end coordinate of ROI#0 and ROI#1 iterative window, (w _d, h _d) and (w _u, h _u) be the width and the height of ROI#0 and ROI#1 iterative window, this paper initial setting w _d=w _u, h _d=3h _u

Through two ROI models target is followed the tracks of, judges according to the position relation of two window center whether follow the tracks of window for two takes place to separate or lose:

$\sqrt{{(x_{\mathrm{dc}}-x_{\mathrm{uc}})}^2+{(y_{\mathrm{dc}}-y_{\mathrm{uc}})}^2}<E---\left(15\right)$

(x wherein _Dc, y _Dc), (x _Uc, y _Uc) being respectively the centre coordinate of Camshift#0 and Camshift#1 tracing area, E is a threshold value.If formula (15) is set up, explain that Camshift#0 and Camshift#1 have robustness preferably.

If formula (15) is false, explain that one of them follows the tracks of window and depart from or lose, consider the continuity of target interframe movement this moment, and the motion change of adjacent two frames is not too large, and then the variation of the center of present frame and previous frame is not too large.

$\sqrt{{(x_{\mathrm{dc}}\left(k\right)-x_{\mathrm{dc}}(k-1))}^2+{(y_{\mathrm{dc}}\left(k\right)-y_{\mathrm{dc}}(k-1))}^2}<E_1---\left(16\right)$

$\sqrt{{(x_{\mathrm{uc}}\left(k\right)-x_{\mathrm{uc}}(k-1))}^2+{(y_{\mathrm{uc}}\left(k\right)-y_{\mathrm{uc}}(k-1))}^2}>E_1---\left(17\right)$

If above two formulas are set up, illustrate that the Camshift#1 tracing area departs from or loses, and adjusts tracking direction and the zone of Camshift#1 based on step 23.

$\sqrt{{(x_{\mathrm{dc}}\left(k\right)-x_{\mathrm{dc}}(k-1))}^2+{(y_{\mathrm{dc}}\left(k\right)-y_{\mathrm{dc}}(k-1))}^2}>E_1---\left(18\right)$

$\sqrt{{(x_{\mathrm{uc}}\left(k\right)-x_{\mathrm{uc}}(k-1))}^2+{(y_{\mathrm{uc}}\left(k\right)-y_{\mathrm{uc}}(k-1))}^2}<E_1---\left(19\right)$

If above two formulas are set up, explain that the Camshift#0 tracing area departs from or loses, and adjusts tracking direction and the zone of Camshift#0.(x in formula (16)-(19) _Dc(k), y _Dc(k)), (x _Uc(k), y _Uc(k)) be respectively the centre coordinate that Camshift#0 and Camshift#1 tracing area present frame are followed the tracks of window, (x _Dc(k-1), y _Dc(k-1)), (x _Uc(k-1), y _Uc(k-1)) be respectively the centre coordinate that Camshift#0 and Camshift#1 former frame are followed the tracks of window.

Such scheme is the ultimate principle of visual tracking method of the present invention.Step 21-23 obtains the process that motion feature information and colouring information merge the acquisition state estimation for the present invention among Fig. 2; Step 24 is calculating processes that utilization Camshift algorithm obtains the target location; Step 25 is differentiated process for Window state, and it has determined the step 23 to adopt which kind of useful information.Below in conjunction with specific embodiment shown in Figure 3, the method under Fig. 2 is elaborated.

Referring to Fig. 3, carry out the method flow diagram of a specific embodiment of vision track for using principle shown in Figure 2, this method may further comprise the steps:

Step 31 is selected home window.

In first frame of video sequence, confirm two and follow the tracks of window; Follow the tracks of size, the shape of window and distribute and confirm by operator oneself based on characteristics such as the size of tracked target, shapes. two tracking windows are distributed up and down; The rectangle red block is the Camshift#0 tracing area, and blue frame is the Camshift#1 tracking area;

Step 32 is obtained two ROI field color information.

The RGB image transitions to the hsv color space, is handled through image denoising,, obtained the color characteristic probability distribution graph of two tracing areas of former frame from each frame of second frame.

Step 33, position and velocity information when obtaining the Kalman discreet value or obtaining lose objects.

The location estimation information (x (k|k-1), y (k|k-1)) among the corresponding predicted state vector X (k|k-1), position correction information x (k-1), y (k-1) is sent to step 34 place.The acquisition of state vector can obtain according to the method at step 22 place, gets in this experiment: Q=diag (881616), and R=diag (0.10.2), P0=diag (1111), P0 are the initial error variance matrix.

Step 34, the direction of search of state fusion, estimating searching window size, centroid position or definite search window.

If still in video window and taking place to follow the tracks of window, target separates and track rejection, according to the center of positional information (x (k|k-1), y (k|k-1)) initialization Camshift search window at step 33 places; If separation or track rejection situation have still still taken place, the position that can use velocity information ((20)-(21) formula) linear prediction target in present frame, possibly occur in target in video window; If target is moved out to outside the video window, when not searching target yet after search window and the form collision, at this moment give shown in (8) of the new velocity information of motion window like step 23 place-(9) formula.

$\stackrel{\&CenterDot;}{x}\left(k\right)=\frac{x\left(k\right|k-1)-x(k-1)}{T}---\left(20\right)$

$\stackrel{\&CenterDot;}{y}\left(k\right)=\frac{y\left(k\right|k-1)-y(k-1)}{T}---\left(21\right)$

Wherein, T is the time interval of two image frame grabbers.

Step 35 calculates the target centroid position.

Shown in step 24, obtain two center point coordinates of following the tracks of window of present frame through the Camshift algorithm according to search window location estimation information (x (k|k-1), y (k|k-1)); If lose in the window or disturb then set by step the velocity information at 34 places by the center point coordinate of the tracking window of

linear prediction present frame; If the target grand window then set by step the velocity reversal at 23 places search for.Follow the tracks of window centre coordinate x in the present embodiment _cAnd y _cValue change less than 1 or coordinate when iterating to maximum times 10 times be exactly the target centroid position of present frame.

Step 36, the state of the two ROI range searching windows of judgement.

State according to step 25 place, obtain two and follow the tracks of window centre coordinate (x _Dc, y _Dc), (x _Uc, y _Uc), judge whether and video window (form is 320 * 240 pixel windows) bumps that because of video window marginal position noise is strong, be judged as by search window easily " target ", setting " collision edge remaining " in the present embodiment is 3 pixels.If collision, the centre coordinate ((x of search window when acquisition search window and video window bump _m(k), y _mAnd set the coordinate (x of the point that window collision rear hatch will shift to (k)), _l, y _l), this paper is set at the center of video window.If whether not collision is then differentiated the tracking window based on (15)-(19) formula and taken place to separate or lose, wherein the E value can be set based on video window and target sizes, then its state is delivered to the estimation that step 34 place carries out state.

Referring to Fig. 5 a-f, be the tracing process synoptic diagram of the embodiment of the invention based on the motion target tracking method of two area-of-interests.A among Fig. 5 is an initialized pair of ROI area tracking window; Figure b is the tracking effect figure of target when receiving similar look and disturbing; As can be seen from the figure, owing to receive the interference of similar look, the blue window of following the tracks of has the sign of dispersing; But because this paper has adopted two ROI regional models and the corresponding adjustment criterion of separating; Can make the tracking window of dispersing constantly obtain adjustment, lock onto the generation that stops to follow the tracks of this situation on the interference colour thereby avoided following the tracks of window, adjusted tracking effect is shown in figure c, d; Figure e is that target shifts out the direction of search synoptic diagram of following the tracks of window after video window and tracking window and video window bump; Figure f is after target reenters video window, follows the tracks of seizure location and the tracking synoptic diagram of window to target.

Protection scope of the present invention is not limited to the above embodiments.

Claims (6)

1. moving target visual tracking method based on two area-of-interests; It is characterized in that; This method obtains the state estimation based on two ROI field color information and motion state information fusion according to the movable information of target and the regional color probability distribution of two ROI of tracking target; And then the quick real-time tracking and the reentry positioning of realization target, concrete steps comprise:

Step 21 is obtained two ROI field color information, and detailed process is following:

Steps A is confirmed two and follows the tracks of window that said tracking window comprises the target area in first frame of one section video sequence, said target area comprises the object of tracking, and the tracking window shape is provided with according to the shape of target;

A selected ROI zone as a Camshift search matched zone after; The one ROI zone is made as the zone of loseing interest in the 2nd Camshift search matched zone; Set of the tracking of the 2nd Camshift search matched zone simultaneously, and the 2nd ROI zone is made as the zone of loseing interest in Camshift search matched zone the 2nd ROI zone;

Step B for each frame from second frame, obtains the color characteristic probability distribution graph of two tracing areas of former frame;

Step 22 adopts the Kalman wave filter to obtain the discreet value information of moving target;

Step 23, state merge, obtain the init state estimation of the search window or the direction of search, and implementation method is following:

The target state judged result that obtains according to last round of step 25 place is not if target loses with separating and then adopt method for estimating state, according to the size and the position of the search window of the initial present frame of size of the search window of discreet value and former frame;

If target is lost in video window and is separated then size according to the search window of the initial present frame of size and position of the search window of former frame with the position and give the position that the window certain speed information of separating comes the linear prediction target in present frame, possibly occur;

If shifting out search window, target then comes the initialization direction of search by the velocity information of regulation;

Step 24 according to the described state estimation result of step 23, calculates the target centroid position;

Step 25 according to two ROI regional aim centroid positions that step 24 obtains, is judged the state of moving target, and court verdict is delivered to step 23;

Step 26, thus circulation execution in step 21-25 realizes the tracking of dynamic object in the successive frame, until withdrawing from.

2. the moving target visual tracking method based on two area-of-interests according to claim 1, the implementation method that it is characterized in that obtaining in the said step 22 discreet value information is following:

Step C, state equation and the observation equation of setting up moving target are following:

X(k)＝AX(k-1)+W(k) (1)

Z(k)＝HX(k)+V(k) (2)

Wherein X (k) is a system in k state vector constantly, and it is vectorial that Z (k) be that k system state is constantly measured, and k is a natural number; The state-noise that W (k) and V (k) are respectively normal distribution with measure noise, and W (k) and V (k) all obey P (W (k))～N (0, Q), P (V (k))～N (0, R) distribute, wherein Q is the covariance matrix of state-noise, R is the covariance matrix of measurement noise; The definition status vector sum is measured vector:

$X\left(k\right)={[x\left(k\right),y\left(k\right),\stackrel{\&CenterDot;}{x}\left(k\right),\stackrel{\&CenterDot;}{y}\left(k\right)]}^T$

Z(k)＝[x(k)，y(k)] ^T

Wherein x (k) and y (k) represent the position of target center of gravity on vision window X axle and Y axle;

and

representes corresponding speed; System state matrix A and measurement matrix H have with reference to X (k) and Z (k):

$A=\left[\begin{array}{cccc}1& 0& T& 0\\ 0& 1& 0& T\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right],$ $H=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\end{array}\right],$ T is the sampling time;

Step D, adopt Kalman filter equation predicted position information (x (k|k-1), y (k|k-1)):

Update stage:

X(k|k-1)＝AX(k-1|k-1) (3)

P(k|k-1)＝AP(k-1|k-1)A ^T+Q (4)

Calibration phase:

K _g(k)＝P(k|k-1)H ^T/(HP(k|k-1)H ^T+R) (5)

X(k|k)＝X(k|k-1)+K _g(k)(Z(k)-HX(k|k-1)) (6)

P(k|k)＝(I-K _g(k)H)P(k|k-1) (7)

In above formula (3)-(7), X (k|k-1) utilizes the laststate prediction result, and X (k-1|k-1) is a k-1 optimal value constantly, and P (k|k-1) is the prior uncertainty correlation matrix, and P (k-1|k-1) is the posteriority error correlation matrix of back, K _g(k) be the Kalman gain matrix, Z (k) is given measurement parameter;

With the positional information (x (k|k-1), y (k|k-1)) among the predicted state vector X (k|k-1) is that the discreet value information of moving target is sent to step 23.

3. the moving target visual tracking method based on two area-of-interests according to claim 1 is characterized in that, in the step 23:

Step e, said in video window, losing when target given the separator window certain speed position that information comes the linear prediction target in present frame, possibly occur when separating, and its velocity information is confirmed as follows:

$\stackrel{\&CenterDot;}{x}\left(k\right)=\frac{x\left(k\right|k-1)-x(k-1)}{T}---\left(8\right)$

$\stackrel{\&CenterDot;}{y}\left(k\right)=\frac{y\left(k\right|k-1)-y(k-1)}{T}---\left(9\right)$

Wherein, T is the time interval of two image frame grabbers, and (x (k|k-1), y (k|k-1)) is search window location estimation information, and (x (k-1), y (k-1)) is the previous frame target position information;

If step F, said target shift out search window and then come the initialization direction of search by the velocity information of regulation, and wherein the velocity information regulation is as follows:

$\stackrel{\&CenterDot;}{x}\left(k\right)=\frac{x_m\left(k\right)-x_l}{T}---\left(10\right)$

$\stackrel{\&CenterDot;}{y}\left(k\right)=\frac{y_m\left(k\right)-y_l}{T}---\left(11\right)$

(x wherein _m(k), y _mThe centre coordinate of search window when (k)) bumping for search window and video window, (x _l, y _l) coordinate of the point that will shift to for window collision rear hatch, (x _l, y _l) be the random point in the form, or preestablish.

4. the moving target visual tracking method based on two area-of-interests according to claim 3, it is following to it is characterized in that the implementation method of target centroid position is obtained in the said calculating of step 24:

Step G is not if target has to depart from or lose the centroid position that then obtains target by the Camshift algorithm;

Step H, if target has taken place to lose in video window and departed from and then use the position that formula (8)-(9) linear prediction target possibly occur in present frame, concrete centroid position computing formula is following:

$x(k+1)=x\left(k\right)+T\stackrel{\&CenterDot;}{x}\left(k\right)$ (12)

Wherein x (k+1) is the target centroid position of k+1 frame; X (k) is the target centroid position of k frame; is the target systemic velocity of k frame, and T is the time interval of two image frame grabbers;

Step I if target has shifted out video window, then constantly searches for by the direction of search that formula (10)-give (11), till searching target.

5. the moving target visual tracking method based on two area-of-interests according to claim 1, the implementation method of state that it is characterized in that obtaining in the said step 25 moving target is following:

Step J, the position of following the tracks of window center through two ROI models two concern to judge whether follow the tracks of windows for two takes place to separate or lose:

$\sqrt{{(x_{\mathrm{dc}}-x_{\mathrm{uc}})}^2+{(y_{\mathrm{dc}}-y_{\mathrm{uc}})}^2}<E---\left(13\right)$

(x wherein _Dx, y _Dc), (x _Uc, y _Uc) being respectively Camshift search matched zone and the regional centre coordinate of the 2nd Camshift search matched, E is a threshold value;

The size of step K, the centroid position through two ROI model following windows and video window is confirmed to follow the tracks of window and whether is shifted out video window.

6. an application rights requires 1 the moving target vision track system based on two area-of-interests; It is characterized in that: this system comprises that two ROI field color characteristic acquisition modules, Kalman estimate module; Feature Fusion and state estimation module, target location determination module, moving target state determination module, wherein:

Two ROI field color characteristic acquisition modules are used to obtain the color characteristic information in two ROI zone, and send Feature Fusion and state estimation module to, and detailed process is following:

1. step confirms two and follows the tracks of window that said tracking window comprises the target area in first frame of one section video sequence, said target area comprises the object of tracking, and the tracking window shape is provided with according to the shape of target;

A selected ROI zone as a Camshift search matched zone after; The one ROI zone is made as the zone of loseing interest in the 2nd Camshift search matched zone; Set of the tracking of the 2nd Camshift search matched zone simultaneously, and the 2nd ROI zone is made as the zone of loseing interest in Camshift search matched zone the 2nd ROI zone;

2. step for each frame from second frame, obtains the color characteristic probability distribution graph of two tracing areas of former frame;

Kalman estimates module, is used for obtaining according to the position detection value of centroid position predicted value and current time the discreet value of next frame, and to Feature Fusion and the state estimation module of sending;

Feature Fusion state estimation module, the discreet value that color characteristic information that is used for two ROI field color characteristic acquisition modules are obtained and Kalman estimate module, the size of initialization search window, centroid position or the direction of search, detailed process is following:

The target state judged result that obtains according to moving target state determination module is not if target loses with separating and then adopt method for estimating state, according to the size and the position of the search window of the initial present frame of size of the search window of discreet value and former frame;

If target is lost in video window and is separated then size according to the search window of the initial present frame of size and position of the search window of former frame with the position and give the position that the window certain speed information of separating comes the linear prediction target in present frame, possibly occur;

If shifting out search window, target then comes the initialization direction of search by the velocity information of regulation;

The target location determination module is used for obtaining through interative computation according to the estimated value of Feature Fusion state estimation module the target location of present frame;

Moving target state determination module is used to judge the state of target following window, if follow the tracks of window in form and the centroid position between two windows greater than certain threshold value, judge that then it departs from and loses.

Images