CN103106667A - Motion target tracing method towards shielding and scene change - Google Patents

Abstract

The invention discloses a motion target tracing method towards shielding and scene change. The motion target tracing method towards shielding and scene change comprises the following steps: a, conducting prospect motion detection on input video sequence, and extracting a motion target; b, if feature of a tracing target is stored, then entering step d; if feature of the tracing target is not stored, then completing formwork initialization of a target object, SURF feature extraction and initialization of a Kalman filter according to area selected by a user; c, adopting a method based on the Kalman filter to conduct prediction tracing on a motion target, and entering step e till the video content ends; when shielding occurs in the tracing process, then entering step d; d, utilizing matching method based on the SURF feature to determine the tracing target, when feature matching tends to stabilize and judge the shielding as ending, reinitializing the Kalman filter and entering step c; e, outputting and storing the target object feature information. The motion target tracing method towards shielding and scene change is a complete set of video motion target tracing method applied to a fixed background monocular camera and can be manufactured into software, and application is convenient.

Description

It is a kind of towards the Moving Objects method for tracing blocked with scene change

Technical field

The invention belongs to image processing techniques, the tracer technique field of Moving Objects.Concretely relate to be combined based on kalman filtering and SURF methods to realize quick, the accurate method for tracing blocked with Moving Objects in the case of scene change.

Background technology

Current Moving Objects in Video Sequences method for tracing has following several：

One is the tracking based on region, and it is partitioned into the object video of each frame first, the corresponding relation then set up between each cutting object, so as to realize the tracking to object video.Segmentation of this method to object video requires very high, once the Object Segmentation mistake of a certain frame or a few frames in video segment, the then tracking of whole object video will fail.

Two be Graph Cuts methods（Also known as Min-Cut/Max-Flow methods）, it is a kind of classical image partition method, many image partition methods are all based on what Graph Cuts were derived at present.Because the back of the tracking of Moving Objects is generally all the extraction of sport foreground object, therefore this tracking based on region is using than wide.But due to this method can not Ground Split is mutually blocked very well object, therefore this method effect in the scene frequently occurred is blocked is poor.

Three be the tracking based on model, and the tracking for being currently based on model is broadly divided into two classes：Human body tracking i.e. based on model and the vehicle tracking based on model.Due to the characteristic of this method, after the corresponding relation of object 2D image coordinates and 3D coordinates is obtained, even if largely angular transformation occurs for object, it can also be tracked using the 3D models of object.This method requirement is first modeled to tracked object, is then matched the content in the model and video image to realize tracking；And this method requirement has enough prioris to tracked object, could set up out effective object module.

The content of the invention

The task of the present invention is that offer is a kind of towards the Moving Objects method for tracing blocked with scene change, and this method can quickly and accurately realize the tracking to special exercise object in video.

Its technical solution is：

It is a kind of towards the Moving Objects method for tracing blocked with scene change, comprise the following steps：

A carries out foreground moving detection to the video sequence of input, extracts Moving Objects；Subsequently into step b,

If b has preserved the feature of tracing object, into step d；If not preserving the feature of tracing object, the region selected according to user completes the template initialization and SURF feature extractions to target object, and Kalman filter initialization；Subsequently into step c,

C is predicted tracking to moving target using the method based on Kalman filter, until video content terminates, into step e；When being blocked in tracing process, then into step d；

D determines tracing object using the matching process based on SURF features, at the end of characteristic matching tends towards stability and judges to block, and reinitializes and enters step c after Kalman filter；

E is exported and is preserved destination object characteristic information.

In above-mentioned steps a, two reference frame I are set up_bg(x,y)、I_up(x, y), I_bg(x, y) is the background frames of current scene, I_up(x, y) is a reference frame being continuously updated over time；By present frame I (x, y) respectively with I_bg(x,y)、I_up(x, y) carries out difference binaryzation, and obtained result is designated as：F_bg(x,y)、F_up(x, y), legacy and the moving object in scene are told according to the value of the two.

In above-mentioned steps c, Kalman filter is initialized first, tracking is then predicted according to the target object state observed；During tracking, template image is adaptively updated according to the profile variations situation of destination object, and representational characteristic information is preserved；In tracing process, using the determination methods intersected based on profile to whether blocking and being modeled, analyze and judge.

In above-mentioned steps d, automatic search video content simultaneously finds the prospect agglomerates most with tracked object features Point matching, for erroneous matching caused by measurement error and noise, obtain SURF characteristic matchings point rear use RANSAC algorithms are accurately matched and conversion obtaining image homography matrix, spotting object in video；Judgement block whether terminate with it is above-mentioned judge whether to block use same model；The method for reinitializing Kalman filter use is identical with the initial method of above-mentioned Kalman filter.

The present invention can have following advantageous effects：

The method that the present invention is combined using Kalman filter and SURF characteristic matchings, on the one hand in unobstructed and scene change, Kalman filter can quickly finish prediction and follow the trail of, on the other hand when blocking with scene change, the Target Tracking Problem under Kalman filter failure conditions can be effectively solved using characteristics such as the Scale invariants of SURF features；Therefore with fast and accurately advantage, and due to that can adaptively update To Template according to the change of objective contour, robustness is also fine.The present invention is integrated use legacy detection algorithm, Kalman filter, SURF features, shadowing etc., and the Moving Objects in Video Sequences method for tracing suitable for fixed background monocular-camera of the complete set put forward can be made into software, is easy to application.

Brief description of the drawings

The present invention is further described with embodiment below in conjunction with the accompanying drawings：

Fig. 1 is the foreground moving detection process schematic diagram in the present invention.

Fig. 2 is the basic Kalman filter workflow schematic diagram used in the present invention.

The metric space contrast schematic diagram that Fig. 3 builds for the SURF used in the present invention with SIFT algorithms.

Square frame filtering schematic diagrames of the Fig. 4 for the SURF algorithm used in the present invention in three directions.

Fig. 5 is FB(flow block) of the invention.

Embodiment

In order to more fully understand and realize the present invention, the technical background that the present invention is used is described below first：

First, Moving Objects detection algorithm.

1. time-domain calculus of finite differences

Time-domain difference exactly makes the difference adjacent two two field picture in video sequence, and Moving Objects are extracted by obtained pixel value difference.This method is simple and convenient, is adapted to the extraction under dynamic background.But the objective contour obtained by this method is possible and imperfect.For example when moving object move it is very slow, and itself have large area smooth region when, overlapping part cannot be obtained by so making the difference adjacent two two field picture, and obtained profile occurs in " cavity ".

A kind of current improved method is to replace two frame differences using three-frame difference, thus can preferably detect the profile of intermediate frame motion target.Three adjacent two field pictures are in setting video sequence：I_t-1(x,y)、I_t(x,y)、I_t+1(x, y), calculates the pixel value difference of adjacent two frame respectively：

Then binary conversion treatment is carried out to obtained error image, obtains binary image：

To b_{(t, t1)}(x, y) and b_(t+1,t)(x, y) carries out logic "and" operation, obtains bianry image B_t(x,y):

$B_t(x,y)=\left\{\begin{array}{cc}1& b_{(t,t-1)}(x,y)\∩b_{(t+1,t)}(x,y)=1\\ 0& b_{(t,t-1)}(x,y)\∩b_{(t+1,t)}(x,y)=0\end{array}\right.---\left(3\right)$

The processing such as burn into expansion finally is carried out to obtained bianry image, to eliminate noise and " cavity ".

2. optical flow method

Optical flow method detects that the general principle of moving object is：To one velocity of each pixel definition in image, if not having moving object in image, light stream vector is consecutive variations in whole image region；If there is moving object in image, there is relative motion between moving object and background, the velocity of the two is different, so as to detect moving object.Do not used because the computational methods of light stream are sufficiently complex and amount of calculation is very big, therefore typically by real-time system.

3. background modeling

Background modeling and time-domain difference method have similarity, are all the differences for doing two field pictures, the difference is that background modeling method is by present frame and reference frame（Background frames）Carry out difference operation.Background modeling method is widely used in the motion detection of static camera, the selection of background frames is the key of whole algorithm, modeling of the background modeling namely to background frames, background frames ideally should be free of moving object, and can be updated according to certain strategy with adapt to the leaf swing in the dynamic change of scene, such as illumination variation, background, ripple, sleet descends slowly and lightly situations such as.Existing background modeling method is broadly divided into six classes：Increment type Gauss is average, sequential medium filtering, mixed Gauss model, Density Estimator, order cuclear density are approximate and feature background model.

The method for testing motion that the present invention is used is exactly a kind of simple and quick method based on background modeling thought：Legacy detection method, with reference to Fig. 1.Set up two reference frame I_bg(x,y)、I_up(x,y)。I_bg(x, y) is the background frames of current scene, I_up(x, y) is a reference frame being continuously updated over time.By present frame I (x, y) respectively with I_bg(x,y)、I_up(x, y) carries out difference binaryzation, and obtained result is designated as：F_bg(x,y)、F_up(x, y) and legacy and moving object in scene are differentiated according to the value of the two.

2nd, the tracing algorithm based on Kalman filter, with reference to Fig. 2.

1. discrete Kalman filter

This be the present invention use it is unobstructed, without scene change when Moving Objects method for tracing.Basic Kalman filtering is the method for solving linear filtering and forecasting problem, with the minimum criterion of mean square error, with it is simple, quick the characteristics of.

Kalman filter predicts process：

${\hat{X}}_k^{-}=F_k{\hat{X}}_{k-1}+B_{k-1}{u}_{k-1}---\left(1\right)$

$P_k^{-}=F_{k-1}{P}_{k-1}{F_{k-1}}^T+Q_{k-1}---\left(2\right)$

Kalman filter renewal process：

$K_k=P_k^{-}{H_k}^T{(H_k{P}_k^{-}{H_k}^T+R_k)}^{-1}---\left(3\right)$

${\hat{X}}_k={\hat{X}}_k^{-}+K_k(Z_k-H_k{\hat{X}}_k^{-})---\left(4\right)$

$P_k=(I-K_k{H}_k)P_k^{-}---\left(5\right)$

Wherein F is state-transition matrix, and H is calculation matrix, and Z is observation, and B is Input transformation matrix, and u is input value（New input value is not needed in some systems, therefore B and u can be omitted）.Q and R represent the variance of noise vector in state migration procedure and observation process respectively.

Represent the k-1 moment to k moment states X_kBest predictor,

Represent the observation Z with the k moment_kWith predicted value of the previous moment to this moment

To X_kThe state made updates.P is covariance, descends target implication identical with X thereon, and K represents Kalman gains.

2. the Kalman filter of extension

For the second best measure of Nonlinear state space model, the class method being most widely used is the Kalman filter of extension（Extended Kalman filtering, EKF）.EKF basic ideas are first to linearize nonlinear system, and the processing similar with linear Kalman filter device is then carried out again.Its specific method is that the Taylor expansion of nonlinear function is blocked, so that nonlinear function be linearized.What is carried out according to Taylor expansion is single order or second order interception, and EKF can be divided mainly into single order EKF（first order EKF）With second order EKF（second order EKF）.

Although the Kalman filter of extension has outstanding performance in terms of nonlinear model is solved, there is also clearly disadvantageous in actual applications for it：One is that unstable filtering may be produced during nonlinear model is linearized；Two be, when calculating the derivative of Jacobian matrix, to realize complex；Three be the situation that pattern function in reality there may be non-differentiability, so may result in EKF failures.Therefore when the noise in the non-linear relatively strong and system of model is non-gaussian distribution, EKF estimated accuracy will be substantially reduced, and ultimately result in failure.

3rd, SIFT, SURF characteristic matching method, with reference to Fig. 3, the left figure in figure is the image pyramid built in conventional method, and last layer image is the down-sampling to previous tomographic image；Right figure is the method for structure metric space in SURF algorithm, and image is constant, the size of the simply Filtering Template of change.

SURF algorithm is SIFT innovatory algorithm.SURF algorithm will be far superior to SIFT algorithms in characteristic matching speed, therefore SURF algorithm can be further applicable in real-time images match scene.The present invention carries out Feature Points Matching using SURF algorithm.

SURF algorithm is divided into metric space structure, feature point detection, feature descriptor generation and four parts of Feature Points Matching.

1. metric space is built

The metric space of SURF algorithm is constituted

Traditional metric space is described as a pyramid, and Gaussian convolution core is the unique linear core for realizing change of scale, gives piece image I (x, y), then its metric space is defined as：

L(x,y,δ)=G(x,y,δ)*I(x,y)    （1）

Wherein G (x, y) is changeable scale Gaussian function,

$G(x,y,\mathrm{\δ})=\frac{1}{{2\mathrm{\π\δ}}^2}{e}^{-(x^2+y^2)/{2\mathrm{\δ}}^2---\left(2\right)}$

(x, y) is space coordinate, and δ is yardstick coordinate, and δ size determines the smoothness of image.Using above formula, the number of plies for including image in the exponent number in pyramid scale space, and every rank pyramid is finally determined according to the size of image.The pyramidal first layer of first rank is original image, and each layer up is to carry out Laplace conversion to preceding layer afterwards（Gaussian convolution, δ values gradually increase）.From the point of view of intuitively, image more up is fuzzyyer.

In order to effectively detect stable key point in metric space, Lowe et al. proposes Gaussian difference scale space（DOG scale-space）.

D(x,y,δ)=(G(x,y,kδ)-G(x,y,δ))*I(x,y)=L(x,y,kδ)-L(x,y,δ)    (3)

Each layer of DoG pyramid metric spaces is subtracted each other by gaussian pyramid adjacent two layers to be obtained, therefore DoG pyramids, compared with Gauss pyramid, the exponent number of tower is identical, but subtracts one per the number of plies in rank tower.

The metric space of SIFT algorithms is constituted

The method that SIFT algorithms build metric space, its shortcoming is that the foundation of every tomographic image will depend on previous tomographic image, and the size of image needs to reset, therefore the operand of this method is larger.

What SURF algorithm changed when building image pyramid is not picture size, but the size of Filtering Template.SURF algorithm can be with parallel processing when building metric space, and need not carry out double sampling to image, so as to improve arithmetic speed.The difference of SURF algorithm and the metric space of SIFT algorithm constructions is as shown in Figure 3.

2. feature point detection

Piece image I (x, y) is given, its integral image is

$I_{\mathrm{\Σ}}(x,y)=\underset{i=0}{\overset{i\≤x}{\mathrm{\Σ}}}\underset{j=0}{\overset{j\≤y}{\mathrm{\Σ}}}I(i,j)---\left(4\right)$

SURF feature point detections judge whether certain point is extreme point on image using Hessian determinants of a matrix.If f (x, y) is the function that a second order can be micro-, then its Hessian matrix is

$H\left(f(x,y)\right)=\left[\begin{array}{cc}\frac{{\∂}^{2}f}{{\∂x}^2}& \frac{{\∂}^{2}f}{\∂x\∂y}\\ \frac{{\∂}^{2}f}{\∂x\∂y}& \frac{{\∂}^{2}f}{{\∂y}^2}\end{array}\right]---\left(5\right)$

The then determinant of matrix H

$\det H=\frac{{\∂}^{2}f}{{\∂x}^2}\frac{{\∂}^{2}f}{{\∂x}^2}-{\left(\frac{{\∂}^{2}f}{\∂x\∂y}\right)}^2---\left(6\right)$

It is the product of H characteristic value, if det H<0, then point (x, y) is not Local Extremum；If det H>0, then point (x, y) is Local Extremum.Then Hessian matrixes of the image I (x, y) under yardstick δ is

$H(x,y,\mathrm{\&delta;})=\left[\begin{array}{cc}{L}_{\mathrm{xx}}(x,y,\mathrm{\&delta;})& L_{\mathrm{xy}}(x,y,\mathrm{\&delta;})\\ L_{\mathrm{xy}}(x,y,\&delta;)& L_{\mathrm{yy}}(x,y,\mathrm{\&delta;})\end{array}\right]---\left(7\right)$

Wherein L_xx(x, y, δ) is image at point (x, y) place and Gaussian function second order local derviation

Convolution, similarly definable L_xy(x, y, δ) and L_yy(x,y,δ)。

Bay et al. carries out rational discretization to convolution kernel and replaced after cutting with square frame Filtering Template, and accelerates convolution speed to reduce amount of calculation using integral image.Three convolution kernels are respectively D after cutting_xx、D_yyAnd D_xy, they are L_xx、L_yyAnd L_xySimplify expression.9 × 9 square frame Filtering Template is as shown in figure 4, corresponding second order Gauss filter scale factor delta=1.2.

Because square frame filtering is the approximate evaluation of second order Gauss filtering, therefore the error for replacing second order Gauss filtering calculating Hessian matrix determinants value and causing is filtered with square frame in order to make up, then had

detH=D_xxD_yy-(0.9D_xy)²    (8)

The length of side for 9 convolution kernel be smallest dimension convolution kernel, with being continuously increased for yardstick, the size of convolution kernel (Filtering Template) also proportional increase.If Filtering Template size is N*N, yardstick δ=N*1.2/9 is corresponded to.

After the extreme point that each yardstick is obtained by Hessian matrix determinants, with 8 consecutive points of its same yardstick and above and below it, each 9 points of two yardsticks are compared each extreme point, when maximum or minimum during the value of the extreme point is worth at 26, just the extreme point is regard as candidate feature point.

The method being finally previously mentioned with M.Brown carries out interpolation arithmetic, obtains the characteristic point position and place scale-value of sub-pixel precision.Remove the low characteristic point of contrast and unstable skirt response point simultaneously（Because DoG methods can produce stronger skirt response）, to improve the stability of noise resisting ability and enhancing matching.

Because characteristic point is the local stability point spatially chosen in graphical rule, so meeting the requirement of characteristic matching in the case of dimensional variation.

Why higher than the operation efficiency of SIFT algorithm SURF algorithm is, is because SURF algorithm accelerates the detection of characteristic point using integral image and Hessian matrixes.

3.SURF feature descriptors are generated

The generation of SURF Feature Descriptors is divided into two steps：Principal direction determines and built descriptor.

Principal direction is determined.To ensure the rotational invariance of characteristic point, 6 δ around characteristic point（δ is characterized a place yardstick）Neighborhood in pixel point sampling, and in sample point ask for the Harr wavelet convolutions core response in x and y direction that the length of side is 4 δ by step-length δ.To make the response contribution close to characteristic point big, the response contribution away from characteristic point is small, carries out Gauss weighting by the δ of σ=2 to small echo response, is then expressed as the point on a two-dimensional coordinate, can finally obtain all sampled points and respond the distribution map on two dimensional surface.Then slided with a subtended angle for 60 ° of sliding window by fixed step size, be added to form new vector by the response in the range of 60 ° every time, the direction of selection most long vector is the principal direction of this characteristic point

Build descriptor.Then centered on characteristic point, reference axis is rotated into principal direction.The square area that the length of side is 20 δ is chosen, the window is divided into 4 × 4 subregion, the Harr responses of 25 sampled points are calculated in each sub-regions, d is designated as respectively_xAnd d_y.Then d is obtained with the δ of σ=3.3 Gaussian function weighting on each subwindow_xAnd d_yAccumulated value, i.e. ∑ d_x, ∑ d_y, ∑ │ d_x│, ∑ │ d_y│.One four-dimensional new vector of formation per sub-regions, because square area includes 16 subwindows, then the descriptor that each characteristic point formation 16 × 4=64 is tieed up.

The method alignd by principal direction, SURF features have rotational invariance, available for the characteristic matching under rotational case.

4.SURF Feature Points Matchings

Similarity determination measurement of the matching of characteristic point by the use of the Euclidean distance of characteristic vector as key point in two images.The formula of Euclidean distance is：

d=sqrt(∑(x_i1-x_i2)²)    (9)

x_i1Represent the i-th dimension coordinate of certain point on piece image, x_i2Represent the i-th dimension coordinate of certain point on the second width image.

Specific practice is：Some key point in piece image is taken, the first two key point nearest with its Euclidean distance in the second width image is found out.If nearest distance divided by secondary near distance are less than some threshold value in the two key points, then it is assumed that find a pair of match points.Increase this threshold value, the number of match point will increase, but accuracy rate declines；This threshold value is conversely reduced, the number of match point will be reduced, but accuracy rate can be improved.

Deficiency of the 5.SURF algorithms in Moving Objects tracking

Although SIFT/SURF features have relatively stable consistency to the scaling of image, rotation, translation, brightness change etc., it is applied and still there is deficiency in Moving Objects tracking, mainly there is the following aspects：

1）Due to using the method based on image pyramid when building metric space, it is thus possible to occur that layer obtains defective tightness to cause yardstick matching to have the situation of error.When original image is smaller in itself, building extraction of the metric space on characteristic point influences little.

2）The low point of contrast and unstable skirt response point can be removed in SURF characteristic point filter processes.If there is large stretch of smooth region in picture material, so characteristic information of this partial content will be filtered.It is able to may also be equally omitted as the marginal information of key character.

3）SURF features are the local features of picture material, have ignored the global information of image in itself.

4）The search strategy that SURF algorithm is used in Feature Points Matching is inefficient, and can not make full use of the position relationship between adjacent features point, consequently, it is possible to causing erroneous matching.

5）SIFT/SURF algorithms only make use of the gamma characteristic of image in itself, not account for image intrinsic colour information.

6）Compared with kalman filter-tracking technologies, the operand of SURF algorithm wants greatly many；The simple tracking for being used for Moving Objects in Video Sequences using SURF algorithm is difficult the requirement for meeting real-time.

4th, homography matrix.

When carrying out destination object positioning, the concept of homography matrix has been used.Two images AB in the same space, if from there are one-to-one mapping relations image A to image B, the mapping relations represent to be exactly a homography matrix with matrix.If the point coordinates of certain in two images is respectively I (x, y, 1), I'(x', y', 1), homography matrix H, then corresponding projection relation be：

$k\left[\begin{array}{c}{x}^{\&prime;}\\ y^{\&prime;}\\ 1\end{array}\right]=H\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]=\left[\begin{array}{ccc}{h}_1& h_2& h_3\\ h_4& h_5& h_6\\ h_7& h_8& h_9\end{array}\right]\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]---\left(1\right)$

Wherein k is proportionality coefficient, and usual H is the transformation matrix that the free degree is 8, works as h₉When=1, it can be obtained by (1)：

h₁x+h₂y+h₃-h₇xx′-h₈yx′-x′=0    (2)

h₄x+h₅y+h₆-h₇xy-h₈yy-y=0    (3)

As long as the coordinate for so having 4 pairs of match points can just calculate homography matrix H.

With reference to the above and Fig. 1 and Fig. 5, technical scheme is illustrated in detail further below：

A carries out foreground moving detection to the video sequence of input, extracts Moving Objects.

In application scenarios to be processed needed for the present invention, the object that Moving Objects are trapped in same position or background for a long time after entering movement suddenly is all situation about being likely to occur.The present invention extracts Moving Objects using remnant object detection method.

Leave analyte detection different from motion detection, it not only needs to detect non-existent object in original scene, also to judge whether the object stops in this scenario.

Specific method is：Set up two reference frame I_bg(x,y)、I_up(x,y)。I_bg(x, y) is the background frames of current scene（Do not include moving object, it is consistent with the background frames in general background modeling）, I_up(x, y) is a reference frame being continuously updated over time, if current image frame is I (x, y), then update method is：

I_up(x,y)=(1-α)I_up(x,y)+αI(x,y)    (1)

α is renewal speed weight, and α is bigger, and renewal rate is faster, and the smaller renewal rates of α are slower.If so external object is stopped in the scene, I will be incorporated through the object after a period of time_upIn (x, y), the part as " background ".By I (x, y) respectively with I_bg(x,y)、I_up(x, y) carries out difference binaryzation, and obtained result is designated as：F_bg(x,y)、Fup(x,y).Such as the pixel value F at fruit dot (x, y) place_bg(x, y)=1, F_up(x, y)=0 item may determine that the point belongs to legacy, and legacy and the moving object in scene can be specifically told according to the method defined in table 1.

The remnant object detection method decisional table of table 1

	Fbg(x,y)	Fup(x,y)	Judge type
				Ⅰ	1	1	Moving Objects
Ⅱ	1	0	Temporary transient stationary objects（Legacy）
				Ⅲ	0	1	Random noise
Ⅳ	0	0	Stationary body in scene

If b has preserved the feature of tracing object, step d is directly entered；If it is not, the region selected according to user completes the template initialization and SURF feature extractions to target object, and Kalman filter initialization.

If having preserved the feature of tracing object under the assigned catalogue of hard disk, then mean to there occurs scene change, now it is first directed to the Template Information and characteristic information of the destination object preserved under assigned catalogue, then global search matching is carried out in frame of video, if the moving object of a certain frame in exceedes some defined threshold value with the points that match of template, then think that target object occurs in this scenario, now needing initialization Kalman filter to be tracked processing.

If there is no the correlated characteristic for preserving tracing object under the assigned catalogue of hard disk, need user to be drawn by mouse on monitored picture and select target object.Algorithm carries out template initialization and SURF feature extractions according to the target object for drawing choosing.

The initialization of above-mentioned Kalman filter such as the following steps：

1）Due to the limitation of practical application scene of the present invention, big change will not typically occur for the movement velocity of target object in the scene, therefore the present invention is analyzed and handled to object using the equation of motion of uniform motion：

x_t=x_t-1+v_x    (2)

y_t=y_t-1+v_y    (3)

Formula (2) and formula (3) represent the equation of motion of the object in x-axis and y-axis direction, v respectively_x、v_yThe speed of object in the two directions is represented respectively.In t, the state representation of a moving object is X_t=(x_t,y_t,v_x,v_y)^T, observation is Z_t=(x_t,y_t)^T。

2）Control Kalman filter model and the equation of motion understand that the state-transition matrix F and calculation matrix H of system are respectively：

$F=\left[\begin{array}{cccc}1& 0& 1& 0\\ 0& 1& 0& 1\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right],$ $H=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\end{array}\right]---\left(4\right)$

3）According to the related experiment situation and reference of practical application scene of the present invention, the variance matrix Q and R of noise vector value distinguish as follows in the state migration procedure and observation process of the Kalman filter employed in the present invention：

$Q=\left[\begin{array}{cccc}0.01& 0& 0& 0\\ 0& 0.01& 0& 0\\ 0& 0& 0.01& 0\\ 0& 0& 0& 0.01\end{array}\right],$ $R=\left[\begin{array}{cc}0.2845& 0.0045\\ 0.0045& 0.0455\end{array}\right]---\left(5\right)$

4）System initial state vector covariance matrix P is defined as follows：

$P_0=\left[\begin{array}{cccc}100& 0& 0& 0\\ 0& 100& 0& 0\\ 0& 0& 100& 0\\ 0& 0& 0& 100\end{array}\right]---\left(6\right)$

5）When the position adjustment of camera, the above matrix Q, R, P₀Definition can be adjusted accordingly according to actual conditions.For original state

Definition, this paper algorithms allow user-defined selection, i.e. user to draw the target object that select that a certain moment thinks tracking with mouse, and then system is according to the now observation position of the object top left corner apex coordinate and speed initialization

C is predicted tracking to moving target using the method based on Kalman filter, until video content terminates, into step e；When being blocked in tracing process, then into step d；

After being initialized to Kalman filter, system just can be predicted tracking to the position of target object.During tracking, template image is adaptively updated according to the profile variations situation of destination object, and representational characteristic information is preserved.

The present invention determines whether to update To Template and SURF Feature Descriptors in real time according to the profile variation of tracked object.Profile variation herein is primarily referred to as in the case where camera focal length is constant target because of the change of displacement position and direction, the change that the total pixel area in caused image shared by target occurs.I.e.

A_m-A_n|>H    (7)

When, judge that profile variation occurs for target, Am and An represent the elemental area shared by m and n moment targets respectively, and H is a threshold value, and the threshold value can be adjusted according to concrete scene.

After the elemental area shared by judgment object changes, system continues to judge whether the length-width ratio of the target object changes, if the now length-width ratio R of target object_tWith the length-width ratio R of the To Template preserved before_mCompared to generation significant change, then it is assumed that angle change occurs for object, it is necessary to update the Template Information of destination object.

R_t-R_m|>H_R, R_t=W_tH_t    (8)

H_RThreshold value during to judge that destination object length-width ratio is varied widely, W_tAnd H_tThe width and height of t destination object are represented respectively

Judge the generation blocked.The present invention is using the determination methods intersected based on profile, and this method is quickly effective, and it needs blocking for two kinds of situations of solution：One is mutually blocking between object, and two be that destination object is blocked by background.

We understand that when destination object is blocked the elemental area shared by its profile is increased sharply or reduced.Elemental area increase represents to there occurs blocking between object, and elemental area reduces the situation for representing to there occurs background occlusion objects.Therefore after the binary image for obtaining moving object contours by motion detection, following formula is recycled to be judged it is known that at the time of blocking generation.

|S_t-S_t-1|>T    (9)

S_tAnd S_t-1Current time and the elemental area shared by previous moment destination object are represented respectively, and T represents the threshold value set.The threshold value needs to be changed according to the focal length of camera in concrete scene.

D determines tracing object using the matching process based on SURF features, at the end of characteristic matching tends towards stability and judges to block, reinitializes after Kalman filter, into step c；

The characteristic point that some are matched between the two images obtained by SURF algorithm has the erroneous matching caused by measurement error and noise to being not right-on, in them.Therefore, the present invention is obtaining SURF characteristic matchings point to rear, is accurately matched using RANSAC algorithms and obtains the homography matrix of the conversion between image, so that spotting object in video.

RANSAC（random sample consensus）Algorithm is a kind of method for estimating mathematical model parameter iteration, and its basic ideas is to try to achieve most of sample by the method for stochastical sampling and checking（Refer to the characteristic point pair being mutually matched）The parameter for the mathematical modeling that can be met.

The present invention is comprised the following steps that with RANSAC algorithms：

1）Four pairs of SURF match points are randomly selected first as Initial Internal Points set, and transformation matrix H is calculated by four pairs of match points in initial sets.

2）Whether the point in judging outside point set can add the set.The distance between I ' and HI is calculated, if the value is less than the threshold value set, the point is added into interior point set.

3）Repeat 1）With 2）Step n times, choose that group set for counting most in interior point set as the matching point set finally needed.Finally according to the set, transformation matrix H is updated with least square method.

In above process, it is assumed that the ratio that the match point finally given accounts for initial SURF match points sum is p, then it is 1-p to randomly select four pairs of match points to be not all the final probability correctly matched⁴, the probability that four pairs of initial points of iteration n times are all not all correct match point is (1-p⁴)^N, then the probability P for obtaining correct transformation matrix H is：

P=1-(1-p⁴)^N    (10)

In actual applications, both make the iterations N of algorithm less again to ensure that greater probability obtains transformation matrix H, typically take N between 10 ~ 20.

After transformation matrix H is obtained, four summits mapping of destination object in template is just obtained to the general location of objects in images in the past, computational methods are as follows：

x_i′=h₁*x_i+h₂*y_i+h₃    (11)

y_i′=h₄*x_i+h₅*y_i+h₆    (12)

(x_i′,y_i') represent the coordinate on target in video image object i-th of summit, (x_i,y_i) be template in destination object i-th of apex coordinate.

Judge the end blocked.Whether the method that the part is used is blocked with judgement uses that method is identical, i.e., whether judgment object profile intersects.At the end of the process of blocking, significant change can occur for the elemental area that occupies of agglomerate where the destination object observed in bianry image, by formula (9) it may determine that whether certain moment blocks terminate.

At the end of blocking, the result of the search matching before is reinitialized Kalman filter by system, then constantly predicts and more newly arrive continuing to complete tracking processing by Kalman filter.

E is exported and is preserved destination object characteristic information.

The need for user, destination object characteristic information is preserved to assigned catalogue.

Meaning of the present invention is：

The tracking process of object video is the key technology of intelligent video processing, is the premise of the senior semantic operation such as Activity recognition, event recognition, identification and processing.Blocking with the complex scene such as scene change, it is the focus and difficult point studied both at home and abroad fast and accurately to realize the tracking to object video.In recent years, people have carried out substantial amounts of research to the tracking problem under circumstance of occlusion, although existing method can solve subproblem at present, all problems can be solved well but without a kind of method.For example, although image layered method can solve to block tracking problem, complexity is high, it is difficult to meet the requirement of real-time；And the tracking initialization difficulty, object module based on color or profile update difficult, it is difficult to in real system；Using multiple-camera joint-monitoring it is the current popular method for solving occlusion issue in Same Scene, but this method is also very immature compared with the tracking under single camera at present, and cost of implementation and complexity are all very high.Compared to blocking, the tracking in the case of scene change is increasingly complex, and the documents and materials referred to both at home and abroad in the field at present are also less, and main solution is still expands to many scene tracking by the track algorithm under single game scape.And it is of the invention, it is not necessary to multiple cameras, it is only necessary to which the Moving Objects that a video camera can be just directed under fixed background environment carry out fast track, and when blocking with scene change still with higher accuracy rate and robustness.

Special instruction, present invention work is supported by project of national nature science fund project (61170253), the plan of Information Institute innovative research team of University Of Science and Technology Of Shandong.

The relevant technology contents do not addressed in aforesaid way are taken or used for reference prior art and can be achieved.

It should be noted that under the teaching of this specification those skilled in the art can also make it is such or it is such be easily varied mode, such as equivalent way, or substantially mode of texturing.Above-mentioned variation pattern all should be within protection scope of the present invention.

Claims (4)

1. it is a kind of towards the Moving Objects method for tracing blocked with scene change, it is characterised in that to comprise the following steps：

A carries out foreground moving detection to the video sequence of input, extracts Moving Objects；Subsequently into step b,

If b has preserved the feature of tracing object, into step d；If not preserving the feature of tracing object, the region selected according to user completes the template initialization and SURF feature extractions to target object, and Kalman filter initialization；Subsequently into step c,

C is predicted tracking to moving target using the method based on Kalman filter, until video content terminates, into step e；When being blocked in tracing process, then into step d；

D determines tracing object using the matching process based on SURF features, at the end of characteristic matching tends towards stability and judges to block, and reinitializes and enters step c after Kalman filter；

E is exported and is preserved destination object characteristic information.

2. it is according to claim 1 a kind of towards the Moving Objects method for tracing blocked with scene change, it is characterised in that：

In above-mentioned steps a, two reference frame I are set up_bg(x,y)、I_up(x, y), I_bg(x, y) is the background frames of current scene, I_up(x, y) is a reference frame being continuously updated over time；By present frame I (x, y) respectively with I_bg(x,y)、I_up(x, y) carries out difference binaryzation, and obtained result is designated as：F_bg(x,y)、F_up(x, y), legacy and the moving object in scene are told according to the value of the two.

3. it is according to claim 1 or 2 a kind of towards the Moving Objects method for tracing blocked with scene change, it is characterised in that：

In above-mentioned steps c, Kalman filter is initialized first, tracking is then predicted according to the target object state observed；During tracking, template image is adaptively updated according to the profile variations situation of destination object, and representational characteristic information is preserved；In tracing process, using the determination methods intersected based on profile to whether blocking and being modeled, analyze and judge.

4. it is according to claim 3 a kind of towards the Moving Objects method for tracing blocked with scene change, it is characterised in that：

In above-mentioned steps d, automatic search video content simultaneously finds the prospect agglomerates most with tracked object features Point matching, for erroneous matching caused by measurement error and noise, obtain SURF characteristic matchings point rear use RANSAC algorithms are accurately matched and conversion obtaining image homography matrix, spotting object in video；Judgement block whether terminate with it is above-mentioned judge whether to block use same model；The method for reinitializing Kalman filter use is identical with the initial method of above-mentioned Kalman filter.

Images