CN102592135A - Visual tracking method of subspace fusing target space distribution and time sequence distribution characteristics - Google Patents

Abstract

The invention discloses a visual tracking algorithm of subspace fusing target space distribution and time sequence distribution characteristics, which mainly comprises steps of firstly, developing target regional characteristics from three angles, and conducting subspace learning respectively; secondly, applying an updating strategy to update the mean value and subspace under three modes on line; finally, integrating the three subspace modes into a likelihood function through tensor reconstruction to be used for evaluating candidate detecting images. The visual tracking method achieves an effective target tracing method, and is a universal method. Experimental results shows that compared with other classical subspace tracking algorithm, the visual tracking method is more effective and robust, and has good application prospect.

Description

Merge the visual tracking method of object space distribution and timing distribution proper subspace

Technical field

The present invention relates to computer vision field, particularly relate to a kind of visual tracking method that merges object space distribution and timing distribution proper subspace.

Background technology

Target following is very important studying a question in the computer vision field, because it is the basis of researchs such as high-rise visual problem such as motion analysis and behavior identification.Generally speaking, present main two critical problems of target tracking algorism: (1) apparent model; (2) follow the tracks of framework.

In general, apparent model is exactly how target object to be carried out effectively expressing, and carries out real-time renewal.To Template be a kind of the most directly Target Modeling method, but this model lacks identification and robustness.Although for target scale, rotation and non-rigid shape deformations robust comparatively, because it has ignored the apparent color space distributed intelligence of target, there is certain defective in the color histogram of target area.Though the apparent model based on Density Estimator has well solved this defective, but the cost that brings is the increase of calculating and storage complexity.In addition, come the internal relation between the modeling neighborhood pixels through Markov random field, but its training cost is very huge based on the apparent model of condition random field.

In recent years, more reasonable based on the apparent model of subspace study owing to the constant hypothesis in its subspace, thereby be widely used in the vision track field.But this model needs abundant sample when training, in practice, be difficult to reach the requirement of real-time.Based on this, Levy and Lindenbaum have proposed sequence KL (Sequential Karhunen-Loeve) mapping algorithm and have been used for the characteristic base that image is learnt on increment ground.Lim etc. have expanded sequence KL mapping algorithm, and average and the characteristic base to target image carries out incremental update simultaneously, and with this algorithm first Application in the vision track of target.Then; The Robust Estimation strategy; The reinforcement adaptive approach of Yang based on data-driven; Liao based on the tracking of robust Kalman filtering and Gai and Stevenson based on the method for dynamic model, though in some specific scene, obtained tracking performance preferably, certain weak point is arranged: above-mentioned all at first will be based on the track algorithm of subspace with image generate one-dimensional vector; The apparent space distribution information of target is almost completely lost, thereby makes model very responsive to target apparent variation of overall importance and noise.To this shortcoming, Hu etc. introduce tensor thought, have played effectiveness to a certain extent.When carrying out but because it has brought error using the R-SVD renewal process only to keep preceding R the big pairing proper vector of eigenwert, and along with tracking, error can progressively add up, and causes the model drift.Though a kind of model based on dynamic tensor analysis has been avoided above-mentioned error, obtained accurate more result, because small sample problem makes the covariance matrix that calculates can't describe the distribution situation of sample, thereby cause the calculating of subspace to be degenerated.

Summary of the invention

Only pay close attention to property time correlation between the characteristic of target area in order to overcome traditional apparent model, and ignored the space distribution information of target area based on subspace study.The present invention provides a kind of visual tracking method that merges object space distribution and timing distribution proper subspace, with respect to other classons space track algorithm, and the effective more and robust of this tracking.

To achieve these goals, the technical scheme below the present invention has adopted:

A kind of visual tracking method that merges object space distribution and timing distribution proper subspace may further comprise the steps:

(1) the utilization tensor theories is expressed as one three rank tensor with the training sample of target image observation, and this tensor is launched into three split-matrixes from the width of image, these three different directions of height and image time shaft of image;

Wherein, the row sample subspace of preceding two split-matrixes is for the space distribution information of target; The capable sample subspace of the 3rd split-matrix is corresponding to the time distributed intelligence of target;

(2), calculate the sample covariance matrix of each split-matrix, and obtain the pattern subspace of target level and vertical direction through covariance matrix being done feature decomposition for preceding two split-matrixes; For the 3rd split-matrix, adopt singular value decomposition algorithm to obtain the pattern subspace of target on time shaft;

(3) for given object candidate area, calculate its reconstructed error on the apparent space of target, and the observation likelihood function of objective definition candidate region thus;

(4) according to the observation likelihood function of definition, the utilization particle filter calculates the posterior probability of dbjective state, adopts maximum a posteriori estimation strategy from a series of candidate's observed readings, to select the maximum dbjective state as current time of posterior probability;

(5) result who utilize to follow the tracks of to the target pattern subspace that obtains in the step (2) is upgraded, and adds forgetting factor in renewal process;

Wherein, forgetting factor is a kind of weight between 0 and 1, and big more forgetting factor means that model pays close attention to recent data more in the model of subspace.

Furtherly, described step (1) specifically comprises following substep:

At first, with the training sample of target image observation with one three rank tensor representation;

Secondly, with the tensor that obtains from width, highly, these three different directions of time are launched into three split-matrixes;

Furtherly, described step (2) specifically comprises following substep:

At first, for preceding two split-matrixes, calculate the sample covariance matrix of each split-matrix, through covariance matrix being done the subspace that feature decomposition obtains each pattern;

Secondly, for the 3rd split-matrix, adopt singular value decomposition algorithm to obtain the pattern subspace of target on time shaft.

Furtherly, described step (3) specifically comprises following substep:

At first, obtain reconstructed error;

Secondly, by reconstructed error definition likelihood function.

Furtherly, described step (4) specifically comprises following substep:

At first, the motion of hypothetical target between the two continuous frames image is affine motion, and the state of target is by the target affine motion parameter characterization between two frames, and hypothetical target state dynamic transfer probability model is a Gauss model;

Secondly, utilize likelihood function to calculate the posterior probability of dbjective state;

At last, adopt maximum a posteriori to estimate that strategy is from selecting the maximum dbjective state as current time of posterior probability from a series of candidate's observed readings.

Furtherly, described step (5) specifically comprises following substep:

For preceding two split-matrixes,

At first, the covariance matrix to the sample matrix of acquisition in the step (2) upgrades;

Secondly, forgetting factor is added in the renewal equation covariance matrix that is more tallied with the actual situation.

For the 3rd split-matrix,

At first, the data matrix that newly obtains being carried out QR decomposes;

Secondly, the data matrix that merges new legacy data is carried out svd.

The invention has the beneficial effects as follows:

1, track algorithm proposed by the invention is a kind of method in common, and any tracking target type all is suitable for;

2, the fusion object space distribution that the present invention adopted and the vision track algorithm of timing distribution proper subspace; Fully excavated space distribution information and property time correlation between the characteristic of target area on the vertical and horizontal direction of target area, made algorithm reduce the apparent variation of overall importance of target and the sensitivity of noise compared to classic algorithm;

3, the present invention proposes a kind of update strategy, for two spatial models of level and vertical direction, through upgrading the covariance matrix more spatial model of fresh target distribution online, has improved update efficiency effectively;

4, the present invention proposes a kind of update strategy; For the time distribution pattern; Adopt the more time series pattern of the fresh target distribution online of increment singular value decomposition method; Not only solved owing to number of samples much smaller than the subspace degenerate problem that the sample dimension produces, also alleviated simultaneously the model drifting problem of introducing in calculating to a certain extent.

Description of drawings

Below in conjunction with accompanying drawing and embodiment the present invention is further specified.

Fig. 1 is the general frame of tracker of the present invention;

Fig. 2 is an image observation decomposing schematic representation of the present invention;

Fig. 3 is that three rank tensors of the present invention launch synoptic diagram.

Embodiment

Through embodiment the present invention is carried out concrete description below; Only be used for the present invention is further specified; Can not be interpreted as the qualification to protection domain of the present invention, the technician in this field can make some nonessential improvement and adjustment to the present invention according to the content of foregoing invention.

As shown in Figure 1, Fig. 1 is a general frame of the present invention.The present invention is a kind of based on multi-angle subspace fusion goal tracking, and the hardware and the programming language of the concrete operation of method of the present invention do not limit, and can accomplish with any language, and other mode of operation repeats no more for this reason.

Embodiments of the invention adopt one to have the Pentium 4 computing machine of 3.2G hertz central processing unit and 1G byte of memory and worked out the working routine that the sequence particle group optimizing is followed the tracks of framework with the Matlab language; Realized method of the present invention, fusion object space of the present invention distributes and the visual tracking method of timing distribution proper subspace may further comprise the steps:

Calculating is estimated modules such as (MAP) based on the expansion of tensor subspace, the renewal of covariance matrix, singular value decomposition algorithm, particle filter Condensation and the maximum a posteriori that the R-SVD algorithm is increment, and concrete steps are described below:

(1) training sample with target image observation is expressed as one three rank tensor

Represent, wherein l ₁, l ₂And l ₃Be expressed as the width of image observation, highly and the length on time shaft respectively.According to tensor theories, this three rank image observation tensor can be by l ₁, l ₂And l ₃Direction is launched into three split-matrix A respectively ₍₁₎, A ₍₂₎, A ₍₃₎Carry out corresponding with Fig. 2.

For A ₍₁₎And A ₍₂₎The row sample covariance matrix, A ₍₃₎Capable sample covariance matrix Distribution calculation following:

$C_1=\underset{i}{\mathrm{\Σ}}(A_{\left(1\right)}^i-{\mathrm{\μ}}_1){(A_{\left(1\right)}^i-{\mathrm{\μ}}_1)}^T$

$C_2=\underset{i}{\mathrm{\Σ}}(A_{\left(2\right)}^j-{\mathrm{\μ}}_2){(A_{\left(2\right)}^j-{\mathrm{\μ}}_2)}^T$

$C_3=\underset{i}{\mathrm{\Σ}}(A_{\left(3\right)}^{\′k}-{\mathrm{\μ}}_3){(A_{\left(3\right)}^{\′k}-{\mathrm{\μ}}_3)}^T$

(2) wherein

Be respectively matrix A ₍₁₎, A ₍₂₎, A ₍₃₎I, j, k row sample.μ ₁, μ ₂Be respectively matrix A ₍₁₎, A ₍₂₎Column mean, μ ₃Be A ₍₃₎Column mean, i.e. matrix A ₍₃₎Capable average.After obtaining the covariance matrix of sample, the subspace of each pattern can obtain through covariance matrix being done simple feature decomposition:

$C_d=U_d{S}_d{U}_d^T,d=\mathrm{1,2,3}$

(3) For a given target candidate region?

and its expansion in vector form?

The target candidate regions in the target apparent subspace reconstruction error can be calculated as follows:

${\mathrm{RE}}_1={\left|\right|(o_t-u_1)-(o_t-u_1)U_1{U}_1^T\left|\right|}^2$

${\mathrm{RE}}_2={\left|\right|(o_t^{\′}-u_2)-(o_t^{\′}-u_2)U_2{U}_2^T\left|\right|}^2$

${\mathrm{RE}}_3={\left|\right|(v_t-u_3)-(v_t-u_3)U_3{U}_3^T\left|\right|}^2$

RE＝RE ₁+RE ₂+RE ₃

U wherein ₁And u ₂Define as follows:

Therefore, the observation likelihood function of object candidate area can define as follows:

p(o _t|x _t)∝exp(-RE)

(4) utilization particle filter Condensation calculates the posterior probability of dbjective state, and method is following: the motion of hypothetical target between the two continuous frames image is affine motion, and the target affine motion parameter between two frames just can be used for characterizing the state of target

T wherein _x, t _y, η _t, s _t, β _t,

Represent position translation amount, rotation angle, yardstick, aspect ratio and vergence direction respectively.Like this, given a series of observed reading O _t={ o ₁..., o _tCan calculate the posterior probability of dbjective state:

p(x _t|O _t)∝p(o _t|x _t)∫p(x _t|x _t-1)p(x _t-1|O _t-1)dx _t-1

Wherein, p (o _t| x _t) be illustrated in the probability that observed reading ot under the given dbjective state xt takes place, p (x _t| x _T-1) expression dbjective state dynamic transfer probability model.Adopt the strategy of maximum a posteriori estimation (MAP) from a series of candidate's observed readings, to select the maximum dbjective state of posterior probability as current time.

(5) according to the result of target following, the covariance matrix that preceding two patterns are decomposed sample matrix upgrades, and in renewal process, adds forgetting factor, and wherein the more new formula of covariance can be expressed as follows:

$C_d\←\mathrm{\λ}{C}_d+(X_{\left(d\right)}-u_d^{\′}){({X^T}_{\left(d\right)}-u_d^{\′})}^T+\frac{\mathrm{mn}}{m+n}({\mathrm{\μ}}_d-{\mathrm{\μ}}_d^{\′}){({\mathrm{\μ}}_d-{\mathrm{\μ}}_d^{\′})}^T,d=\mathrm{1,2}$

C wherein _dBe the current sample covariance matrix of d pattern, X _(d)It is new sample data; λ is a forgetting factor, and λ ∈ [0,1] in experiment makes that in the model of subspace recent new data occupy bigger weight, means that also model pays close attention to recent data more.After having upgraded the sample covariance matrix of all patterns, try to achieve the feature structure of each covariance matrix again, can realize the online updating of antithetical phrase spatial model.

(6) according to the result of target following, to the 3rd split-matrix utilization R-SVD algorithm, thereby to the online updating of this pattern subspace.Specific algorithm is following:

At first right k the new apparent data E={I of target _T+1..., I _T+kCarry out the QR decomposition, obtain the orthogonal basis of E

In addition, order

Secondly, order $V^{\′}=\left(\begin{array}{cc}V& 0\\ 0& M_k\end{array}\right)$ Wherein, M _kIt is k * k unit matrix.So just have

${\mathrm{\Σ}}^{\′}=U^{\′T}{A}^{\′}{V}^{\′}=\left(\begin{array}{c}{U}^T\\ \tilde{E}\end{array}\right)\left(A\right|E)\left(\begin{array}{cc}V& 0\\ 0& M_k\end{array}\right)$

$=\left(\begin{array}{cc}{U}^T\mathrm{AV}& U^{T}E\\ {\tilde{E}}^T\mathrm{AV}& {\tilde{E}}^{T}E\end{array}\right)=\left(\begin{array}{cc}\mathrm{\Σ}& U^{T}E\\ 0& {\tilde{E}}^{T}E\end{array}\right)$

At last; To ∑ ' carry out svd; Promptly

therefore, the svd of data matrix A '.

A '=U ' ∑ ' V ' ^TJust can be expressed as:

$A^{\′}=U^{\′}\left(\tilde{U}\widetilde{\mathrm{\Σ}}{\tilde{V}}^T\right)V^{\′T}=\left(U^{\′}\tilde{U}\right)\widetilde{\mathrm{\Σ}}\left({\tilde{V}}^T\right)V^{\′T}=\hat{U}\widehat{\mathrm{\Σ}}{\hat{V}}^T$

Claims (6)

1. a visual tracking method that merges object space distribution and timing distribution proper subspace is characterized in that, may further comprise the steps:

(1) the utilization tensor theories is expressed as one three rank tensor with the training sample of target image observation, and this tensor is launched into three split-matrixes from the width of image, these three different directions of height and image time shaft of image;

Wherein, the row sample subspace of preceding two split-matrixes is for the space distribution information of target; The capable sample subspace of the 3rd split-matrix is corresponding to the time distributed intelligence of target;

(2), calculate the sample covariance matrix of each split-matrix, and obtain the pattern subspace of target level and vertical direction through covariance matrix being done feature decomposition for preceding two split-matrixes; For the 3rd split-matrix, adopt singular value decomposition algorithm to obtain the pattern subspace of target on time shaft;

(3) for given object candidate area, calculate its reconstructed error on the apparent space of target, and the observation likelihood function of objective definition candidate region thus;

(4) according to the observation likelihood function of definition, the utilization particle filter calculates the posterior probability of dbjective state, adopts maximum a posteriori estimation strategy from a series of candidate's observed readings, to select the maximum dbjective state as current time of posterior probability;

(5) result who utilize to follow the tracks of to the target pattern subspace that obtains in the step (2) is upgraded, and adds forgetting factor in renewal process;

Wherein, forgetting factor is a kind of weight between 0 and 1, and big more forgetting factor means that model pays close attention to recent data more in the model of subspace.

2. the visual tracking method of fusion object space distribution according to claim 1 and timing distribution proper subspace is characterized in that described step (1) specifically comprises following substep:

At first, with the training sample of target image observation with one three rank tensor representation;

Secondly, with the tensor that obtains from width, highly, these three different directions of time are launched into three split-matrixes;

3. the visual tracking method of fusion object space distribution according to claim 1 and timing distribution proper subspace is characterized in that described step (2) specifically comprises following substep:

At first, for preceding two split-matrixes, calculate the sample covariance matrix of each split-matrix, through covariance matrix being done the subspace that feature decomposition obtains each pattern;

Secondly, for the 3rd split-matrix, adopt singular value decomposition algorithm to obtain the pattern subspace of target on time shaft.

4. the visual tracking method of fusion object space distribution according to claim 1 and timing distribution proper subspace is characterized in that described step (3) specifically comprises following substep:

At first, obtain reconstructed error;

Secondly, by reconstructed error definition likelihood function.

5. the visual tracking method of fusion object space distribution according to claim 1 and timing distribution proper subspace is characterized in that described step (4) specifically comprises following substep:

At first, the motion of hypothetical target between the two continuous frames image is affine motion, and the state of target is by the target affine motion parameter characterization between two frames, and hypothetical target state dynamic transfer probability model is a Gauss model;

Secondly, utilize likelihood function to calculate the posterior probability of dbjective state;

At last, adopt maximum a posteriori to estimate that strategy is from selecting the maximum dbjective state as current time of posterior probability from a series of candidate's observed readings.

6. according to the visual tracking method of described fusion object space distribution and timing distribution proper subspace shown in any claim of claim 1-5, it is characterized in that: described step (5) specifically comprises following substep:

For preceding two split-matrixes,

At first, the covariance matrix to the sample matrix of acquisition in the step (2) upgrades;

Secondly, forgetting factor is added in the renewal equation covariance matrix that is more tallied with the actual situation.

For the 3rd split-matrix,

At first, the data matrix that newly obtains being carried out QR decomposes;

Secondly, the data matrix that merges new legacy data is carried out svd.

Images