CN104484890A - Video target tracking method based on compound sparse model - Google Patents

Abstract

The invention discloses a video target tracking method based on a compound sparse model and belongs to the field of computer vision. According to the method, a combined sparse coefficient matrix capable of observing all particles is divided into set sparsity, element sparsity and abnormal sparsity under the condition that a compound sparse appearance model is under a particle filtering framework; sharing and unsharing characteristics of the particle in a dictionary and additive sparse noise are represented; norms from L1 to infinite number and norms L1 and 1 are regularized to implement the compound sparsity; the optimization problem is solved by using a direction-variable mutiplier method, so that the higher calculation efficiency is realized. The invention also provides a dynamic dictionary updating method, so that the change of target appearance can be adapted. Experiment shows that the tracking performance and robustness of the algorithm are superior to several compared conventional video target tracking algorithm. The video target tracking method can be applied to the fields of man-machine interaction, intelligent monitoring, intelligent traffic, vision navigation, video retrieval, and the like.

Description

Based on the video target tracking method of compound sparse model

Technical field

What the present invention relates to is a kind of technology of field of video processing, specifically a kind of video target tracking method based on compound sparse model being applied to man-machine interaction, intelligent monitoring, intelligent transportation, vision guided navigation, video frequency searching.

Background technology

Video tracking is a major issue of computer vision field, and its task is the two-dimensional image sequence that analysis shot by camera arrives, then to the location that interested target or region continue.No matter be in civilian or military domain, video tracking technology is designated as application prospect widely.Modern intelligent monitor system needs to realize detecting automatically the target in visual field, follow the tracks of and identifying, catches abnormal conditions, and makes early warning.Lacking reliable and efficient video frequency object tracking algorithm is one of Main Bottleneck of field of intelligent monitoring.Field of human-computer interaction, people have no longer been satisfied with the simple interaction models only based on mouse and keyboard.Along with the method for virtual reality technology, the man-machine interaction mode of view-based access control model has become the study hotspot of industry member and academia.In order to realize this target, first computing machine will have the ability of perception human motion, and its prerequisite is the position locating human body from video camera, then understands the behavior of people on this basis.Thus can say that vision tracking is the basis of man-machine interaction.Intelligent transportation system refers to, analyzes the video image of monitored road, draws the information such as vehicle behavior, pedestrian behavior and the magnitude of traffic flow, then controls wagon flow or take precautions against traffic hazard.One in intelligent transportation very crucial problem is segmentation and the tracking of vehicle and pedestrian, and this also belongs to the category of video tracking.In addition, video frequency object tracking in fields such as vision guided navigation, video frequency searching, video compress and vision guided navigations all with important application.But, due to the apparent change of target, attitudes vibration, imaging circumstances change, background interference and the factor such as to block, design accurately, stable, fast video tracking algorithm remain a very challenging task.

In recent years, rarefaction representation, in the application in video tracking field, makes the performance of track algorithm, especially in anti-blocking, is significantly improved.Usually, the video tracking algorithm based on rarefaction representation realizes under particle filter framework, and it uses some particles with weight to represent the probability density function of current goal state, the then position of estimating target.The initial observation of each particle is the pixel of corresponding region in present frame, by sparse apparent model, they can be expressed as by template in a dictionary one group of sparse coefficient on the subspace of opening.The likelihood that candidate target belongs to tracked target is defined by the reconstructed error of the linear combination of this group sparse coefficient on dictionary and former observation.

Video tracking algorithm based on rarefaction representation can rough classification two class: single task algorithm and multitasked algorithm.Typical single task algorithm, as L ₁tracker [X.Mei and H.Ling, " Robust visual tracking using L ₁minimization, " inComputerVision, 2009 IEEE 12th International Conference on, 1436 – 1443, IEEE (2009)], when solving sparse coefficient, separately L is carried out to each particle ₁optimization.This practice has two shortcomings.The first, calculated amount is larger, as L ₁tracker uses interior point method to solve the L of each particle ₁optimization problem, its computation complexity is O (m ²nn mrn), wherein: m, n and r are template number and population in intrinsic dimensionality, dictionary respectively.The second, separate between hypothetical particle, ignore the contact between particle completely.But due to the Gaussian distribution that particle filter uses when sampling particle, most of particle is spatially close, and thus their apparent also has similarity.Utilize this similarity can improve the integrative reconstruction performance of all particles, thus improve tracking accuracy.Multi-task learning can utilize the essential connection between multiple relational learning task, from theoretical and be empirically all proved to be and obviously can improve overall performance.The sparse coefficient solving all particles can be regarded as multiple relevant linear regression problem.Thus, the object based on the rarefaction representation video tracking algorithm of multitask is the performance utilizing the advantage of multi-task learning to improve video tracking.Representational algorithm is multitask tracker (MTT) [T.Zhang, B.Ghanem, S.Liu, andN.Ahuja, " Robust visual tracking via multi ?tasksparse learning, " in Computer Vision andPattern Recognition (CVPR), 2012 IEEE Conferenceon, 2042 – 2049, IEEE (2012)].Although experimental result shows, tracking effect and the counting yield of multitask tracker are all better than L ₁tracker, but it has following problem.The joint sparse that in multitask tracker, particle is apparent is by L _{1, q}norm constraint realizes.L _{1, q}norm can cause the contact of overemphasizing between particle, and particularly during q > 2, the non-zero sparse coefficient of all particles all gets similar value.The general knowledge institute contradiction that the characteristic of this and particle should be made up of the general character between particle and himself individual character.

Through finding the retrieval of prior art, open (bulletin) the day 2014.8.13 of Chinese patent literature CN103985143A, discloses a kind of based on the online method for tracking target of distinctive in the video of dictionary learning.Candidate samples and template are carried out piecemeal by the method, rarefaction representation coefficient is extracted to each piece, again using the sorter of these coefficients as corresponding blocks, obtain the judgement degree of confidence of this block candidate samples, then the judgement degree of confidence of each piece is randomly drawed K and sued for peace, travel through all possibilities, finally in various possibility, be selected the maximum candidate samples of number of times, the most the tracking results of present frame.But in this technology, the optimization problem of the rarefaction representation coefficient of all candidate samples is isolated consideration, and the relation between candidate samples is unheeded.That is, it belongs to the track algorithm of single task always, and thus the estimated accuracy of its sparse coefficient is difficult to satisfied industrial needs.

Summary of the invention

The present invention is directed to the problem ignoring general character between contact and the undue intensity particle of existing multitask rarefaction representation track algorithm between particle completely of the single task rarefaction representation track algorithm that prior art exists, propose a kind of video target tracking method based on compound sparse model.

The present invention is achieved by the following technical solutions, comprises the following steps:

The first step: at the initial frame of video to be tested, namely selects tracked target in the first frame by hand, generates primary collection with its initial position around initial position, tight sampling generates initial dictionary D, selection unit matrix I _mas trifling template.

Second step: in each frame of video to be tested, uses the motion model prediction particle state of target

The motion model of described target adopts random walk model, and namely new state obtains from previous moment state according to Gaussian distribution sampling,

3rd step: extract the pixel value in region of each particle of prediction and boil down to observing matrix Y;

Described region is a parallelogram, defined, comprise parameter and comprise center, length and width, angle of inclination by the state of particle.

Described compression refers to: carry out down-sampling process to obtained pixel value, and merges into vector as observed reading to the pixel value after sampling, finally by multiple Vector Groups synthesis observing matrix Y.

4th step: utilize observing matrix Y, dictionary D, trifling template I _m, solve L _{1, ∞}and L _1,1norm optimization problem, calculates sparse coefficient matrix B, S and T of the sparse apparent model of compound;

The sparse apparent model of described compound is a kind of union feature extraction model of all particles, namely observation rarefaction representation coefficient in dictionary template of all particles is unifiedly calculated, the joint sparse matrix of coefficients of all particles is divided into three parts: organize openness, element is openness and abnormal openness, correspond respectively to the sharing feature of particle, personal characteristics and off-note, be specially: when moment t, in particle filter, the predicted state of n particle is the original of them is observed wherein: for according to state the image-region cut out from present frame; These observed down-samplings and is pressed into vector, obtaining final observation when dictionary is D _t={ d ₁..., d _r, wherein: r is the number of template; Initial dictionary is in initial frame, obtains, in order to the problem solving noise He block, introduce abnormal template I near the original state of target according to tight sampling policy sampling _m, i.e. a unit matrix, by particle observe on noise or block the sparse noise being expressed as an additivity, it can get value large arbitrarily in its support.In the sparse apparent model of compound, observing matrix Y can be expressed as $Y=\left[\begin{array}{cc}D& I_m\end{array}\right]\left[\begin{array}{c}B+S\\ T\end{array}\right]=D(B+S)+I_{m}T,$ Wherein, joint sparse matrix of coefficients B represents the sharing feature of particle on dictionary D, and the element sparse matrix S of personal characteristics represents the sparse coefficient of particle in abnormal template, and element sparse matrix T represents additive noise.

Needed for the L that solves _{1, ∞}and L _1,1norm optimization problem is:

${\min }_{E,B,S,T}{\left|\right|E\left|\right|}_F^2+{\mathrm{\λ}}_b{\left|\right|B\left|\right|}_{1,\∞}+{\mathrm{\λ}}_S{\left|\right|S\left|\right|}_{\mathrm{1,1}}+{\mathrm{\λ}}_t{\left|\right|T\left|\right|}_{\mathrm{1,1}}$

s.t.Y＝D(B+S)+I _mT+E

This model solves especially by multiplier method of changing direction, and namely utilizes linear separability the optimization problem of above-mentioned complexity to be decomposed into several comparatively simple subproblem and solves one by one.This method has the feature of Fast Convergent.Compare other optimization methods, higher convergence precision can be got in less iterations.Concrete, by multiplier method of changing direction, the framework that substantially solves of above-mentioned optimization problem is:

$B^{k+1}=\arg {\min }_B{\mathrm{\λ}}_b{\left|\right|b\left|\right|}_{1,\∞}+\frac{\mathrm{\ρ}}{2}{\left|\right|D(B+S^k)+I_m{T}^k+E^k-Y+U^k\left|\right|}_F^2$

$S^{k+1}=\arg {\min }_S{\mathrm{\λ}}_S{\left|\right|S\left|\right|}_{1,1}+\frac{\mathrm{\ρ}}{2}{\left|\right|D(B+S^k)+I_m{T}^k+E^k-Y+U^k\left|\right|}_F^2$

$T^{k+1}=\arg {\min }_t{\mathrm{\λ}}_t{\left|\right|T\left|\right|}_{1,1}+\frac{\mathrm{\ρ}}{2}{\left|\right|D(B+S^k)+I_m{T}^k+E^k-Y+U^k\left|\right|}_F^2$

$E^{k+1}=\arg {\min }_E{\left|\right|B\left|\right|}_{1,\∞}+\frac{\mathrm{\ρ}}{2}{\left|\right|D(B+S^k)+I_m{T}^k+E^k-Y+U^k\left|\right|}_F^2$

$U^{k+1}=U^k+D(B^{k+1}+S^{k+1})+I_m{T}^{k+1}+E^{k+1}-Y$

Can find out, former optimization problem is decomposed in order to B, S, T, E tetra-subproblems and an iteration variable upgrade.Re-using multiplier method of changing direction is the L of standard by four sub-question variation _{1, ∞}norm and L _1,1norm regularization problem, thus obtain final method for solving.

5th step: calculate reconstructed error E, and the weight to each particle upgrade;

6th step: select to upgrade the target of the maximum particle of rear weight as present frame, the state of this target is corresponding is observed

7th step: use observation template in dictionary and sparse is upgraded by adaptive mode.

8th step: the state estimation exporting all frames, namely for the tracking results of target in this video.

Described adaptive mode specifically comprises the following steps:

1) norm of each template in dictionary is first adjusted, and by the weight of norm as each template.

2) then each frame follow the tracks of terminate after consider the newly apparent of target, when the similarity of all templates in this performance and dictionary is all very low, so think that it is an apparent new pattern, and substitute with its template that in dictionary, weight is minimum; The row of nonzero element more than 80% in the sparse matrix of consideration group simultaneously B, namely by the template with common trait that most of particle is shared, think that they are also the patterns that the apparent probability of occurrence of current target is larger, then improve the weight of its correspondence when the weight of these row is less; Again the weights of that newly add and shared template are set to median, to prevent these templates from playing larger effect at subsequent frame, prevent its effect too leading simultaneously.

3) finally to the weight of dictionary, namely the norm of template is normalized, and makes it and is 1.

Accompanying drawing explanation

Fig. 1 is the sparse apparent model schematic of compound;

Fig. 2 is system flowchart;

Fig. 3 is embodiment tracking results schematic diagram.

Embodiment

Elaborate to embodiments of the invention below, the present embodiment is implemented under premised on technical solution of the present invention, give detailed embodiment and concrete operating process, but protection scope of the present invention is not limited to following embodiment.

Embodiment 1

In the present embodiment, the state of target adopts the affine model of six variablees, i.e. x _t={ a _t, b _t, θ _t, s _t, α _t, φ _t, wherein: 6 parameters represent position coordinates, the anglec of rotation, yardstick, length breadth ratio, angle of inclination respectively; The motion model of target adopts random walk model, and namely new state obtains from previous moment state according to Gaussian distribution sampling, the original pixel value being observed the region that state defines of target, is then compressed into vector by its down-sampling, becomes the actual observation used in compound sparse model; Likelihood function p (O _t| x _t) apparently by particle to define with the similarity of dictionary template, i.e. the reconstructed error Gaussian distributed of the sparse apparent model of the compound of particle.At moment t, current dictionary is D _t-1, n particle is respectively so in the present embodiment, new dbjective state is estimated new target is apparent n new particle new dictionary D _tconcrete steps as described below:

Step 1, resampling is carried out to particle, and result is designated as

Step 2, according to motion model the new state of prediction particle

Step 3, state according to particle, generate observing matrix Y from present frame

Step 4, utilization observation Y, dictionary D _t-1with abnormal template I _m, calculate sparse coefficient matrix B, S and T of the sparse apparent model of compound

4.1) initialization: establish B ⁰=0, S ⁰=0, T ⁰=0, E ⁰=Y, U ⁰=0

4.2) major cycle: For k=0,1 ..., until convergence

4.2.1) B subcycle: G=Y-D _t-1s ^k-I _mt ^k-E ^k-U ^k, B ⁽⁰⁾=B ^k, P ⁽⁰⁾=B ^k, for j=0,1 ..., until convergence

$a)B^{(j+1)}={(\mathrm{\ρ}{D_{t-1}}^T{D}_{t-1}+\mathrm{\τ}{I}_r)}^{-1}[\mathrm{\ρ}{D_{t-1}}^{T}G+\mathrm{\τ}(P^{\left(j\right)}-U_B^{\left(j\right)})]$

$b)\mathrm{For\; i}=\mathrm{0,1},...,r,P_{i\&CenterDot;}^{(j+1)}=\max (0,\mathrm{sgn}(1-\frac{{\mathrm{\&lambda;}}_b}{\mathrm{\&tau;}\left|\right|B_i^{(j+1)}+U_{\mathrm{Bi}\&CenterDot;}^{(j+1)}\left|\right|}))a,$ Wherein: A _ii-th row of representing matrix A, sgn () is-symbol function.A is a row vector, and the computing method of each element in the middle of it are, l=1 ..., n.Wherein: a _land z _lbe l the element of a and z, variable u is | the descending sort of z|, $\hat{J}=\max (j:{\mathrm{\&Sigma;}}_{r=1}^j(u_r-u_j)<\frac{{\mathrm{\&lambda;}}_b}{\mathrm{\&tau;}}).$

C) subcycle iteration variable is upgraded, $U_B^{(j+1)}=U_B^{\left(j\right)}+B^{(j+1)}-P^{(j+1)}$

4.2.2) S subcycle: H=Y-D _t-1s ^k+1-I _mt ^k-E ^k-U ^k, S ⁽⁰⁾=S ^k, A ⁽⁰⁾=S ^k, for j=0,1 ..., until convergence

$a)S^{(j+1)}={(\mathrm{\&rho;}{D_{t-1}}^T{D}_{t-1}+\mathrm{\&tau;}{I}_r)}^{-1}[\mathrm{\&rho;}{D_{t-1}}^{T}H+\mathrm{\&tau;}(A^{\left(j\right)}-U_S^{\left(j\right)})]$

C) subcycle iteration variable is upgraded, $U_S^{(j+1)}=U_S^{\left(j\right)}+S^{(j+1)}-A^{(j+1)}$

4.2.3) T subcycle:

4.2.4) E subcycle: $E^{k+1}=\frac{\mathrm{\&rho;}}{\mathrm{\&rho;}+2}(Y-D_{t-1}(B^{k+1}+S^{k+1})-I_m{T}^{k+1}-U^k)$

4.2.5) iteration variable upgrades: U ^k+1=U ^k+ D _t-1(B ^k+1+ S ^k+1)+I _mt ^k+1+ E ^k+1-Y

The reconstructed error E=Y-D of step 5, calculating particle _t-1(B+S)-I _mt

The weight of step 6, more new particle, $w_t^i={\tilde{w}}_{t-1}^i\exp (-{\left|\right|E^i\left|\right|}_2^2)$

Step 7, select the maximum particle of weight as the estimated value of current goal state it is observed

Step 8, utilization d _t-1, B, S, T carry out dynamically updating of dictionary, obtains new dictionary D _t

8.1) according to current dictionary D _t-1the norm of middle template, calculates present weight π _i=|| d _i||

8.2) C=D _t-1(B+S), at D _t-1on reconstruction coefficients c be respective column in C

8.3) use reconstruction coefficients c, upgrade weight

8.4) median of note π is θ=median (π)

8.5) if new observation with the template d that reconstruction coefficients is maximum _isimilarity less, namely i=arg max _1≤i≤r| c _i|, so: j=arg min _1≤i≤rπ _i, replace the template that weight is minimum π _j=θ

8.6) index set of the row of nonzero element in B more than 80% is designated as Q, to any k ∈ Q, π _j< θ, is set to π by its weight _j=θ

8.7) normalized weight π, makes ∑ _iπ _i=1

8.8) norm of normalization dictionary, makes || d _i||=π _i.Finally obtaining new dictionary is D _t

The final tracking results of step 9, target is

The present embodiment is by the mixing Mex programming realization of Matlab and C++, wherein Matlab is responsible for the main process of algorithm, as particle filter, sampling observation, weight calculation, template state estimation, dictionary updating etc., in the sparse apparent model of compound, the calculating of the coefficient of colligation matrix of particle adopts C++ to realize, and can improve the real-time of system like this.Process flow diagram as shown in Figure 2.

The present embodiment uses Matlab language to test in several sections of real video sequences the proposed video frequency object tracking algorithm based on compound sparse model.The video sequence that test uses is all from following network address:

https://sites.google.com/site/trackerbenchmark/benchmarks/v10

As shown in Figure 3, be deer, davidindoor, sylvester, faceocc, football and car11 respectively.In order to verify the performance of the present embodiment algorithm, testing and the algorithm of the present embodiment proposition and several traditional video tracking algorithm are compared, is L respectively ₁tracker, frag, IVT, MS, TLD and CT.The tracking results of all algorithms in test data as shown in Figure 3.The difficult point of deer video is the fast reserve of fawn, motion blur and similar purpose interference; The difficult point of davidindoor video is illumination variation, dimensional variation, the expression shape change of people and partial occlusion; The difficult point of sylvester video is long-time tracking, attitudes vibration and rotation; The difficult point of faceocc video is large area and blocks for a long time; The difficult point of football video is rapid movement, a large amount of similar purpose interference and blocks; The difficult point of car11 video is the image blur that stronger illumination variation and rainwater cause.By the relative position of solid box in comparison diagram 3 with other frames, can find out that the video frequency object tracking algorithm in compound sparse model that the present embodiment proposes all achieves the result being better than traditional track algorithm under above-mentioned difficulties, thus there is better tracking performance and robustness.

Claims (7)

1. based on a video target tracking method for compound sparse model, it is characterized in that, comprise the following steps:

The first step: at the initial frame of video to be tested, namely selects tracked target in the first frame by hand, generates primary collection with its initial position around initial position, tight sampling generates initial dictionary D;

Second step: in each frame of video to be tested, uses the motion model prediction particle state of target

3rd step: extract the pixel value in region of each particle of prediction and boil down to observing matrix Y;

4th step: at the sparse apparent model of compound $Y=\left[\begin{array}{cc}D& I_m\end{array}\right]\left[\begin{array}{c}B+S\\ T\end{array}\right]=D(B+S)+I_{m}T$ In, pass through L _{1, ∞}and L _1,1norm optimization calculates joint sparse matrix of coefficients B, represents the element sparse matrix S of the sharing feature of particle on dictionary D and personal characteristics and represent the element sparse matrix T of the sparse coefficient of particle in abnormal template respectively;

5th step: calculate reconstructed error E, and the weight to each particle upgrade;

6th step: to upgrade the target of the maximum particle of rear weight as present frame, the state of this target is be observed

7th step: use observation template in dictionary and sparse is upgraded by adaptive mode;

8th step: the state estimation exporting all frames, namely for the tracking results of target in this video.

2. method according to claim 1, is characterized in that, the motion model of described target adopts random walk model, and namely new state obtains from previous moment state according to Gaussian distribution sampling,

3. method according to claim 1, is characterized in that, the region described in the 3rd step is a parallelogram, and by the state of particle, namely center, length and width, angle of inclination defined.

4. method according to claim 1, is characterized in that, described compression refers to: carry out down-sampling process to obtained pixel value, and merges into vector as observed reading to the pixel value after sampling, finally by multiple Vector Groups synthesis observing matrix Y.

5. method according to claim 1, it is characterized in that, the sparse apparent model of described compound is a kind of joint sparse Feature Selection Model of all particles, namely the rarefaction representation coefficient of the observation of all particles is unifiedly calculated, the joint sparse matrix of coefficients of all particles is divided into three parts by it: group's sparse coefficient matrix B, element sparse coefficient matrix T and abnormal sparse coefficient matrix T, correspond respectively to the sharing feature of particle, personal characteristics and off-note.

6. method according to claim 1, is characterized in that, the solving-optimizing equation of the sparse apparent model of described compound is:

${\min }_{E,B,S,T}{\left|\right|E\left|\right|}_F^2+{\mathrm{\&lambda;}}_b{\left|\right|B\left|\right|}_{1,\&infin;}+{\mathrm{\&lambda;}}_s{\left|\right|S\left|\right|}_{\mathrm{1,1}}+{\mathrm{\&lambda;}}_t{\left|\right|T\left|\right|}_{\mathrm{1,1}}$

s.t.Y＝D(B+S)+I _mT+E

This model solves especially by multiplier method of changing direction, and namely utilizes linear separability the optimization problem of above-mentioned complexity to be decomposed into several comparatively simple subproblem and solves one by one.

7. method according to claim 6, is characterized in that, former optimization problem is decomposed in order to B, S, T, E tetra-subproblems and an iteration variable upgrade by described multiplier method of changing direction, and re-using multiplier method of changing direction is the L of standard by four sub-question variation _{1, ∞}norm and L _1,1norm regularization problem, thus obtain final method for solving, the framework that substantially solves of this optimization problem is:

$B^{k+1}=\arg {\min }_B{\mathrm{\&lambda;}}_b{\left|\right|B\left|\right|}_{1,\&infin;}+\frac{\mathrm{\&rho;}}{2}{\left|\right|D(B+S^k)+I_m{T}^k+E^k-Y+U^k\left|\right|}_F^2$

$S^{k+1}=\arg {\min }_S{\mathrm{\&lambda;}}_s{\left|\right|S\left|\right|}_{1,1}+\frac{\mathrm{\&rho;}}{2}{\left|\right|D(B+S^k)+I_m{T}^k+E^k-Y+U^k\left|\right|}_F^2$

$T^{k+1}=\arg {\min }_t{\mathrm{\&lambda;}}_t{\left|\right|T\left|\right|}_{1,1}+\frac{\mathrm{\&rho;}}{2}{\left|\right|D(B+S^k)+I_m{T}^k+E^k-Y+U^k\left|\right|}_F^2$

$E^{k+1}=\arg {\min }_E{\left|\right|B\left|\right|}_{1,\&infin;}+\frac{\mathrm{\&rho;}}{2}{\left|\right|D(B+S^k)+I_m{T}^k+E^k-Y+U^k\left|\right|}_F^2$

U ^k+1＝U ^k+D(B ^k+1+S ^k+1)+I _mT ^k+1+E ^k+1-Y。