CN107368785A - The video target tracking method of multinuclear local restriction - Google Patents

Abstract

The present invention provides a video target tracking method with multi-core local constraints, including: adopting a local constraint linear coding method, introducing the local structure of sample data into a cooperative representation method to obtain a sample representation with good classification performance; using a kernel function to The cooperative representation is extended to the kernel space of multi-feature fusion, which enhances the class discrimination ability of the dictionary and sparse representation coefficients on the target features; the target tracking is regarded as a binary classification problem, and the candidate target with the highest classifier score is used as the tracking target under the particle filter framework Target. The method can be used for accurate and robust tracking of video objects.

Description

Multi-core Local Constraint Video Object Tracking Method

technical field

The present invention relates to the technical field of computer vision, in particular to a video target tracking method with multi-core local constraints.

Background technique

Visual target tracking is an important research content in the field of computer vision. It has been widely used in visual navigation, human-computer interaction, intelligent transportation, video surveillance and other fields. It is a variety of subsequent advanced processing, such as target recognition, behavior analysis, video image compression. Fundamentals of high-level video processing and applications such as coding and application understanding. However, due to factors such as occlusion, illumination changes, scale changes, sudden changes, and angle changes in tracking videos, it makes accurate and robust video object tracking a very important task.

With the development of compressed sensing theory and sparse coding theory, sparse representation has been applied to the tracking of video targets, and its core is to regard the target as a sparse representation problem under the particle filter framework. In sparse representation tracking, the l ₁ tracking method has strong robustness, but it is difficult and time-consuming to solve the l ₁ norm minimization problem. Therefore, a cooperative sparse representation based on the l ₂ norm was proposed and applied to target tracking. Although the cooperative representation under the l ₂ norm is not as sparse as the reconstruction coefficient under the l ₁ norm, the mapping can be calculated in advance The matrix does not need to update each particle, thus improving the computational efficiency; however, the cooperative representation method is essentially a linear method, which will lead to tracking failure when the target undergoes nonlinear changes (strong illumination changes, sudden movements, violent background shakes) .

Contents of the invention

Aiming at the deficiencies in the prior art, the present invention provides a video target tracking method with multi-core local constraints, improves the rendering speed by designing a single-pass rendering scheme, and enhances the rendering quality through EWA.

The technical scheme that the present invention realizes that its object of the invention adopts is:

A video target tracking method with multi-core local constraints, comprising the steps of:

(1) Introduce the local structure of the sample data into the collaborative representation method to construct a locally constrained linear coding of the sample features; (2) use the kernel method to construct a multi-core local constrained collaborative coding of the sample features; (3) Based on the support vector machine SVM , embed the classifier score of the sample into the particle filter framework to track the video target; (4) dynamically update the target template, background template and classifier according to the changes of the target and background samples. Support vector machine is Support Vector Machine, SVM.

Preferably, in step (1), in order to obtain a sample representation with good classification performance, the local structure of the sample data is introduced into the collaborative representation to construct a locally constrained collaborative encoding of sample features, specifically:

(a) Under the sparse representation framework, the test sample Expressed as y=d ₁ x ₁ +d ₂ x ₂ +...+d _n x _n = Dx, where the dictionary Represents a dictionary atom, and the objective function of its cooperative representation is The synergy of samples obtained by minimizing is expressed as Optimal solution It is only the linear projection of y, and P=(D ^T D+λI) ^-1 D ^T is independent of y, so that the projection matrix P can be calculated in advance, avoiding the need to optimize each test sample separately under the l ₁ norm processing, greatly improving the computing speed.

(b) The collaborative representation method is essentially a linear method that uses non-local dictionary atoms to reconstruct candidate target coefficients. However, the local structure of the data often carries more information than the global structure, such as a sample and its surrounding samples should be have similar encodings, so the sample reconstruction using local constraints is more accurate; the local structure of the sample data is introduced into the collaborative representation, and its objective function is in is a locally constrained diagonal matrix, and the local constrained co-encoding of the sample is obtained by minimizing as

As a preference, in step (2), based on the kernel method, the multi-core local constraint collaborative encoding of sample features is constructed, specifically:

(a) Using the kernel method to map the nonlinear data to the high-dimensional linear kernel space, the image of the sample y in the high-dimensional space is given by Φ=[φ(d ₁ ),φ(d ₂ ),…,φ(d _n ) ] to linear representation, which obviously increases the computational complexity, so it is necessary to reduce the dimensionality of the feature space. Using the random projection matrix PT to map the high ^- dimensional data to the low-dimensional space, the objective function of the locally constrained kernel cooperative coding is

Take its partial derivative and equal to 0 to get

Where [x ₁ ,x ₂ ,…,x _n ] ^T is the n-dimensional coefficient vector x, denote

Then the solution of formula (1) can be written as

According to the nature of the kernel function, record P=ΦB, and substitute into formula (2) and formula (3) respectively to get

Where K=Φ ^T Φ is the kernel Gram matrix (positive semi-definite symmetric matrix). Inner product of samples in feature space can be calculated by kernel function, for any two samples φ(x), φ(y), there is φ(x) ^T φ(y)=(φ(x)·φ(y)) ＝k(x,y), then

(b) In order to track the target stably, multiple features are used to describe the target, so the fusion kernel function is obtained by multi-kernel fusion as Multi-kernel locally constrained collaborative coding of sample features

The present invention uses the spatial color histogram and the spatial gradient histogram to represent the target; for each target sample, it is divided into four sub-regions with equal areas, and then the color features of the sub-regions are respectively extracted and combined into the target color feature h ^c . For the gradient feature, first use the kernel [-0.5,0,0.5] and [-0.5,0,0.5] ^T for filtering to obtain the image gradient; then use the same method to obtain the gradient histogram of the four sub-regions, and combine The histograms of the four sub-regions obtain the spatial gradient histogram h ^g . Let K _c and K _g be the kernel matrices of color features and gradient features respectively, and each element in K _c and K _g represents the similarity between two histograms; the corresponding elements in K _c are according to K _c (i,j ) = BhaCoff to calculate, where with are two color histograms, BhaCoff(·) is Bhattacharyya coefficient function; K _g , K _c (·,y), K _g (·,y) are calculated in the same way as K _c .

As preferably, in step (3), based on the support vector machine SVM, the classifier score of the sample is embedded into the tracking of the video target under the particle filter framework, specifically:

(a) Extract a certain number of target samples and background samples according to the Gaussian distribution around the target area in the first frame. The target samples and background samples are also positive samples and negative samples respectively. Calculate the multi-core locality of the positive and negative samples under the current dictionary. Constrained collaborative encoding z _i , the encoding of positive and negative samples and their corresponding labels Substituting the support vector machine SVM for training, by minimizing the cost function To learn a classifier, the score of the classifier is calculated as

(b) Embedding classification into the particle filter framework to achieve target tracking; particle filter is an important sampling technique for Bayesian sequences, and is often used to estimate the posterior density of state variables in dynamic systems; assume that the target state variable at time t is s _t , given the target observation set y _1:t = {y ₁ ,y ₂ ,…,y _t }, the current state s _t can be determined by the maximum posterior probability:

The recursive calculation of the posterior probability is divided into two steps: formula (5) prediction and formula (6) update

p(s _t |y _1:t )＝∫p(s _t |s _t-1 )p(s _t-1 |y _1:t-1 )ds _t-1 (5)

where p(s _t |s _t-1 ) is the state transition probability used to describe the dynamic model, p(y _t |s _t ) is the observation likelihood function used to describe the observation model; use the six affine transformation parameters of the image to To define the target motion in adjacent frames, let s _t = (υ ₁ ,υ ₂ ,υ ₃ ,υ ₄ ,t _x ,t _y ), where (υ ₁ ,υ ₂ ,υ ₃ ,υ ₄ ) represent the rotation angle, Scale, area ratio, and tilt direction, (t _x , _ty ) represent 2D position parameters, and the state transition between adjacent frames is expressed as

p(s _t |s _t-1 )＝N(s _t ; s _t-1 ,Σ)

where N(·) is a Gaussian distribution function, and Σ is a diagonal covariance matrix (the diagonal elements of which are the variances of the corresponding motion parameters in s). Based on the learned classifier, the observation model is defined as p(y|s)∝f(z), f(z) is the classification score calculated by formula (4), then the candidate target with the highest score is regarded as the tracking result .

As preferably, in step (4), in order to realize robust video target tracking, dynamically update target template and background template and classifier, specifically:

(a) Regarding the update of the target template D _f , let α be the coefficient of the new tracking result y on the target dictionary template, s is the atom corresponding to the largest coefficient in y and α, that is, the Barthel coefficient between samples, and s _i is y and the Barthel coefficient of each atom in the target template D _f and s _m as its minimum value, and set two thresholds τ ₁ <τ ₂ . If s>τ ₂ , it means that the tracking result can be well represented by the target template; if s<τ ₁ , it means that the tracking target has undergone a strong appearance change, and at this time replace the target sample corresponding to s _m with y;

(b) Regarding the update of the background template D _b , after determining the tracking target in the current frame, select M _B background samples according to the Gaussian distribution around the target area, and randomly replace the samples in the current background dictionary;

(c) Regarding the update of the classifier, the current updated target template _Df and the samples in the background template _Db , that is, the current positive and negative samples, are brought into the classifier for training to obtain the current classifier.

The beneficial effects of the present invention are:

The present invention adopts the local constraint linear coding method, introduces the local structure of the sample data into the cooperative representation method, and obtains the sample representation with good classification performance; uses the kernel function to extend the cooperative representation to the kernel space of multi-feature fusion, so that the dictionary and The class discrimination ability of sparse representation coefficients on target features is enhanced; under the particle filter framework, the candidate target with the highest score of the binary classifier is used as the tracking target, so the video target can be tracked accurately and robustly.

Description of drawings

Figure 1-4 is the video target tracking effect diagram and the tracking effect comparison diagram of the case of the present invention, compared with other four methods: the IVT method proposed by Ross in 2008, the L1 method proposed by Mei in 2011, and the VID method proposed by Kwon in 2010 , MIL law proposed by Babenko in 2009.

Note: In Figure 1-4, each rectangular box selects the effect of each method, and the rectangular boxes corresponding to the effect of each method are respectively represented by labels A, B, C, D, and E, and the descriptions of the methods represented by each label as follows:

A (IVT); B (L1); C (VTD); D (MIL); E (this invention)

Fig. 1-Oclcusion2 sequence (note: in this sequence, target rotation and occlusion change occurred); as can be seen from the tracking result of Fig. 1, the present invention can track target accurately when occlusion and rotation occur, IVT, VTD tracking Failed (eg #501), MIL is able to track but cannot estimate rotation changes, l ₁ tracking drifts.

Figure 2-DavidIndoor sequence (note: as the target moves forward and backward, the target's head rotates, and the target size changes due to the distance, and the background lighting also changes); Figure 3-Car11 Sequence (Note: the background is complex, the target is blurred and the target size has changed); from Figure 2 and Figure 3, it can be seen that the IVT algorithm is superior to the other three algorithms, and has rotation adaptability, but poor adaptability to scale changes. The present invention is superior to others in object rotation and scale change adaptability.

Fig. 4 Deer sequence (note: target fuzzy phenomenon has occurred because of the rapid motion of target); As can be seen from Fig. 4, only the present invention and VTD algorithm can locate the target, other algorithms all occur the phenomenon of missing target (as #40), simultaneously There is an offset in the tracking of VTD. The invention can realize accurate and robust target tracking in complex environments.

detailed description

The technical solution of the present invention will be described in further detail below through specific embodiments and in conjunction with the accompanying drawings.

With reference to Fig. 1-4, a kind of video target tracking method of multi-core local constraint, described method comprises:

(1) Introduce the local structure of sample data into the collaborative representation method to construct a locally constrained linear encoding of sample features. The details are as follows: (a) Under the sparse representation framework, the test sample y=d ₁ x ₁ +d ₂ x ₂ +...+d _n x _n ＝Dx, the objective function of its collaborative representation is The synergy of samples obtained by minimizing is expressed as (b) Introduce the local structure of the sample data into the collaborative representation, and its objective function is The locally constrained co-encoding of samples obtained by minimization is

(2) Using the kernel method, a multi-kernel local constraint collaborative encoding of sample features is constructed. The details are as follows: (a) The image of sample y in high-dimensional space is represented linearly by Φ=[φ(d ₁ ),φ(d ₂ ),…,φ(d _n )], and the random projection matrix P ^T is used to Map the high-dimensional data to the low-dimensional space to obtain the objective function of the locally constrained kernel cooperative coding (Formula (1)), calculate its partial derivative and equal to 0, and according to the nature of the kernel function, record P=ΦB to obtain the objective function

(b) The fusion kernel function obtained by multi-core fusion is Use the spatial color histogram and the spatial gradient histogram to represent the target: for each target sample, it is divided into four sub-regions with equal areas, and then the color features of the sub-regions are respectively extracted and merged into the target color feature h ^c . For the gradient feature, first use the kernel [-0.5,0,0.5] and [-0.5,0,0.5] ^T for filtering to obtain the image gradient; then use the same method to obtain the gradient histogram of the four sub-regions, and combine The histograms of the four sub-regions obtain the spatial gradient histogram h ^g . according to To calculate the elements in K _c , the calculation methods of K _g , K _c (·, y), and K _g (·, y) are the same as K _c . Multi-kernel locally constrained collaborative coding of sample features

(3) Based on SVM, the classifier score of the sample is embedded into the particle filter framework to track the video target. The details are as follows: (a) Extract the target samples (positive samples) and background samples (negative samples) according to the Gaussian distribution around the target area in the first frame, and calculate the multi-core local constraint cooperative coding z _i of the positive and negative samples under the current dictionary respectively, and set Encoding of positive and negative samples and their corresponding labels Substitute into SVM for training, by minimizing the cost function To learn the classifier, the score f(z) of the classifier is obtained by formula (4); (b) Based on the learned classifier, the observation model under the particle filter framework is defined as p(y|s)∝f( z), find the candidate target with the highest score, which is the tracking result.

(4) Dynamically update the target template, background template and classifier according to the changes of the target and background samples. The details are as follows: (a) The update of the target template _Df . If s>τ ₂ , it means that the tracking result can be well represented by the target template, and D _f is retained; if s<τ ₁ , it means that the tracking target has undergone a strong appearance change, and at this time, replace s _m with y target sample. (b) The update of the background template _Db . After determining the tracking target in the current frame, select M _B background samples around the target area according to the Gaussian distribution, and randomly replace the samples in the current background dictionary. (c) The update of the classifier, the samples in the current updated target template D _f and the background template D _b are brought into the classifier for training to obtain the current classifier.

Referring to Figures 1-4, the tracking effect of the present invention is compared with the tracking effects of the other four methods. It can be seen that the tracking effect of the present invention is better when the target motion blur, scale change and fast movement, occlusion, and illumination change occur. Compared with the other four methods, accurate and robust object tracking is achieved.

Claims (5)

1. a kind of video target tracking method of multinuclear local restriction, it is characterised in that the tracking comprises the steps： (1) partial structurtes of sample data are incorporated into collaboration method for expressing, build the local restriction uniform enconding of sample characteristics； (2) kernel method is utilized, builds the multinuclear local restriction collaboration coding of sample characteristics；(3) SVMs is based on, by point of sample Class device score is embedded into the tracking that video object is realized under particle filter framework；(4) change according to target and background sample, move State updates To Template and background template and grader.

2. the video target tracking method of multinuclear local restriction as claimed in claim 1, it is characterised in that：In step (1), Represented to obtain the sample with good classification performance, the partial structurtes of sample data are incorporated into collaboration expression, carry out structure The local restriction collaboration coding of sample characteristics is built, is specifically：(a) under framework of sparse representation, test sampleIt is expressed as Y=d₁x₁+d₂x₂+…+d_nx_n=Dx, wherein dictionaryIts cooperate with represent object function beThe collaboration that sample is obtained by minimizing is expressed as (b) partial structurtes of sample data are incorporated into collaboration to represent, its object function is WhereinIt is the diagonal matrix of a local restriction, the local restriction that sample is obtained by minimizing is assisted It is same to be encoded to

3. the video target tracking method of multinuclear local restriction as claimed in claim 1, it is characterised in that：In step (2), Based on kernel method, the multinuclear local restriction collaboration coding of sample characteristics is built, is specifically：(a) kernel method is used by non-linear number According to High-dimensional Linear nuclear space is mapped to, images of the sample y in higher dimensional space is by Φ=[φ (d₁),φ(d₂),…,φ(d_n)] Carry out linear expression, using accidental projection matrix P^THigh dimensional data is mapped to lower dimensional space again, local restriction nuclear coordination coding Object function is

Local derviation and the property according to kernel function are asked it, Note P=Φ B obtain object function and areWherein K=Φ^TΦ It is core Gram matrixes；(b) in order to stably track target, target is described using multiple features, then passes through the side of multi-core integration Formula obtains merging kernel functionSo as to obtain the multinuclear local restriction of sample characteristics collaboration coding

4. the video target tracking method of multinuclear local restriction as claimed in claim 1, it is characterised in that：In step (3), Based on SVMs, the grader score of sample is embedded into the tracking that video object is realized under particle filter framework, specifically It is：(a) a number of target sample and background sample, target sample are extracted according to Gaussian Profile around the first frame target area Originally, background sample is also respectively positive sample, negative sample, calculates multinuclear local restriction of the positive and negative samples under current dictionary respectively Collaboration coding z_i, by the coding of positive and negative samples and its corresponding labelSubstitute into branch Hold vector machine to be trained, by minimizing cost functionCarry out learning classification Device, the score of grader are calculated as(b) based on the grader learnt, by the sight under particle filter framework Survey model definition is p (y | s) ∝ f (z), and the candidate target with top score is tracking result.

5. the video target tracking method of multinuclear local restriction as claimed in claim 1, it is characterised in that：In step (4), In order to realize the video frequency object tracking of robust, To Template and background template and grader are dynamicallyd update, is specifically：(a) On To Template D_fRenewal, if tracking result y can well by To Template represent if retain D_f, otherwise obtain y and D_fIn The minimum dictionary atom of Pasteur's coefficient, and replaced it by y；(b) on background template D_bRenewal, in the current frame determine tracking After target, M is selected according to Gaussian Profile around target area_BIndividual background sample, the sample in random replacement current background dictionary This；(c) renewal on grader, by the To Template D after current renewal_fWith background template D_bIn sample, bring grader into It is trained and just obtains current grader.