CN103413312A - Video target tracking method based on neighborhood components analysis and scale space theory - Google Patents

Abstract

The invention belongs to the technical field of computer vision, and in particular relates to a video target tracking method based on neighborhood component analysis and scale space theory. The present invention proposes to use the feature transformation function of the Neighborhood Component Analysis (NCA) to obtain the optimal feature for distinguishing the target and the background, and to obtain the best linear classifier for distinguishing the target and background pixels in any frame image. Angle solves the problem of updating target features; a particle confidence calculation method based on multi-scale normalized Laplacian filter function is proposed, and the state diversity and convergence characteristics of particle filter are used to avoid the algorithm falling into the local optimal point problem after the target is occluded. , which guarantees the tracking accuracy based on the scale space theory. The invention can more accurately locate the position and scale of the target, more effectively adapt to the problems of target illumination and color change, and simultaneously robustly process target occlusion.

Description

Video Object Tracking Method Based on Neighborhood Component Analysis and Scale Space Theory

technical field

The invention belongs to the technical field of computer vision, and in particular relates to a video target tracking method based on neighborhood component analysis and scale space theory.

Background technique

In many applications in the field of computer vision, such as intelligent monitoring, robot vision, human-computer interaction interface, etc., it is necessary to track moving objects in video sequences. Due to the diversity of tracking target shapes and the uncertainty of target motion, how to achieve robust real-time tracking in various environments and achieve reliable estimation of variable scales as the target distance changes has always been a research hotspot. Mean shift (mean shift) is the most researched target tracking method in academia in recent years. It is a non-parametric density gradient ascent algorithm, which is used to find the extreme value of the probability density function, along the gradient direction of the target state probability function, local Iteratively searches the image for the most likely state of the object. This type of algorithm is fast and widely respected. However, the adaptive mechanism of scale in mean-shift methods has been an important issue. The traditional mean shift algorithm uses a fixed-bandwidth kernel function, which cannot adaptively track the scaling of the target, which easily leads to inaccurate positioning. When the selected bandwidth is too large, the feature probability distribution of the extracted candidate area will contain background interference, which will affect the positioning; conversely, when the bandwidth is too small, only the local feature probability distribution of the target can be obtained, which will also lead to positioning errors.

In 1998, Bradski et al. improved the basic mean shift method, and used the higher-order moments of the target state probability distribution to obtain a more accurate target scale while searching for the target position, that is, the scale-adaptive mean shift method (CamShift) . However, this method is prone to failure when the color of the surrounding environment is close to the color of the target. Dong Rong (Dong Rong, Li Bo, Chen Qimei, multi-degree-of-freedom mean-shift tracking algorithm based on SIFT features. Control and Decision, 2012(03):399-402+407) proposed a method using SIFT features to obtain The target scale and target direction are used to adjust the bandwidth and direction of the kernel function of the mean shift algorithm, so as to improve the adaptability of the mean shift tracking algorithm to the change of the target scale. Because the mean shift algorithm itself may converge to the local optimal point in the state space, the algorithm still cannot theoretically guarantee the tracking accuracy. Qin Jian (Qin Jian, Zeng Xiaoping, Li Yongming, Mean-Shift Kernel Window Width Adaptive Algorithm Based on Boundary Force. Journal of Software, 2009(07):p.1726-1734.) proposed to introduce the region likelihood to extract The local information of the target compares the regional likelihood between adjacent frames and constructs the boundary force. Through the calculation of the boundary force, the position of the boundary point is obtained, and then adaptively updates the kernel function bandwidth based on the mean value shift of the boundary force. tracking algorithm. For the same reason, limited by the shortcomings of the mean shift algorithm itself, this algorithm cannot theoretically guarantee the tracking accuracy. Similar problems also exist in other algorithms based on mean shift (such as Wang Yong, Chen Fenxiong, Guo Hongxiang, offset-corrected kernel space histogram target tracking. Acta Automatica Sinica, 2012(03): p.430- 436.). In the authorized invention patent with publication number CN101281648A (low-complexity scale-adaptive video target tracking method), the algorithm proposed by the inventor uses the particle filter algorithm as the framework and uses the mean shift algorithm in the importance sampling function , which improves the sampling efficiency and overcomes the shortcoming of the single-scale mean shift method converging on the local optimum, but because it does not consider the adaptive problem of changes in the characteristics of the target itself, it is difficult to ensure the positioning accuracy when the illumination changes significantly.

Neighborhood Components Analysis (NCA) is a method proposed by Goldberger (J. Goldberger, S. Roweis, G. Hinton, R. Salakhutdinov. (2005) Neighborhood Component Analysis. Advances in Neural Information Processing Systems. 17, 513-520.) A supervised learning method for distance metrics, the purpose of which is to maximize the average leave-one-out classification effect in the new transition space by learning a linear space transition matrix on the training set. It measures sample data according to a given distance measure algorithm and then classifies multi-class clusters. In terms of function, it has the same purpose as the k-nearest neighbor algorithm, directly using the concept of random neighbors to determine the labeled training samples that are close to the test samples.

Its typical processing process is described as follows: let _xi represent the i-th sample in the training set sample, and _ci be its category, then the sample transformed by the linear space transfer matrix A is AX, and consider in the transformed new space The entire dataset is used as random nearest neighbors. Use the square Euclidean distance function to define the distance between the leave-one-out data point and other data in the new transformation space, the function is defined as follows:

其梯度函数经过推导为p _ij is also the probability that the sample point x _j is the nearest neighbor of x _i . The classification accuracy of x _i is the classification accuracy of its adjacent nearest neighbor set C _i (C _i ={j| _ci =c _j }): Select A that can maximize the classification accuracy as A _new , namely $A_{\mathrm{new}}=\arg \underset{A}{\max }\underset{i}{\mathrm{\Σ}}{p}_i.$ For the convenience of calculation, the objective function $f\left(A\right)=\underset{i}{\mathrm{\Σ}}{p}_i=\underset{i}{\mathrm{\Σ}}\underset{j\∈C_i}{\mathrm{\Σ}}{p}_{\mathrm{ij}}$ re-written as:

Its gradient function is derived as

$\frac{<span\; class="notranslate">\; \&PartialD;</span>\mathrm{<span\; class="notranslate">\; g</span>}}{<span\; class="notranslate">\; \&PartialD;</span>\mathrm{<span\; class="notranslate">\; A</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; A</span>}\underset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; ik</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; ik</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; ik</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; -</span>\frac{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}}{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ;</span>$

where x _ij =x _i -x _j . The conjugate gradient multivariate optimization method can be used to obtain $A_{\mathrm{new}}=\arg \underset{A}{\max }g\left(A\right)=\arg \underset{A}{\max }\underset{i}{\mathrm{\&Sigma;}}\log \left(\underset{j\&Element;C_i}{\mathrm{\&Sigma;}}{p}_{\mathrm{ij}}\right).$ A _new will maximize the average leave-one-out classification effect of the training sample set in the new transformation space.

x=(x₁,...x_D)^T，t被称为L的尺度参数。以L_xx和L_yy分别表示L在水平和垂直方向上的二阶偏导数，则可如下确定多尺度规范化Laplacian滤波函数在(x,y,t)处的值：Laplacian(x,y,t)=(t(L_xx(x,y)+L_yy(x,y)))²。当灰度图像中包含若干大小不同的正方形灰度块时，该函数会在不同尺度参数t下，先后在各个灰度块的中心取得极大值，通过检查这些极大值所在的尺度和位置，就能确定各灰度块的中心位置和尺度。Multi-scale normalized Laplacian filtering is used to detect different The calculation formula of the grayscale block on the scale. It regards the grayscale image as a two-dimensional function f, namely f: R ² →R. Its linear scale space representation L is defined as a convolution with a 3-dimensional Gaussian kernel g with variable width t: L(·;t)=g(·;t)*f(·), where g:

x=(x ₁ ,...x _D ) ^T , t is called the scale parameter of L. Let L _xx and L _yy represent the second-order partial derivatives of L in the horizontal and vertical directions, respectively, then the value of the multi-scale normalized Laplacian filter function at (x, y, t) can be determined as follows: Laplacian(x, y, t )=(t(L _xx (x,y)+L _yy (x,y))) ² . When the grayscale image contains several square grayscale blocks of different sizes, the function will successively obtain the maximum value at the center of each grayscale block under different scale parameters t, by checking the scale and position of these maximum values , the center position and scale of each gray block can be determined.

Contents of the invention

The object of the present invention is to propose a video target tracking method based on neighborhood component analysis and scale space theory, aiming at the above-mentioned deficiencies in the prior art.

A video target tracking method based on neighborhood component analysis and scale space theory, the video target tracking method includes the following steps:

Step 1: Determine the initial rectangular frame of the target by detection or manual labeling in the first frame, obtain the initial state of the target, and initialize the particle filter;

Determine the initial rectangular frame of the target through the target detection method or manual labeling. The coordinates of the upper left corner of the target rectangle are (r, c), the width and height of the rectangular frame are (w, h), and the initial scale parameter s of the target is calculated by the following formula :

s=((13+(w-34)*0.47619)) ² ;

记录目标的宽度与高度比例为： $\mathrm{asr}=\frac{w}{h};$ Among them, the target center point is

The width-to-height ratio of the record target is: $\mathrm{asr}=\frac{w}{h};$

Set the lower limit of sampling in the initial state of the target as lb, and the upper limit of sampling as ub;

将各粒子的各分量初始化为在[lb,ub]范围内均匀分布的随机向量；Set N particles to describe the diversity of the target state, and initialize the weights of all particles to be uniform

Initialize each component of each particle as a random vector uniformly distributed in the range [lb,ub];

Step 2: Sampling the background domain within the rectangular frame where the target is located and outside the rectangular frame where the target is located in the current frame to obtain a training set X of 2K samples;

In the rectangular frame where the target is located, the target area is sampled with the coordinates of the target center point as the expectation of the two-dimensional Gaussian distribution, and K sampling positions are obtained, and the target pixel features at the K sampling positions are used to form K in the training sample set Target class samples; take the center of the smallest adjacent ellipse of the target rectangle as the pole, establish polar coordinates as the polar axis parallel to the direction of the target width, and perform random sampling in the polar coordinates, and the angle of each sampling point is within [0,2π) A uniformly distributed random number, the polar diameter is a multiple of the polar diameter of the point on the ellipse at the same angle, the multiple is the sum of an exponentially distributed random number and a floating point number greater than 1, and K samples are obtained outside the target area, around The new sampling point above the background domain, the K background class samples in the training sample set are composed of the features of the background pixels at the new sampling point position; the training set X is obtained by sampling 2K samples;

Step 3: Neighborhood component analysis NCA is performed on the training set X of 2K samples, and a new linear space transfer matrix A _new is obtained by using the vector BFGS multivariate optimization algorithm. The 2K training samples are then obtained according to the new linear space transfer matrix A _New is transformed to obtain the transformed training sample set AX;

Step 4: Obtain the next frame of image to become the current frame, and form a test sample set from the vectors composed of pixel features of all positions in the frame image, and transform according to A _new to obtain the transformed test sample set S _new ; use the above The transformed training sample set AX in one frame classifies the transformed test sample set S _new in this frame, and obtains the classification probability p _post of each test sample, and takes the probability of belonging to the target class as the pixel value at the position of each test sample , then a target probability distribution map I _likelihood of a new grayscale image is obtained;

Step 5: On the target probability distribution map I _likelihood , calculate the multi-scale normalized Laplacian filter function centered at the position of each particle, and the value at the position of the particle; the maximum value is vmax, which is normalized by the multi-scale of each particle Laplacian filter function value, the confidence of the particle is calculated based on the normalized distance from the maximum value vmax;

Step 6: Update the particle filter, obtain the output state of the filter, and obtain the new position of the target represented by a rectangular box in the current frame image. If the tracking has not ended, go to step 2, otherwise stop.

The specific process of sampling on the background is as follows:

Generate a uniformly distributed random number on the [0,2π) interval as the angle α in polar coordinates;

其中β>1，w和h分别为目标的宽和高，β为控制采样点到目标边缘距离的参数；The random number χ of the exponential distribution with the generation rate parameter λ=0.5 is used to calculate the polar diameter

Where β>1, w and h are the width and height of the target respectively, and β is a parameter controlling the distance from the sampling point to the edge of the target;

处的像素的特征构成一个背景类样本，重复上述过程K次即可获得K个背景类样本。Get image coordinates in

The features of the pixels at are a background class sample, and K background class samples can be obtained by repeating the above process K times.

The specific process of calculating the probability p _post of each test sample classification is as follows:

The dimension of each sample is the set value n, then the i-th sample x _i is expressed as an n-dimensional row vector x _i =(x _i1 , _xi2 ,…,x _in ), and x _in is the n-th dimension component of the sample , with each sample as a row, the linearly transformed training samples from the target and the background form a 2K×n matrix AX, the first K rows come from the target, and the last K rows come from the background, construct a 2K×2 matrix G, each row The value corresponds to the category of the corresponding row of training samples, if the i-th sample belongs to the target, then the i-th behavior of G is (1,0), if the i-th sample belongs to the background, then the i-th behavior of G is (0,1) ;Similarly, the test sample set S _new constitutes a matrix of W×H×n, where W and H are the width and height of the image respectively; with AX as the training sample set and G as the supervision result, the transformed test sample set S _new classifies, obtains the category belonging of each sample in S _new , and the probability p _post of belonging to the target class, replaces the pixel value at each pixel position in the image corresponding to the test sample with the corresponding p _post , and generates the target probability Distribution map I _likelihood .

The specific calculation process of the multi-scale normalized Laplacian filter function is as follows:

(1) Gaussian kernel function T with continuous scale variable t and discrete space variable n: Z×R ₊ → R is T(n;t)=e ^-t I _n (t), where I _n (t) is the A class of modified Bessel functions whose second order differential It can be calculated by difference:

其中*表示一维离散信号卷积；

where * represents one-dimensional discrete signal convolution;

卷积，将结果矩阵的每一列再与T(n;t)卷积，以L_xx(x,y)表示第二次卷积的第一结果矩阵；类似地，将目标概率分布图I_likelihood矩阵的每一列与

卷积，将结果矩阵的每一行再与T(n;t)卷积，以L_yy(x,y)表示第二次卷积的第二结果矩阵。则多尺度规范化Laplacian滤波函数如下式在(x,y,t)处的取值为：(2) Given a scale variable t, combine each row of the target probability distribution graph I _likelihood matrix with

Convolution, each column of the result matrix is convolved with T(n;t), and L _xx (x, y) represents the first result matrix of the second convolution; similarly, the target probability distribution map I _likelihood Each column of the matrix with

Convolution, each row of the result matrix is convolved with T(n;t), and L _yy (x, y) represents the second result matrix of the second convolution. Then the value of the multi-scale normalized Laplacian filter function at (x, y, t) is as follows:

Laplacian(x,y,t)=(t(L _xx (x,y)+L _yy (x,y))) ² .

The calculation process of the particle confidence is as follows:

Obtain the maximum value vmax among the multi-scale normalized Laplacian filter function values of all particles, calculate the distance d between the multi-scale normalized Laplacian filter function values of each particle and vmax, and the variance var of all particle Laplacian filter function values, and obtain the value according to the following formula Particle confidence conf:

$\mathrm{<span\; class="notranslate">\; conf</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; e</span>}}^{\frac{{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="d"\; filenames="HDA0000368657550000081.png"\; state="\{\{state\}\}">d</figure-callout></span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; var</span>}}}<span\; class="notranslate">\; .</span>$

The acquisition of the new position of the target represented by a rectangular box in the current frame image, the specific process is as follows:

则目标的宽度由如下公式计算：The state of the filter output is

Then the width of the target is calculated by the following formula:

$\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; =</span>\frac{\sqrt{\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 13</span>}}{\mathrm{<span\; class="notranslate">\; 0.47619</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 34</span>}<span\; class="notranslate">\; ;</span>$

height is $h=\frac{w}{\mathrm{asr}};$ The coordinates of the upper left corner of the target are $(r=\stackrel{\&CenterDot;}{r}-\frac{\mathrm{<figure-callout\; id="2"\; label="h"\; filenames="HDA0000368657550000081.png"\; state="\{\{state\}\}">h</figure-callout>}}{2},c=\stackrel{\&CenterDot;}{c}-\frac{w}{2}).$

Beneficial effects of the present invention: the present invention uses feature transformation to obtain the optimal feature for distinguishing the target and the background, obtains the best classifier for distinguishing the target and background pixels in any frame image, and solves the problem of target features from the perspective of classifier design Update the problem; using the state diversity and convergence characteristics of the particle filter, while avoiding the algorithm from falling into the local optimum problem after the target is occluded, the tracking accuracy is guaranteed on the basis of the scale space theory. It avoids the danger of traditional algorithms such as mean shift being trapped in a local optimal state, and at the same time obtains the optimal estimated value of the target position and scale. The invention can more accurately locate the position and scale of the target, more effectively adapt to the problems of target illumination and color change, and simultaneously robustly process target occlusion.

Description of drawings

Fig. 1 is the work flowchart of tracking method of the present invention;

Figure 2 is the tracking effect of the Camshift method in the prior art when the color characteristics of the target change rapidly; where (a) is the second frame when the method starts to deviate from the real target; (b)-(f) are the The tracking effect diagram of each typical frame to the target;

Fig. 3 is the tracking effect figure of the present invention adaptively updating the best feature when the target color feature changes rapidly;

Figure 4 is the tracking effect of the L1APG method in the prior art when the brightness and scale of the target change at the same time; where (a) and (b) show that the method can track the glass bottle as the target in the first few frames The situation of ; (c)-(f) is the situation where the method produces a deviation in the direction of the target height when the surface brightness on the target changes;

Figure 5 is a tracking effect diagram of the present invention when the target brightness and scale change simultaneously, adaptively updating the best features and adjusting the scale; where (a) is the situation when the tracking starts; (b) (c) is when the target The situation that the algorithm of the present invention can still track accurately when the surface becomes dark and the size shrinks; (d)-(f) are the situations where the target surface changes from dark to bright and then dark, and the algorithm of the present invention can still accurately track it when the size first enlarges and then shrinks situation;

Fig. 6 is a target probability distribution diagram generated during the tracking process of the present invention; among them, (a)-(f) show the target probability distribution diagram corresponding to each frame image in Fig. 5;

Fig. 7 is a schematic diagram of the sampling results of the target and the background in the tracking process of the present invention; wherein the target area is inside the white square, and the background area is outside; the circle represents the sampling point of the target area, and x represents the sampling point of the background area;

Fig. 8 is the first two-dimensional projection diagram of each sample in the feature space before the neighborhood component analysis is performed on the training sample set in the example of the present invention; the circle represents the sample point from the target, and x represents the sample point from the background;

Fig. 9 is the first two-dimensional projection diagram of each sample in the transformed feature space after analyzing the neighborhood components of the training sample set in the example of the present invention, the circle represents the sample point from the target, and x represents the sample point from the background;

Fig. 10 is the comparison result of the target width accuracy between the embodiment of the present invention and L1APG in the prior art;

Fig. 11 is the comparison result of the target height accuracy between the embodiment of the present invention and L1APG in the prior art;

Figure 12 is a tracking effect diagram of the present invention when the target is partially and completely occluded; where (a) (b) (c) are the algorithm tracking situations before the first partial occlusion, during occlusion, and after occlusion respectively; (d )-(f) are the algorithm tracking situations before, during and after the second complete occlusion, respectively.

Detailed ways

In this embodiment, target tracking is performed on the "glass" video sequence. The design parameters are set as follows: the total number of particles N=800; the state vector dimension stateN=3, corresponding to the coordinates in the width and height directions of the particle location, and the scale parameter, the observation vector dimension measureN=3, and the target area sampling samples The number K=300, the maximum cycle times of neighborhood component analysis is maxIter=100 times.

In the process of tracking, the brightness and color of the target may change due to changes in the environment, and the characteristics of the target should be corrected in time to adapt to these changes while tracking. Therefore, the present invention uses Neighborhood Component Analysis (NCA) to calculate and obtain A linear transformation matrix, which projects the pixel features of the target and the background into a new space, in which the mahalanobis distance between the target class and the background class samples reaches the maximum, so that the two images trained on the transformed features A classifier whose classification error is smaller than that of the original feature. Transform each pixel feature in a new frame of image according to this transformation matrix, and classify it according to this classifier, and a better target probability distribution map can be obtained than that obtained by using the original feature. Targets appear as areas of higher gray values in this distribution plot, and the background appears as darker areas. In order to further improve the accuracy, a particle filter is introduced to estimate the state of the target. According to the Lindeberg scale space theory, a particle confidence calculation method based on the multi-scale normalized Laplacian filter function is proposed, which avoids traditional algorithms such as mean shift from being trapped in a local optimal state. , and at the same time obtain the optimal estimate of the target position and scale.

Compared with the prior art, the present invention proposes to use the feature transformation function of the Neighborhood Component Analysis (NCA) to obtain the optimal feature for distinguishing the target and the background, and obtain the best linear classification for distinguishing the target and background pixels in any frame image From the perspective of classifier design, it solves the update problem of target features; proposes a particle confidence calculation method based on multi-scale normalized Laplacian filter function, and uses the state diversity and convergence characteristics of particle filter to prevent the target from being occluded. While falling into the problem of local optimum, the accuracy of tracking is guaranteed on the basis of scale space theory. For the widely popular mean shift tracking algorithm and the newly proposed L1APG tracking algorithm in 2012 (Chenglong, B., et al. Realtime robust L1tracker using accelerated proximal gradient approach. in ComputerVision and Pattern Recognition (CVPR), 2012IEEE Conference on.2012 .), under the same experimental conditions, the present invention can more accurately locate the position and scale of the target, more effectively adapt to the problem of target illumination and color changes, and at the same time robustly deal with target occlusion.

As shown in Figure 1, the video target tracking method based on neighborhood component analysis and scale space theory in this embodiment includes the following steps:

Step 1: Determine the initial rectangular frame of the target by detection or manual labeling in the first frame, obtain the initial state of the target, and initialize the particle filter;

Initialize the target state: In the first frame of the sequence, determine the rectangular box where the target is located through the target detection method or manual labeling, and obtain the coordinates of the upper left corner of the target rectangle as (r,c), and the width and height of the rectangular box as (w,h ), then the target initial scale parameter s is obtained by calculating the following formula;

s=((13+(w-34)*0.47619)) ² (1)

记录目标的宽度与高度比例： $\mathrm{asr}=\frac{w}{h};$ Among them, the target center point is

Record the object's width-to-height ratio: $\mathrm{asr}=\frac{w}{h};$

Set the sampling lower limit of the initial state of each particle as lb=[r-3h,c-3w,0.1s], and the sampling upper limit as ub=[r+4h,c+4w,2s];

Step 2: Sampling the background domain within the rectangular frame where the target is located and outside the rectangular frame where the target is located in the current frame to obtain a training set X of 2K samples;

方差为 $\mathrm{\&sigma;}=\left[\begin{array}{cc}\frac{h}{3}& 0\\ 0& \frac{w}{3}\end{array}\right]$ 矩阵的二维高斯分布对图像像素位置进行采样，选取K=300个位于目标所在矩形框内的像素构成训练样本集中的目标类样本；以目标矩形的最小邻接椭圆的中心为极点，平行于目标宽度的方向为极轴建立极坐标，在极坐标中进行随机采样，每个采样点的角度为[0,2π)内均匀分布的随机数，极径为相同角度下椭圆上点的极径的倍数，该倍数为一个指数分布的随机数加上1.2，这样采样得到K=300个位于目标所在矩形框之外的，位于目标周围背景之上的像素位置，以它们构成背景类样本；Take the center position of the target rectangle as the expectation

Variance is $\mathrm{\&sigma;}=\left[\begin{array}{cc}\frac{h}{3}& 0\\ 0& \frac{w}{3}\end{array}\right]$ The two-dimensional Gaussian distribution of the matrix samples the pixel positions of the image, and selects K=300 pixels located in the rectangular frame of the target to form the target class samples in the training sample set; the center of the smallest adjacent ellipse of the target rectangle is the pole, parallel to the target The direction of the width establishes polar coordinates for the polar axis, and random sampling is performed in the polar coordinates. The angle of each sampling point is a random number uniformly distributed within [0,2π), and the polar diameter is the polar diameter of the point on the ellipse at the same angle. Multiple, the multiple is an exponentially distributed random number plus 1.2, so that K=300 pixel positions located outside the rectangular frame of the target and above the surrounding background of the target are obtained by sampling, and they constitute background samples;

Step 3: Neighborhood component analysis NCA is performed on the training set X of 2K samples, and a new linear space transfer matrix A _new is obtained by using the vector BFGS multivariate optimization algorithm. The 2K training samples are then obtained according to the new linear space transfer matrix A _New is transformed to obtain the transformed training sample set AX;

Neighborhood component analysis is performed on the training set X of 2K samples obtained in step 2, and the initial linear space transfer matrix is $A=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right],$ Use the vector BFGS (Broyden-Fletcher-Goldfarb-Shanno) multivariate optimization algorithm to solve A _new , set the maximum number of algorithm loops to maxIter=100 times, and the exit condition of the loop is to exit when the gradient norm is less than 10 ^-3 . If the optimization algorithm successfully returns and obtains the minimum value, the corresponding multivariate parameter will be used as A _new , otherwise set A _new = A; transform the 2K training samples according to A _new , and obtain the transformed training sample set AX = A _new ×X';

Step 4: Obtain the next frame of image and make it the current frame, and arrange the RGB three-components of each pixel in the frame of image in the order of pixel positions from left to right and from top to bottom to form a matrix S of W×H×3 , each of which is a three-component RGB component of a pixel, W is the image width, and H is the image height. Transform S according to A _new , and obtain the transformed test sample set S _new = A _new × S'; use AX obtained in step 3 as the training set, classify S _new according to target and background, and obtain S _new The probability p _post of each sample (corresponding to each pixel) belonging to the target, the original RGB component is replaced by the p _post value of each pixel, and the target probability distribution map I _likelihood with a size of W×H is obtained;

The specific process of calculating the probability p _post of each test sample classification is as follows:

Assuming that the dimension of each sample is n, the i-th sample x _i can be expressed as an n-dimensional row vector x _i =(x _i1 , _xi2 ,…,x _in ), where x _in is the n-th dimension component of the sample. Taking each sample as a row, the linearly transformed training samples from the target and the background can form a 2K×n matrix AX, the first K rows are from the target, and the last K rows are from the background. Construct a 2K×2 matrix G, the value of each row corresponds to the category of the training sample in the corresponding row, if the i-th sample belongs to the target, then the i-th row of G is (1,0), if the i-th sample belongs to the background, Then the i-th row of G is (0,1); similarly, the test sample set S _new constitutes a W×H×n matrix, where W and H are the width and height of the image, respectively. Taking AX as the training sample set and G as the supervision result, classify the transformed test sample set S _new , not only the category of each sample in S _new can be obtained, but also the probability p _post of its belonging to the target class can be obtained. Replace the pixel value at each pixel position in the image corresponding to the test sample with the corresponding p _post to generate the target probability distribution map I _likelihood ;

其中

计算其二阶微分

的离散模板：d2t_mask=t_mask*(1,-2,1)。Step 5: On the target probability distribution map I _likelihood , calculate the multi-scale normalized Laplacian filter function centered at the position of each particle, and the value at the position of the particle; the maximum value is vmax, which is normalized by the multi-scale of each particle Laplacian filter function value, calculate the confidence of the particle based on the normalized distance from the maximum value vmax; get the current state (x, y, t) of the particle, and calculate the T(n; t) at the scale t The discrete template size is The discrete template is

in

Calculate its second order differential

Discrete template for : d2t _mask =t _mask *(1,-2,1).

的离散模板d2t_mask及T(n;t)的离散模板t_mask卷积的第一结果矩阵L_xx(x,y)和第二结果矩阵L_yy(x,y)。Calculate the target distribution map I _likelihood in two directions at the scale where the particle is located.

The first result matrix L _xx (x, y) and the second result matrix L yy (x, y) of the convolution of the discrete template d2t _mask of T(n;t) and the discrete template t _mask of _T (n;t).

Then the multi-scale normalized Laplacian filter function value at the position of the particle can be calculated by formula 2: Laplacian(x,y,t)=(t(L _xx (x,y)+L _yy (x,y))) ² ( 2)

Obtain the maximum value vmax among the multi-scale normalized Laplacian filter function values of all particles, calculate the distance d between the multi-scale normalized Laplacian filter function values of each particle and vmax, and the variance var of all particle Laplacian filter function values, according to formula (3) Get the confidence conf of the particle:

$\mathrm{<span\; class="notranslate">\; conf</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; e</span>}}^{\frac{{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="d"\; filenames="HDA0000368657550000081.png"\; state="\{\{state\}\}">d</figure-callout></span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; var</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Step 6: Update the particle filter, obtain the output state of the filter, and obtain the new position of the target represented by a rectangular box in the current frame image. If the tracking has not ended, go to step 2, otherwise stop.

The specific process of obtaining the target rectangle is as follows:

The state of the filter output is Then the width of the target is calculated by the formula (4):

$\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; =</span>\frac{\sqrt{\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 13</span>}}{\mathrm{<span\; class="notranslate">\; 0.47619</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 34</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

height is $h=\frac{w}{\mathrm{asr}};$ The coordinates of the upper left corner of the target are $(r=\stackrel{\&Center\; Dot;}{r}-\frac{\mathrm{<figure-callout\; id="2"\; label="h"\; filenames="HDA0000368657550000081.png"\; state="\{\{state\}\}">h</figure-callout>}}{2},c=\stackrel{\&Center\; Dot;}{c}-\frac{w}{2}).$

According to the above steps, multiple color test sequences are tracked. During the tracking process, the color, illumination or size of the tracked targets have obvious changes, and there are occlusion problems.

Subfigures (a)–(f) in Figures 2, 3, 4, 5, and 12 depict the location and size changes of objects in chronological order.

As shown in Fig. 3, the tracking result of the "color-changing square" sequence obtained by using this embodiment, the tracked target is a square, and the tracking result of the algorithm is marked with a light-colored square. During the movement of the "color-changing square", the color rapidly changes from pure red to pure blue in the 60 frames before and after. Figure 2 shows a typical algorithm in the mean shift method - the situation where Camshift fails completely. Camshift cannot adapt to such a drastic change in the color characteristics of the target, as shown in the black ellipse in Figure 2, because the color of the square changes rapidly (cannot be seen on ordinary printers. The color printer can see this change), and the target will be lost soon, but the present embodiment uses the method of the present invention because it can find the best transformation feature to distinguish the target and the background in the current frame, so it is very good Adapting to this change in the target's color, the square is tracked accurately.

As shown in Fig. 5, the tracking result of the "glass bottle" sequence obtained in this embodiment is shown. The target to be tracked is a glass bottle, and the tracking result of the algorithm is marked with a white box. The glass bottle repeatedly enters and exits the shadow in the background during the tracking process, causing drastic changes in the brightness of its surface. At the same time, the size of the glass bottle in the image also changes significantly due to the change in distance from the camera. Although the newly published L1APG tracking algorithm in 2012 can track the glass bottle, it gradually produced large errors in the scale parameters, which eventually affected the positioning accuracy of the target, as shown in Figure 4. Fig. 10 and Fig. 11 respectively provide the real width of the target, the real height of the target, the target width and target height obtained in this embodiment, and the curves of the width and height obtained by the L1APG algorithm with the number of video frames. Calculated from the curve in the figure: the average error between the width obtained in this embodiment and the target real width is 4.474529, and the variance is 13.178290. The average error between the width obtained by the L1APG algorithm and the true width of the target is 8.821549, and the variance is 106.027808. The average error between the height obtained in this embodiment and the true height of the target is 1.009198, and the variance is 12.256838. However, the average error between L1APG and the true height of the target is 41.400282, and the variance is 689.699398, which is obviously greater than the result of this embodiment. It can be seen that the method of this embodiment can track the size change of the target very well, and at the same time adapt to the change of the target brightness caused by the illumination, and achieve higher target positioning accuracy. Fig. 6 shows the target probability distribution map corresponding to each frame image in Fig. 5 in the processing process of this embodiment. It can be seen that the neighborhood component analysis technology is used to highlight the area where the target is located, while suppressing the background close to the target periphery, in order to improve the positioning accuracy Conditions are provided. FIG. 7 is a schematic diagram of sampling the target and the background during the tracking process of this embodiment; the target area is inside the white square, and the background area is outside the square. The circles represent the sampling points of the target area, and the x represent the sampling points of the background area. It can be seen that the sampling points of the target are dense near the center of the target and scarce at the boundary of the target, which effectively reduces the possibility of interference caused by errors that may exist near the boundary. At the same time, the background sampling points are dense in the area around the target, and few are far away from the target, which ensures that the trained classifier has the best classification effect between the target and its immediate background. Fig. 8 is a schematic diagram of the projection of each sample in the first two dimensions of the feature space before performing neighborhood component analysis on the training sample set in this example. Circles represent sample points from the target, and x represent sample points from the background. It can be seen that because there is an area close to the target feature in the background, the two types of sample sets overlap and it is difficult to classify. FIG. 9 is a schematic diagram of the first two-dimensional projections of each sample in the transformed feature space after neighborhood component analysis is performed on the training sample set in this example. Circles represent sample points from the target, and x represent sample points from the background. It can be seen that the class distance between the two types of sample sets after the transformation is significantly larger, and it is easy to classify.

As shown in Fig. 12, the tracking result of a certain "cyclist" sequence obtained by using this embodiment, the target is marked with a white box. The target was occluded by the background interference twice successively, but each time after the occlusion ends, the method of this embodiment can continue to track the target. It can be seen that this method can effectively deal with the problem of target occlusion.

Claims (6)

1. based on the video target tracking method of neighbourhood's constituent analysis and metric space theory, it is characterized in that, this video target tracking method comprises the following steps:

Step 1: in the first frame, by the rectangle frame detected or manually mark is determined the initial place of target, obtain the original state of relevant target, and the initialization particle filter;

By object detection method or manual mark, determine the initial place of target rectangle frame, target rectangle upper left corner point coordinate is (r, c), and the wide and high of rectangle frame is (w, h), and target initial gauges parameter s is calculated acquisition by following formula:

s=((13+(w-34)*0.47619)) ²；

Wherein, target's center's point is

The width of record object and height ratio are: $\mathrm{asr}=\frac{w}{h};$

Under the sampling of the original state of target setting, be limited to lb, in sampling, be limited to ub;

Set N particle and describe the diversity of dbjective state, be initialized as the weights of all particles unified

Each component of each particle is initialized as to equally distributed random vector in [lb, ub] scope;

Step 2: to sampling in the rectangle frame at target place in present frame and on the background field of the outer rectangular frame at target place, obtain the training set X of 2K sample;

In the rectangle frame at target place, take target's center's point coordinate as the expectation that dimensional Gaussian distributes, sampled in target area, obtain K sampling location, with K the concentrated target class sample of object pixel feature composition training sample at place, K sampling location, the minimum of target rectangle of take is limit in abutting connection with oval center, the direction that is parallel to target width is that pole axis is set up polar coordinates, in polar coordinates, carry out stochastic sampling, the angle of each sampled point is [0, 2 π) equally distributed random number in, utmost point footpath is the multiple in the utmost point footpath of oval upper point under equal angular, this multiple is the random number of an exponential distribution and is greater than 1 floating number sum, sampling obtains K and is positioned at outside target area, new sampled point on background field on every side, feature with the background pixel at new sampling point position place forms K the background classes sample that training sample is concentrated, 2K sample by sampling obtains training set X,

Step 3: the training set X to 2K sample carries out neighbourhood's constituent analysis NCA, and uses vectorial BFGS multivariate optimized algorithm to solve to obtain new linear space shift-matrix A _new, 2K training sample is again according to obtaining new linear space shift-matrix A _newCarry out conversion, obtain the training sample set AX after conversion;

Step 4: obtain the next frame image, become present frame, the vector that the pixel characteristic of all positions in this two field picture is formed forms the test sample book collection, also according to A _newCarry out conversion, obtain the test sample book collection S after conversion _newTest sample book collection S after utilizing the training sample set AX after conversion in previous frame to conversion in this frame _newClassify, obtain the Probability p of each test sample book classification _Post, will belong to the pixel value of the probability of target class as each place, test sample book position, obtain the destination probability distribution plan I of the gray-scale map that a width is new _Likelihood

Step 5: at destination probability distribution plan I _LikelihoodUpper, calculate the Multi-scale normalized laplacian filter function centered by each place, particle position, the value at place, particle position; Wherein maximal value is vmax, with the Multi-scale normalized laplacian filter functional value of each particle, is the degree of confidence of this particle of basic calculation apart from the normalized distance of maximal value vmax;

Step 6: upgrade particle filter, obtain the output state of wave filter, obtain target means with rectangle frame in current frame image reposition, finish if follow the tracks of not yet, go to step 2, otherwise stop.

2. according to claim 1 based on the video target tracking method of neighbourhood's constituent analysis and metric space theory, it is characterized in that, the detailed process of described background up-sampling is as follows:

Generate [0,2 π) on interval equally distributed random number as the angle [alpha] in polar coordinates;

The random number χ of the exponential distribution of production rate parameter lambda=0.5, calculate utmost point footpath

β wherein>1, w and h are respectively the wide and high of target, and β is for controlling the parameter of sampled point to the object edge distance;

Obtain in image coordinate

The feature of the pixel at place forms a background classes sample, repeats said process and can obtain K background classes sample for K time.

3. according to claim 1 based on the video target tracking method of neighbourhood's constituent analysis and metric space theory, it is characterized in that the Probability p of described each test sample book classification _PostThe computation process detailed process is as follows:

The dimension of each sample is setting value n, i sample x _iBe expressed as n dimension row vector x _i=(x _I1, x _I2..., x _In), x _InFor the n dimension component of sample, using each sample as delegation, from the training sample through linear transformation of target and background, form the matrix A X of a 2K * n, front K is capable of target, rear K is capable of background, builds the matrix G of 2K * 2, and the value of each row is corresponding to the classification of corresponding line training sample, if i sample belongs to target, the i behavior (1,0) of G, if i sample belongs to background, the i behavior (0,1) of G; Similarly, test sample book collection S _newFormed the matrix of W * H * n, wherein W and H are respectively the wide and high of image; The AX of take is training sample set, and G is as the supervision result, to the test sample book collection S after conversion _newClassify; Obtain S _newIn the classification ownership of each sample, and the Probability p that belongs to target class _Post, the pixel value of each pixel position in the corresponding image of test sample book is replaced with to corresponding p _Post, generate destination probability distribution plan I _Likelihood.

4. according to claim 1 based on the video target tracking method of neighbourhood's constituent analysis and metric space theory, it is characterized in that, the concrete computation process of described Multi-scale normalized laplacian filter function is as follows:

(1) have continuous yardstick variable t, gaussian kernel function T:Z * R of discrete space variable n ₊→ R is T (n; T)=e ^-tI _n(t), I wherein _n(t) be first kind modified Bessel function, its second-order differential

Can pass through Difference Calculation:

Wherein * means one-dimensional discrete signal convolution;

(2) give dimensioning variable t, by destination probability distribution plan I _LikelihoodEvery delegation of matrix with Convolution, by matrix of consequence each row again with T (n; T) convolution, with L _Xx(x, y) means the first matrix of consequence of convolution for the second time; Similarly, by destination probability distribution plan I _LikelihoodMatrix each row with

Convolution, by every delegation of matrix of consequence again with T (n; T) convolution, with L _Yy(x, y) means the second matrix of consequence of convolution for the second time.The Multi-scale normalized laplacian filter function as shown in the formula the value of locating at (x, y, t) is:

Laplacian(x,y,t)=(t(L _xx(x,y)+L _yy(x,y))) ²。

5. according to claim 1 based on the video target tracking method of neighbourhood's constituent analysis and metric space theory, it is characterized in that, the computation process of described particle degree of confidence is as follows:

Obtain the maximal value vmax in all particle Multi-scale normalized laplacian filter functional values, calculate between each particle Multi-scale normalized laplacian filter functional value and vmax apart from d, and the variance var of all particle Laplacian filter function values, according to following formula, obtain the degree of confidence conf of this particle:

$\mathrm{conf}=e^{\frac{{-d}^2}{2\&times;\mathrm{var}}}.$

6. according to claim 1 based on the video target tracking method of neighbourhood's constituent analysis and metric space theory, it is characterized in that, the acquisition of described target means with rectangle frame in current frame image reposition, detailed process is as follows:

The state of wave filter output is

The width of target is calculated by following formula:

$w=\frac{\sqrt{s}-13}{0.47619}+34;$

Be highly $h=\frac{w}{\mathrm{asr}};$ Target upper left corner point coordinate is $(r=\stackrel{\&CenterDot;}{r}-\frac{h}{2},c=\stackrel{\&CenterDot;}{c}-\frac{w}{2}).$

Images