CN106651909A - Background weighting-based scale and orientation adaptive mean shift method - Google Patents

Abstract

The invention relates to a background weighting-based scale and orientation adaptive mean shift method. The method includes the following steps that: a) an initial frame is read, a target position y0 is initialized; b) a target model which is represented by a symbol described in the descriptions of the invention and a background feature model which is represented by a symbol described in the descriptions of the invention are calculated; c) a target candidate position is determined; d) a candidate target model which is represented by a symbol described in the descriptions of the invention and a background template are calculated; e) an image weight wi is calculated; f) a new target position y1 is calculated iteratively; g) whether a distance d between two frames of objects is smaller than 0.1 namely a set value zeta, is judged, or whether the number K of iterations is larger than a set maximum number N of iterations is judged, and whether background template similarity rho is smaller than a set zeta2 is judged, if the distance d between the two frames of objects is smaller than 0.1, or the number K of iterations is larger than the set maximum number N of iterations, and the background template similarity rho is smaller than the set zeta2, the method enters step h, otherwise, the method enters step c; h) the length, width, and dimension orientation of a target are estimated; i) the target position of a current frame is determined: the current target model which is represented by the symbol described in the descriptions of the invention and the background feature model represented by the symbol described in the descriptions of the invention are calculated; j) the background feature model is updated; and k) whether the current frame is the last frame is judged, if the current frame is the last frame, the method terminates, if the current frame is not the last frame, the method returns to step b.

Description

Scale-direction adaptive mean shift method based on background weighting

Technical Field

The invention relates to the field of target tracking, in particular to a method for tracking a moving target under a complex background.

Background

Target tracking is one of the core problems of machine vision as a one-field-crossing-disciplinary frontier technology. The target tracking technology has flourished since the sixties of the last century and a series of methods have been developed to date. Due to the diversity of the tracked target itself and the complexity of the external environment, the tracking of moving targets is a very challenging subject. A robust target tracking algorithm must be able to ideally solve various difficulties (such as direction rotation change, size change, illumination change, occlusion change, background similarity change, etc.) encountered in the tracking process, and a Mean Shift algorithm (MS for short) is widely applied in the field of target tracking due to its simple principle and high operation efficiency. The MS algorithm was first proposed by Fukunaga and hoststler et al in 1975 for data analysis, Cheng Y equals to the first use of the MS algorithm in 1995 to the field of pattern recognition, Bradski applies the MS algorithm to face tracking and proposes the CAMSHIFT algorithm, and comenicou and Meer et al successfully apply the MS algorithm to image segmentation and target tracking. The mean shift algorithm is a nonparametric method based on a color histogram, and the target position is found through iterative operation to realize target tracking.

In the traditional MS algorithm, the problem of the size direction of the target is not solved, and Jifeng ni in 2012 proposes a mean shift target algorithm (Scale and Orientation Adaptive mean shift Tracking, SOAMST for short) based on size direction adaptation. To solve the problem of tracking drift caused by the change of the size direction of a moving target, but the problem of Background similarity is easy to fall into local optimization, which causes the loss of the tracked target, and to solve the problem of tracking drift caused by the Background similarity, comiciu et al propose a Background Weighted color Histogram-based mean shift algorithm (Background-Weighted Histogram MS), which incorporates Background information into the Histogram, but does not achieve the practical effect. Jifeng proposes a Corrected Background weighting algorithm (CBWH-MS for short), which reduces tracking drift caused by Background complexity to some extent, but does not consider the problem of the size direction of the target.

The invention provides a moving target tracking algorithm under a complex background aiming at the defects, and provides a size direction self-adaptive mean shift tracking algorithm based on background weighting aiming at the tracking shift problem caused by background similarity, size change, occlusion and the like in the tracking process of the classical mean shift algorithm. The target color features are extracted by combining with background weighting, the spatial information of a video image sequence is fully utilized, the information features of a target area are highlighted, and the tracking drift phenomenon caused by background similarity and background blurring is inhibited. And a size direction self-adaptive covariance matrix estimation method is adopted to adapt to the real-time change of the size direction of the moving target, so that the tracking accuracy is ensured. Experiments prove that compared with other classical mean shift algorithms, the moving target tracking algorithm provided by the method is remarkably improved in precision and efficiency.

Disclosure of Invention

The invention aims to provide a size direction self-adaptive mean shift tracking method based on background weighting aiming at the defects of the prior art, which comprises the following steps: a) reading in an initial frame and aligning the target position y₀Initializing; b) computing object modelsAnd background feature modelc) Determining a target candidate position; d) computing candidate object modelsAnd a background template; e) calculating an image weight w_i(ii) a f) Iteratively calculating a new target position y₁G) judging whether d of the distance between two frame objects is less than 0.1 (namely a set value ξ) or whether the iteration number K is more than a set maximum iteration number N and whether the similarity of the rho background template is calculated to be less than a set ξ₂Whether or not it is true, ifIf yes, entering the step h, and if not, returning to the step c; h) estimating the length, the width and the size direction of the target; i) determining the target location of the current frame, including calculating a current target modelAnd background feature modelj) Updating the background feature model; k) and c, judging whether the frame is the last frame or not, if so, ending, otherwise, returning to the step b.

Preferably, the object modelThe calculation method comprises the following steps:

where u represents a histogram component and m is typically 8, 16, or 32; k (x) is a kernel function for weighting of pixels; n represents the number of pixels of the target window; b (xi) is the corresponding x_iA color histogram index of the pixel at (a); is a KroneckerDelta function; and C is a normalized constant coefficient.

Preferably, the candidate object modelThe calculation method comprises the following steps:

wherein,representing features u in a candidate object modelThe probability density of (d); h denotes the tracked bandwidth.

Preferably, the image weight w_iThe calculation method comprises the following steps:

preferably, said calculating a new target position y₁The method comprises the following steps:

preferably, the method for estimating the length, width and dimension direction of the target is as follows:

wherein A is pi × a × b and represents the area of the target region, a and b are the major and minor axes of the target region, and lambda₁、λ₂Is the matrix eigenvalue.

The invention discloses a method for tracking a moving target under a complex background. The size direction self-adaptive mean shift tracking algorithm based on background weighting is provided aiming at the problem of tracking shift caused by background similarity, size change, occlusion and the like in the tracking process of the classical mean shift algorithm. The target color features are extracted by combining with background weighting, the spatial information of a video image sequence is fully utilized, the information features of a target area are highlighted, and the tracking drift phenomenon caused by background similarity and background blurring is inhibited. And a size direction self-adaptive covariance matrix estimation method is adopted to adapt to the real-time change of the size direction of the moving target, so that the tracking accuracy is ensured. Experiments prove that compared with other classical mean shift algorithms, the moving target tracking algorithm provided by the method is remarkably improved in precision and efficiency.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

Drawings

Further objects, features and advantages of the present invention will become apparent from the following description of embodiments of the invention, with reference to the accompanying drawings, in which:

FIG. 1 shows a flow chart of a background weighting based scale-direction adaptive mean-shift algorithm according to the present invention;

fig. 2-4 show the center point position error images of four tracking algorithms in 3 different video sequences.

Detailed Description

The objects and functions of the present invention and methods for accomplishing the same will be apparent by reference to the exemplary embodiments. However, the present invention is not limited to the exemplary embodiments disclosed below; it can be implemented in different forms. The nature of the description is merely to assist those skilled in the relevant art in a comprehensive understanding of the specific details of the invention.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the drawings, the same reference numerals denote the same or similar parts, or the same or similar steps.

FIG. 1 is a flow chart of a background weighting-based scale-direction adaptive mean shift algorithm according to the present invention; the method comprises the following specific steps:

step 101: reading in an initial frame and aligning to a target position y₀Initializing;

step 102: computing object modelsAnd background feature model

According to one embodiment of the invention, the object modelThe calculation method comprises the following steps:

where u represents a histogram component and m is typically 8, 16, or 32; k (x) is a kernel function for weighting of pixels; n represents the number of pixels of the target window; b (xi) is the corresponding x_iA color histogram index of the pixel at (a); is a KroneckerDelta function; and C is a normalized constant coefficient.

Step 103: determining a target candidate position;

step 104: computing candidate object modelsAnd a background template;

according to one embodiment of the invention, the candidate object modelThe calculation method comprises the following steps:

wherein,representing features u in a candidate object modelThe probability density of (d); h denotes the tracked bandwidth.

Step 105: calculating an image weight w_i；

According to one embodiment of the invention, the image weight w_iThe calculation method comprises the following steps:

step 106: iteratively calculating a new target position y₁；

According to one embodiment of the invention, the new target position y is calculated₁The method comprises the following steps:

step 107: determining if two frames of objects are presentD of the distance between p and p is less than 0.1 (namely a set value ξ) or whether the iteration number K is more than a set maximum iteration number N and whether the calculated p background template similarity is less than a set value ξ₂If yes, go to step 108, if not, go back to step 103;

step 108: estimating the length, the width and the size direction of the target;

according to an embodiment of the present invention, the method for estimating the length, width and dimension direction of the target comprises:

wherein A is pi × a × b and represents the area of the target region, a and b are the major and minor axes of the target region, and lambda₁、λ₂Is the matrix eigenvalue.

Step 109: determining the target location of the current frame, including calculating a current target modelAnd background feature model

Step 110: updating the background feature model;

step 111: and judging whether the frame is the last frame, if so, ending, otherwise, returning to the step 102.

In order to solve the problems existing in the target tracking process of the MS algorithm, the background weighting is combined into the size direction self-adaptive MS algorithm. Experiments show that the size direction self-adaptive mean shift algorithm based on background weighting has strong robustness for target tracking in a complex scene.

In classical mean-shift target tracking, the representation of the target is usually represented by a rectangle or ellipse, and the target is represented by a triangle or ellipseIs based on a color object model, which can be interpreted as the probability density of all colors of the object region. Specifying a pixel of a normalized object model in a color-based object model asObject modelIs calculated as follows:

wherein: u represents a histogram component, m is usually 8, 16 or 32, and m is taken as 16; k (x) is a kernel function for weighting of pixels; n represents the number of pixels of the target window; b (xi) is the corresponding x_iA color histogram index of the pixel at (a); is the Kronecker Delta function; c is a normalized constant coefficient calculated as follows:

similar to the target model, the candidate model of the target is calculated as follows:

wherein,representing features u in a candidate object modelThe probability density of (d); h denotes the tracked bandwidth. And (3) calculating the similarity between the target model and the candidate target model by adopting the Bhattacharyya correlation coefficient, wherein the calculation is as follows:

and calculating similarity by using the Papanicolaou coefficient, and obtaining the target position through an iterative optimization process. Initialized first frame target position at y using Taylor's formula₀Unfolding:

wherein, w_iWeight coefficient for image:

target region estimated in an iterative process from y₀Move to y₁The specific calculation is as follows:

the MS algorithm of the formula (8) can find the most similar region to the target model in the new image sequence, namely, the target is tracked.

In a video sequence with a background similar to a target, a color feature model is directly adopted, and the tracking effect is poor. In order to enable the mean shift algorithm based on the color histogram to better distinguish a background area and a target area, reduce the interference of a flat area in an adjacent frame of an image and enhance the color characteristics of the flat area, the invention adopts a color characteristic model fusing background weighting to solve the problems. Setting the image area of the background weighting area three times as large as the area around the target window according to the empirical value, and then calculating the color histogram of the background area to be represented asWhereinSuppose thatA medium non-zero minimum value ofThe color histogram based on background weighting has the corresponding weight coefficients:

characteristics of the object modelThe probability density of (a) is:

object background feature modelCharacteristic model of current frame candidate area backgroundModel (III)Similarity calculation is carried out, and the expression is as follows:

if rho <, the background change is larger, the background model is updated, namely the background model under the current frame is assigned to the target background model, so that the problem of target tracking loss caused by severe background change is solved.

Aiming at the problem of the size direction in the tracking process, Jifeng Ning et al provides a method combining an image density distribution function and an image weight matrix, and improves the anti-interference capability of an MS algorithm on the size direction of a tracked target. Calculating the current matrix characteristics of the target area, estimating the length, width and direction of the target, decomposing the covariance matrix by using a singular value decomposition matrix, and calculating as follows:

let a and b be the major and minor axes of the target region, and the ratio relationship between λ and b is known₁/λ₂A/b, so k can be assumed to be a scale factor, a k × λ₁，b＝k×λ₂This is expressed as follows:

where a ═ pi × a × b denotes the target region area, and the new covariance matrix is as follows:

with the above change, once the position is determined, the size direction of the target region is estimated, and the covariance matrix of the candidate target region is defined as follows:

where Δ d represents the increment of the target candidate region of the next frame, the position of the initialization target candidate region is calculated as follows:

aiming at the problem of tracking drift caused by similar change of a background, change of a size direction, change of shielding and the like in the tracking process of a moving target, the background weighting operator is fused into an SOAMS algorithm, the target background is processed by using the background weighting operator, the spatial information of an image sequence is utilized, the information characteristic of a target area is highlighted, the interference of a flat area in an image is reduced, the color characteristic of the flat area is enhanced, and the tracking drift phenomenon caused by similar background and fuzzy background is reduced. Target model features fusing background weighting operatorsIs (10).

In the original SOAMS algorithm, although the size direction change of the moving object can be reflected, the following problems exist:

1) when the background of the target is similar or background blurring exists, the tracking precision is reduced, and when the tracking is serious, the tracking is lost.

2) When the moving position of the target between adjacent frames of the moving target is large, the traditional SOAMS algorithm causes an increase in the calculation amount of the tracking process or loss of tracking due to the condition limitations of real-time performance and iteration times when the target is tracked.

3) In the original SOAMS algorithm, a long-axis mode is adopted in the traversal process of the target back selection area, so that the calculated amount of the algorithm is redundant when the target is too large in the tracking process, and the real-time requirement of target tracking cannot be met. The invention provides an improvement aiming at the problems of the original SOAMS algorithm.

The invention provides an improvement aiming at the problems of the original SOAMS algorithm.

For problem 1), the present invention fuses the background weighting operator into the SOAMS algorithm, as already mentioned above.

For the problem 2), in the conventional SOAMS algorithm, the extended range of the target candidate area is achieved by adding a Δ d, where the Δ d represents the increment of the target candidate area of the next frame, and the threshold value is generally 5 to 15. The main reason for this kind of problem is that the position moving distance of the target in the adjacent frame is not fixed during the target tracking process. When the position of the moving target exceeds the range of the candidate target area expanded by the fixed threshold when the position of the target moves greatly in the adjacent frame of the target in the target tracking process by the SOAMS algorithm, the iteration times of the SOAMS algorithm are increased, and if the target cannot be accurately tracked within the limited iteration times, the current similar coefficient is mistakenly found at the large position according to a calculation formula of the bus coefficient to serve as the target, so that the tracking loss is caused. The invention adaptively sets the threshold value by calculating the absolute displacement of the target candidate area and the central point of the target area of the previous frame as delta d. The modified Δ d is calculated as:

wherein (X)₀，Y₀) Is the central point position coordinate of the current frame target candidate area, and (Xcen, Ycen) is the central point position coordinate of the previous frame target area. The self-adaptive threshold value is obtained through the fact change of the target area, the absolute distance of the movement of the target position between adjacent frames is used as the increment of the target post-selection area, the displacement change of the moving target can be better met, particularly when the movement of the target between adjacent frames of the moving target is absolutely, only and only increased suddenly, the algorithm can track the target more quickly and accurately, and the tracking drift phenomenon caused by sudden and violent position change of the target is reduced.

Aiming at the problem 3), a candidate region model and a search region tracked by the original SOAMS algorithm target are optimized, the calculation process of the target candidate region is simplified, namely, the traversal range of the target candidate region is changed into a long axis and a short axis on the basis of the original adoption of a long axis, and the area expanded by the long axis is pi × (a + delta d)²The area of the target candidate region using the major and minor axes is pi ((a + Δ d) × (b + Δ d), and the calculation interval is reduced by pi × ((a-b) × (a + Δ d) in comparison>b) The calculation amount of the algorithm is reduced. The invention adopts the expanded target area to corrode to obtain the target area, and adopts the ellipse model to represent the target area of interest, so that the original traversing mode of adopting a long shaft is changed into the mode of adopting a long shaft and a short shaft to combine, the information of effective data cannot be reduced, the accuracy of reflecting the size direction of the target is improved, and meanwhile, the running efficiency of the algorithm is improved due to the reduction of the calculated amount of redundant pixels.

The iteration times of four algorithms of different video sequences are shown in table 1, and the comparison shows that the operation iteration times of the algorithm is relatively optimal.

TABLE 1 average number of iterations per frame (frame/number) for the four algorithms

Table 2 shows the average run time per frame for the four algorithms, which is reduced by more than a factor of two by comparing the run speed of the inventive algorithm with the original SAOMS algorithm.

TABLE 2 average run time per frame (unit frame/ms) for the four algorithms

Compared with the prior art, the size direction self-adaptive mean shift algorithm based on background weighting has the beneficial effects that:

(1) the size direction estimation of the improved covariance matrix can well cope with the size direction change of the moving target;

(2) meanwhile, a color model based on background characteristic weighting is fused, so that the influence of a complex background on tracking can be reduced, and the tracking precision and efficiency are improved. Compared with the traditional MS tracking algorithm, the MS tracking algorithm based on the simple background weighting (CBWH-MS) and the size direction change (SOAMST), the algorithm has higher efficiency and better robustness in processing the scene sequence with complex and completely shielded background.

Fig. 2-4 show the center point position error of four tracking algorithms under 3 different video sequences, and the tracking accuracy is more accurate by comparing the algorithms of the invention. 2-4, the improved scale-direction adaptive mean shift algorithm based on background weighting is higher in tracking accuracy compared with other three target tracking algorithms. And the robustness of the algorithm provided by the invention in the target tracking process is more stable.

Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.

Claims (6)

1. A scale direction self-adaptive mean shift method based on background weighting is characterized by comprising the following steps:

a) reading in an initial frame and aligning to a target position y₀Initializing;

b) computing object modelsAnd background feature model

c) Determining a target candidate position;

d) computing candidate object modelsAnd a background template;

e) calculating an image weight w_i；

f) Iteratively calculating a new target position y₁；

g) Judging and judging whether d of the distance between the two frame objects is smaller than a set value ξ or whether the iteration number K is larger than a set maximum iteration number N and whether the similarity of the calculated rho background template is smaller than a set ξ₂If yes, entering step h, and if not, returning to step c;

h) estimating the length, the width and the size direction of the target;

i) determining the target location of the current frame, including calculating a current target modelAnd background feature model

j) Updating the background feature model;

k) and c, judging whether the frame is the last frame or not, if so, ending, otherwise, returning to the step b.

2. The method of claim 1, wherein: the object modelThe calculation method comprises the following steps:

$\hat{q}={\left\{{\hat{q}}_u\right\}}_{u=1,\mathrm{...},m}$

$\widehat{q_u}=C\underset{i=1}{\overset{n}{\&Sigma;}}k\left(\right|\left|x_i^{*}\right|{|}^2)\mathrm{\&delta;}\&lsqb;\left(x_i^{*}\right)-u\&rsqb;$

where u represents a histogram component and m is typically 8, 16, or 32; k (x) is a kernel function for weighting of pixels; n represents the number of pixels of the target window; b (xi) is the corresponding x_iA color histogram index of the pixel at (a); is the Kronecker Delta function; and C is a normalized constant coefficient.

3. The method of claim 1, wherein: the candidate object modelThe calculation method comprises the following steps:

$\hat{p}\left(y\right)={\left\{\widehat{p_u}\left(y\right)\right\}}_{u=1,\mathrm{...},m}$

${\hat{p}}_u\left(y\right)=C_h\underset{i=1}{\overset{n_h}{\&Sigma;}}k\left(\right|\left|\frac{y-x_i}{h}\right|{|}^2)\mathrm{\&delta;}\&lsqb;b\left(x_i\right)-u\&rsqb;$

wherein,representing features u in a candidate object modelThe probability density of (d); h denotes the tracked bandwidth.

4. The method of claim 1, wherein: the image weight w_iThe calculation method comprises the following steps:

$w_i=\underset{u=1}{\overset{m}{\&Sigma;}}\sqrt{\frac{\widehat{q_u}}{{\hat{p}}_u\left(y_0\right)}}\mathrm{\&delta;}\&lsqb;b\left(x_i\right)-u\&rsqb;$

5. the method of claim 1, wherein: said calculating a new target position y₁The method comprises the following steps:

$y_1=\frac{i={\&Sigma;}_{i=1}^{n_h}{x}_i{w}_{i}g\left(\right|\left|\frac{y-x_i}{h}\right|{|}^2)}{{\&Sigma;}_{i=1}^{n_h}{w}_{i}g\left(\right|\left|\frac{y-x_i}{h}\right|{|}^2)}$

6. the method of claim 1, wherein: the method for estimating the length, the width and the size direction of the target comprises the following steps:

$\begin{array}{cc}a=\sqrt{\frac{{\mathrm{\&lambda;}}_{1}A}{\left({\mathrm{\&pi;\&lambda;}}_2\right)}}& b=\sqrt{\frac{{\mathrm{\&lambda;}}_{2}A}{\left({\mathrm{\&pi;\&lambda;}}_1\right)}}\end{array}$

wherein A is pi × a × b and represents the area of the target region, a and b are the major and minor axes of the target region, and lambda₁、λ₂Is the matrix eigenvalue.