CN101226592A - Part-Based Object Tracking Method - Google Patents

Abstract

A component-based object tracking method in the field of image processing technology, comprising the following steps: firstly, for the tracking object that appears, the tracking component of the tracking object is positioned by using the method of accelerated area corner detection, and then the tracking is described by a grayscale histogram The component is tracked by the Kalman filter in subsequent frames, and the parameters of the Kalman filter are corrected by measuring the observed value in each frame, and the component is updated, and finally the tracked object is identified. The present invention can track the target more accurately and has a very fast tracking speed. The present invention uses a small window centered on the corner of the object as the tracking component, which can effectively overcome problems such as occlusion.

Description

Part-Based Object Tracking Method

technical field

The invention relates to a method in the technical field of image processing, in particular to a component-based object tracking method.

Background technique

Object tracking is an important research field in computer vision, and it has a wide range of applications in the fields of video surveillance, image compression and 3D reconstruction. Moving objects will inevitably encounter the problem of occlusion in the process of movement. Occlusion is the main factor affecting the stability of tracking. Overcoming occlusion is one of the difficulties of tracking algorithms.

After searching the prior art documents, it was found that Robust Fragments-based Tracking using the Integral Histogram published by Amit Adam et al. in Computer Vision and Pattern Recognition (January 2006: 798-805). Histogram tracking) proposes to divide the tracking window into several sub-windows, and compensate the blocked sub-window histogram, but its disadvantage is that the matching of two windows by using the poor search method reduces the real-time performance of tracking, and the result cannot reflect the tracking The angle of motion of the object.

Contents of the invention

The purpose of the present invention is to overcome the above-mentioned deficiencies in the prior art, and propose a component-based tracking method, which uses multiple components in the target as tracking objects, and uses kernel-based gray histograms to describe the tracking objects. For each component, the parameters of the component are predicted by the Kalman filter, and then the kernel-based gray histogram is used for correction to complete the tracking, which not only effectively overcomes the occlusion problem, but also overcomes the problems of relative motion and non-rigid body deformation inside the object , with good real-time performance and good tracking effect.

The present invention is achieved through the following technical solutions, comprising the steps of:

First, for the tracking object that appears, use the method of FAST (acceleration area) corner detection to locate the tracking part of the tracking object;

Then describe the tracking component through the gray histogram, track the tracking component through the Kalman filter in subsequent frames, and modify the parameters of the Kalman filter through the measurement of the observed value in each frame, and update the component;

Finally, the tracked object is identified.

The described positioning of the tracking component of the tracking object refers to: adopting the method of FAST corner point detection to detect the corner point in the moving object, using the rectangular window as the center of the corner point as the tracking component, the method of FAST corner point detection is specifically as follows : To check the circle around the point c to be detected, find the longest arc among them, if the gray value of all the points in the arc is greater than the gray value of point c by more than t gray values, t is determined by the user according to the needs set, or both are less than the gray value of point c by more than t gray values, it will be judged as a corner point.

The description of the tracking component through the gray histogram refers to: using the kernel-based gray histogram to describe the tracking component, the histogram is an n-dimensional vector, and n is set by the user according to the needs. First, the color space is mapped by 256 dimensions To the n dimension, and then use the Biweight (double weight) kernel function to weight each point, so that the weight of the pixel farther from the center is smaller, which can reduce the influence of background noise and improve the stability of the histogram.

The tracking part is tracked by Kalman filtering, specifically: Kalman filtering includes two parts: prediction and correction: the prediction part adopts prediction equations, and uses the state value and prediction error of the previous moment to make a prediction, and obtains each tracking For the position of the component at the current moment, there will be some errors in the prediction results. The modification part uses the correction equations, and uses the obtained observations at the current moment to correct the prediction results.

The parameters of the Kalman filter are corrected by the measurement of the observed values, which refers to: using a spiral search method around the position of the tracking component predicted by the prediction equations to find a point, and the histogram of the window with this point as the neighborhood The Euclidean distance between the graph and the histogram of the original part is less than the set threshold, and the position of this point is taken as the observation value of the current frame. The correction equations use this observation value to correct the current prediction value of the Kalman filter, and the corrected The state estimate and noise variance estimate of .

The update of the components is specifically: if no qualified point can be found around the predicted point, use the judgment method to mark the object, and keep the components located in the marked area, that is, keep the components caused by the rotation of the object or the occlusion. Parts that disappear but are still within the tracking object, eliminate the parts that are outside the tracking object due to tracking failure, and add the gray histogram of the original part to the gray histogram of the current position of the part. Update the grayscale histogram.

Identifying the tracking object refers to: after determining each component of the tracking object, use the rectangle with the smallest area that can contain the center points of all tracking components to identify the tracking object, so that each component can be unified into one object, specifically: Use the Graham method to determine the convex hull of the corner point set of the object. After obtaining the convex hull of the point set, use the extension line of a side on the convex hull as one side of the rectangle to find the rectangle containing all the points in the point set and rotate them in turn. Find the rectangular region with the smallest area.

Compared with the prior art, the present invention has the following beneficial effects: the present invention can track the target more accurately and has a very fast tracking speed, and uses a small window centered on the corner of the object as the tracking component, which can effectively overcome problems such as occlusion . Object description uses a kernel-based grayscale histogram to adapt to object size changes and rotations, and to reduce the impact of background noise. Predicting the parameters of the component in the next frame through Kalman filtering, and using the histogram to correct the filter parameters ensure the accuracy of component tracking. Finally, the Graham method is used to determine the rectangle where the center of the object is located, so that the inclination and size of the rectangle are more in line with the shape characteristics of the target. Moreover, the present invention has good real-time performance and stable tracking effect, and can effectively overcome problems such as occlusion and non-uniform speed among objects.

Description of drawings

Fig. 1 is the schematic diagram of FAST corner detection method in the embodiment of the present invention;

The flow chart of Kalman filtering among Fig. 2 the present invention;

Fig. 3 is a schematic diagram of determining the minimum rectangular area in the present invention;

Fig. 4 is the initial position of parts in the embodiment experiment of the present invention and the position figure in subsequent frame;

Fig. 5 is the result figure of experiment one in the embodiment of the present invention;

Fig. 6 is the result figure of experiment two in the embodiment of the present invention;

Fig. 7 is the result diagram of occlusion experiment in the embodiment of the present invention;

Fig. 8 is a comparison diagram of the effect of the present invention and the mean translation tracking method.

Detailed ways

The embodiments of the present invention are described in detail below in conjunction with the accompanying drawings: this embodiment is implemented on the premise of the technical solution of the present invention, and detailed implementation methods and specific operating procedures are provided, but the protection scope of the present invention is not limited to the following the described embodiment.

What this embodiment adopts is to track the moving object in a section of video sequence, comprise following specific steps:

1. Part positioning: The tracking part in this embodiment selects a rectangular area centered on the corner point in the tracking object, and the corner point detection adopts the FAST corner point detection method. FAST is a simple and intuitive corner point detection method, which can ensure Real-time requirements of the tracking system. The corner point detected by the FAST corner point detection method is that there are enough points around the point in a different area from the point, specifically: check the circle around the point c to be detected, and find the longest arc among them. If the gray values of all points are greater than the gray value of point c by more than t gray values (t=15), or are all smaller than the gray value of point c by more than t gray values, they are determined as corner points.

As shown in Figure 1, if the gray values of two opposite points on the circumference (such as pixel points 1 and 9) are similar to the gray value of point c, then it is obviously not necessary to detect 12 points to determine whether c is an angle or not. point, so the above method can be optimized, first detect pixel points 1, 9, and then detect 5 and 13. Through experimental statistics, after optimization, for a point to be detected in the image, it only needs to detect 3.8 points around it on average to determine whether it is a corner point.

来描述跟踪部件，假设{x_i ^*}_i＝1...m表示部件中心角点c周围以半径为r的圆内所有像素相对于c的归一化坐标，将目标颜色的分布由原来的256维化为n维，本实施例中n设定为16，由函数b(x)将位置为x的像素映射到它所在的n维颜色空间中的一维，则灰度直方图

的计算公式为：2. Use the kernel-based grayscale histogram vector

To describe the tracking component, assuming that {x _i ^* } _i=1...m represents the normalized coordinates of all pixels in a circle with radius r around the central corner point c of the component relative to c, and the distribution of the target color is changed from the original The 256 dimensions of are converted into n dimensions. In this embodiment, n is set to 16, and the pixel at position x is mapped to one dimension in its n-dimensional color space by function b(x), then the grayscale histogram

The calculation formula is:

$\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; (</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>}$

Wherein: b(x)=I(x)/n, I(x) expression position is the gray value of the pixel of x; k(x) is kernel function, in the present embodiment, adopts Biweight kernel function, its expression Formula is $k\left(x\right)=\left\{\begin{array}{cc}\frac{15}{16}{(1-x^2)}^2& \mathrm{ifx}<1\\ 0& \mathrm{otherwise}\end{array}\right.,$ The weight of pixels farther from the center is smaller, the influence of background noise is reduced, and the stability of the histogram is improved.

3. As shown in Figure 2, the Kalman filter is used to track the tracking components, and the tracking step of the Kalman filter is divided into two parts: prediction and correction:

The prediction part is: the prediction equation group uses the state value and prediction error at the previous moment to obtain the prediction value at the current moment, and the prediction equation group is:

x _k '=Ax _k-1 +Bu _k-1 (B=0 in this embodiment)

P _k '=AP _k-1 A ^T +Q

Among them: x _k ′ is the predicted state variable at time k, x _k-1 is the state variable at time k-1, A is the state transition matrix, P _k ′ is the prediction error correlation matrix at time k, and P _k-1 is k- Correction value of the error correlation matrix at time 1, Q is the motion noise correlation matrix.

The correction part is: after obtaining the predicted value and prediction-related error, the correction equation group corrects the predicted value and prediction-related error through the observation value at the current moment. The correction equation group is:

K _k ＝P _k ′H ^T (HP _k ′H ^T +R) ^-1

x _k ＝x _k ′+K _k (z _k -Hx _k ′)

P _k ＝(IK _k H)P _k ′

Among them: K _k is the Kalman filter gain matrix, H is the measurement matrix, R is the measurement noise correlation matrix, z _k is the observed variable, P _k is the correction value of the error correlation matrix at k time, x _k is the state variable at k time.

The correction equations modify the current forecast value through the observed value z _k , and obtain the corrected state estimated value and noise variance estimated value.

In this embodiment, the observed value is set to track the position of the center of the component, the state variable is set to the position, velocity and acceleration of the center of the component, and assuming that the component moves at a uniform acceleration, the parameters are set as follows:

z=[s _x , s _y ] ^T , x=[s _x , v _x , a _x , s _y , v _y , a _y ] ^T , (z is the observed value, subscript x, y represent the abscissa and ordinate coordinate)

state transition matrix $A=\left[\begin{array}{cccccc}1& 1& 0.5& 0& 0& 0\\ 0& 1& 1& 0& 0& 0\\ 0& 0& 1& 0& 0& 0\\ 0& 0& 0& 1& 1& 0.5\\ 0& 0& 0& 0& 1& 1\\ 0& 0& 0& 0& 0& 1\end{array}\right],$

measurement matrix $h=\left[\begin{array}{cccccc}1& 0& 0& 0& 0& 0\\ 0& 0& 0& 1& 0& 0\end{array}\right]$

The parameter P _k , the initial value of the motion noise correlation matrix Q and the measurement noise correlation matrix R needs to be determined by using prior knowledge. In this embodiment, it is assumed that the observed value falls into any position in the component window with equal probability. Set the measurement noise The initial value of the correlation matrix R is:

$\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{cc}{<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 2</span>}}{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& {<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 2</span>}}{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right]$

N _x , N _y are the number of pixels on the horizontal axis and vertical axis of the component window respectively.

4. Measurement of observed values and update of components: take the spiral search method in the small neighborhood centered on the predicted displacement coordinates (s _x ′, s _y ′), quickly find the coordinates of the observed values, and use the spiral search To the first coordinate that meets ρ[p _k , q _k-1 ]<l as z _k , where ρ[p _k , q _k-1 ] is the distance between the histogram of the current coordinate and the object color model, and l is the distance threshold.

If no qualified point can be found around the predicted point, it is mainly due to the following three reasons: (1) The component has failed to track, that is, the component is already outside the tracking object; (2) The component disappears due to object rotation, etc. But the tracking window is still within the tracking object; (3) The component disappears due to occlusion. In order to ensure the stability of tracking and a sufficient number of components, it is necessary to eliminate the components in the first case and retain the latter two components. The judging method adopted in this embodiment is to reserve the parts located in the marked area after the object is marked, otherwise, eliminate them.

In addition, because the tracking object is constantly changing, and there are problems such as illumination changes, noise, and occlusion in the image, its color model needs to be updated. In this embodiment, the histogram of the observed value of the current frame is weighted to update the color model, the update formula is as follows: q _k = (1-α)q _k-1 + αp _k , where: q _k , q _k-1 are the kernel-based gray histograms of frames k and k-1 respectively, and α is the update factor, which is set to 0.02 in this embodiment.

5. Identifying objects: After determining each component of the tracking object, use the rectangle with the smallest area that can contain the center points of all tracking components to identify the tracking object, so that each component can be unified into one object. Since there are abundant corner points on the edge of the object, This method can more accurately identify the size, position and inclination of the object.

In order to find the rectangle with the smallest area that can contain the center points of all tracking components, the convex hull of the corner point set must be determined first. This embodiment adopts the Graham method to determine the convex hull of the object corner point set. The convex hull of the plane point set is defined as the smallest convex set containing the point set, that is, a convex polygon with some points in the point set as vertices. Any side of the polygon, all points in the point set that are not on the side are on the same side of the side.

The Graham method (Zhou Peide. "Computational Geometry-Algorithm Design and Analysis (Second Edition)". Tsinghua University Press. 2005.), specifically:

①Set the point with the smallest coordinates in the convex set as p ₁ , connect p ₁ with other points in the convex set with line segments, and calculate the angle between these line segments and the horizontal line, and then sort according to the size of the included angle and the distance to p ₁ , to obtain A sequence p ₁ , p ₂ ,...p _n , point p ₁ is the vertex of the convex hull boundary, p ₂ and p _n must also be;

② Determine whether it is a point of a convex hull, and delete the points in p ₃ , p ₄ , ..., p _n-1 that are not on the convex hull, as follows:

[1] Set k=4;

[2] set j=2;

[3] If p ₁ and p _k are respectively on both sides of the line segment p _k-j+1 p _kj , p _k-j+1 is deleted, and the subsequent vertex number is reduced by 1, k=k-1, j=j-1, n=n-1; otherwise p _k-j+1 is temporarily the vertex of the convex hull and recorded;

[4] j=j+1, execute [3] until j=k-2;

[5] k=k+1, execute [2] until k=n.

Among them: use vector cross product to judge whether two points are on both sides of a certain line segment or the same side, ratio: there are two points P, Q, line segment AB, calculate the cross product PA×PB, QA×QB, if the same number means the same line segment side, otherwise on the opposite side.

③Sequentially output convex hull vertices;

After obtaining the convex hull of the point set, take the extension line of a side on the convex hull as one side of the rectangle, find the rectangle containing all the points in the point set, and rotate in turn to find the smallest area of the rectangle.

As shown in Figure 3, assuming that the determined convex hull vertex is ABCDE, the area of the rectangle with AB as the side is:

Similarly, the area of the rectangle with BC, CD, DE, and EA as sides can be calculated, and the rectangle with the smallest area can be selected to mark the tracking object.

As shown in Figures 5, 6, and 7, they are three experimental result diagrams of this embodiment. As can be seen from the figures, in the video sequence, there is a large amount of noise caused by illumination changes and camera shakes. The tracking method can still accurately describe the position, size and rotation angle of the object. In the process of the target from far to near and turning, there are still stable tracking results.

As shown in Figure 4, Figure (a) is the initial position of the parts of Experiment 1 and Experiment 2, and Figure (b) is the position in subsequent frames;

As shown in Figure 5, it is the result of Experiment 1, (a) (b) (c) (d) are the images of 185 frames, 259 frames, 574 frames, and 704 frames respectively, and the 574th frame in (c) A person stepped out of the car, and the person has moved away from the tracked vehicle in frame 704 in (d), which proves that the method of this embodiment can overcome occlusion and recover from occlusion.

As shown in Figure 6, it is the result of Experiment 2. Figures (a), (b), (c) and (d) are images of 38 frames, 91 frames, 118 frames, and 171 frames respectively. The background trees in the figure swing with the wind. And the speed of each part of the tracking character's body is not consistent. It can be seen that this method can overcome the problems of complex changes in the background and inconsistent internal speed of the tracking object.

As shown in Figure 7, it is an occlusion experiment. In the experiment, an occlusion area with a pixel value of 250 is artificially added. Figures (a) and (b) are images of 230 frames and 260 frames respectively. From the experimental results, it can be seen that the method of this embodiment has Stable anti-occlusion ability.

As shown in Table 1, comparing the speed of the method of this embodiment with the particle filter tracking method, the speed of the method of this embodiment is greatly improved, which provides real-time guarantee for further identification and analysis of targets.

Table 1 is the speed comparison between the method of this embodiment and the particle filter tracking method;

tracking method The method of this embodiment particle filter Performance parameters Number of frames Number of parts/frame ms/frame ms/frame Experiment 1 1302 38 3.04 62.3 Experiment 2 150 51 2.92 59.7

As shown in Figure 8, figure (a), (b) is the effect figure of embodiment method, figure (c), (d) is the effect figure of Mean-Shift (mean shift) tracking method, present embodiment method and mean value Compared with the translation method, this method can locate the target more accurately, change the size and rotation angle of the tracking window more flexibly, and has higher stability.

Claims (8)

1. object tracking based on parts, it is characterized in that, comprise the steps: at first tracing object to occurring, use the acceleration region angular-point detection method that the tracking unit of tracing object is positioned, describe tracking unit by grey level histogram then, in follow-up frame, tracking unit is followed the tracks of, in every frame, revise the parameter of Kalman filtering by the measurement of observed reading by Kalman filtering, and carry out the renewal of parts, at last tracing object is identified out.

2. the object tracking based on parts according to claim 1, it is characterized in that, described tracking unit to tracing object positions, be meant: adopt the angle point in the acceleration region angular-point detection method detection motion object, the rectangular window that will be the center with the angle point is as tracking unit, the method of acceleration region Corner Detection is specially: for checking measuring point c to be checked circle on every side, seek wherein the longest circular arc, if gray-scale value t that the gray-scale value of all points is all ordered greater than c in the circular arc more than the gray-scale value, t is set as required by the user, gray-scale value t that perhaps all orders less than c more than the gray-scale value, then be judged as angle point.

3. the object tracking based on parts according to claim 1, it is characterized in that, describedly tracking unit is described by grey level histogram, be meant: adopt based on the grey level histogram of nuclear and describe tracking unit, histogram is a n-dimensional vector, and n is set as required by the user, at first color space is mapped to the n dimension by 256 dimensions, adopt two weight kernel functions that each is put weighting then, make the weights of distance center pixel far away less.

4. the object tracking based on parts according to claim 1, it is characterized in that, describedly tracking unit is followed the tracks of by Kalman filtering, be specially: Kalman filtering comprises prediction and revises two parts: predicted portions adopts the predictive equation group, utilize the state value and the predicated error of previous moment to make prediction, obtain the position of each tracking unit at current time, can there be certain error owing to predict the outcome, revise part and adopt the update equation group, utilize the observed reading correction of the current time that obtains to predict the outcome.

5. the object tracking based on parts according to claim 1, it is characterized in that, the parameter of Kalman filter is revised in described measurement by observed reading, be meant: the method that around the tracking unit position that the predictive equation group dopes, adopts spiral search, find a bit, with the histogram of this some window that is neighborhood and the histogrammic Euclidean distance of former parts less than preset threshold, the position of this point is just as the observed reading of present frame, the update equation group is revised the predicted value of current Kalman filter by this observed reading, obtains revised state estimation value and Noise Variance Estimation value.

6. the object tracking based on parts according to claim 1, it is characterized in that, the renewal of described parts, be specially: if around future position, can't find qualified point, adopt decision method with object identity, keep the parts that are positioned within the identified areas, promptly keep because object rotation or owing to block the parts disappearance that causes but still parts within tracing object, eliminate because of following the tracks of failure and be in parts outside the tracing object, and the method for the grey level histogram weighting summation of the grey level histogram of former parts and current time parts present position is upgraded grey level histogram.

7. according to claim 1 or 6 described object trackings, it is characterized in that based on parts, the renewal of described parts, more new formula is as follows for it: q _k=(1-α) q _K-1+ α p _k, wherein: q _k, q _K-1Be respectively the grey level histogram based on nuclear of k, k-1 frame, α is for upgrading the factor.

8. the object tracking based on parts according to claim 1, it is characterized in that, described tracing object is identified out, be meant: determine after each parts of tracing object, rectangle with the area minimum that can comprise all tracking unit central points identifies out with tracing object, make each parts unification to an object, be specially: adopt the Graham Sodd method of investing method to determine the convex hull of object angle point set, obtain after the convex hull of point set, extended line with a limit on the convex hull is one side of rectangle, find out and comprise a little the rectangle of concentrating all points, the rectangular area of area minimum is found out in rotation successively.

Images