CN105184779B - One kind is based on the pyramidal vehicle multiscale tracing method of swift nature - Google Patents

Abstract

The present invention provides a vehicle multi-scale tracking method based on a fast feature pyramid, using aggregated channel feature ACF features, and using a fast feature pyramid to approximate features on adjacent scales, thereby quickly calculating tracking results on different scales. Aggregate channel features to obtain gradient images from L, U, and V channels of vehicle images to obtain gradient channels; cascade these channel features to form ACF features. Fast Pyramid scales and approximates the original feature map through the idea of statistics. According to the energy statistical relationship between the scaling of each channel, the image on the original scale can be directly calculated to obtain the image on the scale after scaling. Vehicle tracking based on aggregated channel features not only utilizes the global information of multiple channels, but also makes full use of the local information of the vehicle on each channel, which improves the robustness of vehicle tracking; and uses the fast feature pyramid to achieve tracking accuracy. Multi-scale, and ensure the real-time tracking.

Description

A Multi-Scale Vehicle Tracking Method Based on Fast Feature Pyramid

technical field

The invention belongs to digital image processing technology, in particular to computer vision recognition technology.

technical background

With the continuous development of economy and technology and the continuous improvement of scientific and technological level, people's quality of life has also undergone earth-shaking changes. In recent years, with the development of machine vision technology, machine vision has been widely used in intelligent transportation systems, such as traffic incidents and flow monitoring, road surface disease detection, and automatic navigation of intelligent vehicles. As an important part of the intelligent transportation system, intelligent safety vehicles are the current research hotspots. Intelligent vehicles use intelligent algorithms to understand the real-time environment of vehicles and provide guarantees for safe driving.

Vehicle tracking technology is another big new technical field after vehicle detection technology. The two are closely connected and indispensable. According to the positional relationship between the camera and the target, tracking methods can be divided into target tracking under static background and dynamic background tracking.

(1) Target tracking method in static background:

Target tracking in a static background means that the camera is fixed at a certain orientation, and the field of view it observes is also static. Background subtraction and Gaussian background modeling are usually used. The background subtraction method first makes the difference between the foreground image and the background image, and then the target object entering the field of view can be obtained. For the description of the target, the size of the target is usually expressed by the number of pixels in the connected area of the target, or the aspect ratio of the target area. The location information of the target can be located by means of projection. Gaussian background modeling, that is, to model the background first, usually using K (basically 3 to 5) Gaussian models to characterize the characteristics of each pixel in the image, update the mixed Gaussian model after a new frame of image is obtained, and then Read the image from the video stream (we call it the foreground image), and use each pixel in the current image to match the mixed Gaussian model. If it succeeds, it is determined that the point is a background point, otherwise it is a foreground point.

(2) Target tracking method in dynamic background:

Object tracking in a dynamic background means that the camera is not fixed at a certain position, and the field of view observed by the camera is not static, that is, the background is also moving relative to the camera. For this case, the detection-based tracking method is usually used to detect single-frame images to achieve the effect of continuous detection.

Common tracking algorithms are: Kalman filter, particle filter, TLD, CT.

(1) Kalman filtering: Kalman filtering (Kalman filtering) is an algorithm that uses the linear system state equation to optimally estimate the system state through the input and output observation data of the system. Since the observation data includes the influence of noise and interference in the system, the optimal estimation can also be regarded as a filtering process. The vehicle tracking method based on the Kalman filter uses past state information and current measurement results to predict the current optimal state information to achieve the purpose of tracking.

(2) Particle filter: The idea of particle filter (PF: Particle Filter) is based on the Monte Carlo method (Monte Carlo methods), which uses particle sets to represent probability and can be used in any form of state space model. Its core idea is to express its distribution through random state particles extracted from the posterior probability, which is a sequential importance sampling method (Sequential Importance Sampling). In simple terms, the particle filter method refers to the process of approximating the probability density function by finding a group of random samples propagated in the state space, and replacing the integral operation with the sample mean value, so as to obtain the minimum variance distribution of the state. The samples here refer to the particles, and when the number of samples is N→∝, any form of probability density distribution can be approximated.

(3) TLD: The foothold of TLD (Tracking-Learning-Detection) is to solve the shape change, illumination condition change, scale change, occlusion, etc. of the tracked target during long-term tracking. The detector localizes the detected features (representing the target object) and continuously corrects the tracker as needed. The learner estimates the errors of the detector and updates the detector in time to avoid these errors in the future. In the traditional tracking algorithm, the front end needs to cooperate with the detection module. When the tracked target is detected, it starts to enter the tracking module. After that, the detection module will not intervene in the tracking process. The significant difference between this algorithm and the traditional tracking algorithm is that the traditional tracking algorithm and the traditional detection algorithm are combined to solve the deformation and partial occlusion of the tracked target during the tracking process.

(iv) CT: CT (Compressive Tracking) is based on the theory of sparse perception, using a very sparse measurement matrix that satisfies the RIP condition to project the original image feature space, and a low-dimensional compressed subspace can be obtained. The low-dimensional compressed subspace can well preserve the information of the high-dimensional image feature space. Therefore, we use the sparse measurement matrix to extract the features of the foreground target and background, as the positive and negative samples of the online learning update classifier, and then use the naive Bayesian classifier to classify the target image slice of the next frame of image.

(v) MOSSE: The MOSSE (Minimum Output Sum of Squared Error filter) filter is a tracking method with good real-time performance and using the correlation convolution matrix to obtain the minimum output mean square error. And it is robust to lighting, size, pose, and non-rigid deformation. The algorithm first assumes that the output response of the tracking target area has a Gaussian distribution. Using the convolution property, the convolution filter can be obtained, and then at the current position of the next frame image, use this filter to filter to obtain a new output response. , according to the new output belief to determine the position of the target in the current frame.

Contents of the invention

The technical problem to be solved by the invention is to provide a fast and accurate vehicle tracking method.

The technical solution adopted by the present invention to solve the above-mentioned technical problems is a multi-scale vehicle tracking method based on fast feature pyramid. When a target is detected in the previous frame, (x ₀ , y ₀ ) is the detected The center point of the target area, then perform target tracking on the current frame, the specific method is as follows:

Step 1. Calculate the aggregated channel features of the current frame image at the original scale to obtain the feature map at the original scale:

1-1 Convert the current frame image to LUV color space to obtain L, U, V three-channel features;

1-2 Find the gradient map of the LUV image to obtain the HOG feature of the gradient histogram;

1-3 Concatenate the L, U, V three-channel features with the HOG channel features in each direction to obtain the aggregated channel features of the current frame at the original scale;

Step 2. Calculate the feature pyramid:

According to the various scales under the feature pyramid, according to the energy statistical relationship of each channel feature of the aggregation channel feature, perform feature pyramid sampling on the feature map at the original scale to obtain the feature map at the corresponding scale; the scale of the feature map after each feature pyramid sampling is the same as the original scale The scale ratio of the feature map below is s ₁ , s ₂ ,...s _n , n is the total number of different scales; each feature pyramid sampling includes upsampling and downsampling;

Step 3. Calculate tracking results on different scales:

3-1 The width and height of the feature channel of the feature map under the original size are W ₀ and H ₀ respectively, and the center point of the target area of the feature map under the original size is (x ₀ , y ₀ ); The center position of the target area is (x ₀ s ₁ ,y ₀ s ₁ ),(x ₀ s ₂ ,y ₀ s ₂ ),...,(x ₀ s _n ,y ₀ s _n );

3-2 Calculate the convolution filter:

For the case where the target was detected for the first time in the previous frame:

Under the sampling of each feature pyramid, a random Gaussian distribution is generated with the center position of the target area as the extreme value of the Gaussian distribution, and the random Gaussian distribution corresponds to the output responses G ₁ , G ₂ ,...,G _n of the target area. Initialization Filter coefficients for convolution filters i=1,2,...,n, F _i is the Fourier transform of the target area, G _i is the random Gaussian distribution centered on the target area;

For the case where the target was not detected for the first time in the previous frame:

The update of the convolution filter adopts the following strategy:

η is the weight coefficient, is the filter coefficient of the convolution filter of the previous frame;

3-3 Input the feature map of each size after feature pyramid sampling into the convolution filter, and the output responses of the convolution filter are g ₁ , g ₂ ,...,g _n respectively, and the tracking values at each size are obtained respectively PSR ₁ ,PSR ₂ ,...,PSR _n , g _max is the extreme value of the output response at the current size, μ _s and σ _s are the mean and variance of the output response at the current size, respectively;

3-4 Take the largest of the tracking values PSR ₁ , PSR ₂ ,...,PSR _n as the tracking result PSR. When the PSR is greater than the set threshold, it is considered that the tracking is successful, and the tracking of the next frame will continue, and the tracking result will be Map the corresponding scale to the original scale to determine the target area; when the PSR is less than the set threshold, it is considered that the tracking fails, and the current tracking target is canceled.

First, the present invention uses a new, more discriminative aggregated channel feature ACF feature to replace the gray feature in the MOSSE algorithm, and second, uses a fast feature pyramid to approximate features on adjacent scales, thereby quickly calculating tracking results. The ACF feature first transforms the collected RGB vehicle image into the Luv space to obtain the L, U, and V channels respectively; on this basis, the gradient image is obtained to obtain the gradient channel; these channel features are cascaded to form the ACF feature. The fast pyramid is to zoom and approximate the original feature map through the idea of statistics. Different from the traditional image zoom method, the traditional method needs to use the nearest neighbor or linear interpolation to calculate the zoom result, which brings more computational complexity. Higher than the Fast Pyramid method. According to the energy statistical relationship between the scaling of each channel, the fast pyramid can directly calculate the image on the original scale to obtain the image on the scale after scaling.

The beneficial effect of the present invention is that the vehicle tracking algorithm based on aggregated channel features not only utilizes the global information of multiple channels, but also fully utilizes the local information of the vehicle on each channel, which improves the robustness of vehicle tracking; and The multi-scale tracking is achieved by using the fast feature pyramid, and the real-time performance of the tracking is guaranteed.

Description of drawings

Fig. 1 is a schematic flow chart of the present invention.

specific implementation plan

In order to describe the content of the present invention conveniently, some terms are explained first.

Luv channel features. The full name of the LUV color space is CIE 1976 (L, u, v) (also known as CIELUV) color space, L represents the brightness of the object, and u and v are the chromaticity. Proposed by the International Commission on Illumination CIE in 1976, it is obtained by simple transformation of the CIE XYZ space and has visual unity. A similar color space is CIELAB. For general images, the values of u and v range from -100 to +100, and the brightness ranges from 0 to 100.

Gradient channel features. The gradient channel feature is the gradient map of an image, which reflects the edge information of the target. There are many ways to find the gradient, such as Prewitt operator and Sobel operator. However, the simplest operator [-1 0 1] has better performance. Gradients are used to describe the edges of vehicle images. Since there is only a linear change between the Luv channel and the RGB channel, for convenience, the gradient map of the image can be obtained on the Luv channel after obtaining the Luv channel.

Gradient histogram. The gradient histogram is HOG (Histogram of oriented gradient). First, the image is divided into small connected areas, which we call cell units. Then the gradient or edge direction histogram of each pixel in the cell unit is collected. Finally, these histograms are combined to form a feature descriptor. In order to improve performance, we can also contrast-normalize these local histograms in a larger range of the image (we call it interval or block). The method used is: first calculate each histogram The density of the graph in this interval (block), and then normalize each cell unit in the interval according to this density. After this normalization, better effects on lighting changes and shadows can be obtained.

Adopt the inventive method, realize algorithm on C++ platform, concrete steps are as follows:

Step 1. Calculate the aggregated channel features of the current frame image at the original scale to obtain the feature map at the original scale:

1-1 Convert the current frame image to LUV color space to obtain L, U, V three-channel features;

The images collected by the camera are generally RGB images, which are not conducive to color clustering. In order to describe the grayscale and chromaticity information of the vehicle well, it is necessary to convert the RGB image into an LUV image. The specific method is:

First convert the RGB image to CIE XYZ

Then convert CIE XYZ to Luv

u＝13L(u'-u _n ') (3)

v=13L(v'-v _n ') (4)

in,

Y _n is its brightness, and (u _n ', v _n ') is its chromaticity coordinates.

1-2 Find the gradient map of the LUV image to obtain the HOG feature of the gradient histogram;

There are many methods for gradient calculation, such as Prewitt operator and Sobel operator and However, the simplest operator [-1 0 1] and The effect obtained by filtering is better.

1-3 sampling and normalization

Since the 4*4 cells are assigned to 6 directions when calculating the gradient histogram, that is, the aspect ratio of the gradient histogram is 1/4 of the original sample image, in order to keep the aspect ratio of all channels consistent, The Luv channel image and gradient image need to be down-sampled, and the sampling here does not affect the detection result. Use bilinear interpolation during sampling to get better results

In order to suppress the influence of noise in the gradient calculation, it is necessary to normalize the gradient map. The normalization operations are L1-norm, L2-norm and L1-sqrt.

L1-norm: v→v/(||v|| ₁ +ε) (5)

L2-norm:

L1-sqrt:

Among them, ε is a very small number, such as 0.01, v is the gradient, ||·|| ₁ represents a norm, and ||·|| ₂ represents a two-norm. In this embodiment, L2-norm is used.

Get the gradient map, and vote on the gradient direction histogram for each pixel in the 4*4 unit as the direction of the gradient element, thereby forming a direction gradient histogram. The direction of the histogram is equally divided between 0-180 degrees or 0-360 degrees. In order to reduce aliasing, gradient voting requires bilinear interpolation in direction and position between the centers in two adjacent directions. The weight of the vote is calculated according to the magnitude of the gradient, which can be the magnitude itself, the square of the magnitude, or the square root of the magnitude. Practice shows that using the gradient itself as the voting weight works best.

Due to the change of local illumination and the change of foreground and background contrast, the gradient strength varies greatly, which requires local contrast normalization of the gradient. The specific method is to form cell units into larger spatial blocks, and then perform contrast normalization for each block. The normalization process is the same as step 3. The final descriptor is the histogram of the cell units in all blocks within the detection window. A vector of diagrams. In fact, there is overlap between the blocks, that is, the histogram of each cell unit will be used multiple times for the calculation of the final descriptor. This method may seem redundant, but it can improve performance significantly.

2-3 Concatenate the L, U, V three-channel features with the HOG channel features in each direction to obtain the aggregated channel ACF features of the current frame at the original scale; if the gradient histogram has six directions, a total of 10 channels are obtained. These 10 channels are aggregated channel features.

Step 2. Calculate the feature pyramid

According to the various scales under the feature pyramid, according to the energy statistical relationship of each channel feature of the aggregation channel feature, perform feature pyramid sampling on the feature map at the original scale to obtain the feature map at the corresponding scale; the scale of the feature map after each feature pyramid sampling is the same as the original scale The scale ratio of the feature map below is s ₁ , s ₂ ,...s _n , n is the total number of different scales; each feature pyramid sampling includes upsampling and downsampling;

upsampling

Let the original image be I(x,y), and the image obtained after being sampled by an upsampling factor k is I _k (x,y)=I(x/k,y/k), which is defined by the gradient That is, the ratio of the change on the upsampled image is 1/k times that of the original scale image. Assume is the gradient size of the upsampled image, then

Therefore, in the original image and the upsampled image, their gradient sum is related to the upsampled k. Similarly, the gradient direction

Therefore, it can be seen from the above definition that in the upsampled image, its gradient relationship is h' _q ≈ kh _q .

downsampling

Downsampling cannot be derived from upsampling due to high frequency loss and energy loss in the previously sampled image. Let I _k be the image after the original image I is downsampled by the downsampling factor k, and the relationship between them should be h' _q ≤ h _q /k. Since the energy channel image of the image and the downsampled image of the energy channel are only related to the relative scale of the energy channel, they are independent of the original image. If f(I,s _i ) is used to represent the energy of the channel after sampling at the scale of s _i , then

E[f(I,s ₁ )/f(I,s ₂ )]=E[f(I,s ₁ )]/E[f(I,s ₂ )]=r(s ₁ -s ₂ ) ( 11)

established. Therefore E[f(I,s ₁ )/f(I,s ₂ )] must have the following form:

E[f(I,s+s ₀ )/f(I,s ₀ )]=ae ^-λs (12)

A and λ can be obtained by doing data statistics on a large number of target images and natural images. Therefore, the relationship between the downsampled image and the original image is f(I,s)=ae ^-λs f(I,0).

Step 3. Multi-scale tracking:

Replace the gray features in the original MOSSE tracking algorithm with ACF features, and vectorize the features of each channel, then get multiple feature vectors, combine them into a matrix, and perform Fourier transform as F _i , so that Tracking is more robust, and tracking results are computed at different scales:

3-1 The width and height of the feature channel of the feature map under the original size are W ₀ and H ₀ respectively, and the center point of the target area of the feature map under the original size is (x ₀ , y ₀ ); The center position of the target area is (x ₀ s ₁ ,y ₀ s ₁ ),(x ₀ s ₂ ,y ₀ s ₂ ),...,(x ₀ s _n ,y ₀ s _n );

3-2 Calculate the convolution filter:

For the case where the target was detected for the first time in the previous frame:

Under the sampling of each feature pyramid, a random Gaussian distribution is generated with the center position of the target area as the extreme value of the Gaussian distribution, and the random Gaussian distribution corresponds to the output responses G ₁ , G ₂ ,...,G _n of the target area. Initialization The filter coefficients of the convolution filter:

F _i is the Fourier transform of the target area, G _i is the random Gaussian distribution centered on the target area;

For the case where the target was not detected for the first time in the previous frame:

The update of the convolution filter adopts the following strategy:

η is the weight coefficient, is the filter coefficient of the convolution filter of the previous frame;

3-3 Input the feature map of each size after feature pyramid sampling into the convolution filter, and the output responses of the convolution filter are g ₁ , g ₂ ,...,g _n respectively, and obtain the tracking values at each size PSR ₁ ,PSR ₂ ,...,PSR _n ;

g _max is the extreme value of the output response at the current size, μ _s and σ _s are the mean and variance of the output response at the current size, respectively;

3-4 Take the largest of the tracking values PSR ₁ , PSR ₂ ,...,PSR _n as the tracking result PSR. When the PSR is greater than the set threshold, it is considered that the tracking is successful, and the tracking of the next frame will continue, and the tracking result will be Map the corresponding scale to the original scale to determine the target area; when the PSR is less than the set threshold, it is considered that the tracking fails, and the current tracking target is canceled.

Claims (1)

1. A vehicle multi-scale tracking method based on a fast feature pyramid is disclosed, when a target is detected in a previous frame, (x)₀,y₀) And tracking the target of the current frame for the central point of the target area detected by the previous frame, wherein the target tracking comprises the following steps:

step 1, calculating the aggregation channel characteristics of the current frame image under the original scale to obtain a characteristic map under the original scale:

1-1, converting the current frame image into an LUV color space to obtain L, U, V three-channel characteristics;

1-2, solving a gradient map of the LUV image to obtain a HOG (histogram of gradients) feature;

1-3, cascading the L, U, V three-channel characteristics and the HOG channel characteristics in each direction to obtain the aggregation channel characteristics of the current frame in the original scale;

step 2, calculating a characteristic pyramid:

according to various scales under the characteristic pyramid, performing characteristic pyramid sampling on the characteristic graph under the original scale according to the energy statistical relationship of the characteristics of all channels of the aggregation channel characteristics to obtain a characteristic graph under the corresponding scale; the ratio of the dimension of the feature map after sampling by each feature pyramid to the dimension of the feature map under the original dimension is s₁,s₂,...s_nN is the total number of different scales; each characteristic pyramid sampling comprises up sampling and down sampling;

and 3, calculating tracking results on different scales:

3-1 the width and height of the characteristic channel of the characteristic diagram under the original size are respectively W₀And H₀Let the center point of the target region of the feature map under the original size be (x)₀,y₀) (ii) a The central position of the corresponding target area after each characteristic pyramid is sampled is (x)₀s₁,y₀s₁),(x₀s₂,y₀s₂),...,(x₀s_n,y₀s_n) ；

3-2 calculate convolution filter:

for the case where the previous frame was the first time the target was detected:

under the sampling of each characteristic pyramid, a random Gaussian distribution is generated by taking the central position of the target area as an extreme value of the Gaussian distribution, and the random Gaussian distribution corresponds to the output response G of the target area₁,G₂,...,G_nInitializing the filter coefficients of the convolution filterF_iIs Fourier transform of the target area, G_iRandom Gaussian distribution with a target area as a center position;

for the case where the previous frame was not the first time the target was detected:

the convolution filter is updated by adopting the following strategy:

η are the weight coefficients of the image data,the filter coefficient of the convolution filter of the previous frame;

3-3, inputting the feature graphs of all sizes after the feature pyramid is sampled into a convolution filter, wherein the output responses of the convolution filters are g respectively₁,g₂,...,g_nSeparately obtaining the tracking value PSR of each size₁,PSR₂,...,PSR_n，g_maxFor the extreme value of the output response at the current size, mu_sAnd σ_sRespectively mean and variance of output response under the current size;

3-4 taking tracking value PSR₁,PSR₂,...,PSR_nThe maximum PSR is used as a tracking result PSR, when the PSR is larger than a set threshold value, the tracking is considered to be successful, the tracking of the next frame is continued, and the tracking result is mapped to the original scale according to the corresponding scale to determine a target area; and when the PSR is smaller than the set threshold, the tracking is considered to be failed, and the current tracking target is cancelled.