CN103198493B - A kind ofly to merge and the method for tracking target of on-line study based on multiple features self-adaptation - Google Patents

Abstract

The invention discloses a target tracking method based on multi-feature adaptive fusion and online learning, which extracts target features and uses them as template features; respectively extracts three types of features for new candidate target areas; performs automatic tracking according to the discrimination and correlation of each feature Adapt to fusion; calculate the Bhattacharyachian distance between the fused feature and the template feature, and normalize the Bhattacharyachian distance as the weight of the new candidate target area; judge the overlap between the new candidate target area and the target area with the largest weight, if the overlap rate is less than Overlap rate threshold Input the multiple area of the new candidate target area with the maximum weight into the detector. When the output of the recognizer is yes, it means that the tracking is successful and the recognizer, template features and target area are updated; if the output is no, it means that a new target is found; if it overlaps rate is greater than or equal to the overlap rate threshold, update the recognizer, template features, and target regions. Enhanced the adaptability of target tracking in different scenes and under certain deformation conditions, avoiding the problem of easy tracking drift after occlusion.

Description

A Target Tracking Method Based on Multi-Feature Adaptive Fusion and Online Learning

technical field

The invention relates to the field of target tracking, in particular to a target tracking method based on multi-feature adaptive fusion and online learning.

Background technique

Video object tracking refers to the process of object detection, representation and trajectory extraction in video sequences. Video object tracking has practical application requirements in video surveillance, event analysis, human-computer interaction and other fields.

At present, the world's most advanced surveillance systems cannot perfectly handle dynamic tracking tasks in complex scenes, such as: deformation, occlusion, lighting changes, shadows or tracking in crowded environments. Object tracking remains a challenge, especially when objects are partially occluded and deformed.

Single-feature-based tracking methods usually initialize target regions, extract arbitrary target features, such as color features, and search and match them in the next frame. However, it is difficult for this method to deal with tracking tasks in complex backgrounds, which makes it not robust to target tracking and subsequent trajectory evaluation.

For this reason, an object tracking method based on multi-features is proposed in the prior art. This method usually extracts features such as color and edge, and can better complete tracking tasks in some complex situations.

In the process of realizing the present invention, the inventor finds that at least the following disadvantages and deficiencies exist in the prior art:

The target tracking method based on multi-features cannot adapt to the change of target shape, and cannot solve the problem of tracking drift after partial occlusion between targets.

Contents of the invention

The present invention provides a target tracking method based on multi-feature adaptive fusion and online learning. This method avoids the problem of tracking drift and is well adapted to changes in target shape. See the following description for details:

A target tracking method based on multi-feature adaptive fusion and online learning, said method comprising the following steps:

(1) Select the target area from a frame of image, extract the target features and use them as template features;

(2) Initialize the recognizer, input the next frame of image, and initialize the candidate target area; obtain the new candidate target area according to the transfer formula;

(3) Extract three features of color, edge, and texture from the new candidate target area; perform adaptive fusion according to the discrimination and correlation of each feature;

(4) Calculate the Bhattacharyachian distance between the fused feature and the template feature, and normalize the Bhattacharyachian distance as the weight of the new candidate target area;

(5) Sort the N new candidate target areas according to their weights. If the resampling judgment value is greater than the resampling judgment threshold, perform resampling and perform step (6); if not, perform step (6);

(6) Judging the overlap between the new candidate target area with the maximum weight and the target area, if the overlap rate is less than the overlap rate threshold, perform step (7); otherwise, perform step (8);

(7) Input the multiple area of the new candidate target area with the maximum weight into the detector. If the detector outputs 0, it means that the tracking fails; otherwise, input the output result of the detector to the recognizer. If the recognizer outputs Yes, it means that the tracking is successful and the recognizer, the template feature and the target area are updated; if the output is No, it means that a new target is found, and the process ends;

(8) If the overlap rate is greater than or equal to the overlap rate threshold, it is considered that the tracking is successful, the recognizer, the template feature and the target area are updated, and the process ends.

The step of extracting target features and using them as template features specifically includes:

1) Extract color feature information;

2) Extract edge feature information;

3) Extract texture feature information;

4) Fusing the color feature information, the edge feature information and the texture feature information to obtain a target feature histogram as the template feature.

The step of extracting color feature information specifically includes:

1) Divide the color space into a color area and an achromatic area, perform HSV partition on the color area and the achromatic area, obtain Q _H × Q _S color sub-intervals and Q _v achromatic sub-intervals, divide all The QH×QS color subintervals and the _Qv achromatic subintervals are used as _QH × _{QS + Qv} color intervals _u ;

2) Assign different weights to the pixel according to the distance between the pixel and the center point of the target area, and vote for the corresponding color interval u according to the HSV of the pixel;

3) Count the voting values of each color interval to obtain a color feature histogram.

The step of extracting edge feature information specifically includes:

1) Interpolating the target area to obtain an interpolation area twice the width and height, and then dividing the target area and the interpolation area into blocks respectively;

2) Calculate the edge strength and direction of each sub-block, divide the edge direction into several direction areas within the range of 0°-360°, vote for the direction area according to the edge strength, and obtain the edge feature histogram of each sub-block;

3) The edge feature histograms calculated by each sub-block are connected to obtain a complete edge feature histogram.

The step of extracting texture feature information specifically includes:

1) For each sub-block, calculate the local binary pattern feature histogram;

2) Connect the local binary pattern feature histograms calculated by each sub-block to obtain a complete texture feature histogram.

The discrimination is defined as the degree of similarity between the new candidate target region and the adjacent background with respect to a certain feature, expressed by the Bhattachary coefficient of the two histograms.

The correlation is defined as the degree of similarity between the new candidate target region and the template feature with respect to a certain feature, expressed by the Bhattachary coefficient of the two histograms.

The update identifier is specifically: the positive sample is composed of the new candidate target area verified by the detector, and the negative sample is a randomly selected area in the background with the same size as the new candidate target area;

The updating template feature is specifically: using the feature of the new candidate target region with the maximum weight as the updated template feature;

The updating of the target area specifically includes: taking the new candidate target area with the largest weight as the updated target area.

The beneficial effects of the technical solution provided by the present invention are: the method overcomes the singleness of a single feature, enhances the adaptability of target tracking in different scenes and under certain deformation conditions, and avoids the problem of easy tracking drift after occlusion, greatly improving The accuracy and robustness of target tracking are improved.

Description of drawings

Fig. 1 is a flow chart of a target tracking method based on multi-feature adaptive fusion and online learning;

FIG. 2 is a schematic diagram of initializing a target area;

FIG. 3 is a schematic diagram of occlusion of target 1;

Figure 4 is a schematic diagram of successfully retrieving the lost target through online learning;

5 is a schematic diagram of another initialization target area;

Fig. 6 is a schematic diagram of staggered occlusion of target 1 and target 2;

Fig. 7 is a schematic diagram of accurate tracking without tracking drift.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the implementation manner of the present invention will be further described in detail below in conjunction with the accompanying drawings.

In order to avoid the problem of tracking drift and adapt to the change of target shape, the embodiment of the present invention provides a target tracking method based on multi-feature adaptive fusion and online learning. Online learning can overcome target deformation and The problems caused by tracking drift have better achieved the expected tracking effect, see Figure 1, and see the description below for details:

101: Take a frame of any video sequence as input, select a target area from a frame of image, extract target features and use them as template features;

Wherein, the operation of selecting the target area is well known to those skilled in the art, and the target area is a rectangular area, for example: according to actual needs, the rectangular area where the tracked object is located can be manually selected; or the detected object model (for example: human body detection Model [1]) for automatic detection and calculation of template features.

Among them, the steps of extracting target features and using them as template features are as follows:

1) Extract color feature information;

This method uses the kernel-weighted color feature histogram based on the HSV (hue, saturation, and brightness) color space model to model the target. The basic idea is:

(1) Divide the color space into color area and achromatic area, perform HSV partition on the color area and achromatic area, obtain Q _H × Q _S color sub-intervals and Q _v achromatic sub-intervals, divide Q _H × Q _S color sub-intervals and Q _v achromatic sub-intervals are used as Q _H × Q _S + Q _v color intervals u;

For example: all brightness less than 20% or saturation less than 10% are classified as non-color areas, and are divided into Q _v achromatic sub-intervals according to the brightness value, and the color areas outside the achromatic area are color areas, according to chroma and saturation Degree is divided into Q _H × Q _S color subintervals.

(2) According to the distance between the pixel point and the center point of the target area, different weights are assigned to the pixel point (that is, a smaller weight value is assigned to the pixel point farther from the center point of the target area, thereby reducing the interference of the target boundary and background), Vote for the corresponding color interval u according to the HSV of the pixel;

Define the target area as a rectangular area with width W and height H. Pixels on the boundary of the target area may belong to the background or be partially occluded. In order to increase the reliability of the color distribution, the following function is used to assign weights:

$\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>{<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}& \mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; 1</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; \&Greater\; Equal;</span>\mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right.$

In the formula, k( _xi ) represents the weight assigned to the pixel x _i , y represents the center point of the target area, and x _i is the i-th pixel in the target area; Indicates the target area size.

This method uses the Dirac function δ to determine the color interval corresponding to each pixel point, δ(b( _xi )-u) represents the distribution of pixel point x _i in the color interval u in the histogram: b( _xi ) is the pixel HSV of point x _i , φ, η is a constant. The Dirac function δ is defined as follows:

δ(x-φ)=0, x≠φ

For example: there are 3 pixels, namely: x ₁ , x ₂ and x ₃ , the color interval corresponding to pixel x ₁ is u1, the color interval corresponding to pixel x2 is u1, and the color interval corresponding to pixel x ₃ is u2, the voting result of color interval u1 is the sum of the weights of pixel point x1 and pixel point _{x2 ;} the voting result of color interval u2 is the weight of pixel point _x3 ; the voting result of color interval u3 is 0.

(3) The color feature histogram is obtained through the voting value calculated in each color interval, and the dimension of the color feature histogram is the sum of the number of color intervals and achromatic intervals.

The color feature histogram p _c can be expressed as:

${\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}\underset{\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; u</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; \&delta;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; )</span>$

In the formula, U is the number of color intervals, N is the number of pixels in the target area, and the normalization constant C _h :

${\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span>}$

2) Extract edge feature information;

This method adopts an edge direction feature based on multi-scale block edge intensity weighting, and the steps are as follows:

(1) Interpolate the target area to obtain an interpolation area twice as wide and high, and then divide the target area and the interpolation area into blocks;

Among them, the method of interpolating the target area is well known to those skilled in the art. The block method uses methods such as overlapping blocks or non-overlapping blocks to divide the target area and the interpolation area into several sub-blocks, which is not limited in the embodiment of the present invention .

(2) Calculate the edge strength and direction of each sub-block, divide the edge direction into several direction areas within the range of 0°-360°, vote for the direction area according to the edge strength, and obtain the edge feature histogram of each sub-block ;

Wherein, the embodiment of the present invention uses the Sobel operator [2] to calculate the edge strength and direction, and other methods may also be used, which is not limited in the embodiment of the present invention. The number of direction areas is set according to the needs of practical applications, for example, the edge direction is divided into 18 direction areas within the range of 0°-360° with 20 degrees as the interval. The step of voting for the direction region according to the edge strength is well known to those skilled in the art, and the embodiment of the present invention will not repeat it here.

(3) Connect the edge feature histograms calculated by each sub-block to obtain a complete edge feature histogram.

Among them, the dimension of the complete edge feature histogram is the product of the number of direction regions and the total number of sub-blocks.

3) Extract texture feature information;

This method uses the block-based multi-scale local binary model [3] operator to calculate the texture features. Compared with the original local binary pattern features, the block-based multi-scale local binary pattern features have less impact on image noise. It is sensitive, and can extract richer local and global information, has stronger representation and discrimination capabilities for target images, and is more robust.

(1) For each sub-block, calculate the feature histogram of the local binary pattern;

Among them, each sub-block is a sub-block obtained by dividing edge feature information, and this method uses a unified mode [4] containing 59 styles to calculate local binary mode features.

(2) Connect the local binary pattern feature histograms calculated by each sub-block to obtain a complete texture feature histogram.

Among them, the dimensionality of the complete texture feature histogram is 59 times the total number of sub-blocks.

4) The color feature information, edge feature information and texture feature information are fused to obtain the target feature histogram as the template feature.

That is to normalize the color feature information, edge feature information and texture feature information, according to the preset weight (the weight value is set according to the needs of practical applications, for example: color feature information, edge feature information and texture feature information The weight ratio is 1:1:1) The three normalized feature information are connected to obtain the target feature histogram.

102: Initialize the recognizer;

The recognizer is used to retrieve the lost tracked target. The recognizer uses the Boost algorithm [5] to train and learn. The positive samples in the learning are the target areas, and the negative samples randomly select the rectangular area with the same size as the positive samples in the background. The negative samples The selection of the number is determined according to actual needs, and the reference value of this experiment is 3.

103: Input the next frame of image, and initialize the candidate target area;

According to the Gaussian distribution, randomly select N regions around the target region as candidate target regions. The value of N is selected according to actual needs, and the reference value is 20 in this experiment.

104: Obtain a new candidate target area according to the transfer formula;

R _n =A*(R _c -R ₀ )+B*rng+R ₀

Among them, R _n , R _c , and R ₀ are the new candidate target area, the candidate target area, and the target area respectively; A and B are transfer coefficients; rng is a random number generator. For example: the new candidate target area, the candidate target area and the target area are represented by the width, height and center coordinates (x, y) of the rectangular area, that is, the width, height and center coordinates are used to determine the new candidate target area. When calculating the new candidate target area When the width parameter of the candidate target area and the width parameter of the target area are substituted into the transfer formula, the width parameter of the new candidate target area is obtained. By analogy, the height parameter and center coordinates of the new candidate target area are calculated respectively, and finally New candidate target regions.

105: Extract three visual features of color, edge, and texture from the new candidate target area;

Wherein, the specific operation of this step is the same as that of step 101, which is not repeated in this embodiment of the present invention.

106: According to the discrimination and correlation of color, edge and texture features, adaptively fuse each feature;

The discriminative RD _f is defined as the degree of similarity between the new candidate target area and the adjacent background on a certain feature, expressed by the Bhattachary coefficient of the two histograms:

${\mathrm{<span\; class="notranslate">\; RD</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\sqrt{{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; bg</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}}^{\mathrm{<span\; class="notranslate">\; m</span>}}}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; \&Element;</span><span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; HSV</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; EO</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; LBP</span>}<span\; class="notranslate">\; \}</span>$

In the formula, HSV, EO, and LBP respectively represent color, edge, and texture features, and Represents the mth dimension of the feature histogram of the adjacent background and target area respectively; M is the dimension of the color feature histogram or the dimension of the complete edge feature histogram or the dimension of the complete texture feature histogram. The smaller the discriminative RD _f is, the stronger the discrimination of the feature is, and a higher weight is assigned; on the contrary, a lower weight is assigned.

Correlation is defined as the degree of similarity between the new candidate target area and the template feature on a certain feature, expressed by the Bhattachary coefficient of the two histograms:

${\mathrm{<span\; class="notranslate">\; cd</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\sqrt{{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}}^{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}}^{\mathrm{<span\; class="notranslate">\; m</span>}}}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; \&Element;</span><span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; HSV</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; EO</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; LBP</span>}<span\; class="notranslate">\; \}</span>$

In the formula, HSV, EO, and LBP respectively represent color, edge, and texture features, and represent the mth dimension of the feature histogram of the new candidate target area and the target area respectively; M is the dimension of the color feature histogram or the dimension of the complete edge feature histogram or the dimension of the complete texture feature histogram; the correlation CD _f The larger the feature is, the stronger the correlation of the feature is, and a higher weight is assigned; on the contrary, a lower weight is assigned.

The weights α, β, and γ of each feature are finally jointly determined by RDf and CDf:

$\left\{\begin{array}{c}\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; RD</span>}}_{\mathrm{<span\; class="notranslate">\; HSV</span>}}<span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; cd</span>}}_{\mathrm{<span\; class="notranslate">\; HSV</span>}}\\ \mathrm{<span\; class="notranslate">\; \&beta;</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; RD</span>}}_{\mathrm{<span\; class="notranslate">\; EO</span>}}<span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; cd</span>}}_{\mathrm{<span\; class="notranslate">\; EO</span>}}\\ \mathrm{<span\; class="notranslate">\; \&gamma;</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; RD</span>}}_{\mathrm{<span\; class="notranslate">\; LBP</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; cd</span>}}_{\mathrm{<span\; class="notranslate">\; LBP</span>}}\end{array}\right.$

In the formula, is a normalization constant, α+β+γ=1; HSV, EO, and LBP represent color, edge, and texture features, respectively.

107: Calculate the Bhattacharyachian distance d between the fused feature and the template feature, perform normalization processing on the Bhattacharyachian distance d, and use the normalized result as the weight of the new candidate target area;

$\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; =</span>\sqrt{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&rho;</span>}}$

$\mathrm{<span\; class="notranslate">\; \&rho;</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\sqrt{{\mathrm{<span\; class="notranslate">\; p</span>}}^{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; q</span>}}^{\mathrm{<span\; class="notranslate">\; m</span>}}}$

Among them, ρ represents the ^Bhattachary coefficient; ^M is the dimension of the color feature histogram or the dimension of the complete edge feature histogram or the dimension of the complete texture feature histogram; mth dimension.

108: Sort the N new candidate target areas according to their weights, and calculate the resampling judgment value N _j , if the resampling judgment value is greater than the resampling judgment threshold, perform resampling, and execute step 109; if not, execute step 109;

in, N is the number of new candidate target regions; is the weight of the i-th new candidate object region.

The resampling decision threshold is obtained based on experience, and the reference value is 50. Resampling is to delete the low-weight candidate target area, and then copy the new candidate target area with the largest weight to replace the deleted new candidate target area.

109: Judging the overlap between the new candidate target area with the maximum weight and the target area, if the overlap rate is less than the overlap rate threshold, perform step 110; otherwise, perform step 111;

Wherein, the overlapping rate threshold is set according to the actual application situation, for example: 0.3, and this embodiment of the present invention does not limit it during specific implementation.

110: Input the multiple area (width LW, height LH, center (x, y), L is a multiple) of the maximum weighted new candidate target area (width W, height H, center (x, y)) to the detector, If the output of the detector is 0, it means the tracking failed; otherwise, the output of the detector is input to the recognizer, if the output of the recognizer is yes, it means that the tracking is successful and the recognizer, template features and target area are updated; if the output is no, it means that a new target is found, end of process;

A detector is a general-purpose classifier that can detect objects of interest and is used to distinguish tracked objects from other objects. The detectors are trained offline and are not updated during tracking. The detector input can be a frame of image, or a region in a frame of image, and the output is the region of the target object in the image.

The recognizer is used to obtain prior information about a specific target during tracking, and it is only updated in specific situations. The positive samples in the discriminator update are composed of new candidate object regions verified by the detector, and the negative samples are randomly selected regions in the background with the same size as the new candidate object regions. The recognizer prevents the occurrence of tracking drift by identifying new candidate object regions that have been verified. The recognizer input is the object region, and the output is a logical value indicating whether the object belongs to the object represented by the recognizer. The feature of the new candidate target area with the maximum weight is used as the updated template feature; the new candidate target area with the maximum weight is used as the updated target area.

111: If the overlap rate is greater than or equal to the overlap rate threshold, it is considered that the tracking is successful, and the recognizer, template features and target area are updated, and the process ends.

Wherein, the operation of updating the recognizer, the template feature, and the target area in this step is consistent with the updating operation in step 110, and will not be described in detail here in this embodiment of the present invention.

Repeat step 104 to step 111 for other frames of the video sequence until the entire video is traversed.

The following specific experiments are used to verify the feasibility of a target tracking method based on multi-feature adaptive fusion and online learning provided by the embodiment of the present invention. In this experiment, people are used as targets for tracking. The specific results are shown in Figures 2 to 7 Show. Among them, the detector adopts the human detector based on the edge gradient histogram feature and the support vector machine model training.

Figure 2, Figure 3 and Figure 4 are the 16th, 19th, and 23rd frames of the video sequence, respectively. The target area was initialized in the 16th frame. In the 19th frame, the target 1 was occluded and the tracking was lost. In the 23rd frame, the occlusion lost target was successfully retrieved through online learning, which proved that this method can effectively avoid the problem of target loss after occlusion.

Figure 5, Figure 6 and Figure 7 are the 291st, 294th and 299th frames of the video sequence respectively. In frame 291, the target area is initialized. In frame 294, target 1 and target 2 are occluded alternately. In frame 299, this method performs accurate and effective tracking, avoiding the problem of easy tracking drift after target staggered occlusion.

references

[1]N.Dalal and B.Triggs.Histograms of oriented gradients for human detection.CVPR,2005.

[2] Kanopoulos N, Vasanthavada N, Baker R.L. Design of an image edge detection filter using the Sobel operator. IEEE Journal of Volume: 23, Issue.1988.

[3]Wang Xiaoyu, Han, Tony X, Yan Shuicheng. An HOG-LBP human detector with partial occlusion handling. Computer Vision, 32-39, 2009.

[4] T. Ahonen, A. Hadid, and M. Pietikainen. Face Recogniton with Local Binary Patterns. Proc. European Conf. Computer Vision, 469-481, 2004.

[5]P.Viola and M.Jones.Rapid object detection using a boosted cascade of simple features.In Proc.CVPR,volume I,511-518,2001.

Those skilled in the art can understand that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the serial numbers of the above-mentioned embodiments of the present invention are for description only, and do not represent the advantages and disadvantages of the embodiments.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within range.

Claims (5)

1. A target tracking method based on multi-feature adaptive fusion and online learning, characterized in that, the method may further comprise the steps: (1) Select the target area from a frame of image, extract the target feature and use it as a template feature; (2) Initialize the recognizer, input the next frame image, and initialize the candidate target area; obtain a new candidate target area according to the transfer formula; (3) extract three kinds of features of color, edge and texture respectively to described new candidate target area; Carry out self-adaptive fusion according to discrimination and correlation of each feature; (4) Calculating the Bhattacharyachian distance between the fused feature and the template feature, and normalizing the Bhattacharyachian distance as the weight of the new candidate target region; (5) N new candidate target areas are sorted according to the weight size, if the resampling judgment value is greater than the resampling judgment threshold, resampling is performed, and step (6) is executed; if not, step (6) is executed; (6) Carry out overlap judgment to the new candidate target area of maximum weight and described target area, if overlap rate is less than overlap rate threshold value execution step (7); Otherwise, execution step (8); (7) Input multiple regions of the new candidate target region with the maximum weight into the detector, if the detector outputs 0, it means that the tracking fails; otherwise, input the output result of the detector to the recognizer, if the recognizer outputs Yes, it means that the tracking is successful and the recognizer, the template feature and the target area are updated; if the output is No, it means that a new target is found, and the process ends; (8) If the overlap rate is greater than or equal to the overlap rate threshold, it is considered that the tracking is successful, and the identifier, the template feature and the target area are updated, and the process ends; in, The discrimination is defined as the degree of similarity between the new candidate target region and the adjacent background on a certain feature, expressed by the Bhattachary coefficient of the two histograms; The correlation is defined as the degree of similarity between the new candidate target region and the template feature with respect to a certain feature, expressed by the Bhattachary coefficient of the two histograms; The update identifier is specifically: the positive sample is composed of the new candidate target area verified by the detector, and the negative sample is a randomly selected area in the background with the same size as the new candidate target area; The updating template feature is specifically: using the feature of the new candidate target region with the maximum weight as the updated template feature; The updating of the target area specifically includes: taking the new candidate target area with the largest weight as the updated target area. 2. a kind of target tracking method based on multi-feature adaptive fusion and online learning according to claim 1, is characterized in that, the described step of extracting target feature and as template feature specifically comprises: 1) Extract color feature information; 2) Extract edge feature information; 3) Extract texture feature information; 4) Fusing the color feature information, the edge feature information and the texture feature information to obtain a target feature histogram as the template feature. 3. A kind of target tracking method based on multi-feature adaptive fusion and online learning according to claim 2, characterized in that, the step of extracting color feature information specifically comprises: 1) Divide the color space into a color area and an achromatic area, perform HSV partitioning on the color area and the achromatic area, obtain Q _H × Q _S color sub-intervals and Q _v achromatic sub-intervals, and divide all The QH×QS color subintervals and the _Qv achromatic subintervals are used as _QH × _{QS + Qv} color intervals _u ; 2) assign different weights to the pixel according to the distance between the pixel and the center point of the target area, and vote for the corresponding color interval u according to the HSV of the pixel; 3) Count the voting values of each color interval to obtain a color feature histogram. 4. A kind of target tracking method based on multi-feature adaptive fusion and online learning according to claim 2, characterized in that, the step of extracting edge feature information specifically comprises: 1) Interpolating the target area to obtain an interpolation area twice as wide and high, and then dividing the target area and the interpolation area into blocks respectively; 2) Calculate the edge strength and direction of each sub-block, divide the edge direction into several direction areas within the range of 0°-360°, vote for the direction area according to the edge strength, and obtain the edge feature histogram of each sub-block; 3) The edge feature histograms calculated by each sub-block are connected to obtain a complete edge feature histogram. 5. A kind of target tracking method based on multi-feature adaptive fusion and online learning according to claim 2, characterized in that, the step of extracting texture feature information specifically comprises: 1) For each sub-block, calculate the local binary pattern feature histogram; 2) Connecting the feature histograms of local binary patterns calculated by each sub-block to obtain a complete texture feature histogram.