CN111104948A - Target tracking method based on adaptive fusion of double models - Google Patents

Abstract

The embodiment of the present invention proposes a dual-model-based adaptive fusion target tracking method. The dual-model-based adaptive fusion target tracking method utilizes the adaptive fusion coefficient based on relative confidence, so that the correlation filter response and the color classifier response can be optimally fused, thereby fully showing the tracking advantages of the respective models. It effectively solves the problem that the fusion coefficient of the correlation filter and the color classifier in the existing Staple target tracking method is constant and does not fully demonstrate the advantages of the correlation filter tracking model and the color classifier tracking model.

Description

A Target Tracking Method Based on Dual Model Adaptive Fusion

technical field

The invention relates to the technical field of computer vision, in particular to a dual-model-based adaptive fusion target tracking method.

Background technique

Object tracking is one of the main research directions in the field of computer vision. The fields involved in target tracking include digital image processing, machine learning, pattern recognition, neural networks, deep learning, etc., and it has broad prospects for development in many application fields such as video surveillance and intelligent robots.

In recent years, detection-based target tracking methods have been greatly developed, and one of the most mainstream research directions is the correlation filter-based target tracking method. In 2014, Henriques et al. extended the single-channel grayscale feature used by MOSSE and CSK to a multi-channel histogram of directional gradient feature (HOG), and mapped the feature to a high-dimensional space with a kernel technique, thus proposing the KCF algorithm. The proposal of KCF makes the correlation filtering target tracking method develop rapidly. In 2015, the SRDCF proposed by Danelljan et al. used spatial regularization to solve the inherent boundary effect of the correlation filter, and came out on top in the VOT2015 target tracking competition. In 2016, Luca et al. proposed the Staple algorithm based on the linear kernel version of KCF, DCF. The Staple algorithm improved the performance of the tracking algorithm by solving two ridge regression equations combined with correlation filtering and color classifiers, and obtained a very good result. However, the fusion coefficient of the correlation filter and the color classifier in the Staple algorithm is constant, which leads to the Staple algorithm not fully showing the advantages of the correlation filter tracking model and the color classifier tracking model.

Therefore, in view of the problem that the existing Staple algorithm does not fully demonstrate the advantages of the correlation filter tracking model and the color classifier tracking model, it is necessary to provide a method that can fully demonstrate the advantages of the correlation filter tracking model and the color classifier tracking model. target tracking method.

SUMMARY OF THE INVENTION

Aiming at the problem that the existing Staple algorithm does not fully demonstrate the advantages of the correlation filter tracking model and the color classifier tracking model, an embodiment of the present invention proposes a dual-model-based adaptive fusion target tracking method. The target tracking method based on the dual-model adaptive fusion proposed in the embodiment of the present invention utilizes the adaptive fusion coefficient based on relative confidence, so that the correlation filter and the color classifier can perform optimal fusion, and then fully demonstrate the performance of the respective models. Track advantage.

The specific scheme of the dual-model-based adaptive fusion target tracking method is as follows: a dual-model-based adaptive fusion target tracking method, comprising step S1: obtaining initial target information according to the initial frame; step S2: separately from the foreground The color histogram is extracted from the region and the background region, and the ridge regression equation is used to solve and train the color classifier; step S3: extract features from the relevant filtering region, and train the relevant filter; step S4: initialize the scale filter, extract different scale image blocks to train the scale filter; Step S5: use the color classifier to detect the target, and obtain the response of the color classifier; Step S6: detect the target with the correlation filter in the correlation filter area, and obtain the response of the correlation filter; Step S7: according to Calculate the relative confidence in the response of the correlation filter, calculate the adaptive fusion coefficient based on the relative confidence, and use the adaptive fusion coefficient to fuse the response of the correlation filter and the response of the color classifier to obtain the position of the detection target; step S8: Extract the features of the target, and update the correlation filter and color classifier; Step S9: Detect scale changes, update the target, foreground area, background area and scale filter; Step S10: Repeat steps S5 to S9 , until the video ends.

Preferably, the target initial information includes target position, target length, and target width.

Preferably, the process of extracting the color histogram in the step S2 is as follows: dividing the color space into several color intervals, defining each color interval as a column of the histogram, and counting the foreground area or the background area falling within each The number of pixels in the histogram.

Preferably, the value of the width of the bar of the color histogram is 8.

Preferably, the expression of the ridge regression equation is:

χ _t represents the training sample and its corresponding regression value, β is the color classifier to be solved, L _hist represents the loss function of the classifier, and λ _hist is the regularization coefficient.

Preferably, the method of training the correlation filter is to minimize the following formula:

Among them, f represents the sample, d is the dimension of the sample f, h is the correlation filter, g represents the output required by the correlation filter, g is the Gaussian function, * represents the convolution operation, and λ is the regularization coefficient.

Preferably, the expression of the relative confidence is:

Among them, r _t is the relative confidence of the detection result of the correlation filter in the t-th frame relative to the global, APCE _t is the average correlation peak energy of the t-th frame response y _t , and the specific calculation expression of APCE _t is:

Preferably, the expression of the adaptive fusion coefficient is:

Among them, α _t is the adaptive fusion coefficient at the t-th frame, ρ is the influence factor of the relative confidence, r _t is the relative global confidence of the detection result of the correlation filter at the t-th frame, and α is a constant weighting coefficient.

Preferably, the adaptive fusion coefficient is used to fuse the response of the correlation filter and the response of the color classifier, and the specific calculation expression is as follows:

response=(1-αt)response_cf+αt·response_p,

Among them, response_cf is the response of the correlation filter, response_p is the response of the color classifier, α _t is the adaptive fusion coefficient at the t-th frame, and response is the final response.

Preferably, the impact factor ρ is used to adjust the weight of the correlation filter discrimination result and the color classifier discrimination result.

As can be seen from the above technical solutions, the embodiments of the present invention have the following advantages:

The embodiment of the present invention proposes a dual-model-based adaptive fusion target tracking method, which enables the correlation filter and the color classifier to perform optimal fusion through an adaptive fusion coefficient based on relative confidence, thereby fully displaying the tracking advantages of the respective models.

Description of drawings

1 is a schematic flowchart of a dual-model-based adaptive fusion target tracking method provided in an embodiment of the present invention;

FIG. 2 is a schematic diagram of the comparison of experimental results between a dual-model-based adaptive fusion target tracking method and other target tracking methods proposed in an embodiment of the present invention on OTB2013;

3 is a schematic diagram of qualitative comparison between a dual-model-based adaptive fusion target tracking method proposed in an embodiment of the present invention and DSST and KCF tracking in different images;

FIG. 4 is a schematic diagram showing another step flow of the embodiment shown in FIG. 1 .

Detailed ways

In order to make those skilled in the art better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only Embodiments are part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The terms "first", "second", "third", "fourth", etc. (if present) in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to Describe a particular order or sequence. It is to be understood that data so used may be interchanged under appropriate circumstances so that the embodiments described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

As shown in FIG. 1 , an embodiment of the present invention provides a dual-model-based adaptive fusion target tracking method. The method includes ten steps, and the specific content of each step is as follows.

Step S1: Acquire initial information of the target according to the initial frame. The initial information of the target includes the target position, the length of the target, and the width of the target. Further, in step S1, it also includes some initialization parameters and normal initialization operations of the initialization area.

Step S2: extracting color histograms from the foreground area and background area, respectively, and using the ridge regression equation to solve and train the color classifier.

The specific process of extracting a color histogram from the foreground area or from the background area is: divide the color space into several color intervals, define each color interval as a histogram of the histogram, and count the foreground area or background area falling in each The number of pixels in the histogram. In a specific embodiment, the value of the width of the bar of the color histogram is 8.

The response of the color classifier is obtained by solving the ridge regression equation. The specific expression of the ridge regression equation is shown in Equation 1:

χ _t represents the training sample and its corresponding regression value, β is the color classifier to be solved, L _hist represents the loss function of the classifier, and λ _hist is the regularization coefficient.

其中包括正样本(p,1)。X表示图像。所有采样图像的损失可以表示为：可将公式1转变成公式2：Let (q, y) ∈ W denote a series of rectangular sampling boxes q and their corresponding regression labels

which includes positive samples (p,1). X represents an image. The loss of all sampled images can be expressed as: Equation 1 can be transformed into Equation 2:

l _hist (x,p,β)=∑ _(q,y)∈W (β ^T [∑ _u∈H ψ _T(x,q) [u]]-y) ² (Equation 2)

ψ _{T(x, q)} represents the feature transformation of an M channel, and β is the model. Apply linear regression on each pixel u to make the regression value of the pixel belonging to the background area B to 0, the regression value of the pixel belonging to the foreground area O to be 1, and use ψ as the abbreviation of ψ _{T(x, q)} , then for The loss function for a single image can be written as:

In the above formula, O represents the rectangular foreground area closely surrounding the target, and B represents the rectangular background area containing the target.

In the embodiment of the present invention, the color image adopts the RGB color space, therefore, the RGB color histogram is used as a feature. The loss function is decomposed into the sum of each histogram in the histogram, and the preferred value in the embodiment of the present invention is M=32.

β ^T ψ[u] can be quickly obtained by constructing a lookup table k, which maps the pixel value u to the serial number of the histogram to which it belongs, that is, back-projecting with a color histogram. Let β ^T ψ[u]=β ^k(u) , Equation 4 can be obtained:

Wherein, N ^j (A)={x _i ∈ A:k(u)=j} is the sum of the number of elements in the j-th histogram in the area A.

Therefore, the solution to the ridge regression problem of Equation 1 is shown in Equation 5:

对

使用积分图计算,获得颜色分类器的响应。in,

right

Using the integral plot calculation, obtain the response of the color classifier.

Step S3: Extract features from the correlation filter region, and train the correlation filter. The sample template x is extracted according to the target center, and a large number of training samples x _i are constructed by cyclic shift. Extract multi-channel histogram of oriented gradient features (HOG) and train to generate correlation filters.

The correlation filter can be obtained by solving a ridge regression equation. For a sample f composed of d-dimensional features, a d-dimensional correlation filter h can be trained by minimizing Equation 6:

Among them, f represents the sample, d is the dimension of the sample f, h is the correlation filter, g represents the output required by the correlation filter, g is the Gaussian function, * represents the convolution operation, and λ is the regularization coefficient. λ is used to prevent overfitting.

Step S4: Initialize the scale filter, and extract image blocks of different scales to train the scale filter. With the target position determined in the previous frame as the center, a series of image block features of different scales are extracted to construct a feature pyramid. Taking H×W as the target size, a total of S image blocks of size an H×a ⁿ W are ^extracted near the target position, and a represents the scale coefficient. The specific expression of n is shown in Equation 7:

Wherein, in this embodiment, let S=33. Of course, in other embodiments, the specific value of S may also be other numbers.

Step S5: Use the color classifier to detect the target, and obtain the response of the color classifier. After minimizing Equation 6, convert Equation 6 to the frequency domain to obtain the expression of the filter H ^l in the frequency domain. The specific expression is shown in Equation 8:

表示F^k对应的复共轭。where the capital letters in Equation 8 mean the corresponding discrete Fourier transform,

represents the complex conjugate corresponding to F ^k .

Step S6: use the correlation filter to detect the target in the correlation filter area, and obtain the response of the correlation filter. The response of the correlation filter can be obtained by doing the inverse Fourier transform of Equation 8.

Step S7: Calculate the relative confidence according to the response of the correlation filter, calculate the adaptive fusion coefficient based on the relative confidence, use the adaptive fusion coefficient to fuse the response of the correlation filter and the response of the color classifier, and obtain the detection target s position.

Out of the idea of combining the two tracking models to achieve complementary advantages, the Staple target tracking method integrates the response_cf of the correlation filter and the response_p of the color classifier through a weighted average of the constant coefficient α. The specific combination expression is as shown in Equation 9. Show:

response=(1-α)response_cf+α·response_p (Equation 9)

where α is a constant coefficient, response_cf is the response of the correlation filter, and response_p is the response of the color classifier. Although this weighted integration method effectively fuses two complementary models, only one fixed fusion coefficient leads to the failure of the optimal combination of the correlation filter and the color classifier. In response to this problem, the embodiments of the present invention propose an adaptive fusion coefficient based on relative confidence.

In the embodiment of the present invention, the average correlation peak energy (APCE for short) is used to implement self-adaptive adjustment of the fusion coefficient. The average correlation peak energy (APCE for short) is an index used to evaluate the confidence of the detection result of the correlation filter. The larger the average correlation peak energy (APCE for short), the higher the confidence of the detection result. The expression for the APCE of frame t in response to y _t is shown in Equation 10:

表示响应y_t中的最大值，

表示响应y_t中的最小值，mean表示求均值。in,

represents the maximum value in the response y _t ,

represents the minimum value in the response y _t , and mean represents the mean value.

The expression of the relative confidence of the detection result of the correlation filter relative to the global at the t-th frame is shown in Equation 11:

Among them, r _t represents the relative confidence of the detection result of the correlation filter relative to the global at the t-th frame.

Therefore, the constant coefficient α in Equation 9 can be represented by the adaptive fusion coefficient α _t . The specific expression of the adaptive fusion coefficient α _t is shown in Equation 12:

Among them, α _t is the adaptive fusion coefficient at the t-th frame, ρ is the influence factor of the relative confidence, r _t is the relative global confidence of the detection result of the correlation filter at the t-th frame, and α is a constant weighting coefficient. The impact factor ρ is used to adjust the weights of the correlation filter discrimination results and the color classifier discrimination results. When the relative confidence of the detection result of the correlation filter is greater than 1, the judgment result of the correlation filter is more trusted, otherwise the judgment result of the color classifier is more trusted.

Step S8: Extract the features of the target, and update the correlation filter and color classifier.

Step S9: Detect scale change, update the target, foreground area, background area and scale filter.

At the new position, 33 image blocks with different scales are extracted, and the 33 image blocks with different scales are adjusted to the same size, and candidate scale images are generated by cyclic shift. The scale correlation filter is called to detect the candidate scale images, and the scale with the largest response is selected as the new scale. Update the target, foreground area and background area.

Step S10: Repeat steps S5 to S9 until the video ends.

The embodiment of the present invention proposes a dual-model-based adaptive fusion target tracking method. The method enables the correlation filter and the color classifier to perform optimal fusion through the adaptive fusion coefficient based on relative confidence, and then fully Demonstrate the tracking advantages of the respective models.

The dual-model-based adaptive fusion target tracking method proposed in the embodiment of the present invention runs at a speed of up to 28 frames per second using Matlab R2016a on a computer with an I7-4710HQ2.5GHZ processor and 8G memory.

As shown in FIG. 2 , a schematic diagram of the comparison of the experimental results of a dual-model-based adaptive fusion target tracking method proposed in an embodiment of the present invention and other target tracking methods on OTB2013. It can be seen from the comparison in FIG. 2 that the target tracking method based on the dual-model adaptive fusion proposed in the embodiment of the present invention (the Our algorithm is shown in the figure), compared with the original algorithm Staple, the accuracy and success rate are improved by 1.9% and 1.9% respectively. 2%. As shown in FIG. 3 , a schematic diagram of qualitative comparison between a dual-model-based adaptive fusion target tracking method proposed in an embodiment of the present invention and DSST and KCF tracking in different images. It can be seen from FIG. 3 that the target tracking method based on the dual-model adaptive fusion provided by the embodiment of the present invention can track the corresponding target more accurately.

As shown in FIG. 4 , it is a schematic diagram showing another step flow of the embodiment shown in FIG. 1 . Initialize after starting tracking; train three filters after initialization: training scale filter, training color classifier, and training correlation filter; use the trained correlation filter to detect the target to obtain the correlation filter response; use the trained correlation filter The color classifier detects the target to obtain the classifier response; calculates the self-fusion system, fuses the correlation filter response and the classifier response with the self-fusion coefficient, and obtains the fused target position; uses the trained scale filter to detect the scale change; Update some columns; judge whether the video is over, if the video is not over, continue, otherwise end.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

Although the embodiments of the present invention have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and should not be construed as limiting the present invention. Embodiments are subject to variations, modifications, substitutions and variations.

Claims (10)

1. A target tracking method based on adaptive fusion of dual models, characterized in that the method comprises: Step S1: obtain the initial information of the target according to the initial frame; Step S2: extract the color histogram from the foreground area and the background area respectively, and use the ridge regression equation to solve and train the color classifier; Step S3: extracting features from the relevant filtering area, and training the relevant filter; Step S4: initialize the scale filter, and extract image blocks of different scales to train the scale filter; Step S5: utilize the color classifier to detect the target, and obtain the response of the color classifier; Step S6: use the correlation filter to detect the target in the correlation filter area, and obtain the response of the correlation filter; Step S7: Calculate the relative confidence according to the response of the correlation filter, calculate the adaptive fusion coefficient based on the relative confidence, use the adaptive fusion coefficient to fuse the response of the correlation filter and the response of the color classifier, and obtain the detection target s position; Step S8: extract the feature of the target, and update the correlation filter and the color classifier; Step S9: detecting scale changes, updating the target, foreground area, background area and scale filter; Step S10: Repeat steps S5 to S9 until the video ends. 2 . The dual-model-based adaptive fusion target tracking method according to claim 1 , wherein the initial target information includes target position, target length, and target width. 3 . 3. a kind of target tracking method based on the self-adaptive fusion of dual models according to claim 1, is characterized in that, the process of extracting color histogram in described step S2 is: the color space is equally divided into several color intervals , define each color interval as a bar of the histogram, and count the number of pixels in the foreground area or background area that fall in each bar. 4 . The dual-model-based adaptive fusion target tracking method according to claim 3 , wherein the value of the width of the histogram of the color histogram is 8. 5 . 5. a kind of target tracking method based on the self-adaptive fusion of dual models according to claim 1, is characterized in that, the expression of described ridge regression equation is:

χ _t represents the training sample and its corresponding regression value, β is the color classifier to be solved, L _hist represents the loss function of the classifier, and λ _hist is the regularization coefficient. 6. a kind of target tracking method based on the self-adaptive fusion of dual models according to claim 1, is characterized in that, the method for training correlation filter is to minimize following formula:

Among them, f represents the sample, d is the dimension of the sample f, h is the correlation filter, g represents the output required by the correlation filter, g is the Gaussian function, * represents the convolution operation, and λ is the regularization coefficient. 7. a kind of target tracking method based on dual-model adaptive fusion according to claim 1, is characterized in that, the expression of described relative confidence is:

Among them, r _t is the relative confidence of the detection result of the correlation filter in the t-th frame relative to the global, APCE _t is the average correlation peak energy of the t-th frame response y _t , and the specific calculation expression of APCE _t is:

8. a kind of target tracking method based on dual-model adaptive fusion according to claim 7, is characterized in that, the expression of described adaptive fusion coefficient is:

Among them, αt is the adaptive fusion coefficient at the t-th frame, ρ is the influence factor of the relative confidence, rt is the relative global confidence of the detection result of the correlation filter at the _t -th frame, and α is a constant weighting coefficient. 9. a kind of target tracking method based on dual-model adaptive fusion according to claim 8, is characterized in that, adopting described adaptive fusion coefficient to fuse the response of correlation filter and the response of color classifier, the concrete calculation expression The formula is as follows: response=(1-α _t )response_cf+α _t ·response_p, Among them, response_cf is the response of the correlation filter, response_p is the response of the color classifier, α _t is the adaptive fusion coefficient at the t-th frame, and response is the final response. 10 . The dual-model-based adaptive fusion target tracking method according to claim 8 , wherein the influence factor ρ is used to adjust the weight of the correlation filter discrimination result and the color classifier discrimination result. 11 .

Images