CN112927261A - Target tracking method integrating position prediction and related filtering - Google Patents

Target tracking method integrating position prediction and related filtering Download PDF

Info

Publication number
CN112927261A
CN112927261A CN202110249790.7A CN202110249790A CN112927261A CN 112927261 A CN112927261 A CN 112927261A CN 202110249790 A CN202110249790 A CN 202110249790A CN 112927261 A CN112927261 A CN 112927261A
Authority
CN
China
Prior art keywords
target
tracking
sample
frame
filter
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110249790.7A
Other languages
Chinese (zh)
Inventor
卢有亮
罗壹航
陈勇
唐豪
郑伟生
罗建平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Electronic Science and Technology of China
Original Assignee
University of Electronic Science and Technology of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Electronic Science and Technology of China filed Critical University of Electronic Science and Technology of China
Priority to CN202110249790.7A priority Critical patent/CN112927261A/en
Publication of CN112927261A publication Critical patent/CN112927261A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24133—Distances to prototypes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Abstract

The invention discloses a target tracking method integrating position prediction and related filtering, belongs to the technical field of computer vision, and introduces a Kalman filter to correct a tracking result on the basis of tracking by a related filter. At the start of tracking, initial coefficients of the filter are calculated in the initial frame of the video image, according to the position and size of the target in the tag given in advance by the video sequence. In the subsequent frames, the target position of the previous frame is taken as the center, the target range is expanded by 2.5 times to be used as a target search area, the response of a correlation filter is calculated, the position corresponding to the maximum response value is used as an initial target result, the result is used as an observation value, a Kalman filter is used for correcting the tracking result and updating the coefficient of the correlation filter, and the process is repeated until the tracking is finished. The invention combines position prediction and related filtering, and can effectively process the target tracking effect under the influence of factors such as rapid movement, target shielding and the like.

Description

Target tracking method integrating position prediction and related filtering
Technical Field
The invention belongs to the technical field of computer vision, and particularly relates to a target tracking technology.
Background
Target tracking is an important research branch in computer vision technology, and has wide application in the fields of military affairs, industry, security protection, unmanned driving and the like. The common target tracking task refers to single target tracking in particular, namely, on the premise that the initial position and size of a target frame are given in an initial frame of a video, accurate tracking of a target is completed in a subsequent frame. Common target tracking methods are classified into a generating equation and a discriminant equation. The correlation filter is a discriminant tracking method, and a discriminant model of the appearance change of the target and the background is established by establishing the separability of the target and the background and utilizing the autocorrelation of the target, so that the target is stably tracked.
From the 2010 proposal of MOSSE (least output sum of squares and errors) algorithm, the correlation filter becomes the mainstream algorithm in the field of target tracking. The MOSSE algorithm utilizes the idea of minimizing errors to train the classifier continuously during the tracking process to form a model of a correlation filter. The target tracking method comprises the steps of continuously inputting a target image and carrying out correlation operation on the target image and a correlation filter, finding out a point set of a maximum response position after a response value is obtained, namely a target tracking result, continuously updating a filter model according to the tracking result, and stably and efficiently carrying out long-term tracking. The KCF (kernel correlation filtering) algorithm introduces a circulation matrix and a kernel space on the basis of the MOSSE algorithm, improves the tracking precision by extracting the characteristics of a target FHO G (fast gradient histogram), and greatly improves the tracking effect of a correlation filter algorithm. However, the KCF has the disadvantage that the target template is updated in real time when the target is occluded, which may cause false or missed tracking in the case of occlusion.
Kalman filtering is commonly used in engineering applications such as robot navigation, sensor data fusion, and radar tracking, and is also commonly used in computer vision tasks in recent years. The Kalman filtering algorithm utilizes a state equation of a linear system, and predicts the position and the speed of an object in an observation result containing noise through system input and output, so that a more accurate measurement value is obtained.
Disclosure of Invention
In order to solve the technical problems, the invention provides a target tracking method integrating position prediction and related filtering, which effectively improves the target tracking precision under the influence of factors such as rapid movement, target shielding and the like.
The technical scheme adopted by the invention is as follows: a target tracking method fusing position prediction and correlation filtering comprises the following steps:
s1, calculating an initial coefficient of a kernel correlation filter according to the target position and size in a label preset by a video sequence in an initial frame video image;
s2, taking the target position of the previous frame of video image as the center, expanding the target range by 2.5 times to be used as a target search area, and calculating the response of the kernel correlation filter;
s3, taking the position corresponding to the response maximum value as an initial target result, taking the result as an observation value, and correcting the tracking result by using a Kalman filter;
s4, taking the corrected predicted value as a new target center, recording the predicted value into a motion track point set, and selecting an image area with the size 2.5 times of the target range as a new target sample by using the new target center, so as to update the coefficient of the kernel correlation filter;
and S5, repeating the steps S2-S4 until the tracking is finished, and obtaining the target tracking track.
Step S1 specifically includes the following substeps:
s11, obtaining a target selection frame according to a label preset by a video sequence in the initial frame video image, and selecting a video image range which is 2.5 times as large as the target selection frame and contains a background image as a target sample x;
s12, extracting fHoG characteristic x of target sample xfHoGGenerating a training sample C by utilizing the characteristics of the cyclic matrix;
s13, marking the training samples C through Gaussian distribution, and respectively assigning numbers of (0,1) according to the distance between the sample center and the target;
s14, mapping the training samples processed in the step S13 into a kernel space, training a classifier based on ridge regression, minimizing the difference between the sample markers predicted by the classifier and the real markers, solving a kernel correlation filter coefficient alpha, and storing the coefficient as a parameter model.
In step S13, the closer value to the target is closer to 1, and the farther value is closer to 0.
Step S2 specifically includes the following substeps:
s21, selecting a window with the size 2.5 times that of the predicted target frame of the previous frame as a target sample in the subsequent frame;
s22, extracting fHoG characteristics of the target sample and generating a training sample C by utilizing a cyclic matrix; and calculating the response of the kernel correlation filter corresponding to each sample, wherein the maximum response position is the position of the new target selection frame.
The invention has the beneficial effects that: the invention provides a solution that the target can not be effectively tracked when the target moves fast and is shielded in a complex motion scene, and the target tracking precision under the influence of factors such as fast movement, target shielding and the like can be simply and effectively improved by combining position prediction and related filtering, and the introduced algorithm has low time complexity and almost has no influence on the tracking speed of the algorithm.
Drawings
FIG. 1 is an overall flow diagram of the method of the present invention;
FIG. 2 shows the results of the experiment at frame 77 in one tracking example;
FIG. 3 shows the results of an 86 th experiment in one tracking example;
FIG. 4 shows the results of the 91 st frame of the one-pass tracking example;
fig. 5 is a comparison of tracking accuracy of the correlation filter algorithm.
Detailed Description
In order to facilitate the understanding of the technical contents of the present invention by those skilled in the art, the present invention will be further explained with reference to the accompanying drawings.
The present invention will be described with reference to specific embodiments. This example uses an OTB50 data set David3 video sequence with a resolution of 640 x 480 pixels. Experiments were performed in a 4-core 3.6ghz cpu, 4G memory environment. The flow chart of the target tracking method integrating position prediction and related filtering is shown in fig. 1, and specifically comprises the following steps:
the method comprises the following steps: reading the video sequence David3, labeling the pictures sequentially as f1，f2，…，fn]}；
Step two: reading the 1 st frame image f1Given the calibrated target box, target box B1 is [83,200,35,131]I.e. the coordinates of the upper left corner of the target box are (200,83) and the size is (131,35), the target center p1(rounded down) to (265,100). Selecting an image area with the size 2.5 times of the range of a target frame as a target sample x;
step three: extracting fHoG characteristics of the target sample, generating a training sample by using a cyclic matrix, and performing FFT (fast Fourier transform) to convert the sample into a frequency domain X. Training a classifier by utilizing ridge regression of a kernel space to obtain an initial coefficient alpha of a correlation filter;
the generation mode of the target sample fHoG feature training sample C is as follows:
C＝{PuxfHoG|u＝0,1,…,n-1}
wherein x isfHoGAnd (3) generating an n-dimensional fHoG characteristic matrix, wherein P is an n-dimensional cyclic matrix, and the generation of the cyclic matrix is completed by calculating the u-th power of P (1 < u < n).
Figure BDA0002965514090000031
Suppose the observed value of the target tracking system for the input is z, and the transpose of the input weight w is wTThe system function of (f), (z) ═ wTz, given sample X ═ X i1, …, n, given the label y ═ yiThe ridge regression method for i ═ 1, …, n } is:
Figure BDA0002965514090000032
in the formula, xiFor each input sample in X, yiFor the label corresponding to each sample (the sample label is a given content of the dataset, a certain value in the regression problem), λ is the regularization parameter. For sample X, w has a unique closed-form solution: w ═ XHX+λI)-1XHy. Wherein XHTranspose the conjugate of X, i.e. XH＝(X*)T。
By introducing a Fourier coefficient matrix F
Figure BDA0002965514090000041
Where ω is a complex constant ω ═ e-j2πThen w can be expressed as follows:
Figure BDA0002965514090000042
conversion into the frequency domain yields:
Figure BDA0002965514090000043
wherein |, indicates a dot product operation.
Introducing a kernel space approach, for each sample X in the input sample set XiW may be a non-linear function
Figure BDA0002965514090000044
Is linearly expressed as
Figure BDA0002965514090000045
Representing variable as xiThe regression formula can be expressed as
Figure BDA0002965514090000046
Wherein
Figure BDA0002965514090000047
Then
Figure BDA0002965514090000048
Where I is the identity matrix, K is the kernel correlation matrix,
Figure BDA0002965514090000049
α is the filter coefficient.
Figure BDA00029655140900000410
Wherein the content of the first and second substances,
Figure BDA00029655140900000411
a non-linear functional representation representing a variable as z,
Figure BDA00029655140900000412
representing variable as xjA non-linear functional representation of (a);
converting the calculation of alpha into frequency domain to obtain Fourier transform thereof
Figure BDA00029655140900000413
The conjugates are obtained from the above formula
Figure BDA00029655140900000414
Comprises the following steps:
Figure BDA00029655140900000415
the following can be obtained:
Figure BDA00029655140900000416
wherein
Figure BDA00029655140900000417
Is KxxThe fourier transform of (d).
Step four: initial position of targetI.e. the center of the target frame, as the initialization parameter of the position in the kalman filter, and set the speed to 0, at which time the moving state x of the targetkf＝[265；100；0；0]. Setting a state transition matrix a ═ 1,0,1, 0; 0,1,0, 1; 0,0,1, 0; 0,0,0,1]The prediction noise covariance matrix Q is set to diag ([0.05, 0.05,0.05, 0.05)]) Setting an observation matrix H to [1,0, 0, 0; 0,1,0,0]Setting an observation noise covariance matrix as R ═ 2, 0; 0,2]。
Initializing a Kalman filter
For the state of motion of the object
Figure BDA0002965514090000051
Wherein p isrow，pcolIs the center coordinate of the target; v. ofrow，vcolThe initial velocity for the target is 0. A state transition matrix A and an error matrix P of the linear system are set.
And setting an observation matrix D, a prediction noise covariance matrix Q and an observation noise covariance matrix R of the linear system.
Step five: sequentially reading the ith frame image (i)>1)fiWith pi-1As the center, an image of 2.5 times the size of the target range is selected as an input sample x. Extracting fHoG characteristics of the target sample and generating a training sample C. And calculating a correlation response g according to C and the filter coefficient alpha: solving the frequency domain expression G of the response according to the solving method of the content sample response, converting the G into a time domain G through IFFT, and finding the position of the maximum value in the G, namely the tracking result pi。
The response method of the calculation sample combines the solution of the filter coefficient alpha, and converts the response G into the frequency domain to obtain G
Figure BDA0002965514090000052
Wherein the content of the first and second substances,
Figure BDA0002965514090000053
is KxzThe fourier transform of (d).
Step six: will trace the result piAs the observed value z, according to the kalman filtering algorithm in the content of the invention, the kalman gain K is calculated, and the predicted value is updated accordingly
Figure BDA0002965514090000054
And an error matrix P. The specific process is as follows:
setting the motion state of an object as
Figure BDA0002965514090000055
The object motion model is as follows:
state matrix: x is the number ofk＝Axk-1+Buk+wk
Observation matrix: z is a radical ofk＝Dxk+vk
Wherein xkIs a state variable at time k; z is a radical ofkAs a result of observation of the time of day, ukInputting the quantity for the system; w is akAnd vkRespectively satisfying the process noise of normal distribution with mean value of 0 and covariance of Q and the observation noise of normal distribution with mean value of 0 and covariance of R; a is a state transition model, B is an input control model, and D is an observation model.
In the Kalman filtering algorithm, the target state x and the error matrix P are predicted as
Figure BDA0002965514090000056
Pk′＝APk-1′AT+Q
Figure BDA0002965514090000061
Representing a target state xkPredicted value of (A), Pk' indicating error PkPredicted value of (u), uk-1Representing the system input quantity at the k-1 moment;
after the observation results are obtained, the correction values for x and P are
Figure BDA0002965514090000062
Pk＝(1-PkD)Pk′
Wherein KkKalman gain, whose value is:
Kk＝Pk′DT(DPk′DT+R)-1
step seven: will predict the value
Figure BDA0002965514090000063
As a new target centre piAnd recording a motion track point set pos as { p ═ as a result1,…,piAnd selecting an image area with the size 2.5 times of the target range as a new target sample x, so as to update the coefficient alpha of the relevant filter.
As shown in fig. 2, fig. 3, and fig. 4, the tracking effect of the target when encountering the occlusion in the tracking process of this example is shown, it can be seen that after the target passes through the occlusion, the method of the present invention can effectively track the target.
As shown in FIG. 5, the tracking effect of the invention is compared with that of the conventional KCF algorithm, the tracking accuracy of the method of the invention is always superior to that of the conventional KCF algorithm in the process that the threshold value reaches 30, the tracking speed of the KCF algorithm in the data set is 201.93 frames/second, and the tracking speed of the method provided by the invention is 198.64 frames/second without obvious reduction.
It will be appreciated by those of ordinary skill in the art that the embodiments described herein are intended to assist the reader in understanding the principles of the invention and are to be construed as being without limitation to such specifically recited embodiments and examples. Various modifications and alterations to this invention will become apparent to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims (4)

1. A target tracking method fusing position prediction and correlation filtering is characterized by comprising the following steps:
s1, calculating an initial coefficient of a kernel correlation filter according to the target position and size in a label preset by a video sequence in an initial frame video image;
s2, taking the target position of the previous frame of video image as the center, expanding the target range by 2.5 times to be used as a target search area, and calculating the response of the kernel correlation filter;
s3, taking the position corresponding to the response maximum value as an initial target result, taking the result as an observation value, and correcting the tracking result by using a Kalman filter;
s4, taking the corrected predicted value as a new target center, recording the predicted value into a motion track point set, and selecting an image area with the size 2.5 times of the target range as a new target sample by using the new target center, so as to update the coefficient of the kernel correlation filter;
and S5, repeating the steps S2-S4 until the tracking is finished, and obtaining the target tracking track.
2. The target tracking method with fusion of position prediction and correlation filtering according to claim 1, wherein the step S1 specifically includes the following sub-steps:
s11, obtaining a target selection frame according to a label preset by a video sequence in the initial frame video image, and selecting a video image range which is 2.5 times as large as the target selection frame and contains a background image as a target sample x;
s12, extracting fHoG characteristic x of target sample xfHoGGenerating a training sample C by utilizing the characteristics of the cyclic matrix;
s13, marking the training samples C through Gaussian distribution, and respectively assigning numbers of (0,1) according to the distance between the sample center and the target;
s14, mapping the training samples processed in the step S13 into a kernel space, training a classifier based on ridge regression, enabling the difference between the sample mark predicted by the classifier and the real mark to be minimum, and solving a kernel correlation filter coefficient alpha.
3. The method of claim 2, wherein in step S13, the closer to the target, the value is 1, and the farther away, the value is 0.
4. The target tracking method with fusion of position prediction and correlation filtering according to claim 3, wherein the step S2 specifically comprises the following sub-steps:
s21, selecting a window with the size 2.5 times that of the predicted target frame of the previous frame as a target sample in the subsequent frame;
s22, extracting fHoG characteristics of the target sample and generating a training sample C by utilizing a cyclic matrix; and calculating the response of the kernel correlation filter corresponding to each sample, wherein the maximum response position is the position of the new target selection frame.
CN202110249790.7A 2021-03-08 2021-03-08 Target tracking method integrating position prediction and related filtering Pending CN112927261A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110249790.7A CN112927261A (en) 2021-03-08 2021-03-08 Target tracking method integrating position prediction and related filtering

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110249790.7A CN112927261A (en) 2021-03-08 2021-03-08 Target tracking method integrating position prediction and related filtering

Publications (1)

Publication Number Publication Date
CN112927261A true CN112927261A (en) 2021-06-08

Family

ID=76171843

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110249790.7A Pending CN112927261A (en) 2021-03-08 2021-03-08 Target tracking method integrating position prediction and related filtering

Country Status (1)

Country Link
CN (1) CN112927261A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108198209A (en) * 2017-12-22 2018-06-22 天津理工大学 It is blocking and dimensional variation pedestrian tracking algorithm
CN108876816A (en) * 2018-05-31 2018-11-23 西安电子科技大学 Method for tracking target based on adaptive targets response
CN110956653A (en) * 2019-11-29 2020-04-03 中国科学院空间应用工程与技术中心 Satellite video dynamic target tracking method with fusion of correlation filter and motion estimation
CN111145216A (en) * 2019-12-26 2020-05-12 电子科技大学 Tracking method of video image target

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108198209A (en) * 2017-12-22 2018-06-22 天津理工大学 It is blocking and dimensional variation pedestrian tracking algorithm
CN108876816A (en) * 2018-05-31 2018-11-23 西安电子科技大学 Method for tracking target based on adaptive targets response
CN110956653A (en) * 2019-11-29 2020-04-03 中国科学院空间应用工程与技术中心 Satellite video dynamic target tracking method with fusion of correlation filter and motion estimation
CN111145216A (en) * 2019-12-26 2020-05-12 电子科技大学 Tracking method of video image target

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
JOÃO F. HENRIQUES，RUI CASEIRO，PEDRO MARTINS，JORGE BATISTA: "High-Speed Tracking with Kernelized Correlation Filters", 《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》 *
栗宝俊: "基于多表观模型的长期视觉目标跟踪算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *
毛贻豪: "基于相关滤波的抗遮挡行人跟踪方法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Similar Documents

Publication Publication Date Title
CN108734723B (en) Relevant filtering target tracking method based on adaptive weight joint learning
CN108549839B (en) Adaptive feature fusion multi-scale correlation filtering visual tracking method
CN111127513A (en) Multi-target tracking method
CN110276785B (en) Anti-shielding infrared target tracking method
CN109801311B (en) Visual target tracking method based on depth residual error network characteristics
CN110363165B (en) Multi-target tracking method and device based on TSK fuzzy system and storage medium
CN111080675A (en) Target tracking method based on space-time constraint correlation filtering
CN112634333B (en) Tracking device method and device based on ECO algorithm and Kalman filtering
CN110070578B (en) Loop detection method
Nunes et al. Entropy minimisation framework for event-based vision model estimation
CN109685830B (en) Target tracking method, device and equipment and computer storage medium
Dai et al. Object tracking in the presence of shaking motions
CN110706253B (en) Target tracking method, system and device based on apparent feature and depth feature
CN107194950B (en) Multi-person tracking method based on slow feature analysis
CN112183675B (en) Tracking method for low-resolution target based on twin network
CN111161323B (en) Complex scene target tracking method and system based on correlation filtering
Wang et al. Improving RGB-D SLAM accuracy in dynamic environments based on semantic and geometric constraints
CN117011381A (en) Real-time surgical instrument pose estimation method and system based on deep learning and stereoscopic vision
CN112927261A (en) Target tracking method integrating position prediction and related filtering
CN113033356B (en) Scale-adaptive long-term correlation target tracking method
CN107194947B (en) Target tracking method with self-adaptive self-correction function
Shan et al. Visual tracking using IPCA and sparse representation
Liu et al. Fast tracking via spatio-temporal context learning based on multi-color attributes and pca
Li et al. Multitarget tracking of pedestrians in video sequences based on particle filters
CN110147747B (en) Correlation filtering tracking method based on accumulated first-order derivative high-confidence strategy

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20210608

RJ01 Rejection of invention patent application after publication