CN113838093B - Self-adaptive multi-feature fusion tracking method based on spatial regularization correlation filter - Google Patents
Self-adaptive multi-feature fusion tracking method based on spatial regularization correlation filter Download PDFInfo
- Publication number
- CN113838093B CN113838093B CN202111121555.8A CN202111121555A CN113838093B CN 113838093 B CN113838093 B CN 113838093B CN 202111121555 A CN202111121555 A CN 202111121555A CN 113838093 B CN113838093 B CN 113838093B
- Authority
- CN
- China
- Prior art keywords
- tracking
- target
- correlation filter
- feature
- frame image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 38
- 238000000034 method Methods 0.000 title claims abstract description 22
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 53
- 238000012549 training Methods 0.000 claims description 6
- 230000003044 adaptive effect Effects 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 5
- 239000013598 vector Substances 0.000 claims description 5
- 230000003190 augmentative effect Effects 0.000 claims description 2
- 230000001419 dependent effect Effects 0.000 claims description 2
- 239000011159 matrix material Substances 0.000 claims description 2
- 238000004364 calculation method Methods 0.000 abstract description 7
- 238000009825 accumulation Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 abstract description 2
- 230000000007 visual effect Effects 0.000 description 12
- 238000005286 illumination Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000000295 complement effect Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012417 linear regression Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000003909 pattern recognition Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a self-adaptive multi-feature fusion tracking method based on a spatial regularization correlation filter, which comprises the steps of adopting a residual convolution network to extract the depth convolution feature of a target, adopting a gradient histogram and a gray level diagram to extract the manual feature of the target, carrying out feature fusion on the depth convolution feature and the manual feature, respectively sending the fusion feature of a first frame image and the fusion feature of a current frame image tracking result into two spatial regularization correlation filters, and carrying out accumulation summation on confidence values of the two spatial regularization correlation filters to obtain a t+1 frame tracking result. The invention can effectively, accurately and robustly express the appearance of the target, and improves the tracking precision and the robustness while reducing the calculation cost, thereby improving the tracking performance of the tracking algorithm.
Description
Technical Field
The invention relates to the field of computer vision, in particular to a self-adaptive multi-feature fusion tracking method based on a spatial regularization correlation filter.
Background
Visual object tracking is a key problem in the field of computer vision, particularly for continuous tracking of objects in video sequences. Visual target tracking involves almost every aspect in a real life scenario, from human-machine interaction to autopilot. In addition, attention has been paid to various practical applications such as medical diagnosis, activity recognition and video monitoring. Most importantly, continuous monitoring of moving objects by visual object tracking technology has a wider range of applications than searching for objects by static images, which makes visual object tracking a very active research area in computer vision, with many new tracking algorithms being proposed each year.
After we specify an initial bounding box for a target object in the first frame of the video sequence, the visual target tracking algorithm is responsible for estimating the bounding box for this target object in subsequent consecutive video frames. A good tracking algorithm needs to achieve a balance between performance such as accuracy, robustness and efficiency in all practical application fields. Although tracking performance has improved significantly over the last decade, visual object tracking remains a challenging task due to problems with occlusion, scale changes, illumination changes, low resolution, fast motion, motion blur, and distortion in video scenes.
Currently, although various tracking algorithms have been proposed, most share similar components. In all tracking algorithms, one of the key factors that improves tracking performance is the apparent representation of the target object. Existing visual target tracking algorithms can be broadly divided into two categories according to different appearance representation strategies: tracking algorithms based on traditional manual features and tracking algorithms based on deep convolution features. Tracking algorithms based on traditional manual features use underlying features such as SIFT, gradient histogram, color histogram, etc. to represent local features of the target object. The deep convolution feature may learn global and local features that best fit to represent the appearance of an object from a given image, as opposed to being based on traditional manual features. Thus, a tracking algorithm based on a depth convolution feature appearance model may achieve more accurate, robust tracking performance because the depth convolution features may better represent the appearance of the target. However, the tracking algorithm based on the convolution structure has low calculation efficiency and large calculation amount, and does not meet the real-time performance requirement between continuous video frames.
While multi-feature fusion-based representations have proven to be an effective method of improving the tracking performance of tracking algorithms, these existing multi-feature fusion-based tracking algorithms simply fuse together various different manual features or extract multi-layer deep convolutional features in a convolutional network. In various challenging video sequences, previously proposed tracking algorithms based on a variety of manual features or multi-layer deep convolution features are prone to tracking failures.
Disclosure of Invention
The technical problem solved by the invention includes that the existing tracking algorithm based on manual characteristics has high calculation speed, but the tracking algorithm based on manual characteristics adopts low-level characteristics to represent local information of a target object, so that the target object cannot be accurately described; the existing tracking algorithm based on the depth convolution features has low calculation efficiency and large calculation amount, does not meet the real-time performance requirement between continuous video frames, and is easy to fail tracking. The self-adaptive multi-feature fusion visual target tracking method based on the spatial regularization correlation filter fully utilizes the complementary advantages of the traditional manual features and the depth convolution features, can effectively, accurately and robustly represent the appearance of the target, reduces the calculation cost and improves the tracking precision and the robustness at the same time, thereby improving the tracking performance of a tracking algorithm.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: the adaptive multi-feature fusion tracking method based on the spatial regularization correlation filter comprises the following steps:
(1) And reading the video frame, determining a tracking target, and marking the tracking target by a target frame in the first frame image.
(2) And extracting the depth convolution characteristics of the target by adopting a residual convolution network, and extracting the manual characteristics of the target by adopting a gradient histogram and a gray level map.
(3) And carrying out feature fusion on the depth convolution features and the manual features.
(4) And respectively sending the fusion characteristic of the first frame image and the fusion characteristic of the tracking result of the current frame image into two spatial regularization correlation filters.
(5) And carrying out accumulated summation on confidence values of the two spatial regularization correlation filters to obtain a tracking result of the t+1 frame.
By adopting the technical scheme, the invention has the following beneficial technical effects.
The tracking algorithm of the invention achieves satisfactory tracking performance on five reference data sets of OTB-2015, sample-Color, PTB-TIR, UAV20L and VOT-2016. Experimental results show that the tracking performance of the tracking algorithm in the long-term video sequence, the short-term video sequence and the infrared video sequence is remarkably improved. Experiments are carried out on the highly crowded video scene on the sample-Color reference data set, so that the effectiveness, accuracy and robustness of the tracking algorithm are further verified.
Specifically, the invention fully utilizes the complementary advantages of the multi-layer deep convolution feature and the manual feature, and uses the feature fusion technology based on the manual feature and the deep convolution feature representation. This feature fusion-based representation is extended to multiple color channels and the results are concatenated together to form a new representation. Through the feature fusion technology, the tracking algorithm can accurately and robustly learn the appearance model of the target, and has obvious advantages for improving tracking performance.
Second, an alternate direction multiplier-algorithm (alternating direction algorithm of multipliers, ADMM) is introduced into the spatially regularized correlation filter, solving the problem of how to accurately search for targets in complex contexts and how to remove the influence of redundant boundaries.
Drawings
FIG. 1 is a diagram of an overall tracking framework of the present invention.
Detailed Description
Referring to fig. 1, the method of the present invention comprises the steps of:
(1) Reading a video frame, determining a tracking target, and marking the tracking target by a target frame in a first frame image;
(2) Extracting depth convolution characteristics of a target by adopting a residual convolution network based on ResNet-101, and extracting manual characteristics of the target by adopting a gradient histogram (gradient of histogram, HOG) and a gray level map (Grayscale);
(3) Feature fusion is carried out on the depth convolution feature and the manual feature,
(4) Respectively sending the fusion characteristics of the first frame image and the fusion characteristics of the tracking result of the current frame image into two spatial regularization correlation filters;
(5) And carrying out accumulation summation on the confidence values of the two spatial regularization correlation filters, and optimizing the final confidence values through a Newton algorithm to obtain a more accurate tracking result of the t+1 frame.
More specifically, the residual convolution network employs a residual convolution network based on ResNet-101 to extract depth convolution features of the target, and employs a manual feature based on gradient histograms (gradient of histogram, HOG) and gray scale maps (Grayscale) to extract the target. In FIG. 1 (e), the parameter set-up, training and updating of the depth residual convolutional network ResNet-101 of the present invention is consistent with the description in the literature "K.He, X.Zhang, S.Ren, J.Sun, deep residual learning for image recognition, in: proceedings of the IEEE conference on computer vision and pattern recognition,2016, pp.770-778.
The fusion process of the depth convolution feature and the traditional manual feature comprises the steps of respectively extracting the convolution feature and the traditional manual feature by using a tracking result of a first frame image with a target frame and a tracking result of a current frame, and fusing the manual feature serving as a layer of the convolution feature with the depth convolution feature. The target frame is indicated by red (small frame of the first frame image in fig. 1), and the region containing the target sampled based on the first frame image of the video sequence is indicated by yellow dotted line (larger dotted frame of the first frame image in fig. 1), as shown in fig. 1 (a). In general, the first frame image is important for the tracking algorithm, as this corresponds to the actual target position and target size. Thus, the present invention uses the first frame image information to correct for target drift in each frame. In order to better handle the change of the target size, 5 image blocks with different sizes are sampled for the t frame image, wherein the scale factor is 1.01 and the scale step length is 5. Then, at the t-th frame, depth residual convolution features and manual fusion features are extracted for these 5 image blocks of different sizes, respectively, as shown in fig. 1 (d) and 1 (e). Finally, in order to effectively utilize the feature fusion technology of the depth convolution feature and the manual feature for tracking, the invention provides a feature fusion learning framework of two-path spatial canonical correlation filtering. The fusion characteristics of the first frame image and the fusion characteristics of the tracking result of the current frame image are respectively sent into two spatial regularization correlation filters, as shown in fig. 1 (f). The spatial correlation filter values of the first frame image and the current frame image are added and then the newton value is obtained to obtain the tracking result of the next video frame t+1, and the tracking result of the first frame image and the current frame image is utilized to continuously update the subsequent frame of the video sequence, as shown in fig. 1 (g).
When the tracking algorithm fails to track some video frames due to occlusion, deformation, rapid motion and the like, the tracking algorithm of the invention can re-track the rest of the video sequence by the real target frame based on the first frame image. If a tracking failure is detected, the first frame image and the last run containing the next video frame image are used for repositioning and the process is repeated until the last frame image of the video sequence.
For visual target tracking, it is of paramount importance to learn a robust and accurate appearance model in various complex environments. At the same time, accurate and robust representation of the target object is also very important for improved tracking performance, although this is often at the expense of efficiency at runtime. In the tracking algorithm of the invention, the basic idea of feature learning is to take the fusion feature of the deep convolution feature and the traditional manual feature as the appearance feature of the target object. By fusing traditional manual features with deep convolution features, feature representation of the target appearance is enhanced. In addition, the fusion of the deep convolution features and conventional manual features provides a more comprehensive appearance representation for more accurate and robust tracking results.
The deep convolution feature is based on the residual network ResNet-101.ResNet-101 is widely used in the field of image classification. ResNet-101 is incorporated into the visual target tracking algorithm to accomplish the tracking task. Manual features are based on HOG and Grayscale, which are two methods commonly used in conventional tracking algorithms to represent the appearance of a target object. In general, the depth convolution feature is insensitive to illumination changes, so that in an indoor environment, a tracking algorithm based on the depth convolution feature has difficulty in achieving ideal tracking performance. However, tracking algorithms based on manual features of HOG and granslide are very useful when the target object is subject to severe illumination changes, full or partial occlusion, and when the target object is moving outside the camera's field of view. This is because HOG and Grayscale have strong invariance to high light, illumination intensity and shadow induced illumination changes. Therefore, in the characteristic learning process, the invention fully utilizes the complementary advantages of the deep convolution characteristic and the traditional manual characteristic to express the appearance characteristic of the moving object.
For manual features, the present invention uses a 31-dimensional HOG feature and 5-scale gray feature vectors obtained from a 4x 4 grid. The depth convolution feature is then combined with the center view feature as the other channel feature to describe the appearance of the target object.
After feature learning, two types of features can be obtained to describe the target object. The two types of fusion features are transformed into successive spatial domains by two spatially regularized correlation filters. For each characteristic channel k, x is used k The representation of the sample is made,is a sample set used to train the multichannel correlation filter. t represents the t-th video frame, every training sample with k channels +.>Is composed of D-dimensional feature maps of size mxn extracted from a target area of each video frame image. Correlation filter y k ∈R D Is a gaussian tag function. For any spatial position (m, n) ∈Ω in each video frame image: = {0, & gt, M-1 × {0, & gt, N-1}, there is one D-dimensional eigenvector +.>Thus, the spatial regularization correlation filter may be defined as:
wherein ∈is a spatial correlation operation,., is a hadamard product, α k 0 represents each training sample x k The weight of each sample determines the impact of each sample. w and f d Representing the regularization weights and the actual output of the correlation filter, respectively, w may determine f from the spatial position of the correlation filter d Is of importance.Representing d-dimensional training samples with k channels, epsilon (f) is the actual output value of the correlation filter that satisfies the minimum error. Thus (S)>Is a spatially canonical term of the correlation filter. To solve this linear least squares problem, equation (1) can be converted to a standard equation in the fourier domain using the Parseval equation. The spatial regularization correlation filter can be effectively learned by obtaining a plurality of samples according to the historical tracking result. If z is of size MxN, which is a feature map of each video frame image, then the classification score s for each location can be determined using a discrete Fourier transform (Discrete Fourier Transformed, DFT) f (z) is defined as:
wherein s is f (z) is equivalent to a linear regression based on correlation filters, Λ is a discrete fourier function,is an inverse fourier transform. By adding a spatially regularized term weight in the correlation filter f>Unnecessary boundary effects can be eliminated. The value of the weight w changes along with the change of the space position, and the farther the weight w is away from the target area, the larger the weight w penalizes the background area around the target and vice versa.
The over-fitting problem that may occur in an occlusion environment is eliminated by passively updating the correlation filter. Because the correlation filter with spatially regularized terms in equation (1) is a convex optimization problem, the present invention can effectively utilize the global optimal solution that solves the correlation filter f with the alternating direction multiplier Algorithm (ADMM). However, unlike the STRCF tracking algorithm, the tracking algorithm of the present invention does not add the temporal regularization term f-f t-1 || 2 The linear regression problem is solved using only spatially-correlated canonical correlation terms. Firstly, introducing an auxiliary variable g, requiring f=g, setting a step size parameter as beta to obtain the augmented lagrangianDay form:
wherein s is Lagrangian multiplier whenWhen equation (3) is expressed as:
the invention uses ADMM algorithm to solve the following sub-problems alternately:
f (i+1) ，p (i+1) and q (i+1) Each representing a sub-problem of alternate solution, i representing the number of times of alternate solution.
For filter f, applying the Parseval theorem to the first line of equation (5), in the Fourier domain, can equivalently yield f as:
in formula (6)Is the discrete fourier transform of the correlation filter f, decomposing equation (6) into MN sub-problems:
in the labelIs dependent on the correlation filter only +.>Is the j-th element of (c). /> Each represents a vectorization of the corresponding scalar value function, consisting of the j-th element of f in all D channels. The +.A can be obtained by solving the minimum of equation (7) (i.e., the derivative of equation 7 is zero)>Closed solution of (2):
wherein I represents a unit vector, h is in the form ofIs a vector of (a).Is a matrix with rank of 1, and can be obtained by Sherman-Morrison formula:
finally, it can passIs inverse fourier transformed to f.
In the HCDC-SRCF tracking algorithm provided by the invention, the derivation of the sub-problem p and the updating process of the step size parameter beta are as follows from the document F.Li, C.Tian, W.Zuo, L.Zhang, M. -H.Yang, learning spatial-temporal regularized correlation filters forvisual tracking, in: proceedings of the IEEE conference on computer vision and pattern recognition,2018, pp.4904-4913, "same.
In visual object tracking, it is important how quickly and accurately to estimate candidate objects for the next frame. There are two traditional prediction methods: probability estimation and exhaustive search. The present invention uses the document "M.Danelljan, G.Hager, F.Shahbaz Khan, m.felsberg, learning spatially regularized correlation filters for visual tracking, in: proceedings of the IEEE international conference oncomputer vision,2015, pp.4310-4318", generates target candidates for the next frame. The object of the invention is to estimate the position of a target object in a next frame from the positions of the current frame and the first frame. The model can be robustly updated by correcting past errors using the information of the first frame image. Particularly, when the tracking algorithm of the present invention fails to track the target, the target can be easily re-tracked.
The invention mainly adopts two strategies to improve the positioning and tracking performance of the tracking algorithm on the target object in each video frame image. First, the present invention relies on a simple correlation filter that predicts a target frame for each video frame image in each sequence during tracking, as compared to other target detection and search schemes. The correlation filter f can be dynamically, incrementally maintained and updated to determine the position p of the object in each video frame t t And the size alpha of the target t . Most importantly, the invention adds a spatial regularization term in the correlation filterTo improve the positioning accuracy of the tracking algorithm in each video frame t. Secondly, the advantages of a space-time regularization related filter tracking algorithm STRCF, a space regularization related filter tracking algorithm SRDCF and a BACF tracking algorithm are comprehensively considered, and ADMM is added into the space regularization related filter, so that accurate and robust positioning and tracking are realized.
In addition, the ADMM is utilized to eliminate the interference of the background to the target object, so that the tracking performance of the algorithm is further improved. ADMM can be used to model the appearance of tracking target objects. The proposed tracking algorithm does not use negative samples when training the correlation filter model. In the HCDC-SRCF tracking algorithm provided by the invention, each frame fuses ADMM and a spatial regularization term into a tracking frame, and then fusion features are sent into a correlation filter so as to better locate a target object and further enhance the tracking robustness. During tracking, the present invention utilizes online updates of the target representation to describe the change in appearance of the target object. In each frame of video image, according to the tracking results of the first frame and the current frame, using the multi-channel fusion characteristics of the depth convolution characteristics and the manual characteristics to train the correlation filter in an iterative manner, and continuously updating the spatial correlation filter to determine the position and the size of the target object in the next frame of image.
Algorithm 1 gives the overall flow of the HCDC-SRCF tracking algorithm proposed by the present invention.
Those skilled in the art will appreciate that all or part of the steps in implementing the above-described implementation method may be implemented by a program to instruct related software, where the program may be stored in a computer readable storage medium, and the program when executed includes the steps of: (steps of the method), the storage medium, such as: ROM/RAM, magnetic disks, optical disks, etc.
Claims (5)
1. The self-adaptive multi-feature fusion tracking method based on the spatial regularization correlation filter is characterized by comprising the following steps of:
(1) Reading a video frame, determining a tracking target, and marking the tracking target by a target frame in a first frame image;
(2) Extracting the depth convolution characteristics of the target by adopting a residual convolution network, and extracting the manual characteristics of the target by adopting a gradient histogram and a gray level map;
(3) Carrying out feature fusion on the depth convolution features and the manual features;
(4) Respectively sending the fusion characteristics of the first frame image and the fusion characteristics of the tracking result of the current frame image into two spatial regularization correlation filters; the spatial regularization correlation filter is expressed as:
where, x is a spatial correlation operation, x is a hadamard product, α k 0 represents each training sample x k Weights, w and f of (2) d The regularized weights and the actual outputs of the correlation filters, respectively, epsilon (f) being the minimum error value, w determining f from the spatial position of the correlation filter d Is used for determining the importance of the product,is a spatial regularization term of the correlation filter;
(5) Accumulating and summing confidence values of the two spatial regularization correlation filters to obtain a tracking result of the t+1 frame, and continuously updating a subsequent frame of the video sequence by using the tracking results of the first frame image and the current frame image;
introducing an alternate direction multiplier algorithm into the spatial regularization correlation filter to solve a global optimal solution of the correlation filter f, wherein the process is as follows:
introducing an auxiliary variable g, requiring f=g, and obtaining an augmented lagrangian form with a step size parameter of beta:
wherein s is Lagrangian multiplier whenWhen the above formula is expressed as:
the following sub-problems are solved alternately by using an alternate direction multiplier algorithm:
wherein f (i+1) ，p (i+1) And q (i+1) Respectively representing the sub-problems of the alternate solution, i representing the number of times of the alternate solution;
for the filter f, the first line in the above equation, in the fourier domain, equivalently gets f as:
in the middle ofIs the discrete fourier transform of the correlation filter f, samples +.>All D channels are included, further broken down into MN sub-problems:
in the labelIs dependent on the correlation filter only +.>Is the j-th element of-> Each representing a vectorization of the corresponding scalar value function consisting of f the j-th element in all D-channels,/for>The closed solution of (2) is:
wherein I is a unit vector and h is in the form ofIs used for the vector of (a),is a matrix with rank of 1, and can be obtained by Sherman-Morrison formula:
finally, byIs inverse fourier transformed to f.
2. The adaptive multi-feature fusion tracking method based on spatial regularization correlation filter of claim 1, wherein: and (3) fusing the characteristic, namely fusing a layer of manual characteristic serving as a convolution characteristic with the depth convolution characteristic.
3. The adaptive multi-feature fusion tracking method based on spatial regularization correlation filter of claim 1 or 2, characterized in that: the first frame image information is used to correct for target drift in each frame.
4. The adaptive multi-feature fusion tracking method based on spatial regularization correlation filter of claim 3, wherein: when a tracking failure is detected, the last running result of the next video frame image is repositioned by using the real target frame of the first frame image, and the process is repeated until the last frame image of the video sequence.
5. The adaptive multi-feature fusion tracking method based on spatial regularization correlation filter of claim 1, 2 or 4, characterized in that: step (5) further comprises optimizing the final confidence value using a newton algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111121555.8A CN113838093B (en) | 2021-09-24 | 2021-09-24 | Self-adaptive multi-feature fusion tracking method based on spatial regularization correlation filter |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111121555.8A CN113838093B (en) | 2021-09-24 | 2021-09-24 | Self-adaptive multi-feature fusion tracking method based on spatial regularization correlation filter |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113838093A CN113838093A (en) | 2021-12-24 |
| CN113838093B true CN113838093B (en) | 2024-03-19 |
Family
ID=78969766
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111121555.8A Active CN113838093B (en) | 2021-09-24 | 2021-09-24 | Self-adaptive multi-feature fusion tracking method based on spatial regularization correlation filter |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113838093B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109359663A (en) * | 2018-08-22 | 2019-02-19 | 南京信息工程大学 | A kind of video tracing method returned based on color cluster and space-time canonical |
| CN109461172A (en) * | 2018-10-25 | 2019-03-12 | 南京理工大学 | Manually with the united correlation filtering video adaptive tracking method of depth characteristic |
| CN111402303A (en) * | 2020-02-17 | 2020-07-10 | 绍兴文理学院 | Target tracking architecture based on KFSTRCF |
| CN111429485A (en) * | 2020-04-07 | 2020-07-17 | 东北大学 | Cross-modal filtering tracking method based on self-adaptive regularization and high-reliability updating |
| WO2020228446A1 (en) * | 2019-05-13 | 2020-11-19 | 腾讯科技（深圳）有限公司 | Model training method and apparatus, and terminal and storage medium |
| CN112330716A (en) * | 2020-11-11 | 2021-02-05 | 南京邮电大学 | Space-time channel constraint correlation filtering tracking method based on abnormal suppression |
| CN113344973A (en) * | 2021-06-09 | 2021-09-03 | 南京信息工程大学 | Target tracking method based on space-time regularization and feature reliability evaluation |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150146928A1 (en) * | 2013-11-27 | 2015-05-28 | Electronics And Telecommunications Research Institute | Apparatus and method for tracking motion based on hybrid camera |
-
2021
- 2021-09-24 CN CN202111121555.8A patent/CN113838093B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109359663A (en) * | 2018-08-22 | 2019-02-19 | 南京信息工程大学 | A kind of video tracing method returned based on color cluster and space-time canonical |
| CN109461172A (en) * | 2018-10-25 | 2019-03-12 | 南京理工大学 | Manually with the united correlation filtering video adaptive tracking method of depth characteristic |
| WO2020228446A1 (en) * | 2019-05-13 | 2020-11-19 | 腾讯科技（深圳）有限公司 | Model training method and apparatus, and terminal and storage medium |
| CN111402303A (en) * | 2020-02-17 | 2020-07-10 | 绍兴文理学院 | Target tracking architecture based on KFSTRCF |
| CN111429485A (en) * | 2020-04-07 | 2020-07-17 | 东北大学 | Cross-modal filtering tracking method based on self-adaptive regularization and high-reliability updating |
| CN112330716A (en) * | 2020-11-11 | 2021-02-05 | 南京邮电大学 | Space-time channel constraint correlation filtering tracking method based on abnormal suppression |
| CN113344973A (en) * | 2021-06-09 | 2021-09-03 | 南京信息工程大学 | Target tracking method based on space-time regularization and feature reliability evaluation |
Non-Patent Citations (1)
| Title |
|---|
| "基于相关判别模型的视觉跟踪算法研究";张海洋;《中国优秀硕士学位论文全文数据库 信息科技辑》;第I138-1010页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113838093A (en) | 2021-12-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108090919B (en) | Improved kernel correlation filtering tracking method based on super-pixel optical flow and adaptive learning factor | |
| CN113807187B (en) | Unmanned aerial vehicle video multi-target tracking method based on attention feature fusion | |
| CN108549839B (en) | Adaptive feature fusion multi-scale correlation filtering visual tracking method | |
| CN108734723B (en) | Relevant filtering target tracking method based on adaptive weight joint learning | |
| CN108776975B (en) | Visual tracking method based on semi-supervised feature and filter joint learning | |
| CN107369166B (en) | Target tracking method and system based on multi-resolution neural network | |
| CN110120064B (en) | Depth-related target tracking algorithm based on mutual reinforcement and multi-attention mechanism learning | |
| CN111080675B (en) | Target tracking method based on space-time constraint correlation filtering | |
| CN112132149B (en) | Semantic segmentation method and device for remote sensing image | |
| CN107240122A (en) | Video target tracking method based on space and time continuous correlation filtering | |
| CN110276785B (en) | Anti-shielding infrared target tracking method | |
| CN111311647B (en) | Global-local and Kalman filtering-based target tracking method and device | |
| CN110942471B (en) | Long-term target tracking method based on space-time constraint | |
| CN107358623A (en) | A kind of correlation filtering track algorithm based on conspicuousness detection and robustness size estimation | |
| CN111260738A (en) | Multi-scale target tracking method based on relevant filtering and self-adaptive feature fusion | |
| CN111310582A (en) | Turbulence degradation image semantic segmentation method based on boundary perception and counterstudy | |
| CN110084201B (en) | Human body action recognition method based on convolutional neural network of specific target tracking in monitoring scene | |
| Huang et al. | Siamatl: Online update of siamese tracking network via attentional transfer learning | |
| CN112183675B (en) | Tracking method for low-resolution target based on twin network | |
| CN112329784A (en) | Correlation filtering tracking method based on space-time perception and multimodal response | |
| CN112785622A (en) | Long-time tracking method and device for unmanned ship on water surface and storage medium | |
| Fu et al. | Robust multi-kernelized correlators for UAV tracking with adaptive context analysis and dynamic weighted filters | |
| Teng et al. | Robust multi-scale ship tracking via multiple compressed features fusion | |
| CN113033356B (en) | Scale-adaptive long-term correlation target tracking method | |
| CN111161323B (en) | Complex scene target tracking method and system based on correlation filtering |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |