CN112330716A - Space-time channel constraint correlation filtering tracking method based on abnormal suppression - Google Patents

Space-time channel constraint correlation filtering tracking method based on abnormal suppression Download PDF

Info

Publication number
CN112330716A
CN112330716A CN202011251969.8A CN202011251969A CN112330716A CN 112330716 A CN112330716 A CN 112330716A CN 202011251969 A CN202011251969 A CN 202011251969A CN 112330716 A CN112330716 A CN 112330716A
Authority
CN
China
Prior art keywords
frame
filter
feature
formula
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011251969.8A
Other languages
Chinese (zh)
Other versions
CN112330716B (en
Inventor
范保杰
王雪艳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Posts and Telecommunications
Original Assignee
Nanjing University of Posts and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Posts and Telecommunications filed Critical Nanjing University of Posts and Telecommunications
Priority to CN202011251969.8A priority Critical patent/CN112330716B/en
Publication of CN112330716A publication Critical patent/CN112330716A/en
Application granted granted Critical
Publication of CN112330716B publication Critical patent/CN112330716B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/262—Analysis of motion using transform domain methods, e.g. Fourier domain methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a space-time channel constraint correlation filtering tracking method based on abnormal suppression, which comprises the following steps: step S1, extracting the HOG feature, the first depth feature and the second depth feature of the t frame; step S2, carrying out fusion processing on the HOG feature, the first depth feature and the second depth feature to obtain a first fusion feature X, and then determining the position and the scale of a target in the t frame image based on the first fusion feature X and a filter; step S3, updating the filter based on the spatial-temporal channel constraint related filtering model which can restrain abnormity according to the characteristic diagram of the t-th frame; and step S4, repeating the steps S2-S4 until all the frame tracking is finished, and finally obtaining the tracking result. The invention uses manual characteristics to combine depth characteristics to remarkably improve the characteristic representation capability of the target template and passes the characteristic2,1The norm realizes self-adaptive channel characteristic selection, and effectively solves the problems of boundary effect and background clutter.

Description

Space-time channel constraint correlation filtering tracking method based on abnormal suppression
Technical Field
The invention relates to the technical field of computer vision image processing, in particular to a spatio-temporal channel constraint correlation filtering tracking method based on abnormal suppression.
Background
Target tracking is a very popular research topic in the field of computer vision and has been widely applied to video surveillance, unmanned driving, human-computer interaction and the like. Target tracking is given the position and scale of the target in the first frame, thereby predicting the position of the target in subsequent video frames.
Discriminant correlation filter based trackers have achieved excellent results in many common video reference datasets and competitions. Starting from the dominant MOSSE filter, the discriminant correlation filter based tracker achieves very good performance in visual tracking. The KCF uses the multiple-channel HOG features to build an appearance model of the target, thereby significantly improving the performance of the algorithm. The HOG characteristic has good tracking performance on color change, and the CN characteristic has good tracking effect on target deformation and motion blur. Thus, stack proposes a fusion algorithm that uses the combined features of HOG + CN to track the target. The BACF multiplies the target center region by a fixed binary matrix to obtain real samples, and proposes an effective ADMM method to learn the filter. CACF proposes a new framework that can add more context information and context information to the learned filter. The AutoTrack trains the tracker by using self-adaptive space-time regularization, and provides a hyper-parameter correlation filtering tracker method based on self-adaptively adjusting space-time constraint terms.
Recently, convolutional neural networks have been widely used for correlation filters in order to achieve a more accurate and comprehensive representation of the appearance of the target. Feature representations combining manual features and depth features have been widely used in some trackers. C-COT performs sub-grid tracking by learning discriminant continuous convolution operators. ECO, a lightweight version of C-COT, uses decomposition convolution operators and generation of sample space to reduce model complexity and increase computation speed. LADCF selects only 5% of the handcrafted and 20% of the deep features for filter learning using a method with adaptive spatial feature selection. ASRCF provides an adaptive spatial regularization method, which can effectively acquire spatial weight to adapt to the change of target appearance and realize stronger tracking performance. GFS-DCF introduced a tracking framework for Res-Net50 containing rich semantic information and, in addition, introduced a group sparse feature selection method to learn adaptive feature selection and achieve superior performance.
However, when the tracking target is interfered by occlusion, fast motion and background noise, the target sample used for training the filter may be interfered, resulting in a gradual decrease in the discrimination of the filter template.
Therefore, the invention provides a spatio-temporal channel constraint correlation filtering tracking method based on abnormal suppression, aiming at the problems.
Disclosure of Invention
In view of this, in order to solve the problem that the filter learning capability is degraded when the target template is interfered by the existing target tracking method, the invention provides a spatio-temporal channel constraint correlation filtering tracking method based on abnormal suppression.
In order to achieve the above object, the present invention provides a spatio-temporal channel constraint correlation filtering tracking method based on abnormal suppression, which comprises the following steps:
step S1, acquiring a region of a target in a t frame according to the position and the scale of the target in the t-1 frame image, taking the region as a target region, and extracting the HOG feature, the first depth feature and the second depth feature of the target region;
step S2, carrying out fusion processing on the HOG feature, the first depth feature and the second depth feature of the t-th frame to obtain a fused first fusion feature X, and then determining the position and the scale of a target in the image of the t-th frame based on the first fusion feature X and a filter, wherein the filter is trained on the target image in the image of the t-1 th frame in advance;
step S3, updating the filter to obtain a new filter based on a spatial-temporal channel constraint related filtering model capable of inhibiting abnormity according to the characteristic diagram of the t-th frame;
and step S4, repeating the steps S2-S4 until all the frame tracking is finished, and finally obtaining the tracking result.
Further, in the step S1, a preset ResNet50 depth model is adopted to extract the Conv4-3 as the first depth feature and the Conv4-6 as the second depth feature.
Further, in step S2, the determining the position and the scale of the target in the t-th frame image specifically includes:
and acquiring 7 second fusion features X with sequentially increasing scales according to the first fusion feature X, performing Fourier transform on the 7 second feature X by using the filter to obtain a Fourier domain, performing convolution operation to obtain a response map, taking the position of the maximum response value in the response map as the position of the target in the t-th frame image, and taking the scale corresponding to the maximum response value in the response map as the target scale in the t-th frame image.
Further, step S3 specifically includes:
301, the spatial domain expression of the spatial-temporal channel constraint related filtering model capable of suppressing the abnormality is as follows:
Figure BDA0002771879980000021
in formula (1), y is an ideal Gaussian function, and is a spatial correlation operator, p and q represent the position difference of two peaks in two response maps in two-dimensional space, [ psi [p,q]Indicating a shifting operation performed in order to make two peaks coincide with each other,
Figure BDA0002771879980000022
representing the filter on the kth channel at the t-1 frame,
Figure BDA0002771879980000023
representing the filter on the kth channel at the tth frame,
Figure BDA0002771879980000024
and
Figure BDA0002771879980000025
respectively representing the characteristic samples and filters, λ, from K channels1And λ2Is a regular parameter, gamma is an abnormal punishment parameter;
step 302, converting the space-time channel constraint related filtering model capable of suppressing the abnormity into a Fourier domain by using Parseval's theorem, and obtaining an expression as follows:
Figure BDA0002771879980000031
in the formula (2), the upper criterion Λ is the discrete fourier transform of a given signal,
Figure BDA0002771879980000032
for the discrete fourier transform of the introduced filter auxiliary variables, T is the size of the input data, F is an orthogonal matrix of size T x T, which maps the arbitrary T-dimensional vectorized signal into the fourier domain,
Figure BDA0002771879980000033
is defined as Mt-1[ψp,q]Discrete fourier transform of (d);
step 303, constructing formula (2) in step S302 as an augmented lagrangian function, where the expression is:
Figure BDA0002771879980000034
in the formula (3), the first and second groups,
Figure BDA0002771879980000035
in order to be a lagrange multiplier,
Figure BDA0002771879980000036
is a fourier domain transform of the lagrange multiplier,
Figure BDA0002771879980000037
expressed as the transpose of the lagrange multiplier fourier domain transform, mu is a penalty factor,
Figure BDA0002771879980000038
for the discrete fourier transform of the introduced filter auxiliary variables,
Figure BDA0002771879980000039
is the discrete fourier transform of the filter auxiliary variables introduced in the t-1 th frame.
Step 304, given
Figure BDA00027718799800000310
And
Figure BDA00027718799800000311
converting the augmented Lagrange function of step S303 into a subproblem solving filter
Figure BDA00027718799800000312
Figure BDA00027718799800000313
In the formula (4), the first and second groups,
Figure BDA00027718799800000314
expressed as the transpose of the lagrange multiplier fourier domain transform,
Figure BDA00027718799800000315
representing the solution of the filter at the t +1 th frame.
Solving for each channel yields:
Figure BDA00027718799800000316
in the formula (5), the first and second groups,
Figure BDA00027718799800000317
representing the filter auxiliary variables on the kth channel at the tth frame.
Step 305, given
Figure BDA00027718799800000318
And
Figure BDA00027718799800000319
converting the augmented Lagrange function of step S303 toSolving auxiliary variables for sub-problems
Figure BDA00027718799800000320
Figure BDA00027718799800000321
Equation (6) is then decomposed into N subproblems, each expressed as:
Figure BDA0002771879980000041
solving the formula (7) and optimizing by using a Sherman Morrison formula to obtain:
Figure BDA0002771879980000042
in the formula (8), the first and second groups,
Figure BDA0002771879980000043
and
Figure BDA0002771879980000044
it should be noted that the above variables are not of practical significance, but are merely combined for computational convenience.
Step 306, give
Figure BDA0002771879980000045
And
Figure BDA0002771879980000046
updating Lagrange multiplier terms
Figure BDA0002771879980000047
Figure BDA0002771879980000048
In the formula (9), the reaction mixture,
Figure BDA0002771879980000049
is the lagrangian multiplier term at the ith iteration,
Figure BDA00027718799800000410
and
Figure BDA00027718799800000411
is solved by the formulas in steps S305 and S304 in the (i + 1) th iteration;
step 307, updating the regular penalty factor μ:
μ(i+1)＝min(βμ(i),μmax) (10)
in the formula (10), β is 1.5, μmax＝1；
Step 308, update the filter template
Figure BDA00027718799800000412
Figure BDA00027718799800000413
The compound of the formula (11),
Figure BDA00027718799800000414
representing the updated filter template at the t-th frame,
Figure BDA00027718799800000415
representing the filter template at frame t-1,
Figure BDA00027718799800000416
representing the result of solving the formula of step S304,
Figure BDA00027718799800000417
is the online update rate.
Further, before performing step S1, it is determined whether the t-th frame is the first frame in the video sequence, if not, step S1 is performed, if so, the position of the target and the relevant filter are initialized, after the initialization operation is completed, it is determined whether tracking of all frames is currently completed, if so, the tracking result is output, and if not, step S1 is returned to.
The invention has the beneficial effects that:
the invention significantly improves the feature representation capability of the target template in the aspect of feature representation by combining manual features with depth features. In the spatial-temporal channel constraint correlation filtering model based on suppressible abnormity2,1The norm realizes self-adaptive channel characteristic selection, and effectively solves the problems of boundary effect and background clutter ground; similarity constraint is carried out on the target template from the time sequence through a time constraint item, and historical frame information is reserved for learning of the filter, so that the degradation problem of the filter is relieved; more robust and more accurate target tracking is realized by suppressing the abnormal constraint item; by optimizing the model using the ADMM algorithm, the time complexity is significantly reduced and the computation speed is increased.
Drawings
Fig. 1 is a flow chart of a first embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
For better illustration of the present invention, two terms are explained first, the overall name of HOG being Historgram ordered Gradients, representing Histogram of Oriented Gradients; ADMM is generally called Alternating Direction Method of Multipliers, and is expressed as the Alternating Direction multiplier Method.
Example 1
Referring to fig. 1, the present embodiment provides a spatio-temporal channel constraint correlation filtering tracking method based on suppressible anomalies, the method uses a lemming sequence in a target tracking reference data set (OTB100) as a verification set, the video size is 640 × 480, and total 1336 frames include illumination changes, scale scaling, occlusion, fast motion, background clutter and other appearance significant changes; before the method is used, namely in a first frame, initializing the position of a target to be tracked and a relevant filter, and completing target tracking of a subsequent frame based on the method, wherein the general flow chart is shown in fig. 1, and the method comprises the following steps:
step S1, obtaining the area of the target in the t frame according to the target position and scale in the t-1 frame image, taking the area as the target area, and extracting the HOG feature, the first depth feature and the second depth feature of the target area.
Specifically, in the embodiment, a preset ResNet50 depth model is adopted to extract Conv4-3 as a first depth feature and Conv4-6 as a second depth feature.
Step S2, fusion processing is carried out on the HOG feature, the first depth feature and the second depth feature of the t-th frame to obtain a first fusion feature X after fusion, then the position and the scale of the target in the image of the t-th frame are determined based on the first fusion feature X and a filter, and the filter is trained by the target image in the image of the t-1 th frame in advance.
Specifically, in this embodiment, the determining the position and the scale of the target in the t-th frame image specifically includes: and acquiring 7 second fusion features X with sequentially increasing scales according to the first fusion feature X, performing Fourier transform on the 7 second feature X by using the filter to obtain a Fourier domain, performing convolution operation to obtain a response map, taking the position of the maximum response value in the response map as the position of the target in the t-th frame image, and taking the scale corresponding to the maximum response value in the response map as the target scale in the t-th frame image.
Step S3, updating the filter to obtain a new filter based on a spatial-temporal channel constraint related filtering model capable of inhibiting abnormity according to the characteristic diagram of the t-th frame;
specifically, step S3 includes the following sub-steps:
301, the spatial domain expression of the spatial-temporal channel constraint related filtering model capable of suppressing the abnormality is as follows:
Figure BDA0002771879980000061
in formula (1), y is an ideal Gaussian function, and is a spatial correlation operator, p and q represent the position difference of two peaks in two response maps in two-dimensional space, [ psi [p,q]Indicating a shifting operation performed in order to make two peaks coincide with each other,
Figure BDA0002771879980000062
representing the filter on the kth channel at the t-1 frame,
Figure BDA0002771879980000063
representing the filter on the kth channel at the tth frame,
Figure BDA0002771879980000064
and
Figure BDA0002771879980000065
respectively representing the characteristic samples and filters, λ, from K channels1And λ2Is a regular parameter, gamma is an abnormal punishment parameter;
in the above formula, the first term is a ridge regression term, so that the response map and the label y are fitted as much as possible; the second term being the filter htChannel regularization penalty term of using l2,1Norm to realize self-adaptive feature selection; the third term is a temporal regularization term, wherein
Figure BDA0002771879980000066
Represents the filter on the kth channel at the t-1 frame; the fourth term is an abnormal term inhibition term, wherein the response graph of the t-1 frame
Figure BDA0002771879980000067
Can simplify Mt-1[ψp,q]And (4) showing.
Step 302, converting the space-time channel constraint related filtering model capable of suppressing the abnormity into a Fourier domain by using Parseval's theorem, and obtaining an expression as follows:
Figure BDA0002771879980000068
in the formula (2), the upper criterion Λ is the discrete fourier transform of a given signal,
Figure BDA0002771879980000069
for the discrete fourier transform of the introduced filter auxiliary variables, T is the size of the input data, F is an orthogonal matrix of size T x T, which maps the arbitrary T-dimensional vectorized signal into the fourier domain,
Figure BDA00027718799800000610
is defined as Mt-1[ψp,q]Discrete fourier transform of (d);
step 303, constructing formula (2) in step S302 as an augmented lagrangian function, where the expression is:
Figure BDA0002771879980000071
in the formula (3), the first and second groups,
Figure BDA0002771879980000072
in order to be a lagrange multiplier,
Figure BDA0002771879980000073
is a fourier domain transform of the lagrange multiplier,
Figure BDA0002771879980000074
expressed as the transpose of the lagrange multiplier fourier domain transform, mu is a penalty factor,
Figure BDA0002771879980000075
for the discrete fourier transform of the introduced filter auxiliary variables,
Figure BDA0002771879980000076
is the discrete fourier transform of the filter auxiliary variables introduced in the t-1 th frame.
Step 304, given
Figure BDA0002771879980000077
And
Figure BDA0002771879980000078
converting the augmented Lagrange function of step S303 into a subproblem solving filter
Figure BDA0002771879980000079
Figure BDA00027718799800000710
In the formula (4), the first and second groups,
Figure BDA00027718799800000711
expressed as the transpose of the lagrange multiplier fourier domain transform,
Figure BDA00027718799800000712
representing the solution of the filter at the t +1 th frame.
Solving for each channel yields:
Figure BDA00027718799800000713
in the formula (5), the first and second groups,
Figure BDA00027718799800000714
representing the filter auxiliary variables on the kth channel at the tth frame.
Step 305, given
Figure BDA00027718799800000715
And
Figure BDA00027718799800000716
converting the augmented Lagrange function of step S303 into a subproblem solving auxiliary variable
Figure BDA00027718799800000717
Figure BDA00027718799800000718
Equation (6) is then decomposed into N subproblems, each expressed as:
Figure BDA00027718799800000719
solving the formula (7) and optimizing by using a Sherman Morrison formula to obtain:
Figure BDA00027718799800000720
in the formula (8), the first and second groups,
Figure BDA00027718799800000721
and
Figure BDA0002771879980000081
it is to be noted that; the above variables are not of practical significance, but are merely combined for computational convenience.
Step 306, give
Figure BDA0002771879980000082
And
Figure BDA0002771879980000083
updating Lagrange multiplier terms
Figure BDA0002771879980000084
Figure BDA0002771879980000085
In the formula (9), the reaction mixture,
Figure BDA0002771879980000086
is the lagrangian multiplier term at the ith iteration,
Figure BDA0002771879980000087
and
Figure BDA0002771879980000088
is solved by the formulas in steps S305 and S304 in the (i + 1) th iteration;
step 307, updating the regular penalty factor μ:
μ(i+1)＝min(βμ(i),μmax) (10)
in the formula (10), β is 1.5, μmax＝1；
Step 308, update the filter template
Figure BDA0002771879980000089
Figure BDA00027718799800000810
The compound of the formula (11),
Figure BDA00027718799800000811
filter template updated at the time of representing the t-th frame
Figure BDA00027718799800000812
Representing the filter template at frame t-1,
Figure BDA00027718799800000813
representing the result of solving the formula of step S304,
Figure BDA00027718799800000814
is the online update rate.
And step S4, repeating the steps S2-S4 until all the frame tracking is finished, and finally obtaining the tracking result.
According to the technical scheme, the target tracking is carried out on the video sequence, no deviation of the tracked target is generated all the time in the scenes with shielding, rapid motion or background clutter, and the tracking precision is high.
The invention significantly improves the feature representation capability of the target template in the aspect of feature representation by combining manual features with depth features. In the spatial-temporal channel constraint correlation filtering model based on suppressible abnormity2,1The norm realizes self-adaptive channel characteristic selection, and effectively solves the problems of boundary effect and background clutter ground; the filter degradation problem is alleviated through a time constraint term; more robust and more accurate target tracking is realized by suppressing the abnormal constraint item; by optimizing the model using the ADMM algorithm, the time complexity is significantly reduced and the computation speed is increased.
It should be noted that the above is only a specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes and substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (5)

1. A spatial-temporal channel constraint correlation filtering tracking method based on suppressive abnormality is characterized by comprising the following steps:
step S1, acquiring a region of a target in a t frame according to the position and the scale of the target in the t-1 frame image, taking the region as a target region, and extracting the HOG feature, the first depth feature and the second depth feature of the target region;
step S2, carrying out fusion processing on the HOG feature, the first depth feature and the second depth feature of the t-th frame to obtain a fused first fusion feature X, and then determining the position and the scale of a target in the image of the t-th frame based on the first fusion feature X and a filter, wherein the filter is trained on the target image in the image of the t-1 th frame in advance;
step S3, updating the filter to obtain a new filter based on a spatial-temporal channel constraint related filtering model capable of inhibiting abnormity according to the characteristic diagram of the t-th frame;
and step S4, repeating the steps S2-S4 until all the frame tracking is finished, and finally obtaining the tracking result.
2. The method for spatio-temporal channel constraint correlation filtering tracking based on suppressible abnormality of claim 1, characterized in that in said step S1, a preset ResNet50 depth model is used to extract Conv4-3 as the first depth feature and Conv4-6 as the second depth feature.
3. The method for spatio-temporal channel constraint correlation filtering tracking based on suppressible abnormality according to claim 2, characterized in that in said step S2, said determining the position and scale of the target in the t-th frame image is specifically:
and acquiring 7 second fusion features X with sequentially increasing scales according to the first fusion feature X, performing Fourier transform on the 7 second feature X by using the filter to obtain a Fourier domain, performing convolution operation to obtain a response map, taking the position of the maximum response value in the response map as the position of the target in the t-th frame image, and taking the scale corresponding to the maximum response value in the response map as the target scale in the t-th frame image.
4. The method for spatio-temporal channel constraint correlation filtering tracking based on suppressible abnormalities as claimed in claim 3, wherein said step S3 specifically comprises:
301, the spatial domain expression of the spatial-temporal channel constraint related filtering model capable of suppressing the abnormality is as follows:
Figure FDA0002771879970000011
in formula (1), y is an ideal Gaussian function, and is a spatial correlation operator, p and q represent the position difference of two peaks in two response maps in two-dimensional space, [ psi [p,q]Indicating a shifting operation performed in order to make two peaks coincide with each other,
Figure FDA0002771879970000012
representing the filter on the kth channel at the t-1 frame,
Figure FDA0002771879970000013
representing the filter on the kth channel at the tth frame,
Figure FDA0002771879970000014
and
Figure FDA0002771879970000015
respectively representing the characteristic samples and filters, λ, from K channels1And λ2Is a regular parameter, gamma is an abnormal punishment parameter;
step 302, converting the space-time channel constraint related filtering model capable of suppressing the abnormity into a Fourier domain by using Parseval's theorem, and obtaining an expression as follows:
Figure FDA0002771879970000021
in the formula (2), the upper criterion Λ is the discrete fourier transform of a given signal,
Figure FDA0002771879970000022
for the discrete fourier transform of the filter auxiliary variables introduced in the t-th frame,
Figure FDA0002771879970000023
filter assistance introduced for t-1 frameDiscrete Fourier transform of variables, T being the size of the input data, F being an orthogonal matrix of size T x T, F mapping any T-dimensional vectorized signal into the Fourier domain,
Figure FDA0002771879970000024
is defined as Mt-1[ψp,q]Discrete fourier transform of (d);
step 303, constructing formula (2) in step S302 as an augmented lagrangian function, where the expression is:
Figure FDA0002771879970000025
in the formula (3), the first and second groups,
Figure FDA0002771879970000026
in order to be a lagrange multiplier,
Figure FDA0002771879970000027
is a fourier domain transform of the lagrange multiplier,
Figure FDA0002771879970000028
expressed as the transpose of the lagrange multiplier fourier domain transform, mu is a penalty factor,
Figure FDA0002771879970000029
for the discrete fourier transform of the introduced filter auxiliary variables,
Figure FDA00027718799700000210
discrete Fourier transform of filter auxiliary variables introduced in the t-1 frame;
step 304, given
Figure FDA00027718799700000211
And
Figure FDA00027718799700000212
converting the augmented Lagrange function of step S303 into a subproblem solving filter
Figure FDA00027718799700000213
Figure FDA00027718799700000214
In the formula (4), the first and second groups,
Figure FDA00027718799700000215
expressed as the transpose of the lagrange multiplier fourier domain transform,
Figure FDA00027718799700000216
represents the solution of the filter at the t +1 th frame;
solving for each channel yields:
Figure FDA00027718799700000217
in the formula (5), the first and second groups,
Figure FDA00027718799700000218
Figure FDA00027718799700000219
represents the filter auxiliary variable on the kth channel at the tth frame;
step 305, given
Figure FDA00027718799700000220
And
Figure FDA00027718799700000221
converting the augmented Lagrange function of step S303 into a subproblem solving auxiliary variable
Figure FDA00027718799700000222
Figure FDA00027718799700000223
Equation (6) is then decomposed into N subproblems, each expressed as:
Figure FDA0002771879970000031
solving the formula (7) and optimizing by using a Sherman Morrison formula to obtain:
Figure FDA0002771879970000032
in the formula (8), the first and second groups,
Figure FDA0002771879970000033
and
Figure FDA0002771879970000034
step 306, give
Figure FDA0002771879970000035
And
Figure FDA0002771879970000036
updating Lagrange multiplier terms
Figure FDA0002771879970000037
Figure FDA0002771879970000038
In the formula (9), the reaction mixture,
Figure FDA0002771879970000039
is the lagrangian multiplier term at the ith iteration,
Figure FDA00027718799700000310
and
Figure FDA00027718799700000311
is solved by the formulas in steps S305 and S304 in the (i + 1) th iteration;
step 307, updating the regular penalty factor μ:
μ(i+1)＝min(βμ(i),μmax) (10)
in the formula (10), β is 1.5, μmax＝1；
Step 308, update the filter template
Figure FDA00027718799700000312
Figure FDA00027718799700000313
In the formula (11), the reaction mixture,
Figure FDA00027718799700000314
representing the updated filter template at the t-th frame,
Figure FDA00027718799700000315
representing the filter template at frame t-1,
Figure FDA00027718799700000316
representing the result of solving the formula of step S304,
Figure FDA00027718799700000317
is the online update rate.
5. The method as claimed in claim 4, wherein before step S1, it is determined whether the t-th frame is the first frame in the video sequence, if not, step S1 is performed, if it is, the position of the target and the correlation filter are initialized, and after the initialization operation is completed, it is determined whether the tracking of all frames is currently completed, if yes, the tracking result is output, and if not, step S1 is returned to.
CN202011251969.8A 2020-11-11 2020-11-11 Space-time channel constraint correlation filtering tracking method based on abnormal suppression Active CN112330716B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011251969.8A CN112330716B (en) 2020-11-11 2020-11-11 Space-time channel constraint correlation filtering tracking method based on abnormal suppression

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011251969.8A CN112330716B (en) 2020-11-11 2020-11-11 Space-time channel constraint correlation filtering tracking method based on abnormal suppression

Publications (2)

Publication Number Publication Date
CN112330716A true CN112330716A (en) 2021-02-05
CN112330716B CN112330716B (en) 2022-08-19

Family

ID=74318899

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011251969.8A Active CN112330716B (en) 2020-11-11 2020-11-11 Space-time channel constraint correlation filtering tracking method based on abnormal suppression

Country Status (1)

Country Link
CN (1) CN112330716B (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113128558A (en) * 2021-03-11 2021-07-16 重庆邮电大学 Target detection method based on shallow space feature fusion and adaptive channel screening
CN113344973A (en) * 2021-06-09 2021-09-03 南京信息工程大学 Target tracking method based on space-time regularization and feature reliability evaluation
CN113470074A (en) * 2021-07-09 2021-10-01 天津理工大学 Self-adaptive space-time regularization target tracking algorithm based on block discrimination
CN113538340A (en) * 2021-06-24 2021-10-22 武汉中科医疗科技工业技术研究院有限公司 Target contour detection method and device, computer equipment and storage medium
CN113838093A (en) * 2021-09-24 2021-12-24 重庆邮电大学 Self-adaptive multi-feature fusion tracking method based on spatial regularization correlation filter
CN115018906A (en) * 2022-04-22 2022-09-06 国网浙江省电力有限公司 Power grid power transformation overhaul operator tracking method based on combination of group feature selection and discrimination related filtering

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108280808A (en) * 2017-12-15 2018-07-13 西安电子科技大学 The method for tracking target of correlation filter is exported based on structuring

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108280808A (en) * 2017-12-15 2018-07-13 西安电子科技大学 The method for tracking target of correlation filter is exported based on structuring

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ZIYUAN HUANG等: "Learning Aberrance Repressed Correlation Filters for Real-Time UAV Tracking", 《ICCV2019》 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113128558A (en) * 2021-03-11 2021-07-16 重庆邮电大学 Target detection method based on shallow space feature fusion and adaptive channel screening
CN113344973A (en) * 2021-06-09 2021-09-03 南京信息工程大学 Target tracking method based on space-time regularization and feature reliability evaluation
CN113344973B (en) * 2021-06-09 2023-11-24 南京信息工程大学 Target tracking method based on space-time regularization and feature reliability evaluation
CN113538340A (en) * 2021-06-24 2021-10-22 武汉中科医疗科技工业技术研究院有限公司 Target contour detection method and device, computer equipment and storage medium
CN113470074A (en) * 2021-07-09 2021-10-01 天津理工大学 Self-adaptive space-time regularization target tracking algorithm based on block discrimination
CN113838093A (en) * 2021-09-24 2021-12-24 重庆邮电大学 Self-adaptive multi-feature fusion tracking method based on spatial regularization correlation filter
CN113838093B (en) * 2021-09-24 2024-03-19 重庆邮电大学 Self-adaptive multi-feature fusion tracking method based on spatial regularization correlation filter
CN115018906A (en) * 2022-04-22 2022-09-06 国网浙江省电力有限公司 Power grid power transformation overhaul operator tracking method based on combination of group feature selection and discrimination related filtering

Also Published As

Publication number Publication date
CN112330716B (en) 2022-08-19

Similar Documents

Publication Publication Date Title
CN112330716B (en) Space-time channel constraint correlation filtering tracking method based on abnormal suppression
CN108549839B (en) Adaptive feature fusion multi-scale correlation filtering visual tracking method
CN107369166B (en) Target tracking method and system based on multi-resolution neural network
CN112560695A (en) Underwater target tracking method, system, storage medium, equipment, terminal and application
Huang et al. Robust visual tracking via constrained multi-kernel correlation filters
CN109859241B (en) Adaptive feature selection and time consistency robust correlation filtering visual tracking method
CN110349190B (en) Adaptive learning target tracking method, device, equipment and readable storage medium
Lu et al. Learning transform-aware attentive network for object tracking
CN109166139B (en) Scale self-adaptive target tracking method combined with rapid background suppression
Lin et al. Learning temporary block-based bidirectional incongruity-aware correlation filters for efficient UAV object tracking
CN113344973B (en) Target tracking method based on space-time regularization and feature reliability evaluation
CN111401178B (en) Video target real-time tracking method and system based on depth feature fusion and adaptive correlation filtering
CN114359347A (en) Space-time regularization self-adaptive correlation filtering target tracking algorithm based on sample reliability
EP1801731B1 (en) Adaptive scene dependent filters in online learning environments
CN110706253B (en) Target tracking method, system and device based on apparent feature and depth feature
CN111862167A (en) Rapid robust target tracking method based on sparse compact correlation filter
Zeng et al. Deep stereo matching with hysteresis attention and supervised cost volume construction
CN113487530B (en) Infrared and visible light fusion imaging method based on deep learning
Zhang et al. Learning target-aware background-suppressed correlation filters with dual regression for real-time UAV tracking
CN110580712B (en) Improved CFNet video target tracking method using motion information and time sequence information
CN117011342A (en) Attention-enhanced space-time transducer vision single-target tracking method
CN115100740B (en) Human motion recognition and intention understanding method, terminal equipment and storage medium
CN116342653A (en) Target tracking method, system, equipment and medium based on correlation filter
CN113538509B (en) Visual tracking method and device based on adaptive correlation filtering feature fusion learning
CN115393400A (en) Video target tracking method for single sample learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant