CN110060280B - Target tracking method based on appearance self-adaptive spatial regularization correlation filter - Google Patents

Target tracking method based on appearance self-adaptive spatial regularization correlation filter Download PDF

Info

Publication number
CN110060280B
CN110060280B CN201910349109.9A CN201910349109A CN110060280B CN 110060280 B CN110060280 B CN 110060280B CN 201910349109 A CN201910349109 A CN 201910349109A CN 110060280 B CN110060280 B CN 110060280B
Authority
CN
China
Prior art keywords
target
matrix
filter
correlation filter
class
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910349109.9A
Other languages
Chinese (zh)
Other versions
CN110060280A (en
Inventor
周武能
傅衡成
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Donghua University
Original Assignee
Donghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Donghua University filed Critical Donghua University
Priority to CN201910349109.9A priority Critical patent/CN110060280B/en
Publication of CN110060280A publication Critical patent/CN110060280A/en
Application granted granted Critical
Publication of CN110060280B publication Critical patent/CN110060280B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20024—Filtering details

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Probability & Statistics with Applications (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a target tracking method based on an appearance self-adaptive spatial regularization correlation filter, which comprises the following steps: segmenting the target and the background of the tracking result image block by utilizing an online K-means clustering algorithm to obtain a target area template; generating a spatial regularization weight matrix by using a target area template; training and learning a correlation filter with space regularization by an Alternating Direction Multiplier Method (ADMM) to track the target. The invention can effectively limit the learning content of the correlation filter, reduce the background information of the correlation filter and inhibit the boundary effect of the correlation filter. Compared with the traditional correlation filter with spatial regularization, the method has the advantages that the target area and the background area are more accurately inhibited to different degrees; the search range of the correlation filter can be expanded, and the robustness of the correlation filter to large displacement of the target is improved.

Description

Target tracking method based on appearance self-adaptive spatial regularization correlation filter
Technical Field
The invention relates to a target tracking method based on an appearance self-adaptive spatial regularization correlation filter, and belongs to the technical field of video target tracking.
Background
The target tracking has important significance for the development of the fields of robots, unmanned planes, automatic driving, navigation, guidance and the like. For example, in the human-computer interaction process, the camera continuously tracks the human behavior, and the robot achieves the understanding of the human posture, the human motion and the human gesture through a series of analysis processing, so that the friendly communication between the human and the machine is better realized; in the unmanned aerial vehicle target tracking process, visual information of a target is continuously acquired and transmitted to a ground control station, and a video image sequence is analyzed through an algorithm to obtain real-time position information of the tracked target so as to ensure that the tracked target is within the visual field range of the unmanned aerial vehicle in real time.
When tracking an object in a video using a KCF algorithm, a fast motion or a violent motion of the tracked object may cause a search area of the KCF algorithm not to be completely covered, thereby causing a tracking failure. One way to ensure that the search area covers the target is by enlarging the area of the search area, but this introduces boundary effects that cause the filter to learn too much background information. There is a need to find an algorithm that can both enlarge the search area and suppress the background information of the filter.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: although the normalized spatial regularization related filtering algorithm can well solve the boundary effect, the regularization strategy of the algorithm only applies different penalty coefficients to the filter according to the positions of the coefficients of the filter, but the filter learning target information is not beneficial, so that the regularization strategy which can inhibit the boundary effect and better learn the target information needs to be found.
In order to solve the above technical problem, a technical solution of the present invention is to provide a target tracking method based on an appearance adaptive spatial regularization correlation filter, which is characterized by comprising the following steps:
(1) initializing the learning rate of a filter, the maximum iteration times of an ADMM algorithm, a Lagrange penalty factor and the size of a search box;
(2) extracting an image block containing a target from the t frame image, sorting each pixel point of the image block into a sample, and sequentially putting all sample points into a sequence D;
(3) clustering the sample points in the array D by using a K-means algorithm, and specifically setting as follows: measuring the similarity of the sample points by Euclidean distance, wherein the initial 5 centroids are designated as four vertexes of the rectangular image block and the central point of the rectangle, and finally obtaining the category of each sample point;
(4) arranging the sample points into a matrix P with the same size as the original image block according to the original sequence, wherein the elements of the matrix P are corresponding samplesThe class value of the point in the step (3) is that a matrix P with the same center point as that of the matrix P is intercepted from the matrix P1But matrix P1Is 0.6 times the current target size, for matrix P1Counting and sorting the number of the belonged classes, considering the class with the largest number as the class where the target is located, naming as the target class, if the current frame is not the first frame, adding a class which is closest to all the previous target classes, adding the target class, setting the position of the class belonging to the target class to be 1 by using the elements in the matrix P, and if not, setting the positions of the classes to be 0, finally obtaining a Mask matrix of the area where the target is located, resetting the value of the position with the value of 1 to be 0.01 according to the Mask matrix, resetting the value of the position with the value of 0 to be 100000, and naming as a weight matrix w;
(5) solving the filter by using an alternating direction multiplier method, wherein an objective function L (f, g) of the filter is as follows:
Figure BDA0002043341350000021
where f is the filter, g is the auxiliary variable, y is the label generated by the Gaussian function,
Figure BDA0002043341350000022
represents the D-th feature channel of the target image block of the t-th frame, D represents the total number of feature channels,
Figure BDA0002043341350000025
is a Lagrange multiplier, mu is a Lagrange penalty factor;
the ADMM algorithm solves the objective function by iteratively solving the following subproblems:
Figure BDA0002043341350000023
the above subproblems are all closed-form solutions:
Figure BDA0002043341350000024
the horizontal line on the matrix represents the frequency domain form of the matrix, and the elements of the matrix N are all 1;
(5) the filter trained in step (4) is recorded as
Figure BDA0002043341350000031
And updating the previous filter, wherein the updated formula is shown as follows:
Figure BDA0002043341350000032
in the formula (I), the compound is shown in the specification,
Figure BDA0002043341350000033
and eta is the learning rate of the filter.
(7) If t frame is not the last frame, use hi tAnd (3) scoring the candidate samples to obtain a response graph, taking the position with the maximum response value as the position of the target central point, and returning to the step (2), otherwise, ending the tracking.
Preferably, in step (2), the sample comprises 5 dimensions, and the sequence is: and carrying out dimension normalization processing on the R channel value, the G channel value, the B channel value, the X-axis coordinate and the Y-axis coordinate, so that the values are distributed in a [0,1] interval.
The invention can effectively limit the learning content of the correlation filter, reduce the background information of the correlation filter and inhibit the boundary effect of the correlation filter. Compared with the traditional correlation filter with spatial regularization, the method has the advantages that the target area and the background area are more accurately inhibited to different degrees; the search range of the correlation filter can be expanded, and the robustness of the correlation filter to large displacement of the target is improved.
Drawings
FIG. 1 is a flow algorithm implemented by the present invention;
fig. 2 is a process for obtaining the weight matrix according to the embodiment of the present invention.
Detailed Description
The invention will be further illustrated with reference to the following specific examples. It should be understood that these examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Further, it should be understood that various changes or modifications of the present invention may be made by those skilled in the art after reading the teaching of the present invention, and such equivalents may fall within the scope of the present invention as defined in the appended claims.
With reference to fig. 1, the target tracking method based on the appearance adaptive spatial regularization correlation filter provided by the present invention includes the following steps:
(1) initializing the learning rate of the filter, the maximum iteration times of the ADMM, a Lagrange penalty factor and the size of a search box.
(2) And extracting image blocks containing the target from the t frame image, and sorting each pixel point of the image blocks into a sample. This sample contains 5 dimensions, arranged in the order: r channel, G channel value, B channel value, X axis coordinate, and Y axis coordinate. And carrying out dimension-based normalization processing on the first 3 dimensions, so that the values are distributed in a [0,1] interval. Finally, all the sample points are put into the array D in sequence.
(3) Clustering the sample points in the array D by using a K-means algorithm, and specifically setting as follows: the similarity of the sample points is measured in euclidean distances, and the initial 5 centroids are specified as the four vertices of the rectangular image block and the center point of the rectangle. And finally obtaining the category of each sample point.
(6) Arranging the sample points into a matrix P with the same size as the original image block according to the original sequence, wherein the elements of the matrix P are the belonged values of the corresponding sample points in the step (3). Intercepting a matrix P with the same center point as the matrix P in the matrix P1But matrix P1Is 0.6 times the current target size. For matrix P1And counting and ordering the number of the belonged categories, and considering the category with the largest number as the category where the target is located, and naming the category as the target category. In addition, if the current frame is not the first frame, adding a frame closest to all the previous target classesAnd (4) adding the target class. And (3) setting the position of the element in the matrix P to be 1 according to the category belonging to the target class, otherwise, setting the position to be 0, finally obtaining a Mask matrix of the region where the target is located, resetting the value of the position with the value of 1 to be 0.01 according to the Mask matrix, resetting the value of the position with the value of 0 to be 100000, and naming the element as a weight matrix w.
(5) And solving the filter by using an alternating direction multiplier method, wherein the objective function of the filter is as follows:
Figure BDA0002043341350000041
where f is the filter, g is the auxiliary variable, y is the label generated by the Gaussian function,
Figure BDA0002043341350000042
represents the D-th feature channel of the target image block of the t-th frame, D represents the total number of feature channels,
Figure BDA0002043341350000044
is a lagrange multiplier and μ is a lagrange penalty factor. The ADMM algorithm solves the objective function by iteratively solving the following subproblems:
Figure BDA0002043341350000043
the subproblems in the above equation all have a closed form solution, as shown in the following equation:
Figure BDA0002043341350000051
in the above formula, the horizontal lines on the matrix represent the frequency domain form of the matrix, and the elements of the matrix N are all 1.
(6) The filter trained in step (5) is recorded as
Figure BDA0002043341350000052
And for the previous filterAnd updating the line, wherein the updated formula is shown as the following formula:
Figure BDA0002043341350000053
in the formula (I), the compound is shown in the specification,
Figure BDA0002043341350000054
and eta is the learning rate of the filter.
(7) If t frame is not the last frame, use
Figure BDA0002043341350000055
And (3) scoring the candidate samples to obtain a response graph, taking the position with the maximum response value as the position of the target central point, and returning to the step (2), otherwise, ending the tracking.
Fig. 2 is a process for obtaining the weight matrix according to the embodiment of the present invention.

Claims (2)

1. A target tracking method based on an appearance self-adaptive spatial regularization correlation filter is characterized by comprising the following steps:
(1) initializing the learning rate of a filter, the maximum iteration times of an ADMM algorithm, a Lagrange penalty factor and the size of a search box;
(2) extracting an image block containing a target from the t frame image, sorting each pixel point of the image block into a sample, and sequentially putting all sample points into a sequence M;
(3) clustering sample points in the number series M by using a K-means algorithm, and specifically setting as follows: measuring the similarity of the sample points by Euclidean distance, wherein the initial 5 centroids are designated as four vertexes of the rectangular image block and the central point of the rectangle, and finally obtaining the category of each sample point;
(4) arranging the sample points into a matrix P with the same size as the original image block according to the original sequence, wherein the elements of the matrix P are the values to which the corresponding sample points belong in the step (3), and intercepting one sample point in the matrix P to be consistent with the center point of the matrix PThe same matrix P1But matrix P1Is 0.6 times the current target size, for matrix P1Counting and sorting the number of the belonged classes, considering the class with the largest number as the class where the target is located, naming as the target class, if the current frame is not the first frame, adding a class which is closest to all the previous target classes, adding the target class, setting the position of the class belonging to the target class to be 1 by using the elements in the matrix P, and if not, setting the positions of the classes to be 0, finally obtaining a Mask matrix of the area where the target is located, resetting the value of the position with the value of 1 to be 0.01 according to the Mask matrix, resetting the value of the position with the value of 0 to be 100000, and naming as a weight matrix w;
(5) solving the filter by using an alternating direction multiplier method, wherein an objective function L (f, g) of the filter is as follows:
Figure FDA0002812036950000011
where f is the filter, g is the auxiliary variable, y is the label generated by the Gaussian function,
Figure FDA0002812036950000012
representing the target image block x of the t-th frametD represents the total number of feature channels,
Figure FDA0002812036950000013
is a Lagrange multiplier, mu is a Lagrange penalty factor;
the ADMM algorithm solves the objective function by iteratively solving the following subproblems:
Figure FDA0002812036950000021
the above subproblems are all closed-form solutions:
Figure FDA0002812036950000022
the horizontal line on the matrix represents the frequency domain form of the matrix, and the elements of the matrix N are all 1;
(6) The filter trained in step (4) is recorded as
Figure FDA0002812036950000023
And updating the previous filter, wherein the updated formula is shown as follows:
Figure FDA0002812036950000024
in the formula (I), the compound is shown in the specification,
Figure FDA0002812036950000025
a filter representing the ith feature of the t-th frame, wherein eta is the learning rate of the filter;
(7) if t frame is not the last frame, use
Figure FDA0002812036950000026
And (3) scoring the candidate samples to obtain a response graph, taking the position with the maximum response value as the position of the target central point, and returning to the step (2), otherwise, ending the tracking.
2. The method for tracking a target based on an appearance adaptive spatial regularization correlation filter according to claim 1, wherein in step (2), the samples comprise 5 dimensions, and are arranged in the following order: and carrying out dimension normalization processing on the R channel value, the G channel value, the B channel value, the X-axis coordinate and the Y-axis coordinate, so that the values are distributed in a [0,1] interval.
CN201910349109.9A 2019-04-28 2019-04-28 Target tracking method based on appearance self-adaptive spatial regularization correlation filter Active CN110060280B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910349109.9A CN110060280B (en) 2019-04-28 2019-04-28 Target tracking method based on appearance self-adaptive spatial regularization correlation filter

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910349109.9A CN110060280B (en) 2019-04-28 2019-04-28 Target tracking method based on appearance self-adaptive spatial regularization correlation filter

Publications (2)

Publication Number Publication Date
CN110060280A CN110060280A (en) 2019-07-26
CN110060280B true CN110060280B (en) 2021-03-30

Family

ID=67321280

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910349109.9A Active CN110060280B (en) 2019-04-28 2019-04-28 Target tracking method based on appearance self-adaptive spatial regularization correlation filter

Country Status (1)

Country Link
CN (1) CN110060280B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111767941B (en) * 2020-05-15 2022-11-18 上海大学 Improved spectral clustering and parallelization method based on symmetric nonnegative matrix factorization
CN112926693B (en) * 2021-04-12 2024-05-24 辽宁工程技术大学 Nuclear related filtering method for fast motion and motion blur

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104966304A (en) * 2015-06-08 2015-10-07 深圳市赛为智能股份有限公司 Kalman filtering and nonparametric background model-based multi-target detection tracking method
US9558712B2 (en) * 2014-01-21 2017-01-31 Nvidia Corporation Unified optimization method for end-to-end camera image processing for translating a sensor captured image to a display image
CN107818573A (en) * 2016-09-12 2018-03-20 杭州海康威视数字技术股份有限公司 A kind of method for tracking target and device
CN108664918A (en) * 2018-05-09 2018-10-16 吉林大学 Pedestrian tracting method in front of intelligent vehicle based on context-aware correlation filter
CN108776975A (en) * 2018-05-29 2018-11-09 安徽大学 A kind of visual tracking method based on semi-supervised feature and filter combination learning
CN108986139A (en) * 2018-06-12 2018-12-11 南京师范大学 A kind of band for target following is made a difference the feature integration method of figure

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101382873B1 (en) * 2012-06-29 2014-04-08 엘지이노텍 주식회사 Forward Collision Warning System and Forward Collision Warning Method
US9684951B2 (en) * 2014-03-31 2017-06-20 Los Alamos National Security, Llc Efficient convolutional sparse coding
CN104240256B (en) * 2014-09-25 2017-03-15 西安电子科技大学 A kind of image significance detection method based on the sparse modeling of stratification
CN105654069B (en) * 2016-02-03 2019-05-10 江南大学 Based on LpThe increment subspace method for tracking target of norm regularization

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9558712B2 (en) * 2014-01-21 2017-01-31 Nvidia Corporation Unified optimization method for end-to-end camera image processing for translating a sensor captured image to a display image
CN104966304A (en) * 2015-06-08 2015-10-07 深圳市赛为智能股份有限公司 Kalman filtering and nonparametric background model-based multi-target detection tracking method
CN107818573A (en) * 2016-09-12 2018-03-20 杭州海康威视数字技术股份有限公司 A kind of method for tracking target and device
CN108664918A (en) * 2018-05-09 2018-10-16 吉林大学 Pedestrian tracting method in front of intelligent vehicle based on context-aware correlation filter
CN108776975A (en) * 2018-05-29 2018-11-09 安徽大学 A kind of visual tracking method based on semi-supervised feature and filter combination learning
CN108986139A (en) * 2018-06-12 2018-12-11 南京师范大学 A kind of band for target following is made a difference the feature integration method of figure

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
《Correlation Filters with Limited Boundaries》;Galoogahi HK et al;《IEEE》;20151015;全文 *
《Robust Visual Tracking via Local-Global Correlation Filter》;Fan H et al;《ResearchGate》;20170204;全文 *
《多相关滤波自适应融合的鲁棒目标跟踪》;陈倩茹;《中国图象图形学报》;20180409;第23卷(第2期);全文 *
《成像探测系统中的红外弱小目标跟踪点漂移抑制方法研究》;孟晔铭;《中国优秀硕士学位论文全文数据库信息科技辑》;20160315(第2016年第03期);全文 *

Also Published As

Publication number Publication date
CN110060280A (en) 2019-07-26

Similar Documents

Publication Publication Date Title
CN109800689B (en) Target tracking method based on space-time feature fusion learning
CN108053419B (en) Multi-scale target tracking method based on background suppression and foreground anti-interference
CN111241931B (en) Aerial unmanned aerial vehicle target identification and tracking method based on YOLOv3
CN103295242B (en) A kind of method for tracking target of multiple features combining rarefaction representation
CN106845430A (en) Pedestrian detection and tracking based on acceleration region convolutional neural networks
CN110706266A (en) Aerial target tracking method based on YOLOv3
CN110688965B (en) IPT simulation training gesture recognition method based on binocular vision
CN110120065B (en) Target tracking method and system based on hierarchical convolution characteristics and scale self-adaptive kernel correlation filtering
CN111665842A (en) Indoor SLAM mapping method and system based on semantic information fusion
CN110021033A (en) A kind of method for tracking target based on the twin network of pyramid
CN111582349B (en) Improved target tracking algorithm based on YOLOv3 and kernel correlation filtering
CN110728694B (en) Long-time visual target tracking method based on continuous learning
CN112784869B (en) Fine-grained image identification method based on attention perception and counterstudy
CN111738355A (en) Image classification method and device with attention fused with mutual information and storage medium
CN109448023A (en) A kind of satellite video Small object method for real time tracking of combination space confidence map and track estimation
CN110060280B (en) Target tracking method based on appearance self-adaptive spatial regularization correlation filter
CN111445497A (en) Target tracking and following method based on scale context regression
US20070147678A1 (en) Adaptive Scene Dependent Filters In Online Learning Environments
Araujo et al. Air-sslam: A visual stereo indoor slam for aerial quadrotors
CN111914832A (en) SLAM method of RGB-D camera in dynamic scene
Alsanad et al. Real-time fuel truck detection algorithm based on deep convolutional neural network
CN113420648B (en) Target detection method and system with rotation adaptability
CN112241736A (en) Text detection method and device
CN114627339A (en) Intelligent recognition and tracking method for border crossing personnel in dense jungle area and storage medium
CN116523957A (en) Multi-target tracking method, system, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant