CN114897932A - Infrared target tracking implementation method based on feature and gray level fusion - Google Patents
Infrared target tracking implementation method based on feature and gray level fusion Download PDFInfo
- Publication number
- CN114897932A CN114897932A CN202210343922.7A CN202210343922A CN114897932A CN 114897932 A CN114897932 A CN 114897932A CN 202210343922 A CN202210343922 A CN 202210343922A CN 114897932 A CN114897932 A CN 114897932A
- Authority
- CN
- China
- Prior art keywords
- target
- scale
- point
- feature
- frame image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 77
- 230000004927 fusion Effects 0.000 title claims abstract description 14
- 238000000605 extraction Methods 0.000 claims abstract description 17
- 239000011159 matrix material Substances 0.000 claims description 51
- 238000012549 training Methods 0.000 claims description 16
- 238000010586 diagram Methods 0.000 claims description 14
- 230000004044 response Effects 0.000 claims description 13
- 238000012946 outsourcing Methods 0.000 claims description 9
- 238000000354 decomposition reaction Methods 0.000 claims description 7
- 230000009466 transformation Effects 0.000 claims description 5
- 230000001131 transforming effect Effects 0.000 claims description 4
- 238000000926 separation method Methods 0.000 claims 1
- 238000003331 infrared imaging Methods 0.000 abstract description 6
- 230000008859 change Effects 0.000 description 14
- 238000001514 detection method Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 8
- 239000013598 vector Substances 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000005286 illumination Methods 0.000 description 4
- 238000005070 sampling Methods 0.000 description 4
- 230000011218 segmentation Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000000877 morphologic effect Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000005057 refrigeration Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000002187 spin decoupling employing ultra-broadband-inversion sequences generated via simulated annealing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Aiming, Guidance, Guns With A Light Source, Armor, Camouflage, And Targets (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an infrared target tracking implementation method based on feature and gray level fusion. For each frame, firstly, searching a full image, taking out an area block with the same size as the gray template and carrying out normalized correlation operation on the area block and the gray template by taking each pixel point as a center to obtain a roughly predicted target central point position, cutting out a predicted area, carrying out feature extraction and position filter correlation operation to obtain the accurate position of the target central point, then selecting area blocks with multiple scales according to the accurate position of the target central point, extracting features and scale filter correlation operation to obtain a target scale, and simultaneously updating based on the target position and the target scale to obtain a new filter for subsequent prediction. The method improves the matching probability of template matching, can be used for locking and tracking the end target of the infrared imaging air-ground guided weapon, and has good application prospect.
Description
Technical Field
The invention relates to an infrared target tracking implementation method based on feature and gray level fusion, in particular to a target tracking method in an uncooled infrared image terminal guidance stage in an air-to-ground guidance weapon, and belongs to the field of accurate guidance.
Background
The uncooled infrared imaging guidance has the advantages of small size, light weight, low power consumption, low cost and the like, becomes an important direction for the development of infrared technology, has huge development space in the military field, and is the key point of recent tactical guidance weapon research. Because the uncooled infrared imaging sensitivity is low, the edge of an image is fuzzy and is easily influenced by ambient illumination, noise interference and the like, the texture or color characteristics of the image often fluctuate greatly between adjacent frames, the imaging quality is greatly different from that of a refrigeration detector, and ideal matching accuracy and accuracy are difficult to obtain by using a traditional matching method.
Common target tracking methods include those based on gray scale models, on target contours, and on target features. The method based on the gray scale model mainly comprises an average absolute difference (MAD), normalization product correlation (NCC) and the like, the method adopts gray scale information of a target, the tracking effect is poor when the target template is shielded to a large extent or illumination changes, and the template updating strategy problem exists when the target is subjected to scale changes. The contour-based tracking method obtains a target contour by a certain method, and then recursively solves the target contour in each frame of image through a differential equation until the contour converges to a local smaller value of an energy function. However, the method is sensitive to the initial position when initializing the target contour, and the tracking accuracy is limited. The tracking method based on the characteristics has wider application range, the detection algorithm mainly comprises the detection of angular points, edges, gradient characteristics and the like, the most appropriate area is selected as the final target position by comparing the corresponding characteristics of each image area, and the method can be based on the characteristics of the local areas of the images, so that the target tracking can be realized even if the target is partially shielded. The feature detection algorithm based on the corner points comprises a Moravec detection algorithm, a SUSAN detection algorithm, a Maximum Stable Extremum Region (MSER) detection algorithm, a SIFT algorithm, a PCA-SIFT algorithm, a GLOD algorithm, a SURF algorithm and the like; the detection algorithm based on the edge and the gradient comprises a Sobel operator, a Roberts operator, a Canny operator, a LOG operator, a HOG and the like; however, when the tracking method based on the characteristics is applied to short-time tracking in which the speed of a projectile is fast enough relative to a target and the target is generally hit within tens of seconds from shooting to hitting, in addition to the conventional target position change, factors such as rapid scale change when the projectile approaches the target quickly need to be considered, and the traditional characteristic matching method is not suitable for the problem of rapid view angle change.
Disclosure of Invention
The invention aims to overcome the defects and provides an infrared target tracking implementation method based on feature and gray level fusion. For each frame, firstly searching in a full graph, taking out an area block with the same size as the gray template and the gray template by taking each pixel point as the center, carrying out normalization correlation operation to obtain the position of a roughly predicted target central point, cutting out the predicted area, carrying out feature extraction, carrying out correlation operation on the features and a position filter to obtain the accurate position of the target central point, selecting area blocks with multiple scales according to the accurate position of the target central point, extracting the features, carrying out correlation operation on the features and the scale filter to obtain the target scale, and updating based on the target position and the target scale to obtain a new filter for subsequent prediction. The method improves the matching probability of template matching, can be used for locking and tracking the end target of the infrared imaging air-ground guided weapon, and has good application prospect.
In order to achieve the above purpose, the invention provides the following technical scheme:
an infrared target tracking implementation method based on feature and gray level fusion comprises the following steps:
marking the position of the target central point in the t frame image as P t The target frame is denoted as Rect t T is an integer of 1 or more;
when t is 1, according to the position P of the target central point in the 1 st frame image 1 Determining a target frame Rect 1 ；
Target frame Rect in the 1 st frame image 1 The inner area is used as a gray template Mask 1 (ii) a Mask for gray template 1 Extracting features to obtain a feature map, and training a position filter and a scale filter by using the feature map;
when t > 1, according to the increasing order of t value, the following steps S1-S4 are executed in a loop:
s1 taking the pixel point in the whole picture range of the t-th frame image as the central point, taking out the pixel point and taking out the gray template Mask t-1 Region blocks of the same size; mask according to the region block and the gray level template i-1 Preliminarily predicting the position P of the target center point in the t frame image by the correlation operation result t ′；
S2 pairs with P t Mask with gray scale template as central point t-1 Extracting the features of the region blocks with the same size to obtain a feature map, and predicting the position P of the target center point in the t-th frame image according to the correlation operation result of the feature map and the position filter t ；
S3 pairs with P t Extracting features of the region blocks with multiple scales of the central point to obtain a feature map, and obtaining a target frame Rect in the t frame image according to the correlation operation result of the feature map and the scale filter t ；
S4 extracting target frame Rect in t frame image t The inner area is used as a gray template Mask t (ii) a For gray level template Mask t And extracting features to obtain a feature map, and training a position filter and a scale filter by using the feature map.
Further, according to the position P of the target central point in the 1 st frame image 1 Determining a target frame Rect 1 The method comprises the following steps:
according to the position P of the target central point in the 1 st frame image 1 Preliminarily determining the range of a target area;
the position P of the center point of the target in the 1 st frame image 1 Taking the target contour point as a center, and locally dividing the target area range to obtain a target contour point;
obtaining a target minimum outsourcing rectangle according to the target contour points;
the minimum outsourcing rectangle is extended by 3-5 pixels to obtain the target frame Rect 1 。
Further, the position P of the target center point in the 1 st frame image is taken 1 Taking the target contour point as a center, locally dividing the range of the target area to obtain the target contour point, wherein the method comprises the following steps:
will P 1 The surrounding area is divided into 8-20 directions with P 1 And determining whether the gradient of the pixel in each direction is greater than a preset threshold value or not for the starting point, and taking the pixel as a target contour point when the gradient of a certain pixel is greater than the preset threshold value and the gradient of the previous pixel is less than or equal to the preset threshold value.
Further, the feature extraction is HOG feature extraction.
Further, the HOG features include gradient direction and gradient magnitude:
gradient amplitude of any pixel point I (x, y)
The gradient direction β (x, y) of any pixel point I (x, y) is arctan (G) 1y /G 1x )；
Wherein G is 1x Is the gradient, G, of the pixel point I (x, y) in the horizontal direction 1y The gradient of the pixel point I (x, y) in the vertical direction, and β (x, y) is the angle of the gradient of the pixel point I (x, y) relative to the x-axis.
Further, in step S1, taking each pixel point searched by the interval point in the whole image range of the t-th frame image as a central point, and taking out the pixel point and the gray template Mask t-1 The same size of region block.
Further, the method for training the position filter and the scale filter by using the feature map comprises the following steps:
obtaining a Gaussian matrix with the same size as the characteristic diagram according to the characteristic diagram;
and training a position filter and a scale filter by utilizing the Gaussian matrix and the characteristic diagram after Fourier transform.
Further, in step S1, Mask is generated according to the area block and the gray-scale template i-1 Preliminarily predicting the position P of the target center point in the t frame image by the correlation operation result t The method of' is:
mask the area block and the gray template i-1 Carrying out normalized correlation operation to obtain a correlation coefficient matrix;
taking the pixel point corresponding to the maximum correlation coefficient in the correlation coefficient matrix as P t ′。
Further, in step S2, the position P of the target center point in the t-th frame image is predicted according to the correlation operation result of the feature map and the position filter t The method comprises the following steps:
in the t-th frame image with P t ' as center point, cut and Mask i-1 Extracting the features of the region blocks with the same size, and calculating a feature map and a template Mask i-1 Transforming the Gaussian kernel of the characteristic map to a frequency domain through Fourier transformation, matching the Gaussian kernel transformed to the frequency domain with a position filter to obtain a response map, and taking a position coordinate corresponding to a maximum peak value in the response map subjected to inverse Fourier transformation as a position P of a target center point in the t-th frame image t 。
Further, in step S3, the target frame Rect in the t-th frame image is obtained according to the correlation operation result of the feature map and the scale filter t The method comprises the following steps:
adjusting the characteristic diagram into the same size to form a scale characteristic matrix, performing singular value decomposition on the scale characteristic matrix to obtain characteristic values, and before utilization, performing the singular value decomposition on the characteristic values
Obtaining a characteristic matrix to be detected by the largest characteristic value, carrying out Fourier transform on the characteristic matrix to be detected to obtain a frequency domain scale matrix, and calculating the frequency domain scale matrix and a gray template Mask by using a scale filter t-1 The scale corresponding to the maximum peak value in the matching response map is taken as Rect t 。
Compared with the prior art, the invention has the following beneficial effects:
(1) the method adopts a combined template matching method to realize coarse positioning of the target, adopts a characteristic matching method to realize fine positioning of the target, solves the problems that the common algorithm is not suitable for scale change, visual angle change and the like in the target tracking process, improves the robustness of the target tracking process, is particularly suitable for target tracking of large-scale change at the last guidance stage of an accurately guided weapon, and is a tracking method suitable for an infrared imaging air-ground guided weapon ground target;
(2) according to the method, the target area in the first frame image is automatically detected, so that the target area can be effectively hit, the actual operation process is simplified, the subjectivity of manual selection is avoided, and the algorithm is more objective; the accuracy can be further improved by combining the automatic contour detection method with manual adjustment;
(3) the method integrates the gradient amplitude, gradient direction and gray level of the image as features, and can furthest reserve the common features of the infrared image after the position change and the scale change of the target;
(4) according to the method, the gradient direction image is constructed, and the gradient direction information of the target area in the infrared image is extracted, so that the same characteristics can be obtained when the target scale change difference of the infrared image is large; and then, the most similar position in the real-time image is found out through a matching algorithm, so that the accuracy of the position prediction of the target is improved.
Drawings
FIG. 1 is a flow chart of an infrared target tracking implementation method of the present invention;
FIG. 2 is a schematic diagram illustrating detection of a target frame in a first frame image according to the present invention;
FIG. 3 is a schematic diagram of the tracking effect obtained by the infrared target tracking implementation method of the present invention; wherein (a) is a schematic diagram of a first frame image, (b) is a schematic diagram of a 150 th frame image, (c) is a schematic diagram of a 300 th frame image, and (d) is a schematic diagram of a 450 th frame image.
Detailed Description
The features and advantages of the present invention will become more apparent and appreciated from the following detailed description of the invention.
The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
When the seeker tracks the target, the speed is fast enough relative to the target, and the target can be finished within ten seconds from shooting to hitting, which belongs to short-time tracking. In the short-time tracking process, besides the conventional target position change, the rapid change of the scale when the projectile body rapidly approaches the target is one of the main problems to be considered in the tracking process. The method considers the influence of factors such as appearance transformation, illumination change and the like of the target in the actual tracking process, the edge shape characteristic of the target has better robustness to illumination or noise, the target tracking method based on the gradient characteristic is selected, and a plurality of templates of the target are considered as references, so that the robustness of the tracking method is improved.
In the method for realizing the infrared target tracking algorithm based on feature and gray level fusion, the size of a target is automatically detected through a target central point pixel, a gray level template is established, gradient features of a template area are extracted, and then training is carried out to obtain a target tracker. For each frame, firstly, searching at the interval points of the whole image, taking out the area block with the same size as the gray template obtained in the previous frame by taking each pixel point as the center, carrying out normalized correlation operation on the area block and the gray template to obtain a correlation coefficient matrix, taking the point corresponding to the maximum correlation coefficient in the matrix as the position of the target central point predicted in the image of the frame, cutting down the predicted area, carrying out feature extraction, calculating the Gaussian kernel of the feature image and the template, transforming the Gaussian kernel to the frequency domain through Fourier transform, then multiplying the frequency domain by a position filter to obtain a response image, carrying out inverse Fourier transform on the result, obtaining the area where the maximum response point is located as the new position of the target to be tracked, and then training and updating the new position area to obtain a new position filter for subsequent prediction. The matching of a single template has great limitation, two or more templates can be combined, correlation matching is carried out in a real-time image, and robustness and accuracy are obviously improved. The method solves the problem that the common template matching target tracking method is not suitable for scale change and the common characteristic matching method is not suitable for rapid view angle change, can effectively improve the robustness of target tracking, can be used for tracking the end target of an infrared imaging air-ground guided weapon, and has good application prospect.
The invention relates to an infrared target tracking algorithm implementation method based on feature and gray level fusion, which specifically comprises the following steps:
(1) firstly, according to the 1 st frame image target central point pixel P 1 (x, y) preliminarily determining a target area to be tracked; the method comprises the steps of carrying out local segmentation on a target area, searching for a target edge, calculating a target minimum outsourcing rectangle by combining a morphological method, expanding 3-5 pixels on the basis of the minimum outsourcing rectangle, and determining a target frame Rect 1 Size;
the specific method comprises the following steps: by the target center point pixel P 1 The (x, y) position is used as the center of a circle, the surrounding gray distribution condition is calculated, 360-degree direction angles are divided into 8 directions at a unit interval of 45 degrees, an initial contour is obtained by judging points with gradient of each direction being larger than a threshold value (set to be 20 in practical application), and then region growing or OTSU binary segmentation and morphological filtering are combined, so that the segmentation region is more accurate, the target minimum outsourcing rectangle is obtained through calculation, and the size of a target frame is obtained.
(2) According to the target frame Rect 1 Creating a grayscale template Mask 1 Initializing model parameters of a Tracker model based on a position filter and a scale filter, and intercepting target frame Rect 1 And (3) extracting the features of the image of the target area in the Tracker, generating a Gaussian matrix with the same size as the feature map, and updating parameters of a position estimation model and a scale estimation model in the Tracker by combining a fast Fourier transform training position filter and a scale filter.
(3) For the non-first frame image, firstly, searching at the interval points of the whole image, taking out the gray obtained from the previous frame by taking each point as the centerArea blocks with the same size as the degree template and the gray template are subjected to normalized correlation operation to obtain a correlation coefficient matrix, and a point corresponding to the maximum correlation coefficient in the matrix is taken as a preliminarily predicted target point position P in the frame image t ' intercepting a target candidate region for feature extraction; performing correlation operation (preferably Gaussian kernel correlation) by using the position filter and the feature map, searching the maximum correlation position, and obtaining the predicted target central point position P in the frame image t ；
P t The specific determination method comprises the following steps:
with P t Taking the center as the center, intercepting image blocks of the target area for feature extraction, calculating Gaussian kernels of a candidate area and a template, transforming the candidate area and the template into a frequency domain through Fourier transform (FFT), calculating a matching response graph of the template and the candidate area, and searching a maximum peak value and a corresponding coordinate position, namely the maximum peak value and the corresponding coordinate position are the position P of the target center point t 。
(4) Intercepting a plurality of scale candidate areas according to the position of a target central point predicted by the frame, and extracting features; performing correlation operation (preferably Gaussian kernel correlation) by using a scale filter and the feature map, searching the maximum correlation scale, and obtaining the target scale estimation predicted in the frame image, namely a target frame;
the specific method comprises the following steps: intercepting a plurality of scale target image blocks for feature extraction, readjusting all scale feature blocks to be of the same size (down sampling or up sampling) to form a scale feature matrix, performing singular value decomposition on the scale feature matrix, and taking the previous scale feature matrix before the singular value decomposition
And forming a projection matrix by using the corresponding characteristic vectors of the maximum characteristic values, obtaining a compressed characteristic matrix to be detected after the action of the projection matrix, obtaining a frequency domain scale matrix by performing Fourier transform, calculating a relevant response by using a scale filter, solving the relevant response, and seeking the maximum response value and the corresponding scale, namely the latest scale of the target.
(5) According to the position and the scale of the target central point of the frame, intercepting a target area of the frame, extracting a feature training position filter and a scale filter, updating parameters of a position estimation model and a scale estimation model, and completing infrared target tracking;
in the step, target feature extraction is carried out according to the target position and the target scale which are continuously updated along with the number of the image frames to obtain a new Tracker model. In the frequency domain, a position filter and a scale filter are trained according to a characteristic diagram obtained by a Tracker model, so that the process of training first, then detecting, then training and then detecting is realized, and the process is continuously circulated.
Example 1:
as shown in fig. 1, the specific steps of this embodiment are as follows:
the method comprises the following steps: firstly, according to the pixel position P of the target central point in the first frame image (default is a gray level image) 1 (x, y) preliminarily determining a target area to be tracked by the position; with P 1 (x, y) as the center, locally dividing the target area, searching the target edge, extracting the target contour point, calculating the minimum outsourcing rectangle of the target, and then extending 4 pixels to determine the target frame Rect on the basis of the minimum outsourcing rectangle 1 Size, as shown in fig. 2.
Step two: according to the target frame Rect 1 Size-establishing gray level template Mask 1 . Initializing Tracker model parameters and extracting Rect 1 Obtaining a feature map tmpl by HOG features of the target region, generating a Gaussian matrix prob with the same size as the feature map tmpl, training a position filter and a scale filter by the feature map tmpl and the Gaussian matrix prob by combining fast Fourier transform, and updating parameters of a position estimation model and a scale estimation model in a Tracker model.
The feature extraction can be divided into the following steps: determining a target detection window, normalizing an image, calculating a gradient, counting a gradient histogram and normalizing the gradient histogram to finally obtain a feature vector, namely a feature map;
HOG extraction features refer to gradient direction and gradient amplitude, and for gradient amplitude | G of pixel point I (x, y) in image 1 (X, Y) | satisfies
Wherein G is 1x Is the gradient of pixel point I (x, y) in the horizontal direction, G 1y The gradient of the pixel point I (x, y) in the vertical direction is shown.
The gradient direction of the pixel point I (x, y) in the image is defined as:
β(x,y)＝arctan(G 1y /G 1x )
where β (x, y) is the angle of the gradient of the pixel point I (x, y) with respect to the x-axis.
Step three: for the non-first frame image, it is recorded as the t-th frame image, t > 1, and the target center point position estimation is performed first. In the full-image alternate point search, taking each point as the center, taking out the area block with the same size as the gray template obtained in the previous frame and carrying out the normalized correlation operation with the gray template to obtain a correlation coefficient matrix, and taking the point corresponding to the maximum correlation coefficient in the matrix as the target central point position P preliminarily predicted in the image t ', by P t ' intercepting a target candidate region as a center to extract features; performing correlation operation (Gaussian kernel correlation) by using the position filter and the feature map, searching the maximum correlation position, and obtaining the predicted target central point position P t . In this step, P t ' As a coarse prediction result, P t In order to accurately predict the result, the searching at the interval points of the whole image is beneficial to improving the processing speed.
Assuming that an input image block is f, obtaining a feature map corresponding to each image block through a feature extraction algorithm, wherein a feature dimension is d, and a feature map of each dimension (l ∈ {1, …, d }) is marked as f l Then for each image block the goal is to minimize the loss function epsilon:
in the above formula, the subscript t represents the t-th frame, λ is a regularization coefficient, which represents a correlation operation, g represents a response output (correlation output),
a grayscale template generated for the ith frame of dimension is represented. After applying the Parseval theorem formula to the above formula and performing discrete Fourier transform, the optimal filter can be obtained as follows:
filter trained on t-th frame
Only with respect to the ideal output G of the object and the characteristics F of the image block, and considering the information of the time dimension
Respectively update the numerator and denominator of
As its molecule, B t And (3) respectively carrying out iterative updating for denominators, wherein eta is the learning rate, and the updating steps are as follows:
when a new frame (such as a t-th frame) comes, selecting an image block Z from the t frame according to the target prediction position, obtaining a feature matrix Z of the image block Z after the same feature extraction algorithm, wherein the dimension is d, and a feature map of each dimension (l ∈ {1, …, d }) is marked as d
Then in the frequency domain, the corresponding fourier output Y t Comprises the following steps:
for Y t After inverse discrete Fourier transform, a confidence map of the target in a t frame can be obtained
By finding y t We obtain the position of the target at the tth frame,
representing a fourier transform.
Step four: and estimating the target scale according to the predicted position of the target center point in the t-th frame. Intercepting a plurality of scale (actual use parameters are 21) candidate regions, and extracting features; performing correlation operation (Gaussian kernel correlation) by using the scale filter and the feature map, searching the maximum correlation scale, and obtaining the target scale Rect of the t-th frame t 。
The scale estimation of the target is similar to the position estimation method of the target, and the scale estimation of the target is to combine the characteristics of the target to construct a scale information vector filter of the target. When a new frame of the target comes, the rough position of the target is obtained by using the gray template, and then the accurate position P of the target is obtained according to the position filter t . Assuming S scale levels, for each
We will use P t Is taken as the center, with a n w〃a n h (a is a scale step length which is set to be 1.07 in practical use, w and h are the width and the height of a target frame in a previous frame) are different in size to extract image blocks of different scales of a target in a new frame, all the image blocks with the scales are readjusted to be the same in size (down sampling or up sampling), a feature extraction algorithm is used for extracting corresponding d-dimensional features for the image blocks with each scale, and a feature column vector of the image block corresponding to the scale n is recorded as f t (n), then f t The dimension of (n) should be d × 1.
Composing all the dimensional characteristic column vectors into one dimensionFeature matrix f t ＝[f t (1)…f t (S)]Then f t The corresponding feature dimension is dXS, and in general, the feature dimension extracted by the feature extraction algorithm is far higher than the number of the selected target scales, i.e. d > S, so the matrix f t Is less than or equal to S, i.e., rank (f) t ) S ≦ that is, our scale feature matrix may be compressed to save computation. To f t Performing singular value decomposition, and taking the previous value
Forming a projection matrix P by the corresponding eigenvectors t f Then, then
Projection matrix
To f t Performing line compression with dimensions of
The compressed scale feature expression matrix is
Having a dimension of
We use the compressed scale feature matrix instead of the original scale feature matrix to perform fourier transform, thereby reducing the time required for fourier transform. Fourier transform of a scaled feature matrix of the object to be registered
Then B is t The corresponding updating mode is as follows:
remember u t ＝(1-η)u t-1 +ηf t To u, to u t The same singular value decomposition operation is also carried out, and a projection matrix P with the same dimension is obtained t u 。
From the linear nature of the Fourier transform
The equivalent update form of (1) is:
when the t-th frame of a new frame comes, the predicted position and the scale of the previous frame are taken as the reference, and the features z of the image blocks of S scales are extracted t Through the projection matrix
Obtaining a compressed characteristic matrix to be detected after action
Then to
By performing Fourier transform to obtain
The frequency domain scale matrix can be obtained in an substituting way
And calculating the scale corresponding to the position of the maximum value of the matrix and recording the scale corresponding to the t frame target.
Step five: according to the latest target position and scale, the target area is intercepted, the feature training filter is extracted, the position estimation model and the scale estimation model parameters are updated, the infrared target tracking method is completed, the effect is shown in figure 3, and it can be seen that the accurate tracking effect is obtained by using the method.
The invention has been described in detail with reference to specific embodiments and illustrative examples, but the description is not intended to be construed in a limiting sense. Those skilled in the art will appreciate that various equivalent substitutions, modifications or improvements may be made to the technical solution of the present invention and its embodiments without departing from the spirit and scope of the present invention, which fall within the scope of the present invention. The scope of the invention is defined by the appended claims.
Those skilled in the art will appreciate that those matters not described in detail in the present specification are well known in the art.
Claims (10)
1. An infrared target tracking implementation method based on feature and gray level fusion is characterized by comprising the following steps:
marking the position of the target central point in the t frame image as P t The target frame is denoted as Rect t T is an integer of 1 or more;
when t is 1, according to the position P of the target central point in the 1 st frame image 1 Determining a target frame Rect 1 ；
Target frame Rect in the 1 st frame image 1 The inner area is used as a gray template Mask 1 (ii) a Mask for gray template 1 Extracting features to obtain a feature map, and training a position filter and a scale filter by using the feature map;
when t > 1, according to the increasing order of t value, the following steps S1-S4 are executed in a loop:
s1 taking out the pixel point in the whole picture range of the t-th frame image as the central point and taking out the pixel point and the gray template Mask t-1 Region blocks of the same size; mask according to the region block and the gray level template i-1 Phase ofPreliminarily predicting the position P of the target central point in the t frame image according to the correlation operation result t ′；
S2 pairs with P t Mask with gray scale template as central point t-1 Extracting the features of the area blocks with the same size to obtain a feature map, and predicting the position P of the target center point in the t-th frame image according to the correlation operation result of the feature map and the position filter t ；
S3 pairs with P t Extracting features of the region blocks with multiple scales of the central point to obtain a feature map, and obtaining a target frame Rect in the t frame image according to the correlation operation result of the feature map and the scale filter t ；
S4 extracting target frame Rect in t frame image t The inner area is used as a gray template Mask t (ii) a Mask for gray template t And extracting features to obtain a feature map, and training a position filter and a scale filter by using the feature map.
2. The method for realizing infrared target tracking based on feature and gray scale fusion as claimed in claim 1, wherein the method is implemented according to the position P of the target center point in the 1 st frame image 1 Determining a target frame Rect 1 The method comprises the following steps:
according to the position P of the target central point in the 1 st frame image 1 Preliminarily determining the range of a target area;
the position P of the center point of the target in the 1 st frame image 1 Taking the target contour point as a center, and locally dividing the target area range to obtain a target contour point;
obtaining a target minimum outsourcing rectangle according to the target contour points;
the minimum outsourcing rectangle is extended by 3-5 pixels to obtain the target frame Rect 1 。
3. The method for realizing infrared target tracking based on feature and gray scale fusion as claimed in claim 2, wherein the target central point position P in the 1 st frame image is used 1 Taking the target contour point as a center, locally dividing the range of the target area to obtain the target contour point, wherein the method comprises the following steps:
will P 1 The surrounding area is divided into 8-20 directions with P 1 And determining whether the gradient of the pixel in each direction is greater than a preset threshold value or not for the starting point, and taking the pixel as a target contour point when the gradient of a certain pixel is greater than the preset threshold value and the gradient of the previous pixel is less than or equal to the preset threshold value.
4. The method as claimed in claim 1, wherein the feature extraction is HOG feature extraction.
5. The method of claim 4, wherein the HOG features comprise gradient direction and gradient amplitude:
gradient amplitude of any pixel point I (x, y)
The gradient direction β (x, y) of any pixel point I (x, y) is arctan (G) 1y /G 1x )；
Wherein G is 1x Is the gradient of pixel point I (x, y) in the horizontal direction, G 1y The gradient of the pixel point I (x, y) in the vertical direction, and β (x, y) is the angle of the gradient of the pixel point I (x, y) relative to the x-axis.
6. The method for realizing infrared target tracking based on feature and gray scale fusion as claimed in claim 1, wherein in step S1, taking out and gray scale template Mask with each pixel point searched for at a separation point in the whole picture range of the t-th frame image as a center point t-1 The same size of region block.
7. The method for realizing infrared target tracking based on feature and gray scale fusion as claimed in claim 1, wherein the method for training the position filter and the scale filter by using the feature map comprises:
obtaining a Gaussian matrix with the same size as the characteristic diagram according to the characteristic diagram;
and training a position filter and a scale filter by utilizing the Gaussian matrix and the characteristic diagram after Fourier transform.
8. The method for realizing infrared target tracking based on feature and gray scale fusion as claimed in claim 1, wherein in step S1, Mask is applied according to the area block and gray scale template i-1 Preliminarily predicting the position P of the target center point in the t frame image by the correlation operation result t The method of' is:
mask the area block and the gray template i-1 Carrying out normalized correlation operation to obtain a correlation coefficient matrix;
taking the pixel point corresponding to the maximum correlation coefficient in the correlation coefficient matrix as P t ′。
9. The method as claimed in claim 1, wherein in step S2, the position P of the target center point in the t-th frame image is obtained by predicting according to the correlation result between the feature map and the position filter t The method comprises the following steps:
in the t-th frame image with P t ' as center point, cut and Mask i-1 Extracting the features of the region blocks with the same size, and calculating a feature map and a template Mask i-1 Transforming the Gaussian kernel of the characteristic map to a frequency domain through Fourier transformation, matching the Gaussian kernel transformed to the frequency domain with a position filter to obtain a response map, and taking a position coordinate corresponding to a maximum peak value in the response map subjected to inverse Fourier transformation as a position P of a target center point in the t-th frame image t 。
10. The method as claimed in claim 1, wherein in step S3, the target frame Rect in the t-th frame image is obtained according to the correlation operation result between the feature map and the scale filter t The method comprises the following steps:
adjusting the feature map to the same size to form a scale feature matrix, and aligning the scaleThe degree characteristic matrix is subjected to singular value decomposition to obtain characteristic values before utilization
Obtaining a characteristic matrix to be detected by the largest characteristic value, carrying out Fourier transform on the characteristic matrix to be detected to obtain a frequency domain scale matrix, and calculating the frequency domain scale matrix and a gray template Mask by using a scale filter t-1 The scale corresponding to the maximum peak value in the matching response map is taken as Rect t 。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210343922.7A CN114897932B (en) | 2022-03-31 | 2022-03-31 | Infrared target tracking realization method based on feature and gray level fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210343922.7A CN114897932B (en) | 2022-03-31 | 2022-03-31 | Infrared target tracking realization method based on feature and gray level fusion |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114897932A true CN114897932A (en) | 2022-08-12 |
| CN114897932B CN114897932B (en) | 2024-07-19 |
Family
ID=82715811
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210343922.7A Active CN114897932B (en) | 2022-03-31 | 2022-03-31 | Infrared target tracking realization method based on feature and gray level fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114897932B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116503933A (en) * | 2023-05-24 | 2023-07-28 | 北京万里红科技有限公司 | Periocular feature extraction method and device, electronic equipment and storage medium |
| CN117011196A (en) * | 2023-08-10 | 2023-11-07 | 哈尔滨工业大学 | Infrared small target detection method and system based on combined filtering optimization |
| CN117576380A (en) * | 2024-01-16 | 2024-02-20 | 成都流体动力创新中心 | Target autonomous detection tracking method and system |
| CN118071826A (en) * | 2024-01-29 | 2024-05-24 | 广东永浩信息技术有限公司 | Target positioning method and system based on video monitoring |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170091952A1 (en) * | 2015-09-30 | 2017-03-30 | Apple Inc. | Long term object tracker |
| CN106570486A (en) * | 2016-11-09 | 2017-04-19 | 华南理工大学 | Kernel correlation filtering target tracking method based on feature fusion and Bayesian classification |
| CN110009664A (en) * | 2019-04-11 | 2019-07-12 | 南京航空航天大学金城学院 | A kind of infrared object tracking method and device based on response diagram fusion |
| CN111563915A (en) * | 2020-05-06 | 2020-08-21 | 南京航空航天大学 | KCF target tracking method integrating motion information detection and Radon transformation |
| CN112561845A (en) * | 2020-12-21 | 2021-03-26 | 阜阳强松航空科技有限公司 | Long-term tracking method based on infrared and visible light fusion |
-
2022
- 2022-03-31 CN CN202210343922.7A patent/CN114897932B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170091952A1 (en) * | 2015-09-30 | 2017-03-30 | Apple Inc. | Long term object tracker |
| CN106570486A (en) * | 2016-11-09 | 2017-04-19 | 华南理工大学 | Kernel correlation filtering target tracking method based on feature fusion and Bayesian classification |
| CN110009664A (en) * | 2019-04-11 | 2019-07-12 | 南京航空航天大学金城学院 | A kind of infrared object tracking method and device based on response diagram fusion |
| CN111563915A (en) * | 2020-05-06 | 2020-08-21 | 南京航空航天大学 | KCF target tracking method integrating motion information detection and Radon transformation |
| CN112561845A (en) * | 2020-12-21 | 2021-03-26 | 阜阳强松航空科技有限公司 | Long-term tracking method based on infrared and visible light fusion |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116503933A (en) * | 2023-05-24 | 2023-07-28 | 北京万里红科技有限公司 | Periocular feature extraction method and device, electronic equipment and storage medium |
| CN116503933B (en) * | 2023-05-24 | 2023-12-12 | 北京万里红科技有限公司 | Periocular feature extraction method and device, electronic equipment and storage medium |
| CN117011196A (en) * | 2023-08-10 | 2023-11-07 | 哈尔滨工业大学 | Infrared small target detection method and system based on combined filtering optimization |
| CN117011196B (en) * | 2023-08-10 | 2024-04-19 | 哈尔滨工业大学 | Infrared small target detection method and system based on combined filtering optimization |
| CN117576380A (en) * | 2024-01-16 | 2024-02-20 | 成都流体动力创新中心 | Target autonomous detection tracking method and system |
| CN118071826A (en) * | 2024-01-29 | 2024-05-24 | 广东永浩信息技术有限公司 | Target positioning method and system based on video monitoring |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114897932B (en) | 2024-07-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114897932B (en) | Infrared target tracking realization method based on feature and gray level fusion | |
| CN108154118B (en) | A kind of target detection system and method based on adaptive combined filter and multistage detection | |
| Zhu et al. | Beyond local search: Tracking objects everywhere with instance-specific proposals | |
| CN111354017A (en) | Target tracking method based on twin neural network and parallel attention module | |
| CN106338733B (en) | Forward-Looking Sonar method for tracking target based on frogeye visual characteristic | |
| CN104574445A (en) | Target tracking method and device | |
| CN108564598B (en) | Improved online Boosting target tracking method | |
| CN111080674B (en) | Multi-target ISAR key point extraction method based on Gaussian mixture model | |
| CN107609571B (en) | Adaptive target tracking method based on LARK features | |
| CN112085765B (en) | Video target tracking method combining particle filtering and metric learning | |
| CN111429485B (en) | Cross-modal filtering tracking method based on self-adaptive regularization and high-reliability updating | |
| CN111640138A (en) | Target tracking method, device, equipment and storage medium | |
| CN111539987B (en) | Occlusion detection system and method based on discrimination model | |
| Hou et al. | Robust human tracking based on DPM constrained multiple-kernel from a moving camera | |
| CN113763274A (en) | Multi-source image matching method combining local phase sharpness orientation description | |
| CN110738098A (en) | target identification positioning and locking tracking method | |
| CN116342653A (en) | Target tracking method, system, equipment and medium based on correlation filter | |
| Raj et al. | Deep manifold clustering based optimal pseudo pose representation (dmc-oppr) for unsupervised person re-identification | |
| CN114842506A (en) | Human body posture estimation method and system | |
| CN114429593A (en) | Infrared small target detection method based on rapid guided filtering and application thereof | |
| CN114022510A (en) | Target long-time tracking method based on content retrieval | |
| Ma et al. | Affine object tracking using kernel-based region covariance descriptors | |
| CN108346158B (en) | Multi-target tracking method and system based on main block data association | |
| CN112580679A (en) | Infrared target tracking method, computer device and computer readable storage medium | |
| CN110660079A (en) | Single target tracking method based on space-time context |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |