CN109978908A - A kind of quick method for tracking and positioning of single goal adapting to large scale deformation - Google Patents

Abstract

The present invention relates to it is a kind of adapt to large scale deformation but the quick method for tracking and positioning of target, the following steps are included: template matching, determine the coordinate and size of target template image, cyclic convolution is done with current frame image and target template image and acquires target response matrix with Ridge Regression Modeling Method, as rough coordinates of targets at response matrix maximum value；Motion detection, before and after frames images match obtain visual angle and shake parameter, and acquire difference diagram；It counts difference diagram and obtains motion detection output coordinate；The rough coordinates of targets is corrected with the motion detection output coordinate, obtains precision target coordinate；Template renewal updates target template with the precision target coordinate, updates ridge regression parameter；The present invention can rapidly and accurately track set goal position, and suitable for the scene that target shape has large scale to change, there is good robustness, can be applied in the scenes such as shooting auto-focusing, monitor video target lock-on.

Description

A fast tracking and localization method for single target adapting to large-scale deformation

technical field

The invention belongs to the technical field of video signal processing, and in particular relates to a method for fast tracking and positioning of targets that is adaptable to large-scale deformation.

Background technique

Object tracking is an important research direction in computer vision. It has a wide range of applications in the fields of human-computer interaction, machine recognition, and artificial intelligence.

In the field of target tracking, the change of target shape has always been a difficult problem. Most of the existing methods establish a standard template in the first frame, and perform template matching in the latter frame to obtain the target position. However, once the target is deformed during the tracking process (such as the change of the shooting angle, the flip of the human body, etc.), the template matching is likely to fail.

The change of shape necessarily means that the pixel where the target is located has changed drastically, so it is very suitable for motion detection to detect the target.

SUMMARY OF THE INVENTION

In order to solve the technical problem of template matching failure after the target in the prior art is deformed, the present invention provides the following technical solutions:

A single-target fast tracking and positioning method adapting to large-scale deformation, including the following steps:

Step 1, template matching, determine the coordinates and size of the target template image, use the current frame image and the target template image to do circular convolution and use the ridge regression method to obtain the target response matrix, and the maximum value of the response matrix is used as the rough target coordinate;

Step 2, motion detection,

Step 2.1, match the front and rear frame images, obtain the viewing angle shaking parameters, and obtain the difference map;

Step 2.2, obtain motion detection output coordinates from statistical difference map;

Step 2.3, correcting the rough target coordinates with the motion detection output coordinates to obtain precise target coordinates;

Step 3, template update, update the target template with the precise target coordinates, and update the ridge regression parameters.

As a further description of the present invention, the circular convolution matrix of the current frame image and the target template image in the step 1 is,

Let the target template image be z, the current frame image be img, where ||img|| is the norm of img, ||z|| is the norm of z, FFT and IFFT represent the two-dimensional fast Fourier of the image respectively Leaf transformation and inverse transformation, is the conjugate of FFT(z), and σ is the standard deviation of the Gaussian kernel method;

The target response matrix is,

R=IFFT(FFT(K)·FFT(γ))

In the above formula, γ is the ridge regression parameter matrix, R is the target response matrix, and the position of the highest peak is the rough target coordinate, which is expressed as [x _t , y _t ].

As a further description of the present invention, the method for predicting the shaking parameters in the step 2.1 is:

In the above formula, θ=[α, β, ε] is the shaking parameter, α is the parameter of shaking up and down, β is the parameter of shaking left and right, ε is the zoom parameter of the lens, Ic is the current frame image, Ip is the previous frame image, Z(Ip, ε) is the image after scaling the current frame with the parameter ε;

The difference map of the front and rear frame images is,

In the above formula, D is the difference map.

As a further description of the present invention, the motion detection output coordinates in the step 2.2 are,

Let the target size be [size _x , size _y ], where x _md is the x-coordinate position of the moving target, y _md is the y-coordinate position of the moving target, and D(i, j) is the difference map at coordinates [i, j ] at the value;

In the above formula, D _sum is the sum of the weights of the entire difference map window.

As a further description of the present invention, the precise target coordinates in the step 2.3 are,

In the above formula, ρ is the weight coefficient, and [x _final , y _final ] is the precise template coordinate.

As a further description of the present invention, the updated target template in the step 3 is,

In the above formula is the updated target template, δ is the weight coefficient, and the updated ridge regression parameters are,

γ=(K+λI) ^-1 S

In the above formula, γ is the updated ridge regression parameter, I is the unit matrix of the same size as K, I cooperates with the regularization coefficient λ to play a regularizing role, and S is the standard response matrix.

Compared with the prior art, the beneficial effects obtained by the present invention are:

The invention eliminates the shaking of the viewing angle by matching the front and rear frames, detects the moving target by making the frame difference, and weights the results of the two to quickly and accurately track the position of the target, and is suitable for scenes with large-scale changes in the shape of the target. It has good robustness and can be used in scenes such as shooting autofocus, surveillance video target locking, etc.

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

Description of drawings

Figure 1 is a system block diagram of the method.

Figure 2 is the target response matrix R.

FIG. 3 is a current frame effect image of the inter-frame difference method.

Fig. 4 is the effect image of the previous frame of the inter-frame difference method.

FIG. 5 is a difference diagram between the current frame and the previous frame of the inter-frame difference method.

Figure 6 is a current frame effect diagram of the inter-frame difference method when the target moves and the lens is shaken.

Figure 7 is the effect diagram of the previous frame of the frame-to-frame difference method when the target moves and the lens is shaken.

Figure 8 is a difference diagram between the current frame and the previous frame by the inter-frame difference method when the target moves and the lens is shaken.

FIG. 9 is the first effect diagram of the current frame after the shaking parameters are corrected by this method.

FIG. 10 is the first rendering of the previous frame after the shaking parameters are corrected by this method.

FIG. 11 is a difference diagram 1 between the current frame and the previous frame obtained directly by the inter-frame difference method after correcting the shaking parameters by this method.

Fig. 12 is a difference diagram between the current frame and the previous frame obtained by using the method of inter-frame difference after correcting the shaking parameters by this method, and then correcting the lens shaking parameters.

FIG. 13 is the second effect diagram of the current frame after the shaking parameters are corrected by this method.

FIG. 14 is the second effect diagram of the previous frame after correcting the shaking parameters by this method.

FIG. 15 is a second difference diagram between the current frame and the previous frame obtained directly by the inter-frame difference method after correcting the shaking parameters by this method.

Fig. 16 is the difference between the current frame and the previous frame obtained by the method of inter-frame difference after correcting the shaking parameters by this method, and then correcting the lens shaking parameters.

Figure 17 is the standard response matrix S.

Figure 18 is a first effect diagram of the location of the tracking target.

FIG. 19 is a tracking effect diagram of FIG. 18 after being processed by the KCF method.

FIG. 20 is a tracking effect diagram of FIG. 18 after being processed by the SAMF method.

Fig. 21 is a tracking effect diagram of Fig. 18 after being processed by the DSST method.

FIG. 22 is a tracking effect diagram of FIG. 18 after processing by the Staple method.

FIG. 23 is a tracking effect diagram of FIG. 18 after processing by this method.

Figure 24 is the second effect diagram of the location of the tracking target.

FIG. 25 is a tracking effect diagram of FIG. 24 after being processed by the KCF method.

FIG. 26 is a tracking effect diagram of FIG. 24 after being processed by the SAMF method.

Fig. 27 is a tracking effect diagram of Fig. 24 after being processed by the DSST method.

FIG. 28 is a tracking effect diagram of FIG. 24 after processing by the Staple method.

Fig. 29 is a tracking effect diagram of Fig. 24 after processing by this method.

Detailed ways

In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose, the specific embodiments, structural features and effects of the present invention are described in detail below with reference to the accompanying drawings and examples.

In order to solve the technical problems existing in the prior art, the technical solution provided in this embodiment is mainly divided into three parts: a template matching phase, a motion detection phase and a template updating phase. As shown in FIG. 1 , an artificial target is required in the first frame. At the position and size of the first frame, the original image is then cropped to obtain a standard image centered on the tracking target.

First, the template matching stage,

Let the target template image be z and the current frame image be img. In order to speed up the technology, the frequency domain product is used instead of the time domain convolution, then the circular convolution matrix K of the current frame image and the target template image is:

In formula (1) ||img|| is the norm of img, ||z|| is the norm of z, FFT and IFFT represent the two-dimensional fast Fourier transform and inverse transform of the image, respectively, is the conjugate of FFT(z), σ is the standard deviation of the Gaussian kernel method; then the target response matrix is,

R=IFFT(FFT(K)·FFT(γ)) (2)

In formula (2), γ is the ridge regression parameter matrix. As shown in Figure 2, R is the target response matrix. The target response matrix R is a response matrix that is consistent with the size of the image. The value of each coordinate in R represents the correlation between the modified position and the target. The greater the correlation, the more likely the target is at this coordinate point, so the target The position is the position of the maximum value in the matrix R. By traversing the R matrix, find its maximum value, record its coordinates, which are expressed as [x _t , y _t ], and the coordinates are used to assist the motion detection method in the following.

2. Motion detection stage

The inter-frame difference method is often used to detect moving objects. Its basic principle is to obtain the difference map by calculating the Euclidean distance between the pixels of two adjacent frames. The value of each point in the difference map represents the current frame and the previous frame. The degree of change, the target of the movement will appear at the point with a large degree of change. The effect is shown in Figure 3, Figure 4, and Figure 5. 3 and 4 are respectively the current frame image and the previous frame image, and FIG. 5 is the difference diagram between the two. It can be clearly seen from the figure that when the camera is stationary, the inter-frame difference method can accurately obtain the position of the moving target. But once the two frames have not only the movement of the target, but also the shaking of the lens, a lot of background pixels also change. At this time, the output difference map of the inter-frame difference method cannot accurately identify the location of the moving target, as shown in Figure 6, Figure 7, and Figure 8.

Therefore, this method first predicts the shaking parameter θ=[α, β, ε] of the current shooting angle of the video by matching the front and rear frames, and then performs the inter-frame difference method after the lens shake is corrected. The prediction method of the sloshing parameter is:

In formula (3), θ is the shaking parameter, α is the parameter of shaking up and down, β is the parameter of shaking left and right, ε is the lens zoom parameter, Ic is the current frame image, Ip is the previous frame image, Z(Ic, ε) is the image after scaling the current frame with the parameter ε;

Taking the angle of view shaking parameter θ as a reference, find the Euclidean distance between the two adjacent images after correction, and then the difference map D between the two frames before and after can be obtained,

In Equation (4), D is the difference map after the lens shake parameter is corrected, and its effect is shown in Figures 9 to 16 . 9 and 13 are images of the current frame, and FIGS. 10 and 14 are images of the previous frame. Figure 11 and Figure 15 are the difference diagrams obtained directly by the frame difference method for two frames of images. As shown in the figure, the background pixels in the image are also moving due to the shaking of the lens, so it is also difficult to lock the location of the moving object in the difference map. However, after correcting the lens shake parameters, the difference maps of the two frames before and after are shown in Figure 12 and Figure 16. Obviously, the background information has been suppressed, and the position of the moving target shows a larger difference value, and the effect is obvious.

On the difference map, take the rough target coordinate output [x _t , y _t ] of the template matching stage as the center, take a retrieval window twice the size of the template, and use the value of the difference map as the weight in this window, for each pixel The coordinates are weighted and averaged to obtain the position of the moving target. If the size of the moving target is [size _x , size _y ], the output coordinates of the motion detection are,

In formula (5), x _md is the x coordinate position of the moving target, y _md is the y coordinate position of the moving target, D(i, j) is the value of the difference map at coordinates [i, j] D _sum is the entire difference map The sum of the weights of the windows;

Finally we take the weighted average of the output of template matching and the output of motion detection,

In the above formula, ρ is the weight coefficient, and [x _final , y _final ] is the precise template coordinate.

Third, the template update stage,

After calculating the template position [x _final , y _final ] in the current frame, take this coordinate as the center and the target size [size _x , size _y ] as the scale, re-acquire the current frame target template z′, the current frame template z′ and the above Frame template z-weighted average to get new template The new template will replace the old template to participate in the tracking task of the next frame:

In formula (8), δ is the weight coefficient. After the new template is obtained, the response matrix K is recalculated according to formula (1) to update the ridge regression parameter γ,

γ=(K+λI) ^-1 S (9)

In formula (9), α is the updated ridge regression parameter, I is the unit matrix of the same size as K, I cooperates with the regularization coefficient λ to play a regularizing role, and S is the standard response matrix. where λ is a small positive real number. The value of S is consistent with the size of the image and has a very small standard deviation of the two-dimensional Gaussian kernel shape, as shown in Figure 17.

Compared with the advanced tracking methods, this method has higher robustness and can complete the tracking tasks that other methods cannot. The effect is shown in Figure 6.

To sum up, the main features of this method are: 1. Predict and eliminate lens shake by matching the front and rear frame images. Then perform frame differential detection on moving objects; 2) Combine motion detection algorithm and correlation filter tracking. Motion detection method is used to correct the results of the tracking algorithm to achieve better tracking results. 3) The algorithm can complete more complex tracking tasks when the target moves and deforms rapidly.

The above content is a further detailed description of the present invention in combination with specific preferred embodiments, and it cannot be considered that the specific implementation of the present invention is limited to these descriptions. For those of ordinary skill in the technical field of the present invention, without departing from the concept of the present invention, some simple deductions or substitutions can be made, which should be regarded as belonging to the protection scope of the present invention.

Claims (6)

1. a single target fast tracking and positioning method adapted to large-scale deformation, is characterized in that, comprises the following steps: Step 1, template matching, determine the coordinates and size of the target template image, use the current frame image and the target template image to do circular convolution and use the ridge regression method to obtain the target response matrix, and the maximum value of the response matrix is used as the rough target coordinate; Step 2, motion detection, Step 2.1, match the front and rear frame images, obtain the viewing angle shaking parameters, and obtain the difference map; Step 2.2, obtain motion detection output coordinates from statistical difference map; Step 2.3, correcting the rough target coordinates with the motion detection output coordinates to obtain precise target coordinates; Step 3, template update, update the target template with the precise target coordinates, and update the ridge regression parameters. 2. method according to claim 1, is characterized in that: the circular convolution matrix of current frame image and target template image in described step 1 is, Let the target template image be z, the current frame image be img, where ||img|| is the norm of img, ||z|| is the norm of z, FFT and IFFT represent the two-dimensional fast Fourier of the image respectively Leaf transformation and inverse transformation, is the conjugate of FFT(z), and σ is the standard deviation of the Gaussian kernel method; The target response matrix is, R=IFFT(FFT(K)·FFT(γ)) In the above formula, γ is the ridge regression parameter matrix, R is the target response matrix, and the position of the highest peak is the rough target coordinate, which is expressed as [x _t , y _t ]. 3. method according to claim 2, is characterized in that: the prediction method of shaking parameter in described step 2.1 is, In the above formula, θ=[α, β, ε] is the shaking parameter, α is the parameter of shaking up and down, β is the parameter of shaking left and right, ε is the zoom parameter of the lens, Ic is the current frame image, Ip is the previous frame image, Z(Ic, ε) is the image after scaling the current frame with the parameter ε; The difference map of the front and rear frame images is, In the above formula, D is the difference map. 4. The method according to claim 3, characterized in that: in the step 2.2, the motion detection output coordinates are, Let the target size be [size _x , size _y ], where x _md is the x-coordinate position of the moving target, y _md is the y-coordinate position of the moving target, and D(i, j) is the difference map at coordinates [i, j ] at the value; In the above formula, D _sum is the sum of the weights of the entire difference map window. 5. method according to claim 4 is characterized in that: in described step 2.3, precise target coordinates are, In the above formula, ρ is the weight coefficient, and [x _final , y _final ] is the precise template coordinate. 6. method according to claim 5, is characterized in that: the target template after updating in described step 3 is, In the above formula is the updated target template, δ is the weight coefficient, and the updated ridge regression parameters are, γ=(K+λI) ^-1 S In the above formula, γ is the updated ridge regression parameter, I is the unit matrix of the same size as K, I cooperates with the regularization coefficient λ to play a regularizing role, and S is the standard response matrix.