CN114241008B - A Long-term Region Tracking Method Adapting to Scene and Object Variations - Google Patents

Abstract

The invention discloses a long-term region tracking method adaptive to scene and target changes, which comprises the following steps: the tracking position is quickly and accurately predicted in a mode of combining rough and fine tracking and center position correction; multi-thread research and judgment, wherein the tracking confidence coefficient is corrected by combining the apparent similarity of the target, the multi-frame interval constraint is considered, the tracking result is comprehensively researched and judged, the tracking false alarm is eliminated, and the reliability of the tracking result is improved; and (4) flexible tracking, namely updating a tracking area in a self-adaptive manner when the tracking is successful according to a tracking research and judgment result, and re-capturing the target when the tracking is failed. And when the tracking result is judged to be successful, adapting to the weak change between frames by adopting a frame-by-frame weighting updating mode at short-time small scene intervals, and adapting to the apparent significant change of a large-interval target through an active restarting module at long-time intervals. The method has the advantages of fast tracking response, high precision and reliable confidence coefficient.

Description

A Long-term Region Tracking Method Adapting to Scene and Object Variations

technical field

The invention belongs to the technical field of digital image processing, and focuses on solving the problem of long-term tracking of a specific target under the condition of imaging equipment moving. The algorithm can actively adapt to the target scale and appearance changes and has a small amount of calculation, which can be well deployed on various low-performance hardware platforms.

Background technique

The main purpose of target tracking is to imitate the motion perception function of the physiological visual system. After obtaining the initial state of the target of interest to be tracked, analyze the subsequent image sequence captured by the imaging device and calculate the position of the target in each frame of the image. Obtain the trajectory of the target within a specific space-time range, and provide technical support for advanced processing and applications such as target recognition, target behavior analysis, and 3D reconstruction. In recent years, target tracking technology has been developed continuously and rapidly, and has been widely used in national defense and military fields, such as battlefield reconnaissance and surveillance, border patrol, key target positioning, calibration and strike, electronic warfare and damage assessment. This technology has also been widely used in civil fields such as security monitoring, traffic operation, aerial photography, and disaster monitoring.

A long-term area tracking method that adapts to scene and target changes is mainly aimed at UAV monitoring systems or high-altitude observation systems. The monitoring target is a specific rigid body target, and the monitoring distance can reach several kilometers or even more than ten kilometers. Infrared monitoring equipment is generally installed. In order to lock the target, various posture changes may occur on the equipped equipment or the gimbal. In this application mode, the difficulty of target tracking is reflected in the following aspects: (1) From the perspective of the target's own imaging, the apparent change is large. When the shooting distance is long, limited by the limitations of infrared imaging itself, the target occupies fewer pixels, lacks sufficient texture information, and has weaker apparent features. In the process of continuous tracking of the locked target, there will inevitably be changes in its own apparent imaging, including changes in the target scale and appearance details caused by changes in the distance between the imaging device and the target, as well as changes in the target’s appearance due to changes in attitude or trajectory. Variety. (2) From the perspective of the external environment, similar targets may interfere with each other and block each other during operation during long-distance shooting. In addition, due to the overhead shooting angle, the target is also easily submerged in the surrounding background; finally, the movement of the imaging platform may also cause Motion blur, even causing objects to move out of view. (3) In terms of the limitations of the tracking algorithm itself, in the current real-time target tracking algorithm, whether it is a generative model or a discriminative model, their basis is in the same video, the size of the same object in the two frames before and after and the spatial position will not change dramatically, and the position of the target is determined in the next frame based on a given target template or a trained classifier. But the essential difference between tracking and other vision tasks is the ability to adapt to gradual changes in moving objects. So for visual tracking, online updates play a crucial role. However, online updating is also a double-edged sword in balancing dynamic information description and unexpected noise introduction, accumulating errors for a long time, collecting inappropriate samples when the target disappears or overfitting the available data, which easily degrades the performance of the tracker, lead to tracking drift.

In the actual tracking process, although many tracking algorithms have made significant progress in building appearance models and robust tracking, target tracking is still a very complicated problem in the face of many practical difficulties. Compared with the popular deep learning algorithms in recent years, in real-time processing systems, the correlation filter tracking algorithm has become a current research hotspot due to its excellent performance in target tracking accuracy, robustness to target apparent changes, and speed. This method learns a filter template, performs convolution operation on the image of the next frame and the filter template, predicts the target position according to the output response, and uses FFT to convert the convolution operation of the image into a point product in the frequency domain. , which greatly reduces the computational complexity, and can achieve a processing speed of hundreds of frames per second when using the original pixel value to describe the target. In order to meet the real-time processing requirements, and to be able to judge the prediction results after completing the basic position prediction, and to retrieve the target after the target is lost, the present invention proposes an adaptive update long-term target tracking under the framework of correlation filtering method, focusing on solving the following technical problems.

(1) Accurate predicted position: In view of the fact that the target is weak and easily submerged in the surrounding background in long-distance infrared imaging, if the target location is only limited to the target's own appearance, the current method of determining the target position at one time based on the maximum response value of confidence method can easily cause tracking position offset. The present invention needs to be able to make full use of the target and surrounding background information, accurately determine the final position of the target, and ensure the accuracy of the tracking result.

(2) Tracking results are credible: For the target drift problem caused by various external disturbances, the main reason is that the tracking algorithm cannot reliably determine the tracking results, which leads to problems in the model update mechanism. Currently, when these algorithms are updated online, Generally, the target historical tracking results are used as training samples, which implicitly assumes that the target tracking results are correct or the error is small, but in fact it is difficult for us to guarantee that the tracking results will not be wrong. A robust tracker should be able to verify the tracking results through various external physical constraints to ensure the reliability of the tracking results.

(3) The update content is correct: In view of the practical problem of target appearance changes, the existing trackers adopt the frame-by-frame update method to adapt to the dynamic changes of the target object's pose or scale after the first frame is determined to track the target object, but this This update method of fixed tracking target object is suitable for the scene where the apparent change of the target in the field of view is slow and the motion trajectory is continuous, and it cannot completely solve the problem of drastic scale change. When the tracking object exceeds the field of view and only the target part remains in the field of view, this update method will cause tracking drift due to edge filling, and the amount of calculation will also increase significantly. The present invention requires the tracker to continuously adapt to update the target template during the whole tracking process, to adapt to the drastic scale and appearance changes of the target during the tracking process, and to ensure the tracking speed and precision.

Contents of the invention

In order to realize wide-area and long-term target tracking under continuous viewing angles, the present invention proposes a long-term area tracking method that adapts to scene and target changes.

In order to achieve the above object, the present invention adopts the following technical solutions:

A long-term area tracking method adapted to scene and target changes, comprising the following steps:

Based on the three-level combination of coarse and fine tracking and center position correction, the target position is tracked on the images collected by the UAV monitoring system or the high-altitude observation system;

Calculate the target position tracking confidence, and correct the target position tracking confidence based on the apparent similarity of the target, combined with the multi-frame interval constraints, comprehensively judge the corrected tracking results;

According to the comprehensive research and judgment results, the tracking model is adaptively updated when the tracking is successful, and the target is recaptured when the tracking fails.

As described above, a long-term area tracking method adapting to scene and target changes is characterized in that the images collected by the UAV monitoring system or the high-altitude observation system are processed based on the three-level combination of coarse and fine tracking and center position correction. Object location tracking includes:

According to the target position P _t-1 in the previous frame, select an image block with the same size as the rough tracking filter template at the corresponding position in this frame as the rough tracking search area to perform rough tracking search, and obtain the preliminary position estimate P _c of the target. If this tracking response If the value corresponding to the position in the graph is higher than the threshold thr _ρc , it is determined that the position of the center point of the fine tracking search is this point, otherwise P _t-1 is still used as the center point position of the fine tracking search of this frame;

Select an image block with the same size as the fine tracking filter template in the previous frame at the center point of the fine tracking search determined above as the fine tracking search area to perform fine tracking search to obtain the fine tracking position P _f , if the tracking response map corresponds to the position If the value is higher than the threshold thr _ρc , accept the fine tracking search result and go to the next step; otherwise, the tracking of this frame fails and the lost recapture is performed;

Select an image area with the same size as the image apparent template T _a on and around the fine tracking position _Pf , and perform the mean absolute difference algorithm MAD with the image apparent template T _a respectively, and take the position point with the largest similarity as the final tracking position _Pt .

A long-term area tracking method that adapts to scene and target changes as described above, is characterized in that calculating the target position tracking confidence, and correcting the target position tracking confidence based on the target apparent similarity includes:

For the tracking response graph, calculate the peak side lobe ratio PSR as shown in formula (1), denoted as psr _cur , which reflects the intensity of the main peak relative to the side lobe. In the formula, F _max is the response value of the peak, and μ _sub and σ _sub are the side lobe mean and standard deviation;

Calculate the ratio of the peak side lobe ratio psr _cur of the current frame to the average value psr _avg of the PSR tracking response of the recent consecutive successful M frames in history, reflecting the degree of shock of the PSR, and determining the target position tracking confidence ρ _c of the current frame;

Calculate the _MAD value of the image apparent similarity between the image block and the image apparent template T a in the 5*5 area centered on the final tracking position P _t , and compare the MAD value of the current frame with the MAD mean value of the latest consecutive successful M frame tracking results in history Compare to obtain the normalized image apparent similarity ρ _a ;

The weighted average of target position tracking confidence ρ _c and image apparent similarity ρ _a is the corrected current tracking confidence ρ.

As described above, a long-term area tracking method adapting to scene and target changes is characterized in that the multi-frame interval constraint threshold is jointly determined by the inter-frame displacement change of the most recent consecutive successful N frames in the historical trajectory plus a constant value c, When the tracking confidence ρ is greater than the threshold thr _a and the interval displacement between the current frame and the previous N frames is smaller than the multi-frame interval constraint threshold thr _m , it is determined that the tracking result is correct, otherwise the tracking fails.

A long-term area tracking method adapting to scene and target changes as described above, characterized in that, when the tracking is successful, adaptively updating the tracking area includes:

When the number of consecutive successful tracking is less than N frames, the rough tracking filter template and the fine tracking filter template are weighted and updated frame by frame to adapt to the weak difference between frames;

When the number of consecutive successful tracking is equal to N frames, use the tracking results of the current frame to re-initialize the rough tracking filter template, fine tracking filter template and image appearance template T _a to adapt to the apparent change of the target;

When reinitializing the rough tracking filter module and the fine tracking filter template, the determination method of the size of the tracking area and the search area takes into account both the target size and the calculation speed limit, specifically:

According to the object distance d ₀ , focal length f ₀ , shooting angle θ ₀ at the initial frame 0 and the object distance d _t , focal length f _t and shooting angle θ _t at the current frame t, estimate the target expansion coefficient, and determine the size of the target in the current frame. The rough estimation method of the expansion coefficient γ is:

Considering the calculation speed limit, when the short side of the target imaging is less than or equal to 54 pixels, a rectangular frame with a size of 10 pixels outside the length of the short side is selected as the tracking area centered on the tracking point. When the short side of the target image is larger than 54 pixels, select a 64*64 area centered on the tracking point as the tracking area, expand the fine tracking by 1 times as the search area, and expand the rough tracking by 2 times as the search area, respectively create Coarse and fine tracking filter template.

As described above, a long-term area tracking method adapting to scene and target changes is characterized in that when the tracking fails, in order to achieve long-term tracking, it is necessary to perform lost recapture, mainly including:

Select the frame with the highest confidence from the tracking buffer in turn according to the size of the tracking area at the current moment, and expand the search area twice to prepare a filter template to search again in the current frame;

The tracking buffer stores the original frame image, target position and confidence degree that were recently determined to be successful;

After the search is successful, re-initialize the rough tracking filter template, fine tracking filter template and image appearance template in the current frame;

If the search fails, continue to repeat the above search method for the next frame. When the target position cannot be reacquired in consecutive N frames, the target is declared lost, and the target tracking program is terminated.

A long-term area tracking method adapted to scene and target changes designed by the present invention has the following advantages:

(1) Fast response: The correlation filtering method is used to predict the target position, and the normal tracking processing time of the algorithm frame by frame on the embedded platform is less than 10ms. When the target is lost and needs to be recaptured, the maximum processing time is no more than 20ms.

(2) Tracking accuracy: Compared with the existing correlation filter tracking algorithm, this method uses the combination of rough and fine tracking, and can use the tracking response results and historical tracking information to determine the timing of updating, and is compatible with the existing CSK, KCF, and ECO Compared with the -HC method, it has the characteristics of high accuracy of tracking results.

(3) Judgment pair: This method combines multiple clues and combines space-time information to conduct comprehensive research and judgment on the tracking results. Based on the predicted location of the target and the confidence judgment of the results, the output results of the method are more reliable than previous methods.

Description of drawings

The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

Fig. 1 is the overall framework of the long-term area tracking method adapting to scene and target changes in the present invention;

Fig. 2 is a flow chart of frame-by-frame tracking processing;

Figure 3 is a multi-frame buffer management strategy;

Fig. 4 is a schematic diagram of tracking target lost and recaptured.

Detailed ways

As mentioned above, the present invention proposes a long-term area tracking method that adapts to scene and target changes. The specific implementation manners of the present invention will be clearly and completely described below in conjunction with the accompanying drawings.

This embodiment discloses a long-term area tracking method that adapts to scene and target changes, and the overall framework is shown in Figure 1:

When the target to be tracked is obtained in the initial frame, rough and fine tracking filter templates and image appearance template T _a are prepared. When a subsequent new frame of image arrives, the candidate search area is selected to predict the target position, and then a combination of multiple The clue calculates the confidence degree of the tracking result and conducts comprehensive research and judgment on the target tracking confidence degree to eliminate tracking false alarms; finally, the tracking model is updated according to the research and judgment results.

In this embodiment, the specific processing flow of each frame is shown in Figure 2, including:

Step 1: Target location prediction

It mainly includes two parts: coarse and fine tracking search and center position correction.

(a) First, rough tracking search is performed to obtain the initial target position P _c , if the maximum response value of this position is greater than the acceptable threshold thr _ρc , fine tracking is performed at this position, otherwise, it is directly performed at the position P _t-1 of the last frame fine tracking. Both coarse and fine tracking adopt the tracking principle based on correlation filtering. The specific method is to select an image block with the same size as the tracking filter template (including the rough tracking filter template and the fine tracking filter template) at the corresponding position of the frame according to the target position in the previous frame as Search the area, extract features and add a cosine window to eliminate the influence of the boundary. Assume that the search area feature is z, as shown in formula 1, perform operations with the target feature model x of the previous frame to obtain the kernel matrix k; then as shown in formula 2, use the previous The filter parameter a of the frame calculation is multiplied by the kernel matrix k in the frequency domain, and the calculation result is transferred to the time domain through the inverse Fourier transform F ^-1 , and the response graph y after the correlation filtering operation is obtained; finally, through Search for the maximum value Fmax and the corresponding coordinates (px, py) in the response graph y to calculate the position where the target appears with the highest probability in the current frame.

y=F ^-1 (F(a)·F(k)) (2)

In this process, both coarse and fine tracking adopt the correlation filtering method. The difference between the two is that after the tracking area is selected, the size of the search area is set differently. The specific size is set to the initial value according to the actual situation. Downsampling operation for nearest neighbor interpolation.

(b) When the maximum response value corresponding to the fine tracking position P _f is greater than the acceptable threshold thr _ρc , use the image apparent similarity to correct the center position at this position. The specific method is to select four candidate points P1, P2, P3 and P4 at the current frame position point P _f and its surroundings up, down, left, and right. Select an image area with the same size as the image apparent template T _a centered on these 5 points, in this embodiment, select a 5*5 size area to form new target apparent images S, S1, S2, S3 and S4, Then, the grayscale-based image matching algorithm MAD (mean absolute difference) is carried out with the image appearance template T _a respectively, and the position point with the largest similarity is taken as the final tracking point instead of P _f as the final tracking result P _t of the tracker. The calculation of MAD is shown in formula (3):

Wherein: M=N=5, d represents the average absolute difference of pixel values between the image block S and the image apparent template T.

Step 2: Confidence Correction

It includes two parts: the spatial-temporal analysis of the tracking filter's own response and the calculation of the apparent similarity of the tracking area image.

Spatio-temporal analysis of the tracker's own response First, calculate the Peak to Sidelobe Ratio (PSR) from the tracking response map solved by the correlation filter tracking algorithm, denoted as psr _cur . PSR can be used to represent the correlation peak intensity. The correlation output g is divided into two parts: the peak value and the side lobe, the peak value is the maximum value, and the side lobe is selected as a 11*11 area centered on the peak value. It mainly reflects the ratio of the main peak to the side lobe. When the mean value of the side lobe is large or the distribution is uneven, the value of the PSR will be reduced. The specific calculation is shown in formula (4):

F _max is the peak response value, μ _sub and σ _sub are the mean and standard deviation of the side lobes.

The changes in the tracking response graph will not change drastically during normal tracking, but only when the target is occluded or lost, so the historical tracking information can also provide a reference for the calculation of the current tracking confidence. In this method, the mean value of the historical effective frame tracking results is used to normalize the peak side lobe ratio, as shown in formula (5), to represent the target position tracking confidence ρ _c , this method uses the current PSR value psr _cur and the historical effective The ratio of the frame PSR average value psr _avg is used as the final tracking filter confidence index. The advantage is that it can adapt to each test scene and effectively overcome the disadvantages of poor generalization of the fixed threshold.

Calculation method of tracking confidence based on image apparent similarity: normalize the previously calculated MAD. The specific method is to calculate the MAD average value of the latest consecutive successful M frame tracking results in history. In this embodiment, the first three are specifically selected. The MAD calculation average value d _avg of the frame (M=3) tracking results is used as the apparent reference similarity value of the current tracking image, and then the tracking point MAD value d _cur calculated in the current frame is compared with the reference similarity value d _avg , such as Formula (6) shows.

The weighted average of target position tracking confidence ρ _c and image apparent similarity ρ _a is the corrected current tracking confidence ρ, and the specific calculation is shown in formula (7).

ρ＝0.5*ρ _c +0.5*ρ _a (7)

Step 3: Research and judge the tracking results

Combining the tracking confidence and the historical track information of the target tracking to judge the tracking results.

The multi-frame interval constraint specifically refers to the movement displacement constraint of the tracking target between multiple frames, and calculates the predicted displacement deviation threshold between the current frame and the historical frame according to the inter-frame displacement changes of the historical continuous tracking frame N. In order to realize this strategy, it is necessary to store the tracking data of the most recent consecutive frames (such as N frames, N=5 in the present invention). Three cache queues are used to store data. The specific storage and update strategy is shown in Figure 3, and the specific description is as follows:

Cache 1: It is used to calculate the deviation distance of the currently processed N frames, store the tracking track information of the latest N frames, and the update strategy is that the current frame is processed and stored in the cache, and the earliest frame processing result is deleted;

Cache 2: It is used to save the N frame data of the previous stage of the current stage, and is used to update "cache 3". The update strategy is to initialize "cache 2" when "cache 1" is full for the first time, and then "cache 1 "Every update, the earliest value in "cache 1" is pushed into "cache 2", and the earliest value in cache 2 is removed at the same time.

Cache 3: It is used to calculate the latest historical prediction, store the last consecutive successful N frames in the history, and the update strategy is to copy the consecutive successful N frames from "Cache 2", and calculate the average frame interval displacement once, which can be used multiple times after it Frame prediction; it is cleared after calculating the average frame interval displacement.

The final tracking result judgment strategy based on the tracking confidence and multi-frame interval displacement constraints is: only when the tracking confidence ρ is greater than the acceptable threshold thr _a (value 0.15) and the current frame is displaced from the previous N frames (if the current frame is When frame t (that is, the displacement between frame t and frame tN) is less than the acceptable multi-frame interval constraint threshold thr _m , it is determined that the tracking result is correct, otherwise the tracking fails.

Step 4: Track Model Management

When the tracking result determines that the frame is successfully tracked, the tracking model management is performed. The present invention adopts a frame-by-frame weighted update and stage restart preparation method for filter template update to adapt to target scale and appearance changes. The specific method is:

First count the number of successful continuous tracking, and judge whether to actively restart according to the number of successful continuous tracking.

If the number of consecutive successful tracking is less than N, in this embodiment, when N=5, in order to adapt to the slight difference between frames in short-term small scenes, a strategy of weighted update template frame by frame is adopted. However, the results of each frame are used for updating, or updating every other frame is risky, especially when the target is occluded, or the tracker is not following well, then updating the model will only make the tracker worse. It becomes impossible to identify the target. The present invention only updates the tracking model when it is judged that the tracking is successful in step 3 and the tracking confidence ρ is greater than the threshold thr _l (thr _a <thr _l <1, thr _l takes a value of 0.3), so as to avoid the target model being Pollution, reducing model drift and update times, while increasing speed. The calculation method of updating the tracking model frame by frame is shown in formulas (8), (9).

F(x) _t ＝(1-r)F(x) _t-1 +rF(x) _t (8)

F(a) _t = (1-r)F(a) _t-1 +rF(a) _t (9)

Among them, F( ) represents the Fourier transform operation, x represents the extracted target and background features, a represents the filter parameters, and r represents the update rate. The larger the r value, the greater the weight of the current frame, but the larger r value is very It is easy to pollute the template with the current frame. Therefore, it is necessary to adjust the update rate adaptively according to the level of tracking confidence. When the pre-confidence ρ is low (thr _l <ρ<thr _h , thr _h =0.7), the update rate is 0.035, and when the confidence ρ is high (thr _h ≤ρ≤1), the update rate is 0.085.

If the number of consecutive successful tracking is equal to N, when N=5 in this embodiment, in order to adapt to the huge change in the appearance of the long-term large scene, the present invention retrains the filter in the current frame, initializes the tracking parameters, the tracking filter template and the target table concept template. When preparing the tracking filter template, the determination of the tracking area and the search area takes into account both the target size and the calculation speed limit. According to the object distance d ₀ , focal length f ₀ , shooting angle θ ₀ at the initial frame 0 and the object distance d _t , focal length f _t and shooting angle θ _t at the current frame t, estimate the target expansion coefficient, and determine the size of the target in the current frame. The rough estimation method of the expansion coefficient γ is:

Secondly, consider the calculation speed limit. When the target image is small (short side is less than or equal to 54 pixels), select a rectangular frame with a short side length of 10 pixels as the center of the tracking point as the tracking area. When the target image becomes larger ( The short side is greater than 54 pixels), select a 64*64 fixed-size area centered on the tracking point as the tracking area. Coarse and fine tracking are expanded outward by 1 time and 2 times respectively as the search area to create a rough and fine tracking filter template.

The target appearance template selects a 5*5 image area with the tracking point as the center.

Step 5: Lost Recover

When the tracking result is judged to be the tracking failure of this frame, it is necessary to carry out the lost and recaptured work. First, judge whether to stop tracking, that is, judge whether continuous multi-frame tracking has failed, and then end the tracking process if continuous N=5 frames have failed. Otherwise, use the historical successful tracking information to restart the tracking in the failed state to recapture the lost target. specific method:

Select the frame with the highest confidence from the tracking buffer in turn according to the size of the tracking area at the current moment, and expand the search area twice to prepare a tracking filter template to search again in the current frame;

Calculate the confidence degree ρ and multi-frame interval displacement for the search results, and judge the tracking results according to the strategy in step 3. If the search is successful, re-initialize the rough and fine tracking filter template in the current frame according to the setting method of the tracking area and search area in step 4 with the image appearance template;

If the search fails, transfer to the next frame for processing.

The long-term target tracking method for template adaptive update provided by the present invention has been described in detail above, but obviously the specific implementation form of the present invention is not limited thereto. For those skilled in the art, various obvious changes made to it without departing from the scope of the claims of the present invention are within the protection scope of the present invention.

Claims (1)

1. A long-time region tracking method adapting to scene and target changes is characterized by comprising the following steps:

tracking the target position of an image acquired by an unmanned aerial vehicle monitoring system or a high-altitude observation system based on a three-stage combination mode of rough and fine tracking and center position correction;

calculating a tracking confidence coefficient of the target position, correcting the tracking confidence coefficient of the target position based on the apparent similarity of the target, and comprehensively studying and judging the corrected tracking result by combining multi-frame interval constraint;

according to the comprehensive judgment result, the tracking model is updated in a self-adaptive mode when the tracking is successful, and the target is captured again when the tracking is failed;

the self-adaptive updating tracking model when the tracking is successful comprises the following steps:

when the continuous successful tracking times are less than N frames, the coarse tracking filter template and the fine tracking filter template are weighted and updated frame by frame to adapt to weak difference between frames;

when the continuous successful tracking times are equal to N frames, the coarse tracking filter template, the fine tracking filter template and the image apparent template T are reinitialized by using the tracking result of the current frame _a To accommodate target apparent significant changes;

the method for determining the sizes of the tracking area and the search area when the coarse tracking filter template and the fine tracking filter template are reinitialized simultaneously considers the target size and the calculation speed limit, and specifically comprises the following steps:

according to the initial frame 0 time object distance d ₀ Focal length f ₀ And a shooting angle theta ₀ Object distance d from current frame t _t Focal length f _t Angle of incidence theta _t Estimating an expansion coefficient of a target, and determining the size of the target in the current frame scale, wherein the rough estimation method of the expansion coefficient gamma comprises the following steps:

considering the calculation speed limit, when the short side of the target imaging is smaller than or equal to 54 pixels, selecting a rectangular frame with the length of the short side expanded by 10 pixels as a tracking area by taking a tracking point as the center, when the short side of the target imaging is larger than 54 pixels, selecting a 64 × 64 area with the tracking point as the center as the tracking area, expanding the fine tracking outwards by 1 time as a search area, expanding the coarse tracking outwards by 2 times as the search area, and respectively creating a fine tracking filter template and a coarse tracking filter template;

when the tracking fails, in order to realize the long-term tracking, lost recapture is needed, which mainly comprises:

sequentially selecting the frame with the highest confidence from the tracking cache according to the size of the tracking area at the current moment, expanding the frame by two times to be used as a search area for preparing a filter template, and searching again in the current frame;

tracking the original frame image, the target position and the confidence coefficient which are recently judged to be successful and stored in the cache;

after the search is successful, the coarse tracking filter template, the fine tracking filter template and the image appearance template are initialized again in the current frame;

if the searching fails, the method is continuously repeated for the next frame, and when the target position cannot be obtained again in the continuous N frames, the target loss is declared, and the target tracking program is terminated;

the tracking of the target position of the image acquired by the unmanned aerial vehicle monitoring system or the high-altitude observation system based on the mode of three-stage combination of rough and fine tracking and center position correction comprises the following steps:

according to the target position P of the previous frame _t-1 Selecting and coarse tracking filtering at the corresponding position of the frameImage blocks with the same template size are used as coarse tracking search areas for coarse tracking search to obtain a primary position estimate P of the target _c If the value of the current tracking response map corresponding to the position is higher than the threshold value thr _ρc Determining the position of the fine tracking search center point as the point, otherwise, still adopting P _t-1 The position of the central point of the fine tracking search of the current frame is taken as the position of the central point of the fine tracking search of the current frame;

selecting an image block with the same size as the template of the last frame fine tracking filter at the determined position of the fine tracking search central point as a fine tracking search area to perform fine tracking search, and obtaining a fine tracking position P _f If the value of the current tracking response map corresponding to the position is higher than the threshold value thr _ρc Receiving the fine tracking search result, and turning to the next step, otherwise, losing and recapturing the frame if the tracking of the frame fails;

at the fine tracking position P _f Top and surrounding selection and image appearance template T _a Image regions of the same size as the image apparent template T _a Performing average absolute difference algorithm MAD, and taking the position point with the maximum similarity as the final tracking position P _t ；

Calculating a target position tracking confidence coefficient, and correcting the target position tracking confidence coefficient based on the target apparent similarity comprises the following steps:

the peak-to-side lobe ratio PSR is calculated as shown in equation (1) for the tracking response plot and is designated as PSR _cur Reflecting the intensity of the main peak relative to the side lobe, in the formula F _max Is the response value of the peak, μ _sub And σ _sub Is the mean and standard deviation of the sidelobes;

calculating the peak sidelobe ratio psr of the current frame _cur PSR mean PSR of M frame tracking response with latest continuous success _avg The ratio of (a) to (b) reflects the oscillation degree of the PSR, and determines the target position tracking confidence coefficient rho of the current frame _c ；

Calculated to finally track position P _t Image block and image apparent template T in 5*5 area as center _a Comparing the MAD value of the current frame with the MAD average value of the latest continuous successful M frame tracking result to obtain the normalized image apparent similarity rho _a ；

Target position tracking confidence ρ _c Apparent similarity to image ρ _a Weighted average is carried out to obtain a corrected current tracking confidence coefficient rho;

the multi-frame interval constraint threshold is determined by the interframe displacement change of the latest continuous successful N frames in the historical track and a constant value c, and when the tracking confidence coefficient rho is greater than the threshold thr _a And the interval displacement between the current frame and the previous N frames is less than a multi-frame interval constraint threshold thr _m And judging that the tracking result is correct, otherwise, failing to track.

Images