CN114241008B - Long-time region tracking method adaptive to scene and target change - Google Patents

Long-time region tracking method adaptive to scene and target change Download PDF

Info

Publication number
CN114241008B
CN114241008B CN202111573298.1A CN202111573298A CN114241008B CN 114241008 B CN114241008 B CN 114241008B CN 202111573298 A CN202111573298 A CN 202111573298A CN 114241008 B CN114241008 B CN 114241008B
Authority
CN
China
Prior art keywords
tracking
target
frame
fine
template
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111573298.1A
Other languages
Chinese (zh)
Other versions
CN114241008A (en
Inventor
李波
辛明
张贵伟
张超
刘偲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beihang University
Original Assignee
Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beihang University filed Critical Beihang University
Priority to CN202111573298.1A priority Critical patent/CN114241008B/en
Publication of CN114241008A publication Critical patent/CN114241008A/en
Application granted granted Critical
Publication of CN114241008B publication Critical patent/CN114241008B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/251—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/262—Analysis of motion using transform domain methods, e.g. Fourier domain methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)
  • Radar Systems Or Details Thereof (AREA)

Abstract

The invention discloses a long-term region tracking method adaptive to scene and target changes, which comprises the following steps: the tracking position is quickly and accurately predicted in a mode of combining rough and fine tracking and center position correction; multi-thread research and judgment, wherein the tracking confidence coefficient is corrected by combining the apparent similarity of the target, the multi-frame interval constraint is considered, the tracking result is comprehensively researched and judged, the tracking false alarm is eliminated, and the reliability of the tracking result is improved; and (4) flexible tracking, namely updating a tracking area in a self-adaptive manner when the tracking is successful according to a tracking research and judgment result, and re-capturing the target when the tracking is failed. And when the tracking result is judged to be successful, adapting to the weak change between frames by adopting a frame-by-frame weighting updating mode at short-time small scene intervals, and adapting to the apparent significant change of a large-interval target through an active restarting module at long-time intervals. The method has the advantages of fast tracking response, high precision and reliable confidence coefficient.

Description

Long-time region tracking method adaptive to scene and target change
Technical Field
The invention belongs to the technical field of digital image processing, and mainly solves the problem of long-time tracking of a specific target under the condition of movement of imaging equipment. The algorithm can actively adapt to target scale and apparent change, has small calculation amount, and can be well deployed on various low-performance hardware platforms.
Background
The main purpose of target tracking is to simulate the motion perception function of a physiological visual system, analyze an image sequence captured subsequently by imaging equipment after obtaining the initial state of an interested target to be tracked, calculate the position of the target in each frame of image, obtain the motion trajectory of the target in a specific space-time range, and provide technical support for advanced processing and application such as target identification, target behavior analysis, three-dimensional reconstruction and the like. In recent years, the target tracking technology has been developed continuously and rapidly, and has been widely applied to the fields of national defense and military, such as battlefield reconnaissance and monitoring, border patrol, key target positioning, shooting and striking, electronic warfare and damage assessment. The technology is widely applied to the civil fields of security monitoring, traffic operation, aerial photography, disaster monitoring and the like.
A long-time region tracking method adaptive to scene and target changes mainly aims at an unmanned aerial vehicle monitoring system or an overhead observation system, a monitored target is a specific rigid target, the monitoring distance can reach several kilometers or even more than ten kilometers, in order to meet all-weather requirements all day long, infrared monitoring equipment is generally installed, and in order to lock the target, various posture changes of carrying equipment or a holder can occur. In this application mode, the difficulty of target tracking is shown in the following aspects: (1) the apparent change is large from the imaging of the object itself. When the shooting distance is long, the infrared imaging method is limited by the limitation of infrared imaging, the target occupies few pixels, insufficient texture information is lacked, and the apparent characteristic is weak. The self apparent imaging change inevitably occurs in the process of continuously tracking the locked target, which comprises the change of the target dimension and the change of the appearance detail caused by the change of the distance between the imaging device and the target, and also comprises the apparent change of the target caused by the change of the posture or the running track. (2) In terms of external environment, similar targets may interfere with each other and shield each other in the operation process when shooting at a long distance, and in addition, due to a nodding view angle, the targets are easily submerged in the surrounding background; finally, the motion of the imaging platform may also cause motion blur and even movement of the target out of view. (3) From the limitation of the tracking algorithm, in the current real-time target tracking algorithm, no matter a generative model or a discriminant model, the basis is that the sizes and the spatial positions of the same object in two frames before and after are not changed greatly in the same video, and the position of the target is determined in the next frame based on a given target template or a trained classifier. The essential difference in tracking over other visual tasks is the ability to accommodate gradual changes in moving objects. Online updates play a crucial role for visual tracking. On-line updating, however, is also a double-edged sword in terms of balanced dynamic information description and unexpected noise introduction, accumulating errors for a long time, collecting inappropriate samples or overfitting the available data when the target disappears tends to degrade the performance of the tracker, leading to tracking drift.
In the actual tracking process, although many tracking algorithms make obvious progress in establishing an appearance model and robust tracking at present, target tracking is still a very complex problem in the presence of many practical difficulties. Compared with the deep learning algorithm with great heat in recent years, in a real-time processing system, the correlation filtering tracking algorithm is a hot spot of current research by virtue of excellent performances of target tracking accuracy, robustness to target apparent change and speed. The method carries out convolution operation on the image of the next frame and the filtering template by learning the filtering template, predicts the target position according to the output response, converts the convolution operation of the image into dot multiplication in a frequency domain by using FFT (fast Fourier transform) in actual calculation, greatly reduces the calculation complexity, and can achieve the processing speed of hundreds of frames per second when the target is described by using the original pixel value. In order to meet the real-time processing requirement, study and judge the prediction result after completing basic position prediction and retrieve the target again after the target is lost, the invention provides a self-adaptive updating long-term target tracking method under a related filtering framework, and mainly solves the following technical problems.
(1) The predicted position is accurate: aiming at the real problems that targets are weak and small and are easily submerged in surrounding backgrounds in long-distance infrared imaging, if the targets are only limited to be positioned on the appearance of the targets, the tracking position is easily shifted by the current method for determining the positions of the targets once based on the maximum response value of confidence coefficient. The method needs to be capable of fully utilizing the target and the surrounding background information, accurately determining the final position of the target and ensuring the accuracy of the tracking result.
(2) The tracking result is credible: the method aims at the problem of target drift caused by various external interferences, and the main reason is that the tracking algorithm cannot reliably judge the tracking result of the tracking algorithm, so that a model updating mechanism has problems. A robust tracker should be able to perform multi-stage verification on the tracking result through various external physical constraints, so as to ensure the reliability of the tracking result.
(3) The updated content is correct: for the realistic problem of the apparent change of the target, the conventional tracker adopts a frame-by-frame updating method to adapt to the dynamic change of the posture or the scale of the target object at the later stage after the target object is determined and tracked in the first frame, but the updating mode of the fixed tracking target object is suitable for the scene with slow apparent change and continuous motion track of the target in the field of view, and the problem of severe scale change cannot be thoroughly solved. When the tracked object exceeds the visual field and only the target is partially left in the visual field, the updating mode causes tracking drift due to edge filling, and meanwhile, the calculation amount is greatly increased. The invention needs the tracker to continuously adapt to the updating of the target template in the whole tracking process, adapts to the severe scale and the apparent change of the target in the tracking process and can ensure the tracking speed and the tracking precision.
Disclosure of Invention
In order to realize wide-area long-time target tracking under continuous visual angles, the invention provides a long-time area tracking method adaptive to scene and target changes.
In order to achieve the purpose, the invention adopts the following technical scheme:
a long-time region tracking method adapting to scene and target changes comprises the following steps:
tracking the target position of an image acquired by an unmanned aerial vehicle monitoring system or a high-altitude observation system based on a three-stage combination mode of rough and fine tracking and center position correction;
calculating a tracking confidence coefficient of the target position, correcting the tracking confidence coefficient of the target position based on the apparent similarity of the target, and comprehensively studying and judging the corrected tracking result by combining multi-frame interval constraint;
and according to the comprehensive judgment result, the tracking model is updated in a self-adaptive manner when the tracking is successful, and the target is recaptured when the tracking is failed.
The long-term area tracking method adaptive to scene and target changes is characterized in that target position tracking of images acquired by an unmanned aerial vehicle monitoring system or a high-altitude observation system based on a mode of three-stage combination of rough and fine tracking and center position correction comprises the following steps:
according to the target position P of the previous frame t-1 Selecting image blocks with the same size as the coarse tracking filter template at the corresponding positions of the frame as coarse tracking search areas to perform coarse tracking search, and obtaining a primary position estimate P of the target c If the value of the current tracking response map corresponding to the position is higher than the threshold value thr ρc Determining the position of the fine tracking search center point as the point, otherwise, still adopting P t-1 The position of the central point of the fine tracking search of the frame is used;
selecting an image block with the same size as the template of the last frame fine tracking filter at the determined position of the fine tracking search central point as a fine tracking search area to perform fine tracking search to obtain a fine tracking position P f If the value of the current tracking response map corresponding to the position is higher than the threshold value thr ρc Receiving the fine tracking search result, and turning to the next step, otherwise, losing and recapturing the frame if the tracking of the frame fails;
at the fine tracking position P f Top and surrounding selection and image appearance template T a Image regions of the same size as the image appearance template T a Performing average absolute difference algorithm MAD, and taking the position point with the maximum similarity as the final tracking position P t 。
The long-term region tracking method adapting to scene and target changes is characterized in that calculating a target position tracking confidence coefficient, and correcting the target position tracking confidence coefficient based on the target apparent similarity comprises the following steps:
the peak-to-side lobe ratio PSR is calculated as shown in equation (1) for the tracking response plot and is designated as PSR cur Reflecting the intensity of the main peak relative to the side lobe, in the formula F max Is the response value of the peak, μ sub And σ sub Is the mean and standard deviation of the side lobes;
Figure BDA0003424500750000031
calculating the peak sidelobe ratio psr of the current frame cur PSR mean PSR of M frame tracking response with latest continuous success avg The ratio of (a) to (b) reflects the oscillation degree of the PSR, and determines the target position tracking confidence coefficient rho of the current frame c ；
Figure BDA0003424500750000032
Calculated to finally track position P t Image block and image apparent template T in 5*5 area as center a Comparing the MAD value of the current frame with the MAD average value of the latest continuous successful M frame tracking result to obtain the normalized image apparent similarity rho a ；
Target position tracking confidence ρ c Apparent similarity to image ρ a And obtaining the corrected current tracking confidence coefficient rho by weighted average.
The long-term region tracking method adapting to scene and target changes is characterized in that a multi-frame interval constraint threshold is determined by the interframe displacement change of the latest continuous successful N frames in the historical track plus a constant value c, and when the tracking confidence coefficient rho is greater than a threshold thr a And the interval displacement between the current frame and the previous N frames is less than a multi-frame interval constraint threshold thr m And judging that the tracking result is correct, otherwise, failing to track.
The long-term region tracking method adapting to scene and target changes is characterized in that when tracking is successful, the self-adaptive updating of the tracking region comprises the following steps:
when the continuous successful tracking times are less than N frames, the coarse tracking filter module and the fine tracking filter module are weighted and updated frame by frame to adapt to weak difference between frames;
when the continuous successful tracking times are equal to N frames, the tracking result of the current frame is used for reinitializing the coarse tracking filter module, the fine tracking filter module and the image apparent template T a To accommodate significant changes in target appearance;
the method for determining the sizes of the tracking area and the search area when the coarse tracking filter template and the fine tracking filter template are reinitialized simultaneously considers the target size and the calculation speed limit, and specifically comprises the following steps:
according to the initial frame 0 time object distance d 0 Focal length f 0 Angle of photographing theta 0 Object distance d from current frame t t Focal length f t Angle of incidence theta t Estimating an expansion coefficient of a target, and determining the size of the target in the current frame scale, wherein the rough estimation method of the expansion coefficient gamma comprises the following steps:
Figure BDA0003424500750000041
considering the calculation speed limit, when the imaging short side of the target is equal to or less than 54 pixels, a rectangular frame with the short side extended by 10 pixels is selected as the tracking area with the tracking point as the center. And when the short side of the target imaging is larger than 54 pixels, selecting a 64 × 64 area taking the tracking point as the center as a tracking area, expanding the fine tracking outwards by 1 time as a search area, expanding the coarse tracking outwards by 2 times as a search area, and respectively creating a coarse and fine tracking filter template.
The long-term area tracking method adaptive to scene and target changes is characterized in that loss recapture is required to realize long-term tracking when tracking fails, and mainly comprises the following steps:
sequentially selecting the frame with the highest confidence from the tracking cache according to the size of the tracking area at the current moment, and expanding a double search area to prepare a filter template to search again in the current frame;
tracking the original frame image, the target position and the confidence coefficient which are recently judged to be successful and stored in the cache;
after the search is successful, the coarse tracking filter template, the fine tracking filter template and the image appearance template are initialized again in the current frame;
if the searching fails, the method is continuously repeated for the next frame, the target is declared lost after the target position cannot be obtained again by the continuous N frames, and the target tracking program is terminated.
The long-term region tracking method adaptive to scene and target changes designed by the invention has the following advantages:
(1) The response is quick: and (3) predicting the target position by adopting a related filtering method, wherein the normal tracking processing time of the algorithm on the embedded platform frame by frame is less than 10ms. The processing time does not exceed 20ms at the maximum when target loss requires reacquisition.
(2) And (3) accurate tracking: compared with the existing related filtering tracking algorithm, the method adopts a coarse and fine tracking combination mode, can utilize the tracking response result and historical tracking information to carry out update opportunity judgment, and has the characteristic of high accuracy of the tracking result compared with the existing CSK, KCF and ECO-HC methods.
(3) And (4) judging: the method combines various clues, comprehensively studies and judges the tracking result by combining the time-space information, provides the confidence judgment of the result on the basis of giving the target prediction position, and is more reliable compared with the output result of the traditional method.
Drawings
The present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
FIG. 1 is a general framework of the long-term region tracking method of the present invention adapted to scene and target changes;
FIG. 2 is a flow diagram of a frame-by-frame tracking process;
FIG. 3 is a multi-frame buffer management policy;
fig. 4 is a schematic diagram of tracking target loss recapture.
Detailed Description
The invention proposes a long-term region tracking method adaptive to scene and target changes, and the following description clearly and completely describes the specific embodiments of the invention with reference to the accompanying drawings.
The embodiment discloses a long-term region tracking method adaptive to scene and target changes, and the overall framework is as shown in fig. 1:
after the initial frame obtains the target to be tracked, a coarse and fine tracking filter is carried outTemplate and image appearance template T a Preparing, when a new subsequent frame of image arrives, selecting a candidate search area, predicting a target position, then combining various clues to calculate confidence coefficient of a tracking result and comprehensively studying and judging the target tracking confidence coefficient, and eliminating a tracking false alarm; and finally, updating the tracking model according to the research and judgment result.
In this embodiment, a specific processing flow per frame is shown in fig. 2, and includes:
step 1: target location prediction
The device mainly comprises two parts: and carrying out rough and fine tracking search and center position correction.
(a) Firstly, coarse tracking search is carried out to obtain a primary position P of a target c If the maximum response value for that location is greater than the acceptable threshold thr ρc Then fine tracking is performed at this position, otherwise the position P is directly at the previous frame t-1 And performing fine tracking. The rough and fine tracking adopts a tracking principle based on relevant filtering, specifically, according to the position of a target of a previous frame, an image block with the same size as a tracking filter template (including the rough tracking filter template and the fine tracking filter template) is selected as a search area at the corresponding position of a current frame, characteristics are extracted, a cosine window is added to eliminate boundary influence, the characteristic of the search area is assumed to be z, and the search area is operated with a target characteristic model x of the previous frame to obtain a kernel matrix k as shown in formula 1; then, as shown in formula 2, the filter parameter a calculated in the previous frame and the kernel matrix k are used to perform dot multiplication operation in the frequency domain, and the calculation result is subjected to inverse Fourier transform F -1 In the time domain, a response graph y after the relevant filtering operation is obtained; and finally, the position of the maximum probability of the target in the current frame can be calculated by searching the maximum value Fmax and the corresponding coordinates (px, py) in the response graph y.
Figure BDA0003424500750000061
y＝F -1 (F(a)·F(k)) (2)
In the process, the coarse tracking and the fine tracking all adopt related filtering methods, the difference between the two methods is that after a tracking area is selected, the size of the searching area is set to be different, the specific size sets an initial value according to the actual situation, and meanwhile, in order to ensure the speed, the coarse tracking carries out downsampling operation based on nearest neighbor interpolation on the searching area.
(b) When the position P is accurately tracked f The corresponding maximum response value is greater than the acceptable threshold thr ρc Then, the center position is corrected at this position by the apparent similarity of the images. The specific method is that the current frame position point P f And four candidate points P1, P2, P3 and P4 are selected up, down, left and right around. Respectively selecting and imaging the apparent template T by taking the 5 points as the center a The image areas with the same size are selected 5*5 in the embodiment to form new target apparent images S, S1, S2, S3 and S4, and then are respectively matched with the image apparent template T a Performing image matching algorithm MAD (mean absolute difference) based on gray scale, and taking the position point with the maximum similarity as the final tracking point to replace P f As final tracking result P of the tracker t . The MAD calculation is shown in equation (3):
Figure BDA0003424500750000062
wherein: m = N =5,d represents the average absolute difference in pixel values of the image block S and the image appearance template T.
Step 2: confidence correction
The device comprises two parts: and the tracking filter responds to the space-time analysis and the apparent similarity calculation of the tracking area image.
The tracker self-response space-time analysis firstly calculates Peak to side lobe Ratio (PSR) of a tracking response graph solved by a correlation filtering tracking algorithm, and records the PSR as PSR cur . PSR can be used to represent the correlation peak intensity. The correlation output g is divided into two parts: the peak value is the maximum value, and the side lobe is selected to be in an area with the peak value as the center 11 x 11. The ratio of the main peak to the side lobe is mainly reflected, and when the mean value of the side lobe is large or the distribution is uneven, the PSR value can be reduced. The specific calculation is shown in formula (4):
Figure BDA0003424500750000071
F max is the response value of the peak, μ sub And σ sub Is the mean and standard deviation of the side lobes.
The change of the tracking response graph does not change violently during normal tracking, and only when the target is shielded or lost, the change can change violently, so that the historical tracking information can also provide reference for calculation of the current tracking confidence. The method utilizes the mean value of the tracking result of the historical effective frame to normalize and express the peak sidelobe ratio as shown in a formula (5), and the target position tracking confidence coefficient rho is expressed by the normalized expression c The method uses the current PSR value PSR cur And the PSR mean PSR of the historical valid frame avg The ratio of the two-dimensional space-time-domain signal to the confidence coefficient index of the final tracking filter has the advantages that the method can be self-adaptive to each section of test scene, and effectively overcomes the defects of poor generalization of a fixed threshold value and the like.
Figure BDA0003424500750000072
The tracking confidence degree calculation method based on the image apparent similarity comprises the following steps: the previously calculated MAD is normalized by calculating the MAD average of the historical latest continuous successful M-frame tracking results, and in this embodiment, the MAD average d of the tracking results of the previous three frames (M = 3) is specifically calculated avg As the reference similarity value of the current tracking image appearance, then the tracking point MAD value d calculated by the current frame cur Similarity to reference value d avg The ratio is shown in equation (6).
Figure BDA0003424500750000073
Target position tracking confidence ρ c Apparent similarity with image ρ a And (4) obtaining the corrected current tracking confidence coefficient rho by weighted average, wherein the calculation is shown as a formula (7).
ρ＝0.5*ρ c +0.5*ρ a (7)
And step 3: tracking result study and judgment
And (4) studying and judging the tracking result by combining the tracking confidence coefficient and the target tracking historical track information.
The multi-frame interval constraint specifically refers to motion displacement constraint of a tracking target among multiple frames, and a prediction displacement deviation threshold value of a current frame and a historical frame is calculated according to interframe displacement change of a historical continuous tracking successful frame N. To implement this strategy, it is necessary to store historical tracking data for the most recent consecutive frames (e.g., N frames, N =5 in the present invention). Three cache queues are used for storing data, and a specific storage and update strategy is shown in fig. 3 and described as follows:
caching 1: the system is used for calculating the deviation distance of the currently processed N frames, storing the tracking track information of the latest N frames, updating the strategy that the current frame is processed and stored in a cache, and deleting the processing result of the earliest frame;
and (4) caching 2: the method is used for storing N frame data of a previous stage of a current stage and updating a cache 3, wherein the updating strategy is to initialize a cache 2 after the cache 1 is full for the first time, and then press the oldest value in the cache 1 into the cache 2 every time the cache 1 is updated, and simultaneously remove the oldest value in the cache 2.
And (3) caching: the method is used for calculating the latest historical prediction, storing historical continuous latest successful N frames, and adopting the updating strategy of copying the continuous successful N frames from the 'cache 2', and adopting the average frame interval displacement of one-time calculation, so that the method can be used for predicting the next frames for multiple times; the average frame interval displacement is calculated and then emptied.
The final tracking result judgment strategy based on the tracking confidence coefficient and the multi-frame interval displacement constraint is as follows: only if the tracking confidence p is greater than an acceptable threshold thr a (value 0.15) and the interval displacement between the current frame and the previous N frames (if the current frame is t frame, namely the displacement between t frame and t-N frame) is less than the acceptable multi-frame interval constraint threshold thr m And judging that the tracking result is correct, otherwise, failing to track.
And 4, step 4: tracking model management
And when the tracking result judges that the tracking of the current frame is successful, managing the tracking model. The invention adopts a mode of combining frame-by-frame weighted updating and stage restarting preparation to update the filter template to adapt to the target scale and the apparent change. The specific method comprises the following steps:
firstly, counting the continuous tracking success times, and judging whether to restart actively according to the continuous tracking success times.
If the number of continuous successful tracking times is less than N, in this embodiment, when N =5, in order to adapt to a weak difference between short-time small scene frames, a policy of weighting and updating the template frame by frame is adopted. However, the result of each frame is used for updating, or frame-by-frame updating is risky, especially when the target is occluded or the tracker is not good enough, the model is updated, so that the tracker can not identify the target more and more, the invention only judges that the tracking is successful in step 3 and the tracking confidence coefficient p is greater than the threshold thr l (thr a <thr l <1，thr l And (4) updating the tracking model when the value is 0.3), so that the target model is prevented from being polluted, the model drifting and updating times are reduced, and the speed is increased. The calculation method for updating the tracking model frame by frame is shown in equations (8) and (9).
F(x) t ＝(1-r)F(x) t-1 +rF(x) t (8)
F(a) t ＝(1-r)F(a) t-1 +rF(a) t (9)
Wherein F (-) represents Fourier transform operation, x represents extracted target and background features, a represents filter parameters, r represents updating rate, the larger the r value is, the larger the weight of the current frame is, but the larger r value is easy to pollute the template by the current frame. Therefore, the update rate needs to be adaptively adjusted according to the tracking confidence level. The current confidence ρ is low (thr) l <ρ<thr h ,thr h = 0.7), the update rate is 0.035, and the confidence p is high (thr) h Rho is not less than 1) and the update rate is 0.085.
If the number of continuous successful tracking times is equal to N, when N =5 in the embodiment, in order to adapt to the large apparent change of the long-term large scene, the invention retrains the filter at the current frame, initializes the tracking parameters and the tracking filter templateAnd a target apparent template. When preparing the tracking filter template, the determination of the tracking area and the search area considers the target size and the calculation speed limit at the same time. According to the initial frame 0 time object distance d 0 Focal length f 0 Angle of photographing theta 0 Object distance d from current frame t t Focal length f t Angle of incidence theta t Estimating an expansion coefficient of a target, and determining the size of the target in the current frame scale, wherein the rough estimation method of the expansion coefficient gamma comprises the following steps:
Figure BDA0003424500750000091
then, considering the calculation speed limit, when the target image is small (the short side is 54 pixels or less), a rectangular frame with the short side extended by 10 pixels is selected as the tracking area with the tracking point as the center, and when the target image is large (the short side is 54 pixels or more), an area with a fixed size of 64 × 64 with the tracking point as the center is selected as the tracking area. And the coarse tracking and the fine tracking are respectively expanded outwards by 1 time and 2 times to be used as a search area to establish a coarse tracking filter template and a fine tracking filter template.
The target appearance template selects an image area with a tracking point as a center 5*5.
And 5: loss-weight compensation
When the tracking result is that the tracking of the current frame fails, lost recapture work needs to be carried out, whether the tracking is stopped or not is judged firstly, namely whether continuous multi-frame tracking fails or not is judged, and the tracking process is ended if continuous N =5 frames fail or not. Otherwise, restarting tracking in a failure state is carried out by utilizing the historical successful tracking information to carry out target loss recapture. The specific method comprises the following steps:
sequentially selecting the frame with the highest confidence from the tracking cache according to the size of the tracking area at the current moment, and expanding a double search area to prepare a tracking filter template to search again in the current frame;
calculating confidence coefficient rho and multi-frame interval displacement of the search result, studying and judging the tracking result according to the strategy in the step 3, and if the search is successful, re-initializing the coarse and fine tracking filter template and the image appearance template in the current frame according to the tracking area and search area setting method in the step 4;
and if the search fails, the next frame processing is carried out.
The long-term target tracking method for template adaptive update provided by the present invention is described in detail above, but it is obvious that the specific implementation form of the present invention is not limited thereto. It will be apparent to those skilled in the art that various obvious changes may be made therein without departing from the scope of the invention as defined in the appended claims.

Claims (1)

1. A long-time region tracking method adapting to scene and target changes is characterized by comprising the following steps:
tracking the target position of an image acquired by an unmanned aerial vehicle monitoring system or a high-altitude observation system based on a three-stage combination mode of rough and fine tracking and center position correction;
calculating a tracking confidence coefficient of the target position, correcting the tracking confidence coefficient of the target position based on the apparent similarity of the target, and comprehensively studying and judging the corrected tracking result by combining multi-frame interval constraint;
according to the comprehensive judgment result, the tracking model is updated in a self-adaptive mode when the tracking is successful, and the target is captured again when the tracking is failed;
the self-adaptive updating tracking model when the tracking is successful comprises the following steps:
when the continuous successful tracking times are less than N frames, the coarse tracking filter template and the fine tracking filter template are weighted and updated frame by frame to adapt to weak difference between frames;
when the continuous successful tracking times are equal to N frames, the coarse tracking filter template, the fine tracking filter template and the image apparent template T are reinitialized by using the tracking result of the current frame a To accommodate target apparent significant changes;
the method for determining the sizes of the tracking area and the search area when the coarse tracking filter template and the fine tracking filter template are reinitialized simultaneously considers the target size and the calculation speed limit, and specifically comprises the following steps:
according to the initial frame 0 time object distance d 0 Focal length f 0 And a shooting angle theta 0 Object distance d from current frame t t Focal length f t Angle of incidence theta t Estimating an expansion coefficient of a target, and determining the size of the target in the current frame scale, wherein the rough estimation method of the expansion coefficient gamma comprises the following steps:
Figure FDA0004046332940000011
considering the calculation speed limit, when the short side of the target imaging is smaller than or equal to 54 pixels, selecting a rectangular frame with the length of the short side expanded by 10 pixels as a tracking area by taking a tracking point as the center, when the short side of the target imaging is larger than 54 pixels, selecting a 64 × 64 area with the tracking point as the center as the tracking area, expanding the fine tracking outwards by 1 time as a search area, expanding the coarse tracking outwards by 2 times as the search area, and respectively creating a fine tracking filter template and a coarse tracking filter template;
when the tracking fails, in order to realize the long-term tracking, lost recapture is needed, which mainly comprises:
sequentially selecting the frame with the highest confidence from the tracking cache according to the size of the tracking area at the current moment, expanding the frame by two times to be used as a search area for preparing a filter template, and searching again in the current frame;
tracking the original frame image, the target position and the confidence coefficient which are recently judged to be successful and stored in the cache;
after the search is successful, the coarse tracking filter template, the fine tracking filter template and the image appearance template are initialized again in the current frame;
if the searching fails, the method is continuously repeated for the next frame, and when the target position cannot be obtained again in the continuous N frames, the target loss is declared, and the target tracking program is terminated;
the tracking of the target position of the image acquired by the unmanned aerial vehicle monitoring system or the high-altitude observation system based on the mode of three-stage combination of rough and fine tracking and center position correction comprises the following steps:
according to the target position P of the previous frame t-1 Selecting and coarse tracking filtering at the corresponding position of the frameImage blocks with the same template size are used as coarse tracking search areas for coarse tracking search to obtain a primary position estimate P of the target c If the value of the current tracking response map corresponding to the position is higher than the threshold value thr ρc Determining the position of the fine tracking search center point as the point, otherwise, still adopting P t-1 The position of the central point of the fine tracking search of the current frame is taken as the position of the central point of the fine tracking search of the current frame;
selecting an image block with the same size as the template of the last frame fine tracking filter at the determined position of the fine tracking search central point as a fine tracking search area to perform fine tracking search, and obtaining a fine tracking position P f If the value of the current tracking response map corresponding to the position is higher than the threshold value thr ρc Receiving the fine tracking search result, and turning to the next step, otherwise, losing and recapturing the frame if the tracking of the frame fails;
at the fine tracking position P f Top and surrounding selection and image appearance template T a Image regions of the same size as the image apparent template T a Performing average absolute difference algorithm MAD, and taking the position point with the maximum similarity as the final tracking position P t ；
Calculating a target position tracking confidence coefficient, and correcting the target position tracking confidence coefficient based on the target apparent similarity comprises the following steps:
the peak-to-side lobe ratio PSR is calculated as shown in equation (1) for the tracking response plot and is designated as PSR cur Reflecting the intensity of the main peak relative to the side lobe, in the formula F max Is the response value of the peak, μ sub And σ sub Is the mean and standard deviation of the sidelobes;
Figure FDA0004046332940000021
calculating the peak sidelobe ratio psr of the current frame cur PSR mean PSR of M frame tracking response with latest continuous success avg The ratio of (a) to (b) reflects the oscillation degree of the PSR, and determines the target position tracking confidence coefficient rho of the current frame c ；
Figure FDA0004046332940000022
Calculated to finally track position P t Image block and image apparent template T in 5*5 area as center a Comparing the MAD value of the current frame with the MAD average value of the latest continuous successful M frame tracking result to obtain the normalized image apparent similarity rho a ；
Target position tracking confidence ρ c Apparent similarity to image ρ a Weighted average is carried out to obtain a corrected current tracking confidence coefficient rho;
the multi-frame interval constraint threshold is determined by the interframe displacement change of the latest continuous successful N frames in the historical track and a constant value c, and when the tracking confidence coefficient rho is greater than the threshold thr a And the interval displacement between the current frame and the previous N frames is less than a multi-frame interval constraint threshold thr m And judging that the tracking result is correct, otherwise, failing to track.
CN202111573298.1A 2021-12-21 2021-12-21 Long-time region tracking method adaptive to scene and target change Active CN114241008B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111573298.1A CN114241008B (en) 2021-12-21 2021-12-21 Long-time region tracking method adaptive to scene and target change

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111573298.1A CN114241008B (en) 2021-12-21 2021-12-21 Long-time region tracking method adaptive to scene and target change

Publications (2)

Publication Number Publication Date
CN114241008A CN114241008A (en) 2022-03-25
CN114241008B true CN114241008B (en) 2023-03-07

Family

ID=80760614

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111573298.1A Active CN114241008B (en) 2021-12-21 2021-12-21 Long-time region tracking method adaptive to scene and target change

Country Status (1)

Country Link
CN (1) CN114241008B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114663462A (en) * 2022-04-07 2022-06-24 北京远度互联科技有限公司 Target tracking method and device, electronic equipment and storage medium
CN116563348B (en) * 2023-07-06 2023-11-14 中国科学院国家空间科学中心 Infrared weak small target multi-mode tracking method and system based on dual-feature template

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110335293A (en) * 2019-07-12 2019-10-15 东北大学 A kind of long-time method for tracking target based on TLD frame
CN111091583A (en) * 2019-11-22 2020-05-01 中国科学技术大学 Long-term target tracking method
CN111508002A (en) * 2020-04-20 2020-08-07 北京理工大学 Small-sized low-flying target visual detection tracking system and method thereof
CN113327272A (en) * 2021-05-28 2021-08-31 北京理工大学重庆创新中心 Robustness long-time tracking method based on correlation filtering

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105578034A (en) * 2015-12-10 2016-05-11 深圳市道通智能航空技术有限公司 Control method, control device and system for carrying out tracking shooting for object

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110335293A (en) * 2019-07-12 2019-10-15 东北大学 A kind of long-time method for tracking target based on TLD frame
CN111091583A (en) * 2019-11-22 2020-05-01 中国科学技术大学 Long-term target tracking method
CN111508002A (en) * 2020-04-20 2020-08-07 北京理工大学 Small-sized low-flying target visual detection tracking system and method thereof
CN113327272A (en) * 2021-05-28 2021-08-31 北京理工大学重庆创新中心 Robustness long-time tracking method based on correlation filtering

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
基于特征融合的长时目标跟踪算法;葛宝义等;《光学学报》;20180627;第38卷(第11期);第1-13页 *
时空上下文学习长时目标跟踪;刘威等;《光学学报》;20160110;第36卷(第01期);第1-8页 *
自适应目标变化的时空上下文抗遮挡跟踪算法;张晶等;《计算机工程与科学》;20180915;第40卷(第09期);第1653-1661页 *

Also Published As

Publication number Publication date
CN114241008A (en) 2022-03-25

Similar Documents

Publication Publication Date Title
CN111127518B (en) Target tracking method and device based on unmanned aerial vehicle
CN114241008B (en) Long-time region tracking method adaptive to scene and target change
CN110796010B (en) Video image stabilizing method combining optical flow method and Kalman filtering
CN110728697B (en) Infrared dim target detection tracking method based on convolutional neural network
CN107424177B (en) Positioning correction long-range tracking method based on continuous correlation filter
KR100985805B1 (en) Apparatus and method for image stabilization using adaptive Kalman filter
CN101860729A (en) Target tracking method for omnidirectional vision
CN110827321B (en) Multi-camera collaborative active target tracking method based on three-dimensional information
CN112991391A (en) Vehicle detection and tracking method based on radar signal and vision fusion
CN113379801B (en) High-altitude parabolic monitoring and positioning method based on machine vision
KR101308347B1 (en) The Detection and Recovery Method of Occlusion of a Face Image using a Correlation Based Method
CN105374049B (en) Multi-corner point tracking method and device based on sparse optical flow method
CN110717934B (en) Anti-occlusion target tracking method based on STRCF
CN112258553B (en) All-time target tracking method based on multi-source image fusion
CN106296730A (en) A kind of Human Movement Tracking System
CN111931685A (en) Video satellite moving target detection method based on bidirectional tracking strategy
JP2022027464A (en) Method and device related to depth estimation of video
CN117036404A (en) Monocular thermal imaging simultaneous positioning and mapping method and system
CN116381672A (en) X-band multi-expansion target self-adaptive tracking method based on twin network radar
CN115439777A (en) Video satellite target tracking method based on multi-feature fusion and motion estimation
CN110751671A (en) Target tracking method based on kernel correlation filtering and motion estimation
CN116503244A (en) Front-view sonar image stitching method capable of retaining information
CN115511804A (en) Offshore infrared dim target detection method based on pipeline adaptive filtering
CN115953431A (en) Multi-target tracking method and system for aerial video of unmanned aerial vehicle
CN115345845A (en) Feature fusion smoke screen interference efficiency evaluation and processing method based on direction gradient histogram and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant