CN114241008A - Long-time region tracking method adaptive to scene and target change - Google Patents

Long-time region tracking method adaptive to scene and target change Download PDF

Info

Publication number
CN114241008A
CN114241008A CN202111573298.1A CN202111573298A CN114241008A CN 114241008 A CN114241008 A CN 114241008A CN 202111573298 A CN202111573298 A CN 202111573298A CN 114241008 A CN114241008 A CN 114241008A
Authority
CN
China
Prior art keywords
tracking
target
frame
fine
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111573298.1A
Other languages
Chinese (zh)
Other versions
CN114241008B (en
Inventor
李波
辛明
张贵伟
张超
刘偲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beihang University
Original Assignee
Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beihang University filed Critical Beihang University
Priority to CN202111573298.1A priority Critical patent/CN114241008B/en
Publication of CN114241008A publication Critical patent/CN114241008A/en
Application granted granted Critical
Publication of CN114241008B publication Critical patent/CN114241008B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/251—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/262—Analysis of motion using transform domain methods, e.g. Fourier domain methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)
  • Radar Systems Or Details Thereof (AREA)

Abstract

The invention discloses a long-term region tracking method adaptive to scene and target changes, which comprises the following steps: the tracking position is quickly and accurately predicted in a mode of combining rough and fine tracking and center position correction; multi-thread research and judgment, wherein the tracking confidence coefficient is corrected by combining the apparent similarity of the target, the multi-frame interval constraint is considered, the tracking result is comprehensively researched and judged, the tracking false alarm is eliminated, and the reliability of the tracking result is improved; and (4) flexible tracking, namely updating a tracking area in a self-adaptive manner when the tracking is successful according to a tracking research and judgment result, and re-capturing the target when the tracking is failed. And when the tracking result is judged to be successful, adapting to the weak change between frames by adopting a frame-by-frame weighting updating mode at short-time small scene intervals, and adapting to the apparent significant change of a large-interval target through an active restarting module at long-time intervals. The method has the advantages of fast tracking response, high precision and reliable confidence coefficient.

Description

Long-time region tracking method adaptive to scene and target change
Technical Field
The invention belongs to the technical field of digital image processing, and mainly solves the problem of long-time tracking of a specific target under the condition of movement of imaging equipment. The algorithm can actively adapt to target scale and apparent change, has small calculation amount, and can be well deployed on various low-performance hardware platforms.
Background
The main purpose of target tracking is to simulate the motion perception function of a physiological visual system, analyze an image sequence captured subsequently by imaging equipment after obtaining the initial state of an interested target to be tracked, calculate the position of the target in each frame of image, obtain the motion trajectory of the target in a specific space-time range, and provide technical support for advanced processing and application such as target identification, target behavior analysis, three-dimensional reconstruction and the like. In recent years, the target tracking technology has been continuously and rapidly developed, and has been widely applied to the fields of national defense and military, such as battlefield reconnaissance and monitoring, border patrol, key target positioning, correction and striking, electronic warfare and damage assessment. The technology is widely applied to civil fields such as security monitoring, traffic operation, aerial photography, disaster monitoring and the like.
A long-time region tracking method adaptive to scene and target changes mainly aims at an unmanned aerial vehicle monitoring system or an overhead observation system, a monitored target is a specific rigid target, the monitoring distance can reach several kilometers or even more than ten kilometers, in order to meet all-weather requirements all day long, infrared monitoring equipment is generally installed, and in order to lock the target, various posture changes of carrying equipment or a holder can occur. In this application mode, the difficulty of target tracking is shown in the following aspects: (1) the apparent change is large from the point of view of the imaging of the target itself. When the shooting distance is long, the infrared imaging method is limited by the limitation of infrared imaging, the target occupies few pixels, insufficient texture information is lacked, and the apparent characteristic is weak. The self apparent imaging change inevitably occurs in the process of continuously tracking the locked target, which comprises the change of the target dimension and the change of the appearance detail caused by the change of the distance between the imaging device and the target, and also comprises the apparent change of the target caused by the change of the posture or the running track. (2) In terms of external environment, similar targets may interfere with each other and shield each other in the operation process when shooting at a long distance, and in addition, due to a nodding view angle, the targets are easily submerged in the surrounding background; finally, the motion of the imaging platform may also cause motion blur and even movement of the target out of view. (3) From the limitation of the tracking algorithm, in the current real-time target tracking algorithm, no matter a generative model or a discriminant model, the basis is that the sizes and the spatial positions of the same object in two frames before and after are not changed greatly in the same video, and the position of the target is determined in the next frame based on a given target template or a trained classifier. The essential difference in tracking over other visual tasks is the ability to accommodate gradual changes in moving objects. Online updates play a crucial role for visual tracking. However, online updating is also a double-edged sword in terms of balanced dynamic information description and accidental noise introduction, accumulates errors for a long time, collects inappropriate samples when the target disappears or over-fits available data, easily degrades the performance of the tracker, resulting in tracking drift.
In the actual tracking process, although many tracking algorithms make obvious progress in establishing an appearance model and robust tracking at present, target tracking is still a very complex problem in the presence of many practical difficulties. Compared with the deep learning algorithm with great heat in recent years, in a real-time processing system, the correlation filtering tracking algorithm is a hot spot of current research by virtue of excellent performances of target tracking accuracy, robustness to target apparent change and speed. The method carries out convolution operation on the image of the next frame and the filtering template by learning the filtering template, predicts the target position according to the output response, converts the convolution operation of the image into dot multiplication in a frequency domain by using FFT (fast Fourier transform) in actual calculation, greatly reduces the calculation complexity, and can achieve the processing speed of hundreds of frames per second when the target is described by using the original pixel value. In order to meet the real-time processing requirement, study and judge the prediction result after completing basic position prediction and retrieve the target again after the target is lost, the invention provides a self-adaptive updating long-term target tracking method under a related filtering framework, and mainly solves the following technical problems.
(1) The predicted position is accurate: aiming at the real problem that a target is weak and small in remote infrared imaging and is easily submerged in a surrounding background, if the target is positioned only on the appearance of the target, the tracking position is easily deviated by the current method for determining the position of the target once based on the maximum response value of confidence coefficient. The method needs to be capable of fully utilizing the target and the surrounding background information, accurately determining the final position of the target and ensuring the accuracy of the tracking result.
(2) The tracking result is credible: the method aims at the problem of target drift caused by various external interferences, and the main reason is that the tracking algorithm cannot reliably judge the tracking result of the tracking algorithm, so that a model updating mechanism has problems. A robust tracker should be able to perform multi-stage verification on the tracking result through various external physical constraints, so as to ensure the reliability of the tracking result.
(3) The updated content is correct: for the realistic problem of the apparent change of the target, the conventional tracker adopts a frame-by-frame updating method to adapt to the dynamic change of the posture or the scale of the target object at the later stage after the target object is determined and tracked in the first frame, but the updating mode of the fixed tracking target object is suitable for the scene with slow apparent change and continuous motion track of the target in the field of view, and the problem of severe scale change cannot be thoroughly solved. When the tracked object exceeds the visual field and only the target is partially left in the visual field, the updating mode causes tracking drift due to edge filling, and meanwhile, the calculation amount is greatly increased. The invention needs the tracker to continuously adapt to the updating of the target template in the whole tracking process, adapts to the severe scale and the apparent change of the target in the tracking process and can ensure the tracking speed and the tracking precision.
Disclosure of Invention
In order to realize wide-area long-time target tracking under continuous visual angles, the invention provides a long-time area tracking method adaptive to scene and target changes.
In order to achieve the purpose, the invention adopts the following technical scheme:
a long-time region tracking method adapting to scene and target changes comprises the following steps:
tracking the target position of an image acquired by an unmanned aerial vehicle monitoring system or a high-altitude observation system based on a three-stage combination mode of rough and fine tracking and center position correction;
calculating a tracking confidence coefficient of the target position, correcting the tracking confidence coefficient of the target position based on the apparent similarity of the target, and comprehensively studying and judging the corrected tracking result by combining multi-frame interval constraint;
and according to the comprehensive judgment result, the tracking model is updated in a self-adaptive manner when the tracking is successful, and the target is recaptured when the tracking is failed.
The long-term area tracking method adaptive to scene and target changes is characterized in that target position tracking of images acquired by an unmanned aerial vehicle monitoring system or a high-altitude observation system based on a mode of three-stage combination of rough and fine tracking and center position correction comprises the following steps:
according to the target position P of the previous framet-1Selecting image blocks with the same size as the coarse tracking filter template from the corresponding positions of the frame as coarse tracking search areas to perform coarse tracking search, and obtaining a primary position estimate P of the targetcIf the value of the current tracking response map corresponding to the position is higher than the threshold value thrρcDetermining the position of the fine tracking search center point as the point, otherwise, still adopting Pt-1The position of the central point of the fine tracking search of the frame is used;
selecting the image block with the same size as the template of the last frame fine tracking filter at the determined position of the fine tracking search central point as a fine tracking search area to perform fine tracking search to obtain a fine tracking position PfIf the value of the current tracking response map corresponding to the position is higher than the threshold value thrρcReceiving the fine tracking search result, and turning to the next step, otherwise, losing and recapturing the frame if the tracking of the frame fails;
at the fine tracking position PfTop and surrounding selection and image appearance template TaImage regions of the same size as the image appearance template TaPerforming average absolute difference algorithm MAD, and taking the position point with the maximum similarity as the final tracking position Pt。
The long-term region tracking method adapting to scene and target changes is characterized in that calculating a target position tracking confidence coefficient, and correcting the target position tracking confidence coefficient based on the target apparent similarity comprises the following steps:
the peak-to-side lobe ratio PSR is calculated as shown in equation (1) for the tracking response plot and is designated as PSRcurReflecting the intensity of the main peak relative to the side lobe, in the formula FmaxIs the response value of the peak, μsubAnd σsubIs the mean and standard deviation of the side lobes;
Figure BDA0003424500750000031
calculating the peak sidelobe ratio psr of the current framecurPSR mean PSR of M frame tracking response with latest continuous successavgThe ratio of (a) to (b) reflects the oscillation degree of the PSR, and determines the target position tracking confidence coefficient rho of the current framec；
Figure BDA0003424500750000032
Calculated to finally track position PtImage blocks and image appearance templates T in 5-by-5 regions as centersaComparing the MAD value of the current frame with the MAD average value of the latest continuous successful M frame tracking result to obtain the normalized image apparent similarity rhoa；
Target position tracking confidence ρcApparent similarity to image ρaAnd obtaining the corrected current tracking confidence coefficient rho by weighted average.
The long-term region tracking method adapting to scene and target changes is characterized in that a multi-frame interval constraint threshold is determined by the interframe displacement change of the latest continuous successful N frames in a historical track and a constant value c, and when a tracking confidence coefficient rho is greater than a threshold thraAnd the interval displacement between the current frame and the previous N frames is less than a multi-frame interval constraint threshold thrmTemporal decision tracking resultCorrect, otherwise the trace fails.
The long-term region tracking method adapting to scene and target changes is characterized in that when tracking is successful, the self-adaptive updating of the tracking region comprises the following steps:
when the continuous successful tracking times are less than N frames, the coarse tracking filter module and the fine tracking filter module are weighted and updated frame by frame to adapt to weak difference between frames;
when the continuous successful tracking times are equal to N frames, the tracking result of the current frame is used for reinitializing the coarse tracking filter module, the fine tracking filter module and the image apparent template TaTo accommodate significant changes in target appearance;
the method for determining the sizes of the tracking area and the search area when the coarse tracking filter template and the fine tracking filter template are reinitialized simultaneously considers the target size and the calculation speed limit, and specifically comprises the following steps:
according to the initial frame 0 time object distance d0Focal length f0Angle of photographing theta0Object distance d from current frame ttFocal length ftAngle of incidence thetatEstimating an expansion coefficient of a target, and determining the size of the target in the current frame scale, wherein the rough estimation method of the expansion coefficient gamma comprises the following steps:
Figure BDA0003424500750000041
considering the calculation speed limit, when the imaging short side of the target is equal to or less than 54 pixels, a rectangular frame with the short side extended by 10 pixels is selected as the tracking area with the tracking point as the center. And when the short side of the target imaging is larger than 54 pixels, selecting a 64 x 64 area taking the tracking point as the center as a tracking area, expanding the fine tracking outwards by 1 time as a search area, expanding the coarse tracking outwards by 2 times as a search area, and respectively creating a coarse and fine tracking filter template.
The long-term area tracking method adaptive to scene and target changes is characterized in that loss recapture is required to realize long-term tracking when tracking fails, and mainly comprises the following steps:
sequentially selecting the frame with the highest confidence from the tracking cache according to the size of the tracking area at the current moment, and expanding a double search area to prepare a filter template to search again in the current frame;
tracking the original frame image, the target position and the confidence coefficient which are recently judged to be successful and stored in the cache;
after the search is successful, the coarse tracking filter template, the fine tracking filter template and the image appearance template are initialized again in the current frame;
if the searching fails, the method is continuously repeated for the next frame, the target is declared lost after the target position cannot be obtained again by the continuous N frames, and the target tracking program is terminated.
The long-term region tracking method adaptive to scene and target changes designed by the invention has the following advantages:
(1) the response is quick: and (3) predicting the target position by adopting a related filtering method, wherein the normal tracking processing time of the algorithm on the embedded platform frame by frame is less than 10 ms. The processing time does not exceed 20ms at the maximum when target loss requires reacquisition.
(2) And (3) accurate tracking: compared with the existing related filtering tracking algorithm, the method adopts a coarse and fine tracking combination mode, can utilize the tracking response result and historical tracking information to carry out update opportunity judgment, and has the characteristic of high accuracy of the tracking result compared with the existing CSK, KCF and ECO-HC methods.
(3) And (4) judging: the method combines various clues, comprehensively studies and judges the tracking result by combining the time-space information, provides the confidence judgment of the result on the basis of giving the target prediction position, and is more reliable compared with the output result of the traditional method.
Drawings
The present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
FIG. 1 is a general framework of the long-term region tracking method of the present invention that adapts to scene and target changes;
FIG. 2 is a flow diagram of a frame-by-frame tracking process;
FIG. 3 is a multi-frame buffer management policy;
fig. 4 is a schematic diagram of tracking target loss recapture.
Detailed Description
The invention proposes a long-term region tracking method adaptive to scene and target changes, and the following description clearly and completely describes the specific embodiments of the invention with reference to the accompanying drawings.
The embodiment discloses a long-term region tracking method adaptive to scene and target changes, and the overall framework is as shown in fig. 1:
after the initial frame obtains the target to be tracked, a coarse and fine tracking filter template and an image apparent template T are carried outaPreparing, when a new subsequent frame of image arrives, selecting a candidate search area, predicting a target position, then combining various clues to calculate confidence coefficient of a tracking result and comprehensively studying and judging the target tracking confidence coefficient, and eliminating a tracking false alarm; and finally, updating the tracking model according to the research and judgment result.
In this embodiment, a specific processing flow per frame is shown in fig. 2, and includes:
step 1: target location prediction
The device mainly comprises two parts: and carrying out rough and fine tracking search and center position correction.
(a) Firstly, coarse tracking search is carried out to obtain a primary position P of a targetcIf the maximum response value for that location is greater than the acceptable threshold thrρcThen fine tracking is performed at this position, otherwise the position P is directly at the previous framet-1And performing fine tracking. The rough and fine tracking adopts a tracking principle based on relevant filtering, specifically, according to the position of a target of a previous frame, an image block with the same size as a tracking filter template (including the rough tracking filter template and the fine tracking filter template) is selected as a search area at the corresponding position of the current frame, characteristics are extracted, a cosine window is added to eliminate boundary influence, and the search area characteristics are assumed to be z, and the search area characteristics and a target characteristic model x of the previous frame are operated to obtain a kernel matrix k as shown in formula 1; then, as shown in formula 2, the filter parameter a calculated in the previous frame and the kernel matrix k are used to perform dot multiplication operation in the frequency domain, and the calculation result is subjected to inverse Fourier transform F-1In the time domain, the phase is obtainedClosing the response graph y after the filtering operation; and finally, the position of the maximum probability of the target in the current frame can be calculated by searching the maximum value Fmax and the corresponding coordinates (px, py) in the response graph y.
Figure BDA0003424500750000061
y＝F-1(F(a)·F(k)) (2)
In the process, the coarse tracking and the fine tracking all adopt related filtering methods, the difference between the two methods is that after a tracking area is selected, the size of the searching area is set to be different, the specific size sets an initial value according to the actual situation, and meanwhile, in order to ensure the speed, the coarse tracking carries out downsampling operation based on nearest neighbor interpolation on the searching area.
(b) When the position P is accurately trackedfThe corresponding maximum response value is greater than the acceptable threshold thrρcThen, the center position is corrected at this position by the apparent similarity of the images. The specific method is that the current frame position point P isfAnd four candidate points P1, P2, P3 and P4 are selected up, down, left and right around. Respectively selecting and imaging the apparent template T by taking the 5 points as the centeraImage regions of the same size, in this embodiment, 5 × 5 regions are selected to form new target apparent images S, S1, S2, S3 and S4, and then the new target apparent images S, S1, S2, S3 and S4 are respectively associated with the image apparent templates TaPerforming image matching algorithm MAD (mean absolute difference) based on gray scale, and taking the position point with the maximum similarity as the final tracking point to replace PfAs final tracking result P of the trackert. The MAD calculation is shown in equation (3):
Figure BDA0003424500750000062
wherein: m is equal to N is equal to 5, and d represents the average absolute difference between the pixel values of the image block S and the image apparent template T.
Step 2: confidence correction
The device comprises two parts: and the tracking filter responds to the space-time analysis and the apparent similarity calculation of the tracking area image.
The tracker self-response space-time analysis firstly calculates Peak to side lobe Ratio (PSR) of a tracking response graph solved by a correlation filtering tracking algorithm, and records the PSR as PSRcur. PSR can be used to represent the correlation peak intensity. The correlation output g is divided into two parts: the peak value is the maximum value, and the side lobe is selected to be in an area 11 x 11 with the peak value as the center. The ratio of the main peak to the side lobe is mainly reflected, and when the mean value of the side lobe is large or the distribution is uneven, the PSR value can be reduced. The specific calculation is shown in formula (4):
Figure BDA0003424500750000071
Fmaxis the response value of the peak, μsubAnd σsubIs the mean and standard deviation of the side lobes.
The change of the tracking response graph does not change violently during normal tracking, and only when the target is shielded or lost, the change can change violently, so that the historical tracking information can also provide reference for calculation of the current tracking confidence. The method utilizes the mean value of the tracking result of the historical effective frame to normalize and represent the peak sidelobe ratio as shown in a formula (5), and represents the tracking confidence coefficient rho of the target position according to the normalized expressioncThis method uses the current PSR value PSRcurAnd the PSR mean PSR of the historical effective frameavgThe ratio of the two-dimensional space-time-domain signal to the confidence coefficient index of the final tracking filter has the advantages that the method can be self-adaptive to each section of test scene, and effectively overcomes the defects of poor generalization of a fixed threshold value and the like.
Figure BDA0003424500750000072
The tracking confidence degree calculation method based on the image apparent similarity comprises the following steps: the previously calculated MAD is normalized by calculating the MAD average value of the latest continuous successful M-frame tracking results in the history, and in this embodiment, the MAD average value d of the tracking results of the first three frames (M is 3) is specifically takenavgAs a reference similarity value apparent from the current tracking image, and then currentTracking point MAD value d calculated by framecurSimilarity to reference value davgThe ratio is shown in equation (6).
Figure BDA0003424500750000073
Target position tracking confidence ρcApparent similarity to image ρaAnd (4) obtaining the corrected current tracking confidence coefficient rho by weighted average, wherein the calculation is shown as a formula (7).
ρ＝0.5*ρc+0.5*ρa (7)
And step 3: tracking result study and judgment
And (4) studying and judging the tracking result by combining the tracking confidence coefficient and the target tracking historical track information.
The multi-frame interval constraint specifically refers to motion displacement constraint of a tracking target among multiple frames, and a prediction displacement deviation threshold value of a current frame and a historical frame is calculated according to interframe displacement change of a historical continuous tracking successful frame N. In order to implement the strategy, it is necessary to store the tracking data of the most recent continuous multiframe (such as N frames, where N is 5 in the present invention). Three cache queues are used for storing data, and a specific storage and update strategy is shown in fig. 3 and described as follows:
caching 1: the system is used for calculating the deviation distance of the currently processed N frames, storing the tracking track information of the latest N frames, updating the strategy that the current frame is processed and stored in a cache, and deleting the processing result of the earliest frame;
and (4) caching 2: the method is used for storing N frame data of a previous stage of a current stage and updating a cache 3, wherein the updating strategy is to initialize a cache 2 after the cache 1 is full for the first time, and then press the oldest value in the cache 1 into the cache 2 every time the cache 1 is updated, and simultaneously remove the oldest value in the cache 2.
And (3) caching: the method is used for calculating the latest historical prediction and storing the historical continuous latest successful N frames, the updating strategy is to copy the continuous successful N frames from the 'cache 2', and the average frame interval displacement is calculated once and can be used for the prediction of the next frames for a plurality of times; the average frame interval displacement is calculated and then emptied.
The final tracking result judgment strategy based on the tracking confidence coefficient and the multi-frame interval displacement constraint is as follows: only if the tracking confidence p is greater than an acceptable threshold thra(value 0.15) and the interval displacement between the current frame and the previous N frames (if the current frame is t frame, namely the displacement between t frame and t-N frame) is less than the acceptable multi-frame interval constraint threshold thrmAnd judging that the tracking result is correct, otherwise, failing to track.
And 4, step 4: tracking model management
And when the tracking result judges that the tracking of the current frame is successful, managing the tracking model. The invention adopts a mode of combining frame-by-frame weighted updating and stage restarting preparation to update the filter template to adapt to the target scale and the apparent change. The specific method comprises the following steps:
firstly, counting the continuous tracking success times, and judging whether to restart actively according to the continuous tracking success times.
If the number of times of continuous successful tracking is less than N, in this embodiment, when N is 5, in order to adapt to a weak difference between short-time small scene frames, a policy of weighting and updating the template frame by frame is adopted. However, the result of each frame is used for updating, or frame-by-frame updating is risky, especially when the target is occluded or the tracker is not good enough, the model is updated, so that the tracker can not identify the target more and more, the invention only judges that the tracking is successful in step 3 and the tracking confidence coefficient p is greater than the threshold thrl(thra<thrl<1，thrlThe value of 0.3) is taken, the tracking model is updated, the target model is prevented from being polluted, the model drifting and updating times are reduced, and meanwhile, the speed is increased. The calculation method for updating the tracking model frame by frame is shown in equations (8) and (9).
F(x)t＝(1-r)F(x)t-1+rF(x)t (8)
F(a)t＝(1-r)F(a)t-1+rF(a)t (9)
Wherein F (-) represents Fourier transform operation, x represents extracted object and background features, a represents filter parameters, r represents update rate, and the larger the value of rLarge means that the current frame has a larger weight, but a larger value of r easily causes the template to be contaminated by the current frame. Therefore, the update rate needs to be adaptively adjusted according to the tracking confidence level. The current confidence ρ is low (thr)l<ρ<thrh,thrh0.7), the update rate is 0.035, and the confidence p is high (thr)hRho is not less than 1) and the update rate is 0.085.
If the number of continuous successful tracking times is equal to N, in this embodiment, when N is equal to 5, in order to adapt to a large change in appearance of a long-term large scene, the present invention retrains the filter at the current frame, and initializes the tracking parameters, the tracking filter template, and the target appearance template. When preparing the tracking filter template, the determination of the tracking area and the search area considers the target size and the calculation speed limit at the same time. According to the initial frame 0 time object distance d0Focal length f0Angle of photographing theta0Object distance d from current frame ttFocal length ftAngle of incidence thetatEstimating an expansion coefficient of a target, and determining the size of the target in the current frame scale, wherein the rough estimation method of the expansion coefficient gamma comprises the following steps:
Figure BDA0003424500750000091
then, considering the calculation speed limit, when the target image is small (the short side is 54 pixels or less), a rectangular frame with the short side extended by 10 pixels is selected as the tracking area with the tracking point as the center, and when the target image is large (the short side is 54 pixels or more), a region with a fixed size of 64 × 64 with the tracking point as the center is selected as the tracking area. And the coarse tracking and the fine tracking are respectively expanded outwards by 1 time and 2 times to be used as a search area to establish a coarse tracking filter template and a fine tracking filter template.
The target appearance template selects an image area with a tracking point as a center 5 x 5.
And 5: loss-weight compensation
When the tracking result is that the tracking of the current frame fails, lost recapture work is needed, whether the tracking is stopped is judged firstly, namely whether continuous multi-frame tracking fails is judged, and the tracking process is ended if continuous N is 5 frames which fail. Otherwise, restarting tracking in a failure state is carried out by utilizing the historical successful tracking information to recapture the lost target. The specific method comprises the following steps:
sequentially selecting the frame with the highest confidence from the tracking cache according to the size of the tracking area at the current moment, and expanding a double search area to prepare a tracking filter template to search again in the current frame;
calculating confidence coefficient rho and multi-frame interval displacement of the search result, studying and judging the tracking result according to the strategy in the step 3, and if the search is successful, re-initializing the coarse and fine tracking filter template and the image appearance template in the current frame according to the tracking area and search area setting method in the step 4;
and if the search fails, the next frame processing is carried out.
The long-term target tracking method for template adaptive update provided by the present invention is described in detail above, but it is obvious that the specific implementation form of the present invention is not limited thereto. It will be apparent to those skilled in the art that various obvious changes may be made therein without departing from the scope of the invention as defined in the appended claims.

Claims (6)

1. A long-time region tracking method adapting to scene and target changes is characterized by comprising the following steps:
tracking the target position of an image acquired by an unmanned aerial vehicle monitoring system or a high-altitude observation system based on a three-stage combination mode of rough and fine tracking and center position correction;
calculating a tracking confidence coefficient of the target position, correcting the tracking confidence coefficient of the target position based on the apparent similarity of the target, and comprehensively studying and judging the corrected tracking result by combining multi-frame interval constraint;
and according to the comprehensive judgment result, the tracking model is updated in a self-adaptive manner when the tracking is successful, and the target is recaptured when the tracking is failed.
2. A long-term region tracking method adapted to scene and object changes as claimed in claim 1, characterized by:
the tracking of the target position of the image acquired by the unmanned aerial vehicle monitoring system or the high-altitude observation system based on the mode of three-stage combination of rough and fine tracking and center position correction comprises the following steps:
according to the target position P of the previous framet-1Selecting image blocks with the same size as the coarse tracking filter template from the corresponding positions of the frame as coarse tracking search areas to perform coarse tracking search, and obtaining a primary position estimate P of the targetcIf the value of the current tracking response map corresponding to the position is higher than the threshold value thrρcDetermining the position of the fine tracking search center point as the point, otherwise, still adopting Pt-1The position of the central point of the fine tracking search of the frame is used;
selecting the image block with the same size as the template of the last frame fine tracking filter at the determined position of the fine tracking search central point as a fine tracking search area to perform fine tracking search, and obtaining a fine tracking position PfIf the value of the current tracking response map corresponding to the position is higher than the threshold value thrρcReceiving the fine tracking search result, and turning to the next step, otherwise, losing and recapturing the frame if the tracking of the frame fails;
at the fine tracking position PfTop and surrounding selection and image appearance template TaImage regions of the same size as the image appearance template TaPerforming average absolute difference algorithm MAD, and taking the position point with the maximum similarity as the final tracking position Pt。
3. A long-term region tracking method adapted to scene and object changes as claimed in claim 1, characterized by:
calculating a target position tracking confidence coefficient, and correcting the target position tracking confidence coefficient based on the target apparent similarity comprises the following steps:
the peak-to-side lobe ratio PSR is calculated as shown in equation (1) for the tracking response plot and is designated as PSRcurReflecting the intensity of the main peak relative to the side lobe, in the formula FmaxIs the response value of the peak, μsubAnd σsubIs the mean and standard deviation of the side lobes;
Figure FDA0003424500740000011
calculating the peak sidelobe ratio psr of the current framecurPSR mean PSR of M frame tracking response with latest continuous successavgThe ratio of (a) to (b) reflects the oscillation degree of the PSR, and determines the target position tracking confidence coefficient rho of the current framec；
Figure FDA0003424500740000021
Calculated to finally track position PtImage blocks and image appearance templates T in 5-by-5 regions as centersaComparing the MAD value of the current frame with the MAD average value of the latest continuous successful M frame tracking result to obtain the normalized image apparent similarity rhoa；
Target position tracking confidence ρcApparent similarity to image ρaAnd obtaining the corrected current tracking confidence coefficient rho by weighted average.
4. A long-term region tracking method adapted to scene and object changes as claimed in claim 1, characterized by:
the multi-frame interval constraint threshold is determined by the interframe displacement change of the latest continuous successful N frames in the historical track and a constant value c, and when the tracking confidence coefficient rho is greater than the threshold thraAnd the interval displacement between the current frame and the previous N frames is less than a multi-frame interval constraint threshold thrmAnd judging that the tracking result is correct, otherwise, failing to track.
5. A long-term region tracking method adapted to scene and object changes as claimed in claim 1, characterized by:
the self-adaptive updating tracking model when the tracking is successful comprises the following steps:
when the continuous successful tracking times are less than N frames, the coarse tracking filter module and the fine tracking filter module are weighted and updated frame by frame to adapt to weak difference between frames;
when the continuous successful tracking times are equal to N frames, the tracking result of the current frame is used for reinitializing the coarse tracking filter module, the fine tracking filter module and the image apparent template TaTo accommodate significant changes in target appearance;
the method for determining the sizes of the tracking area and the search area when the coarse tracking filter template and the fine tracking filter template are reinitialized simultaneously considers the target size and the calculation speed limit, and specifically comprises the following steps:
according to the initial frame 0 time object distance d0Focal length f0Angle of photographing theta0Object distance d from current frame ttFocal length ftAngle of incidence thetatEstimating an expansion coefficient of a target, and determining the size of the target in the current frame scale, wherein the rough estimation method of the expansion coefficient gamma comprises the following steps:
Figure FDA0003424500740000022
considering the calculation speed limit, when the short side of the target imaging is smaller than or equal to 54 pixels, a rectangular frame with the length of the short side expanded by 10 pixels is selected as a tracking area by taking the tracking point as the center, when the short side of the target imaging is larger than 54 pixels, a 64 × 64 area with the tracking point as the center is selected as the tracking area, the fine tracking is expanded outwards by 1 time to be used as a search area, the coarse tracking is expanded outwards by 2 times to be used as the search area, and a fine tracking filter template and a coarse tracking filter template are respectively created.
6. A long-term region tracking method that accommodates scene and object changes as recited in claim 5, wherein:
when the tracking fails, in order to realize the long-term tracking, lost recapture is needed, which mainly comprises:
sequentially selecting the frame with the highest confidence from the tracking cache according to the size of the tracking area at the current moment, expanding the frame by two times to be used as a search area for preparing a filter template, and searching again in the current frame;
tracking the original frame image, the target position and the confidence coefficient which are recently judged to be successful and stored in the cache;
after the search is successful, the coarse tracking filter template, the fine tracking filter template and the image appearance template are initialized again in the current frame;
if the searching fails, the method is continuously repeated for the next frame, the target is declared lost after the target position cannot be obtained again by the continuous N frames, and the target tracking program is terminated.
CN202111573298.1A 2021-12-21 2021-12-21 Long-time region tracking method adaptive to scene and target change Active CN114241008B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111573298.1A CN114241008B (en) 2021-12-21 2021-12-21 Long-time region tracking method adaptive to scene and target change

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111573298.1A CN114241008B (en) 2021-12-21 2021-12-21 Long-time region tracking method adaptive to scene and target change

Publications (2)

Publication Number Publication Date
CN114241008A true CN114241008A (en) 2022-03-25
CN114241008B CN114241008B (en) 2023-03-07

Family

ID=80760614

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111573298.1A Active CN114241008B (en) 2021-12-21 2021-12-21 Long-time region tracking method adaptive to scene and target change

Country Status (1)

Country Link
CN (1) CN114241008B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114663462A (en) * 2022-04-07 2022-06-24 北京远度互联科技有限公司 Target tracking method and device, electronic equipment and storage medium
CN116563348A (en) * 2023-07-06 2023-08-08 中国科学院国家空间科学中心 Infrared weak small target multi-mode tracking method and system based on dual-feature template

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180284777A1 (en) * 2015-12-10 2018-10-04 Autel Robotics Co., Ltd. Method, control apparatus, and system for tracking and shooting target
CN110335293A (en) * 2019-07-12 2019-10-15 东北大学 A kind of long-time method for tracking target based on TLD frame
CN111091583A (en) * 2019-11-22 2020-05-01 中国科学技术大学 Long-term target tracking method
CN111508002A (en) * 2020-04-20 2020-08-07 北京理工大学 Small-sized low-flying target visual detection tracking system and method thereof
CN113327272A (en) * 2021-05-28 2021-08-31 北京理工大学重庆创新中心 Robustness long-time tracking method based on correlation filtering

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180284777A1 (en) * 2015-12-10 2018-10-04 Autel Robotics Co., Ltd. Method, control apparatus, and system for tracking and shooting target
CN110335293A (en) * 2019-07-12 2019-10-15 东北大学 A kind of long-time method for tracking target based on TLD frame
CN111091583A (en) * 2019-11-22 2020-05-01 中国科学技术大学 Long-term target tracking method
CN111508002A (en) * 2020-04-20 2020-08-07 北京理工大学 Small-sized low-flying target visual detection tracking system and method thereof
CN113327272A (en) * 2021-05-28 2021-08-31 北京理工大学重庆创新中心 Robustness long-time tracking method based on correlation filtering

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
刘威等: "时空上下文学习长时目标跟踪", 《光学学报》 *
张晶等: "自适应目标变化的时空上下文抗遮挡跟踪算法", 《计算机工程与科学》 *
葛宝义等: "基于特征融合的长时目标跟踪算法", 《光学学报》 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114663462A (en) * 2022-04-07 2022-06-24 北京远度互联科技有限公司 Target tracking method and device, electronic equipment and storage medium
CN116563348A (en) * 2023-07-06 2023-08-08 中国科学院国家空间科学中心 Infrared weak small target multi-mode tracking method and system based on dual-feature template
CN116563348B (en) * 2023-07-06 2023-11-14 中国科学院国家空间科学中心 Infrared weak small target multi-mode tracking method and system based on dual-feature template

Also Published As

Publication number Publication date
CN114241008B (en) 2023-03-07

Similar Documents

Publication Publication Date Title
CN111127518B (en) Target tracking method and device based on unmanned aerial vehicle
CN114241008B (en) Long-time region tracking method adaptive to scene and target change
CN110796010B (en) Video image stabilizing method combining optical flow method and Kalman filtering
CN110728697B (en) Infrared dim target detection tracking method based on convolutional neural network
CN101860729A (en) Target tracking method for omnidirectional vision
KR100985805B1 (en) Apparatus and method for image stabilization using adaptive Kalman filter
CN110827321B (en) Multi-camera collaborative active target tracking method based on three-dimensional information
KR101308347B1 (en) The Detection and Recovery Method of Occlusion of a Face Image using a Correlation Based Method
CN107255468A (en) Method for tracking target, target following equipment and computer-readable storage medium
CN113379801B (en) High-altitude parabolic monitoring and positioning method based on machine vision
CN105374049B (en) Multi-corner point tracking method and device based on sparse optical flow method
CN111161309B (en) Searching and positioning method for vehicle-mounted video dynamic target
CN102436639A (en) Image acquiring method for removing image blurring and image acquiring system
CN110717934B (en) Anti-occlusion target tracking method based on STRCF
IL175632A (en) Method, system and computer product for deriving three dimensional information progressively from a streaming video sequence
CN112489091B (en) Full strapdown image seeker target tracking method based on direct-aiming template
JP2020149641A (en) Object tracking device and object tracking method
JP3850602B2 (en) Moving body detection device and moving body detection method
CN106296730A (en) A kind of Human Movement Tracking System
JP2022027464A (en) Method and device related to depth estimation of video
CN117036404A (en) Monocular thermal imaging simultaneous positioning and mapping method and system
CN113781523A (en) Football detection tracking method and device, electronic equipment and storage medium
CN115511804A (en) Offshore infrared dim target detection method based on pipeline adaptive filtering
CN115082519A (en) Airplane tracking method based on background perception correlation filtering, storage medium and electronic equipment
CN113379787A (en) Target tracking method based on 3D convolution twin neural network and template updating

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant