CN115393281A - Infrared weak and small target detection tracking method based on mask and adaptive filtering - Google Patents

Abstract

A detection and tracking method for small infrared targets based on masks and adaptive filtering. Firstly, a target foreground and background mask is generated based on the initial multi-frame (5-10 frames) infrared image sequence by using the characteristics of the fixed field of view of satellite remote sensing and reconnaissance satellites. It is used to record the characteristic information of the interference target and the real target; and then detect the new input infrared image on a basis again, and construct an adaptive filter according to the target feature information contained in the mask, which can detect different targets at different moving moments Dynamic feature extraction is performed to improve the matching and tracking accuracy of the target. The method of the present invention uses the target mask and the adaptive filter to solve the problem of noise interference and multi-target interference of infrared weak and small targets when they move at high speed, has stronger robustness, and can realize multiple infrared weak and small The high-reliability matching and tracking of targets meets the real-time and high-reliability reconnaissance requirements of sensitive targets on the planet.

Description

A Method for Detection and Tracking of Small Infrared Targets Based on Mask and Adaptive Filtering

technical field

The invention designs a method for detecting and tracking small and small infrared targets, in particular an on-orbit fast and highly reliable detection and tracking method for small and high-speed infrared targets, belonging to the field of aerospace remote sensing.

Background technique

Infrared cameras mainly observe by receiving the infrared radiation of the target itself, especially for missiles, aircraft and other high-speed and high-heat radiation targets, which make infrared target detection and tracking technology play an important role in military reconnaissance and early warning. For wide-range satellite infrared remote sensing images, these sensitive flying targets are small in size and often appear as spots or dots. The signal-to-noise ratio of the targets is very low, and they are easily disturbed by noise, clutter or clouds, and are often submerged in the background. . Therefore, for small and weak infrared targets, the motion or change characteristics of the target are usually used for detection, such as the frame difference method and the background difference method, which have the characteristics of simple, direct and fast calculation, making it more suitable for real-time reconnaissance on-orbit by satellites with limited resources early warning mission.

However, the inter-frame difference method and the background difference method are more sensitive to interference factors such as background changes caused by noise and target motion. Moreover, when the target is occluded, the algorithm can easily mistake the reappearing target as another moving target and cannot cope with occlusion changes. In addition, since the inter-frame difference method mainly uses the difference in gray value to detect moving objects, when the gray values of most of the pixels inside the target are the same, the difference image obtained by the inter-frame difference method only includes the two sides of the target object. image, but a "hole" phenomenon is generated inside the target, and it is difficult to obtain a complete outline of the target. Moreover, due to the changeable motion state of the target, it is difficult to completely eliminate various real targets from the background constructed by the background difference method.

Contents of the invention

The technical problem solved by the present invention is: to overcome the deficiencies in the prior art, and to solve the problem of high false alarms and poor trajectory correlation in the detection and tracking of small and small infrared targets on orbit (the target trajectory point is incomplete or the tracking is lost when occlusion occurs, and the two objects cannot be separated when they reappear). trajectories), and provides a real-time high-precision infrared weak and small target detection and tracking method.

Technical solution of the present invention is:

A method for detecting and tracking infrared dim and small targets based on masks and adaptive filtering, comprising the following steps:

从当前帧的图像I^t中提取获得多个疑似目标，获得由当前帧中多个疑似目标坐标组成的疑似目标坐标集

和每个疑似目标的图像切片

若当前帧的疑似目标集为空时，进入步骤6)；反之进入步骤2)；1) Use the background image corresponding to the previous frame image I ^t-1

Extract multiple suspected targets from the image I ^t of the current frame, and obtain a suspected target coordinate set composed of multiple suspected target coordinates in the current frame

and image slices for each suspected target

If the suspected target set of the current frame is empty, enter step 6); otherwise, enter step 2);

中的第k个疑似目标的坐标

在当前帧图像I^t中以

位置处的像素点为中心提取一个图像块，对图像块进行边缘检测，将不符合目标尺寸范围的疑似目标从中疑似目标坐标集

剔除，遍历疑似目标坐标集

中的所有元素，然后进入步骤3)；2) For the suspected target coordinate set obtained in step 1)

The coordinates of the kth suspected target in

In the current frame image I ^t with

The pixel at the position is taken as the center to extract an image block, and the edge detection is performed on the image block, and the suspected target that does not meet the target size range is extracted from the suspected target coordinate set

Eliminate, traverse the suspected target coordinate set

All elements in , then go to step 3);

中的第k个疑似目标的坐标

从上一帧图像I^t-1中提取出疑似目标k对应的多个候选匹配目标组成候选匹配目标集，并获得每个候选匹配目标的图片切片

3) According to the suspected target coordinate set

The coordinates of the kth suspected target in

Extract multiple candidate matching targets corresponding to the suspected target k from the previous frame image I ^t-1 to form a candidate matching target set, and obtain a picture slice of each candidate matching target

为空，则判定疑似目标k为新检测到的疑似目标，将疑似目标k的信息赋值到掩膜集合M{m×n}中，并返回步骤3)，对下一个疑似目标进行处理，直至遍历所有疑似目标后进入步骤6)；反之，则进入步骤5)；掩膜集合M{m×n}由m行n列的元素组成，m等于图像I^t中长度方向上像素的个数，n等于图像I^t中宽度方向上像素的个数；每个元素内包括用于表征图像上对应像素点的信息；4) If the candidate matching target set corresponding to a suspected target k

If it is empty, it is determined that the suspected target k is a newly detected suspected target, and the information of the suspected target k is assigned to the mask set M{m×n}, and returns to step 3), and the next suspected target is processed until Go to step 6) after traversing all suspected targets; otherwise, go to step 5); the mask set M{m×n} is composed of elements in m rows and n columns, m is equal to the number of pixels in the length direction of the image I ^t , n is equal to the number of pixels in the width direction in the image I ^t ; each element includes the information used to characterize the corresponding pixel on the image;

和步骤3)获得的候选匹配目标图片切片

确定疑似目标k与对应的J_k个候选匹配目标的匹配系数，提取出满足阈值要求的候选匹配目标作为疑似目标k的候选目标

对掩膜集合M{m×n}中与候选目标

位置对应元素的信息进行更新；若某疑似目标k不存在对应的候选目标

则判定疑似目标k为新检测到的疑似目标，将疑似目标k的信息赋值到掩膜集合M{m×n}中，然后返回步骤3)，对下一个疑似目标进行处理，直至遍历所有疑似目标后进入步骤6)；5) Using the suspected target image slice obtained in step 1)

and the candidate matching target picture slice obtained in step 3)

Determine the matching coefficient between the suspected target k and the corresponding J _k candidate matching targets, and extract the candidate matching targets that meet the threshold requirements as the candidate targets for the suspected target k

For the mask set M{m×n} and the candidate target

The information of the element corresponding to the position is updated; if there is no corresponding candidate target for a certain suspected target k

Then it is determined that the suspected target k is a newly detected suspected target, and the information of the suspected target k is assigned to the mask set M{m×n}, and then returns to step 3), and the next suspected target is processed until all suspected targets are traversed. Go to step 6 after the goal);

进行更新，获得当前帧图像I^t对应的背景图像

当前帧的疑似目标集为空时，前帧图像I^t对应的背景图像

等于上一帧图像I^t-1对应的背景图像

6) Using the mask set M{m×n}, for the background image corresponding to the previous frame image I ^t-1

Update to obtain the background image corresponding to the current frame image I ^t

When the suspected target set of the current frame is empty, the background image corresponding to the previous frame image I ^t

Equal to the background image corresponding to the previous frame image I ^t-1

7) Use the mask set M{m×n} to judge whether the suspected target is a real target or an interference target, and update the information of the corresponding element of the suspected target in the mask set M{m×n};

8) Use the information of the corresponding elements of the real target in the mask set M{m×n} to perform trajectory association and complete the complete tracking of the same target.

Preferably, the information used to characterize the corresponding pixel on the image in each element in the mask set M{m×n} is represented by 6 feature vectors, which are:

M{i}{1} is the target type of the location corresponding to the element M{i}; the target type includes: background point, corresponding to M{i}{1} value 0; interference target, corresponding to M{i}{1} value is 1; for suspected targets, the value corresponding to M{i}{1} is 2; for real targets, the value corresponding to M{i}{1} is 3; 1≤i≤m×n;

M{i}{2} is the number of the target to which the position corresponding to the element M{i} belongs;

M{i}{3} is the number of pixels in the length and width directions of the image at the latest moment of the target corresponding to the position of element M{i};

M{i}{4} is the complete trajectory set of the target corresponding to the position of element M{i}; the complete trajectory set is composed of the position information of the central element of the target corresponding to the trajectory in each frame of image;

M{i}{5} is the speed of the target at the latest moment of the position corresponding to the element M{i};

M{i}{6} is the latest movement direction of the target to which the position corresponding to the element M{i} belongs.

和每个疑似目标的图像切片

的方法，具体为：Preferably, in the step 1), a suspected target coordinate set consisting of multiple suspected target coordinates in the current frame is obtained

and image slices for each suspected target

method, specifically:

11) Solve the pixel difference between the current frame image and the previous frame image, and obtain the difference image D _f =I ^t -I ^t-1 between the current frame image and the previous frame image; simultaneously solve the pixels of the current frame and the background frame image Difference, to obtain the difference image between the current frame image and the background frame image

k＝1,2,3,…,K；Thr_f与Thr_b的取值范围为8～12；12) Reserving the difference point with an absolute value greater than Thr _f in the difference image D _f or the difference point with an absolute value greater than Thr _b in the difference image D _b as a candidate point, and finding the position of the candidate point from the current frame image Corresponding pixel points, and generate Unicom areas, and then record the central coordinate points of these Unicom areas, according to the target type of the element corresponding to the position of the central coordinate point in the mask set, remove the target type from the central coordinate point as the center of the interference target Coordinate points, take the rest of the central coordinate points as the central coordinates of the suspected target and add them to the suspected target coordinate set to obtain the suspected target coordinate set

k=1,2,3,...,K; Thr _f and Thr _b range from 8 to 12;

中的第k个疑似目标的坐标

在当前帧图像I^t中以

位置处的像素点为中心提取一个图像块，对该图像块进行边缘检测，提取出疑似目标k的边缘形状，并计算得到疑似目标k边缘形状所包围的像素构成的图像切片；13) For the obtained suspected target coordinate set

The coordinates of the kth suspected target in

In the current frame image I ^t with

The pixel at the position is the center to extract an image block, edge detection is performed on the image block, the edge shape of the suspected target k is extracted, and an image slice composed of pixels surrounded by the edge shape of the suspected target k is calculated;

中剔除，反之将步骤13)获得的图像切片作为疑似目标k的图像切片

14) If the size of the image slice formed by the pixels surrounded by the edge shape of the suspected target k obtained in step 13) is not within the range of the prior size of the target, then the suspected target k from the suspected target coordinate set

Otherwise, the image slice obtained in step 13) is used as the image slice of the suspected target k

Preferably, the size of the image block in step 13) is larger than 1.25-1.7 times of the target maximum size and smaller than 2 times of the target maximum size.

的方法，具体为：Preferably, the step 3) extracts a plurality of candidate matching targets corresponding to the suspected target k from the previous frame image I ^t-1 to form a candidate matching target set, and obtains a picture slice of each candidate matching target

method, specifically:

位置处的像素点为中心，选定一个大小为q×q的图像块作为目标候选区域；31) In the previous frame of image I ^t-1 , the same

The pixel at the position is the center, and an image block with a size of q×q is selected as the target candidate area;

,j＝1,2,3,…,J_k，其中，J_k为疑似目标k对应的目标候选区域中的候选匹配目标的个数；32) Combining with the mask set M{m×n}, from the target candidate area, select elements corresponding to the target type of the elements in the mask set that are not background targets as candidate matching targets to form a candidate matching target set

,j=1,2,3,...,J _k , where J _k is the number of candidate matching targets in the target candidate area corresponding to the suspected target k;

33) According to the information of the feature vector M{i}{3} in the mask set M{m×n}, obtain the image slice of each candidate matching target

Preferably, the value range described in step 31) is as follows:

3v·Δt/r≤q≤5v·Δt/r

Among them, r is the image resolution, v is the maximum moving speed of the target, and Δt is the time difference between adjacent frame images.

Preferably, step 5) determines the method for matching coefficient, specifically:

为自适应滤波器DF_k对疑似目标图像切片

的计算结果，

为自适应滤波器DF_k对候选匹配目标图片切片

的计算结果；

和

为候选匹配目标

在当前帧图像I^t中在行方向和列方向上的预测位置；

为第k个疑似目标的坐标。Among them, α is the filter difference coefficient, and the value range of α is 0.01～0.2;

Slice suspected object images for adaptive filter DF _k

The calculation result of

For the adaptive filter DF _k pair candidate matching target picture slice

calculation results;

and

Candidate match target

The predicted position in the row direction and the column direction in the current frame image I ^t ;

is the coordinate of the kth suspected target.

和

的确定方法，具体为：Preferably, the

and

The method of determination is as follows:

为候选匹配目标

在上一帧图像I^t-1中的位置坐标，Δt为两相邻帧之间的时间间隔，

为由掩膜集合M{m×n}获得的候选匹配目标

的运动速度，

为由掩膜集合M{m×n}获得的候选匹配目标

与图像行方向之间的夹角。in,

Candidate match target

The position coordinates in the previous frame image I ^t-1 , Δt is the time interval between two adjacent frames,

is the candidate matching target obtained from the mask set M{m×n}

speed of movement,

is the candidate matching target obtained from the mask set M{m×n}

Angle with the image row direction.

的方法，具体为：Preferably, said step 6) obtains the background image corresponding to the current frame image I ^t

method, specifically:

表示根据掩膜集合，将当前帧图像I^t中对应目标类型为背景或干扰目标的像素点的像素值保持不变，其余像点的像素值至零处理后获得的像素矩阵；

表示在当前帧图像I^t中，将疑似目标与真实目标所对应的像素块作为被替换区域，将被替换区域整体替换为邻域背景的像素块，并根据掩膜集合，将当前帧图像I^t中对应目标类型为背景与干扰目标的像素点的像素置零处理后获得的像素矩阵；邻域背景的像素块中每个像素的值等于被替换区域外侧上下左右四个方向上等大小像素块中对应像素的均值。Among them, λ is the background update coefficient, and the value range of λ is 0.7-0.9;

Indicates that according to the mask set, the pixel value of the pixel point corresponding to the target type in the current frame image I ^t is kept unchanged, and the pixel value of the remaining image points is zero-processed to obtain the pixel matrix;

Indicates that in the current frame image I ^t , the pixel block corresponding to the suspected target and the real target is used as the replaced area, and the replaced area is replaced with the pixel block of the neighborhood background as a whole, and according to the mask set, the current frame image I The pixel matrix obtained by zeroing the pixels corresponding to the target type in ^t as the background and the interfering target pixels; the value of each pixel in the pixel block of the neighborhood background is equal to the pixels of the same size in the four directions of the upper, lower, left, and right outside the replaced area Mean value of corresponding pixels in the block.

Preferably, the step 7) is a method for judging whether the suspected target is a real target or an interference target, specifically: for an element whose feature vector target type is a suspected target in the mask set, when the number of times the target appears in the entire image sequence exceeds Num _d or when the update of the target track exceeds Numo, calculate the length of the complete track set of the target to which the corresponding position of the element belongs, if the track length exceeds Thr _track ; if so, determine the target type of the element as a real target, otherwise it will be It is judged as an interference target; the value range of Num _d is equal to the number of image frames corresponding to 5-10s; the value range of Numo is equal to the number of image frames corresponding to 10-15s; the value range of Thr _track is 3-5.

Preferably, said step 8) carries out the method for trajectory association, specifically:

When the track corresponding to the element whose target type is the real target in the mask set stops updating for more than Numo frames, it is determined that the real target movement is completed, and the target number and track are added to the track set Track;

When a new trajectory is added to the trajectory set, the trajectory prediction is performed according to the movement speed and direction of the end trajectory point of the known trajectory in the trajectory set, and the predicted position of the known trajectory in the corresponding new trajectory is obtained. Compare the speed and direction of the target corresponding to the old and new trajectories, so as to complete the complete tracking of the target; the value range of Numo is equal to the number of image frames corresponding to 10-15s.

中每个像素的像素值等于初始获得的N帧图像中对应位置像素点的平均值，N的取值范围为5～10。Preferably: an initial background image

The pixel value of each pixel in is equal to the average value of the corresponding pixel points in the initially obtained N frames of images, and the value range of N is 5-10.

The advantage of the present invention compared with prior art is:

(1) The present invention records various types of target information by generating a target mask, thereby greatly reducing the influence of interfering targets during detection and improving detection accuracy.

(2) Aiming at the characteristics of dynamic changes of sensitive targets during flight, the present invention constructs an adaptive filter, which can extract the feature information of the target at all times, and increases the matching accuracy.

(3) The present invention correlates the trajectories of all real targets, which can solve the problem of target occlusion to a certain extent, and can generate complete situation information of the target in combination with the target features included in the mask.

Description of drawings

Fig. 1 is a flow chart of the method of the present invention;

Fig. 2 is a diagram of a method for generating an adaptive filter according to the present invention.

Detailed ways

For satellite on-orbit infrared reconnaissance and early warning systems, the reliability and timeliness of intelligence information is the key to battlefield support, especially for high-speed flying targets. Only real-time or quasi-real-time detection and tracking can play a real role in space-based reconnaissance and early warning systems. Therefore, the frame difference method and the background difference method are used for joint detection to realize the rapid processing of massive infrared image sequences. However, this type of method is susceptible to interference from noise, imaging bad pixels, imaging bright spots, and moving clouds. These interferences are significantly different from the surrounding environment, similar to real infrared weak and small targets, and often exist in the form of spots or points in the field of view. As a result, false alarms are high when detecting real targets, and tracking loss will also occur when the target is occluded. However, different from real targets, the shape features and motion characteristics of these interfering targets are different from real targets. Therefore, based on these prior knowledge, the present invention uses image sequences to generate a mask information, and extracts The complete contour features of various targets are used to support the detection, and at the same time, various important feature information during the detection and tracking process are stored and analyzed in real time, and the information is used to build an adaptive filter to further confirm the real target and greatly reduce false alarms. Improve the reliability of intelligence information while ensuring real-time processing.

As shown in Figure 1, the specific implementation steps of the inventive method are as follows:

(1) Pre-generate an initial mask set M{m×n} according to the acquired remote sensing infrared image size m×n, where m is the number of pixels in each row of the remote sensing infrared image, and n is each column of the remote sensing infrared image The number of pixels, that is, M{m×n} contains all observation point information in the field of view observed by the camera. For any element M{i}, 1≤i≤m×n, the element can be found according to its index i The corresponding position in the image observation field of view. Each element M{i} in the mask set M{m×n} corresponds to 6 feature vectors M{i}{j}, 1≤i≤m×n, 1≤j≤6.

in,

M{i}{1} is the target type of the corresponding position of the element; the target type includes: background point, the corresponding M{i}{1} value is 0; interference target, the corresponding M{i}{1} value is 1; For suspected targets, the corresponding M{i}{1} value is 2; for real targets, the corresponding M{i}{1} value is 3; only the center point element of the target slice has a value, and the rest are empty. Suspected targets include: interference targets and real targets. For background points, M{i} contains 1 feature vector, and the rest of the feature vectors are empty. For suspected targets, jamming targets and real targets, M{i} contains 6 feature vectors.

M{i}{2} is the number of the target to which the corresponding position of the element belongs;

M{i}{3} is the size feature of the target at the latest moment corresponding to the element's position (that is, the number of pixels in the length and width directions of the remote sensing infrared image);

M{i}{4} is the complete trajectory set of the target corresponding to the position of the element; the complete trajectory set is composed of the position information of the central element of the target corresponding to the trajectory in each frame of image;

M{i}{5} is the speed of the latest moment of the target to which the position corresponding to the element belongs;

M{i}{6} is the movement direction of the target at the latest moment to which the corresponding position of the element belongs.

The initial value of the feature vector of each element in the initial mask set is set to 0, that is, it is assumed that the initial target type of all pixels is the background point.

初始的背景图像中每个像素的像素值等于N帧遥感红外图像中对应位置像素点的平均值。(2) According to the initially obtained N frames of remote sensing infrared images (N is generally 5 to 10 images) {I ¹ ,I ² ,...,I ^N }, sum and average the pixels at each corresponding position processing, get the initial background image

The pixel value of each pixel in the initial background image is equal to the average value of the corresponding pixel points in the N frames of remote sensing infrared images.

若当前帧的疑似目标集为空时，重复步骤(3)对下一帧的遥感红外图像进行疑似目标提取，直至疑似目标坐标集不为空时，进入步骤(4)；若当前帧的疑似目标集为空时，则当前帧的背景图像

与上一帧的背景图像相同；(3) Through the initial background image, extract the suspected target from the remote sensing infrared image of the current frame, and obtain the suspected target coordinate set composed of the suspected target coordinates in the current frame and the image slice of each suspected target

If the suspected target set of the current frame is empty, repeat step (3) to extract the suspected target from the remote sensing infrared image of the next frame until the suspected target coordinate set is not empty, enter step (4); When the target set is empty, the background image of the current frame

The same background image as the previous frame;

背景帧图像的差异性来对目标进行检测，分别求解当前帧图像与上一帧图像的像素差值，获得当前帧图像与上一帧图像的差值图像D_f＝I^t-I^t-1；同时求解当前帧与背景帧图像的像素差值，获得当前帧图像与背景帧图像的差值图像

差值图像中的每个点作为差值点，当前帧图像与上一帧图像的差值图像中每个差值点的像素值等于当前帧图像对应像素点的值与上一帧图像对应像素点的值之间的差值。当前帧图像与背景帧图像的差值图像中每个差值点的像素值等于当前帧图像对应像素点的值与背景帧图像对应像素点的值之间的差值。(31) Using the difference between the current frame image and the previous frame image, and the current frame image and the previous frame obtained

The difference of the background frame image is used to detect the target, and the pixel difference between the current frame image and the previous frame image is solved respectively to obtain the difference image D _f =I ^t -I ^t-1 of the current frame image and the previous frame image ; Simultaneously solve the pixel difference between the current frame image and the background frame image, and obtain the difference image between the current frame image and the background frame image

Each point in the difference image is used as a difference point, and the pixel value of each difference point in the difference image between the current frame image and the previous frame image is equal to the value of the corresponding pixel point of the current frame image and the corresponding pixel of the previous frame image The difference between the values of the points. The pixel value of each difference point in the difference image between the current frame image and the background frame image is equal to the difference between the value of the corresponding pixel point of the current frame image and the value of the corresponding pixel point of the background frame image.

k＝1,2,3,…,K，其中，

表示第k个疑似目标中心在观测视野中处于第

列，第

行的像素点(位置)。(初始获得疑似目标坐标集时，疑似目标坐标集中每个元素的所属目标类型均为疑似目标，即

(32) Keep the difference points whose absolute value is greater than Thr _f in the difference image D _f or the difference points whose absolute value is greater than Thr _b in the difference image D _b (according to statistics, the general value range of Thr _f and Thr _b 8 to 12) as candidate points, find the pixel points corresponding to the candidate point positions from the current frame image, and use the Unicom domain marking method (the existing mature method) to connect these pixel points to generate K Unicom regions, and then respectively Record the central coordinate points of these connected areas, according to the target type of the element corresponding to the position of the central coordinate point in the mask set, remove the central coordinate points whose target type is the interference target from the central coordinate points, and use the rest of the central coordinate points as the suspected target Center coordinates and add the suspected target coordinate set to get the suspected target coordinate set

k=1,2,3,...,K, where,

Indicates that the center of the kth suspected target is in the observation field of view

column, No.

The pixel point (position) of the row. (When the suspected target coordinate set is initially obtained, the target type of each element in the suspected target coordinate set is a suspected target, that is

在当前帧图像I^t中以

位置处的像素点为中心提取一个图像块，图像块的尺寸大于目标最大尺寸的1.25～1.7倍，小于目标最大尺寸的2倍。(选取原因是由于红外遥感观测卫星轨道固定，其分辨率也是可确定的，因此可以利用这一先验知识统计得到目标在观测图像中的尺寸范围)，并利用“sobel”索贝尔算子来对该图像块进行边缘检测，提取出疑似目标k的边缘形状，并可计算得到疑似目标大小及边缘形状所包围的像素构成疑似目标的图像切片

如果疑似目标k的图像切片的尺寸不在统计得到的目标对应观测图像中尺寸范围内，则将目标k从疑似目标坐标集

中剔除，遍历疑似目标坐标集中的所有元素，然后进入步骤(4)；(33) For the coordinates of the kth suspected target in the suspected target coordinate set obtained in step (32)

In the current frame image I ^t with

The pixel at the position is taken as the center to extract an image block, and the size of the image block is 1.25-1.7 times larger than the maximum size of the target, and smaller than 2 times the maximum size of the target. (The reason for the selection is that the orbit of the infrared remote sensing observation satellite is fixed, and its resolution can also be determined, so this prior knowledge can be used to statistically obtain the size range of the target in the observation image), and the "sobel" Sobel operator is used to Perform edge detection on the image block, extract the edge shape of the suspected target k, and calculate the suspected target size and the pixels surrounded by the edge shape to form the image slice of the suspected target

If the size of the image slice of the suspected target k is not within the size range of the corresponding observed image of the target obtained by statistics, then the target k is removed from the suspected target coordinate set

Eliminate, traverse all elements in the suspected target coordinate set, and then enter step (4);

(4) According to the suspected target coordinate set obtained in step (3), extract the candidate matching target set and the candidate matching target picture slice corresponding to each suspected target from the previous frame image

位置处的像素点为中心，选定一个大小为q×q的图像块(图像块大小q主要结合图像分辨率r和目标最大运动速度v进行判断，3v·Δt/r≤q≤5v·Δt/r，其中Δt为相邻帧图像之间的时间差)作为目标候选区域；(41) For the suspected target k, in the previous frame of image I ^t-1 , the same

The pixel point at the position is the center, and an image block with a size of q×q is selected (the image block size q is mainly judged by combining the image resolution r and the maximum moving speed v of the target, 3v·Δt/r≤q≤5v·Δt /r, where Δt is the time difference between adjacent frame images) as the target candidate area;

,j＝1,2,3,…,J_k，其中，J_k为疑似目标k对应的目标候选区域中的候选匹配目标数。根据掩模集合M{m×n}中特征向量M{i}{3}的信息，获得每个候选匹配目标的图像切片

(42) Combined with the mask set M{m×n}, from the target candidate area, select the points corresponding to the target type of the elements in the mask set that are not background targets, as candidate matching targets, and form a candidate matching target set

,j=1,2,3,...,J _k , where J _k is the number of candidate matching targets in the target candidate area corresponding to the suspected target k. According to the information of the feature vector M{i}{3} in the mask set M{m×n}, the image slice of each candidate matching target is obtained

(43) Repeat steps (41)-(42) K times to obtain a candidate matching target set for each suspected target.

为空，则说明当前帧中的疑似目标k为新检测到的疑似目标(对应

值为2)，将新编号赋值到掩膜集合

中，并将尺寸特征和目标位置分别赋值到

和

完成该帧目标检测，进入步骤(7)；(5) If the candidate matching target set corresponding to a suspected target k

is empty, it means that the suspected target k in the current frame is a newly detected suspected target (corresponding to

The value is 2), assign the new number to the mask set

, and assign the size feature and target position to

and

Complete the frame target detection, enter step (7);

If the candidate matching target set {T _P _k(j)} is not empty, go to step (6);

和步骤(4)获得的候选匹配目标图片切片

分别确定每个疑似目标与对应的J_k个候选匹配目标的匹配系数，提取出满足阈值要求的候选匹配目标作为疑似目标k的候选目标

根据候选目标

获得疑似目标k的运动方向及速度，并对掩模中候选目标

对应的元素的特征向量进行更新；若某疑似目标k不存在对应的候选目标

则定义疑似目标k为新检测到的疑似目标(即

)，更新掩模中对应位置的元素的特征向量；(6) Using the suspected target image slice obtained in step (3)

and the candidate matching target picture slice obtained in step (4)

Determine the matching coefficients of each suspected target and the corresponding J _k candidate matching targets, and extract the candidate matching targets that meet the threshold requirements as the candidate targets of suspected target k

According to the candidate target

Obtain the movement direction and speed of the suspected target k, and compare the candidate targets in the mask

The feature vector of the corresponding element is updated; if there is no corresponding candidate target for a suspected target k

Then define the suspected target k as the newly detected suspected target (ie

), update the feature vector of the element at the corresponding position in the mask;

和步骤(4)获得的候选匹配目标图片切片

利用一个与目标大小相同的自适应滤波器DF_k对疑似目标k与疑似目标k对应的候选匹配目标集

中所有候选匹配目标进行匹配，匹配系数的计算如下所示：(61) Using the suspected target image slice obtained in step (3)

and the candidate matching target picture slice obtained in step (4)

Use an adaptive filter DF _k with the same size as the target to pair the candidate matching target set corresponding to the suspected target k and the suspected target k

Match all candidate matching targets in , and the calculation of the matching coefficient is as follows:

为自适应滤波器DF_k对疑似目标图像切片

的计算结果，

为自适应滤波器DF_k对候选匹配目标图片切片

的计算结果；

为疑似目标k在图像I^t中所占据的所有像素构成的矩阵，即疑似目标图像切片(切片主要利用步骤33中的边缘检测结果进行提取)，

为第j个候选匹配目标

在图像I^t-1中的候选匹配目标图像切片(表示目标所占据的所有像素构成的矩阵，并等比例缩放到与图像I^t相同大小)，

和

为候选匹配目标

在当前帧图像I^t中在行方向和列方向上的预测位置，计算方法如下：In the above formula, α is the filter difference coefficient (α is generally 0.01 to 0.2, which is negatively correlated with the dynamic range of image pixel values),

Slice suspected object images for adaptive filter DF _k

The calculation result of

For the adaptive filter DF _k pair candidate matching target picture slice

calculation results;

Be the matrix formed by all pixels occupied by the suspected target k in the image I ^t , i.e. the suspected target image slice (the slice mainly utilizes the edge detection result in step 33 to extract),

matches the target for the jth candidate

Candidate matching target image slice in image I ^t-1 (representing a matrix composed of all pixels occupied by the target, and scaled to the same size as image I ^t ),

and

Candidate match target

The prediction position in the row direction and the column direction in the current frame image I ^t is calculated as follows:

为候选匹配目标

在图像I^t-1中的位置，Δt为两相邻帧之间的时间间隔，

和

分别为候选匹配目标

的运动速度和运动方向(方向为运动方向与行方向x之间的夹角)。In the above formula,

Candidate match target

At the position in the image I ^t-1 , Δt is the time interval between two adjacent frames,

and

Candidate matching targets

The movement speed and direction of movement (the direction is the angle between the movement direction and the row direction x).

When constructing a dynamic filter, since the imaging size of the target in the infrared remote sensing image generally changes continuously in the range of 1×1 to 10×10, and the high-speed moving target has a high temperature of the tail flame, the infrared camera also needs to perform imaging. Part of the tail flame will be recorded, resulting in the formation of an elliptical bright spot with direction in the image when the target is moving at high speed, and the axis direction of the bright spot is similar to the direction of target movement. For the dynamically changing weak infrared targets, if a fixed filter is used for feature extraction, it is difficult to achieve stable capture of the target at all times. Therefore _, the present invention designs a rectangular adaptive filter with an angle according to the target feature. The filter is based on a 5×5 filter DF _base , and _is generated from Adaptive filter DF _k , the length of DF _k is l _k +2, the width is w _k +2, its filter coefficients are generated by interpolation or sampling of DF _base coefficients, and its direction is the same as the axis direction of target k, as shown in Figure 2 shown.

之间差异越大，表示匹配系数

越大；

越小，表示疑似目标k与候选匹配目标

越相似，在所有满足

(Thr_R为一个统计量，根据任务不同，设计也不同，在本例中为2～5)的候选匹配目标集中，选择匹配系数值最小的候选匹配目标为k所匹配的候选目标

即认为候选目标

与疑似目标k为同一目标，可通过当前帧与上一帧位置的变化计算出目标k的运动方向及速度，在此基础上将目标的类型、编号保持不变、目标尺寸、轨迹集、速度、运动方向特征于掩模

处更新，并消除掩模

处的历史特征信息；若不存在匹配系数小于Thr_R的目标，同样认为该目标为新出现目标，则其建立新编号，并在掩模

处将目标的编号、尺寸、轨迹进行更新。(62) It can be seen that the suspected target k and the candidate matching target

The greater the difference, the matching coefficient

bigger;

The smaller the value, it means that the suspected target k matches the candidate target

The more similar, in all satisfies

(Thr _R is a statistic, depending on the task, the design is also different, in this example, it is 2 to 5) in the candidate matching target set, select the candidate matching target with the smallest matching coefficient value as the candidate target matched by k

candidate target

It is the same target as the suspected target k, and the movement direction and speed of the target k can be calculated through the position change between the current frame and the previous frame. On this basis, the type, number, target size, trajectory set, and speed of the target remain unchanged. , the direction of motion is characterized by the mask

update and remove the mask

The historical feature information at the place; if there is no target with a matching coefficient less than Thr _R , it is also considered as a new target, then it will create a new number, and in the mask

Update the number, size and trajectory of the target.

中疑似目标及真实目标对应的像素值更新，更新原则为：(7) After completing the current frame image I ^t detection, use the mask M to complete the background image corresponding to the current frame

The pixel values corresponding to the suspected target and the real target are updated, and the update principle is:

表示根据掩膜集合，从当前帧图像I^t中提取出目标类型为背景或干扰目标的像素点的像素值保持不变，其余像点(疑似目标与真实目标像素点)的像素值至零处理后获得的像素矩阵(背景、干扰目标、疑似目标以及真实目标所对应的像素点可以根据M{i}{1}与M{i}{3}的值得到)。

表示在当前帧图像I^t基础上，将疑似目标与真实目标所在对应的像素块整体替换为该目标所在邻域背景(邻域背景表示目标在图像I^t中所在位置上下左右四个方向的背景像素)像素块，邻域背景像素块的大小取决于对应目标的大小，由目标切片周围上下左右四个相同大小的背景像素块平均加权得到，并将背景与干扰目标所对应的像素置零。In the above formula, λ is the background update coefficient, generally between (0.7 and 0.9),

Indicates that according to the mask set, the pixel values of the pixels whose target type is background or interference target extracted from the current frame image I ^t remain unchanged, and the pixel values of the remaining pixels (suspected targets and real target pixels) are processed to zero The obtained pixel matrix (the pixels corresponding to the background, interference target, suspected target and real target can be obtained according to the values of M{i}{1} and M{i}{3}).

Indicates that on the basis of the current frame image I ^t , the pixel blocks corresponding to the suspected target and the real target are replaced as a whole with the background of the neighborhood where the target is located (neighborhood background indicates the background of the target in the image I ^t in the four directions of up, down, left, and right pixel) pixel block, the size of the neighborhood background pixel block depends on the size of the corresponding target, which is obtained by the average weighting of four background pixel blocks of the same size around the target slice, and the pixels corresponding to the background and the interference target are set to zero.

处更新目标类型、编号、外形特征、目标速度、运动方向；若为干扰目标，则只保留类型信息，特征向量M{i}{2}～M{i}{6}的值均为空，即：(8) For elements in the mask set whose feature vector target type is a suspected target, when the number of times the target appears in the entire image sequence exceeds Num _d (generally the number of image frames corresponding to imaging 5-10s) or the target trajectory stops updating When it exceeds Numo (generally the number of image frames corresponding to 10-15s of imaging, that is, if the target has no new trajectory in the image sequence of 10-15s continuously, it is judged that the target has disappeared), calculate its trajectory length, if the trajectory If the length exceeds Thr _track (Thr _track ranges from 3 to 5, in the embodiment of the present invention it is 3, that is, the target moves more than 3 pixels), then the target type of the element is determined as a real target, otherwise it is determined as interference target, and update the mask M, the update principle is: if it is a real target, then

Update the target type, serial number, appearance feature, target speed, and direction of motion; if it is an interference target, only the type information will be kept, and the values of the feature vectors M{i}{2}~M{i}{6} are all empty, which is:

为第l条轨迹在第t帧图像中的坐标位置，t_b(l)为第l条轨迹的起始出现帧，t_e(l)为第l条轨迹的末尾出现帧。当新轨迹Track(l+1)加入到轨迹集中时，根据轨迹中结束轨迹点的运动速度和方向进行轨迹预测，得到其在相应新轨迹中的预测位置，通过与新轨迹的起始位置和新旧两个轨迹对应目标的运动速度及方向进行比对，从而完成目标的完整跟踪。(9) For the element O _T in the mask set whose eigenvector target type is a real target, when the target track stops updating for more than Numo frames, it is judged that the target movement is completed, and the target number and track are added to the track set Track. for new track

is the coordinate position of the lth track in the image frame t, t _b (l) is the initial frame of the lth track, and t _e (l) is the end frame of the lth track. When the new trajectory Track(l+1) is added to the trajectory set, the trajectory prediction is performed according to the motion speed and direction of the end trajectory point in the trajectory, and its predicted position in the corresponding new trajectory is obtained. The old and new trajectories are compared against the moving speed and direction of the target, so as to complete the complete tracking of the target.

Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention, and any person skilled in the art can use the methods disclosed above and technical content to analyze the present invention without departing from the spirit and scope of the present invention. Possible changes and modifications are made in the technical solution. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention, which do not depart from the content of the technical solution of the present invention, all belong to the technical solution of the present invention. protected range. In the case of no conflict, the embodiments of the present application and the technical features in the embodiments may be combined with each other.

The content that is not described in detail in the description of the present invention belongs to the well-known technology of those skilled in the art.

Claims (12)

1. A method for detecting and tracking infrared dim targets based on masks and adaptive filtering is characterized by comprising the following steps:

1) Using the last frame image I ^t-1 Corresponding background image

From picture I of the current frame ^t Extracting to obtain multiple suspected targets, and obtaining a suspected target coordinate set composed of multiple suspected target coordinates in the current frame

And image slices of each suspected object

If the suspected target set of the current frame is empty, entering step 6); otherwise, entering the step 2);

2) For the suspected target coordinate set obtained in step 1)

Coordinates of the kth suspected target in (1)

In the current frame image I ^t To Chinese

Extracting an image block by taking a pixel point at the position as the center, carrying out edge detection on the image block, and carrying out suspected target coordinate set on a suspected target which does not conform to the size range of the target

Removing and traversing suspected target coordinate set

Then go to step 3);

3) According to the suspected target coordinate set

Coordinates of the kth suspected target in (1)

From the last frame image I ^t-1 Extracting a plurality of candidate matching targets corresponding to the suspected target k to form a candidate matching target set, and obtaining a picture slice of each candidate matching target

4) If a candidate matching target set corresponding to a suspected target k

If the suspected target k is empty, judging the suspected target k as a newly detected suspected target, assigning the information of the suspected target k to a mask set M { M × n }, returning to the step 3), processing the next suspected target until all the suspected targets are traversed, and entering the step 6); otherwise, entering the step 5); the set of masks M { M × n } is composed of M rows and n columns of elements, M being equal to the image I ^t Number of pixels in the medium-length direction, n being equal to image I ^t The number of pixels in the medium width direction; each element comprises information used for representing a corresponding pixel point on the image;

5) By using stepsStep 1) obtaining a suspected target image slice

And 3) obtaining candidate matching target picture slices

Determining a suspected target k and a corresponding J _k Extracting the candidate matching target meeting the threshold requirement as the candidate target of the suspected target k according to the matching coefficient of each candidate matching target

For the neutralization candidate target in the mask set M { M × n }

Updating the information of the elements corresponding to the positions; if some suspected target k does not have a corresponding candidate target

Judging the suspected target k as a newly detected suspected target, assigning the information of the suspected target k to a mask set M { M × n }, then returning to the step 3), processing the next suspected target until all the suspected targets are traversed, and then entering the step 6);

6) Using mask set M { M × n }, for the previous frame image I ^t-1 Corresponding background image

Updating to obtain current frame image I ^t Corresponding background image

When the suspected target set of the current frame is empty, the image I of the previous frame ^t Corresponding background image

Is equal toOne frame image I ^t-1 Corresponding background image

7) Judging whether the suspected target is a real target or an interference target by using the mask set M { M multiplied by n }, and updating information of elements corresponding to the suspected target in the mask set M { M multiplied by n };

8) And (4) carrying out track association by using the information of the corresponding elements of the real target in the mask set M { M multiplied by n } to complete the complete tracking of the same target.

2. The method for detecting and tracking the infrared weak and small target based on the mask and the adaptive filtering as claimed in claim 1, wherein the information used for characterizing the corresponding pixel points on the image in each element in the mask set M { M × n } is represented by 6 eigenvectors, which are respectively:

m { i } {1} is the target type of the corresponding position of the element M { i }; the object types include: a background point corresponding to a value of M { i } {1} of 0; an interference target, corresponding to a value of 1 for M { i } {1 }; a suspected target, corresponding to a value of M { i } {1} of 2; the real target, corresponding to M { i } {1} value is 3; i is more than or equal to 1 and less than or equal to mxn;

m { i } {2} is the number of the target to which the corresponding position of the element M { i } belongs;

m { i } {3} is the number of pixels in the length and width directions of the image at the latest moment of the target to which the corresponding position of the element M { i } belongs;

m { i } {4} is a complete track set of the target to which the corresponding position of the element M { i } belongs; the complete track set consists of position information of central elements of the track corresponding to the target in each frame of image;

m { i } {5} is the speed of the target latest moment to which the corresponding position of the element M { i } belongs;

m { i } {6} is the motion direction of the object at the latest moment to which the corresponding position of the element M { i } belongs.

3. The method for detecting and tracking infrared dim target based on mask and adaptive filtering as claimed in claim 2, wherein said step 1) is obtained from the current frameA suspected target coordinate set composed of multiple suspected target coordinates

And image slices of each suspected object

The method specifically comprises the following steps:

11 Solving the pixel difference between the current frame image and the previous frame image to obtain a difference image D between the current frame image and the previous frame image _f ＝I ^t -I ^t-1 (ii) a Simultaneously solving the pixel difference value of the current frame image and the background frame image to obtain a difference value image of the current frame image and the background frame image

12 Differential image D) _f Medium absolute value is greater than Thr _f Difference point or difference image D of _b Medium absolute value is greater than Thr _b The difference points are reserved as candidate points, pixel points corresponding to the candidate points are found from the current frame image, communication areas are generated, then central coordinate points of the communication areas are recorded respectively, according to the target type of elements corresponding to the central coordinate points in the mask set, the central coordinate points with the target type as an interference target are removed from the central coordinate points, the rest central coordinate points are used as the central coordinates of the suspected target and are added into the suspected target coordinate set, and the suspected target coordinate set is obtained

Thr _f And Thr _b The value range of (A) is 8-12;

13 For the obtained set of coordinates of suspected object

Coordinates of the kth suspected target in (1)

At the current frame image I ^t To Chinese

Extracting an image block by taking a pixel point at the position as a center, carrying out edge detection on the image block, extracting the edge shape of the suspected target k, and calculating to obtain an image slice formed by pixels surrounded by the edge shape of the suspected target k;

14 ) if the size of the image slice formed by the pixels surrounded by the edge shape of the suspected target k obtained in the step 13) is not in the prior size range of the target, the suspected target k is selected from the suspected target coordinate set

Removing the image slices obtained in the step 13) as the image slices of the suspected target k

4. The method for detecting and tracking infrared weak and small objects based on the mask and adaptive filtering as claimed in claim 3, wherein the size of the image block in step 13) is greater than 1.25-1.7 times the maximum size of the object and less than 2 times the maximum size of the object.

5. The method for detecting and tracking infrared weak and small target based on mask and adaptive filtering as claimed in claim 2, wherein said step 3) is performed from the previous frame of image I ^t-1 Extracting a plurality of candidate matching targets corresponding to the suspected target k to form a candidate matching target set, and obtaining a picture slice of each candidate matching target

The method specifically comprises the following steps:

31 In the previous frame of image I) ^t-1 The same as in (1)

Selecting an image block with the size of qxq as a target candidate region by taking a pixel point at the position as a center;

32 In combination with the mask set M { M × n }, from the target candidate region, selecting elements whose target types corresponding to the elements in the mask set are not background targets as candidate matching targets to form a candidate matching target set

Wherein, J _k The number of candidate matching targets in a target candidate area corresponding to the suspected target k;

33 Obtaining an image slice for each candidate matching object based on information of the feature vector M { i } {3} in the mask set M { M × n }

6. The method for detecting and tracking the infrared dim target based on the mask and the adaptive filtering as claimed in claim 5, wherein the value range in step 31) is as follows:

3v·Δt/r≤q≤5v·Δt/r

wherein r is the image resolution, v is the maximum motion speed of the target, and Δ t is the time difference between adjacent frame images.

7. The method for detecting and tracking the infrared weak and small target based on the mask and the adaptive filtering as claimed in claim 2, wherein the method for determining the matching coefficient in step 5) specifically comprises:

wherein alpha is a filter difference coefficient, and the value range of alpha is 0.01-0.2;

for adaptive filters DF _k Slicing the suspected target image

As a result of the calculation of (a),

for adaptive filters DF _k Slicing candidate matching target picture

The calculation result of (2);

and

matching targets for candidates

In the current frame image I ^t In the row direction and in the column direction;

the coordinates of the kth suspected target.

8. The method as claimed in claim 7, wherein the method for detecting and tracking infrared dim target based on mask and adaptive filtering is characterized in that

And

the determination method specifically comprises the following steps:

wherein,

matching targets for candidates

Last frame image I ^t-1 At, the time interval between two adjacent frames,

for candidate matching targets obtained from the set of masks M { M × n }

The speed of movement of (a) is,

for candidate matching targets obtained from the set of masks M { M × n }

The angle to the image line direction.

9. The method for detecting and tracking infrared dim target based on mask and adaptive filtering as claimed in claim 2, wherein said step 6) obtains current frame image I ^t Corresponding background image

The method specifically comprises the following steps:

wherein, lambda is a background updating coefficient, and the value range of lambda is 0.7-0.9;

representing the current frame image I according to the mask set ^t The pixel values of the pixel points of which the corresponding target types are background or interference targets are kept unchanged, and the pixel values of the other pixel points are processed to zero to obtain a pixel matrix;

is shown in the current frame image I ^t Taking pixel blocks corresponding to the suspected target and the real target as a replaced area, replacing the whole replaced area with the pixel block of the neighborhood background, and according to a mask set, replacing the current frame image I ^t The pixel matrix is obtained after the zero setting processing of the pixels of the pixel points of which the corresponding target types are the background and the interference target; the value of each pixel in the pixel blocks of the neighborhood background is equal to the average value of the corresponding pixels in the pixel blocks with equal size in four directions of upper, lower, left and right outside the replaced area.

10. The method for detecting and tracking the infrared weak and small target based on the mask and the adaptive filtering according to any one of claims 1 to 9, wherein the step 7) is a method for judging whether the suspected target is a real target or an interference target, specifically: for the elements of which the feature vector target type in the mask set is a suspected target, when the number of times that the target appears in the whole image sequence exceeds Num _d Or the target track stops updating beyond Num _o Then, calculating the length of the complete track set of the target of the corresponding position of the element, and if the track length exceeds Thr _track (ii) a If so, judging the target type of the element as a real target, otherwise, judging the element as an interference target; num _d The value range of the image frames is equal to the number of the image frames corresponding to 5 to 10 s; num _o The value range of (a) is equal to the number of image frames corresponding to 10-15 s; thr (Thr) _track The value range is 3-5.

11. The method for detecting and tracking the infrared dim target based on the mask and the adaptive filtering according to any one of claims 1 to 9, wherein the step 8) is a method for performing track association, and specifically comprises the following steps:

when the track corresponding to the element with the target type being the real target in the mask set stops updating and exceeds Num _o When the frame is in use, judging that the real target motion is finished, and adding a target number and a Track into a Track set Track;

when a new track is added into the track set, the track is predicted according to the movement speed and direction of the ending track point of the known track in the track set to obtain the predicted position of the known track in the corresponding new track, and the predicted position is compared with the movement speed and direction of the target corresponding to the initial position of the new track and the new and old tracks to complete the complete tracking of the target; num _o The value range of (a) is equal to the number of image frames corresponding to 10-15 s.

12. The method for detecting and tracking the infrared dim target based on the mask and the adaptive filtering according to any one of claims 1 to 9, characterized in that: initial background image

The pixel value of each pixel in the image is equal to the average value of pixel points at corresponding positions in the initially obtained N frames of images, and the value range of N is 5-10.

Images