CN114372936A

CN114372936A - Infrared dim target detection method based on image inpainting technology

Info

Publication number: CN114372936A
Application number: CN202210022778.7A
Authority: CN
Inventors: 安玮; 卢德勇; 林再平; 盛卫东; 龙云利; 杨俊刚; 李淼; 凌强
Original assignee: National University of Defense Technology
Current assignee: National University of Defense Technology
Priority date: 2022-01-10
Filing date: 2022-01-10
Publication date: 2022-04-19

Abstract

The application relates to an infrared small and weak target detection method and device based on an image inpainting technology. The method comprises the following steps: extracting a current frame infrared image to be processed from an infrared video image, respectively detecting weak and small targets through a spatial domain algorithm and a time domain algorithm, merging suspected weak and small target areas obtained through the two ways into a candidate weak and small target area, digging out the candidate target area in an original infrared image, recovering and estimating pixels where the candidate target is located by using an image repairing technology, reconstructing an infrared background image of the current frame, then subtracting the infrared background image of the current frame from the infrared background image of the current frame to obtain a target saliency map, and segmenting the target saliency map to obtain the weak and small targets. The method effectively improves the detection rate of the infrared dim target, reduces false alarms caused by factors such as noise, clutter and edges, has low algorithm complexity and small calculation amount, and is easy to meet the real-time requirement in practical engineering application.

Description

Infrared dim target detection method based on image inpainting technology

Technical Field

The application relates to the field of infrared detection, in particular to an infrared dim target detection method and device based on an image repairing technology.

Background

An infrared searching and tracking system adopts an important passive and passive detection technology, has strong stealth and anti-interference capability, and targets often appear in the form of weak targets in an infrared image, only comprise a few pixels to dozens of pixels, lack shape and texture information, have weak target energy and low signal-to-noise ratio, and can be submerged in strong clutter and complex backgrounds.

The currently proposed infrared weak and small target detection method can be divided into the following steps according to the main technical approaches: the method is based on a traditional filtering method, a human visual system method, a low-rank sparse decomposition method and a learning method.

Aiming at the detection of weak and small targets in an infrared video image, the main defects of the prior art are low reliability and detection rate, high false alarm rate, low real-time property and the like, and a better infrared weak and small target detection technology still needs to be further researched.

Disclosure of Invention

Based on this, it is necessary to provide a method, an apparatus, a computer device and a storage medium for detecting a small and weak target based on an image patching technology, which can further improve the accuracy and the real-time performance of detecting a small and weak target in an infrared video image.

An infrared small and weak target detection method based on an image inpainting technology, the method comprising:

acquiring a current frame infrared image in an infrared video image, and obtaining a first detection result of a suspected small target through a spatial domain filtering algorithm and a threshold segmentation method according to the current frame infrared image; the infrared video image comprises a weak and small target to be detected;

acquiring a previous frame infrared image of the current frame infrared image, and obtaining a second detection result of the suspected weak small target through a time domain difference algorithm and a threshold segmentation algorithm according to the previous frame infrared image and the current frame infrared image;

combining the first detection result and the second detection result to obtain a candidate target area of the weak and small target, digging the candidate target area from the current frame infrared image, and reconstructing the candidate target area by using an image repairing technology to obtain an infrared background image of the current frame infrared image;

and obtaining a target saliency map by subtracting the current frame infrared image from the infrared background image, and obtaining the detected weak and small target through a threshold segmentation algorithm according to the target saliency map.

In one embodiment, the method further comprises the following steps: filtering the current frame infrared image through a Top-Hat morphological filtering algorithm to obtain a filtered image;

and segmenting suspected dim targets from the filtered images through a threshold segmentation algorithm to obtain first detection results of the suspected dim targets.

In one embodiment, the method further comprises the following steps: and obtaining a second detection result of the suspected weak small target by an interframe difference method and a threshold segmentation algorithm.

In one embodiment, the method further comprises the following steps: reconstructing the candidate target region by utilizing an algorithm based on a neighborhood weighted mean, an image inpainting algorithm based on diffusion, an image inpainting algorithm based on a Navier-Stokes equation or an image inpainting algorithm based on Fast Marching Method.

In one embodiment, the method further comprises the following steps: the threshold value calculation formula in the threshold value segmentation algorithm is as follows:

T＝mean+k·σ

wherein, T is the threshold value, mean is the pixel gray value mean value of the image subjected to threshold value segmentation, sigma is the pixel gray value variance of the image subjected to threshold value segmentation, and k is a constant larger than zero.

An infrared small target detection device based on an image inpainting technology, the device comprising:

the spatial domain detection module is used for acquiring a current frame infrared image in an infrared video image, and obtaining a first detection result of a suspected dim target through a spatial domain filtering algorithm and a threshold segmentation method according to the current frame infrared image; the infrared video image comprises a weak and small target to be detected;

the time domain detection module is used for acquiring a previous frame infrared image of the current frame infrared image, and obtaining a second detection result of the suspected weak small target through a time domain difference algorithm and a threshold segmentation algorithm according to the previous frame infrared image and the current frame infrared image;

the image reconstruction module is used for combining the first detection result and the second detection result to obtain a candidate target area of the weak and small target, digging the candidate target area from the current frame infrared image, and reconstructing the candidate target area by using an image repairing technology to obtain an infrared background image of the current frame infrared image;

and the detection result output module is used for subtracting the current frame infrared image and the infrared background image to obtain a target saliency map, and obtaining the detected small and weak target through a threshold segmentation algorithm according to the target saliency map.

A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:

A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:

The infrared small target detection method based on the image inpainting technology comprises the steps of extracting a current frame infrared image to be processed from an infrared video image containing a small target, respectively detecting the small target of the image to be processed through a spatial domain algorithm and a time domain algorithm, combining suspected small target areas obtained through the two ways into a candidate small target area, digging out the candidate target area in an original infrared image, recovering and estimating pixels where the candidate target is located through the image inpainting technology, reconstructing a current frame infrared background image, then performing difference on the current frame infrared image and the current frame infrared background image to obtain a target significant image, and finally obtaining the small target from the target significant image. The method fully utilizes the information of two detection methods by combining the target detection algorithms of a space domain and a time domain, excavates a weak and small target candidate region from the space-time dimension, ensures that a real target is in the candidate region with higher probability, and combines a quick and efficient image repairing technology, thereby effectively ensuring the high detection rate of the infrared weak and small target detection algorithm, reducing false alarms caused by noise, clutter, edges and other factors, and having low algorithm complexity and small calculation amount and being easy to meet the real-time requirement in practical engineering application.

Drawings

FIG. 1 is a flow diagram illustrating an exemplary method for detecting small and infrared targets based on image inpainting;

FIG. 2 is a schematic flow chart illustrating a method for detecting infrared small and weak targets based on image inpainting technology in another embodiment;

FIG. 3 is an original infrared image containing a small and weak target as employed in an embodiment of the present invention;

FIG. 4 is a diagram illustrating an original infrared image Top-Hat filtered according to an embodiment of the present invention;

FIG. 5 is a diagram illustrating a suspected target region segmented from an image obtained after Top-Hat filtering in an exemplary embodiment of the invention;

FIG. 6 is a time-domain difference image obtained by subtracting the previous frame of infrared image from the original infrared image of FIG. 3 in an embodiment of the present invention;

FIG. 7 is a suspected target area segmented from the time-domain difference image of FIG. 6 in an embodiment of the present invention;

FIG. 8 is a block diagram illustrating candidate target regions obtained by combining the suspected target regions of FIGS. 5 and 7 according to an embodiment of the present invention;

FIG. 9 is an image of the original IR image of FIG. 3 with candidate target areas removed in accordance with an embodiment of the present invention;

FIG. 10 is a background image obtained by reconstructing a candidate object region by using an image inpainting technique according to an embodiment of the present invention;

fig. 11 is a target saliency map obtained in an embodiment of the present invention, obtained by subtracting the background estimation image of fig. 10 from the original infrared image of fig. 2;

FIG. 12 is a diagram of a small and weak target obtained by final segmentation in an embodiment of the present invention;

FIG. 13 is a block diagram of an embodiment of an apparatus for detecting infrared small and weak objects based on image inpainting technology;

FIG. 14 is a diagram illustrating an internal structure of a computer device according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The infrared small and weak target detection method based on the image inpainting technology can be applied to the following application environments. The terminal extracts a current frame infrared image to be processed from an infrared video image containing a small target, respectively detects the small target of the image to be processed through a spatial domain algorithm and a time domain algorithm, combines suspected small target areas obtained through the two ways into a candidate small target area, then in an original infrared image, after the candidate target area is dug out, restores and estimates the pixel where the candidate target is located by using an image repairing technology, reconstructs a current frame infrared background image, then performs subtraction on the current frame infrared image and the current frame infrared background image to obtain a target saliency map, and finally obtains the small target from the target saliency map. The terminal may be, but is not limited to, various personal computers, notebook computers, and tablet computers.

In one embodiment, as shown in fig. 1, there is provided an infrared small and weak target detection method based on image inpainting technology, including the following steps:

step 102, acquiring a current frame infrared image in the infrared video image, and obtaining a first detection result of the suspected dim target through a spatial domain filtering algorithm and a threshold segmentation method according to the current frame infrared image.

The infrared video image comprises a weak target to be detected. The spatial filtering method comprises high-pass filtering, median filtering, mean filtering, Top-Hat filtering and the like.

And 104, acquiring a previous frame infrared image of the current frame infrared image, and obtaining a second detection result of the suspected weak small target through a time domain difference algorithm and a threshold segmentation algorithm according to the previous frame infrared image and the current frame infrared image.

And 106, combining the first detection result and the second detection result to obtain a candidate target area of the weak and small target, digging out the candidate target area from the current frame infrared image, and reconstructing the candidate target area by using an image repairing technology to obtain an infrared background image of the current frame infrared image.

The suspected small target areas segmented twice are merged into candidate small target areas, target information on a time domain and a space domain is comprehensively considered, the obtained candidate target areas can contain real small targets with higher probability, and therefore the small target detection probability of the whole algorithm is improved.

In order to ensure the real-time requirement of the whole algorithm, the candidate target region is reconstructed by using an image inpainting algorithm based on the neighborhood weighted mean.

And 108, subtracting the current frame infrared image and the infrared background image to obtain a target saliency map, and obtaining the detected weak and small target through a threshold segmentation algorithm according to the target saliency map.

In the infrared small target detection method based on the image inpainting technology, a current frame infrared image to be processed is extracted from an infrared video image containing a small target, the small target detection is respectively carried out on the image to be processed through a spatial domain algorithm and a time domain algorithm, suspected small target areas obtained through the two ways are combined into candidate small target areas, then the candidate target areas are dug out in an original infrared image, the pixel where the candidate target is located is recovered and estimated through the image inpainting technology, a current frame infrared background image is reconstructed, then the current frame infrared image and the current frame infrared background image are subjected to subtraction to obtain a target saliency map, and finally the small target is obtained from the target saliency map. The method fully utilizes the information of two detection methods by combining the target detection algorithm of a space domain and a time domain, more comprehensively excavates detail information, effectively improves the detection rate of infrared dim targets by combining an image repairing technology, reduces false alarms caused by noise, clutter, edges and other factors, has low algorithm complexity and small calculated amount, and is easy to meet the real-time requirement in practical engineering application.

In one embodiment, the method further comprises the following steps: and reconstructing the candidate target region by utilizing an algorithm based on a neighborhood weighted mean, an image inpainting algorithm based on diffusion, an image inpainting algorithm based on a Navier-Stokes equation or an image inpainting algorithm based on Fast Marching Method.

In an application scene of detecting the weak and small targets of the infrared video images, because the weak and small targets of each frame of image are required to be detected, the real-time requirement of the algorithm is high, the infrared background images can be quickly reconstructed through a simple and efficient image repairing technology based on neighborhood weighted average, the operation efficiency of the whole algorithm is improved, and the image repairing algorithm which is quick and easy to realize enables the whole algorithm to have better real-time performance.

Image inpainting technology based on neighborhood weighted average:

(a) the image repairing is to estimate and fill the region to be repaired by using the peripheral information of the region to be repaired. In the process of reconstructing the background image, in order to avoid the influence of an excessively bright (or excessively dark) target area on the reconstruction of the background image, the candidate target area is firstly dug away, and then the candidate target area is filled and repaired by using an image repairing technology to reconstruct the background image. The image inpainting selects candidate target region pixels from left to right and from top to bottom, a window calculated by neighborhood weighted average can be selected to be a weak target region which is expanded by a few pixels, and the window size can be 9 × 9, 11 × 11 and 13 × 13 if the weak target is smaller than 9 × 9.

(b) And after a calculation window is selected, carrying out weighted summation on pixel points of a non-candidate target area in the window, wherein the result is the corresponding pixel point of the reconstructed background image. And if the selected window is too small, causing no non-candidate target area pixel points in the window, expanding the window.

(c) In order to further avoid the influence of the too bright (or too dark) pixels of the target area on the reconstruction of the background image, a method different from a general weighting coefficient is adopted. The weighting method is generally that the closer the distance is, the larger the coefficient is, and here, the rule that the closer the distance is, the smaller the coefficient is adopted, and the square root of the distance can be taken as the weighting coefficient.

T＝mean+k·σ

It should be understood that, although the steps in the flowchart of fig. 1 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in fig. 1 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

In one embodiment, as shown in FIG. 2, a method of infrared small and weak target detection is provided. Firstly, respectively calculating a suspected dim target area by a spatial domain filtering method and a time domain difference method; secondly, merging the suspected small target areas divided twice into candidate small target areas, and digging out the candidate small target areas; thirdly, restoring and estimating the pixels of the candidate target area by using an image repairing technology, and reconstructing an infrared background image of the current frame; and finally, subtracting the reconstructed background image from the original infrared image of the current frame to obtain a target saliency map, and segmenting the weak and small targets by using a threshold segmentation method.

With reference to fig. 3 to 12, the specific implementation process of the present invention is as follows:

1. selecting proper structural element parameters, and performing background clutter suppression on the current frame original image by using a Top-Hat morphological filtering method to obtain a filtered image, as shown in fig. 4. By means of a threshold segmentation method, a suspected small object is segmented from the image, as shown in fig. 5.

2. And (4) selecting suspected dim targets by a time domain difference method. The difference between the current frame image and the previous frame image is used to obtain a difference image shown in image 6, and threshold segmentation is performed to obtain a suspected dim target, as shown in fig. 7.

3. The suspected small target areas obtained by the spatial filtering method and the time domain difference method are merged to obtain a candidate target area, as shown in fig. 8.

4. And (5) digging out a candidate target area from the current frame infrared image 3 to obtain a graph 9.

5. Image reconstruction and restoration are performed on the excavated candidate target region by using an image inpainting technique, so as to obtain an estimated background image, as shown in fig. 10. In this embodiment, the image inpainting technique used is a fast and efficient method based on the neighborhood weighted mean.

6. The reconstructed background image of fig. 10 is subtracted from the original image of fig. 3 to obtain a target saliency map, as shown in fig. 11.

7. The final weak and small targets are segmented from the target saliency map by a threshold segmentation method, as shown in fig. 12. The threshold value is calculated by the formula

T₁＝mean₁+k₁·σ₁

In the formula, mean₁Mean, σ, of the target saliency map₁Is the variance, k, of the target saliency map₁Is a constant greater than zero, when the gray value of the pixel in the target saliency map is greater than T₁The hour is indicated as the target.

In one embodiment, as shown in fig. 13, there is provided an infrared weak and small target detection apparatus based on an image inpainting technology, including: a spatial domain detection module 1302, a temporal domain detection module 1304, an image reconstruction module 1306, and a detection result output module 1308, wherein:

the spatial domain detection module 1302 is configured to obtain a current frame infrared image in the infrared video image, and obtain a first detection result of a suspected dim target according to the current frame infrared image through a spatial domain filtering algorithm and a threshold segmentation method; the infrared video image comprises a weak target to be detected;

the time domain detection module 1304 is configured to obtain a previous frame infrared image of the current frame infrared image, and obtain a second detection result of the suspected weak small target through a time domain difference algorithm and a threshold segmentation algorithm according to the previous frame infrared image and the current frame infrared image;

the image reconstruction module 1306 is configured to combine the first detection result and the second detection result to obtain a candidate target region of a weak target, dig out the candidate target region from the current frame infrared image, and reconstruct the candidate target region by using an image inpainting technology to obtain an infrared background image of the current frame infrared image;

the detection result output module 1308 is configured to perform subtraction on the current frame infrared image and the infrared background image to obtain a target saliency map, and obtain a detected small target through a threshold segmentation algorithm according to the target saliency map.

The spatial domain detection module 1302 is further configured to filter the current frame infrared image through a Top-Hat morphological filtering algorithm to obtain a filtered image; and segmenting suspected dim targets from the filtered images through a threshold segmentation algorithm to obtain first detection results of the suspected dim targets.

The time domain detecting module 1304 is further configured to obtain a second detection result of the suspected weak small target by using an inter-frame difference method and a threshold segmentation algorithm.

The image reconstruction module 1306 is further configured to reconstruct the candidate target region using a neighborhood weighted mean-based algorithm, a diffusion-based image inpainting algorithm, a Navier-Stokes equation-based image inpainting algorithm, or a Fast Marching Method-based image inpainting algorithm.

For specific limitations of the infrared small and weak target detection apparatus based on the image inpainting technology, reference may be made to the above limitations of the infrared small and weak target detection method based on the image inpainting technology, and details are not repeated here. All or part of the modules in the infrared small and weak target detection device based on the image inpainting technology can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 14. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to realize an infrared small and weak target detection method based on an image inpainting technology. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the architecture shown in fig. 14 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In an embodiment, a computer device is provided, comprising a memory storing a computer program and a processor implementing the steps of the above method embodiments when executing the computer program.

In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. An infrared small and weak target detection method based on an image inpainting technology is characterized by comprising the following steps:

2. The method of claim 1, wherein obtaining the first detection result of the suspected dim target by the spatial domain filtering algorithm and the threshold segmentation method comprises:

filtering the current frame infrared image through a Top-Hat morphological filtering algorithm to obtain a filtered image;

3. The method of claim 1, wherein obtaining a second detection result of the suspected weak and small target by a time domain difference algorithm and a threshold segmentation algorithm comprises:

and obtaining a second detection result of the suspected weak small target by an interframe difference method and a threshold segmentation algorithm.

4. The method of claim 3, wherein reconstructing the candidate target region using an image inpainting technique comprises:

reconstructing the candidate target region by utilizing an algorithm based on a neighborhood weighted mean, an image inpainting algorithm based on diffusion, an image inpainting algorithm based on a Navier-Stokes equation or an image inpainting algorithm based on Fast Marching Method.

5. The method of claim 4, wherein the threshold calculation formula in the threshold segmentation algorithm is:

T＝mean+k·σ

6. An infrared small and weak target detection device based on image inpainting technology, which is characterized by comprising:

7. The apparatus of claim 6, wherein the spatial domain detection module is further configured to:

8. The apparatus of claim 6, wherein the time domain detection module is further configured to:

9. The apparatus of claim 6, wherein the image reconstruction module is further configured to:

10. The apparatus of claim 6, wherein the detection result output module is further configured to:

obtaining a target saliency map by subtracting the current frame infrared image from the infrared background image, and obtaining a detected weak target through a threshold segmentation algorithm according to the target saliency map; the threshold value calculation formula in the threshold value segmentation algorithm is as follows:

T＝mean+k·σ