KR102456018B1 - Method for infrared counter countermeasure using dual-band infrared images - Google Patents

Abstract

The present embodiments detect a spoof from a dual-band infrared image including a main band image and a sub-band image, extract a target candidate from the main band image, and obtain an image capable of accurately tracking a target by excluding the spoof from the target candidate A device and a countermeasure against a deceptive body are provided.

Description

How to respond to infrared decoys using dual-band infrared images {METHOD FOR INFRARED COUNTER COUNTERMEASURE USING DUAL-BAND INFRARED IMAGES}

The technical field to which the present invention pertains relates to a method for responding to an infrared spoof using a dual-band infrared image.

The content described in this section merely provides background information for the present embodiment and does not constitute the prior art.

In the process of detecting/tracking a target using an infrared image, it may be tracked in the wrong direction by an infrared decoy (InfraRed CounterMeasure, IRCM). IRCCM (InfraRed Counter CounterMeasure) technology is needed to counter these deceptive entities.

Korean Patent Publication No. 10-1868094 (2018.06.08.)

Embodiments of the present invention are mainly for detecting a spoof from a dual-band infrared image including a main band image and a sub-band image, extracting a target candidate from the main band image, and accurately tracking a target by excluding the spoof from the target candidate. There is a purpose.

Other objects not specified in the present invention may be additionally considered within the scope that can be easily inferred from the following detailed description and effects thereof.

According to an aspect of the present embodiment, an infrared detector for acquiring a dual-band infrared image including a main band image and a sub-band image, a spoof detector for extracting spoof information from the dual-band infrared image, and the main band image There is provided an image acquisition device including a target tracking unit for extracting target candidate information from and outputting target information by excluding the deceptive information.

The infrared detector may set a wavelength band in the range of 3 to 5 μm as the wavelength band of the main band image, and set a wavelength band lower than the wavelength band of the main band image as the wavelength band of the sub-band image.

The spoof detector may determine the presence or absence of a spoof from the dual-band infrared image, and determine the location of the spoof.

The spoof detection unit is a method of classifying based on a first threshold value that maximizes the variance of the two groups based on the histogram distribution of the dual-band infrared image, or a method of classifying using the target information by the target tracking unit Through this, the target candidate region and the background region may be separated from the dual-band infrared image.

The spoof detection unit calculates an average pixel value for the target candidate region and a moving average value of the average pixel value, compares a difference between a current average value and a moving average value of the average pixel value with a second threshold value, The presence or absence of deception can be determined.

The spoof detection unit calculates an image ratio of the target candidate region and a moving average value of the image ratio for the target candidate region based on the average pixel value of the main band image and the average pixel value of the sub-band image, , by comparing the difference between the current image ratio and the moving average value of the image ratio with a third threshold value, it is possible to determine the presence or absence of the deceptive object.

The spoof detection unit determines the location of the spoof by using the spoof probability of each pixel of the dual-band infrared image, and determines the spoof probability for each pixel as energy of the main band in the main band image. Depending on the magnitude between the intensity and the energy intensity of the sub-band in the sub-band image, (i) the ratio between the energy intensity of the main band in the main band image and the energy intensity of the sub-band in the sub-band image, or (ii) fixed It can be set as a value.

The target tracking unit tracks the target candidate information by applying a track filter to the target candidate information extracted from the main band image, and applies a spoof mask filter having a spoof mask to the tracked target candidate information to apply the spoofing mask. A line of sight (LOS) may be adjusted by extracting target information excluding information and applying a LOS compensation filter to the target information and the deceptive information.

According to another aspect of this embodiment, in a method for countering a spoof by an image acquisition device, acquiring a dual-band infrared image including a main band image and a sub-band image, and extracting spoof information from the dual-band infrared image extracting target candidate information from the main band image, applying a track filter to the target candidate information extracted from the main band image to track the target candidate information, and a deceptive mask on the tracked target candidate information extracting target information excluding the spoof information by applying a spoof mask filter having It provides a method for countering a deceptive body comprising the steps.

The track filter manages a track for multiple tracking targets, stores the track history, maintains a deceptive state for target candidate information previously determined as a deceptive object, and at a field of view (FOV) Newly appeared deceptives can be excluded from target candidate information.

The spoof mask filter divides the spoof region and the background region into a spoof region and a background region based on the spoof probability of each pixel of the dual-band infrared image, and an IOU (Intersection Over Union) between the spoof region and the target candidate information. By comparing with the fourth threshold, the corresponding target candidate information may be determined as a deceptive object.

The spoofing countermeasure method performs the step of applying a kinematics filter after the step of extracting the target information excluding the deceptive information, and the kinematics filter stores the motion trajectory of the target candidate information to store the target candidate information If it accelerates and decelerates, it can be judged as a deceptive body.

The LOS compensation filter may maintain the relative movement of the line of sight and the target information in a situation where the deceptive information and the target information overlap.

The LOS compensation filter may move a field of view (FOV) in a direction in which the target information moves so that the spoof information disappears from the screen in a situation where the spoof information and the target information are distinguished.

As described above, according to the embodiments of the present invention, a spoof is detected from a dual-band infrared image including a main band image and a sub-band image, a target candidate is extracted from the main band image, and a spoof is excluded from the target candidate. This has the effect of accurately tracking the target.

Even if effects not explicitly mentioned herein, the effects described in the following specification expected by the technical features of the present invention and their potential effects are treated as described in the specification of the present invention.

1 and 2 are diagrams illustrating an image acquisition apparatus according to an embodiment of the present invention.
FIG. 3 is a diagram illustrating dual-band infrared characteristics processed by the image acquisition apparatus according to an embodiment of the present invention and an emission form of a deceit.
4 is a diagram illustrating a spoof ROI mask based on a spoof probability for each pixel processed by the image acquisition apparatus according to an embodiment of the present invention.
5 is a diagram illustrating a plurality of filters applied to an image acquisition apparatus according to an embodiment of the present invention.
6 is a diagram illustrating a result of applying a track filter by the image acquisition apparatus according to an embodiment of the present invention.
7 is a diagram illustrating a result of applying the LOS compensation filter by the image acquisition apparatus according to an embodiment of the present invention.

Hereinafter, in the description of the present invention, when it is determined that the subject matter of the present invention may be unnecessarily obscured as it is obvious to those skilled in the art with respect to related known functions, the detailed description thereof will be omitted, and some embodiments of the present invention will be described. It will be described in detail with reference to exemplary drawings.

This embodiment relates to an infrared countermeasure (InfraRed Counter CounterMeasure, IRCCM) for efficient target tracking in a situation where an infrared response (InfraRed CounterMeasure, IRCM) of a target is operated based on an image simultaneously acquired with a dual-band infrared detector.

1 and 2 are diagrams illustrating an image acquisition apparatus according to an embodiment of the present invention.

The image acquisition apparatus 100 includes an infrared detection unit 110 , a decoy detection unit 120 , and a target tracking unit 130 .

The infrared detector 110 acquires a dual-band infrared image including a main band image and a sub-band image. The infrared detector 110 may be implemented as a dual-band infrared detector, and the dual-band infrared detector is a detector that acquires an infrared image and simultaneously acquires images of two bands.

The infrared detector 110 may set a wavelength band in the range of 3 to 5 μm as the wavelength band of the main band image, and set a wavelength band lower than the wavelength band of the main band image as the wavelength band of the auxiliary band image.

The spoof detector 120 extracts spoof information from the dual-band infrared image. The spoof detector 120 determines the presence or absence of a spoof from the dual-band infrared image, and determines the location of the spoof. The spoof detection unit 120 performs a function of determining whether there is a spoof in the acquired image and calculating the position of the spoof in the image with a probability. The spoof detector 120 determines the presence and location of a spoof in the image by utilizing a sudden increase in energy received in the two infrared bands and a sudden change in the reception ratio.

The spoof detection unit 120 performs classification based on the first threshold value that maximizes the variance of the two groups based on the histogram distribution of the dual-band infrared image, or the classification method using target information by the target tracking unit. separates the target candidate region and the background region in the dual-band infrared image.

In order to determine whether there is a deceptive body, the first step is to separate the target candidate area and the background area. The target candidate region broadly refers to both the region for the target to be tracked and the region generated by the deceptive body. For example, separation may be performed based on a histogram distribution of an image obtained using the otsu threshold method, or may be separated using a target tracking result of a target tracking unit.

The spoof detector 120 calculates an average pixel value and a moving average value of the average pixel value for the target candidate region, compares the difference between the current average value and the moving average value of the average pixel value with a second threshold value, The presence or absence of a body can be determined.

The spoof detector 120 calculates an average pixel value (Target _avg ) for the target area, and also calculates a moving average value (Target moving _avg ) for this. When the difference between the current average value and the moving average value is greater than a threshold _decoy , it can be determined that a decoy has been detected in the image. This is a detection method for the sudden appearance of a deceptive object in the acquired image area.

The spoof detector 120 calculates a moving average value of the image ratio and the image ratio of the target candidate region based on the average pixel value of the main band image and the average pixel value of the sub-band image with respect to the target candidate region, and the current image By comparing the difference between the ratio and the moving average of the image ratio with the third threshold value, the presence or absence of a deceptive body may be determined.

Using the average pixel value of the infrared main band for the target candidate region (Target _{avg (p)} ) and the average pixel value of the auxiliary band (Target _{avg (s)} ) with respect to the target candidate region, the ratio for the target candidate region (Target _ratio ) is calculated, A moving average value (Target _{moveinavg (ratio)} ) is also calculated for this. Also, when the difference between the current average value and the moving average value is greater than a threshold _decoy(ratio) , it can be determined that a decoy has been detected in the image. This is a detection method using the characteristic that the reaction of the target and the decoy is different for each infrared band.

The spoof detector 120 determines the location of the spoof by using the spoof probability of each pixel of the dual-band infrared image. The decoy probability (P _decoy ) for each pixel in the image uses the energy intensity (I _p ) of the main band with respect to the energy intensity (I _s ) of the sub-band of each pixel.

The spoof detection unit 120 calculates the spoof probability for each pixel according to the magnitude between the energy intensity of the main band in the main band image and the energy intensity of the sub band in the sub-band image (i) the main band in the main band image. It is set as a ratio between the energy intensity of the band and the energy intensity of the sub-band in the sub-band image, or (ii) a fixed value.

The target tracking unit 130 extracts target candidate information from the main band image and outputs target information by excluding deceptive information from the target candidate information.

When the existence of the deceptive object is confirmed, the target tracking unit 130 performs an operation of maintaining the relative movement of the existing target except for the newly detected track, an operation of tracking the target in the ROI except for the screen area where the deceptive object is located, the existing target By biasing in the relative movement direction of

The target tracking unit 130 tracks target candidate information by applying a track filter to target candidate information extracted from the main band image. The target tracking unit 130 applies a spoof mask filter having a spoof mask to the tracked target candidate information to extract target information excluding the spoof information. The target tracking unit 130 adjusts a line of sight (LOS) by applying a LOS compensation filter to the target information and the deceptive information.

FIG. 3 is a diagram illustrating dual-band infrared characteristics processed by the image acquisition apparatus according to an embodiment of the present invention and an emission form of a deceit.

As shown in (a) of FIG. 3 , a band in which the target can express the target well including the peak region among the radiation characteristics is selected as the main infrared band, and the infrared region in which the radiation energy ratio of the target and the decoy is different is supported. Select the band

For example, the hot engine part of a typical aircraft has a temperature range of 1300 K to 2000 K, and according to Planck's law, it has a peak energy radiation at a wavelength of 3 to 5 um, so when targeting an aircraft, set that band as the main band, In the case of a flare, since it emits more energy than an aircraft, the auxiliary band is set to a lower wavelength than the main band.

By selecting the band in this way, the main infrared band can be used as a main image for spoof detection and target tracking, and the infrared sub band can be used as an auxiliary role for spoofing detection.

Referring to (b) of FIG. 3 , the decoy shows the kinematic movement after detaching from the platform, shows a rapid decrease in movement due to aerodynamic drag, and proceeds to fall due to the earth's gravity.

4 is a diagram illustrating a spoof ROI mask based on a spoof probability for each pixel processed by the image acquisition apparatus according to an embodiment of the present invention.

4, 410 denotes a main band image, 420 denotes a spoof probability per pixel, 430 denotes a spoof mask, 440 denotes a target candidate, and 450 denotes a target candidate after spoofing mask filter.

5 is a diagram illustrating a plurality of filters applied to an image acquisition apparatus according to an embodiment of the present invention.

A method for countering a spoof by an image acquisition device includes acquiring a dual-band infrared image including a main band image and a sub-band image, extracting spoof information from the dual-band infrared image, and obtaining target candidate information from the main band image. extracting, applying a track filter to target candidate information extracted from the main band image to track target candidate information, applying a spoof mask filter having a spoof mask to the tracked target candidate information to exclude the spoof information A step of extracting information and a step of adjusting a line of sight (LOS) by applying a LOS compensation filter to the target information and the deceptive information are performed.

The target tracking unit performs image tracking and applies a tracking result filter.

Image tracking tracks multiple targets for images in the infrared main band. The tracking method may use a general image tracking algorithm capable of tracking multiple targets. For example, a tracking learning detection (TLD) technique may be used.

The tracking result filter outputs useful target information excluding deceptive objects among the results tracked by the image tracking algorithm. As shown in FIG. 5 , the tracking result filter includes a track filter, a deceptive mask filter, a kinematics filter, and a LOS compensation filter. If a deceptive object is not detected, the value generated from the video tracking block is transmitted as it is, and filtering is performed only when a deceptive object is detected.

The track filter memorizes the history of the video tracking result and maintains it as a deceptive for a track determined as a deceptive in the previous time. In addition, when a spoof is detected, a newly created tracking track is determined as a spoof, and a spoof newly appearing in the field of view (FOV) of the image acquisition device is excluded from target candidates.

As shown in FIG. 6 , when the spoof detection unit determines that there is no spoof, tracking tracks by all tracking targets are maintained ( 610 ). When the spoof is determined to be within the FOV by the decoy detection unit as at time t ₁ to t ₃ , and a new tracking track is created as a result of video tracking, the tracking is maintained to memorize the tracking track, but the tracking track is stored in the deceiver. It is set as a result of by (620, 620, 630).

As shown in Figure 4, the spoof mask filter divides the spoof probability per pixel of the spoof detection unit into a spoof area and a background area using the otsu threshold technique. After that, if the target candidate and the intersection over union (IOU) are above the threshold _IoU , the target candidate is judged as a deceptive.

The kinematics filter memorizes the motion trajectory in the image of the target candidate and judges it as a deceptive object when it decelerates rapidly. This is, as shown in Fig. 3 (b), the kinetic energy when the deceptive body is separated from the platform is rapidly decelerated by the aerodynamic drag. In the case of an aircraft, it can be used to discharge in the left/right or left/right/down directions in the direction of travel from the rear of the aircraft. In the case of a ship (or tank), it is discharged in a direction away from the surface and then falls.

The LOS compensation filter is used when the direction of the LOS movement cannot be clearly determined, such as overlapping the decoy and the target, and moves the LOS by considering two things.

Maintain the relative motion of the LOS of the image acquisition device and the target. For example, virtual constant acceleration motion is maintained for the tracking track that is the result of the image tracking algorithm, and the LOS value is maintained as the image tracking result in case the entire system is an inertial navigation missile ( S710 ).

When the distinction between the target and the deceit is clear, the FOV is moved in the forward direction of the target in order to quickly disappear from the image screen (720).

According to the method for countering the spoof performed by the image acquisition device, it is possible to detect the spoof by combining the characteristics of the spoof according to the laws of physics to accurately determine the spoof. The decoy emits energy from the moment it leaves the platform and shows a sharp increase in the received energy. Since the wavelength of energy emission of the decoy and the wavelength of energy emission of the target are different, a difference in the reception ratio of the two infrared bands occurs.

According to the deceptive countermeasure method performed by the image acquisition device, it is helpful in maintaining target tracking without being deceived by ignoring the deceptive object in the image acquired using the image tracking result filter, or by quickly sending the deceitful object out of the screen. .

The image acquisition apparatus may be mounted in the form of software, hardware, or a combination thereof on a computing device or server provided with hardware elements. A computing device or server is all or part of a communication device such as a communication modem for performing communication with various devices or wired/wireless communication networks, a memory for storing data for executing a program, a microprocessor for executing the program to perform calculations and commands, etc. It can mean a variety of devices, including

The image acquisition device may include at least one processor, a computer-readable storage medium, and a communication bus.

The processor may control the image acquisition device to operate. For example, the processor may execute one or more programs stored in a computer-readable storage medium. The one or more programs may include one or more computer-executable instructions, which, when executed by a processor, may be configured to cause the image acquisition device to perform operations according to the exemplary embodiment.

The computer-readable storage medium is configured to store computer-executable instructions or program code, program data and/or other suitable form of information. A program stored in a computer-readable storage medium includes a set of instructions executable by a processor. In one embodiment, the computer-readable storage medium includes memory (volatile memory, such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory device , or any other form of storage medium that can be accessed by the SAR sidelobe reduction device and can store desired information, or a suitable combination thereof.

A communication bus interconnects the various other components of the SAR side lobe reduction device, including a processor and a computer readable storage medium.

The image acquisition device may also include one or more input/output interfaces and one or more communication interfaces that provide interfaces for one or more input/output devices. The input/output interface and the communication interface are coupled to the communication bus. The input/output device may be connected to other components of the SAR side lobe reduction device via an input/output interface.

The image acquisition apparatus may be implemented in a logic circuit by hardware, firmware, software, or a combination thereof, or may be implemented using a general-purpose or special-purpose computer. The device may be implemented using a hardwired device, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or the like. In addition, the device may be implemented as a system on chip (SoC) including one or more processors and controllers.

Although it is described that each process is sequentially executed in FIG. 5, this is only illustratively described, and those skilled in the art change the order described in FIG. Alternatively, various modifications and variations may be applied by executing one or more processes in parallel or adding other processes.

The operations according to the present embodiments may be implemented in the form of program instructions that can be performed through various computer means and recorded in a computer-readable medium. Computer-readable medium represents any medium that participates in providing instructions to a processor for execution. Computer-readable media may include program instructions, data files, data structures, or a combination thereof. For example, there may be a magnetic medium, an optical recording medium, a memory, and the like. A computer program may be distributed over a networked computer system so that computer readable code is stored and executed in a distributed manner. Functional programs, codes, and code segments for implementing this embodiment may be easily inferred by programmers in the art to which this embodiment belongs.

The present embodiments are for explaining the technical idea of the present embodiment, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The protection scope of this embodiment should be interpreted by the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present embodiment.

Claims (14)

an infrared detector for acquiring a dual-band infrared image including a main band image and a sub-band image;
a spoof detector for extracting spoof information from the dual-band infrared image; and
and a target tracking unit that extracts target candidate information from the main band image and outputs target information by excluding the deceptive information,
The target tracking unit,
Tracking the target candidate information by applying a track filter to the target candidate information extracted from the main band image,
applying a spoof mask filter having a spoof mask to the tracked target candidate information to extract target information excluding the spoof information,
The image acquisition apparatus according to claim 1, wherein a line of sight (LOS) is adjusted by applying a LOS compensation filter to the target information and the decoy information. According to claim 1,
The infrared detector,
An image acquisition apparatus characterized in that a wavelength band in the range of 3 to 5 um is set as the wavelength band of the main band image, and a wavelength band lower than the wavelength band of the main band image is set as the wavelength band of the auxiliary band image. According to claim 1,
The decoy detection unit,
and determining the presence or absence of a spoof from the dual-band infrared image and determining a location of the spoof. According to claim 1,
The decoy detection unit,
Based on the histogram distribution of the dual-band infrared image, the dual-band infrared image is classified based on a first threshold value maximizing the variance of the two groups or the dual-band infrared image is classified using the target information by the target tracking unit. An image acquisition device, characterized in that the target candidate region and the background region are separated from the image. 5. The method of claim 4,
The decoy detection unit,
For the target candidate region, an average pixel value and a moving average value of the average pixel value are calculated, and the difference between the current average value and the moving average value of the average pixel value is compared with a second threshold value to determine the presence of the deceptive object Image acquisition device, characterized in that. 5. The method of claim 4,
The decoy detection unit,
For the target candidate region, an image ratio of the target candidate region and a moving average value of the image ratio are calculated based on the average pixel value of the main band image and the average pixel value of the sub-band image, and the current image ratio and the and comparing the difference between the moving average values for the image ratio with a third threshold value to determine the presence or absence of the deceptive object. 4. The method of claim 3,
The decoy detection unit,
determining the position of the spoof by using the spoof probability of each pixel of the dual-band infrared image,
(i) Energy intensity of the main band in the main band image according to the magnitude between the energy intensity of the main band in the main band image and the energy intensity of the sub band in the sub band image and (ii) a ratio between the energy intensities of the sub-band in the sub-band image, or (ii) a fixed value. delete A method for countering a deceptive body by an image acquisition device, the method comprising:
acquiring a dual-band infrared image including a main band image and a sub-band image;
extracting deceit information from the dual-band infrared image;
extracting target candidate information from the main band image;
tracking the target candidate information by applying a track filter to the target candidate information extracted from the main band image;
extracting target information excluding the spoof information by applying a spoof mask filter having a spoof mask to the tracked target candidate information; and
and adjusting a line of sight (LOS) by applying a LOS compensation filter to the target information and the spoof information. 10. The method of claim 9,
The track filter is
Manage tracks for multiple tracking targets, store the track history, maintain a spoof state for target candidate information previously determined to be a spoof, and detect a newly emerged spoof from a field of view (FOV). A deceptive countermeasure method, characterized in that it is excluded from target candidate information. 10. The method of claim 9,
The gas mask filter,
A spoof region and a background region are divided based on the spoof probability of each pixel of the dual-band infrared image, and the Intersection Over Union (IOU) between the spoof region and the target candidate information is compared with a fourth threshold value. , a deceptive countermeasure method, characterized in that the corresponding target candidate information is determined as a deceptive object. 10. The method of claim 9,
Performing the step of applying a kinematics filter after the step of extracting the target information excluding the deceptive information,
and the kinematics filter stores the motion trajectory of the target candidate information, and when the target candidate information decelerates, it is determined as a spoof. 10. The method of claim 9,
The LOS compensation filter,
and maintaining the relative movement of the line of sight and the target information in a situation where the spoof information and the target information overlap. 10. The method of claim 9,
The LOS compensation filter,
In a situation where the spoof information and the target information are distinguished, the spoofing countermeasure method, characterized in that the field of view (FOV) is moved in the direction in which the target information moves so that the spoof information disappears from the screen.

Images