WO2019164464A1

WO2019164464A1 - Method for target detection and bad pixel replacement by controlled camera motion

Info

Publication number: WO2019164464A1
Application number: PCT/TR2018/050065
Authority: WO
Inventors: Erdem Akagündüz; Erkan OKUYAN
Original assignee: Aselsan Elektroni̇k Sanayi̇ Ve Ti̇caret Anoni̇m Şi̇rketi̇
Priority date: 2018-02-21
Filing date: 2018-02-21
Publication date: 2019-08-29
Also published as: EP3756341A1

Abstract

A method for improving automatic target detection with replacement of bad pixels by using a controlled motion of the electro-optical systems. By utilizing the controlled motion of the optical system (i.e. the camera), two main improvements are achieved: the field of search is increased and bad/dead pixels on the camera sensor are detected assuming the values of these pixels do not change when the entire scene is in motion. The algorithm that is revealed with the method renders such systems resilient against bad pixels.

Description

METHOD FOR TARGET DETECTION AND BAD PIXEL REPLACEMENT BY

CONTROLLED CAMERA MOTION

Technical Field

The present disclosure relates a method for improving automatic target detection with replacement of bad pixels by using a controlled motion of the electro-optical system.

Description of Related Art

Electro-optical systems are used to keep under surveillance of an area and detect any object (missile, aircraft etc.) that can pose a threat. Bad pixels can be problem for mission critical cases, such as detecting dim targets on infrared electro-optic systems. Any type of array detector includes bad pixels that simply do not create the necessary signals required for the detector to function. In certain applications, especially when pixel or subpixel sized signals are being detected/tracked, bad pixels become an important problem.

The most well-known and practiced method to avoid bad pixels is to construct a list of bad pixels during production, specific to any produced detector and use this list of bad pixels to replace them with functioning neighbors. This list is provided in the detector datasheet, so that the designer takes the necessary precautions to avoid any harm that the bad pixel could cause. The patent numbered US8571346B2 is an example for mentioned method that is based on a comparison of a pixel under evaluation with surrounding pixels and its correction. However new bad pixels may emerge after production. Therefore the detectors are applied some sort of calibration when restarted. For instance, new bad pixels can be detected by using a reference plate the detector sees on each restart.

Unfortunately, bad pixels can still emerge after the restart of camera system because of the vibration that the system experiences, or some other effects, new bad pixels emerge on-the- fly. For example bad pixels can emerge on infrared missile seeker systems at any stage of a missile’s flight, and they may cause the mission to fail.

There are many published documents in the technique similar to US8994819B2 numbered U.S. patent application. The application titled“Integrated optical detection system” discloses an optical detection system integrated within an electro-optical sighting and/or scanning system. Another application numbered US6978965B1 describes a device for target-tracking missiles having an electro-optical seeker assembly mounted in a missile structure through gimbals. U.S. patent no. US8422816B2 entitled“Method and apparatus for performing bad pixel compensation” introduces a method that compensates absolute values of pixels around a target in different directions. Then a processor performs bad pixel compensation on the target pixel according to the first detection value and the second detection value. However this method can only detect bad pixels before operation and unable to detect pixels during the detection operation as stated in the prior art.

Previous literature show no exact implementation of systems that improve target detection via controlled and synchronized camera motion and enlarge the field-of-search.

Purpose of the Invention

The abovementioned problems are addressed and a technical solution is proposed with a method for improving automatic target detection with replacement of bad pixels by using a controlled motion of the electro-optical systems. By utilizing a controlled motion on the optical system (i.e. the camera), two main improvements are achieved: the field of search is increased and bad/dead pixels on the camera sensor are detected assuming the values of these pixels do not change when the entire scene is in motion. The algorithm that is revealed with the method renders such systems resilient against bad pixels.

The presented method running parallel and simultaneously with a visual detection algorithm, comprising the steps of:

• introducing a predefined controlled motion to the electro-optical system,

• keeping track of the results of the detection algorithm in a matrix table determining target or target-like regions having higher values (i.e. over the detection threshold of the system) that will show target candidate strength in each frame,

• increasing the value of a pixel by an integer constant if the candidate strength of the pixel on the current image is above a threshold value,

• decreasing the value of a pixel found as a candidate pixel on the previous image but not in the current image by an integer constant if the value is not already zero,

• considering a pixel as bad if the value of the pixel is above the threshold value,

• decreasing the value of each pixel by a predetermined number to avoid overflow if the value is not already zero. Brief Description of the Drawings

Figure 1 shows a scene pixel movement and blur region graphically according to different frequency and radius values indicated in the undermentioned Table 1.

Detailed Description

Dynamic (on-the-fly) bad pixel detection algorithm

The proposed algorithm aims to detect any bad pixels, at the exact point it emerges. For this purpose, this algorithm is run parallel and simultaneously with the actual visual algorithm (detection, tracking etc.). The detection algorithm simply keeps track of the results of detection filter by using the output of a point target candidate detector system. For example, the correlation or convolution of a Gaussian with the NxN local regions on the image will output higher values for target and target-like regions. The output of such a filter can be used to detect target candidates on a single image. The output of such filter is called the target candidate strength. The proposed dynamic bad pixel detection algorithm can use these strength values of such a filter. Flowever the proposed bad pixel detection can use the output of any other candidate detection algorithm as well.

The dynamic bad pixel detection algorithm simply keeps track of the candidate strengths of the current and the previous frame. By checking the stability of the candidate strength, a pixel on the detector can be labelled as bad. For this purpose, the proposed algorithm keep a “Candidate Pixel Repeat” (CPR) image having the same resolution of the image. This image is a matrix with zero values on each pixel initially. On each new frame the proposed algorithm does the following:

If the candidate strength of a pixel on the current image is above a threshold value (i.e. if the output of the candidate detection filter on the current image considers this pixel as a target candidate), the value of that pixel of CPR image is increased by an integer constant.

If the a pixel is found as a candidate pixel on the previous image, but it is not found as a candidate image in the current image, the value of that pixel of CPR image is decreased by an integer constant. Flowever a negative value on the CPR image is not allowed. Thus CPR pixel value stays zero, if it is already zero.

- After the aforementioned increments and decrements, if the value of a pixel on the CPR image is above threshold value, that pixel is considered to be a bad pixel.

- Finally regardless of the value of the filter candidate filter output of the current or previous frames, every pixel on the CPR image is decreased by an integer to avoid overflow. Again, a negative value on the CPR image is not allowed. Thus CPR pixel value stays zero, if it is already zero.

Critical assumptions and the limitations of the algorithm

The very basic drawback of the algorithm is that for the algorithm not to detect actual targets, which are still, in other words, occupy the same detector pixels for a sufficient amount of time, would be detected as bad. In order to overcome this drawback, we propose a controlled movement on the camera. For a gimbal system, with the ability to make control movement this function is trivial.

For instance, a seeker head system required to detect a point target. During its search mode, the seeker could suffer from a new emerged bad pixel. Using the proposed algorithm and introducing a circular motion around the search axis with a constant radius and frequency, no scene component occupy the same detector pixel on succeeding frames of the detector. However the bad pixels always have the same detector coordinate.

Such a gimbal movement with radius r (pixels) and frequency f (Hz), a detector with fps frame rate (the number of frames created in a second), the amount motion on the detector frame that scene component experience is given by the following formula.

Thus a detector with 100 frame rate, with a circular motion of 5 pixels radius and 2 Hz frequency, a scene component experiences 0.63 pixels displacement on each new frame. The controlled camera motion need not to be circular as long as it is continuous.

If the gimbal motion is very fast, motion blur occurs in the image, which could negatively affect the required vision task. For example, for a system with integration time tint (the amount of time a detector spends to collect radiation to create a single frame), a pixel sized target would resolve into a k = n . fps . tint sized region (see Table 1 and Figure 1 ) Table 1 : Scene pixel movement and blur region for different gimbal movement where fps is

100 and tint is 4ms

Claims

1. A method for improving automatic target detection with replacement of bad pixels, running parallel and simultaneously with a visual detection algorithm, comprising the steps of:

• introducing a predefined controlled motion which is continuous to an electro-optical system,

• keeping track of the results of the detection algorithm in a matrix table determining target or target-like regions having values which exceed a predetermined threshold and show target candidate strength in each frame with same resolution,

• decreasing the value of each pixel by a predetermined number to avoid overflow if the value is not already zero.