JP2008311834A - Defective pixel correcting device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a defective pixel correcting device capable of suppressing an unnatural image. <P>SOLUTION: The defective pixel correcting device has a frame memory part 4 for sequentially storing image data supplied from an infrared imaging part 1 in a frame unit, and a determination circuit 4 for making respective pixels constituting the infrared imaging part 1 determination target pixels, calculating a difference between a brightness value in the current frame supplied from the infrared imaging part 1 and in each of the brightness values of a plurality of delay frames which are stored in the frame memory part 4 and temporally continue to the current frame about the determination target pixels and examining a relation of the difference to a threshold. When the relation satisfies a determination condition showing a normal output, the determination circuit 4 outputs the brightness value of the current frame as an output of the determination target pixels as it is, and when the relation satisfies a determination condition showing an abnormal output, it outputs a brightness value regarded as a normal output among the brightness values of the plurality of delay frames. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an infrared imaging device including a two-dimensional array type infrared detector.

  In manufacturing an infrared imaging device, it is very difficult to always produce an infrared detector without defective pixels. The defective pixel includes a fixed defective pixel whose output is always abnormal because the sensitivity is higher (or lower) than other pixels, and a blinking defective pixel in which normal output and abnormal output are repeated irregularly. When all of the infrared detectors including such defective pixels are handled as defective products, the yield in manufacturing the infrared imaging device is deteriorated. In order to improve the yield, it is necessary to correct the defective pixels so that the infrared detector including the defective pixels can be used without degrading the display quality.

Patent Document 1 describes a solid-state imaging device capable of detecting and correcting a blinking defective pixel of an infrared imaging device composed of a plurality of pixels arranged two-dimensionally. This solid-state imaging device includes, for each pixel, a frame memory integrator that calculates an integrated pixel output by integrating an output signal from the pixel over a plurality of frames, and an average of integrated pixel outputs in peripheral pixels of the detection target pixel. Takes the difference between the adjacent pixel average value calculation means to calculate, the integrated pixel output value of the detection target pixel and the average integrated pixel output value of the surrounding pixels, and blinks the pixels whose difference value exceeds the criterion A peripheral pixel comparator that identifies a defective pixel, and defect correction means that replaces the output of the identified blinking defective pixel with the output of another pixel. By replacing the output of the defective blinking pixel with the output of another normal pixel, the defective blinking pixel can be regarded as a normal pixel.
JP 2006-319602 A

  However, the solid-state imaging device described in Patent Document 1 has the following problems because the output of the blinking defective pixel is replaced with the output of another normal pixel.

  There is a blinking defective pixel in the first area where the infrared ray from the subject A is incident, and the output of the pixel in the second area where the infrared ray from the subject B adjacent to the subject A is incident is used as an alternative to the output of the blinking defective pixel. In this case, when the difference between the infrared radiation amount of the subject A and the infrared radiation amount of the subject B is particularly large, the luminance of the portion corresponding to the blinking defective pixel in the image of the first region is the original luminance (subject The luminance greatly deviates from the luminance corresponding to the infrared radiation amount of A), and as a result, the image in the first region becomes an unnatural image. For example, when the infrared radiation amount of the subject A is larger than the infrared radiation amount of the subject B, a dark part is displayed in a bright image.

  Further, in the blinking defective pixel, there is a period in which the normal output state is set, and it is desirable to use the output of the blinking defective pixel during this period. In the solid-state imaging device described in Patent Document 1, correction processing is always performed in which the output of the blinking defective pixel is replaced with the output of another normal pixel. In addition, correction processing is performed. As described above, the solid-state imaging device described in Patent Document 1 cannot effectively use the luminance value at the time of normal output.

  An object of the present invention is to provide a defective pixel correction apparatus that solves the above-described problems, can effectively use a luminance value at normal output, and can suppress an unnatural image.

In order to achieve the above object, a defective pixel correction apparatus of the present invention is
A frame memory unit that sequentially stores image data indicating luminance values obtained from each of the plurality of pixels, which is supplied in units of frames from an infrared imaging unit including a plurality of pixels;
Each of the plurality of pixels is set as a determination target pixel, the luminance value in the current frame supplied from the infrared imaging unit, and the current frame stored in the frame memory unit with respect to the determination target pixel in terms of time A determination circuit that obtains a difference from a luminance value in each of a plurality of consecutive delay frames and examines a relationship between the difference and a threshold,
The determination circuit outputs the luminance value of the current frame as the output of the determination target pixel as it is when the relationship satisfies the determination condition indicating normal output, and the relationship satisfies the determination condition indicating abnormal output. As an output of the determination target pixel, instead of the luminance value of the current frame, a luminance value that is a normal output among the luminance values of the plurality of delay frames is output.

  According to the present invention, when it is determined that the output of the blinking defective pixel is a normal output, the luminance value in the current frame is output as it is, so that effective information can be used to the maximum, and the better Images can be provided.

  In addition, when it is determined that the output of the blinking defective pixel is an abnormal output, the luminance value of the frame that has been normally output in the delay frame is output instead of the luminance value of the current frame, resulting in an unnatural image. Since this can be suppressed, a better image can be provided.

  Next, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a schematic configuration of a defective pixel correction apparatus according to an embodiment of the present invention. Referring to FIG. 1, the defective pixel correction device is applied to an infrared imaging device, and the main part thereof is a frame memory unit 4 and a determination circuit 5 that receive a luminance signal from the infrared imaging unit 1. Consists of.

The infrared imaging unit 1 includes an infrared detector 2 composed of a plurality of pixels arranged two-dimensionally, and a preprocessing circuit 3 that converts a luminance signal (analog signal) output from the infrared detector 2 into a digital signal. Have An existing infrared detector can be used as the infrared detector 2. In the infrared detector 2, a luminance signal corresponding to the amount of input infrared light is output for each pixel. The output order of the luminance signal of each pixel is determined in advance. The luminance signal converted into a digital signal by the preprocessing circuit 3 is supplied to the frame memory unit 4 and also to the input I ₀ of the determination circuit 5.

The frame memory unit 4 is a means for delaying the image data (luminance signal of each pixel) supplied from the preprocessing circuit 3 with a time corresponding to one frame period as a delay unit time, and is connected in series. The frame memory 4 ₁ to 4 _M. The frame memories 4 ₁ to 4 _M are composed of frame memories that store image data for one frame (luminance value of each pixel). In this frame memory, image data is written and read out in the same order as the output order of the luminance signal of each pixel in the infrared detector 2.

The frame memory 4 ₁ stores the image data supplied (luminance value of each pixel) from the pre-processing circuit 3. When storing the luminance value of the pixel, the frame memory 4 ₁ supplies the image data stored immediately before to the frame memory 4 ₂ and the determination circuit 5.

The frame memory 4 ₂ inputs the image data from the frame memory 4 _1, when storing the image data, and supplies the image data had been stored immediately before the frame memory 4 ₃ and the determination circuit 5. Frame memory 4 ₃ to 4 _M-1 is also similar to the frame memory 4 _2, and inputs the image data from the previous stage of the frame memory, when storing the image data, subsequent image data had been stored immediately before To the frame memory and to the determination circuit 5.

The frame memory 4 _M receives the image data from the frame memory 4 _M-1 as input, and supplies the image data stored immediately before to the third input of the determination circuit 5 when storing the image data. . Image data from the frame memory 4 ₁ to 4 _M is fed to the input I ₁ ~I _M of each decision circuit 5. Due to the delay processing in the frame memory unit 4, the inputs I _{0 to} I _M of the determination circuit 5 include Nth frame, N−1th frame, N−2th frame,. The luminance signals of the (M-1) th frame and the NM frame are supplied at the same timing.

The determination circuit 5 uses each pixel constituting the infrared detector 2 as a determination target pixel, and determines whether the luminance signal supplied to the input I ₀ is a normal output for the determination target pixel. If it is determined that the output is normal or is unknown, the determination circuit 5 outputs the luminance signal supplied to the input I ₀ as it is. If it is determined that the output is abnormal, the determination circuit 5 checks the input that is a normal output among the other inputs I _{1 to} I _M , and outputs the luminance signal instead of the luminance signal of the input I _0. .

  Next, the defective pixel correction process by the determination circuit 5 will be described in detail.

FIG. 2 is a flowchart showing a procedure of defective pixel correction processing. Referring to FIG. 2, in step S <b> 10, it is determined whether or not the input I ₀ is a normal output based on the relationship between the luminance signal difference between each of the inputs I _{0 to} I _M and the threshold value A.

In this determination, whether the first condition that the difference between the luminance signal (luminance value) of the input I _{0 and each} of the luminance signals (luminance values) of the other inputs I _{1 to} I _M is greater than the threshold A is satisfied. Check for no. When the first condition is not satisfied, the determination result in step S11 is normal. When the first condition is satisfied, it is checked whether or not the second condition that all of the luminance signal differences between the other inputs I _{1 to} I _M are larger than the threshold A is satisfied. When the second condition is satisfied, the determination result in step S11 is unknown. If the second condition is not satisfied, the determination result in step S11 is abnormal.

If the determination result in step S11 is normal or unknown, the luminance signal of the input I ₀ is output as it is in step S12. If the determination result in step S11 becomes abnormal, in step S13, another output that is a normal output is determined based on the relationship between the difference in luminance signal between each of the other inputs I _{1 to} I _M and the threshold A. Determine the input. In step S14, instead of the luminance signal of the input I ₀ , the determined luminance signal of the other input is output.

  According to this defective pixel correction processing, when it is determined that the output of the blinking defective pixel is a normal output, the luminance signal in the current frame (Nth frame) is output as it is, so that effective information is used to the maximum. The image of the area including the blinking defective pixel does not become an unnatural image.

  In addition, when it is determined that the output of the blinking defective pixel is an abnormal output, the frame that is normally output in the delay frame (N-1 frame to NM frame) instead of the luminance signal of the current frame (N frame) Luminance signal (luminance value) is output. In general, the luminance signal (luminance value) of the corresponding pixel between temporally consecutive frames does not change greatly. Therefore, when the luminance value in the delayed frame one frame or several frames before is used instead of the luminance value of the current frame, the output of the blinking defective pixel does not greatly deviate from the original output. Therefore, it is possible to suppress the image of the area including the blinking defective pixel from becoming an unnatural image, and a better image can be obtained as compared with the case where the luminance signal of the blinking defective pixel is replaced with the luminance signal of another pixel. Can be provided.

  In the following, examples of the case where the number of frame memories is two and three are given as an embodiment of the blinking defective pixel correction apparatus of the present invention.

FIG. 3 shows a first embodiment of the blinking defective pixel correction apparatus of the present invention. This blinking defective pixel correction device has the same configuration as that shown in FIG. 1 except that the frame memory unit 4 includes two frame memories 4 ₁ and 4 ₂ connected in series.

The delay processing for the frame memory unit 4, the input I ₀ ~I ₂ of the decision circuit 5, the determination target pixel, N-th frame, N-1 th frame, N-2-th frame of the luminance signal at the same timing Supplied. In the determination circuit 5, defective pixel correction processing based on the inputs I _{0 to} I ₂ is performed. FIG. 4 shows a flowchart of the defective pixel correction process.

Referring to FIG. 4, the condition that the absolute value of the difference between input I ₀ and input I ₁ is greater than threshold A and the absolute value of the difference between input I ₀ and input I ₂ is greater than threshold A in step S20. It is determined whether or not the above is satisfied. If it is determined in step S20 that the condition is not satisfied (corresponding to the normal determination in step S11 of FIG. 2), input I ₀ is selected in step S21.

If it is determined in step S20 that the condition is satisfied, it is determined in step S22 whether or not the condition that the absolute value of the difference between the input I ₁ and the input I ₂ is greater than the threshold A is satisfied. In step S22, when it is determined that the condition is satisfied (corresponding to unknown determination in step S11 in FIG. 2) is, in step S23, selects the input I _0. If it is determined in step S22 that the condition is not satisfied (corresponding to the abnormality determination in step S11 in FIG. 2), the luminance signals of the inputs I ₁ and I ₂ are both regarded as normal outputs, and in step S24. The input I ₁ which is the luminance signal of the frame (N−1 frame) immediately before the current frame (N frame) is selected.

  Here, the reason for selecting the immediately preceding frame in step S24 will be briefly described. In general, in a luminance signal output in units of frames from an infrared imaging device, the luminance signal (luminance value) of the corresponding pixel between two temporally continuous frames does not change significantly. Therefore, when it is determined that the luminance signal is an abnormal output for the first time in the current frame and the luminance signal of another frame is used as the luminance signal of the current frame, the frame closest in time to the current frame is used as the other frame. desirable. The state determined not to satisfy the condition in step S22 is that the luminance signal is normal output up to the (N-1) th frame, and it is unknown whether the luminance signal of the Nth frame is abnormal output or normal. It corresponds to the state. In this state, it is desirable to use the luminance signal (luminance value) of the (N-1) th frame, which is the frame immediately before the Nth frame, instead of the luminance signal of the Nth frame that is regarded as an abnormal output.

FIG. 5 shows a second embodiment of the blinking defective pixel correction apparatus of the present invention. This blinking defective pixel correction device has the same configuration as that shown in FIG. 1 except that it includes _three frame memories 4 ₁ , 4 ₂ , and 4 ₃ in which a frame memory unit 4 is connected in series.

Due to the delay processing in the frame memory unit 4, the inputs I _{0 to} I ₃ of the determination circuit 5 include the Nth frame, the N−1th frame, the N−2th frame, and the N−3th frame for the determination target pixel. Luminance signals are supplied at the same timing. In the determination circuit 5, defective pixel correction processing based on the inputs I _{0 to} I ₃ is performed. FIG. 6 shows a flowchart of the defective pixel correction process.

Referring to FIG. 6, in step S30, the absolute value of the difference between input I ₀ and input I ₁ is greater than threshold A, the absolute value of the difference between input I ₀ and input I ₂ is greater than threshold A, and It is determined whether or not the condition that the absolute value of the difference between the input I ₀ and the input I ₃ is larger than the threshold A is satisfied. If it is determined in step S30 that the condition is not satisfied (corresponding to the normal determination in step S11 of FIG. 2), input I ₀ is selected in step S31.

If it is determined in step S30 that the condition is satisfied, the absolute value of the difference between the input I ₁ and the input I ₂ is greater than the threshold value A and the absolute value of the difference between the input I ₂ and the input I ₃ is determined in step S32. It is determined whether or not the condition that the absolute value of the difference between the input I ₁ and the input I ₃ is greater than the threshold A is greater than the threshold A. If it is determined in step S32 that the condition is satisfied (corresponding to the unknown determination in step S11 of FIG. 2), the input I ₀ is selected in step S33.

If it is determined in step S32 that the condition is not satisfied (corresponding to the abnormality determination in step S11 in FIG. 2), the absolute value of the difference between the input I ₁ and the input I ₂ is greater than the threshold A in step S34. Alternatively, it is determined whether or not the condition that the absolute value of the difference between the input I ₁ and the input I ₃ is larger than the threshold A is satisfied. In step S34, when it is determined that the condition is not satisfied, it is regarded that the input I ₁ and input I ₂ or input I ₃ to be normal output, in step S35, selects the input I _1. If it is determined in step S34 that the condition is satisfied, the input I ₂ is regarded as a normal output, and the input I ₂ is selected in step S35.

  The blinking defective pixel correction apparatus described above is an example of the present invention, and the configuration and operation thereof can be changed as appropriate without departing from the spirit of the invention. For example, in the apparatus shown in FIG. 1, the frame memory unit 4 and the determination circuit 5 may be provided in the infrared imaging unit 1.

  The present invention can be applied to all imaging apparatuses using a two-dimensional infrared detector. Specific infrared imaging devices to which the present invention is applied include, for example, security infrared cameras, aircraft cameras, missile guidance devices, and the like.

  In addition to the infrared imaging device, the present invention can also be applied to an imaging device typified by a CCD image sensor or a CMOS image sensor as long as the problem of blinking defective pixels occurs.

1 is a block diagram illustrating a schematic configuration of a defective pixel correction apparatus according to an embodiment of the present invention. It is a flowchart which shows the procedure of the defective pixel correction process performed with the defective pixel correction apparatus shown in FIG. It is a block diagram which shows the 1st Example of the blinking defect pixel correction apparatus of this invention. It is a flowchart which shows the procedure of the defective pixel correction process performed with the defective pixel correction apparatus shown in FIG. It is a block diagram which shows the 2nd Example of the blinking defect pixel correction apparatus of this invention. It is a flowchart which shows the procedure of the defective pixel correction process performed with the defective pixel correction apparatus shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Infrared imaging part 2 Infrared detector 3 Pre-processing circuit 4 Frame memory part 5 Judgment circuit

Claims (4)

A frame memory unit that sequentially stores image data indicating luminance values obtained from each of the plurality of pixels, which is supplied in units of frames from an infrared imaging unit including a plurality of pixels;
Each of the plurality of pixels is set as a determination target pixel, the luminance value in the current frame supplied from the infrared imaging unit, and the current frame stored in the frame memory unit with respect to the determination target pixel in terms of time A determination circuit that obtains a difference from a luminance value in each of a plurality of consecutive delay frames and examines a relationship between the difference and a threshold,
The determination circuit outputs the luminance value of the current frame as the output of the determination target pixel as it is when the relationship satisfies the determination condition indicating normal output, and the relationship satisfies the determination condition indicating abnormal output. A defective pixel correction apparatus that outputs a luminance value that is a normal output among luminance values of the plurality of delay frames, instead of the luminance value of the current frame, as an output of the determination target pixel.   The determination circuit outputs the determination target pixel when the absolute value of the difference between the luminance value in the current frame and the luminance value in each of the plurality of delay frames does not satisfy the first condition larger than the threshold value. Is the normal output, satisfies the first condition, and does not satisfy the second condition in which the absolute values of the differences between the luminance values in each of the plurality of delay frames are all greater than the threshold value. The defective pixel correction device according to claim 1, wherein the output of the determination target pixel is determined to be an abnormal output.   The determination circuit, when the relationship satisfies a determination condition indicating an abnormal output, the luminance value of the delay frame closest in time to the current frame among the luminance values to be the normal output in the plurality of delay frames The defective pixel correction apparatus according to claim 1, wherein an output of the determination target pixel is used. Sequentially storing, in a frame memory unit, image data indicating a luminance value obtained from each of the plurality of pixels, which is supplied in units of frames from an infrared imaging unit including a plurality of pixels;
Each of the plurality of pixels is set as a determination target pixel, the luminance value in the current frame supplied from the infrared imaging unit, and the current frame stored in the frame memory unit with respect to the determination target pixel in terms of time Obtaining a difference with a luminance value in each of a plurality of consecutive delay frames, and examining a relationship between the difference and a threshold;
When the relationship satisfies the determination condition indicating normal output, the luminance value of the current frame is directly output as the output of the determination target pixel;
When the relationship satisfies the determination condition indicating abnormal output, the luminance value to be a normal output among the luminance values of the plurality of delay frames instead of the luminance value of the current frame as the output of the determination target pixel Outputting a defective pixel correction method.

Images