JP2006319602A - Solid-state image pickup apparatus and method for detecting flicker defect - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To favorably detect flicker defect pixels of a solid-state image pickup device to be used in a solid-state image pickup apparatus such as an infrared camera, even in a period other than a calibration period. <P>SOLUTION: This apparatus is provided with an infrared solid-state image pickup device composed of a plurality of pixels arranged in two dimensions; a frame memory integrator which integrates the output of each of the pixels over a plurality of frames and calculates an integrated pixel output for each pixel; an adjacent pixel average value calculation means for calculating the average of the integrated pixel outputs of neighboring pixels of a detection object pixel; a neighboring pixel comparator which calculates the difference between the value of the integrated pixel output of the detection object pixel and the average value of the integrated pixel outputs of the neighboring pixels of the detection object pixel, and specifies the pixel as a flicker defect pixel, if the difference value exceeds a criterion set beforehand; and a defect correction means which substitutes an alternate value derived from the outputs of the other pixels, for the output of a defective pixel including a flicker defect pixel specified by the neighboring pixel comparator. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

The present invention relates to a solid-state imaging device such as an infrared camera using a solid-state imaging device as an imaging device, and a blinking defect detection method for detecting a blinking defective pixel generated in the solid-state imaging device.

In the solid-state imaging device, a blinking defect exists in addition to the fixed defective pixel. Here, the blinking defective pixel indicates a pixel that behaves like normal luminance in some cases, but outputs abnormal luminance in other cases, such as outputting abnormal luminance.
Since there is almost no periodicity or regularity in the luminance variation of the blinking defective pixel, it is difficult to detect by the same method as the fixed defective pixel detection.

As an example of a system for detecting such blinking defective pixels, a conventional technique is known in which a uniform temperature object is imaged and detected by luminance change and variation. (For example, see Patent Document 1)
JP 2003-298949 A (FIG. 1)

However, in the above-described method, in order to perform offset correction, sensitivity correction, fixed defect correction, and blinking defect correction, an object having a substantially uniform temperature is captured over almost the entire angle of view of the solid-state imaging device (hereinafter, Calibration).
In particular, with respect to detection of blinking defects, abnormal luminance is exhibited non-steadily, and the blinking defect appearance cycle is indefinite, and therefore blinking defective pixels having a cycle longer than the calibration time cannot be detected.
Further, in order to detect a flashing defect with a long cycle, it takes a long time of several hundred frames, and the original subject cannot be imaged during that time.
Further, since the position (x, y) of the generated defective pixel is not constant every time the power is turned on, calibration for a long time must be performed every time the power is turned on.

  The invention described in the present application has been made with the object of solving such problems, and calibration of detection of blinking defective pixels in a solid-state imaging device used in a solid-state imaging device such as an infrared camera is performed. The object is to enable detection outside of the period.

  The solid-state imaging device according to the present invention includes an infrared solid-state imaging device composed of a plurality of pixels arranged two-dimensionally, and an integration pixel obtained by integrating the output of the pixel over a plurality of frames for each of the pixels. A frame memory integrator for calculating an output; an adjacent pixel average value calculating means for calculating an average of the integrated pixel outputs in peripheral pixels of the detection target pixel; a value of the integration pixel output of the detection target pixel; and the detection target A peripheral pixel comparator that takes a difference from the average value of the integrated pixel output in the peripheral pixels of the pixel, and identifies the blinking defective pixel as a pixel with the difference value exceeding a predetermined criterion And defect correcting means for substituting the output relating to the defective pixel including the blinking defective pixel specified by the peripheral pixel comparator with an alternative value derived from the output of another pixel. Was to so that.

Since it is not necessary to perform blinking defective pixel detection during the calibration period, calibration for a long time is unnecessary, and the influence on the original imaging time is reduced.
In addition, when detecting a blinking defect that blinks in a long cycle, the flash cycle that can be detected during the calibration period is limited, but since it is always detected, it can be detected in a long cycle. It can also handle blinking pixel detection.

Embodiment 1 FIG.
Preferred embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same configurations in the embodiments, and the duplicate description is omitted.

FIG. 1 shows the configuration of an infrared camera 100 according to Embodiment 1 of the present invention.

The infrared camera 100 captures an infrared solid-state image sensor (hereinafter referred to as a solid-state image sensor) 1 and a solid when an object (not shown) having a substantially uniform temperature over almost the entire angle of view of the solid-state image sensor 1 is imaged. An offset correction unit 2 that corrects an output offset between pixels using the pre-correction pixel data 101 from the image sensor 1 and a sensitivity difference between pixels using a change in imaging output of each pixel due to a temperature change of the subject. Sensitivity correction means 4 for correcting the defect, blinking defective pixel detection means 50 for detecting the blinking defective pixel and specifying the position of the blinking defective pixel, and an output relating to the defective pixel including the blinking defective pixel is derived from outputs of other pixels. And defect correcting means 6 for correcting by substituting with the alternative value.

In an imaging apparatus such as an infrared camera 100 using the solid-state imaging device 1, processing such as (1) offset correction, (2) sensitivity correction, and (3) defect correction is performed from the solid-state imaging device 1 as shown in FIG. Is applied to the pixel data 101 before correction.
Of the light rays coming from the subject, the light rays belonging mainly to the infrared region are captured by the infrared solid-state imaging device 1.
The solid-state imaging device 1 is composed of a large number of pixels arranged two-dimensionally, and each pixel is assigned an address indicating the position (x, y) of each pixel, and each pixel receives an infrared light receiving intensity. The output according to is exhibited.
Each pixel output is subjected to correction processing in the order of (1) offset correction, (2) sensitivity correction, and (3) defect correction.

First, an output value shift (output offset) may occur between pixels. When data indicating the output offset is arranged according to the pixel arrangement of the solid-state imaging device 1, it can be seen what pattern the output offset between the pixels has on the pixel arrangement. A pattern that the output offset has on the pixel array is called an offset pattern.
The offset pattern can be obtained, for example, from the output of the infrared solid-state image pickup device 1 when an object having a uniform temperature is captured over almost the entire angle of view of the infrared solid-state image pickup device 1 as a subject. It is stored in a ROM (Read Only Memory) or the like at an initial stage.
The offset correction process is a process for removing or suppressing an output offset between pixels from the imaging output of the infrared solid-state imaging device 1 during normal use according to an offset pattern stored in advance in the offset correction data 3. This is executed by the correction means 2.
The offset pattern used for the offset correction process is called an offset correction pattern.
Further, the above-described processing for capturing an image of a uniform temperature object and storing the offset correction pattern obtained as a result in the offset correction data 3 is called calibration.

A sensitivity difference between pixels remains in the post-offset-corrected pixel imaging output 102 that has undergone the offset correction processing by the offset correction means 2.
In order to realize uniform output characteristics over the entire screen (all pixels), a sensitivity correction process for multiplying the post-offset-corrected pixel imaging output 102 through the offset correction means 2 by the sensitivity correction coefficient ΔVave / ΔV (x, y) This is executed by the correction means 4.
(X, y) is the pixel position, ΔV (x, y) is the output change due to temperature in the pixel indicated by (x, y), that is, the offset difference value, and ΔVave is the average value of the entire screen of ΔV (x, y). It is.
Note that ΔV (x, y) can be obtained from, for example, an offset pattern when an object having an arbitrary temperature T is used as a subject and an offset pattern when another object having an arbitrary temperature T + ΔT is used as a subject. ΔVave can be obtained by averaging ΔV (x, y) thus obtained over the entire screen (all pixels), and is stored in a ROM (Read Only Memory) or the like at an initial stage of manufacturing. .

The post-sensitivity pixel imaging output 103 that has undergone the offset correction process and the sensitivity correction process is further subjected to a defect correction process by the defect correction means 6.
The defect correction process is a process for supplying a replacement value for a defective pixel, which is a pixel that exhibits an abnormal luminance output, and correcting the defective pixel instead of the output of the pixel. As an alternative value, for example, an average output of surrounding pixels surrounding the defective pixel is used.

In order to perform this defect correction process, it is necessary to detect defective pixels. First, a detection method of fixed defective pixels among defective pixels will be described next.
For a fixed defective pixel, the pixel position (x, y) does not change. Therefore, in the sensitivity correction process, a pixel whose sensitivity correction coefficient ΔVave / ΔV (x, y) obtained for each pixel position (x, y) exceeds a predetermined threshold is detected as a defective pixel having abnormal sensitivity. can do. The position (x, y) of the defective pixel detected as the defect correction data is stored in the defect correction data 7 at the initial stage of manufacturing.
The defect correction means 6 executes the above-described defect correction process for the pixel at the position (x, y) stored as the defective pixel position in the defect correction data 7.

Next, a method for detecting a blinking defective pixel among defective pixels will be described with reference to FIGS.
FIG. 2 shows an image diagram of a target pixel and peripheral pixels around the target pixel when detecting a blinking defective pixel. FIG. 3 shows a detection flow for detecting a blinking defective pixel according to the first embodiment. 3 that are the same as the numbers in FIG. 1 correspond to the numbers in FIG.

In step S101 of FIG. 3, the frame memory integrator 8 performs, for each pixel position (x, y), in the frame memory in which the post-sensitivity-corrected pixel imaging output 103 subjected to output offset correction and sensitivity correction is written. The data is read out and frame integration is performed for the entire screen, and an integrated pixel output Vn (x, y) 111 is output for each pixel position. Here, as an example, the number of frame integrations n is n = 128.

In step S <b> 102, the peripheral pixel average value calculation unit 12 uses the integrated pixel output Vn (x, y) 111 to calculate the pixel position (x, y) of the pixels constituting the solid-state imaging device 1. Average luminance data Vave (x, y) 112 is calculated.
Here, the peripheral pixels are a plurality of pixels located around the target pixel to be examined as to whether it is a blinking defective pixel. As an example, when the position of the target pixel in FIG. 2 is (xi, yj) and the surrounding pixels are 3 × 3 pixels around the target pixel, the surrounding pixels are 8 pixels indicated by the surrounding dot portions. That means. In the example of FIG. 2, the peripheral pixel average value calculation means 12 calculates the average luminance of the surrounding eight pixels excluding the target pixel.

Next, in step S103, the luminance threshold value calculation means 13 is an average value threshold value that is a determination criterion for determining whether the luminance difference between the target pixel and the peripheral pixels is within a certain ratio based on the average luminance data Vave112 of the peripheral pixels. Calculate Assuming that the ratio at this time is the threshold coefficient ε, for example, when 50% is set as the threshold value with respect to the average value of surrounding pixels, the average value threshold = average luminance data Vave112 of the surrounding pixels × threshold coefficient ε (ε = 0.5) is calculated. Is done. The average value threshold value calculated here is output as average value threshold value data 113.

Next, in S104, the peripheral pixel comparator 9 compares the magnitude of the difference between the integrated pixel output Vn (x, y) 111 and the average luminance data 112 of the peripheral pixels with the average value threshold data 113 calculated in S103. To do. As a result of comparison, a pixel that exhibits an output that does not belong to the average value threshold data 113 is identified as a blinking defective pixel. As a result of the comparison, if it belongs to the average value threshold data 113, the process returns to step S102 and the next pixel is similarly determined.

Here, the blinking pixel is characterized in that it is generated in units of one pixel in principle. In a local area around the pixel, it is considered that the luminance is almost the same when a large subject is imaged. Therefore, it can be determined that the abnormal luminance is in blinking defective pixels for each pixel in the local area.
Furthermore, when the subject is capturing a small subject in units of one pixel, it is considered that the pixel address position will change with time if it is the original subject by making a determination based on the frame-integrated output in the time axis direction. It is unlikely that abnormal luminance will occur only at the pixel address, and on the contrary, in the case of a blinking pixel, abnormal luminance will occur only at a specific pixel address, so that it can be distinguished from the original subject.
Therefore, even during imaging, it is possible to detect a blinking defective pixel by always comparing it with the average pixel output value of the local area integrated with the frame.

The peripheral pixel comparator 9 outputs an address 115 indicating the position (x, y) of the pixel detected as the blinking defective pixel to the defect correction data 7.

In step S <b> 105, the blinking pixel address 115 output from the peripheral pixel comparator 9 is added to the defect correction data 7.
In step S106, it is determined whether or not the detection of the blinking defective pixel is to be ended, and in the case of continuing, the process returns to step S102.
In step S <b> 107, the defect correction unit 6 performs defect correction on the pixel whose position is indicated by the blinking pixel address 115. The defect correction processing is processing for correcting a defective pixel that is a pixel that exhibits an abnormal luminance output by supplying an alternative value instead of the output of the pixel. As an alternative value, for example, an average output of surrounding pixels surrounding the defective pixel is used.

According to this embodiment, it is possible to detect a blinking defect, which has been difficult to detect in the past, without performing long-time calibration.
Also, when detecting a blinking defect that blinks at a long time interval, it has been supported by increasing the number of imaging frames during calibration in the past, but it is always detected during the imaging period. It can also handle flashing defects that change in various cycles.
Furthermore, it is possible to detect not only blinking but also a fixed defect generated during imaging due to dust adhesion or the like.

Embodiment 2. FIG.
FIG. 4 shows the configuration of an infrared camera according to Embodiment 2 of the present invention.
The second embodiment has a configuration in which the peripheral pixel standard deviation calculating unit 11 and the standard deviation threshold calculating unit 14 are provided in the first embodiment.

The peripheral pixel standard deviation calculation means 11 calculates the standard deviation σ of the peripheral pixels for each pixel position (x, y) using the integrated pixel output Vn (x, y) 111.
The standard deviation threshold calculation unit 14 calculates a standard deviation threshold based on the standard deviation σ obtained by the peripheral pixel standard deviation calculation unit 11.

FIG. 5 shows a detection flow for detecting a blinking defective pixel according to the second embodiment. In the detection flow described in the first embodiment, steps S120 and S121 are added, and the determination condition performed in step S104 is changed. The detection flow of the blinking defective pixel according to the second embodiment will be described with reference to FIG. Note that the description of the same step numbers as those in Embodiment 1 is omitted.

In step S101, the frame memory integrator 8 outputs the integrated pixel output Vn (x, y) 111 to the peripheral pixel average value calculating means 12, the peripheral pixel comparator 9, and the peripheral pixel standard deviation calculating means 11.
In step S120, the peripheral pixel standard deviation calculating unit 11 uses the integrated pixel output Vn (x, y) 111 to determine the standard deviation σ of the peripheral pixels for each pixel position (x, y). The standard deviation σ is supplied to the standard deviation threshold calculation means 14 as the peripheral pixel standard deviation data 120.
In step S121, the standard deviation threshold calculation unit 14 uses the standard deviation σ obtained by the peripheral pixel standard deviation calculation unit 11 to calculate a standard deviation threshold that is a determination criterion for determining whether the pixel is a blinking defective pixel. . Then, the calculated standard deviation threshold value is output as standard deviation threshold value data 121 to the peripheral pixel comparator 9. In FIG. 3, the standard deviation threshold calculation means 14 calculates standard deviation threshold = 3 × σ as an example of the standard deviation threshold.
In step S122, the peripheral pixel comparator 9 first determines the peripheral pixel average luminance data 112 and the integrated pixel output Vn (x, y) 111 for each pixel obtained from the peripheral pixel average value calculation means 12, as in the first embodiment. Is determined to belong to the luminance threshold data 113 (= average luminance data Vave112 of the surrounding pixels × threshold coefficient ε) calculated by the luminance threshold calculation means 13 in step S103, and pixels that do not belong are detected.
Further, the peripheral pixel comparator 9 includes the average luminance data 112 of the peripheral pixels and the integrated pixel output Vn (x) for each pixel obtained from the peripheral pixel average value calculation unit 12 among the pixels not belonging to the luminance threshold data 113. , Y) It is determined whether the difference from 111 belongs to the value of the standard deviation threshold data 121 calculated in step S121, and pixels that do not belong are detected as blinking defective pixels.

As described above, in the second embodiment, not only the determination based on the average value that is the absolute value of the luminance output but also the determination based on the distribution of variation due to the luminance standard deviation is added, so that even a noisy image can be obtained. It is possible to detect a blinking pixel with high accuracy.

Embodiment 3 FIG.
FIG. 6 shows the configuration of an infrared camera according to Embodiment 3 of the present invention.
In the third embodiment, the infrared camera has a configuration in which a determination pixel number setting unit 15 is newly provided.

In the peripheral pixel comparator 9 of FIG. 6, whether or not the output of each pixel is within the reference range of the luminance threshold value and the standard deviation threshold value is compared and detected as in the second embodiment.
The continuous pixel number detecting means 15 receives the blinking pixel address data 115 from the peripheral pixel comparator 9. The continuous pixel number detecting means 15 checks whether or not the blinking defective pixels are adjacent to each other with respect to the position of the blinking defective pixel indicated by the blinking pixel address data 115. Count the number. The continuous pixel number detection means 15 determines whether the defective flashing pixel detection is valid or invalid based on the detected continuous number of defective flashing pixels.
In principle, the number of generated pixels of the blinking pixel is one pixel unit, and a plurality of pixels are not generated continuously. For this reason, the continuous pixel number detection means 15 judges that the detection result of the blinking pixel is valid only when only one pixel is detected independently and the blinking defective pixel is detected, and a plurality of other pixels are continuously detected. In the case, it is determined to be invalid.

As described above, in the third embodiment, the number of consecutive pixels detected as blinking defective pixels is checked, and only when only one pixel is detected independently and not a blinking defective pixel, the determination result is valid. As a result, the detection accuracy of the blinking defective pixel can be improved.

It is a block diagram which shows the structure of the infrared camera which concerns on Embodiment 1 of this invention. It is an image figure of the surrounding pixel at the time of detecting the blinking defect pixel which concerns on Embodiment 1 of this invention. It is a detection flowchart which detects the blinking defect pixel which concerns on Embodiment 1 of this invention. It is a block diagram which shows the structure of the infrared camera which concerns on Embodiment 2 of this invention. It is a detection flowchart which detects the blinking defect pixel which concerns on Embodiment 2 of this invention. It is a block diagram which shows the structure of the infrared camera which concerns on Embodiment 3 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Infrared solid-state image sensor, 2 Offset correction means, 3 Offset correction data, 4 Sensitivity correction means, 5 Sensitivity correction data, 6 Defect correction means, 7 Defect correction data, 8 Frame memory integrator, 9 Peripheral pixel comparator, 10 Timing Control circuit, 11 peripheral pixel standard deviation calculating means, 12 peripheral pixel average value calculating means, 13 luminance threshold value calculating means, 14 standard deviation threshold value calculating means, 15 continuous pixel number detecting means, 20 lens, 21 shutter, 50 blinking defective pixel detection Means, 100 infrared camera, 101 pixel data before correction, 102 pixel imaging output after offset correction, 103 pixel imaging output after sensitivity correction,
110 frame memory control signal, 111 integrated pixel output Vn (x, y), 112 average luminance data of peripheral pixels, 113 average value threshold data, 114 comparison timing signal, 115 blinking pixel address data, 120 peripheral pixel standard deviation data, 121 Standard deviation threshold data.

Claims (6)

A solid-state imaging device composed of a plurality of pixels arranged two-dimensionally;
A frame memory integrator that integrates an output value output by the pixel for each pixel position over a plurality of frames to calculate an integrated pixel output;
A peripheral pixel average value calculating means for calculating an average value of the integrated pixel output values with respect to peripheral pixels located around each of the pixels;
A peripheral pixel comparator for detecting a blinking defective pixel based on a comparison result between the value of the integrated pixel output of each pixel and the average value calculated by the peripheral pixel average value calculating unit corresponding to the pixel; A solid-state imaging device comprising: A solid-state imaging device composed of a plurality of pixels arranged two-dimensionally;
A frame memory integrator that integrates an output value output by the pixel for each pixel position over a plurality of frames to calculate an integrated pixel output;
A peripheral pixel average value calculating means for calculating an average value of the integrated pixel output values with respect to peripheral pixels located around each of the pixels;
Luminance threshold value calculating means for calculating an average value threshold value based on the average value;
The difference between the integrated pixel output value of each pixel and the average value of the surrounding pixels corresponding to the pixel is taken, and the pixel blinks with a pixel whose difference value exceeds the average value threshold value. A solid-state imaging device comprising: a peripheral pixel comparator that determines that the pixel is a defective pixel. Peripheral pixel standard deviation calculating means for calculating a standard deviation of the value of the integrated pixel output with respect to the peripheral pixels;
Standard deviation threshold value calculation means for calculating a standard deviation threshold value using the standard deviation value,
The peripheral pixel comparator takes a difference between a value of the integrated pixel output of each pixel and an average value of the integrated pixel output in the peripheral pixel corresponding to the pixel, and the value of the difference is the standard deviation threshold value. 3. The solid-state imaging device according to claim 1, wherein a pixel that exceeds the threshold is determined to be a blinking defective pixel. 4. The continuous pixel number detecting means for determining that the result of the detection is valid when only one pixel of the blinking defective pixel is independently detected by the peripheral pixel comparator. The solid-state imaging device according to any one of 3. 5. The peripheral pixel is an eight pixel excluding the pixel among nine pixels composed of 3 vertical columns and 3 horizontal columns with the pixel at the center. The solid-state imaging device described in 1. A solid-state imaging device composed of a plurality of pixels arranged two-dimensionally, calculating an integrated pixel output by integrating an output value output by the pixel for each pixel position over a plurality of frames;
Calculating an average value of the integrated pixel output values for peripheral pixels located around each of the pixels;
The difference between the integrated pixel output value of each pixel and the average value of the integrated pixel output values in the surrounding pixels corresponding to the pixel is taken, and the difference value exceeds a preset criterion. And a step of determining that the pixel is a blinking defective pixel.

Images