JP4108278B2 - Automatic defect detection and correction apparatus for solid-state image sensor and imaging apparatus using the same - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an automatic defect detection and correction apparatus for a solid-state image sensor that detects a defective pixel present in a solid-state image sensor and corrects the defective pixel.
[0002]
[Prior art]
In a solid-state imaging device such as a CMOS image sensor and a CCD (Charge Coupled Device) formed on a semiconductor substrate, defective pixels may occur due to local crystal defects on the semiconductor substrate. Such a defective pixel has a signal output level that is output from the defective pixel so that the signal output level does not depend on the amount of incident light. There is a problem that the image quality of the image obtained by imaging deteriorates.
[0003]
In conventional video camera and digital camera imaging devices, a defective pixel existing in a solid-state imaging device is detected, address data related to the defective pixel is stored in a storage device such as a nonvolatile memory, and this nonvolatile The signal output from the defective pixel is corrected based on the address data of the defective pixel stored in the memory.
[0004]
For example, in the case of an image pickup apparatus equipped with a CMOS image sensor, a signal output from each pixel of the CMOS image sensor and a pixel to be inspected while the lens of the image pickup apparatus is shielded in the manufacturing stage and no light is incident on the CMOS image sensor. When the difference between the signal output levels exceeds a predetermined threshold, the pixel that outputs this specific signal output level is detected as a defective pixel. Then, the address data of the defective pixel is stored in the nonvolatile memory. When the detection of the defective pixel is completed, the CMOS image sensor and the nonvolatile memory storing the address data of the defective pixel are paired and shipped in the imaging apparatus. When a user takes an image using this imaging device, the defective pixel in the video signal output from the CMOS image sensor is determined based on the address data of the defective pixel of the CMOS image sensor stored in the nonvolatile memory. The corresponding signal is corrected by the signal of the pixel near the defective pixel.
[0005]
[Problems to be solved by the invention]
However, as described above, in the defective pixel correction of the solid-state imaging device, the defective pixel is detected in a state where the lens of the imaging device is shielded in the manufacturing stage and no light is incident on the CMOS image sensor or the like. Address data etc. must be stored in memory. For this reason, there exists a problem that the man-hour in a manufacture stage increases. Another problem is that defective pixels of the CMOS image sensor that are caused by electrostatic breakdown or the like cannot be corrected after the product of the image pickup apparatus equipped with the CMOS image sensor is shipped from the factory.
[0006]
In order to solve such a problem, in Japanese Patent Laid-Open No. 6-6665, when the power of the image pickup apparatus is turned on, the aperture of the lens mounted on the image pickup apparatus is closed to be in a light shielding state, and a defect is detected by the image pickup output signal of the solid-state image pickup device. There has been disclosed a defect correction apparatus that detects pixels, records and holds defect data based on detection signals from the defective pixels, and corrects the defective pixels using the latest defect data at the time of photographing.
[0007]
However, since the defect correction apparatus disclosed in Japanese Patent Laid-Open No. 6-6585 stores defective pixel address data and the like in a memory, a storage device such as a memory for storing defective pixel address data in the imaging apparatus is provided. It is necessary to install. Another problem is that the number of defective pixels that can be corrected depends on the storage capacity of a storage device such as a nonvolatile memory that stores address data of the defective pixels.
[0008]
The present invention solves such problems, and its purpose is to increase the number of man-hours during production, and to store defective pixel address data and the like for defective pixels generated after shipment. It is an object of the present invention to provide an automatic defect detection / correction device for a solid-state imaging device that can detect and correct without mounting the storage device.
[0009]
[Means for Solving the Problems]
An automatic defect detection and correction apparatus for a solid-state image sensor according to the present invention is an automatic defect detection and correction apparatus for a solid-state image sensor that detects defective pixels in a solid-state image sensor and corrects the detected defective pixels. A difference in signal output level with a plurality of pixels around the predetermined pixel is calculated as a defective pixel detection value, and the calculated defective pixel detection value is a preset high-level first defect detection threshold value. If it is larger, the predetermined pixel is determined to be a defective pixel, and if the calculated defective pixel detection value is smaller than the first defect detection threshold and greater than a preset second defect detection threshold at a medium level. Defect detecting means for determining the predetermined pixel as a defective pixel, and defect correcting means for correcting a signal output level from the defective pixel detected by the defect detecting means.
[0010]
The defective pixel detection value is a difference between a signal output level of a predetermined pixel and a maximum value of signal output levels of a plurality of surrounding pixels.
[0011]
The second defect detection threshold is a value obtained by multiplying a peripheral pixel index, which is a difference between the maximum value and the minimum value of the signal output levels of a plurality of surrounding pixels, by a predetermined value.
[0012]
The defect correcting means replaces the signal output level from the defective pixel with an average value of the signal output levels from pixels near the defective pixel.
[0013]
The imaging device of the present invention has the automatic defect detection and correction device for a solid-state imaging device according to any one of claims 1 to 4.
[0014]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0015]
FIG. 1 is a schematic block diagram of an imaging apparatus equipped with an automatic defect detection / correction apparatus for a solid-state imaging device according to an embodiment of the present invention. The image pickup apparatus shown in FIG. 1 includes a lens 1 that forms an image of a subject and a CMOS image sensor 2 that is a solid-state image pickup device, and an output of the CMOS image sensor 2 is input to an automatic defect detection and correction device 12. The automatic defect detection / correction device 12 has a defect detection circuit 4 to which the output of the CMOS image sensor 2 is directly input, and the output of the CMOS image sensor 2 performs scanning in the horizontal scanning period. It is input via a 2H line delay device 3 for delaying. The output of the defect detection circuit 4 for detecting defective pixels is given to a defect correction circuit 5 for correcting defective pixels, and the output of the defect correction circuit 5 is given to a digital signal processing circuit 6. The digital signal processing circuit 6 performs various signal processing on the input signal and outputs it as a YUV digital signal.
[0016]
Incident light obtained from the subject is input to the CMOS image sensor 2 via the lens 1. This incident light is photoelectrically converted into an electrical signal in the CMOS image sensor 2, and the electrical signal is subjected to correlated double sampling processing (CDS: Correlated Double Sampling), auto gain control (AGC: automatic gain control), A / D. It is output as a digital signal through a circuit (not shown) in which processing such as conversion is performed. This digital signal includes a signal from the defective pixel when the CMOS image sensor 2 has a defective pixel. Assuming that the line where the defective pixel exists is A, this signal A is given to the defect detection circuit 4 in a state where it has been subjected to digital processing, and from the line where the defective pixel exists via the 2H line delay unit 3 for two horizontal periods. The delayed signal B is given to the defect detection circuit 4.
[0017]
The defect detection circuit 4 detects a defective pixel by obtaining a difference between a signal level output from the defective pixel and a signal level output from surrounding pixels, and a signal output from the detected defective pixel is: Correction is performed based on a signal input to the defect correction circuit 5 and output from a pixel adjacent to the defective pixel, and is output as a YUV digital signal through the digital signal processing circuit 6.
[0018]
FIG. 2 is a configuration diagram of a color filter of the CMOS image sensor 2 used in the automatic defect detection / correction apparatus according to the embodiment of the present invention. R is a red transmitting filter 7, B is a blue transmitting filter 8, G is a green transmitting filter 9, and the red transmitting filter 7 and the green transmitting filter 9 are alternately arranged in the horizontal direction (x direction). The first filter row 10 that repeats the above and the second filter row 11 that alternately repeats the green transmission filter 9 and the blue transmission filter 8 in the horizontal direction are alternately arranged in the vertical direction (y direction). . G of the first filter row 10 and G of the second filter row 11 are arranged so as not to overlap in the vertical direction. This filter array is generally used as a Bayer array.
[0019]
In FIG. 2, attention is paid to a pixel corresponding to the red transmitting filter 7 disposed in the center of the first filter row 10, and the pixel of interest is X and the signal output level is P. In the first filter row 10 including the pixel of interest X, the portions of the color transmitting filter 7 located on both sides of the pixel are the pixels 2 and 3, the respective signal output levels are P2 and P3, respectively, The pixels of the red transmission filter 7 located on both sides of the pixel 3 are assumed to be pixels 1 and 4, and the respective signal output levels are P1 and P4, respectively. In addition, the pixel of the red transmission filter 7 disposed above the target pixel X in the first filter column 10 disposed with the second filter column 11 sandwiched above the first filter column 10 having the target pixel X. Let the pixel 5 and its signal output level be P5.
[0020]
Here, the pixel delay will be described with reference to FIG. A signal A that is output from the CMOS image sensor 2 and includes a pixel that is detected as a defective pixel is a signal of the L1 line that is the first filter row 10 including the pixel 1, the pixel 2, the pixel 3, the pixel 4, and the pixel X. The 2H line delay unit 3 outputs a signal B that is delayed for two horizontal periods from the signal A in which defective pixels, which are signals of the L1 line, are present. The signal B is a signal of a line L2 that is a horizontal filter row having the pixels 5.
[0021]
The defect detection circuit 5 detects the defective pixel. FIG. 3 is a flowchart showing a procedure of defect detection processing. With reference to FIG. 3, a case where a defect detection process is performed on the target pixel X existing in the color filter layout diagram of the CMOS image sensor 2 shown in FIG. 2 will be described.
[0022]
First, in step S1, the signal output levels of the pixel X of interest and the surrounding pixels 1 to 5 are compared, and the signal output level P of the pixel X is any one of the surrounding pixels 1 to 5. If it is smaller than the signal output level (P1 to P5), it is determined that the pixel X is a normal pixel, and the detection process ends.
[0023]
On the other hand, if the signal output level P of the pixel X is higher than all the signal output levels (P1 to P5) of the surrounding pixels 1 to 5, in step S2, the signal output levels (P1 to P5) of the pixels 1 to 5 are set. ) Is a difference between the peripheral pixel index FLAT, which is the difference between the maximum value and the minimum value, and the difference between the signal output level P of the pixel X and the maximum value of the signal output levels (P1 to P5) of the peripheral pixels 1 to 5. A pixel detection value DIFF is calculated.
[0024]
In step S3, it is detected whether the target pixel X is a defective pixel. In this case, it is assumed that the signal output levels of the target pixel X and the surrounding pixels 1 to 5 have the relationships shown in FIGS. FIG. 4A is a graph showing a comparison of signal output levels between the pixels 1 to 4 arranged in the horizontal direction (x direction) with respect to the position of the pixel X and the pixel X, and FIG. These are graphs showing a comparison of signal output levels between the pixel 5 and the pixel X arranged in the vertical direction (y direction) with respect to the position of the pixel X. 4A and 4B, the maximum value of the signal output levels (P1 to P5) of the pixels 1 to 5 is P3 of the pixel 3, and the defective pixel detection value DIFF is a signal between the pixel X and the pixel 3. It is represented by the difference in output level (P-P3). FIG.4 (c) is a graph which shows each signal output level of each pixel 1-5, From this graph, the maximum value and minimum value of each signal output level (P1-P5) of pixels 1-5 are as follows. The maximum value is P3 of the pixel 3, the minimum value is P2 of the pixel 2, and the peripheral pixel index FLAT is represented by a difference in signal output level between the pixel 3 and the pixel 2 (P3-P2).
[0025]
Next, the first defect detection process when the signal output level from the target pixel X is high will be described. Defective pixel detection value DIFF (P-P3: FIG. 4A), which is the difference between the signal output level P of the pixel X of interest and the maximum value of the signal output levels (P1 to P5) of the surrounding pixels 1 to 5. Is larger than the first defect detection threshold USER_THR determined according to the user's request, it is determined that the pixel X of interest is a defective pixel. By setting the value of the first defect detection threshold USER_THR to a predetermined value or more, it is possible to reliably detect defective pixels having a high signal output level.
[0026]
However, in the case where the signal output levels of the pixels around the pixel X of interest are uniform without unevenness, even if the defective pixel detection value DIFF is smaller than the first defect detection threshold USER_THR, the defect is very noticeable visually. Pixel exists. In order to detect such defective pixels, the first defect detection threshold USER_THR may be further reduced. However, if the first defect detection threshold USER_THR is set to a small value, the signal output level of the pixel and the signal output level of the peripheral pixels are normal in a subject having a complex pattern with unevenness. There is a possibility that the defective pixel detection value DIFF, which is a difference from the maximum value, becomes larger than the first defect detection threshold USER_THR, and even a normal pixel is erroneously determined as a defective pixel.
[0027]
Detection of a defective pixel when the signal output level from the target pixel X is a medium level is performed as the second detection process. In the second detection process, the second defect detection threshold THR is used to detect defective pixels. The second defect detection threshold value THR is given by equation (1).
[0028]
THR = FLAT × REF_USR (1)
Here, FLAT is a peripheral pixel index described above, and REF_USR is a coefficient that can be arbitrarily set according to a user's request. When the defective pixel detection value DIFF, which is the difference between the signal output level of the pixel of interest X and the maximum signal output level of surrounding pixels, is greater than the second defect detection threshold THR, the pixel of interest X is detected as a defective pixel. To do. When the value of the peripheral pixel index FLAT is 0 (zero), it is considered that the image of the pixels around the pixel of interest X is a uniform image on the subject surface with no signal output level unevenness. Further, as the peripheral pixel index FLAT increases from 0, it can be estimated that the periphery of the target pixel X is an image having a complicated pattern with unevenness of the signal output level on the subject surface. When the periphery of the pixel of interest X is a uniform image without unevenness, the second defect detection threshold value THR may be set to a smaller value in order to detect more defective pixels. In the case where the periphery of the pixel X is a complex pattern with unevenness of the signal output level, the second defect detection threshold value THR may be set to a larger value in order to prevent erroneous determination of the defective pixel.
[0029]
For this reason, in step S4, the value of the coefficient REF_USR that can be arbitrarily set according to the user's request is set in the second defect detection threshold value THR shown in the equation (1), which is the detection amount of the defective pixel, and the second defect detection value is set. The value of the threshold value THR is calculated. In step S5, when the defective pixel detection value DIFF is smaller than the second defect detection threshold value THR, the pixel X is determined to be a normal pixel, and the defect detection process ends.
[0030]
On the other hand, if the defective pixel detection value DIFF is greater than the second defect detection threshold value THR, it is determined in step S6 whether the pixel of interest X is not the center of a subject such as a point light source. When the subject is like a point light source, the signal output level of the pixel X and the surrounding pixels 1 to 4 is the highest in the signal output level from the pixel X as shown in FIG. The signal output level is considered to decrease as the distance from X increases. Accordingly, the signal output levels of the peripheral pixels 1 and 2 satisfy the inequality P1 <P2, and the signal output levels of the peripheral pixels 3 and 4 satisfy the inequality P4 <P3. Therefore, when the condition of Pl> P2 or P3 <P4 is not satisfied, it is determined that the pixel X is the center of the subject such as a point light source and is a normal pixel, and the detection process ends.
[0031]
Next, when either P1> P2 or P3 <P4 is satisfied, in step S7, the difference in signal output level between the pixel 2 and the pixel 1 (P1−P2), or the pixel 3 and the pixel 4 Signal output level difference (P4-P3) is smaller than the third defect detection threshold LIGHT determined in response to the user's request, the periphery of the pixel X is uniform with no irregularities in the signal output level. Although it is a pattern, it is considered that the signal output level of the pixel X protrudes from the signal output levels of the surrounding pixels 1 to 4, so that the pixel X is determined to be a defective pixel. The difference in signal output level between the pixel 2 and the pixel 1 (P1-P2) or the difference in signal output level between the pixel 3 and the pixel 4 (P4-P3) is larger than the third defect detection threshold LIGHT. In this case, it is determined that the pixel X is a normal pixel, and the defect detection process ends.
[0032]
Thereafter, in the defect correction circuit 6, when the target pixel X is determined to be a defective pixel in the defect detection circuit 5, the signal output levels of the pixels 2 and 3 of the same color filter around the target pixel X in real time. The average value Poave is obtained from the equation (2).
[0033]
Poave = (P2 + P3) / 2 (2)
Then, the defective pixel is corrected by replacing the signal output level P output from the defective pixel with the average signal output level Poave obtained by the equation (2). Then, by performing two detection processes of the first detection process and the second detection process for all the effective pixels of the CMOS image sensor 2, a defective pixel having a high level signal output level and a medium level signal are detected. A defective pixel having an output level can be automatically detected with high accuracy, and the defective pixel can be corrected.
[0034]
Therefore, in the automatic defect detection / correction device for a solid-state imaging device according to the present invention, a defect detection circuit detects a defective pixel without using a storage device for storing defective pixel address data and the like, and then continuously detects the defect. Correction of the signal output level of the defective pixel by the correction circuit can be easily and accurately performed.
[0035]
In addition, in the imaging apparatus equipped with the automatic defect detection and correction apparatus for the solid-state imaging device of the present invention, since there is no storage device such as a memory, there is no restriction on the number of defective pixels that can be corrected by the storage capacity. It is possible to correct a defective pixel caused by a change with time after shipment, and convenience is improved.
[0036]
【The invention's effect】
The automatic defect detection and correction apparatus for a solid-state imaging device according to the present invention calculates a difference in signal output level between a predetermined pixel and a plurality of pixels around the predetermined pixel as a defective pixel detection value, and calculates the calculated defective pixel If the value is larger than a preset high-level first defect detection threshold, the predetermined pixel is determined as a defective pixel, and the calculated defective pixel detection value is preset smaller than the first defect detection threshold. Defect detection means for determining a predetermined pixel as a defective pixel if it is greater than the medium level second defect detection threshold, defect correction means for correcting a signal output level from the defective pixel detected by the defect detection means, By detecting the defective pixel generated after manufacture and shipment without mounting a storage device such as a memory, the defective pixel is continuously corrected after that. Door can be.
[0037]
In addition, the imaging apparatus according to the present invention eliminates the restriction on the number of defective pixels that can be corrected by the storage capacity, and can correct defective pixels caused by a change with time after manufacture and shipment, thereby improving convenience.
[0038]
[Brief description of the drawings]
FIG. 1 is a schematic block diagram of an automatic defect detection and correction apparatus for a solid-state imaging device according to an embodiment of the present invention.
FIG. 2 is a layout diagram of color filters of a CMOS image sensor used in the automatic defect detection and correction apparatus.
FIG. 3 is a flowchart illustrating a defect pixel detection process procedure according to the present invention.
4A and 4B are graphs showing a comparison of signal output levels between a pixel X and pixels 1 to 5 around the pixel X. FIG. (C) is a graph which shows the comparison of the signal output level of the pixels 1-5.
FIG. 5 is a graph showing a comparison of signal output levels of the pixel X and the pixels 1 to 4 when the subject is a point light source.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Lens 2 CMOS image sensor 3 2H line delay device 4 Defective pixel detection circuit 5 Defective pixel correction circuit 6 Digital signal processing circuit 7 Red transmission filter 8 Blue transmission filter 9 Green transmission filter 10 First filter row 11 Second filter row 12 Automatic defect detection and correction device

Claims (4)

A color filter in which a plurality of types of color transmission filters are provided in a matrix with a predetermined arrangement, and pixels arranged in the horizontal and vertical directions so as to correspond to the color transmission filters in the color filter, respectively, are provided. An automatic defect detection and correction device for a solid-state imaging device that is provided in an imaging device having a solid-state imaging device and detects a defective pixel of the fixed imaging device and corrects the detected defective pixel,
An output signal of all pixels in the first line along the horizontal direction that is output from the solid-state imaging device in one horizontal scanning period, and is output from the solid-state imaging element with a delay of at least one horizontal scanning period from the one horizontal scanning period. Output signals of all the pixels on the second line along the horizontal direction, and the output signal of the target pixel included in the first line and the color corresponding to the target pixel on the first line. From a pixel corresponding to the same color transmission filter as the transmission filter, a pair of first and second pixels closest to the target pixel on both sides of the target pixel, and a pair of first and second pixels on both sides A pair of third and fourth pixels closest to the first and second pixels, and a color transmission filter corresponding to the target pixel at the same horizontal position as the target pixel in the second line; A fifth pixel corresponding to a color transmission filter of a color is selected, and the target pixel is a defective pixel based on first to fifth peripheral pixel signals that are output signals of the first to fifth pixels, respectively. A defect detection means for determining whether or not
A defect correction unit that corrects an output signal level of the target pixel determined as a defective pixel by the defect detection unit ;
When the level of the output signal of the target pixel is higher than all the levels of the first to fifth peripheral pixel signals, the defect detection means and the first to fifth peripherals A difference from the maximum value of the pixel signal is calculated as a defective pixel detection value, and if the calculated defective pixel detection value is larger than a preset first defect detection threshold, the target pixel is determined as a defective pixel, When the defective pixel detection value is smaller than the first defect detection threshold, the peripheral pixel index, which is the difference between the maximum value and the minimum value of the first to fifth peripheral pixel signals, is multiplied by a predetermined value. An automatic defect detection / correction device for a solid-state imaging device , wherein the target pixel is determined to be a defective pixel if it is larger than a set second defect detection threshold . The defect detection unit is configured such that the first pixel and the third pixel are located on the same side with respect to the target pixel, and the level of the first peripheral pixel signal is smaller than the level of the third peripheral pixel signal. Or the second pixel and the fourth pixel are located on the same side of the target pixel, and the level of the second peripheral pixel signal is smaller than the level of the fourth peripheral pixel signal The automatic defect detection and correction apparatus for a solid-state imaging device according to claim 1 , wherein if the condition is not satisfied, the target pixel is not determined as a defective pixel. The defect detection means has a condition that the first pixel and the third pixel are located on the same side with respect to the target pixel, and the level of the first peripheral pixel signal is smaller than the level of the third peripheral pixel signal. When satisfied, the second pixel and the fourth pixel are located on the same side with respect to the target pixel, and the level of the second peripheral pixel signal is smaller than the level of the fourth peripheral pixel signal. If satisfied, the difference between the level of the third peripheral pixel signal and the level of the first peripheral pixel signal or the difference between the level of the fourth peripheral pixel signal and the level of the second peripheral pixel signal is Determining that the pixel of interest is a defective pixel if it is smaller than a preset third defect detection threshold,
Said defect correcting means, when the pixel of interest is determined as a defective pixel is corrected so that the output signal level of the pixel of interest becomes the average value of the first and second peripheral pixel signal level, claim 2 The automatic defect detection and correction apparatus described in 1. The imaging device which has the automatic defect detection correction apparatus of the solid-state image sensor in any one of Claims 1-3 .

Images