JP2013106185A - Solid-state imaging device and image processing device using the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a correction method capable of reducing dark current unevenness of an image sensor and an offset for each output channel generated in clamping due to a defective pixel without increasing new noise.SOLUTION: An imaging device comprises: an image sensor that has a plurality of pixels; dark image acquisition means that acquires a dark image in a state where the image sensor is shielded from light; smoothing means that smooths the dark image in one row or column direction; and subtraction means that subtracts the smoothed dark image.

Description

  The present invention relates to a solid-state imaging device such as a digital camera or a video camera, and more particularly to noise correction of imaging data.

  A solid-state imaging device such as a CMOS image sensor is used in an imaging device such as a digital camera or a video camera.

  In a solid-state imaging device, there is a dark current generated due to factors such as temperature. For this reason, if the output signal of the image sensor is used as it is, the dark current component is superimposed on the effective signal component, which causes deterioration of the image quality of the captured image. In particular, when the temperature of only a part of the image sensor increases due to the heat generated by the image sensor itself or the heat from the peripheral circuit, two-dimensional dark current unevenness occurs.

  The dark current varies depending on the use environment, exposure time, and the like. Therefore, immediately before or after the actual shooting operation, the shooting operation is performed in a state where the image sensor is shielded, and the captured image (dark image) obtained at that time is subtracted from the image obtained by shooting the subject, A technique for removing dark current components is described in Patent Document 1.

  Further, shot noise occurs in the imaged signal of each pixel. Since this shot noise is random noise, when a black image is subtracted from an image obtained by photographing a subject, random noise included in each image is superimposed and image quality deteriorates. Therefore, Patent Document 2 discloses a technique for removing dark current unevenness without increasing random noise by performing median filter processing on a dark image and then using it for subtraction processing.

JP 2003-333435 A JP 2007-180616 A

  By the way, the solid-state imaging device includes an optical black region (OB region) in which the photodiode portion is shielded from light in order to obtain a level signal (black reference signal) serving as a reference for the output signal. A signal obtained from the opening area where the photodiode portion is not shielded from light is clamped based on the signal level obtained from the OB area.

  In addition, in order to read out the output signal of the image sensor at high speed, a technique is well known in which a plurality of output paths of the image sensor are used to simultaneously read out a plurality of pixels. For this reason, in order to speed up the clamping process, a technique for performing each output path is well known.

  In a solid-state image sensor, a defective pixel with an abnormal dark current called white scratch may be formed during the manufacturing process. This defective pixel appears more prominently at higher temperatures and longer seconds.

  If the defective pixel is in the OB area, the output of the OB area is shifted for each channel due to the influence of the defective pixel at the time of high-temperature, long-second shooting, and is processed with an incorrect clamp level for each channel. As a result, there is a problem that a different offset occurs in the captured image for each channel.

  In the technique described in Patent Document 1, it is possible to remove fixed pattern noise such as dark current unevenness and the above-described offset between channels by subtracting a dark image from an image obtained by photographing a subject. Random noise gets worse.

  In the technique described in Patent Literature 2, dark current unevenness can be removed without deteriorating random noise during dark image subtraction processing by performing median filter processing on a dark image. However, the aforementioned inter-channel offset smoothes the dark image and cannot be removed.

  An image pickup device including a plurality of pixels, a dark image acquisition unit that acquires a dark image in a state where the image pickup device is shielded from light, a smoothing unit that smoothes the dark image in one direction of a row or a column, and the smoothing An imaging apparatus is provided that includes subtracting means for subtracting a subsequent dark image.

  According to the present invention, it is possible to effectively correct dark current unevenness and inter-channel offset while suppressing deterioration of image quality due to random noise.

Whole block diagram of imaging device in the present invention Schematic diagram of image sensor in the present invention Circuit of unit pixel of image pickup device in the present invention The figure explaining the output channel of the image pick-up element in this invention Block diagram of AFE in the present invention The flowchart figure of the noise correction process in this invention The figure explaining the smoothing process in Example 1 of this invention The figure explaining another smoothing process in Example 1 of this invention. The figure explaining the smoothing process area | region of the dark image in Example 2 of this invention. The figure explaining the smoothing process in Example 2 of this invention

  Hereinafter, the best mode for carrying out the present invention will be described in detail with reference to examples.

[Example 1]
FIG. 1 is an overall block diagram of a solid-state imaging device according to the present invention.

  Reference numeral 101 denotes a CMOS solid-state image sensor that captures an image formed by a photographing lens (not shown). An analog front end (AFE) 102 performs OB clamping and analog-digital conversion processing. A digital front end (DFE) 103 receives the digital output of each pixel and digitally processes image signal correction, pixel rearrangement, and the like. An image processing unit 104 performs various image processing such as predetermined pixel interpolation processing, color conversion processing, and noise correction processing on the digital output from the DFE 103. The present embodiment is particularly characterized by noise correction processing, and details thereof will be described later. The memory circuit 105 is a working memory for the image processing unit 104, and is also used as a buffer memory in continuous shooting or the like. Reference numeral 106 denotes a control circuit that controls the entire image pickup apparatus and incorporates a known CPU. Reference numeral 107 denotes an operation circuit that electrically receives an operation member in the digital camera. Reference numeral 108 denotes a display unit for displaying images and the like, for example, a TFT (Thin Film Transistor) type LCD (Liquid Crystal Display). The recording circuit 109 is a recording medium such as a memory card or a hard disk. A timing generation circuit (TG) 110 generates various timings for driving the image sensor 101.

  FIG. 2 is a configuration example of a pixel region of a CMOS type solid-state imaging device. As shown in FIG. 2, the solid-state imaging device of this embodiment includes an aperture pixel region 203, a horizontal optical black region (HOB) 201, and a vertical optical black region (VOB) 202. The aperture pixel region 203 accumulates and outputs charges generated according to incident light. The horizontal optical black region (HOB) 201 and the vertical optical black region (VOB) 202 are light-shielded regions provided adjacent to the aperture pixel region 203.

  FIG. 3 is an example of a circuit of unit pixels (for one pixel) of the CMOS image sensor. Reference numeral 301 denotes a photodiode that converts light into electric charge. Reference numeral 302 denotes a transfer switch, which is composed of a MOS transistor. This transfer switch 302 is controlled by a PTX pulse. Reference numeral 303 denotes a floating diffusion (FD) portion. Reference numeral 304 denotes a source follower MOS transistor that amplifies the voltage of the FD section. The unit pixels are arranged in a matrix, and the pixels in the same column are connected to a common vertical output line 307 via a selection switch 305. The selection switch 305 is controlled by a PSEL pulse. Reference numeral 308 denotes a common power supply VDD. Reference numeral 306 denotes a reset switch composed of a MOS transistor, which resets the PD 301 to the potential VDD via the FD 303 and the transfer switch 302. The reset switch 306 is controlled by a PRES pulse.

  FIG. 4 is a diagram for explaining an output channel of the CMOS solid-state imaging device. Reference numeral 300 denotes a unit pixel shown in FIG. Signals such as a common PRES, PTX, and PSEL pulse are input to each pixel from a vertical scanning circuit (not shown) for each row. The output of each pixel is output to the column circuit via the vertical output line 307 to which the current source load 401 is connected. Here, the pixel signals of the even columns are output to the column circuit (CH1) indicated by 402, and the pixel signals of the odd columns are output to the column circuit (CH2) indicated by 403. The pixel signals read by the column circuits 402 and 403 are sequentially transferred to the reading circuits 406 and 407 by the horizontal scanning circuits 404 and 405 and are read out to the outside. In this embodiment, the output channel has a two-channel configuration of CH1 and CH2. However, by increasing the number of output channels, higher-speed reading can be performed.

  FIG. 5 shows the internal configuration of the AFE 102. A gain control amplifier 501 is used for sensitivity adjustment. Reference numeral 502 denotes a horizontal OB clamp circuit for correcting gentle dark shading in the row direction so as to match the black level reference value. Offset correction is performed so that the difference between the output of the HOB area of each row and the black level reference value decreases. The correction here is fed back to the AMP 501. Reference numeral 503 denotes an analog-to-digital conversion circuit (AD), which applies a gain to the pixel signal from the image sensor 101 by an AMP 501 and a horizontal OB clamp circuit 502 so that the pixel output in the HOB area matches the black level reference value. Are converted into, for example, a 14-bit digital signal. Each block 501, 502, 503 of the AFE 102 is provided for each output channel of the image sensor, and high-speed processing can be performed by simultaneously processing the signals of each output channel.

  However, if there is a defective pixel with an abnormal dark current in the area used for the clamping process of the image sensor clamp, the output of the channel with the defective pixel will be shifted at the time of high-temperature, long-second shooting, resulting in an incorrect clamp level. May be processed. As a result, different offsets occur for each channel in the output image.

  Next, noise correction processing performed in the image processing unit 104 in FIG. 1 will be described. FIG. 6 is a flowchart showing a procedure of dark subtraction processing which is noise correction processing of the present invention.

  In step 601, a main image obtained by photographing a subject is acquired. In this photographing process, signal charges accumulated for a predetermined time are read from the image sensor 101, and image data after each process is performed by the AFE 102 and the DFE 103 is written in the memory circuit 105.

  Next, in step 602, it is determined whether correction by dark subtraction processing is performed. If the operation circuit 107 is set to perform dark subtraction processing, the process proceeds to step 603 to acquire a dark image. When the operation circuit 107 is set not to perform the dark subtraction process, the noise correction process is terminated without performing the dark image acquisition and the dark subtraction process, and a predetermined pixel interpolation process is performed in the image processing unit 104. And color conversion processing is performed.

  Further, step 602 may be configured to determine whether to perform correction from the output of a predetermined area of the captured image as follows. For example, a pixel region that does not have a photodiode is arranged in a part of the pixel region in FIG. 2, and a dark current reference value is calculated from a difference between an output of the OB region and an output of the pixel region that does not have a photodiode, Dark subtraction processing is performed when the dark current reference value exceeds a predetermined threshold.

  In step 603, electric charges are accumulated for the same time as the photographing process performed in step 601 while keeping the shutter closed, for example, in the light shielding state. The accumulated charge is read from the image sensor 101, and each process is performed by the AFE 102 and the DFE 103 to generate a dark image.

  In step 604, the dark image generated in step 603 is subjected to smoothing processing, which will be described later, to generate a dark image in which the influence of random noise is reduced. In addition, in order to distinguish from the dark image acquired at step 603, the dark image which performed the smoothing process is called a correction | amendment dark image.

  In step 605, the corrected dark image generated in step 604 is subtracted for each unit pixel from the captured main image of the subject acquired in step 601 to generate a subtracted main image. The corrected dark image signal to be subtracted here is a signal in which random components have been reduced by the smoothing process. Therefore, compared to the case where the captured dark image signal is subtracted as it is, the deterioration of image quality due to random noise is suppressed. Can do. Furthermore, since correction processing is performed by subtraction for each pixel unit, two-dimensional unevenness such as dark current unevenness can be corrected.

  When the dark subtraction process ends and the noise correction process ends, the corrected main image is recorded as image data in the recording circuit 109 after performing a predetermined pixel interpolation process and a color conversion process.

  In this embodiment, the dark image is acquired after the main image is captured. However, the dark image may be acquired before the main image is captured.

  FIG. 7 is a diagram for explaining the smoothing process in step 604. The pixel signal in the i-th row and the j-th column is represented as Sij. For example, when the smoothing process is performed on the pixel signal S33, the median value (median value) is calculated from signals of a total of five pixels S13, S23, S33, S43, and S53 in the same column as the target pixel. If S23 is the median value of the five pixels, the output after the smoothing process of S33, which is the target pixel, becomes the value of S23. The smoothing process may use the average value of the five pixels instead of the median value.

  By performing smoothing processing using pixels in the same column as the pixel to be processed in this way, it is possible to remove two-dimensional unevenness by dark subtraction processing, and clamp processing due to defective pixels with abnormal dark current, etc. It is possible to correct an offset between channels that sometimes occurs.

  In the smoothing process in step 604, as shown in FIG. 8, in addition to the pixels in the same column as the target pixel, pixels of the same output channel on the left and right of the target pixel may be used for the smoothing process. For example, when performing the smoothing process on the pixel signal of S33, in addition to S13, S23, S33, S43, and S53 in the same column as the target pixel, the pixels S11, S21, belonging to the same output channel on the left and right of the target image S31, S41, S51 and S15, S25, S35, S45, and S55 are used for the smoothing process. By doing so, the influence of random noise can be further reduced, and the offset between channels can be detected accurately.

[Example 2]
In this embodiment, a noise correction process when the AFE 102 performs clamping operations with different feedback gains using the pixel outputs of the VOB area 202 and the HOB area 201 will be described.

  First, the clamping operation of this embodiment will be described.

  First, while monitoring the signal output in the VOB area, feedback is performed with a large gain so that the output signal quickly reaches a predetermined reference level, and clamping operation (high-speed clamping) is performed. The clamping operation (low-speed clamping) is continued by feeding back to the area and feeding back with a small gain so that the output signal becomes a predetermined reference level at a low speed. The reason why high-speed clamping is first performed in the VOB region is to make it possible to clamp the OB pixel output to a predetermined level at a high speed even if the dark current is very large by making a large region used for clamping. The reason why the low-speed clamping is performed in the HOB region is to correct the shading in the vertical direction of the dark signal.

  The following problems may occur when performing the clamping process.

  When there is a defective pixel or the like having an abnormal dark current in the VOB area, the feedback level of the VOB clamp is large, so that the clamp level is greatly shifted due to the influence of the output of the defective pixel. As a result, a large inter-channel offset occurs in a row below the defective pixel. If this large channel-to-channel offset is within the VOB region, it can be immediately followed by high-speed clamping and returned to the reference level in several lines. However, if there is a defective pixel near the end of the VOB area, it is switched to a low-speed clamp in the HOB area immediately after a large channel offset occurs. Therefore, the low-speed clamp cannot follow the large offset between channels, and the offset between channels remains for a while in the row below the defective pixel.

  Therefore, in the image processing apparatus 104 of the present embodiment, the smoothing process for the dark image is performed as follows in order to correct the offset between channels by dark subtraction processing while suppressing deterioration of random noise. Note that the flowchart of this embodiment is the same as the flowchart of the first embodiment shown in FIG.

  In the present embodiment, the dark image acquired in step 603 is subjected to different smoothing processes in the three smoothing processing areas 801, 802, and 803 as shown in FIG. 9 to generate a corrected dark image.

  In the region 801 in which high-speed clamping is performed in the VOB, as shown in FIG. 10A, for the target pixel S33 of the smoothing process, signals of a total of three pixels S23, S33, and S43 in the same column are used. Calculate the median value. That is, in this region, the number of pixels used for the smoothing process is reduced so that a large offset between channels can be immediately followed. If the number of pixels used for the smoothing process is small, the random noise suppression effect is reduced. However, since the area 801 is an invalid area where no subject is captured, priority is given to the offset removal, and the number of pixels in the smoothing process is reduced.

  As shown in FIG. 10B, the smoothing process is performed in the region 802 in which a large inter-channel offset may remain when the low-speed clamping is performed and there is a defective pixel near the end of the VOB region. For the target pixel S33, a median value is calculated using signals of a total of five pixels S13, S23, S33, S43, and S53 in the same column. Since this area includes an opening area, the number of smoothing pixels is set so as to follow a large inter-channel offset while giving a random noise suppression effect to some extent.

  In the region 803 in which a large offset between channels does not occur due to the low-speed clamping, as shown in FIG. 10C, for the target pixel S33 of the smoothing process, S03, S13, The median value is calculated using a total of 7 pixel signals of S23, S33, S43, S53, and S63. That is, in the area 803 where the opening area occupies most, the random noise suppression effect is prioritized over following the large offset between channels, and the number of pixels in the smoothing process is increased.

  As described above, in a region where high-speed clamping is performed, the number of pixels used for the smoothing process is reduced so that the corrected dark image can immediately follow even when a large channel offset occurs. In the region where low-speed clamping is performed, a large offset between channels is less likely to occur. Therefore, the random noise suppression effect is prioritized and the number of pixels used for the smoothing process is increased.

  By subtracting the corrected dark image subjected to the smoothing process, dark current unevenness and inter-channel offset can be effectively corrected while suppressing deterioration in image quality due to random noise.

101: imaging device 102: analog front end 103: digital front end 104: image processing unit 105: memory circuit 106: control circuit 107: operation circuit 108: display circuit 109: recording circuit 110: timing generation circuit 201: HOB area 202: VOB region 203: opening region 300: unit pixel 301: photodiode 302: transfer MOS transistor 303: floating diffusion 304: source follower MOS transistor 305: selection MOS transistor 306: reset MOS transistor 307: vertical output line 308: common power supply VDD
401: current source loads 402, 403: column circuits 404, 405: horizontal scanning circuits 406, 407: readout circuit 501: gain control amplifier 502: horizontal OB clamp circuit 503: analog-digital conversion circuit

Claims (4)

An image sensor comprising a plurality of pixels, a dark image acquisition unit that acquires a dark image in a state where the image sensor is shielded from light, a smoothing unit that smoothes the dark image in one direction of a row or a column, and the smoothing An image pickup apparatus comprising: subtracting means for subtracting the dark image after the conversion processing. The image pickup device includes a plurality of output paths, and the smoothing processing unit performs a smoothing process using pixels in a column belonging to the same output path in the vicinity of the smoothing target pixel. The imaging device described. The imaging apparatus according to claim 1, wherein the smoothing processing unit has a different number of pixel units used for the smoothing process depending on a position in the dark image. The imaging apparatus according to claim 1, wherein the smoothing processing unit performs median processing or averaging processing.