JP2010016450A - Image capturing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image capturing apparatus capable of reducing continuous defects (=line defects) on an image for a plurality of pixels, improving correction accuracy, and reducing image quality degradation. <P>SOLUTION: The image capturing apparatus has an image sensor including two-dimensionally arrayed pixels 60 for photoelectrically converting and storing light from an object and a plurality of output lines 67 connected to the pixels 60, for transferring stored charges to the post stage as signals. Then, in the image sensor, the pixels 60 of all colors are connected to the output lines 67, and the output of the pixels 60 disposed in the same column or the same row is alternately output to the different output lines 67 adjacent on the right and left by multiple pixel. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image pickup apparatus using a solid-state image pickup device (hereinafter referred to as an image pickup device) such as a CCD or a CMOS image sensor, and more particularly to an image pickup device characterized by a technique for improving the image quality of a picked-up image.

  At present, electronic image processing apparatuses such as digital cameras that record and reproduce still images and moving images captured by an image sensor such as a CCD or CMOS image sensor using a memory card having a solid-state memory element as a recording medium are widely used. .

  FIG. 10 is a schematic configuration diagram of a conventional image sensor (CMOS image sensor).

  Each pixel 60 is arranged for each of four pixels of three colors of red (R), green (G1, G2), and blue (B) by a color filter (generally called a Bayer arrangement). Each pixel performs photoelectric conversion, and is output to an output line 67 at a predetermined timing by a switch unit or the like, and transferred to an AD conversion circuit or the like at a subsequent stage via an SN circuit block 75 and an output amplifier 74.・ Read out.

  Regarding the image, the color of the subject is reproduced by performing development processing using each color of the Bayer array.

  FIG. 11 is a circuit diagram of the pixel 60 of the image sensor of FIG.

  A photodiode (hereinafter referred to as PD) 61 receives a light image formed by a photographing lens (not shown) and generates and accumulates charges. The transfer switch (hereinafter referred to as TX) 62 is composed of a MOS transistor.

  A floating diffusion (hereinafter referred to as FD) 64 is a capacitor. The electric charge accumulated in the PD 61 is transferred to the FD 64 by the TX 62, converted into a voltage, and output from the amplifier 65 by the source follower.

  The row selection switch 66 outputs a pixel output to the vertical output line 67. The reset switch 63 resets the potential of the FD 64.

  By the way, in the imaging device, a defect for each pixel occurs in a manufacturing process or the like, and a pixel or the like that outputs an abnormal level may appear.

  In order to reduce the influence of defective pixels (defective pixels), in recent imaging apparatuses, pixel defect correction (defect correction) is generally performed.

  Regarding the determination of the defective pixel, mainly, the sensor output at the standard charge accumulation time under a predetermined condition is evaluated in advance, and it is determined as a defective pixel based on the evaluation result. The address of the defective pixel, its defect level, etc. The method of acquiring the data of is taken. This data describes the position data (x, y) of the defective pixel and its level.

  In the defect correction, the defective pixel data for which the defective pixel is determined is stored in the storage unit, the defective pixel is determined based on the data in the storage unit, and image data (pixels) of the same color pixel adjacent to the position of the defective pixel is detected. Output) is performed (for example, refer to Patent Documents 1 and 2).

  Thereby, it is possible to further reduce image quality degradation.

  For adjacent pixels used for interpolation, it is desirable to use adjacent pixels of the same color in both the vertical and horizontal directions around the top, bottom, left, and right (in some cases oblique directions) in order to perform natural interpolation (see FIG. 12). Gray pixels are defective pixels).

  However, since there are various causes for pixel defects, there may be a defect in which a plurality of pixels are connected depending on the location of the defect.

  For example, when a defect occurs in the output line, there is a case where a column in the vertical direction becomes defective in the conventional CMOS image sensor (in the case of a CCD image sensor, one horizontal row becomes defective). . Such a phenomenon is generally called a “line defect”.

  When interpolating (correcting) using peripheral pixels for a line defect, the adjacent pixels above and below (left and right in the case of a CCD image sensor) are also defective, so left and right (up and down in the case of a CCD image sensor) Interpolation with pixels only. For this reason, the interpolation accuracy is lowered, and the correction mark is easily noticeable as an image (see FIG. 13, the output line to which the gray pixel is connected is lost).

  If the output line is not a straight line with respect to the line defect, it is conceivable that the defect portion is dispersed as an image and becomes inconspicuous.

As described above, the output line is not a straight line, but the purpose is different, but the pixel arrangement of the even-numbered and odd-numbered pixels of the Bayer array is shifted by ½ pixel, and the vertical output line is between the shifted pixels. Has been proposed (see Patent Documents 3 and 4).
Japanese Patent Laid-Open No. 10-42201 JP 2003-333435 A Japanese Patent Application Laid-Open No. 07-235655 JP 2001-057420 A

  However, in the techniques proposed in Patent Documents 3 and 4, as shown in FIG. 14, when the defective output line is connected to the green (G1, G2) pixels, the green pixels for two columns of pixels are displayed. It becomes a defect, and the accuracy of defect interpolation is lowered.

  An object of the present invention is to provide an imaging apparatus capable of reducing continuous defects (= line defects) on an image with respect to a plurality of pixels, improving correction accuracy, and reducing image quality deterioration.

  In order to achieve the above object, an imaging apparatus according to claim 1 is arranged in a two-dimensional manner, and photoelectrically converts and accumulates light from a subject and charges accumulated by being connected to the pixels as signals. A plurality of output lines for transferring to the subsequent stage are provided, and the pixels of all colors are connected to the output line, and the outputs of the pixels arranged in the same column or the same row are adjacent to each other on the left and right sides for each of the plurality of pixels. An image pickup device that alternately outputs to different output lines is provided.

  According to the imaging apparatus of the present invention, it is possible to reduce consecutive defects (= line defects) on an image for a plurality of pixels, improve correction accuracy, and reduce image quality deterioration.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
FIG. 1 is a schematic configuration diagram showing a first example of a CMOS image sensor as an image pickup element used in the image pickup apparatus of the present invention.

  Each pixel 60 is arranged two-dimensionally (generally referred to as a Bayer array) every four pixels of three colors (all colors) of red (R), green (G1, G2), and blue (B) by a color filter. Has been.

  Specifically, the pixels 60 (1-1) [pixels in the first row and the first row] to 60 (mn) [pixels in the m column and the nth row] are arranged.

  Each pixel 60 (1-1) to 60 (m−n) accumulates photoelectrically converted signals, and is connected to the output line 67 [specifically, 67 (1) by a switch unit or the like at a predetermined timing. To 67 (m)] is output. Each pixel 60 is passed through an SN circuit block 75 [specifically 75 (1) to 75 (m)] and an output amplifier 74 [specifically 74 (a), 74 (b)]. Then, transfer / reading is performed to an AD converter circuit or the like in the subsequent stage.

  The pixels 60 are connected to different output lines 67 (1) to 67 (m) by two pixels in the vertical direction.

  In the present embodiment, for example, the pixels 60 (1-1) and 60 (1-2) output to the output line 67 (2), and the subsequent two pixels 60 (1-3) and 60 (1-4). Is output to the output line 67 (1). The subsequent two pixels 60 (1-5) and 60 (1-6) output to the output line 67 (2). As a result, the pixels arranged in the same vertical column are alternately output to different output lines adjacent to the left and right for each of a plurality of pixels (two pixels in the present embodiment).

  Each output line 67 is connected to all four pixels of three colors of color filters red (R), green (G1, G2), and blue (B).

  By adopting the above configuration, as shown in FIG. 2, the imaging apparatus of the present invention has a conventional defect “line defect” that occurs when an output line is lost, without occurring in the same column on the image. Since defective pixels occur in a dispersed manner, they are less noticeable.

  In addition, since the adjacent same color pixels of the defective pixel are highly likely to be normal pixels in the same column, the upper and lower sides and the left and right are highly likely to perform defect correction with high accuracy by performing interpolation using the adjacent adjacent same color pixels.

  Although the first embodiment of the present invention has been described with respect to an image sensor, the present invention is not limited thereto.

  FIG. 3 is a schematic configuration diagram of a CCD image sensor as an image pickup element used in the image pickup apparatus of the present invention.

  The CCD image sensor shown in FIG. 3 includes pixels 70 (1-1) to 70 (m−n), vertical transfer CCDs 71 (1) to 71 (m), a horizontal transfer CCD 72, and an output amplifier 73.

  Even in this configuration, the same effect can be obtained even when the pixels are output in different directions adjacent to the left and right of each pixel (every two pixels in FIG. 3) with respect to the vertical transfer CCD that hits the output line from the pixel. Obtainable.

  Furthermore, there is an image sensor having a configuration in which the pixel arrangement is shifted by 1/2 pixel as described in Patent Document 4 described above. However, such an image sensor also has a plurality of meandering output lines as shown in FIG. The same effect can be obtained by connecting to different output lines adjacent to the left and right for each pixel.

  FIG. 15 is a diagram illustrating a pixel array (a) of the conventional image sensor and a modification (b) of the pixel array of the image sensor of FIG.

(Second Embodiment)
The first embodiment is configured to output to a different output line for each of a plurality of pixels. Considering the premise of an image sensor configured individually for each unit pixel as shown in FIG. Not only every 2 pixels as in the example 1, but every 3 pixels, every 4 pixels, every 5 pixels, etc. can be set. However, in recent years, an increasing number of image sensors share a part of the circuit configuration of one pixel for the purpose of reducing the circuit area.

  The second embodiment shows an example in which a part of the pixel configuration is shared by a plurality of pixels.

  FIG. 4 is a schematic configuration diagram showing a second example of a CMOS image sensor as an image pickup element used in the image pickup apparatus of the present invention.

  FIG. 4 is an example showing a basic configuration of a pixel in which a part of the pixel configuration is shared by a plurality of pixels, and a part of the basic pixel configuration of FIG. 11 described above is shared by a plurality of pixels. .

  4A shows an example common to two pixels, and FIG. 4B shows an example common to four pixels.

  The unit pixels 60 (y) and 60 (y + 1) include PDs 61 (y) and 61 (y + 1), TX62 (y) and 62 (y + 1), and reset switches 63 (y) and 63 (y + 1) that perform charge accumulation, respectively. Prepared individually.

  On the other hand, the FD 64, the amplifier 65, and the row selection switch 66 shown in the common configuration 600 are shared by the pixels 60 (y) to 61 (y + 1).

  The unit pixels are pixels 60 (y) to 60 (y + 3) when the four pixels are common. PDs are PD61 (y) to 61 (y + 3) when the four pixels are common. Further, TX62 is TX62 (y) to 62 (y + 3) when the four pixels are common. In addition, the reset switches are reset switches 63 (y) to 63 (y + 3) in common for four pixels.

  FIG. 5 is a diagram illustrating an example of connection to an output line in the image sensor of FIG.

  FIG. 5A is common to two pixels, and is configured to output a set of two pixels in the vertical direction to the same output line, and the next two pixels output to different output lines.

  On the other hand, FIG. 5B is common to four pixels, and is configured to output a set of four pixels in the vertical direction to the same output line, and the next four pixels output to different output lines.

  As described above, in the second embodiment, the defect is connected to the same column on the image by connecting to a different output line for each number of pixels sharing a partial configuration in the circuit configuration of one pixel. In this case, defective pixels are generated in a dispersed manner, so that an effect of making the pixels inconspicuous is obtained.

  In this embodiment, when two pixels are common, the output is output to a different output line every two pixels, and when four pixels are common, the output is output to a different output line. However, the present invention is not limited to this.

  For example, every number of pixels with the minimum number of pixels sharing a part of the circuit configuration as a minimum unit (specifically, when two pixels are common, every four pixels or every six pixels with 2 pixels as the minimum unit) The same effect can be obtained even if the output line is changed.

  In other words, the continuous defect “line defect” that occurs when the conventional output line is lost does not occur in the same column on the image, and the defective pixels are dispersed and become inconspicuous.

(Third embodiment)
When the imaging elements of the first and second embodiments are used, the output lines do not match the pixel arrangement. Therefore, when reading in the order of the output lines, the order of the colors to be output differs depending on the rows. Come.

  For example, in FIG. 1, when the output line 67 (1), 67 (2), 67 (3)... 67 (m) is read out in this order, the first and second rows are output lines 67 (1). The output of the pixels R and G2 is made at the timing of the output line 67 (2).

  On the other hand, in the third and fourth rows, the pixels R and G2 are output at the timing of the output line 67 (1), and the pixels G1 and B are output at the timing of the output line 67 (2).

  Accordingly, there is no output depending on the row, or different color signals are output at the same timing. That is, the output timing shifts.

  The deviation of the output timing becomes a problem that the processing procedure becomes complicated in consideration of the Bayer arrangement in the development processing (white balance, sharpness, contrast, color conversion, etc.) performed using a plurality of pixels. .

  In addition, since all the electrical characteristics of the output lines cannot match, there is a slight difference in the reference potential for each output line. Although the influence of the reference potential shift can be reduced by performing shading correction, the timing at which shading due to the output line is shifted due to the shift in the output timing.

  For example, the output signals of the pixels R and G2 and the pixels G1 and B are generated in the order of the output lines 67 (2), 67 (3). Output signals of the pixels R and G2 and the pixels G1 and B are generated in the order of (1), 67 (2).

  For this reason, by connecting pixels arranged in the same column to different output lines, the data for performing shading correction needs to have correction data for a plurality of lines in accordance with the deviation in output timing, etc. Problems occur.

  The third embodiment relates to an image signal processing method and correction timing for dealing with the above problem in the imaging apparatus using the imaging device described in the first and second embodiments.

  FIG. 6 is an overall block diagram of the imaging apparatus according to the embodiment of the present invention.

  In FIG. 6, the internal configuration of the image sensor 1 is the same as that of FIG. The analog front end (AFE) 2 will be described later with reference to FIG. A digital front end (DFE) 3 receives the digital output of each pixel and performs digital processing such as correction and rearrangement. The DFE 3 will be described later with reference to FIG.

  The memory 4 stores various data. The image processing unit 5 corrects the output received from the DFE 3 and performs development processing. The control unit 6 controls various circuits. The operation unit 7 includes a shooting start switch, a power switch, various modes (AF mode, strobe mode, continuous shooting mode, etc.) changeover switch, and the like.

  The display unit 8 displays an image or the like that has been developed by the image processing unit 5. The recording unit 9 stores captured images and the like. The timing generator 10 outputs a drive timing signal to the image sensor 1, AFE2, DFE3, and the like.

  FIG. 7 is a block diagram of the AFE 2 in FIG.

  The gain control amplifier 21 performs sensitivity adjustment. The horizontal OB clamp unit 22 has a function of clamping optical black (OB), which is a light-shielded pixel arranged in the image sensor 1, as a reference level, and an offset between the OB output of each line of the image sensor 1 and the black level. The factor is gradually followed by a factor. This is intended to correct minute and gentle dark shading. The AD converter 23 converts the output of the gain control amplifier 21 from analog to digital.

  FIG. 8 is a block diagram of the DFE 3 in FIG. FIG. 9 is a diagram illustrating pixel output timing of the imaging apparatus of FIG.

  The readout timing signals of the output lines shown in FIG. 9A are sent to the image sensor 1 by the timing generator 10 shown in FIG.

  In FIG. 8, the horizontal shading data reading unit 31 reads shading data of dark output in a predetermined range of the image sensor 1 stored in advance in the memory 4. The shading data is data for correcting a deviation in units of output lines that cannot be corrected by the gentle dark shading that is corrected by the OB clamping operation of the AFE2.

  The horizontal shading correction unit 32 corrects horizontal shading using the shading data. As a result, shading as shown in FIG. 9C is corrected to a level that substantially matches the reference potential as shown in FIG. 9D.

  The array correcting unit 33 performs horizontal array correction on a row in which the output timing Bayer array shift occurs in the pixel output subjected to the shading correction by the horizontal shading correcting unit 32.

  Specifically, as shown in FIG. 9B, the pixel R is in the same row as the pixel R in the first row and the pixel R in the third row are shifted from each other. As described above, the arrangement of the data in the third row is shifted by one pixel (the fourth row is similarly shifted). The array correction unit 33 sends the image data with the matched Bayer array to the image processing unit 5. In the image processing unit 5, correction of defective pixels and development processing are performed.

  That is, in the present embodiment, both the functions of the horizontal shading correction unit 32 and the array correction unit 33 are provided in the DFE 3, and the Bayer array correction is performed after the shading correction.

  Due to the processing procedure of the present embodiment, since there is no deviation in shading, processing can be performed with only one line of shading correction data, and the Bayer arrangement is also the same as before with shading correction. For this reason, it is possible to avoid complication in image processing.

  In this embodiment, both the shading correction and the array correction are performed by the DFE 3. However, the present invention is not limited to this. For example, there is no problem even if other functional elements are used for shading correction / array correction depending on the configuration of the imaging apparatus, such as performing shading correction with AFE 2 and array correction with image processing unit 5.

  In this embodiment, since an output device connected to a pixel and outputting an output line is described in the vertical direction, the output of the pixels arranged in the same column is a plurality of pixels. An example of outputting to different output lines adjacent to each other on the left and right is given.

  However, for example, in the case where the output lines connected to the pixels are arranged in the horizontal direction as the element layout, the output of the pixels arranged in the same row is adjacent to the upper and lower sides of every plurality of pixels. By outputting to Similar effects can be obtained.

It is a schematic block diagram which shows the 1st example of the CMOS image sensor as an image pick-up element used for the imaging device of this invention. FIG. 2 is a schematic explanatory diagram of pixel arrangement and defect correction of the image sensor in FIG. 1. It is a schematic block diagram of the CCD image sensor as an image pick-up element used for the imaging device of this invention. It is a schematic block diagram which shows the 2nd example of the CMOS image sensor as an image pick-up element used for the imaging device of invention. It is a figure which shows an example of the connection to the output line in the image pick-up element of FIG. 1 is an overall block diagram of an imaging apparatus according to an embodiment of the present invention. It is a block diagram of AFE2 in FIG. It is a block diagram of DFE3 in FIG. It is a figure which shows the pixel output timing of the imaging device of FIG. It is a schematic block diagram of the conventional image pick-up element (CMOS image sensor). It is a figure which shows the unit pixel structure of the image pick-up element of FIG. FIG. 11 is a schematic explanatory diagram of pixel defect correction of the image sensor of FIG. It is a figure which shows an example of the pixel defect (line defect) of the image pick-up element of FIG. It is a figure which shows the other example of the pixel defect (line defect) of the image pick-up element of FIG. It is a figure which shows the pixel array (a) of the conventional image sensor, and the modification (b) of the pixel array of the image sensor of FIG.

Explanation of symbols

1 Image sensor 2 Analog front end (AFE)
3 Digital Front End (DFE)
60 pixels 67 output line 74 output amplifier 75 SN circuit block

Claims (3)

  A pixel that is two-dimensionally arranged and photoelectrically converts and accumulates light from a subject, and a plurality of output lines that are connected to the pixel and transfer the accumulated charge as a signal to a subsequent stage, and the output lines are all colors And an image sensor that alternately outputs the outputs of the pixels arranged in the same column or in the same row to different output lines adjacent to each other on the left and right sides. An imaging device.   The plurality of pixels is a pixel sharing a part of a configuration in a circuit configuration of one pixel or a pixel having a minimum unit of a pixel sharing a configuration of the circuit configuration. The imaging apparatus according to claim 1, wherein: Shading correction means for correcting shading;
After correcting the shading by the shading correction means, an array correction means for performing an array correction on the row of pixels in which a shift in the Bayer array of output timing has occurred;
The imaging apparatus according to claim 1, further comprising:

Images