JP2007208885A - Imaging unit and image sensor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging unit and image sensor which prevents the occurrence of white non-gradation pixel or color reproducibility defect by complementing a sensitivity difference between a white pixel and a color pixel and can perform photographing that is not inferior in image quality, either. <P>SOLUTION: An imaging unit and image sensor can be provided in which exposure times of a white pixel in which a luminance filter is disposed and of a color pixel in which a color filter is disposed, are individually controlled, so that an exposure time of a color pixel with a low sensitivity can be prolonged, the occurrence of white no-gradation pixel or color reproducibility defect is prevented by complementing a sensitivity difference between the white pixel and the color unit and photographing that is not inferior in image quality, either, can be performed. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging unit and an imaging apparatus, and more particularly, to an imaging unit and an imaging apparatus that improve image quality.

  In recent years, imaging devices such as digital cameras and digital video cameras are required to have high-resolution and high-definition performance of imaging elements used in imaging devices as multifunction, high image quality, and miniaturization progress. Therefore, higher pixels and higher density are accelerating.

  On the other hand, in an image sensor having a high pixel density, the photoelectric conversion amount per pixel decreases due to a decrease in the area per pixel, that is, the sensitivity decreases and the signal level of the pixel output decreases, and S / N ( (Signal / Noise) ratio is lowered. The reduction in the S / N ratio has a great influence on the image reproducibility and the image quality. Therefore, it is necessary to improve the sensitivity in order to realize high-quality image formation. In the field of image sensors, various techniques for improving sensitivity have been studied.

  For example, in a solid-state imaging device having photodiodes arranged vertically and horizontally, a luminance filter Y having a high light transmittance, such as a transparent filter and a white filter, is arranged on each photodiode arranged in a checkered pattern. The other photodiodes have a filter configuration in which conventional color filters R (red), G (green), and B (blue) are arranged to obtain color information. A method for improving luminance resolution independent of color information of a subject has been proposed by using a signal charge generated by a photodiode having a high luminance filter Y as a luminance signal (see, for example, Patent Document 1). ).

  However, in the method of Patent Document 1, there is a sensitivity difference of three times or more between a pixel in which the luminance filter Y is arranged (hereinafter referred to as white pixel) and a pixel in which the color filter is arranged (hereinafter referred to as color pixel). Therefore, under the same exposure conditions, white reproducibility in white pixels and poor color reproducibility due to insufficient sensitivity in color pixels occur.

On the other hand, even in an image sensor using a conventional color filter, a method of controlling white balance by changing the exposure time of a color pixel for each color of the color filter has been proposed (for example, see Patent Document 2).
JP 2003-318375 A JP 2003-60992 A

  However, the method disclosed in Patent Document 2 is a delicate exposure time control for white balance in an image sensor composed of normal color pixels, and the sensitivity of white pixels and color pixels having a sensitivity difference of three times or more. It is not assumed that the difference is corrected, and it is not possible to prevent the occurrence of poor color reproducibility due to overexposure in white pixels and insufficient sensitivity in color pixels.

  The present invention has been made in view of the above circumstances, and can compensate for a difference in sensitivity between a white pixel and a color pixel to prevent occurrence of overexposure and poor color reproducibility, and shooting that is not inferior in terms of image quality. It is an object of the present invention to provide an imaging unit and an imaging apparatus that can be used.

  The object of the present invention can be achieved by the following constitution.

In an imaging unit having a plurality of pixels arranged in a 1.2-dimensional matrix and having an imaging device that photoelectrically converts a subject image to generate imaging data,
A white pixel in which a luminance filter is disposed on some of the plurality of pixels of the image sensor;
A color pixel in which a color filter is disposed on a pixel other than the white pixel;
An imaging unit comprising: an exposure time control unit that separately controls an exposure time of the white pixel and an exposure time of the color pixel.

2. The image sensor is a CMOS image sensor,
The exposure time control means includes a white pixel control line that controls reading of the output of the white pixel;
2. The imaging unit according to 1, further comprising a color pixel control line that controls reading of the output of the color pixel.

3. The image sensor is a CCD type image sensor,
The exposure time control means includes a white pixel control signal for controlling transfer of an output of the white pixel;
2. The imaging unit according to 1, further comprising a color pixel control signal for controlling transfer of an output of the color pixel.

  4). 3. The imaging unit according to 2, wherein the exposure time control unit separately controls the white pixel control line and the color pixel control line.

  5. 4. The imaging unit according to 3, wherein the exposure time control unit separately controls the white pixel control signal and the color pixel control signal.

  An imaging apparatus comprising the imaging unit according to any one of 6.1 to 5.

7). Movie mode for shooting movies,
Brightness detection means for detecting the brightness of the subject,
7. The imaging apparatus according to claim 6, wherein the exposure time control means included in the imaging unit controls exposure times of white pixels and color pixels included in the imaging unit based on an output of the luminance detection means.

  8). The exposure time control means controls the exposure time of the color pixel to an integral multiple of the exposure time of the white pixel when the output of the brightness detection means shows a brightness lower than a predetermined brightness. The imaging device described in 1.

  According to the present invention, by separately controlling the exposure time of the white pixel in which the luminance filter is arranged and the color pixel in which the color filter is arranged, the exposure time of the low-sensitivity color pixel can be increased. Therefore, it is possible to provide an imaging unit and an imaging apparatus that can compensate for a difference in sensitivity between the color pixel and the color pixel to prevent occurrence of overexposure and color reproducibility and can perform photographing that is comparable in image quality.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In addition, in each figure, the same code | symbol shows the same or an equivalent part, and the overlapping description is abbreviate | omitted.

  First, a digital camera that functions as an imaging apparatus according to the present invention will be described with reference to FIG. FIG. 1 is a circuit block diagram showing an example of the internal configuration of the digital camera 1. In FIG. 1, an image of a subject Obj is formed on an image sensor 162 by a photographic lens 201, photoelectrically converted by the image sensor 162, and read out as image data 162k. The timing generator 166 controls the photoelectric conversion operation of the image sensor 162 and the reading operation of the photoelectrically converted image data 162k. For example, when a live view image used as a finder for determining the composition of shooting is displayed on the display unit 131, it is not necessary to read out image data of all the pixels of the image sensor 162, so depending on the internal configuration of the image sensor. By using various methods such as thinning readout and pixel addition readout, it is possible to reduce the amount of image data to be read and lighten the load on the subsequent circuit. Here, the imaging element 162 and the timing generator 166 function as an imaging unit in the present invention.

  The read imaging data 162k is amplified by an amplifier 163, and then subjected to preprocessing such as analog / digital conversion and noise removal by a front end processor (FEP) 164, and white balance processing and color processing are performed by an image processing unit 165. Image processing such as processing is performed and output as an image pickup signal 165a toward the image pickup control means 161. The above photographing operation is controlled by the imaging control unit 161 and the timing generator 166 under the control of the camera control unit 150.

  The output imaging signal 165a is temporarily recorded in the image memory 181 from the imaging control unit 161 via the camera control unit 150, and finally recorded on the memory card 182. A preview image for determining the composition at the time of shooting and an after-view image for confirming the shot image are displayed on the display unit 131 by the camera control unit 150.

  The camera control unit 150 calculates the exposure condition of the image sensor 162 using the imaging data 162k of the image sensor 162 such as the above-described live view image as photometric data, and controls the aperture 172 via the aperture control unit 171. A so-called electronic shutter function can be achieved by controlling the amount of light incident on the image sensor 162 and controlling the photoelectric conversion time of the image sensor 162 via the image capture controller 161 and the timing generator 166. Here, the imaging element 162, the imaging control unit 161, and the camera control unit 150 function as a luminance detection unit in the present invention.

  In addition, the camera control unit 150 receives an input from the operation unit 100 and controls the operation of each unit of the digital camera 1. The operation unit 100 includes an AF switch 101a that operates when the shutter button is half-pressed, a release switch 101b that operates when the shutter button is fully pressed, a power switch 111, a mode setting dial 112, and the like.

  Next, the arrangement of luminance filters and color filters in the image sensor of the present invention will be described with reference to FIG. FIG. 2 is a schematic diagram illustrating an example of the filter configuration of the image sensor 162. As shown in FIG. 2A, the imaging element 162 includes a plurality of pixels 162b arranged in a two-dimensional matrix, and a luminance filter Y or R (red) G (green) B is placed on each pixel 162b. Any of color filters R, G, and B that transmit only the blue color is disposed. Hereinafter, a pixel in which the above-described luminance filter Y is disposed is referred to as a white pixel, and a pixel in which the color filters R, G, and B are disposed is referred to as a color pixel.

  In FIG. 2A, the pixels in the horizontal odd rows (n−1 rows, n + 1 rows, etc.) and the vertical odd columns (m−1 columns, m + 1 columns, etc.) are white pixels in which the luminance filter Y is arranged. Pixels in even rows (n rows, n + 2 rows, etc.) and vertical even columns (m columns, m + 2 columns, etc.), that is, pixels in which the luminance filter Y is not arranged are color filters R, G, B in a normal Bayer array. The color pixels are arranged in the same order.

  Note that the brightness filter Y corresponds to an ND filter, a transparent filter, a white filter, a gray filter, a visibility correction filter, and the like, but light is directly emitted without providing anything on the surface of the photoelectric conversion unit of the pixel. The structure that enters the light source also corresponds to a structure provided with a transparent filter. Further, the Bayer array is an array of color filters as shown in FIG. 2B. For example, repetition of R and G is arranged in horizontal odd rows (n−1 rows, n + 1 rows, etc.), and horizontal even rows. A repetition of G and B is arranged in (n rows, n + 2 rows, etc.), and it is a color filter array having a checkered pattern when only the G filter is seen.

  Next, a first embodiment of an image sensor according to the present invention will be described with reference to FIGS.

  FIG. 3 is a block diagram showing a configuration of a CMOS (complementary metal oxide semiconductor) type image pickup device which is the first embodiment of the image pickup unit in the present invention. The CMOS image sensor 162 includes a plurality of pixels 162b arranged in a two-dimensional matrix on the imaging surface 162a, a vertical scanning circuit 162c, a sample hold circuit 162d, an output circuit 162e, an output amplifier 162g, a horizontal scanning circuit 162f, a timing, and the like. It includes components such as a generator 162h, and each horizontal row of pixels 162b and the vertical scanning circuit 162c are connected by a row selection line 162i, and each vertical column of pixels 162b and the sample and hold circuit 162d are vertical. They are connected by a signal line 162j. On each pixel, for example, a luminance filter or a color filter shown in FIG.

  The photographing operation of the image sensor 162 is controlled by a timing generator 162h built in the image sensor 162 in accordance with the image capture control signal 161a from the image capture controller 161, and the image data 162k that is the output of the image sensor 162 is input to the amplifier 163. Is done. The timing generator 162h functions as exposure time control means in the present invention.

  Although FIG. 1 shows an example in which the timing generator 166 is provided separately from the image sensor 162, in particular, in a CMOS image sensor, a control circuit such as a timing generator can be built in the same chip as the pixel. Yes, it is one of the features of the CMOS image sensor. Here, the imaging element 162 functions as an imaging unit in the present invention. Of course, it may be provided separately as in FIG. 1, or a part of the functions of the timing generator may be provided separately, and the remaining functions may be incorporated in the image sensor.

  FIG. 4 is a circuit diagram illustrating an example of a circuit of the pixel 162b that constitutes the CMOS image sensor 162. The pixel 162b includes an embedded photodiode PD (hereinafter referred to as a PD section) and N-channel MOSFETs (metal oxide semiconductor field effect transistors: hereinafter referred to as transistors) Q1 to Q4. A connection portion between the drain of the transistor Q1 and the source of the transistor Q2 is formed of a floating diffusion FD (hereinafter referred to as an FD portion), which is a so-called complete transfer structure. A reset signal RST, a transfer signal TX, and a read signal SX indicate signals (potentials) for each transistor, VDD indicates a power supply, and GND indicates ground.

  The PD unit photoelectrically converts incident light from the subject to generate a photocurrent Ipd corresponding to the amount of incident light, and the photocurrent Ipd is accumulated in the parasitic capacitance Cpd of the PD unit as a signal charge Qpd. Since the PD portion has a buried structure and the photoelectrically converted photocurrent Ipd cannot be directly taken out, the PD portion is connected to the FD portion via a transistor Q1 called a transfer gate (hereinafter referred to as transfer gate Q1).

  At the time of shooting, the transfer signal TX is set to the low potential L, so that the transfer gate Q1 is turned off, and the signal charge Qpd is accumulated in the parasitic capacitance Cpd of the PD section. At the time of signal transfer, the transfer signal TX is set to the high potential H. As a result, the transfer gate Q1 is turned on, and the signal charge Qpd is completely transferred to the FD portion. In the image sensor 162 of the present invention, a white pixel transfer signal (hereinafter referred to as a white pixel control line) YTX and a color pixel transfer signal (hereinafter referred to as a color pixel control line) CTX can be controlled separately. Therefore, it is possible to compensate for the low sensitivity of the color pixel by making the signal charge Qpd accumulation time of the color pixel at the time of photographing, that is, the exposure time longer than that of the white pixel.

  The transistor Q2 is referred to as a reset gate (hereinafter referred to as a reset gate Q2), is controlled by a reset signal RST, and resets the FD portion to the power supply voltage VDD by being turned on.

  The transistor Q3 constitutes a source follower amplifier circuit, and functions to lower the output impedance by performing current amplification on the potential Vfd of the FD section.

  The transistor Q4 is a pixel output readout transistor, and its gate is connected to the row selection line 162i described above, and operates as a switch that is turned on / off in response to the readout signal SX applied by the vertical scanning circuit 162c. To do. The source of the transistor Q4 is connected to the vertical signal line 162j. When the transistor Q4 is turned on, the potential Vfd of the FD portion is reduced in impedance by the transistor Q3, and is output to the vertical signal line 162j as the pixel output VOUT. Is done.

  FIG. 5 is a circuit block diagram showing a driving method of the pixel shown in FIGS. A reset signal line RSTn-1, a white pixel control line YTXn-1, and a read signal line SXn-1 are connected to the white pixel in which the luminance filter Y in the horizontal (n-1) th row shown in FIG. In addition, the reset signal line RSTn, the white pixel control line YTXn, and the readout signal line SXn are arranged in any one of the color filters R, G, and B in the white pixel in which the luminance filter Y in the horizontal (n) row is arranged. A reset signal line RSTn, a color pixel control line CTXn, and a read signal line SXn are input to the color pixels thus obtained. That is, two control lines of a white pixel signal line YTXn and a color pixel control line CTXn are connected to a horizontal (n) row where white pixels and color pixels are mixed, and thereby, signal charges of the white pixels and the color pixels are connected. Therefore, it is possible to compensate for the low sensitivity of the color pixel by making the exposure time of the color pixel longer than that of the white pixel.

  FIG. 6 is a timing chart showing an imaging operation at the time of moving image shooting or live view image shooting in the circuit block shown in FIG. The imaging operation here is basically a so-called rolling shutter type imaging operation in which exposure is performed for a frame rate period for each horizontal parallel of the imaging element.

  In the photographing operation of the (N) th frame (N is a positive integer), the reset signal line RSTn-1 in the horizontal (n-1) th row is set to the high potential H at the timing T1, and the horizontal (n-1) ) The reset gate Q2 of each pixel in the row is turned on, the FD portion of each pixel in the horizontal (n−1) th row is reset to the power supply voltage VDD, and at the timing T2, the horizontal (n−1) th row When the white pixel control line YTXn-1 is set to the high potential H, the transfer gate Q1 of each pixel in the horizontal (n-1) th row is turned on, and the PD portion of each pixel in the horizontal (n-1) th row. Is completely transferred to the FD portion.

  At timing T3, the readout signal line SXn-1 in the horizontal (n-1) th row is set to the high potential H, whereby the potential of the FD portion of each pixel in the horizontal (n-1) th row, that is, the horizontal (n- 1) The imaging data of each pixel in the row is derived to the vertical signal line 162j and stored in the sample hold circuit 162d. At timing T4, the imaging of all the pixels in the horizontal (n−1) row is performed according to the horizontal transfer signal HT. Data is output to the amplifier 163.

  Next, at timing T5, the reset signal line RSTn in the horizontal (n) row is set to the high potential H, so that the reset gate Q2 of each pixel in the horizontal (n) row is turned on, and the horizontal (n) row. The FD portion of each pixel in the eye is reset to the power supply voltage VDD, and the horizontal (n) row white pixel control line YTXn and the horizontal (n) row color pixel control line CTXn are set to the high potential H at timing T6. As a result, the transfer gates Q1 of all the white and color pixels in the horizontal (n) row are turned on, and the signal charge Qpd accumulated in the PD portions of all the pixels in the horizontal (n) row is transferred to the FD portion. Completely transferred.

  At timing T7, the readout signal line SXn in the horizontal (n) row is set to the high potential H, so that the potential of the FD portion of each pixel in the horizontal (n) row, that is, each pixel in the horizontal (n) row. The imaging data is derived to the vertical signal line 162j and stored in the sample hold circuit 162d. At timing T8, imaging data of all pixels in the horizontal (n) row is output to the amplifier 163 according to the horizontal transfer signal HT. . The above operation is performed for all horizontal parallel lines of the image sensor 162, and the (N) th frame is completed. In normal moving image shooting, the operation of the (N) th frame is repeated.

  Next, the operation proceeds to the (N + 1) th frame (N is a positive integer). Basically, this is the same as the shooting operation for the (N) th frame described above, but only the white pixel control line YTXn for the horizontal (n) th row is set to the high potential H at the timing T6 in the (N + 1) th frame. The transfer gate Q1 of the white pixel in the horizontal (n) row is turned on, and the signal charge Qpd accumulated in the PD portion of the white pixel in the horizontal (n) row is completely transferred to the FD portion.

  At this time, since the color pixel control line CTXn in the horizontal (n) row remains at the low potential L, the signal charge Qpd accumulated in the PD portion of the color pixel in the horizontal (n) row is transferred to the FD portion. Accordingly, the accumulation operation is continued, and the exposure time of the color pixels is extended to improve the sensitivity. That is, the exposure time of the white pixel is 1 frame rate from the fall of the timing T2 of the (N) th frame to the rise of the timing T2 of the (N + 1) th frame, and the exposure time of the color pixel is the (N) th frame. This is between two frame rates from the fall of the timing T2 to the rise of the timing T2 of the (N + 2) th frame not shown.

  In addition, since the FD portion of the color pixel in the horizontal (n) row is reset to the power supply voltage VDD, the potential of the FD portion of the color pixel in the horizontal (n) row, that is, the imaging data is the imaging data in the dark state. Become. This imaging data is treated as invalid data by the image processing means 165 at the subsequent stage, for example, and processing such as replacement with imaging data of the color pixel of the (N) frame before one frame is performed.

  With the above operation, in moving image shooting, it is possible to increase the exposure time of the color pixel by outputting the imaging data of the white pixel in all frames of the shooting and outputting the imaging data of the color pixel every other frame. It is possible to provide an imaging device and an imaging apparatus that can compensate for a sensitivity difference between a white pixel and a color pixel to prevent occurrence of overexposure and color reproducibility and can perform photographing that is not inferior in terms of image quality.

  FIG. 7 is a timing chart showing an imaging operation at the time of still image shooting in the circuit block shown in FIG. In FIG. 7, at timing T11, reset signal lines RST (RSTn-1, RSTn, etc.), white pixel control lines YTX (YTXn-1, YTXn, etc.) and color pixel control lines CTX (CTXn, etc.) of all pixels of the image sensor 162. ) Is set to the high potential H, the charges accumulated in the PD portions of all the pixels are completely transferred to the FD portion, and the FD portions of all the pixels are reset to the power supply voltage VDD so that the PD portion and the FD The parts are initialized together. At the end of the timing T11, the reset signal lines RST (RSTn-1, RSTn, etc.), the white pixel control lines YTX (YTXn-1, YTXn, etc.) and the color pixel control lines CTX (CTXn, etc.) of all the pixels of the image sensor 162 are low. By setting the potential to L, photocharge accumulation starts in the PD portions of all pixels, that is, exposure of the electronic shutter starts.

  After the white pixel exposure time SS1 elapses from the end of the timing T11, all the white pixel control lines YTX (YTXn-1, YTXn, etc.) of all the white pixels of the image sensor 162 are set to the high potential H at the timing T12. The photocharge accumulated in the PD portion of the white pixel is completely transferred to the FD portion, and the exposure of the electronic shutter of the white pixel is completed. At timing T13, the readout signal line SXn-1 of all the pixels in the horizontal (n-1) th row of the image sensor 162 is set to the high potential H, so that the FD portion of each pixel in the horizontal (n-1) th row. The potential, that is, the imaging data of each pixel in the horizontal (n−1) th row is derived to the vertical signal line 162j and stored in the sample and hold circuit 162d, and the horizontal (n−1) in accordance with the horizontal transfer signal HT at timing T14. Imaging data of all pixels in the row is output to the amplifier 163. The operations at timings T13 and T14 are sequentially performed on all horizontal parallels including only white pixels, whereby reading of all horizontal parallel imaging data including only white pixels is completed.

  After the color pixel exposure time SS2 has elapsed from the end of the timing T11, the color pixel control lines CTX (CTXn, etc.) of all the color pixels of the image sensor 162 are set to the high potential H at the timing T15, so that the PDs of all the color pixels The photocharge accumulated in the part is completely transferred to the FD part, and the exposure of the electronic shutter of the color pixel is completed. At timing T16, the readout signal lines SXn of all pixels in the horizontal (n) row of the image sensor 162 are set to the high potential H, so that the potential of the FD portion of each pixel in the horizontal (n) row, that is, horizontal (n ) The imaging data of each pixel in the row is derived to the vertical signal line 162j and stored in the sample hold circuit 162d. At timing T17, the imaging data of all the pixels in the horizontal (n) row is amplified according to the horizontal transfer signal HT. It is output toward 163. The operations at timings T16 and T17 are sequentially performed on all horizontal parallels in which white pixels and color pixels are mixed, whereby reading of all horizontal parallel imaging data in which white pixels and color pixels are mixed is completed.

  In this example, after the exposure of the electronic shutter of the white pixel is completed at time T12, all the horizontal parallel imaging data is read, and after the exposure of the electronic shutter of the color pixel is completed, the white pixel and the color pixel are read. Although all the horizontal parallel imaging data is read out, the present invention is not limited to this. For example, all horizontal parallel imaging data may be sequentially read out after timing T15, that is, after the exposure of the electronic shutter of the color pixel. . According to the readout method of this example, readout of all horizontal parallel imaging data consisting of only white pixels is completed during exposure of the color pixels, so that the readout time after timing T15 can be shortened.

  The white pixel exposure time SS1 and the color pixel exposure time SS2 may be obtained, for example, by photometric calculation using imaging data of a live view image before capturing a still image, or from a photometric output of a photometric element independent of the imaging element 162. You may obtain | require by calculating. The ratio between the white pixel exposure time SS1 and the color pixel exposure time SS2 may be a ratio determined in advance based on the transmittance ratio between the luminance filter Y and the color filters R, G, and B. As a result of the calculation, when the subject is brighter than the predetermined value, the white pixel exposure time SS1 and the color pixel exposure time SS2 are set to be the same, and when the brightness is lower than the predetermined value, the color is set to a predetermined ratio. The sensitivity for color may be improved by increasing the pixel exposure time.

  Next, a control flow at the time of photographing with the digital camera 1 will be described with reference to FIGS. FIGS. 8 to 10 are flowcharts showing a control flow at the time of photographing with the digital camera 1. FIG. 8 is a main routine, FIG. 9 is a still image mode subroutine, and FIG. 10 is a moving image mode subroutine.

  In FIG. 8, when the power source of the digital camera 1 is turned on in step S101, it is confirmed in step S102 whether the operation mode of the digital camera 1 is the camera mode. When the operation mode is set to a mode that is not a camera mode such as a playback mode or a recording mode, for example (step S102; NO), the control proceeds to the set mode. A description of the control in each mode other than the camera mode is omitted.

  When the operation mode is set to the camera mode (step S102; YES), display of a live view image used as a finder is started in step S111. In the live view display, as described in the description of FIG. 1, instead of reading out the imaging data of all the pixels of the imaging device 162, for example, by performing a thinning-out readout operation of 30 frames per second, the load of signal processing and image processing is reduced. Can be reduced.

  In step S112, it is confirmed whether or not the AF switch 101a that is operated by half-pressing the shutter button is turned on. The operation of steps S111 and S112 is repeated until it is turned on. When the AF switch 101a is turned on (step S112; YES), in step S113, for example, using the imaging data of white pixels in the live view image, an autofocus (AF) operation on the subject and detection of subject luminance, that is, photometry. (AE) The operation is performed, and the aperture value control of the aperture 172 is performed so that the focusing operation on the subject and the white pixels of the image sensor 162 are not saturated. In step S114, it is confirmed whether or not the release switch 101b that operates when the shutter button is fully pressed is turned on, that is, whether or not shooting is started. The operation from step S111 to step S114 is repeated until it is turned on.

  When the release switch 101b is turned on in step S114 (step S114; YES), it is confirmed in step S121 whether the shooting mode in the camera mode of the digital camera 1 is the still image mode. If the still image mode is selected (step S121; YES), the process proceeds to step S130, the still image subroutine shown in FIG. 9 is executed, the process proceeds to step S151, and it is confirmed whether the power of the digital camera 1 is turned off. If the power is not turned off (step S151; NO), the process returns to step S102, and thereafter, the above-described operation is repeated. If the power is off (step S151; YES), the operation is terminated as it is.

  If the still image mode is not set in step S121 (step S121; NO), it is determined that the shooting mode in the camera mode of the digital camera 1 is the moving image mode, the process proceeds to step S140, and the moving image subroutine shown in FIG. The process proceeds to S151, and thereafter, the above-described operation is repeated.

  FIG. 9 is a flowchart of step S130 (still image subroutine) in FIG. In step S301, the display of the live view image started in step S111 is terminated, and in step S302, the aperture 172 is controlled to an appropriate aperture value that does not saturate the white pixels obtained in the photometry (AE) operation of step S113. In step S303, a still image is taken according to the operation shown in the timing chart of FIG.

  In step S304, the captured still image data is temporarily recorded in the image memory 181. In step S305, the still image data recorded in the image memory 181 is displayed as an after-view image on the display unit 131. In step S306, the image memory The still image data recorded in 181 is finally recorded in the memory card 182, and the process returns to step 130 of the main routine of FIG.

  FIG. 10 is a flowchart of step 140 (moving image subroutine) of FIG. In step S401, it is confirmed whether or not the subject brightness obtained by the photometry (AE) operation in step S113 in FIG. 8 is higher than or equal to a predetermined brightness. If the brightness is higher than or equal to the predetermined brightness (step S401; YES), in step S411, the aperture 172 is an exposure time of one frame (usually 1/30 second) obtained by the photometry (AE) operation in step S113. Control is made to an appropriate aperture value that does not saturate white pixels, and in step S412, only the operation of reading out image data of both white pixels and color pixels shown in the Nth frame in FIG. Then, a moving image having a predetermined number of frames (for example, 30 frames for one second) is captured, and in step S413, the moving image having the predetermined number of frames captured in step S412 is recorded in the image memory 181.

  When it is determined in step S401 that the subject brightness is lower than the predetermined brightness (step S401; NO), in step S421, the aperture 172 is obtained by the photometric (AE) operation in step S113 as in step S411. The aperture value is controlled to an appropriate aperture value that does not saturate white pixels in one frame exposure time. In step S422, the operations shown in the Nth frame and the N + 1th frame in FIG. (30 frames) of moving images are shot, and in step S413, the moving images of the predetermined number of frames shot in step S422 are recorded in the image memory 181. During the shooting in step S412 or step S422, the shot moving image is displayed on the display unit 131 as a live view image.

  Here, the predetermined luminance described above varies depending on various conditions such as the sensitivity of the image sensor itself and the F-number of the photographing lens. For example, the luminance at which the accumulated charge amount of the color pixel decreases and noise starts to be noticeable or slightly lower than that. It is desirable to set a high luminance.

  In step S414, it is confirmed whether or not the release switch 101b is turned off, that is, whether or not the moving image shooting is finished. If it is turned off (step S414; YES), all the moving images recorded in the image memory 181 are recorded in the memory card 182 in step S431, and the process returns to step 140 of the main routine of FIG. If the release switch 101b is not turned off in step S414 (step S4146; NO), the process returns to step S401, and the above-described operation is repeated thereafter.

  Using the method described above, subject brightness is detected, and if the subject is bright, the same shooting as normal movie shooting is performed. If the subject is dark, shooting of low-sensitivity color pixels is performed using a plurality of white pixels. By increasing the sensitivity of the color pixels once per frame (2 frames in this example), a high-quality moving image with good color reproducibility can be obtained even for a dark subject.

  In this example, although it has been described that the normal moving image shooting in step S412 and the moving image shooting specific to the present invention in step S422 are switched in step S401 according to the brightness of the subject, Since there is a difference in sensitivity between the pixels and the color pixels, the determination in step S401 is not performed, and the moving image shooting unique to the present invention shown in step S422 may always be performed.

  Next, a second embodiment of the present invention will be described with reference to FIGS. FIG. 11 is a block diagram showing a configuration of a CMOS type image pickup device which is the second embodiment of the image pickup unit according to the present invention. In contrast to FIG. 5, in FIG. 11, the first vertical signal line 1621 j to which the outputs of horizontal pixels (for example, the horizontal (n−1) th row) composed of only white pixels are connected, and the white pixels and color pixels are connected. A second vertical signal line 1622j to which mixed pixel outputs of horizontal lines (for example, horizontal (n) row) are connected is provided. The vertical signal line 1621j is connected to the first sample hold circuit 1621d and the vertical signal line 1622j. Are connected to a second sample and hold circuit 1622d.

  The pixel output of each horizontal pixel composed of only white pixels stored in the first sample hold circuit 1621d is output by the first output circuit 1621e according to the scanning operation of the first horizontal scanning circuit 1621f to the first output amplifier 1621g. Are output sequentially from the image sensor 162 as horizontal parallel image data 1621k consisting of only white pixels. Similarly, the horizontal pixel output in which white pixels and color pixels are mixed is output as horizontal parallel imaging data 1622k in which white pixels and color pixels are mixed through the second output amplifier 1622g. These operations are controlled by the timing generator 162h. Here, the imaging element 162 functions as an imaging unit in the present invention.

  FIG. 12 is a timing chart showing the operation of the second embodiment of the image sensor of the present invention shown in FIG. In FIG. 12, horizontal parallels (for example, horizontal (n−1) th row) composed of only white pixels and horizontal parallels (for example, horizontal (n) th row) in which white pixels and color pixels are mixed are simultaneously performed in parallel. A so-called rolling shutter type imaging operation is performed in which exposure is performed for a frame rate period.

  In FIG. 12, in the shooting operation of the (N) th frame (N is a positive integer), at the timing T21, the reset signal line RSTn-1 of the horizontal (n-1) th row and the reset signal of the horizontal (n) th row. Since both the lines RSTn are set to the high potential H, the FD portions of all the pixels in the horizontal (n-1) th row composed only of white pixels and the horizontal (n) th row in which white pixels and color pixels are mixed are obtained. At the timing T22, the horizontal (n-1) th white pixel control line YTXn-1, and the horizontal (n) th white pixel control line YTXn and color pixel control line CTXn are all at high potential. By being set to H, the photoelectric charges accumulated in the PD portions of all the pixels on the horizontal (n−1) th row and the horizontal (n) th row are completely transferred to the FD portion.

  At timing T23, the horizontal (n-1) th row readout signal line SXn-1 and the horizontal (n) th row readout signal line SXn are set to the high potential H, so that the horizontal (n-1) th row. The pixel outputs of all the pixels are derived to the first vertical signal line 1621j, and at the same time, the pixel outputs of all the pixels in the horizontal (n) row are derived to the second vertical signal line 1622j.

  At timing T24, the first imaging data 1621k of all the pixels in the horizontal (n−1) -th row is output according to the first horizontal transfer signal HT1, and in parallel therewith, the horizontal (n− 1) First imaging data 1621k of all pixels in the row is output. The first horizontal transfer signal HT1 and the second horizontal transfer signal HT2 may be the same signal or different timing signals. The above operation is performed two rows at a time for the entire horizontal parallel of the image sensor 162, and the imaging of the (N) frame is completed.

  Next, the operation proceeds to the (N + 1) th frame (N is a positive integer). Basically, it is the same as the shooting operation of the (N) th frame described above, but at the (N + 1) th frame, at the timing T22, the horizontal (n−1) th row of white pixel control lines YTXn−1 and the horizontal Only the white pixel control line YTXn in the (n) th row is set to the high potential H, the transfer gate Q1 of the white pixel in the horizontal (n-1) th row and the horizontal (n) th row is turned on, and the horizontal (n− 1) The signal charge Qpd accumulated in the PD portion of the white pixel in the row and the horizontal (n) row is completely transferred to the FD portion.

  At this time, since the color pixel control line CTXn in the horizontal (n) row remains at the low potential L, the signal charge Qpd accumulated in the PD portion of the color pixel in the horizontal (n) row is transferred to the FD portion. Accordingly, the accumulation operation is continued, and the exposure time of the color pixels is extended to improve the sensitivity. In other words, the exposure time of the white pixel is 1 frame rate from the fall of the timing T22 of the (N) th frame to the rise of the timing T22 of the (N + 1) th frame, and the exposure time of the color pixel is the (N) th frame. It is between two frame rates from the fall of the timing T22 to the rise of the timing T22 of the (N + 2) frame (not shown).

  In addition, since the FD portion of the color pixel in the horizontal (n) row is reset to the power supply voltage VDD, the potential of the FD portion of the color pixel in the horizontal (n) row, that is, the imaging data is the imaging data in the dark state. Become. This imaging data is treated as invalid data by the image processing means 165 at the subsequent stage, for example, and processing such as replacement with imaging data of the color pixel of the (N) frame before one frame is performed.

  By the method described above, the horizontal parallel imaging data consisting only of white pixels and the horizontal parallel imaging data in which white pixels and color pixels are mixed can be read out separately, and image processing by the image processing means 165 at the subsequent stage can be performed. In addition to facilitating the selection of imaging data, such as reading out and using only horizontal imaging data consisting of only white pixels for operations using only luminance information such as autofocus and photometry.

  Next, a third embodiment of the present invention will be described with reference to FIGS. FIG. 13 is a block diagram showing the configuration of an interline CCD (Charge Coupled Device) type imaging device and its peripheral circuit, which is the third embodiment of the imaging unit of the present invention. The CCD image sensor 162 includes a photoelectric conversion unit 162w, a transfer unit 162x, a vertical transfer CCD 162y, a horizontal transfer CCD 162z, and the like. Parts other than the photoelectric conversion unit 162w are basically shielded from light.

  The photoelectric conversion unit 162w is composed of a photodiode in which any one of the luminance filter Y or the color filters R, G, and B as shown in FIG. 2A and FIG. Photoelectrically accumulates and accumulates photocharges in the parasitic capacitance of the photodiode. In this embodiment, a photoelectric conversion unit in which a luminance filter is arranged is called a white pixel, and a photoelectric conversion unit in which a color filter is arranged is called a color pixel.

  The transfer unit 162x transfers the photocharge accumulated in the photoelectric conversion unit 162w to the vertical transfer CCD 162y. The transfer timing is controlled by a transfer signal. In this embodiment, the transfer signal for white pixels is called a white pixel control signal YTX, and the transfer signal for color pixels is called a color pixel control signal CTX. The white pixel control signal YTX and the color pixel control signal CTX are controlled by the timing generator 166 under the control of the camera control unit 150 and the imaging control unit 161.

  The vertical transfer CCD 162y sequentially transfers the photocharges transferred by the transfer unit 162x in the vertical direction of the drawing according to the vertical transfer clock VCL. The horizontal transfer CCD 162z transfers the photocharges transferred in the vertical direction of the figure by the vertical transfer CCD 162y in the horizontal direction of the figure in accordance with the water-side transfer clock HCL for each horizontal line, and outputs it to the amplifier 163 as imaging data 162k. The operations of the vertical transfer CCD and the horizontal transfer CCD may be the same as those of a general CCD type image pickup device, and are controlled by the timing generator 166. Here, the imaging element 162 and the timing generator 166 function as an imaging unit in the present invention.

  FIG. 14 is a timing chart showing a driving method at the time of moving image shooting of the CCD type image pickup device 162 shown in FIG. In this example, the reading of the image data of the color pixel will be described as one out of every three readings of the image data of the white pixel.

  In FIG. 14, when the white pixel control signal YTX and the color pixel control signal CTX are both set to the high potential H at the timing T31, all the transfer units 162x of the image sensor 162 are opened and accumulated in the photoelectric conversion unit 162w. In addition, the accumulated photocharge of the (N-1) th frame is transferred to the vertical transfer CCD 162y, and the photoelectric conversion unit 162w is reset to a state in which no photocharge is accumulated.

  At the end of the timing T31, both the white pixel control signal YTX and the color pixel control signal CTX are set to the low potential L, all the transfer units 162x are closed, and accumulation of photocharges in the (N) frame, that is, photographing is started. . (N) During the photographing of the frame, at timing T32, one vertical transfer clock VCL is input, the vertical transfer CCD is transferred for one horizontal line, and the horizontal transfer clock HCL is input for the number of pixels of one horizontal line. The imaging data 162k for one horizontal line is horizontally transferred and output to the amplifier 163. By repeating the above-described operation for the number of horizontal rows, the entire imaging data 162k of the (N-1) th frame is output. In a normal CCD type image pickup device, vertical and horizontal transfer operations are performed using four layers of clocks for the purpose of avoiding mixing with preceding and subsequent image pickup data. In this example, for ease of explanation, one clock is used. It was supposed to be transferred with. Actually, it is only necessary to perform vertical and horizontal transfer operations performed by a normal CCD type imaging device.

  After a predetermined time of one frame has elapsed from the beginning of the timing T31 (after 1/30 seconds in the case of a normal moving image), the white pixel control signal YTX is set to the high potential H at the timing T33, so that all of the image sensor 162 The white pixel transfer unit 162x is opened, and the (N) frame accumulated photocharge accumulated in the white pixel photoelectric conversion unit 162w is transferred to the vertical transfer CCD 162y, and the white pixel photoelectric conversion unit 162w is reset to a state in which no photocharge is accumulated.

  At the end of the timing T33, the white pixel control signal YTX is set to a low potential L, the transfer unit 162x of all white pixels is closed, and shooting of the (N + 1) th frame of white pixels is started. At this time, since the color pixel control signal CTX remains at the low potential L at the timing T33, the photocharge accumulated in the color pixel is not transferred to the vertical transfer CCD 162y, and the photocharge accumulation, that is, photographing is continued.

  During the shooting of the (N + 1) th frame, the entire imaging data 162k of the (N) th frame is output at the timing T34 as in the above-described timing T32. At this time, the imaging data of the portion corresponding to the color pixel of the vertical transfer CCD 162y is data in a state where there is no signal charge, that is, in a dark state. This data is treated as invalid data by the image processing means 165 at the subsequent stage, for example, and processing such as replacement with imaging data of the color pixel of the (N-1) th frame before one frame is performed.

  At timings T35 and T36, the same operations as those at timings T33 and T34 are performed, and the imaging of the (N + 2) frame and the imaging data output of the (N + 1) frame are performed. Even at the timing T35, the transfer of light charges of the color pixels is not performed, and the accumulation of light charges continues.

  After a predetermined time of one frame from the beginning of the timing T35 (after 1/30 seconds in the case of a normal moving image), at the timing T37, the white pixel control signal YTX and the color pixel control signal CTX are both similar to the timing T31. By setting the high potential H, all the transfer units 162x of the image sensor 162 are opened, and the (N + 2) -th frame accumulated photocharges accumulated in the photoelectric conversion unit 162w are transferred to the vertical transfer CCD 162y. At timing T38, the entire imaging data 162k of the (N + 2) th frame is output.

  In this example, it is assumed that the transfer of the photocharge of the color pixel is performed once for the transfer of the photocharge of the white pixel, but this cancels the sensitivity ratio of the white pixel and the color pixel which is usually three times or more. can do. Since the human eye has a slower response to color information than luminance information in a moving image, the imaging data of color pixels, that is, color information, is compared to the imaging data of white pixels, that is, luminance information, as in this example. Even if the reading frequency is reduced, color misregistration is hardly felt, and conversely, by increasing the storage time of color pixels, it is possible to improve the sensitivity to color, thereby enabling rich color reproduction.

  As described above, according to the present invention, the exposure time of a low-sensitivity color pixel is lengthened by separately controlling the exposure time of a white pixel having a luminance filter and a color pixel having a color filter. To provide an imaging unit and an imaging apparatus that can compensate for a difference in sensitivity between white pixels and color pixels, prevent occurrence of whiteout and poor color reproducibility, and can perform shooting that is inferior in terms of image quality Can do.

  It should be noted that the detailed configuration and detailed operation of each component constituting the imaging unit and the imaging apparatus according to the present invention can be changed as appropriate without departing from the spirit of the present invention.

It is a circuit block diagram which shows an example of an internal structure of a digital camera. It is a schematic diagram which shows an example of the filter structure of an image pick-up element. 1 is a block diagram showing a configuration of a CMOS type image pickup device which is a first embodiment of an image pickup unit in the present invention. FIG. It is a circuit diagram which shows an example of the circuit of the pixel which comprises a CMOS type image pick-up element. It is a circuit block diagram which shows the drive method of the pixel of an image pick-up element. It is a timing chart which shows the imaging operation at the time of moving image photography or live view image photography. It is a timing chart which shows the imaging operation at the time of still image photography. It is the main routine of the flowchart which shows the flow of control at the time of imaging | photography of a digital camera. It is a subroutine which shows the flow of control at the time of still image photography. It is a subroutine which shows the flow of control at the time of video recording. It is a block diagram which shows the structure of the CMOS type image pick-up element which is 2nd Embodiment of the imaging unit in this invention. 12 is a timing chart showing the operation of the second embodiment of the image sensor of the present invention shown in FIG. It is a block diagram which shows the structure of the CCD type image pick-up element of the interline system which is 3rd Embodiment of the image pick-up unit in this invention, and its peripheral circuit. 14 is a timing chart showing a driving method during moving image shooting of the CCD type image pickup device shown in FIG. 13.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Digital camera 100 Operation means 131 Display means 150 Camera control means 161 Image pick-up control means 162 Image pick-up element 162b Pixel 163 Amplifier 164 Front end processor (FEP)
165 Image processing means 166 Timing generator 171 Aperture control means 172 Aperture 181 Image memory 182 Memory card 201 Shooting lens Obj Subject Y Luminance filter R, G, B Color filter YTX White pixel control line and white pixel control line CTX Color pixel control line and Color pixel control signal

Claims (8)

In an imaging unit having a plurality of pixels arranged in a two-dimensional matrix and having an imaging element that photoelectrically converts a subject image to generate imaging data,
A white pixel in which a luminance filter is disposed on some of the plurality of pixels of the image sensor;
A color pixel in which a color filter is disposed on a pixel other than the white pixel;
An imaging unit comprising: an exposure time control unit that separately controls an exposure time of the white pixel and an exposure time of the color pixel. The image sensor is a CMOS image sensor,
The exposure time control means includes a white pixel control line that controls reading of the output of the white pixel;
The imaging unit according to claim 1, further comprising a color pixel control line that controls reading of the output of the color pixel. The image sensor is a CCD type image sensor,
The exposure time control means includes a white pixel control signal for controlling transfer of an output of the white pixel;
The imaging unit according to claim 1, further comprising a color pixel control signal for controlling transfer of an output of the color pixel. The imaging unit according to claim 2, wherein the exposure time control unit separately controls the white pixel control line and the color pixel control line. The imaging unit according to claim 3, wherein the exposure time control unit separately controls the white pixel control signal and the color pixel control signal. An imaging apparatus comprising the imaging unit according to claim 1. Movie mode for shooting movies,
Brightness detection means for detecting the brightness of the subject,
The imaging apparatus according to claim 6, wherein the exposure time control unit included in the imaging unit controls exposure times of white pixels and color pixels included in the imaging unit based on an output of the luminance detection unit. The exposure time control means controls the exposure time of the color pixel to an integral multiple of the exposure time of the white pixel when the output of the brightness detection means shows a brightness lower than a predetermined brightness. Item 8. The imaging device according to Item 7.

Images