JP4769851B2 - Imaging device - Google Patents

Description

  The present invention relates to an image pickup apparatus, and more particularly to white balance control of an image pickup apparatus using an image pickup element including a plurality of pixels having different sensitivities.

  In an imaging apparatus such as a digital still camera, white balance control that enables color reproduction faithful to a subject is performed. General white balance control adjusts the gains of the R and B color components so that the colors of the R, G and B color information in the captured image have the same level with the G color component as a reference. ing.

  The functions of white balance control installed in the imaging device include a user's shooting scene, light source, etc. in addition to auto white balance in which the device automatically performs white balance adjustment by performing shooting scene determination according to a predetermined control procedure. There is a manual white balance that adjusts the white balance by selecting the white balance position manually.

  As a conventional example of auto white balance control, a white balance adjustment is performed using a predetermined white balance adjustment gain by determining a shooting scene and a light source at the time of shooting from color information of a shot image signal. Yes (see, for example, Patent Document 1 or 2). Also, as a conventional example of manual white balance control, the photographer can select and select the white balance position set for each shooting scene and light source, such as “Sunny”, “Cloudy”, “Light bulb”, “Fluorescent light”, etc. Some perform white balance adjustment using a gain corresponding to the white balance position (see, for example, Patent Document 2 or 3).

JP-A-6-165189 (page 3-4, FIG. 1) JP 2000-299876 A (page 5-7, FIG. 1) Japanese Patent Laid-Open No. 11-187412 (page 3-5, FIG. 1)

  For example, in an imaging device such as a digital still camera, both a high-sensitivity pixel and a low-sensitivity pixel are mounted on a solid-state image sensor, and a high-sensitivity image captured by a high-sensitivity pixel and a low-sensitivity image captured by a low-sensitivity pixel are combined. Consider a case in which a wide dynamic range is achieved by processing and outputting as one piece of image data. In such an imaging device, the spectral sensitivity characteristics may be different between the high-sensitivity pixel and the low-sensitivity pixel, and R and G are detected between the high-sensitivity pixel and the low-sensitivity pixel even though the image is captured under the same shooting conditions. , B may have different output signal level ratios. For this reason, when the manual white balance control or the auto white balance control is executed, the color balance of the image signal may be lost when the output of the high sensitivity pixel and the output of the low sensitivity pixel are combined. There is.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to appropriately perform white balance control and color reproduction in an imaging apparatus using an imaging device having a plurality of pixels having different spectral sensitivity characteristics. An object of the present invention is to provide an imaging device capable of obtaining a photographed image having excellent properties.

An image pickup apparatus according to the present invention is an image pickup apparatus including an image pickup element having a plurality of pixels having different spectral sensitivity characteristics, and sets a control value for white balance adjustment set corresponding to each of the plurality of pixels. storage means for holding, using the control value, and a white balance adjusting means performs white balance adjustment for each of the image signals obtained by the plurality of pixels of the imaging device, said imaging device, said plurality The pixel includes a high-sensitivity main pixel and a low-sensitivity sub-pixel, and the storage means includes a first control value for white balance adjustment corresponding to the main pixel and a white balance adjustment corresponding to the sub-pixel. A second control value for use, and the white balance adjustment means uses the first control value and the second control value stored in the storage means, To perform serial and white balance adjustment of the image signal by the main pixel and a white balance adjustment of the image signal by the sub-pixels, wherein the.

In the above-described configuration, it is possible to obtain a captured image with excellent color balance by performing white balance adjustment using a control value separately for each of a plurality of pixels having different spectral sensitivity characteristics. In addition, when a wide dynamic range is achieved by an imaging device having a high-sensitivity main pixel and a low-sensitivity sub-pixel, white balance control can be optimized to improve color reproducibility. A photographed image with excellent color reproducibility can be obtained. Further, for example, when so-called manual white balance using a preset control value is performed, by using the first control value corresponding to the main pixel and the second control value corresponding to the sub-pixel, the main pixel and the sub-pixel. In each case, appropriate white balance adjustment is possible.

  As one aspect of the present invention, the storage means stores a control value for white balance adjustment corresponding to the main pixel, and the white balance adjustment means stores the control stored in the storage means. White balance adjustment of the image signal by the main pixel using the value and white balance of the image signal by the sub pixel using the control value and the color shift ratio of the image signal between the main pixel and the sub pixel. Adjustments shall be made.

  In the above configuration, for example, when performing so-called manual white balance using a preset control value, the main pixel and the sub-pixel are obtained by using the control value corresponding to the main pixel and the color shift ratio of the sub-pixel to the main pixel. In each case, appropriate white balance adjustment is possible.

  In one aspect of the present invention, the white balance adjusting unit applies a control value for white balance adjustment set corresponding to the main pixel to an image signal of the main pixel to adjust white balance. Shall be performed.

  In the above configuration, for example, for an image signal read from only the main pixel, a suitable white balance adjustment can be performed by applying a control value set corresponding to the main pixel, and the captured image has excellent color reproducibility. Can be obtained.

  In one aspect of the present invention, the white balance adjusting unit applies a control value for white balance adjustment set corresponding to the sub-pixel to the image signal of the sub-pixel to adjust white balance. Shall be performed.

  In the above configuration, for example, for an image signal read from only the sub-pixel, by applying a control value set corresponding to the sub-pixel, an appropriate white balance can be adjusted, and a captured image with excellent color reproducibility Can be obtained.

  Further, as one aspect of the present invention, the storage means holds control values set for each white balance position corresponding to each of a plurality of shooting conditions.

  In the above configuration, when performing so-called manual white balance using a control value set in advance for each white balance position corresponding to shooting conditions such as a shooting scene and a light source, the main pixel is controlled by the control value corresponding to each white balance position. Appropriate white balance adjustment is possible for each subpixel.

  As described above, according to the present invention, in an imaging apparatus that uses a solid-state imaging device including a plurality of pixels having different spectral sensitivity characteristics, white balance control can be appropriately performed, and a captured image with excellent color reproducibility can be obtained. It is possible to obtain.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus according to an embodiment of the present invention. In this embodiment, a configuration example of a digital still camera is shown as an example of an imaging apparatus. Note that the present invention can be applied not only to a digital still camera but also to other types of imaging devices such as a digital video camera.

  The image pickup apparatus according to the present embodiment includes an image pickup device 1 including a CCD solid-state image pickup device, a CDS circuit 2 that performs correlated double sampling processing, an analog amplifier 3 that performs signal amplification, an AD converter 4 that converts digital signals, and various types. A digital signal processing unit 5 that performs digital image signal processing, a central processing unit (CPU) 6 that performs operation control and arithmetic processing of each unit, a memory 7 that stores various setting values, temporary storage of image data, and the like, an imaging device Are provided with an operation unit 8 having buttons and the like for performing operation input, an external memory 9 for recording image data, and the like.

  The digital signal processing unit 5 includes an offset correction unit 21 that performs offset correction processing of an input image signal, an integration unit 22 that performs integration processing for each RGB color signal of the input image signal, and white balance adjustment that performs white balance adjustment of the image signal. Unit (WB adjustment unit) 23, a synthesis processing unit 24 that synthesizes image signals obtained from a plurality of pixels having different sensitivities, and a YC processing unit that generates luminance signal Y and color difference signals Cr and Cb from RGB color signals. 25. A compression processing unit 26 that compresses an image signal in accordance with a predetermined compression encoding method such as JPEG, a recording processing unit 27 that performs recording processing of the compressed image data in the external memory 9, and the like.

  FIG. 2 is a schematic diagram showing a pixel arrangement configuration of the image sensor 1 used in the present embodiment. In this embodiment, a CCD solid-state image sensor having a so-called honeycomb arrangement in which the arrangement of each pixel is shifted by 1/2 pitch in the horizontal and vertical directions is used as the image sensor 1. The pixels 30 of the image sensor 1 have a pixel arrangement described in, for example, Japanese Patent Application Laid-Open No. 10-136391, and are arranged with a ½ pitch shift in the horizontal and vertical directions with respect to the pixels in adjacent rows and columns. A vertical transfer path (not shown) for transferring the signal charges read from each pixel in the vertical direction is meandered so as to avoid each pixel in the vertical direction.

  In the illustrated example, each pixel 30 includes a wide area high sensitivity pixel (hereinafter referred to as a main pixel) 31 occupying about 4/5 of the light receiving area, and a small area low sensitivity pixel (occupying about 1/5). In the following, one pixel having 32) is divided into a plurality of light receiving regions. Signal charges that are photoelectrically converted and output from each of the main pixel 31 and the sub-pixel 32 having different sensitivities are read out and transferred to the vertical transfer path (not shown) separately from each other, a horizontal transfer path (not shown), It is output as an image signal from the image sensor 1 through an output amplifier.

  Thus, by combining the outputs from a plurality of pixels having different sensitivities into one image, the dynamic range of the captured image can be expanded. For example, the output of the main pixel is dominant when the amount of light is low (low brightness, the subject image is dark), and the output of the subpixel is high when the amount of light is high (high brightness, the subject image is bright) By performing the synthesis process so that the image becomes dominant, it is possible to realize an image pickup apparatus having a wide dynamic range in which a good image can be obtained such that a dark part of an image is not blacked out or a bright part is not blown out. In the present embodiment, a high-sensitivity image (an image obtained by the main pixel 31) and a low-sensitivity image (an image obtained by the sub-pixel 2) are simultaneously acquired by one shooting, and each image signal is sequentially read out. It is designed to be combined.

  Note that the ratio and position at which the pixels 30 of the image sensor 1 are divided are determined by design, and FIG. 2 is merely an example. Further, the image pickup device 1 is not limited to the CCD solid-state image pickup device having the honeycomb arrangement shown in FIG. 2, and may be a CCD solid-state image pickup device having a square lattice arrangement (Bayer method), or a CMOS sensor having a honeycomb arrangement or a square lattice arrangement. A solid-state imaging device can also be used.

  Next, shooting and image processing operations in the imaging apparatus configured as described above will be described. A subject image formed on the light receiving surface of the image sensor 1 via a photographing lens (not shown) is converted into a signal charge of a level corresponding to the amount of incident light by the main pixel 31 and the sub-pixel 32 of each pixel 30. The signal charges accumulated in each pixel 30 are transferred through a vertical transfer path and a horizontal transfer path (not shown), sequentially read out, and output as an imaging signal. The output signal of the image sensor 1 is subjected to correlated double sampling processing and color separation processing in the CDS circuit 2, and R, G, and B color signals are obtained for the images of the main pixel 31 and the subpixel 32.

  The R, G, B color signals for each pixel are amplified to a predetermined level by the analog amplifier circuit 3 and then converted to a digital signal by the AD converter 4. The digitized R, G, B color signals of the digitized main pixel 31 and sub-pixel 32 are input to the digital signal processing unit 5 and subjected to image signal processing.

  In the digital signal processing unit 5, an offset correction process is first performed in the offset correction unit 21 to match the black reference levels of the R, G, and B color signals. Next, in the white balance adjustment unit (white balance adjustment means) 23, as will be described in detail later, a gain value which is a control value for white balance adjustment of each of the main pixel 31 and the sub-pixel 32 set in advance is stored in a memory (stored). Means) The color balance of each of the R, G, and B color components is adjusted by multiplying each color signal of the main pixel 31 and the sub-pixel 32 output from the offset correction unit 21 by the gain value.

  Then, in the synthesis processing unit 24, the R, G, and B color signals whose color balance is adjusted in each of the main pixel 31 and the sub-pixel 32 are synthesized. In this way, by synthesizing the image signal of the main pixel 31 and the image signal of the sub-pixel 32, R, G, and B color signals are obtained by the pixels 30 of the image sensor 1 with an expanded dynamic range. Further, the YC processing unit 25 performs a predetermined calculation on the R, G, and B color signals corresponding to each pixel 30, and generates an image signal composed of the luminance signal Y and the color difference signals Cr and Cb. Then, the compression processing unit 26 compresses the image signal according to a predetermined compression encoding method such as JPEG, and the compressed image data is recorded by the recording processing unit 27 in the external memory 9 such as a memory card such as a flash memory. The

  Although not shown, the digital signal processing unit 5 performs known image processing such as gamma correction, noise removal, contour correction, and tone correction.

  On the other hand, the integrating unit 22 calculates the integrated values of the R, G, and B color signals in each of the main pixel 31 and the sub-pixel 32 when auto white balance control is executed. At this time, the CPU 6 determines the shooting scene and the light source at the time of shooting based on the integrated value calculated by the integrating unit 22. The CPU 6 has a function as a control unit that performs overall control of each unit of the imaging apparatus based on an input from the operation unit 8 and performs autofocus, automatic exposure control, and the like that are not described in the present embodiment.

  Next, the white balance control according to the first embodiment will be described in more detail. FIG. 3 is a flowchart showing processing when manual white balance control is performed in the first embodiment of the present invention.

  First, based on the operation of the operation unit 8 or the like, a photographing sequence is executed to photograph a subject and capture an imaging signal (step S11). At this time, the image signals obtained by the main pixel 31 and the sub-pixel 32 of the pixel 30 of the image sensor 1 are subjected to correlated double sampling and color separation processing by the CDS circuit 2, and the main pixel 31 and the sub-pixel 32 respectively. R, G, and B color signals are generated for each of the images. The RGB color signals of the main pixel 31 and the sub-pixel 32 are amplified by the analog amplifier circuit 3, converted into a digital signal by the AD converter 4, and then input to the digital signal processing unit 5.

  In the digital signal processing unit 5, the offset correction unit 21 performs offset processing of the R, G, and B color signals, and then the white balance adjustment unit 23 performs white balance adjustment corresponding to the selected white balance position. In this embodiment, the white balance adjustment (step S12) for the image signal by the main pixel 31 and the white balance adjustment (step S13) for the image signal by the sub-pixel 32 are separately performed corresponding to the white balance position.

  In this embodiment, for example, as shown in Table 1, the white balance position is determined according to the shooting scene such as “sunny”, “cloudy”, “shade”, “bulb”, “fluorescent light”, and the type of light source. A white balance adjustment gain value is set for each of the main pixel 31 and the sub-pixel 32 corresponding to each white balance position. In the example of Table 1, main pixel gains Rgain1, Ggain1, Bgain1 (first control value) and subpixel gains Rgain2, Ggain2, Bgain2 (second control value) are set, and each gain is set as a white balance control table. The value is stored in the memory 7. These gain values are adjusted and set for each individual imaging apparatus by, for example, measurement using a reference light source at the time of manufacture.

  The white balance adjustment unit 23 reads the white balance adjustment gain values of the R, G, and B color signals corresponding to the selected white balance position from the memory 7 and performs white balance adjustment. At this time, the color signals Radj1, Gadj1, Badj1, Radj2, Gadj2, and Badj2 in which the white balance is adjusted for each of the main pixel 31 and the subpixel 32 are obtained by performing calculations according to the following equations (1) to (6). .

Radj1 = R1 × Rgain1 (1)
Gadj1 = G1 × Ggain1 (2)
Badj1 = B1 × Bgain1 (3)
Radj2 = R2 × Rgain2 (4)
Gadj2 = G2 × Ggain2 (5)
Badj2 = B2 × Bgain2 (6)
Here, R 1, G 1, B 1, R 2, G 2, and B 2 are color signals before white balance adjustment obtained from the main pixel 31 and the sub-pixel 32, respectively.

  The selection of the white balance position is performed by manually setting the operation unit 8 in consideration of the shooting scene, the light source, and the like when the manual white balance control is performed.

  By such white balance adjustment, it is possible to appropriately adjust the color balance of the R, G, and B color components in the respective image signals of the main pixel 31 and the sub-pixel 32. When only the main pixel 31 is read out and image data is formed only from an image signal from the main pixel 31, the gain value for the main pixel is applied during white balance adjustment. When only the subpixel 32 is read out and image data is formed only from the image signal from the subpixel 32, the gain value for the subpixel is applied during white balance adjustment.

  Next, in the synthesis processing unit 24, the R, G, and B color signals whose color balance is adjusted in each of the main pixel 31 and the sub-pixel 32 are synthesized (step S14), and then in the YC processing unit 25, each pixel 30 A predetermined calculation is performed on each of the R, G, and B color signals corresponding to 1 to generate an image signal composed of the luminance signal Y and the color difference signals Cr and Cb (step S15). Then, the compression processing unit 26 compresses the image signal according to a predetermined compression encoding method such as JPEG (step S16), and the compressed image data is recorded in the external memory 9 by the recording processing unit 27 (step S17).

  In an imaging device equipped with an imaging device having a main pixel and a sub-pixel, the output level of both pixels is different even when imaged under the same shooting conditions, so that the spectral sensitivity characteristics are different between the main pixel and the sub-pixel. There is. For example, in such a case, if white balance control is performed using a single gain value for white balance adjustment, the color balance may be lost.

  If the white balance adjustment of the image signal obtained by the sub-pixel is performed using the gain optimized for the image signal obtained by the main pixel, the color balance of the image captured by the sub-pixel is accurate. It will not be reproduced. In particular, a colored state occurs on the high luminance side of an image in which a signal output from the sub-pixel is dominant. Conversely, when the white balance adjustment of the image signal obtained at the main pixel is performed using the gain optimized for the image signal obtained at the sub-pixel, the signal output from the main pixel becomes dominant. It becomes colored on the low brightness side. In any case, the image has a poor color balance.

  On the other hand, in the present embodiment, there are gain values optimally set corresponding to the main pixel 31 and the sub-pixel 32, respectively, and when performing white balance control, these gain values are used for the main pixel 31 and the sub-pixel. White balance adjustment is performed separately for each pixel 32. Thereafter, the image data of the main pixel 31 and the sub-pixel 32 are synthesized to obtain image data.

  As a result, in an imaging apparatus using an imaging device in which two pixels having different sensitivities are provided to realize a wide dynamic range, white balance control can be optimized and color reproducibility can be improved. Therefore, according to the present embodiment, a captured image having a wide dynamic range and excellent color reproducibility can be obtained.

  FIG. 4 is a flowchart showing processing when manual white balance control is performed in the second embodiment of the present invention. In the first embodiment, a configuration has been described in which gain values necessary for white balance adjustment are separately provided for the main pixel and for the sub-pixel. In the second embodiment, the gain value for the main pixel is used. An example in which only the white balance adjustment gain value is provided is shown.

  In this case, for example, as shown in Table 2, corresponding to the white balance position according to the shooting scene such as “sunny”, “cloudy”, “shade”, “bulb”, “fluorescent light”, and the type of light source, Only main pixel gains Rgain1, Ggain1, and Bgain1 are set for each image pickup apparatus, and each gain value is stored in the memory 7 as a white balance control table. Further, the data of the color misregistration ratio of the sub-pixel 32 with respect to the main pixel 31 is held. For the sub-pixel 32, white balance adjustment is performed by multiplying the gain for main pixel by the color shift ratio. Thereby, the color balance is adjusted with a gain value corresponding to the sub-pixel gain. Note that the color misregistration ratio may be appropriately adjusted for each white balance position when the color misregistration ratio is different from that of sunlight such as a fluorescent lamp.

  In the flowchart of FIG. 4, only step S33 for adjusting the white balance of the sub-pixel 32 is different from that of the first embodiment. Other steps S31 to S32 and S34 to S37 are the same as steps 11 to S12 and S14 to S17 of the first embodiment shown in FIG. 3, and the description thereof is omitted here.

  In step S33, the white balance adjusting unit 23 reads main pixel gains Rgain1, Ggain1, and Bgain1 corresponding to the white balance position selected as the white balance adjustment gain value from the memory 7, and the main pixel gain and color shift. The white balance adjustment for the sub-pixel 32 is performed using the ratio. At this time, by performing calculations according to the following equations (7) to (9), the color signals Radj2, Gadj2, and Badj2 in which the white balance is adjusted for the sub-pixel 32 are obtained.

Radj2 = R2 × Rgain1 × Rd (7)
Gadj2 = G2 × Ggain1 × Gd (8)
Badj2 = B2 × Bgain1 × Bd (9)
Here, R2, G2, and B2 are the color signals before white balance adjustment obtained from the sub-pixel 32, and Rd, Gd, and Bd are the color shift ratios of the colors of the sub-pixel 32 with respect to the main pixel 31, for example, Gd = 1. 0, Rd = 1.2, and Bd = 0.8.

  In the second embodiment, a gain value optimally set corresponding to the main pixel 31 is used, and when performing white balance control, the gain value and the color shift ratio of the sub-pixel 32 with respect to the main pixel 31 are used. White balance adjustment is performed separately for each of the main pixel 31 and the sub-pixel 32. As a result, as in the first embodiment, white balance control can be optimized and color reproducibility can be improved in an imaging apparatus using an imaging device that provides two pixels with different sensitivities to achieve a wide dynamic range. .

  In addition, this invention is not limited to embodiment mentioned above at all, In the range which does not deviate from the summary, it can implement in a various aspect. In the above embodiment, manual white balance control has been described, but the same applies to the case where auto white balance control is performed in which the apparatus automatically determines the light source and shooting scene at the time of shooting and performs optimal white balance adjustment. Applicable. In the auto white balance control, for example, the ratio of R / G, B / G is obtained from the integrated values of the R, G, B color signals read from the image sensor, and the value of these ratios and the subject in the automatic exposure control The white balance position is automatically determined based on the information on the luminance (shooting EV value). Then, using the gain values for main pixels and subpixels corresponding to the white balance position selected based on this automatic determination, white balance adjustment may be performed for each of the main pixels and subpixels.

It is a block diagram which shows the structure of the imaging device which concerns on embodiment of this invention. It is a schematic diagram which shows the pixel arrangement configuration of the image sensor used in the present embodiment. It is a flowchart which shows the process in the case of performing manual white balance control in 1st Embodiment of this invention. It is a flowchart which shows the process in the case of performing manual white balance control in 2nd Embodiment of this invention.

Explanation of symbols

1 Image sensor 5 Digital signal processor 6 CPU
7 Memory 8 Operation unit 9 External memory 22 Integration unit 23 White balance adjustment unit 30 Pixel 31 Main pixel 32 Sub pixel

Claims (4)

An imaging device including an imaging device having a plurality of pixels having different spectral sensitivity characteristics,
Storage means for holding a control value for white balance adjustment set corresponding to each of the plurality of pixels;
White balance adjustment means for performing white balance adjustment for each image signal obtained by a plurality of pixels of the image sensor using the control value ;
The imaging device has a high-sensitivity main pixel and a low-sensitivity subpixel as the plurality of pixels,
The storage means stores a first control value for white balance adjustment corresponding to the main pixel and a second control value for white balance adjustment corresponding to the sub-pixel,
The white balance adjustment unit uses the first control value and the second control value stored in the storage unit to adjust the white balance of the image signal by the main pixel and the white balance of the image signal by the sub pixel. An imaging device characterized by performing: The white balance adjustment means performs white balance adjustment by applying a control value for white balance adjustment set corresponding to the main pixel to an image signal of the main pixel. The imaging apparatus according to 1 . The white balance adjustment unit performs white balance adjustment by applying a control value for white balance adjustment set corresponding to the sub-pixel to an image signal of the sub-pixel. The imaging apparatus according to 1 . The imaging apparatus according to claim 1, wherein the storage unit holds a control value set for each white balance position corresponding to each of a plurality of photographing conditions.

Images