JP2004312553A - White balance control method and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a white balance control method capable of accurately carrying out white balance correction even when a dark object or an excessively bright object is photographed and to provide an imaging apparatus. <P>SOLUTION: The imaging apparatus uses a CCD for wide dynamic range imaging with high sensitivity pixels and low sensitivity pixels and acquires high sensitivity image data and a low sensitivity image data optically in phase by one imaging. The imaged image is divided into a plurality of blocks (e.g. 8×8), an RGB integral value is calculated for blocks in the image darker than a threshold value by using information of the high sensitivity pixels, an RGB integral value is calculated for blocks in the image brighter than the threshold value by using information of the low sensitivity pixels and the white balance gain is calculated from the integral values. Thus, the stability of the information as to each block is increased and the calculation accuracy of the white balance gain is enhanced. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a white balance control method and an imaging apparatus, and more particularly to a white balance control technique suitable for a digital camera, a video camera, and the like.
[0002]
[Prior art]
In Patent Document 1, in order to suppress the influence on color reproduction caused by variation in spectral characteristics of CCD units mounted on a digital camera, spectral sensitivity characteristics are detected for each individual camera, and the detected spectral sensitivity characteristics are converted into the detected spectral sensitivity characteristics. An adjustment method and apparatus for calculating a white balance adjustment value based on the adjustment value and setting the adjustment value in a camera is disclosed.
[0003]
Further, in Patent Document 2, in order to compensate for pixel defects caused by dark current accumulation effects during long exposure, pixels having a dark output level value of a certain number or less are detected as defective pixels, and these defects are detected. A technique for compensating for a pixel based on neighboring pixel information is disclosed.
[0004]
[Patent Document 1]
JP 2001-320715 A
[0005]
[Patent Document 2]
JP 2001-177768 A
[0006]
[Problems to be solved by the invention]
By the way, many digital cameras on the market now have a configuration in which the photographing sensitivity can be selected. For example, there is one that can switch the sensitivity in five steps corresponding to ISO 100/200/400/800/1600. In this type of digital camera system, there is no problem if shooting in a high sensitivity mode when shooting a dark scene or the like, but when shooting in a standard sensitivity mode such as ISO 100 or 200, the CCD output pixel level is high. It will be very low. When the CCD output level is low, color noise occurs, and the colors are mixed, making it difficult to achieve white balance with accurate RGB information.
[0007]
Therefore, even if a scene with a certain level of color information, such as a gray chart, is photographed, the RGB distribution of each pixel is not stabilized to the same value, and an image that causes a color shift in order to achieve white balance with the integrated value Become.
[0008]
The present invention has been made in view of such circumstances, and provides a white balance control method and an imaging apparatus capable of performing accurate white balance correction even when a dark scene or an excessively bright scene is photographed. With the goal.
[0009]
[Means for Solving the Problems]
In order to achieve the above object, the white balance control method according to the present invention includes a plurality of high-sensitivity pixels having relatively high sensitivity and low-sensitivity pixels having relatively low sensitivity, which are arranged according to a predetermined arrangement form. An imaging step of imaging a subject by an imaging unit having a structure capable of extracting an image signal from each of the sensitivity pixel and the low sensitivity pixel, and a block dividing step of dividing an image acquired via the imaging unit into a plurality of blocks And a brightness calculation step for calculating a value indicating the brightness of the block based on an image signal obtained from the imaging means for each of the divided blocks, and a threshold value that serves as an evaluation criterion for the brightness of each block. A comparison step of comparing a threshold value setting step, a value indicating the brightness of each block calculated in the brightness calculation step with the threshold value, and the comparison process; According to the comparison result of the above, white blocks are used so that an image signal obtained from the high-sensitivity pixel is used for blocks darker than the threshold value, and an image signal obtained from the low-sensitivity pixel is used for blocks brighter than the threshold value. A switching control step for switching an image signal for balance calculation, a gain calculation step for calculating a gain for white balance adjustment based on the image signal for white balance calculation selected in the switching control step, and the gain calculation step And a white balance adjustment step of performing white balance adjustment by correcting the color signal based on the gain obtained in (1).
[0010]
The imaging means used in the present invention has a composite pixel structure in which a high sensitivity pixel and a low sensitivity pixel are combined. With these two types of pixels (photoelectric conversion units) having different sensitivities, it is possible to acquire optically in-phase image information (high sensitivity image information and low sensitivity image information) by one imaging. A gain for white balance adjustment is calculated based on an image signal output from the image pickup means. In the present invention, information on high sensitivity pixels or information on low sensitivity pixels is used in a stage prior to white balance calculation. Or selectively switching.
[0011]
The captured image is divided into multiple blocks, and white balance calculation is performed using high-sensitivity pixel information for blocks darker than the threshold in the image and low-sensitivity pixel information for blocks brighter than the threshold. Thus, the stability of information for each block is improved, and the white balance gain can be determined based on accurate information.
[0012]
In order to provide an apparatus that embodies the above-described method invention, an imaging apparatus according to the present invention includes a plurality of high-sensitivity pixels having relatively high sensitivity and low-sensitivity pixels having relatively low sensitivity according to a predetermined arrangement form. An image pickup unit having a structure capable of taking out a signal photoelectrically converted by the main photosensitive pixel and a signal photoelectrically converted by the sub-photosensitive pixel, and images captured by the image pickup unit into a plurality of blocks. Block division setting means for dividing, brightness calculation means for calculating a value indicating the brightness of the block based on an image signal obtained from the imaging means for each of the divided blocks, and evaluation criteria for the brightness of each block The threshold value setting means for setting the threshold value to be compared with the threshold value and the value indicating the brightness of each block calculated by the brightness calculation means, and is darker than the threshold value Switch control for switching the image signal for white balance so that the image signal obtained from the high-sensitivity pixel is used for locking and the image signal obtained from the low-sensitivity pixel is used for a block brighter than the threshold A gain calculating means for calculating a gain for white balance adjustment based on the image signal for white balance calculation selected by the switching control means, and correcting the color signal based on the gain obtained by the gain calculating means. And white balance adjustment means for performing white balance adjustment.
[0013]
According to an aspect of the present invention, the imaging unit has a structure in which each light receiving cell is divided into a plurality of light receiving regions including at least the high sensitivity pixel and the low sensitivity pixel, and is identical above each light receiving cell. A color filter having the same color component is disposed for the high-sensitivity pixel and the low-sensitivity pixel in the light-receiving cell, and each light-receiving cell is provided with one microlens for each light-receiving cell. It is characterized by.
[0014]
The imaging means having such a structure can handle the high-sensitivity pixel and the low-sensitivity pixel in the same light receiving cell (pixel cell) as having substantially the same pixel position. Therefore, it is possible to acquire two pieces of image information at the same phase in time and substantially in the same position in a single imaging.
[0015]
According to another aspect of the present invention, the brightness calculation unit calculates an integrated value of R, G, and B color signals obtained from the imaging unit. In the case of an image signal that has been color-separated into three primary colors of R (red), G (green), and B (blue), the R and B gains based on G may be calculated, and the gains for each of the RGB colors May be calculated. There is a mode in which an integrated value of each color of R, G, B is calculated for each divided block, and white balance calculation is performed using the integrated value.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.
[0017]
[Image sensor structure]
First, the structure of the image sensor will be described. FIG. 1 is a schematic plan view showing an example of a CCD solid-state imaging device (hereinafter referred to as a CCD) used in the practice of the present invention, and FIG. 2 is an enlarged view of a main part thereof. As shown in these drawings, the CCD 10 is a two-dimensional imaging device (image sensor) in which a large number of light receiving cells 20 are arranged at a constant arrangement period in the horizontal direction (row direction) and the vertical direction (column direction). The illustrated configuration is a pixel array called a honeycomb array, in which the center points of the geometric shapes of the light receiving cells 20 are shifted by half the pixel pitch (1/2 pitch) every other row direction and column direction. It has become. That is, in the rows (or columns) of the light receiving cells 20 adjacent to each other, the cell arrangement in one row (or column) is in the row direction (or column direction) with respect to the cell arrangement in the other row (or column). The structure is arranged so as to be relatively shifted by about ½ of the arrangement interval.
[0018]
Each light receiving cell 20 includes two photodiode regions 21 and 22 having different sensitivities. The first photodiode region 21 has a relatively large area and constitutes a main photosensitive portion (hereinafter referred to as “high sensitivity pixel”) having high sensitivity. The second photodiode region 22 has a relatively small area and constitutes a secondary photosensitive portion (hereinafter referred to as “low sensitivity pixel”) having low sensitivity.
[0019]
For each light receiving cell 20, the high-sensitivity pixel 21 and the low-sensitivity pixel 22 are provided with color filters of the same color. That is, one primary color filter of RGB is assigned to each light receiving cell 20. As shown in FIG. 1, the BRBR... Row is arranged in the next stage of the GGGG... Row in the horizontal direction, the GGGG... Row is arranged in the next stage, and the RBRB. Further, in the column direction, the GGGG column, the BRBR column, the GGGG column, and the RBRB column are repeated in a cyclic manner.
[0020]
The opening shape of the light receiving cell 20 is not limited to a quadrangle, and may be a polygon such as a hexagon or an octagon, or a circle. Further, the separation shape (divided form) of the light receiving cell 20 is not limited to the shape shown in FIG. 1, and it is sufficient that the accumulated charges can be read out separately from each divided region. The relationship is designed as appropriate. For example, one light receiving cell 20 may be divided into three or more light receiving regions. In carrying out the present invention, the color separation is not limited to the three primary colors of R, G, and B, and color is performed using a complementary color system color filter of yellow (Y), magenta (M), and cyan (C). A mode of decomposition is also possible.
[0021]
A vertical transfer path (VCCD) 30 is formed on the right side of the light receiving cell 20. The vertical transfer path 30 extends in the vertical direction by meandering in a zigzag manner in the vicinity of each column of the light receiving cells 20 and avoiding the light receiving cells 20.
[0022]
On the vertical transfer path 30, transfer electrodes 31, 32, 33, and 34 necessary for four-phase driving (φ1, φ2, φ3, φ4) are arranged. The transfer electrodes 31 to 34 are provided close to each row of the light receiving cells 20 so as to meander while avoiding the opening of the light receiving cells 20 and extend in the horizontal direction in FIG. For example, when the transfer electrode is formed of two-layer polysilicon, the first transfer electrode 31 to which the pulse voltage φ1 is applied and the third transfer electrode 33 to which the pulse voltage φ3 is applied are the first polysilicon layer. The second transfer electrode 32 to which the pulse voltage φ2 is applied and the fourth transfer electrode 34 to which the pulse voltage φ4 is applied are formed of a second polysilicon layer. Of course, in carrying out the present invention, the charge transfer method and the configuration of the transfer electrode are not limited to the above examples.
[0023]
In FIG. 1, a VCCD driving circuit 42 that applies a pulse voltage to the transfer electrodes 31 to 34 is disposed on the right side of the imaging area 40 in which the light receiving cells 20 are arranged. A horizontal transfer path (HCCD) 44 that transfers the signal charges transferred from the vertical transfer path 30 in the horizontal direction is provided below the imaging area 40 (at the lower end side of the vertical transfer path 30).
[0024]
The horizontal transfer path 44 is composed of a two-phase drive transfer CCD, and the last stage (the leftmost stage in FIG. 1) of the horizontal transfer path 44 is connected to the output unit 46. The output unit 46 includes an output amplifier, performs charge detection of the input signal charge, and outputs it to the output terminal 48 as a signal voltage. In this way, the signal photoelectrically converted by each light receiving cell 20 is output as a dot-sequential signal sequence.
[0025]
In the CCD 10 of this example, it is possible to independently read out the signal charges accumulated in the high sensitivity pixel 21 and the low sensitivity pixel 22 by one exposure. That is, as shown in FIG. 2, the signal charges respectively generated by the high sensitivity pixel 21 and the low sensitivity pixel 22 are separately read out and mixed into the vertical transfer path 30 adjacent to the right side of the light receiving cell 20. Without being transferred.
[0026]
As shown in FIG. 3, one microlens 50 is disposed above each light receiving cell 20 for each light receiving cell 20 so that incident light is efficiently incident on the light receiving cell 20. .
[0027]
FIG. 4 is a sectional view of the light receiving cell 20. This figure shows a cross-sectional view taken along line 4-4 of FIG. According to FIG. 4, a p-type well 62 is formed on an n-type semiconductor substrate 60, and two n-type regions 63 and 64 are formed in the surface region of the p-type well 62 to constitute a photodiode. ing. A photodiode in the n-type region indicated by reference numeral 63 corresponds to the high-sensitivity pixel 21, and a photodiode in the n-type region indicated by reference numeral 64 in FIG. 4 corresponds to the low-sensitivity pixel 22.
[0028]
Between the two n-type regions 63 and 64, p ⁺ A mold separation region 65 is formed. The isolation region 65 functions as a channel stop region (channel stopper), and electrically isolates the photodiode region. In addition, p indicated by reference numeral 66 in FIG. ⁺ The mold region is a channel stopper for electrically separating the light receiving cell 20 and the vertical transfer path 30 and the like.
[0029]
On the upper layer surface of the n-type regions 63 and 64 constituting the photodiode, a light shielding film 68 having openings for defining light incident openings to the respective photodiode regions corresponding to the high-sensitivity pixel 21 and the low-sensitivity pixel 22. Is formed. An interlayer insulating film 70 made of phosphosilicate glass or the like is formed on the upper surface of the semiconductor substrate so as to cover the light shielding film 68.
[0030]
The upper surface of the interlayer insulating film 70 is flattened with high accuracy, and a color filter layer (on-chip color filter) 72 that functions as a color separation filter is formed thereon. A microlens (on-chip microlens) 50 is disposed on the color filter layer 72 corresponding to each light receiving cell 20. The microlens 50 has a function of collecting light incident from above into an opening defined by the light shielding film 68.
[0031]
Light incident through the microlens 50 is color-separated by the color filter layer 72 and is incident on the photodiode regions of the high-sensitivity pixel 21 and the low-sensitivity pixel 22, respectively. Light incident on each photodiode region (21, 22) is converted into a signal charge corresponding to the amount of light, and is separately read out to the vertical transfer path 30. Since the high-sensitivity pixel 21 and the low-sensitivity pixel 22 are partitioned by the separation region 65, signal charges accumulated in the respective photodiode regions are not mixed in the light receiving cell 20.
[0032]
Although the vertical transfer path 30 is not shown in FIG. ⁺ An n-type region constituting the vertical transfer path 30 is formed outside the mold region (channel stopper) 66. The n-type region, the n-type regions of the high-sensitivity pixel 21 and the low-sensitivity pixel 22, and the p-type well 62 between them constitute a readout transistor. An insulating layer such as a silicon oxide film is formed on the upper layer surface of the vertical transfer path 30, and a transfer electrode is disposed thereon so as to cover the upper side of the vertical transfer path 30. An insulating layer such as silicon oxide is further formed on the transfer electrode, and a light shielding film (film integrated with the light shielding film 68) covering the vertical transfer path 30 is formed thereon.
[0033]
In this way, charge transfer paths and transfer electrodes are formed between adjacent light receiving cells 20, and each light receiving cell 20 is partitioned through the charge transfer paths and transfer electrodes. In FIG. 20, the light receiving region is partitioned only by the separation region 65 (channel stopper). Therefore, two finer independent light receiving regions can be formed in one light receiving cell 20 treated as one pixel, and two types of image signals (high sensitivity image signals) having different sensitivities can be obtained from one light receiving cell 20. And a low-sensitivity image signal) can be taken out separately, and an optically in-phase image signal can be obtained.
[0034]
FIG. 5 is a graph showing the photoelectric conversion characteristics of the high sensitivity pixel 21 and the low sensitivity pixel 22. The horizontal axis represents the amount of incident light, and the vertical axis represents the image data value (QL value) after A / D conversion. In this example, 12-bit data is illustrated, but the number of bits is not limited to this.
[0035]
As shown in the figure, the sensitivity ratio between the high-sensitivity pixel 21 and the low-sensitivity pixel 22 is 1: 1 / a (where a> 1, a = 16 in this example). The output of the high-sensitivity pixel 21 gradually increases in proportion to the amount of incident light, and when the amount of incident light is “c”, the output reaches a saturation value (QL value = 4095). Thereafter, even if the amount of incident light increases, the output of the high sensitivity pixel 21 becomes constant. This “c” will be referred to as the saturation light amount of the high sensitivity pixel 21.
[0036]
On the other hand, the sensitivity of the low-sensitivity pixel 22 is 1 / a of the sensitivity of the high-sensitivity pixel 21 and is saturated at a QL value = 4095 / b when the incident light quantity is α × c (where b> 1, α = a / b, in this example b = 4, α = 4). “Α × c” at this time is referred to as a saturated light amount of the low sensitivity pixel 22.
[0037]
In this way, by combining the high-sensitivity pixel 21 and the low-sensitivity pixel 22 having different sensitivities and saturations, the dynamic range of the image sensor can be expanded α times compared to the configuration of only the high-sensitivity pixels. In this example, the dynamic range is expanded about four times with a sensitivity ratio of 1/16 and a saturation ratio of 1/4. When the maximum dynamic range when only the high-sensitivity pixel 21 is used is 100%, the dynamic range is expanded up to about 400% by utilizing the low-sensitivity pixel 22 in this example.
[0038]
[Configuration example of digital camera]
Next, a camera equipped with the above-described wide dynamic range imaging CCD 10 will be described.
[0039]
FIG. 6 is a block diagram of the digital camera according to the present embodiment. The taking lens 110 of the camera 100 may be a single focus lens or a variable focal length such as a zoom lens. Light that has passed through the taking lens 110 and the aperture 112 enters the CCD 10. A large number of photosensors (light receiving elements) are two-dimensionally arranged on the light receiving surface of the CCD 10, and red (R), green (G), and blue (B) primary color filters corresponding to each photosensor are predetermined. Arranged in array form.
[0040]
The subject image formed on the light receiving surface of the CCD 10 is converted into a signal charge in an amount corresponding to the amount of incident light by each sensor on the CCD 10.
[0041]
The signal charges accumulated in this way are read out to the shift register by a read gate pulse applied from the CCD drive circuit 116, and sequentially read out as a voltage signal corresponding to the signal charge by a register transfer pulse. The CCD 10 has a so-called electronic shutter function that can sweep out the accumulated signal charge by a shutter gate pulse, thereby controlling the charge accumulation time (shutter speed).
[0042]
The voltage signal (image signal) sequentially read out from the CCD 10 is applied to a correlated double sampling circuit (CDS circuit) 118, where the R, G, B signals for each light receiving element (pixel) are sampled and held. It is added to the A / D converter 120. The A / D converter 120 converts the R, G, and B signals sequentially added from the CDS circuit 118 into digital signals and outputs them. The CCD drive circuit 116, the CDS circuit 118, and the A / D converter 120 are driven in synchronism with a timing signal applied from the timing generation circuit 122.
[0043]
The R, G, B signals output from the A / D converter 120 are temporarily stored in the memory 124, and then the R, G, B signals stored in the memory 124 are added to the digital signal processing circuit 126. The digital signal processing circuit 126 includes a synchronization circuit 128, a white balance adjustment circuit 130, a gamma correction circuit 132, a luminance / color difference signal (YC signal) creation circuit 134, and a memory 136.
[0044]
The synchronization circuit 128 converts the dot-sequential R, G, B signals read from the memory 124 into a simultaneous system and outputs the R, G, B signals to the white balance adjustment circuit 130. The white balance adjustment circuit 130 includes multipliers 130R, 130G, and 130B for increasing and decreasing the digital values of the R, G, and B signals. The R, G, and B signals are respectively multiplied by the multipliers 130R, 130G, and 130B, respectively. Added to 130B.
[0045]
White balance correction values (gain values) Rg, Gg, and Bg for white balance control from a central processing unit (CPU) 138 are added to other inputs of the multipliers 130R, 130G, and 130B. , 130G, and 130B each multiply two inputs, and output R ′, G ′, and B ′ signals that have undergone white balance adjustment by this multiplication to the gamma correction circuit 132. Details of the white balance correction values Rg, Gg, and Bg applied from the CPU 138 to the white balance adjustment circuit 130 will be described later.
[0046]
The gamma correction circuit 132 changes the input / output characteristics so that the white balance adjusted R ′, G ′, B ′ signals have desired gamma characteristics, and outputs them to the YC signal generation circuit 134. The YC signal generation circuit 134 generates a luminance signal Y and chroma signals Cr and Cb from the gamma-corrected R, G, and B signals. These luminance signal Y and chroma signals Cr and Cb (YC signal) are stored in the memory 136 in the same memory space as the memory 124.
[0047]
Here, by reading the YC signal in the memory 136 and outputting it to the liquid crystal monitor 152, the moving image video (through image) being captured or the recorded still image can be displayed on the liquid crystal monitor 152. Further, the YC signal captured in response to pressing of the shutter button included in the operation unit 140 and stored in the memory 136 is compressed according to a predetermined format such as JPEG by the compression / decompression circuit 154, and then the memory card 156 and the like. Recorded on the recording medium.
[0048]
Smart media (Solid-State Floppy Disk Card), PC card, compact flash, magnetic disk, optical disk, magneto-optical disk, memory stick, etc. can be applied as a recording medium as means (recording unit) for storing image data, Various media that can be read and written in accordance with an electronic, magnetic, optical, or combination thereof can be used. A signal processing means and an interface corresponding to the medium to be used are applied. A configuration in which a plurality of media can be mounted regardless of different types or the same type of recording media may be adopted. The means for storing the image is not limited to a removable medium that can be attached to and detached from the camera body, but may be a recording medium (internal memory) built in the camera 100.
[0049]
In the reproduction mode, the image data recorded on the memory card 156 is read out, decompressed by the compression / expansion circuit 154, and then output to the liquid crystal monitor 152. In this way, the reproduced image is displayed on the liquid crystal monitor 152.
[0050]
The camera 100 is provided with an operation unit 140 for a user to input various commands. The operation unit 140 includes various operation units such as a shutter button, a zoom switch, and a mode change switch. The shutter button is an operation means for inputting an instruction to start photographing, and is composed of a two-stroke switch having an S1 switch that is turned on when half-pressed and an S2 switch that is turned on when fully pressed. When S1 is on, shooting preparation operations such as AE and AF processing are performed, and when S2 is on, exposure for recording is performed. The zoom switch is an operation means for changing the photographing magnification and the reproduction magnification. The mode switch is an operation means for switching between the shooting mode and the playback mode.
[0051]
In addition to the above, the operation unit 140 has a shooting mode setting for setting an optimum operation mode (continuous shooting mode, auto shooting mode, manual shooting mode, portrait mode, landscape mode, night view mode, etc.) according to the shooting purpose. Means, a menu button for displaying a menu screen on the LCD monitor 152, a cross button (cursor moving operation means) for selecting a desired item from the menu screen, an OK button for instructing confirmation of the selected item and execution of processing, a selection item, etc. Cancel button, cancel instruction contents, cancel button to input command to return to previous operation state, turn on / off LCD monitor 152, switch display method, display / display on on-screen display (OSD) Display buttons for switching non-display, dynamic range expansion processing (image composition) Operation means such as a D range expansion mode switch for selecting whether to apply or not are also included.
[0052]
The operation unit 140 is not limited to a push-type switch member, dial member, lever switch, or the like, but includes an operation unit 140 realized by a user interface for selecting a desired item from a menu screen. It is.
[0053]
A signal from the operation unit 140 is input to the CPU 138. The CPU 138 controls each circuit of the camera 100 based on an input signal from the operation unit 140, and controls, for example, autofocus (AF), automatic exposure control (AE), auto white balance (AWB), lens drive control, Control of photographing operation, charge readout control from the CCD 10, image processing control, image data recording / reproduction control, file management in the memory card 156, display control of the liquid crystal monitor 152, and the like are performed.
[0054]
Further, the camera 100 of this example has a strobe device 146, and an automatic flash mode in which the strobe device 146 automatically emits light at low brightness in response to an operation of a strobe key (not shown), regardless of subject brightness. Is set to a forced light emission mode in which light is emitted or a light emission inhibition mode in which light emission of the strobe device 146 is prohibited.
[0055]
The CPU 138 controls charging of a main capacitor (not shown), discharge (light emission) timing to a light emitting tube (for example, a xenon tube) and the like according to a strobe mode selected by the user. Further, the CPU 138 performs white balance control in accordance with the setting of the strobe mode.
[0056]
The EEPROM 158 stores data necessary for control of AE, AF, AWB, etc., or customization information set by the user. The information in the EEPROM 158 can be rewritten as necessary, and the information content is retained even when the power is turned off. The CPU 138 refers to the data in the EEPROM 158 as necessary to perform calculations.
[0057]
The AF control of the camera 100 is, for example, contrast AF that moves the taking lens 110 so that the high frequency component of the G signal is maximized, and the lens driving unit so that the high frequency component of the G signal is maximized when the shutter button is half-pressed. The photographing lens 100 is moved to the in-focus position via 142.
[0058]
In the AE control, as shown in FIG. 7, the R, G, B signals are fetched a plurality of times (up to four times) at predetermined exposures (1) to (4), and these R, G, B signals are integrated. Subject brightness (shooting EV value) is obtained based on the integrated value. In this way, photometry is performed in a plurality of times to measure the wide range (5 to 17 EV), and the optimum exposure condition is determined. Note that the range that can be measured by one photometry and the range that should be measured can be appropriately designed for each camera model. Further, an integrated value may be obtained for each of the RGB color signals, or an integrated value may be obtained for only one of these colors (for example, the G signal).
[0059]
Next, details of the measurement of the photographing EV value will be described.
[0060]
As shown in FIG. 8, one screen is divided into a plurality of areas (for example, 8 × 8), luminance signals obtained from R, G, B signals are integrated for each divided area (block), and the integrated value is obtained. Based on the above, the EV value (EVi) of each divided area is obtained. Subsequently, as shown in FIG. 8, the EV value of each divided area is weighted corresponding to the shooting mode, and the EV ′ value of the entire screen is calculated by the following equation.
[0061]
[Expression 1]
EV ′ = log ₂ {Σ (W _i × 2 ^EVi ) / ΣW _i }
However, i: 0 to 63 (subscript indicating an 8 × 8 divided area)
W _i : Weighting factor for each divided area according to the shooting mode
That is, when the shooting mode is the auto shooting mode or the portrait shooting mode, the center-weighted metering method is used as shown in the weighting coefficient of FIG. 8A, and when the shooting mode is the landscape shooting mode, it is shown in FIG. Thus, the photometry method is obtained by reducing the weighting of the divided areas located on the outermost periphery, and in the night scene shooting mode, the average photometry method is used as shown in FIG.
[0062]
The EV ′ calculated as described above is further subjected to exposure correction ΔEV corresponding to the shooting mode as shown in the following equation to obtain the shooting EV value.
[0063]
[Expression 2]
EV = EV′−ΔEV
Note that ΔEV is, for example, ΔEV = 0 in the portrait shooting mode, and ΔEV = 0.3 in the landscape shooting mode and the night scene shooting mode.
[0064]
The aperture value and shutter speed at the time of shooting are finally determined based on the shooting EV value obtained as described above.
[0065]
Then, when the shutter button is fully pressed, the diaphragm 112 is driven via the diaphragm driver 144 of FIG. 6 so that the determined diaphragm value is obtained, and the electric charge is accumulated by the electronic shutter so that the determined shutter speed is achieved. Control the time.
[0066]
The camera 100 of this example reads data from only the high sensitivity pixel 21 during the through image, and creates an image for the through image from the image signal of the high sensitivity pixel 21. Further, the AE process and the AF process associated with S1 = ON of the shutter button are performed based on a signal obtained from the high sensitivity pixel 21. When the mode for performing the wide dynamic range recording is selected, or when the wide dynamic range recording mode is automatically selected based on the AE result (ISO sensitivity or photometric value), the shutter button In response to S2 = ON, the CCD 10 is exposed, and after the exposure, the mechanical shutter is closed to block the entrance of light, and in synchronization with the vertical drive signal (VD), first, the charge of the high-sensitivity pixel 21 is read. Thereafter, the charge of the low sensitivity pixel 22 is read.
[0067]
Next, an auto white balance control method will be described.
[0068]
In the camera 100 of this example, the entire imaging screen is divided into a plurality of area blocks (for example, 8 × 8 as shown in FIG. 9), and the RGB level obtained from the CCD 10 for each pixel is integrated for each block. Then, an average value per pixel is obtained for each color. Then, for each block, the ratio R / G between the integrated average value of the R signal and the integrated average value of the G signal and the ratio B / G of the integrated average value of the B signal and the integrated average value of the G signal are obtained.
[0069]
Note that the average integrated value of the R, G, and B signals for each block is calculated by the integrating circuit 148 shown in FIG. 6 and sent to the CPU 138. In addition, multipliers 150R, 150G, and 150B are provided between the integration circuit 148 and the CPU 138, and adjustment gain values for adjusting variations in devices are added to the multipliers 150R, 150G, and 150B. It has become.
[0070]
The gain value of the white balance is calculated using the values R / G and B / G of each block thus obtained. In this embodiment, the data of the high-sensitivity pixel 21 of the CCD 10 is used in the previous stage, or the low value is calculated. Whether to use the data of the sensitivity pixel 22 is determined based on the luminance. The threshold value of the luminance information serving as a determination criterion is set in the EEPROM 158 in the camera 100. The CPU 138 refers to the threshold value of luminance information set in the EEPROM 158 to determine which pixel data is to be used, and performs control to switch the usage information.
[0071]
An accurate RGB integrated value can be calculated by using high sensitivity pixel information for blocks darker than the threshold in the image, and using low sensitivity pixel information for blocks that are too bright.
[0072]
FIG. 10 is a block diagram showing a main configuration of the camera 100.
[0073]
The image signal output from the CCD 10 is converted into a digital signal by the A / D converter 120. The image data acquired when the shutter button S1 is turned on is sent to the AE integrated value calculation unit 170, where the EV value is calculated. Exposure conditions (aperture and shutter speed) are determined from the EV value obtained here, and information on the EV value is sent to a brightness determination unit 174 in the white balance adjustment unit 172.
[0074]
On the other hand, the image data acquired when the shutter button S2 is turned on is subjected to offset processing after A / D conversion and sent to the linear matrix circuit 176. The offset process is a process for correcting the dark current component of the CCD output, and performs an operation of subtracting the integrated value (average value) of the optical black (OB) signal obtained from the light-shielded pixel on the CCD 10 from the pixel value. The OB integrated value calculation unit 178 calculates an integrated value of the OB signal obtained from the light-shielded pixel on the CCD 10 and stores the value. This OB integrated value is referred to during the offset process.
[0075]
The RGB data of the low sensitivity pixel 22 corrected by the linear matrix circuit 176 is sent to the integrated value calculation unit 182 for the high sensitivity pixel. The integrated value calculation unit 182 divides the entire image for one screen obtained from the low-sensitivity pixels 22 into 8 × 8 blocks, and calculates the RGB integrated value of each block.
[0076]
The RGB data of the high sensitivity pixel 21 corrected in the linear matrix circuit 92 is sent to the integrated value calculation unit 180 for the high sensitivity pixel. The integrated value calculation unit 180 divides the entire image for one screen obtained from the high sensitivity pixel 21 into 8 × 8 blocks, and calculates the RGB integrated value of each block.
[0077]
Information calculated by the integrated value calculation unit 180 for high-sensitivity pixels is sent to the threshold value determination unit 184, and the brightness of each block is compared with a predetermined threshold value. When the brightness of the block is lower than a predetermined threshold, the integrated value information obtained from the high sensitivity pixel 21 is used for the block. When the brightness of the block is higher than a predetermined threshold, the integrated value information obtained from the low sensitivity pixel 22 is used for the block.
[0078]
For example, when the brightness is evaluated with a value of 0 to 255, the luminance threshold is set to “240”, and information on the integrated value obtained from the low-sensitivity pixel 22 is used for a block having a luminance of 240 or higher. For the block with luminance lower than 240, the integrated value information of the high sensitivity pixel 21 is used.
[0079]
Thus, the brightness is determined for each block, and it is determined which integrated value of the high sensitivity pixel 21 or the low sensitivity pixel 22 is used. The integrated value used for the white balance calculation is selected for each 8 × 8 block according to the determination of the threshold determining unit 184, and integrated value information of each selected block is stored in the memory 186 (information on the high-sensitivity pixel 21). And a block using the information of the low-sensitivity pixel 22 may be mixed). Note that the memory 186 may be the same memory space as the memory 24 described in FIG. 6, or may be a different memory space.
[0080]
The integrated value information of the 8 × 8 blocks stored in the memory 186 is sent to the R / G, B / G value detecting unit 188 in FIG. 10, and the integrated average value of the R signal and the integrated average of the G signal for each block. The ratio R / G with the value value and the ratio B / G between the integrated average value of the B signal and the integrated average value of the G signal are calculated.
[0081]
The color information R / G and B / G obtained for each block is sent to the light source discriminating unit 190 and used for detection of the light source. A white balance gain calculation unit 192 calculates a white balance gain based on the determination results of the light source determination unit 190 and the brightness determination unit 174. Details of the light source discrimination and white balance gain calculation processing will be described later.
[0082]
With the determined gain value (white balance correction value), the balance of each color signal is adjusted in the white balance adjustment circuit 30 described with reference to FIG. The image signal that has been subjected to white balance adjustment by the white balance adjustment circuit 130 is converted into a final image through necessary processing such as gamma conversion in the gamma conversion unit 194 shown in FIG. Or the output unit 196 such as the memory card 156.
[0083]
When the image is shot in the wide dynamic range recording mode, the high sensitivity image data obtained from the high sensitivity pixel 21 and the low sensitivity image data obtained from the low sensitivity pixel 22 are combined in the digital signal processing circuit 126. In (not shown), they are combined (added) to create a wide dynamic range image. The synthesis processing unit adds (synthesizes) the multiplication unit that multiplies the high-sensitivity image data by a coefficient, the multiplication unit that multiplies the low-sensitivity image data by a coefficient, and the high-sensitivity image data and the low-sensitivity image data after the coefficient multiplication. And a processing unit. Each coefficient (coefficient indicating the addition ratio) multiplied by the high-sensitivity image data and the low-sensitivity image data is variably set by the CPU 138.
[0084]
In this example, setting information indicating how much the dynamic range during wide dynamic range recording can be input can be input through a predetermined user interface. The gamma coefficient, the synthesis parameter at the time of addition, the gain coefficient of the color difference signal matrix circuit, and the like are controlled according to the specified dynamic range information. A non-volatile memory (ROM or EEPROM 158 not shown) in the camera 100 stores table data defining various parameters, coefficients, etc. corresponding to the specified dynamic range.
[0085]
FIG. 11 is a graph showing the relationship between the level of the final image (synthesized image data) obtained by combining high-sensitivity image data and low-sensitivity image data and relative subject brightness. Relative subject brightness represents subject brightness with reference to 100% as subject brightness that gives a level at which high-sensitivity image data is saturated. In FIG. 11, the image data is represented by 8 bits (0 to 255), but the number of bits is not limited to this.
[0086]
In this example, it is assumed that the dynamic range can be set in six stages from D0 to D5. For example, the dynamic range can be switched in stages such that the relative subject brightness is 100% -130% -170% -220% -300% -400%. Of course, the number of setting stages of the dynamic range is not limited to this example, it can be designed to an arbitrary number of stages, and continuous setting (no stage) is also possible. In addition, since the required dynamic range differs depending on the shooting scene, there is also an aspect in which the dynamic range is automatically set by analyzing the captured image. Furthermore, a mode in which the setting of the dynamic range is automatically switched according to the photographing mode such as the portrait mode and the night view mode is also possible.
[0087]
FIG. 12 shows the overall flow of auto white balance control.
[0088]
When the strobe device 146 described in FIG. 6 is caused to emit light, white balance correction values Rg, Gg, and Bg for performing good white balance with respect to the strobe light are added to the white balance adjustment circuit 130. Hereinafter, white balance control when the strobe does not emit light will be described.
[0089]
As shown in FIG. 12, when an image signal is output from the CCD 10 (step S10), an integrated value of 8 × 8 blocks is calculated for each of the high sensitivity pixel 21 and the low sensitivity pixel 22 (step S12). For each integrated block, the integrated value of the high-sensitivity pixel 21 is compared with a luminance threshold value, and it is determined whether or not the integrated block is darker than the threshold value (step S14).
[0090]
In step S14, the integrated value of the signal obtained from the high-sensitivity pixel 21 is used for the block determined to be darker than the threshold (step S16), and conversely, the block determined to be brighter than the threshold is low-sensitivity. It is assumed that the integrated value of the signal obtained from the pixel 22 is used (step S18).
[0091]
In step S16 or step S18, an integrated value of 8 × 8 blocks for white balance calculation is determined (step S20). Based on this integrated value information, a white balance gain (a coefficient for multiplying each color signal) is calculated, and white balance adjustment is performed (step S22).
[0092]
In this way, when the image creation is completed, it is determined whether or not the next image is captured (step S24). When the next shooting is performed, the process returns to step S10 and the above-described processing is repeated. On the other hand, if the next image is not captured, such as when the shooting mode is canceled in step S24, this sequence ends (step S26).
[0093]
FIG. 13 is a flowchart illustrating a calculation example of the white balance coefficient (gain). FIG. 13 shows an example of auto white balance control when the auto shooting mode or the person shooting mode is selected as the shooting mode.
[0094]
First, the photographing EV value obtained when the shutter button is half-pressed is acquired (step S30). As described with reference to FIG. 10, the EV value information calculated by the AE integrated value calculation unit 170 is sent to the brightness determination unit 174.
[0095]
Subsequently, the R, G, and B signals for one screen output from the A / D converter 120 when the shutter button is fully pressed are divided into 8 × 8 blocks, and an integrated value is obtained for each block to determine a threshold value determination unit 184. Is compared with the threshold value, information on the high sensitivity pixel 21 or the low sensitivity pixel 22 is selected, and R / G and B / G are obtained for each block (step S32 in FIG. 13).
[0096]
The color information R / G and B / G obtained for each block as described above indicates which detection frame is included in the detection frames represented by the color coordinates in FIG. Used to determine Note that the detection frames such as the shade-clouding detection frame and the daylight color detection frame in FIG. 14 are frames represented on the color coordinates with the horizontal axis being R / G and the vertical axis being B / G. A range of color distribution such as a light source type is defined for each frame.
[0097]
In FIG. 14, a blue sky detection frame, a shade-cloudy detection frame, a daylight color detection frame, a green detection frame, a daylight white-white fluorescent lamp detection frame divided into six regions, a warm white detection frame, a skin color detection frame, and divided into two regions. Although a tungsten detection frame is shown, the types and number of detection frames, the setting range, and the like are not limited to this example, and various designs are possible.
[0098]
Using the detection frames shown in FIG. 14, a distribution of 64 points (R / G, B / G) corresponding to each divided area of the imaging screen is obtained, and the number of points output to each detection frame is detected.
[0099]
Next, an evaluation value of shade-cloudiness, an evaluation value of a fluorescent lamp (daylight color, daylight white-white, warm white), and an evaluation value of a tungsten light bulb are expressed by the following formula:
[0100]
[Equation 3]
Evaluation value of shade-cloudy = F (like outdoor) × F (shade-like cloudy) × F (blue sky)
[0101]
[Expression 4]
Daylight color evaluation value = F ₁ (Like indoor) x F (Like daylight fluorescent light)
[0102]
[Equation 5]
Day white-white evaluation value = F ₁ (Looks like indoor) x F (Like white-white fluorescent light)
[0103]
[Formula 6]
Evaluation value of warm white = F ₁ (Indoor) x F (warm white fluorescent light) x F (skin)
[0104]
[Expression 7]
Light bulb evaluation value = F ₂ (Indoor) x F (bulb) x F (skin)
Calculate based on
[0105]
In the above [Equation 3], F (likeness of outdoors) is a value of a membership function representing the likelihood of outdoors with the EV value as a variable as shown in FIG. 15, and the EV value obtained in step S30 of FIG. Can be based on.
[0106]
F (shade-cloudiness) in the equation (3) is a divided area where the EV value Evi of the divided area is 12 or less, and the number of divided areas that fall within the shade-cloudy detection frame as shown in FIG. It is a value of a membership function representing shade-cloudiness as a variable. F (blue sky) is an area in which the EV value Evi of the divided area exceeds 12.5, and a membership function representing the blue sky with the number of divided areas falling within the blue sky detection frame as a variable as shown in FIG. Value. Note that F (blue sky) takes a value that acts in the direction of lowering the evaluation value of shading as the number of areas entering the blue sky detection frame increases.
[0107]
Here, the luminance (EV value Evi) of each of the divided areas is expressed by the following equation:
[0108]
[Equation 8]
Evi = Ev + log ₂ (Gi / 45)
Ev: EV value
Gi: Average integrated value of G in each area
Calculate based on 45 in the above formula is an appropriate value among the values after A / D conversion (provided that the value is represented by an 8-bit digital value (0 to 255)).
[0109]
In addition, F in the formula 4 to the formula 6 ₁ As shown in FIG. 18, the “indoorness” is a value of a membership function representing the indoorness (fluorescent lamp) with the EV value as a variable. ₂ As shown in FIG. 18, (indoorness) is a value of a membership function that indicates the likelihood of indoor (tungsten light bulb) with the EV value (the value in parentheses) as a variable. The EV values shown in FIGS. 15 and 18 use values measured in the center-weighted metering mode in the auto / portrait shooting mode regardless of the shooting mode.
[0110]
Similarly, F (daylight fluorescent lamp-like), F (day white-white fluorescent lamp-like), F (warm white fluorescent lamp-like), and F (bulb-likeness) in [Expression 4] to [Expression 7] are: , Members representing the uniqueness of the light bulb / fluorescent lamp shown in FIG. 19 with the number of areas included in the daylight color detection frame, the day white-white detection frame, the warm white detection frame, and the tungsten light bulb detection frame shown in FIG. The value of the ship function.
[0111]
Further, F (skin) in the equations [6] and [7] is a value of the membership function representing the skin color shown in FIG. 20 with the number of areas falling within the skin color detection frame shown in FIG. 14 as variables. It is. Note that F (skin) acts to lower the evaluation value of the light bulb as the number of areas in the skin color detection frame increases. This is because, in a scene with skin color, if white balance control is strongly applied to the tungsten light bulb color, the reddish color will fly off and the skin color will become worse.
[0112]
As described above, when the shade-cloudy evaluation value, the daylight color evaluation value, the day white-white evaluation value, the warm white evaluation value, and the light bulb evaluation value are calculated (step S34 in FIG. 13). Then, it is determined whether or not the maximum value among these five evaluation values is equal to or greater than a predetermined reference value (0.4 in this example) (step S36 in FIG. 13). If the maximum value is 0.4 or more, white balance control is performed based on the white balance correction value suitable for the light source color of the evaluation value that takes the maximum value (step S38).
[0113]
On the other hand, if the maximum value is less than 0.4, it is determined as daylight (clear), and white balance control based on the white balance correction value suitable for daylight is performed (step S40).
[0114]
Here, the white balance correction value is expressed by the following equation:
[0115]
[Equation 9]
White balance correction value = (Auto setting value-Sunny) x Evaluation value + Sunny
However, the clearness is 1.0. In addition, the auto setting value is prepared in the EEPROM 158 in advance for each light source color. The preset auto setting value (gain value for each RGB color) of the light source type is empirically set with a gain balance that leaves the atmosphere of the light source color.
[0116]
In this example, the auto-set values for shade-cloudy, day-white-white, and tungsten bulb are selected as follows.
[0117]
(1) When shade-cloudy is selected
The number of divided areas that fall within the cloudiness detection frame shown in FIG. 14 is compared with the number of divided areas that fall within the shade detection frame, and the larger auto setting value is adopted. Or, for the two auto setting values, the auto setting value is calculated by increasing the weight in a large number of regions.
[0118]
(2) When lunch white-white is selected
Compared to the number of divided areas that fall within the day white-white detection frame divided into six areas shown in FIG. 14, the auto setting value of the area with the largest number is adopted. Alternatively, the auto setting values are calculated by increasing the weights in the descending order of the number of the six auto setting values.
[0119]
(3) When tungsten bulb is selected
Compared with the number of divided areas that fall within the tungsten detection frame divided into two regions shown in FIG. 14, the auto setting value with the larger number is adopted. Or, for the two auto setting values, the auto setting value is calculated by increasing the weight in a large number of regions.
[0120]
Assuming that the white balance correction values obtained by the equation (9) are Rg, Gg, Bg and the signals to be corrected are R, G, B, the correction results in the white balance adjustment circuit 130 are R ′, G ′, B ′. Then, R ′, G ′, and B ′ are the following expressions:
[0121]
[Expression 10]
R ′ = Rg × R
G ′ = Gg × G
B ′ = Bg × B
Represented by
[0122]
In calculating the white balance correction value, in the above example, the auto setting value set in advance for each detection frame shown in FIG. 14 is used, but the (R / G, B) of the divided blocks that fall within the detection frame are used. The white balance correction value can be calculated based on the average value of the distribution of / G).
[0123]
Also, it is possible to calculate the centroid of the distribution by weighting the number in each detection frame and the average coordinates in each detection frame, and to calculate the white balance correction gain from the coordinates of the centroid and target coordinates. Good.
[0124]
According to the above-described embodiment, the imaging screen, which is a collection of pixels, is divided into a plurality of blocks, the brightness (luminance value) of each block is compared with a certain level threshold value, and the brightness is determined. If there is a darker portion than the threshold value, the RGB integrated value for white balance calculation is calculated using the data of the high-sensitivity pixel 21 for the block corresponding to the dark portion, while the block lighter than the threshold value. If there is a pixel block that is too bright and jumpy, the RGB integrated value for white balance calculation is calculated using the signal from the low-sensitivity pixel 22 for the block corresponding to the bright portion. Accurate integrated value information can be obtained from each block.
[0125]
Since the white balance gain can be determined based on the accurate integrated value information obtained in this way, the calculation accuracy of the white balance gain is improved as compared with the conventional case.
[0126]
In the above-described embodiment, the digital camera that mainly records a still image is exemplified, but the scope of application of the present invention is not limited to this, and an electronic camera such as a digital camera, a video camera, and a DVD camera capable of recording a moving image, or The present invention can also be applied to other imaging devices having an electronic imaging function, such as a mobile phone with camera, a PDA with camera, and a mobile personal computer with camera.
[0127]
【The invention's effect】
As described above, according to the present invention, an image captured using an imaging unit having a high sensitivity pixel and a low sensitivity pixel is divided into a plurality of blocks. While using information on high-sensitivity pixels, white balance calculation is performed using information on low-sensitivity pixels in blocks that are brighter than the threshold, so accurate information can be obtained for each block, and white The accuracy of the balance calculation can be increased.
[Brief description of the drawings]
FIG. 1 is a schematic plan view showing an example of a CCD used to implement the present invention.
FIG. 2 is an enlarged view of the main part of FIG.
FIG. 3 is an enlarged view of a light receiving cell.
4 is a cross-sectional view taken along line 4-4 of FIG.
FIG. 5 is a graph showing photoelectric conversion characteristics of high-sensitivity pixels and low-sensitivity pixels.
FIG. 6 is a block diagram showing a configuration of a digital camera according to an embodiment of the present invention.
FIG. 7 is a diagram used for explaining how to obtain a shooting EV value;
FIG. 8 is a diagram used for explaining the photometry method for each shooting mode.
FIG. 9 is a diagram illustrating an example of image division during auto white balance processing;
FIG. 10 is a block diagram showing the main configuration of the camera of this example.
FIG. 11 is a graph showing the relationship between the level of a final image (synthesized image data) obtained by combining high-sensitivity image data and low-sensitivity image data and relative subject brightness.
FIG. 12 is a flowchart showing the procedure of auto white balance control.
FIG. 13 is a flowchart showing a calculation example of a white balance coefficient (gain).
FIG. 14 is a graph showing a detection frame indicating a color distribution range such as a light source type;
FIG. 15 is a graph showing a membership function representing outdoor appearance
FIG. 16 is a graph showing a membership function representing shade-cloudiness.
FIG. 17 is a graph showing a membership function representing a blue sky.
FIG. 18 is a graph showing a membership function representing indoorness
FIG. 19 is a graph showing a membership function representing the likelihood of a light bulb / fluorescent lamp.
FIG. 20 is a graph showing a membership function representing skin color.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 ... CCD, 20 ... Light receiving cell, 21 ... High sensitivity pixel, 22 ... Low sensitivity pixel, 50 ... Micro lens, 72 ... Color filter layer, 126 ... Digital signal processing circuit, 130 ... White balance adjustment circuit, 138 ... CPU, 148 ... Integration circuit, 152 ... Liquid crystal monitor, 156 ... Memory card

Claims (4)

A structure in which a large number of high-sensitivity pixels having relatively high sensitivity and low-sensitivity pixels having relatively low sensitivity are arranged according to a predetermined arrangement form, and image signals can be respectively extracted from the high-sensitivity pixels and the low-sensitivity pixels. An imaging step of imaging a subject by an imaging means having
A block dividing step of dividing an image acquired via the imaging means into a plurality of blocks;
A brightness calculation step of calculating a value indicating the brightness of the block based on an image signal obtained from the imaging means for each of the divided blocks;
A threshold value setting step for setting a threshold value that serves as an evaluation criterion for the brightness of each block;
A comparison step of comparing the threshold value with a value indicating the brightness of each block calculated in the brightness calculation step;
According to the comparison result of the comparison step, an image signal obtained from the high-sensitivity pixel is used for a block darker than the threshold, and an image signal obtained from the low-sensitivity pixel is used for a block brighter than the threshold. A switching control step of switching the image signal for white balance calculation,
A gain calculating step of calculating a gain for white balance adjustment based on the image signal for white balance calculation selected in the switching control step;
A white balance adjustment step of performing white balance adjustment by correcting the color signal based on the gain obtained in the gain calculation step;
A white balance control method comprising: A large number of high-sensitivity pixels with relatively high sensitivity and low-sensitivity pixels with relatively low sensitivity are arranged in accordance with a predetermined arrangement form, and are photoelectrically converted with the signal photoelectrically converted with the primary photosensitive pixel and the secondary photosensitive pixel. Imaging means having a structure capable of taking out the received signals,
Block division setting means for dividing an image picked up by the image pickup means into a plurality of blocks;
Brightness calculation means for calculating a value indicating the brightness of the block based on an image signal obtained from the imaging means for each of the divided blocks;
A threshold value setting means for setting a threshold value that serves as an evaluation criterion for the brightness of each block;
A value indicating the brightness of each block calculated by the brightness calculation means is compared with the threshold value, and an image signal obtained from the high-sensitivity pixel is used for a block darker than the threshold value, while Switching control means for switching the image signal for white balance calculation so as to use the image signal obtained from the low-sensitivity pixel for the bright block,
Gain calculating means for calculating a gain for white balance adjustment based on the image signal for white balance calculation selected by the switching control means;
White balance adjustment means for performing white balance adjustment by correcting a color signal based on the gain obtained by the gain calculation means;
An imaging apparatus comprising: The imaging means has a structure in which each light receiving cell is divided into a plurality of light receiving regions including at least the high sensitivity pixel and the low sensitivity pixel, and the high sensitivity pixel in the same light receiving cell and The color filter of the same color component is arrange | positioned about the said low-sensitivity pixel, and each micro cell is provided in each light receiving cell with respect to one light receiving cell. Imaging device. The imaging apparatus according to claim 2, wherein the brightness calculation unit calculates an integrated value of R, G, and B color signals obtained from the imaging unit.

Images