JP2003333616A - White balance adjustment method, image pickup device, program, and storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To always obtain accurate white balance regardless of photographing conditions. <P>SOLUTION: A white balance adjustment method is provided with a first color temperature calculation step (S1) for calculating a first color temperature from an image signal within a white detection range out of image signals from an imaging device, a second color temperature calculation step (S2) for calculating a second color temperature from an image signal within a chromatic color detection range out of the image signals from the imaging device, and white balance adjustment steps (S3 to S6) for adjusting white balance on the basis of the first and second color temperatures. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for adjusting the white balance of an image pickup device such as a digital camera or a digital video camera.

[0002]

2. Description of the Related Art A white balance device of a conventional image pickup apparatus will be described.

The image signal output from the image sensor is A /
It becomes a digital signal through the D converter. The digitized image sensor output signal is divided into a plurality of arbitrary blocks as shown in FIG. 3, and the color evaluation value Cx = (RB) / Y Cy = (R + B-2G) / YY = (for each block. R + G + B) / 2 is calculated (example of conversion formula for primary color signal).

Here, when a color evaluation value (Cx, Cy) is included in a preset white detection area, which will be described later, it is assumed that the block is white, and each color pixel of the block assumed to be white. The integrated value of (SumR, SumG, SumB) is calculated. The white balance gain is calculated from the integrated value using the following formula. kWB_R = 1.0 / SumR kWB_G = 1.0 / SumG kWB_B = 1.0 / SumB FIG. 2 is a diagram showing an example of a white detection area. White is photographed at every arbitrary color temperature step under a light source of high color temperature to low color temperature, a color evaluation value is calculated from a signal value obtained from an image sensor, and the value is plotted. As a result, as shown in FIG. 2A, a white judgment line from a high color temperature to a low color temperature is created. In fact, even if it is white, there is some variation in the spectrum, so there is a slight width from the line.

However, the conventional white balance detection device has the following drawbacks.

[0006] For example, when human skin up and the like are photographed under a high color temperature light source such as sunlight, the color evaluation value of a white subject under the sun light source is distributed as (“White” shown in FIG. 2). The color evaluation value of human skin is distributed at almost the same position as white under a low color temperature light source as shown in (Skin ”in FIG. 2). Therefore, when the skin color area is large, there is a problem in that the light source is erroneously determined to be lower than the actual light source.

Conventionally, in order to avoid such misjudgment, the white detection range is narrowed to a range shown by White on the assumption that there is a high possibility that it is external light when the illuminance of the subject is bright, and low white color is detected. It was devised so that the temperature light source was not picked up.

[0008]

However, the conventional white balance device cannot deal with the case where the color temperature is low and the illuminance is very large such as tungsten light in the studio, and the subject such as the shade or the evening is dark. In this case, since it is necessary to detect from a light source having a high color temperature to a light source having a low color temperature, it is necessary to widen the white detection range, which cannot solve the problem of being attracted to the skin color.

Therefore, the present invention has been made in view of the above-mentioned problems, and an object thereof is to always obtain an accurate white balance regardless of photographing conditions.

[0010]

[Means for Solving the Problems]
In order to achieve the object, a white balance adjusting method according to the present invention includes a first color temperature calculating step of calculating a first color temperature from an image signal within a white detection range of image signals from an image sensor. A second color temperature calculating step of calculating a second color temperature from an image signal within a chromatic color detection range of the image signal from the image sensor, the first color temperature and the second color temperature And a white balance adjusting step of adjusting the white balance based on the above.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the accompanying drawings.

(First Embodiment) FIG. 1 is a block diagram showing the schematic arrangement of an image pickup apparatus having a white balance apparatus according to the embodiment of the present invention.

In FIG. 1, the light passing through the lens 18 is received by the CCD 19, and the output signal from the CCD is A
After being converted into a digital signal in the D / D converter 20, white balance is taken in the WB circuit 6 of the embodiment of the present invention, color difference signals U and V are created in the color signal creation circuit 21, and luminance is created in the brightness signal creation circuit A color image can be obtained by creating the signal Y.

2 and 8 are diagrams showing a white detection range and a skin color detection range. In the present embodiment, an image sensor using a primary color filter will be described as an example. In FIG.
The areas indicated by “White” and “Skin” are white,
The flesh colors respectively indicate the areas detected when the image is taken at a high color temperature such as outdoors. In FIG.
The areas indicated by “White” and “Skin” are white,
Each flesh color indicates an area detected when the image is taken at a low color temperature such as tungsten light.

The white detection range is from high color temperature to low color temperature,
Calculate Cx = (RB) / Y Cy = (R + B-2G) / YY = (R + G + B) / 2 (1) from the output value of the image sensor that captured white in any step. It is plotted on a two-dimensional axis (shown by (a) in FIG. 2). Here, the X axis corresponds to the color temperature of the light source, and the Y axis corresponds to the green direction correction amount. In this way, the white detection range (the range indicated by the fine dotted line) is created.

Similarly, for skin color, human skin is photographed for each color temperature and plotted on the axis (shown by (b) in FIG. 2). Here, there are various human skins such as whites, blacks, and yellows, but since the hue value of each human skin does not change so much, the skin detection axis is created using an almost intermediate skin color. The skin detection range (the range indicated by the fine dotted line) is obtained by expanding the detection range in the Y-axis direction with respect to the skin detection axis for performing the green correction.

The operation of detecting the white balance of this embodiment using the above detection axes will be described with reference to the flow chart shown in FIG.

First, one screen shown in FIG. 3 is divided into blocks of arbitrary size. And in each block,
A white evaluation value (Cx, Cy) is calculated based on the color signal from the image sensor, and it is determined whether or not the white detection range and the skin detection range in the figure are entered.

Next, the integrated values SumRw, SumGw, SumBw of the output values of the color filters of the blocks included in the white detection range and the total sample number SampleWNum are calculated. SumRw = ΣR (i) SumGw = ΣG (i) SumBw = ΣB (i) (2) SampleWNum = ΣSample (i) Further, the white evaluation values SumCxW and SumCyW are calculated from the integrated value. SumCxW = (SumRw-SumBw) / Yw SumCyW = (SumRw + SumBw-2SumGw) / Yw (3) Yw = (SumRw + SumGr + SumBg) / 2 Color temperature of the subject (in the image capture screen) from this white evaluation value Predict (first color temperature: CTempFromWhite) (step S
1). Since the X axis of the white detection axis corresponds to the color temperature, the light source color temperature can be calculated if SumCxW is known.

Similarly, the integral value and the total number of samples of the output values of the color filters of the blocks included in the skin detection range are calculated. SumRskin (red), SumGskin (green), SumBskin (blue), SampleSkinNum (total number of samples) (4) Further, the skin evaluation values SumCxSkin and SumCySkin are calculated from the above integrated values. SumCxSkin = (SumRskin-SumBskin) / Yskin SumCySkin = (SumRskin + SumBw-2SumGskin) / Yskin… (5) Yskin = (SumRskin + SumGw + SumBskin) / 2 From this skin evaluation value, the color temperature of the subject (in the imaging screen) (Second color temperature: CTempFromSkin) is predicted (step S
2).

A method of predicting each color temperature from SumCxW, SumCxSkin obtained in step S1 and step S2 will be described with reference to FIG. 6 showing the relationship between SumCx and color temperature.

When the color temperature CtempFromWhite is obtained from SumCxW, the equation L1 in FIG. 6 is used. Formula L1 is expressed as L1: Ctemp = α × SumCx + m (where α is a negative coefficient and m is a value around 4000K) (6) ′.

In addition, the color temperature CtempFromSkin from SumCxSkin
When obtaining, the equation L2 in FIG. 6 is used. Formula L2 is expressed as L2: Ctemp = α × SumCx + n (where n> m, and m is determined by the hue defined as human skin) (7) ′.

The expression L1 is preliminarily obtained by plotting the color temperature when a blank sheet is photographed for each color temperature. Further, in the formula L2 as well, it is assumed that it is obtained in advance by plotting the color temperature when the human skin is photographed for each color temperature.

Then, the color temperature calculated from the white evaluation value and the color temperature calculated from the skin evaluation value are compared (step S
3).

If the correct color temperature can be detected by the white detection (when the color temperature is high and the skin color area is small), CtempFromWhite ≧ CTempFromSkin (6) is satisfied (step S3 Yes). Therefore, the color temperature of the white detection result is determined as the final color temperature of the light source (step S
4). Even when the color temperature of the white detection result is higher than the color temperature of the skin detection result, the color temperature of the white detection result is determined as the final color temperature of the light source.

However, regardless of whether the color temperature is high or low,
When the skin color area is large, the skin temperature is lower than the color temperature predicted from the skin detection range because the skin color is mostly distributed on the low color temperature side of the white detection axis, resulting in CTempFromWhite <CTempFromSkin… (7) (step S3 No). This indicates that the skin color is pulled, and in such a case, the predicted color temperature CTempFromSkin in the skin color detection area is calculated.
Is output (step S5). Further, when the image is taken under a low color temperature such as tungsten light, white is included in the skin detection range even when there is no skin color, and thus the result of Expression (7) is also used.
In this case, the white detection result CTempFromWhite and the skin detection result CtempFromSkin are weighted averaged by the process of step S5 described later, so that the influence is eliminated.

Here, the white detection result CTempFro is digitally obtained.
If mWhite and the skin detection result CTempFromSkin are used for white balance correction, the white balance may change due to a slight scene shift.
Therefore, for the reason of preventing this, it is more preferable to perform the process of weighted averaging two coefficients according to the ratio of the number of samples for which skin is detected (hereinafter, this process is referred to as sample number coefficient MIX). Below is an example of this sample number coefficient
The MIX processing will be described.

The total number of blocks obtained by dividing one screen is AllSampleNumber, and from the total of the sample points detected for skin, the following formula is used to determine what percentage of the screen the skin occupies. SkinSamplePercent = SampleSkinNum × 100 / AllSampleNumber (8) Using this SkinSamplePercent, the final color temperature of the subject is calculated using the following equation. CTempResult = k × CTempFromSkin + (1-k) × CTempFromWhite (9) For example, when SkinSamplePercent is 10% or more, 100% (k = 1) of skin detection result is used and SkinPercent is 10%.
When you turn off, the white detection result starts to be mixed gradually. SkinSample
Percent is 5%, skin detection result is 50% (k = 0.5), white detection result is 50%, SkinSamplePercent is 0%, skin detection result is 0% (k = 0), white detection result is 100%. To be set.

The white balance coefficient is calculated based on the color temperature thus obtained, and the white balance correction processing of the image pickup signal is executed (step 6).

In the present embodiment, CTempFro
CtempFromWhite and C if mWhite <CtempFromSkin
I was doing a weighted average of tempFromSkin, but CtempFromWhi
A weighted average of these color temperatures may be performed when te ≧ CTempFromSkin. In this case, the processing is simplified and the responsiveness is improved.

As described above, appropriate white balance correction can be performed regardless of whether the color temperature of the light source is high or low, and whether or not there is a flesh color in the photographing screen.

(Second Embodiment) In this embodiment, the first embodiment
The method of mixing the color temperature predicted by the white detection and the subject color temperature predicted by the skin detection in the above embodiment is different. FIG. 7 is an operation processing flowchart of the image pickup apparatus according to the second embodiment, and the processing according to the present embodiment will be described with reference to FIG.

First, the color temperature CtempFromWhite from white detection
SumCxW and SumCyW are calculated from (first color temperature), and the simultaneous equations of equation (1) are solved to back-calculate the color pixel output values Rw, Gw, and Bw of the image sensor.

From this, the white balance coefficients WB_Rw, W
B_Gw and WB_Bw are calculated (step S21). WB_Rw = 1.0 / Rw WB_Gw = 1.0 / Gw (10) WB_Bw = 1.0 / Bw Next, from the color temperature CTempFromSkin (second color temperature) from skin detection, CxFromSkin on the skin detection axis , CyFromSkin is calculated. This calculation method is the same as in the first embodiment.
By solving the simultaneous equations as before, R
Find the s, Gs, and Bs values. Further, from this, the white balance coefficients WB_Rs, WB_Gs, and WB_Bs from the skin are calculated (step S22). WB_Rs = 1.0 / Rs WB_Gs = 1.0 / Gs (11) WB_Bs = 1.0 / Bs For the color signals obtained in the pre-designated block of the screen (center part of the screen as shown in Fig. 4) , The above two types
White balance correction is performed based on each WB coefficient, a color difference signal is created from the corrected signal, and an average value of the color difference signals is calculated (step S23). The color difference signal is calculated from the signal corrected based on the white balance coefficient from white, and the sum of the absolute values of the color difference signals is averaged to obtain AveCrw, AveCbw (12), which is used as the white balance coefficient from the skin. A color difference signal is calculated from the signal corrected based on the above, and an average of sums of absolute values of the color difference signals is defined as AveCrs, AveCbs, ... (13).

When the saturation is calculated from this, ChromaW = sqrt (AveCrw1 ² + AveCbw1 ² ) ChromaS = sqrt (AveCrs1 ² + AveCbs1 ² ) ... (14) (step S25).

Here, if the WB coefficient calculated from the white detection is correct (if it is not pulled on the skin), ChromaW = ChromaS (15), but if it is pulled on the skin color of a person when white is detected. It operates so that the flesh color is achromatic (in the direction that the entire screen turns blue). That is, when the human skin is in the center of the screen and the skin color is in the white detection area, ChromaW <ChromaS (16).

When the state of equation (15) is reached, the color temperature CtempFromWhite from white detection is output (step S27). On the other hand, in the case of Expression (16), the white balance coefficient may be calculated using the color temperature CTempFromSkin from the skin detection, but in order to prevent a sudden change of the white balance correction of the screen, the first embodiment is performed. The final color temperature of the subject is predicted using the weighted average similar to the form (step S26). CTempResult = k × CTempFromSkin + (1-k) × CTempFromWhite (17) However, in Expression (17), k, CTempResult, CTempF
romSkin and CTempFromWhite have the same meaning as in the first embodiment. Equation (17) indicates that the more the skin color is detected, the color temperature CTempFromSkin from the skin detection.
Means to attach importance to. For example, if the skin color is detected in all of the screen, the color temperature CTempFromSki from the skin detection is detected.
Since only n is adopted, appropriate white balance correction can be performed even when white is not detected in the screen.

From the color temperature thus obtained, the coefficient for white balance correction is determined (step S28).

In the present embodiment, ChromaW
<CtempFromWhite and CtempFromSkin for ChromaS
However, the weighted average of these color temperatures may be performed when ChromaW ≧ ChromaS. In this case, the processing is simplified and the responsiveness is improved.

In the process of step S25, since the skin color is less detected, the accuracy may be low. In this case, in the process before proceeding to step S25, if the number of samples detected from the skin detection range is smaller than a certain value (for example, SkinSamplePercent is 1% or less), the process proceeds to step S27, and otherwise. Is step S
You may make it progress to the process of 25.

As described above, appropriate white balance correction can be performed also in this embodiment.

(Third Embodiment) In the present embodiment, the first
6 illustrates a mixing method different from the method for mixing the subject color temperature from white detection and the subject color temperature from skin detection (step 5 in FIG. 5) according to the embodiment.

The total number of the plurality of blocks of the image pickup device subjected to white detection and skin detection is AllBlockNum (18).

Number of blocks included in the white detection range SampleWN
Using um and AllBlockNum, the percentage WBlockPercent of blocks determined to be white is calculated. WBlockPercent = SampleWNum / AllBlockNum × 100 (19) Using this ratio, the final subject color temperature (CTempResult) is calculated using the following weighted addition.

(Example) If WBlockPercent> 10%, CTempResult = CTempFromWhite (20) And

When WBlockPercent> 10% is not satisfied, CTempResult = WhiteRatio × CTempWhite + (1-WhiteRati
o) × CtempFromSkin where WhiteRatio is Wh when WBlockPercent = 10%
WhiteRatio when itRatio = 1.0 and WBlockPercent = 0%
= Ratio = a × WBlockPercent + b
The color temperature is obtained by the linear function of (21).

In this embodiment, the skin color is used for the white detection and the chromatic color detection, but the effect can be expected by using not only the skin color but also the green such as the leaf and the blue such as the blue sky.

(Fourth Embodiment) In this embodiment, the first
The method of mixing the subject color temperature from the white detection and the subject color temperature from the skin detection in the embodiment of the present invention is different from the method of the first embodiment (step 5 in FIG. 5).

In this embodiment, when the photographing mode of the camera is designated as the portrait mode (portrait photographing mode), white color balance is obtained using the subject color temperature information from the skin detection.

Alternatively, when the shooting mode of the camera is the portrait mode, a weight is added to increase the ratio of the coefficient k in the equation at the time of mixing.

As described above, the above first to fourth
According to the embodiment of the present invention, in addition to the color information from the white detecting means used in the conventional white balance device, the final subject color temperature is specified by adding the color information obtained by detecting the skin color. Therefore, even if the white area becomes smaller in a shot such as a close-up of the human face, more accurate color temperature determination can be performed.

In addition, it is possible to prevent the skin color from being pulled in a subject having a relatively low illuminance, which has been conventionally erroneously determined to be white and erroneously corrected to be white.

Further, since it is not necessary to narrow the white detection range even in a bright subject as in the conventional case, it is possible to properly perform the white balance correction of the tungsten light of extremely high illuminance such as in a studio.

[0055]

Other Embodiments The purpose of each embodiment is to supply a storage medium (or recording medium) recording a program code of software for realizing the functions of the above-described embodiments to a system or apparatus, It is needless to say that this is also achieved by the computer (or CPU or MPU) of the device reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer is executed based on the instruction of the program code. It goes without saying that a case where some or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing is also included.

Further, after the program code read from the storage medium is written in the memory provided in the function expansion card inserted into the computer or the function expansion unit connected to the computer, based on the instruction of the program code. Needless to say, this also includes a case where a CPU or the like included in the function expansion card or the function expansion unit performs a part or all of the actual processing and the processing realizes the functions of the above-described embodiments.

When the present invention is applied to the above storage medium, the storage medium stores the program code corresponding to the above-described flowchart.

[0058]

As described above, according to the present invention,
It is possible to always achieve accurate white balance regardless of shooting conditions.

[Brief description of drawings]

FIG. 1 is a diagram showing a schematic configuration of an image pickup apparatus including a white balance apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram showing white and skin detection axes.

FIG. 3 is a diagram showing a color detection block.

FIG. 4 is a diagram showing a skin determination block.

FIG. 5 is a flowchart showing an operation of the white balance device according to the first embodiment.

FIG. 6 is a diagram showing an equation for obtaining a color temperature from SumCxW and SumCxSkin.

FIG. 7 is a flowchart showing an operation of the white balance device according to the second embodiment.

FIG. 8 is a diagram showing white and skin detection axes.

[Explanation of symbols]

6 White balance circuit 18 lenses 19 Image sensor 20 A / D converter 21 Color signal generation circuit 22 Luminance signal generation circuit

Claims (17)

[Claims] 1. A first color temperature calculation step of calculating a first color temperature from an image signal within a white detection range of an image signal from the image sensor, and a first color temperature calculation step from among the image signals from the image sensor. A second color temperature calculation step of calculating a second color temperature from the image signal within the color detection range; and a white balance adjustment step of adjusting white balance based on the first color temperature and the second color temperature. A white balance adjusting method comprising: 2. In the white balance adjustment step, the first color temperature and the second color temperature are compared to determine whether the first color temperature and the second color temperature are substantially the same. When the first color temperature is higher than the second color temperature, it is determined that the first color temperature is the color temperature of the light source, and the second color temperature is higher than the first color temperature. Determines that the second color temperature is the color temperature of the light source.
White balance adjustment method described in. 3. In the white balance adjusting step, the first color temperature and the second color temperature are compared to determine whether the first color temperature and the second color temperature are substantially the same. When the first color temperature is higher than the second color temperature, it is determined that the first color temperature is the color temperature of the light source, and the second color temperature is higher than the first color temperature. Determines the color temperature of the light source by performing a weighted average of the first color temperature and the second color temperature based on the ratio of the range of white color and the ratio of the range of chromatic color in the image signal. The white balance adjusting method according to claim 1, wherein 4. The white balance adjusting step calculates a first white balance coefficient from the first color temperature, corrects the image signal with the first white balance coefficient, and outputs the corrected image signal from the corrected image signal. A first saturation value is calculated, a second white balance coefficient is calculated from the second color temperature, the image signal is corrected with the second white balance coefficient, and a second white balance coefficient is calculated from the corrected image signal. The saturation value of 2 is calculated, and the white balance is adjusted by a white balance coefficient corresponding to the larger one of the first saturation value and the second saturation value. White balance adjustment method. 5. In the white balance adjusting step, a first white balance coefficient is calculated from the first color temperature, the image signal is corrected by the first white balance coefficient, and the corrected image signal is obtained. A first saturation value is calculated, a second white balance coefficient is calculated from the second color temperature, the image signal is corrected with the second white balance coefficient, and a second white balance coefficient is calculated from the corrected image signal. The white balance adjusting method according to claim 1, wherein a saturation value of 2 is calculated, and a white balance coefficient is determined by performing a weighted average of the first saturation value and the second saturation value. . 6. The white balance adjusting step calculates a ratio of a white range occupied in the image signal, and sets the first color temperature to a color temperature of a light source when the ratio of the white range is larger than a predetermined value. The color temperature of the light source is determined by performing a weighted average of the first color temperature and the second color temperature when the ratio of the white range is smaller than a predetermined value. White balance adjustment method described in. 7. The weighted average of the first color temperature and the second color temperature is based on a ratio of a white range and a range of a chromatic color in the image signal in the white balance adjusting step. The white balance adjusting method according to claim 1, wherein the color temperature of the light source is determined. 8. The color temperature of the light source is determined by weighted averaging the first color temperature and the second color temperature based on the ratio of the range of white occupying in the image signal. The white balance adjusting method according to claim 1. 9. In the white balance adjusting step, a weighted average of the first color temperature and the second color temperature is calculated based on a ratio of a white range and a range of a chromatic color in the image signal. The white balance adjusting method according to claim 1, wherein the color temperature of the light source is determined. 10. The color temperature of the light source is determined by performing a weighted average of the first color temperature and the second color temperature based on the ratio of the range of white in the image signal. The white balance adjusting method according to claim 1. 11. The white balance adjusting method according to claim 1, wherein the chromatic color is a flesh color. 12. The white balance adjusting step,
11. The white balance adjusting method according to claim 10, wherein the second color temperature is determined to be the color temperature of the light source when the photographing mode of the image pickup apparatus is set to the person photographing mode. 13. The white balance adjusting step,
When the photographing mode of the image pickup apparatus is set to the person photographing mode, the weight of the second color temperature is increased to increase the first color temperature.
The white balance adjusting method according to claim 10, wherein the color temperature of the light source is determined by performing a weighted average with the color temperature of. 14. When calculating the first and second saturation values, the first and second saturation values are calculated based on the image signal of the central portion of the screen among the white balance-corrected image signals.
The white balance adjusting method according to claim 4, wherein the color temperature of the color balance is calculated. 15. A program for causing a computer to execute the white balance adjusting method according to claim 1. Description: 16. A storage medium on which the program according to claim 15 is stored so that it can be read by a computer. 17. An image pickup apparatus, which executes the white balance adjusting method according to claim 1. Description: