JP2012165077A - White balance control apparatus, imaging apparatus using the same, and white balance control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a white balance control apparatus and control method capable of suppressing variation in white balance control value calculated using color information obtained from a subject region.SOLUTION: In the case of dividing an image into plural regions and calculating a white balance control value using the result of integrating the color information obtained from regions overlapping with the subject region, if the size of the subject region is not equal to or greater than a specific threshold, the integration is performed with zero as a weight of the color information. Then the threshold of the size of the subject region is determined to be a size where one or more of the regions are entirely included in the subject region without regard to the position of the subject region.

Description

  The present invention relates to a white balance control device and a control method, and more particularly to a white balance control device and a control method that use a result of subject detection for calculating a white balance control value. The present invention further relates to an imaging apparatus using the white balance control apparatus of the present invention.

  Imaging devices such as digital cameras and digital video cameras have a white balance control function for adjusting the color tone of an image obtained by imaging. White balance control is a process of correcting the pixel value of a captured image with a white balance control value (correction value) so that a white object is imaged white.

  White balance control includes manual white balance control and automatic white balance control. In manual white balance control, a white balance control value is obtained from a pixel value of an image obtained by imaging a white object to be used as a reference, such as white paper. In the auto white balance control, it is common to automatically detect a white part from an image obtained by normal imaging and obtain a white balance control value using the pixel value of the detected part.

  In this general auto white balance control, when the portion that can be determined to be white is small relative to the entire image, or when the portion that is erroneously determined to be white is large, a correct white balance control value cannot be acquired. If a correct white balance control value cannot be obtained, an image subjected to white balance control will be colored.

  In order to solve this problem, a white balance control value calculation method using face detection information has been proposed (Patent Document 1). For example, in Patent Document 1, the white balance control value is corrected as necessary so that the color of the face area to which the white balance control value obtained by general auto white balance control is applied is in an appropriate range as the skin color. Is disclosed.

JP 2009-159596 A

  In Patent Document 1, the white balance control value is corrected on the assumption that the color information obtained from the face area indicates skin color. However, the color information is obtained in units of blocks obtained by dividing the image, and the size and shape of the blocks are determined independently of the size and position of the face area. Therefore, the skin color pixel is not always dominantly included in the block overlapping the face area. In other words, depending on the relationship between the block size and the size and position of the face area, correct skin color information may not be obtained from the block overlapping the face area. For example, if the orientation or position of the subject changes while capturing multiple images continuously in the same scene, the correct skin color is obtained from the block that overlaps the face area even if the image is captured in the same scene Can or can not. In this case, in the method of Patent Document 1, there is a possibility that the white balance control value calculated using the face detection information may fluctuate greatly, and the color tone is not constant even for images captured continuously in the same scene. Cause.

  The present invention has been made in view of such a problem of the prior art, and a white balance control apparatus and control capable of suppressing variations in white balance control values calculated using color information obtained from a subject area. Provide a method.

  That is, the white balance control device according to the present invention includes a subject detection unit that detects a subject region that is a region considered to be a specific subject from an image, a dividing unit that divides the image into a plurality of regions, and a plurality of regions. A color information calculation unit that calculates color information representing a representative color of the region for each of the regions that overlap with the subject region, and a weight determination unit that determines a weight for each of the regions that overlap with the subject region. , An integration means for applying the weight determined for the area overlapping the subject area for which the color information was obtained to each color information, an integration result of the color information, and a predetermined target color And a white balance control value calculation means for calculating a white balance control value so that the representative color becomes the target color from the information, and the weight determination means detects the object detected by the subject detection means. If the size of the region is less than a threshold value Th of one or more sizes where the entire region of the plurality of regions is included in the subject region regardless of the position of the subject region in the image, the weight is determined to be 0. If the threshold Th is equal to or greater than the threshold Th, the weight is determined to be a value greater than zero.

  With such a configuration, according to the present invention, it is possible to provide a white balance control device and a control method capable of suppressing variations in white balance control values calculated using color information obtained from a subject area. .

1 is a block diagram illustrating an example of a functional configuration of an imaging apparatus that uses a white balance control apparatus according to an embodiment of the present invention. The flowchart for demonstrating the process of the to-be-photographed object detection part of FIG. The flowchart for demonstrating the process of the white balance control part of FIG. The flowchart for demonstrating the detail of the 1st auto white balance calculation process performed by S4200 of FIG. The flowchart for demonstrating the detail of the 2nd auto white balance calculation process performed by S4300 of FIG. The figure for demonstrating the example of the weighting weight w [i] determined by S4204 of FIG. The figure for demonstrating the determination method of the addition weight w [i] in the 2nd Embodiment of this invention. The figure for demonstrating the determination method of the addition weight w [i] in the 3rd Embodiment of this invention. The figure for demonstrating the determination method of the addition weight w [i] in the 4th Embodiment of this invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.
FIG. 1 is a block diagram illustrating a functional configuration example of an imaging apparatus 100 using a white balance control apparatus according to an embodiment of the present invention. Note that a configuration related to imaging such as a lens and an imaging element and a configuration related to recording of a captured image that the imaging device has are not essential for the white balance control device of the present invention. Further, the data format of the image to be subjected to white balance control is arbitrary, and may be, for example, RAW data or JPEG data.

  The subject image is formed on the imaging surface of the image sensor 1010 by the lens 1000 which is an imaging optical system. In this embodiment, the image sensor 1010 is, for example, a single-plate color image sensor including a general primary color filter. The primary color filter is composed of three types of color filters each having a transmission main wavelength band in the vicinity of 650 nm, 550 nm, and 450 nm, and photographs color planes corresponding to R (red), G (green), and B (blue), respectively. .

  In a single-plate color image sensor, a color filter in which three color filters are regularly arranged so that each pixel corresponds to one color filter is used, so that each pixel can only obtain the brightness of a single color plane. For this reason, a color mosaic image is output from the image sensor 1010. The A / D conversion unit 1020 converts the color mosaic image output as an analog voltage for each pixel from the image sensor into digital data suitable for subsequent image processing. The subject detection unit 1030 performs a specific subject detection process on the image converted into the digital data by any known method, and the position, size, reliability, etc. for each region (subject region) that seems to be a specific subject. Ask for information. In the present embodiment, a human face is detected as the specific subject.

  The white balance control unit 1040 obtains a white balance control value by a process to be described later, and applies it to the image. Specifically, a gain such that R, G, and B in a region that should be white are the same color is obtained as a white balance control value and applied to the R, G, and B components of the image.

  The color separation unit 1050 generates a color image in which all pixels have R, G, and B information by interpolating the color mosaic image. A basic color image is generated from the generated color image through a matrix conversion unit 1060 and a gamma conversion unit 1070. The color adjustment unit 1080 applies processing for improving the appearance of the image, such as noise reduction, saturation enhancement, hue correction, and edge enhancement. An image output from the color adjustment unit 1080 is referred to as an original image and is stored in the memory 1120. Alternatively, the original image is compressed by the compression unit 1090 using a method such as JPEG, and the recording unit 1100 records it on a recording medium such as a memory card.

A control unit 1110 controls processing of the entire imaging apparatus 100. The control unit 1110 may include, for example, a ROM that stores a control program, a CPU that executes the control program, and a RAM that the CPU uses as a work area when the control program is executed.
The memory 1120 is used for storing intermediate data used for processing in each unit.

The operation unit 1130 includes switches, buttons, and the like for the user to make various instructions and settings to the imaging apparatus 100. Further, the operation unit 1130 may be used as an EVF or may have a display device for displaying various information.
Each unit except the lens 1000 is connected to be communicable with each other through a bus 1140.

The processing (face detection processing) of the subject detection unit 1030 will be described with reference to the flowchart shown in FIG.
In step S3100, the subject detection unit 1030 performs preprocessing such as extracting luminance information suitable for face detection from the image output by the A / D conversion unit 1020. In step S3200, the subject detection unit 1030 extracts a facial feature amount from the extracted luminance information. In step S <b> 3300, the subject detection unit 1030 detects one or more regions (face regions) that appear to be faces from the extracted facial feature values. Then, the subject detection unit 1030 acquires, as detection results, face detection information such as the position and size in the image, the position of face parts (such as eyes and nose), and the reliability of the face for each face area. There is no particular limitation on the face detection method and the configuration of the face detection information, and any known technique can be used.

(Overall operation of white balance control unit)
The overall operation of the white balance control unit 1040 will be described with reference to the flowchart shown in FIG.
In step S4100, the white balance control unit 1040 obtains a first auto white balance control value from color information obtained from the image. In step S4200, the white balance control unit 1040 obtains a second auto white balance control value based on the face detection result of the subject detection unit 1030. In step S4300, the white balance control unit 1040 obtains the second auto white balance control value or the first auto white balance control value if the appropriate second auto white balance control value is not obtained. Auto white balance control value.

  In step S <b> 4400, the white balance control unit 1040 determines a white balance control value to be reflected in the image from a plurality of options according to a user instruction from the operation unit 1130. This option includes a preset white balance control value including an auto white balance control value, a manual white balance control value, and the like. As described above, the white balance control unit 1040 serving as the reflection unit can reflect the white balance control value selected by the user in the image. Further, when the user selects the auto white balance control value, the white balance control value using the face detection information can be reflected. Note that it is not necessary for the user to determine each white balance control value to be reflected in the image in S4400, and for example, a predetermined white balance control value may be reflected.

(First auto white balance control value calculation process)
The first auto white balance control value calculation process performed by the white balance control unit 1040 in S4100 of FIG. 3 will be described using the flowchart shown in FIG.

  First, the white balance control unit 1040 reads an image stored in the memory 1120 from the A / D conversion unit 1020 and divides it into a plurality of blocks (S4101). The number of divisions is m (m is an integer of 2 or more). Although there is no restriction | limiting in the method of dividing | segmenting an image, For example, it can divide | segment into the square or hexagon area | region of the same size.

S4102 to S4106 are performed for each divided block. The white balance control unit 1040 calculates a color average value (R [i], G [i], B [i]) by averaging the pixel values included in each block for each color component (S4102). Further, the white balance control unit 1040 calculates a color evaluation value (Cx [i], Cy [i]) for each block using the following equation (1) (S4103).
Cx [i] = (R [i] − B [i]) / Y [i] × 1024
Cy [i] = (R [i] + B [i]-2G [i]) / Y [i] x 1024 formula (1)
Where Y [i] = R [i] + 2G [i] + B [i], i = 1 ... m

  Next, the white balance control unit 1040 determines whether or not the color evaluation value (Cx [i], Cy [i]) of the i-th block calculated in S4103 is included in a preset white detection range ( S4104). The white detection range is a range in the Cx-Cy coordinate system of a color evaluation value obtained by photographing a white object in advance under light sources having different characteristics (such as color temperature).

When the calculated color evaluation values (Cx [i], Cy [i]) are included in the white detection range (YES in S4104), the white balance control unit 1040 determines that the block is white. Then, the white balance control unit 1040 accumulates the color average values (R [i], G [i], B [i]) of the blocks determined to be white (S4105). On the other hand, when the color evaluation values (Cx [i], Cy [i]) are not included in the white detection range, the white balance control unit 1040 advances the process to S4106 without integrating the color average values of the block. The processing (white extraction processing) in S4104 and S4105 can be expressed by the following equation (2).
SumR + = w [i] * R [i]
SumG + = w [i] * G [i]
SumB + = w [i] * B [i] Equation (2)
However, i = 1 ... m

In Expression (2), when the color evaluation value (Cx [i], Cy [i]) is included in the white detection range, the addition weight w [i] is 1, and when it is not included, the addition weight w [i ] Is set to 0. As a result, only the color average values (R [i], G [i], B [i]) of blocks whose color evaluation values (Cx [i], Cy [i]) are included in the white detection range are integrated. . In step S4106, the white balance control unit 1040 determines whether the processing in steps S4102 to S4105 has been performed for all blocks. If there is an unprocessed block, the processing returns to step S4102, and if processing has been performed for all blocks, the processing is performed in step S4107. Proceed to In S4107, the white balance control unit 1040 calculates the first auto white balance control values (WBCol_R1, WBCol_G1) using the following equation (3) from the obtained integrated values (SumR, SumG, SumB) of the color average values. , WBCol_B1).
WBCol_R1 = SumY × 1024 / SumR
WBCol_G1 = SumY × 1024 / SumG
WBCol_B1 = SumY × 1024 / SumB formula (3)
However, SumY = (SumR + 2 x SumG + SumB) / 4

(Second auto white balance control value calculation process)
The second auto white balance control value calculation process performed by the white balance control unit 1040 in S4200 of FIG. 3 will be described using the flowchart shown in FIG.

  First, the white balance control unit 1040 as a dividing unit reads an image stored in the memory 1120 from the A / D conversion unit 1020 and divides the image into a plurality of blocks (S4201). The number of divisions is n (n is an integer of 2 or more).

  From the viewpoint of reducing the processing load on the white balance control unit 1040, this division processing is desirably performed in the same manner as the division in S4101 (the same block shape and size, that is, n = m). However, when there is a sufficient margin in processing capacity and memory, at least one of the number of divisions and the division shape in S4101 and S4201 may be different according to conditions such as imaging conditions and face detection results.

  S4202 to S4206 are performed for each divided block. The white balance control unit 1040 as the color information calculation unit determines whether at least a part of the block overlaps the face area detected by the subject detection unit 1030 (S4202), and sets the representative color of the block for the overlapping block. The color information to be expressed is calculated. In the present embodiment, the white balance control unit 1040 calculates a color average value (R [i], G [i], B [i]) obtained by averaging pixel values included in a block for each color component as color information. (S4203). However, in order to accurately calculate the color average value for the skin color portion in the face area, it is desirable that the entire block is included in the face area.

  The white balance control unit 1040 serving as a weight determination unit calculates an addition weight w [i] for the block being processed based on information obtained from the pixels in the block and information obtained from the subject detection unit 1030 (face detection result). Calculate (S4204). Then, the white balance control unit 1040 as the integrating means uses the above-described equation (2), and adds the block weights w to the block color average values (R [i], G [i], B [i]). [i] is applied and integrated (weighted addition) (S4205).

  On the other hand, if the block being processed does not overlap the face region at all (S4202, No), the white balance control unit 1040 advances the process to S4206 without integrating the block color average values. This corresponds to setting the block addition weight w [i] to 0 for blocks that do not overlap the face area.

In step S4206, the white balance control unit 1040 determines whether the processing in steps S4202 to S4205 has been performed for all blocks divided in step S4201. The white balance control unit 1040 returns the process to S4202 if there is an unprocessed block, and advances the process to S4207 if the process has been performed for all blocks. In step S4207, the white balance control unit 1040 as the white balance control value calculation unit calculates the second auto white balance control values (WBCol_R2, WBCol_G2, and WBCol_B2) using the following equation (4).
WBCol_R2 = SumY × SkinR × 1024 / SumR
WBCol_G2 = SumY × SkinG × 1024 / SumG
WBCol_B2 = SumY × SkinB × 1024 / SumB formula (4)
However, SumY = (SumR + 2 x SumG + SumB) / 4
SumR, SumG, and SumB are weighted addition values (integration results) of the obtained color average values, and SkinR, SkinG, and SkinB are target skin color color balance control values. The second auto white balance control value is a white balance control value such that the representative color of the block overlapping the face area becomes the target color of the skin color.

  In this embodiment, the white balance control unit 1040 normally sets the second auto white balance control value as the final auto white balance control value in S4300 of FIG. However, in some cases, such as when a face area cannot be detected by the subject detection unit 1030, the weighted addition value of the color average value obtained in S4205 may not be an appropriate value as a value for obtaining the auto white balance control value. In such a case, the white balance control unit 1040 sets the first auto white balance control value as the final auto white balance control value in S4300.

(Weighted weight calculation process)
Next, an example of processing of the weighted weight w [i] in S4204 of FIG. 5 will be described using FIG. Here, a method of calculating the addition weight based on the size information of the face area among the detection results obtained from the subject detection unit 1030 will be described.

  Even when the size of the block is smaller than the face area, depending on the position of the face, there may be no block that is entirely included in the face area. On the other hand, the block size has a lower limit from the viewpoint of speeding up the processing and reliability of the color evaluation value. For this reason, as shown in FIG. 6A, there is a high probability that there is no block including the entire face area, particularly when the face area is small.

  Even if the face area can be detected, if the entire face area does not contain a block, the second auto white balance control value calculated based on the color information of the block partially overlapping the face area is stable. do not do. Therefore, the color tone between images obtained by continuously capturing the same scene may not be constant.

  Therefore, the present embodiment is characterized in that the addition weight of each block is determined in accordance with the relationship between the number of divisions and the shape of the image in S4201 and the size of the face area. More specifically, the addition weight is determined depending on whether or not the size of the face area has such a size that one or more blocks that are entirely included in the face area exist, regardless of the position of the face area. Regardless of the position of the face area, the size of the face area that satisfies the condition that there is at least one block that is entirely included in the face area is determined according to the number of image divisions, the shape of the image, and the detected shape of the face area. can do.

  For example, if the block division shape and the face area detection shape are both square, and the size of the face area is equal to or larger than 2 x 2 blocks in the vertical and horizontal directions, the entire face area is located at any position in the image. The condition that there is at least one block including is satisfied.

  In the present embodiment, as shown in FIG. 6B, the size of the face area obtained from the subject detection unit 1030 is such that there is one or more blocks including the entire face area regardless of the position of the face area. If it is less than Th, the addition weight w [i] is also set to 0 for blocks overlapping the face area. Accordingly, SumR, SumG, and SumB are all 0, and the second auto white balance control value is not calculated. In S4300, the first auto white balance control value becomes the final auto white balance control value.

  On the other hand, when the size of the face area is equal to or larger than Th, the addition weight w [i] of the block having an overlap with the face area is set to be larger than zero. Note that when the size of the face area is equal to or greater than Th, the addition weight w [i] may be uniformly set to 1. However, in that case, if the size of the face region fluctuates in the vicinity of Th, the final auto white balance control value frequently switches between the second auto white balance control value and the first auto white balance control value. Can happen. This becomes a factor that the color tone varies between shots when auto white balance control is applied. Therefore, in this embodiment, the value of the addition weight w [i] is determined so as to approach the maximum value Wmax (here, 1) stepwise or gradually as the size of the face region increases from Th.

  If the rate of change until the addition weight w [i] reaches the maximum value after the face area size becomes Th (the slope of the increased portion of the addition weight in FIG. 6B) is too small, the face detection result is obtained. If it cannot be used sufficiently and is too large, the above-mentioned problem of fluctuation becomes large. Therefore, an appropriate value is obtained and set by, for example, experiments. In addition, as shown in FIG. 6B, it may be increased linearly, or the rate of increase in the vicinity of Th may be low, and thereafter it may be increased in a quadratic curve that increases the rate of increase.

(Modification)
FIG. 6A illustrates a case where there is no tilt or rotation of the face area for easy explanation and understanding. However, as shown in FIG. 6C, when the face region is tilted or rotated, the value of Th changes (when performing block division as shown in FIG. 6A, compared to when there is no tilt). Th increases). Therefore, the second auto white balance control value can be further stabilized by obtaining the face area size threshold Th in consideration of the tilt or rotation amount of the face area.

  As described above, in this embodiment, when it is not certain that there is a block that is entirely included in the face area, the addition weight is set to 0 even for blocks that overlap the face area. Therefore, when it is not certain that the entire face area includes a block, the first auto white balance based on the information of the entire image is the final auto white balance control value.

  Then, if it is certain that the entire face area includes a block, the second auto white balance control value based on the color average value of the blocks overlapping the face area is calculated, and the final auto white Use the balance control value. As a result, the variation in the auto white balance control value calculated based on the color information of the block overlapping the face area is reduced, and it is possible to suppress a change in color tone between images obtained by continuously capturing the same scene. Become.

  Furthermore, the addition weight w [i] is gradually or gradually brought closer to the maximum value as the face area increases from the face area size threshold Th that ensures that the entire face area includes a block that is entirely included. The problem when the size of the face region changes near the threshold Th can be alleviated.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. This embodiment is the same as the first embodiment except for the method of determining the addition weight w [i] of the color evaluation value of the block overlapping the face area in the second auto white balance control value calculation process. Therefore, only the method for determining the addition weight w [i] will be described.

  In the first embodiment, when the size of the face area is equal to or greater than the threshold Th, the addition weight for the color evaluation value (color average value) of the block having an overlap with the face area is set to a uniform value. . However, when it is attempted to accurately extract only the skin color from the face area, it is desirable that there are many blocks that are entirely included in the face area. However, when the face area is small as shown in FIG. 7A, the total number of blocks overlapping the face area (9) is smaller than the total number of blocks (1) included in the face area. Become. In this case, when the color average value is weighted and added to the nine blocks with a uniform addition weight, the ratio of the best color average value obtained from the block including the entire face area to the weighted addition value is small. Become.

  For this reason, in this embodiment, the value of the addition weight w [i] for the block that overlaps the face area is decreased as the ratio of the overlap with the face area decreases, so that the block that is considered not to contain much skin color pixels. Reduce the weight of the average color value. For example, if the block overlaps 90% with the face area, it can be reduced to 90% of the addition weight of the entire block included in the face area, or addition of blocks with less than 10% overlap with the face area Or increase to 9 times the weight. FIG. 7B shows an example of the relative size of the addition weight when the addition weight of the block whose whole face area is 10 is 10 in the situation of FIG. 7A.

  For example, the addition weight as described above is determined by multiplying the reference addition weight obtained in accordance with the relationship between the size of the face area and the addition weight as shown in FIG. 6B in the first embodiment by a coefficient. can do.

  According to the present embodiment, even when the number of blocks that are entirely included in the face area is small, a weighted addition result that sufficiently reflects the color average value of the blocks that are entirely included in the face area is obtained. It is done. Therefore, in addition to the effects of the first embodiment, it is possible to further improve the accuracy of the second auto white balance control value.

(Third embodiment)
Next, a third embodiment of the present invention will be described. Similar to the second embodiment, this embodiment is also the same as the first embodiment except for the method of determining the addition weight w [i] of the color evaluation value of the block overlapping the face area. Therefore, only the method for determining the addition weight w [i] will be described.

  The present embodiment is characterized in that an addition weight w [i] for a block overlapping with a face area is corrected based on information on face parts (such as eyes and nose) obtained as a result of face detection processing in the subject detection unit 1030. And

  When a purer skin color can be obtained, a second auto white balance control value with higher accuracy can be obtained. Therefore, in the present embodiment, the addition weight for the color average value of a block at a position where there is a high possibility that many colors desirable to be acquired as the representative color of the block are included in the color average value of the block corresponding to the other part. Make it larger than the addition weight. If the specific subject is a human face, for example, the addition weight for the color average value of the block corresponding to the flatter part of the face, such as the cheek or forehead, and the addition weight for the color average value of the block corresponding to the other part Larger than.

  The position of the cheek or forehead may be given as a result of the face detection process, or may be estimated from the position of the eyes or nose normally detected in the face detection process. Then, among the blocks whose whole face area is included, the addition weight w [i] of the block overlapping with the cheek or forehead is made larger than the addition weight w [i] of the block overlapping with the other part. For example, the addition weight w [i] of the block overlapping the cheek or forehead is twice the addition weight w [i] of the block overlapping the other face detection area (see FIGS. 8A and 8B). ).

  For example, the addition weight as described above is determined by multiplying the reference addition weight obtained in accordance with the relationship between the size of the face area and the addition weight as shown in FIG. 6B in the first embodiment by a coefficient. can do.

  Also according to the present embodiment, since the ratio of the color average value obtained from the purer skin color in the integrated value of the color average value is increased, the second auto white balance control value is the same as in the second embodiment. It is possible to further improve the accuracy.

(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described. Similar to the second and third embodiments, this embodiment is also the same as the first embodiment except for the method of determining the addition weight w [i] of the color evaluation value of the block overlapping the face area. Therefore, only the method for determining the addition weight w [i] will be described. This embodiment can be combined with the second or third embodiment.

  The present embodiment relates to a method of determining the addition weight w [i] of the color evaluation value of a block overlapping with the face area when a plurality of face areas are detected by the subject detection unit 1030. For example, a person other than the intended subject (main subject) may be imaged and a plurality of face regions may be detected, as in the case of imaging in a park or city. When a plurality of face areas are detected, it is considered that the second auto white balance control value with higher accuracy can be obtained by utilizing information obtained from each face area. However, considering that it is generally desired that the white balance of the main subject is appropriate, it is desirable to increase the proportion of the color average value of the main subject in the integrated value of the color average values from a plurality of face regions. .

  From this point of view, in the present embodiment, among the blocks overlapping with the face area, the addition weight w [i] for the block overlapping with the face area of the main subject is the block overlapping with the face area of the person other than the main subject. Is larger than the addition weight w [i]. For example, the addition weight w [i] for the block overlapping the face area of the main subject is set to three times the addition weight w [i] for the block overlapping the face area of a person other than the main subject (FIG. 9A). And (b)). By using this method, the ratio of the color average value obtained from the main subject to the integrated value of the color average value increases.

  For example, the addition weight as described above is determined by multiplying the reference addition weight obtained in accordance with the relationship between the size of the face area and the addition weight as shown in FIG. 6B in the first embodiment by a coefficient. can do.

  It should be noted that which face region is selected as the main subject among the plurality of face regions detected by the subject detection unit 1030 can be determined by an arbitrary method. For example, the main subject may be determined under conditions such as the largest face area, the face area closest to the center of the image, and the face area with the highest face detection reliability. Alternatively, the main subject may be determined using information such as the degree of focus (defocus amount), the degree of smile, and the position of the line of sight. A combination of these conditions may be used. Alternatively, the user may select the face as the main subject.

  In the present embodiment, the second auto white balance control value is not calculated unless at least the size of the face area as the main subject is greater than or equal to the threshold Th described in the first embodiment ( (Because the addition weight is 0 in all blocks). However, as long as the size of the face area as the main subject is equal to or greater than Th, the size of the other face areas may be less than the threshold Th. This is because the ratio of the color average value obtained from the face area of the person who is not the main subject to the integrated value of the color average value is small due to the determination of the addition weight described above.

  When this embodiment is combined with the second or third embodiment, the correction described in the second or third embodiment is applied to the addition weight of the main subject obtained as shown in FIG. 9B. Good.

According to the present embodiment, when a plurality of face areas are detected, it is expected that the accuracy of the second auto white balance control value is improved by effectively using information obtained from each face area. Furthermore, it is possible to obtain a second auto white balance control value that makes the color tone of the main subject appropriate.
Further, by combining with the second or third embodiment, the effects of these embodiments can also be realized.

(Other embodiments)
In the above-described embodiment, both the first white balance control value and the second auto white balance control value are calculated, and only when the second auto white balance control value cannot be calculated or the reliability is low. A configuration using one white balance control value has been described. However, an object of the present invention is to suppress variations in the second auto white balance control value calculated using color information obtained from the face area, particularly variations in the second auto white balance control value obtained in the same scene. It is. Therefore, the calculation of the first white balance control value is not essential in the present invention.

  Furthermore, in the above-described embodiment, a case has been described in which color information obtained from a human face area is used for calculating the second auto white balance control value. However, the present invention can be applied to the calculation of the second auto white balance control value using color information obtained from an area of an arbitrary specific subject whose target color is known in advance.

The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.
As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

Claims (10)

Subject detection means for detecting a subject region that is a region considered to be a specific subject from an image;
Dividing means for dividing the image into a plurality of regions;
Color information calculating means for calculating color information representing a representative color of each of the plurality of areas overlapping with the subject area;
A weight determining means for determining a weight for each of the regions overlapping with the subject region;
Integrating means for applying the weights determined for areas overlapping with the subject area for which the color information is obtained to each of the color information,
White balance control value calculation means for calculating a white balance control value such that the representative color becomes the target color from the integration result of the color information and information on a predetermined target color;
The weight determination means has one subject area detected by the subject detection means, and one of the plurality of areas is entirely included in the subject area regardless of the position of the subject area in the image. The white balance control apparatus, wherein the weight is determined to be 0 if the threshold value Th is less than the threshold value Th, and the weight is determined to be greater than 0 if the threshold value Th is greater than or equal to the threshold value Th.   2. The weight determination unit according to claim 1, wherein the weight determination unit determines the weight so that the weight approaches a maximum value stepwise or gradually as the subject area increases from a size equal to the threshold Th. White balance control device.   3. The white balance control device according to claim 1, wherein the color information calculation unit calculates, as the color information, a color average value obtained by averaging pixel values included in the region for each color component. 4. .   The weight determination unit determines the weight so that the weight becomes smaller in a region where the ratio of overlapping with the subject region is smaller when the weight is not determined to be 0. Item 4. The white balance control device according to any one of Items 3 to 3.   If the weight determination means does not determine the weight to 0, the weight for an area that is likely to include many colors that are desirably acquired as the representative color among the areas that overlap the subject area, The white balance control apparatus according to any one of claims 1 to 3, wherein the weight is determined so as to be larger than a weight for another region.   When a plurality of the subject areas are detected and the weight is not determined to be 0, the weight determining means selects a subject selected as a main subject among the plurality of subject areas from among the areas overlapping with the subject area 6. The white according to claim 1, wherein the weight is determined so that a weight for a region overlapping with the region is larger than a weight for a region overlapping with another subject region. Balance control device.   When the weight determining unit determines the weight to be 0, the white balance control value calculating unit calculates a white balance control value from color information of a region determined to be white among the plurality of regions. The white balance control device according to claim 1, wherein the white balance control device is a white balance control device. Imaging means for capturing the image;
A white balance control device according to any one of claims 1 to 7,
Reflecting means for reflecting the white balance control value in the image. A subject detection step in which subject detection means detects a subject region that is a region considered to be a specific subject from the image;
A dividing step of dividing the image into a plurality of regions;
A color information calculating step in which color information calculating means calculates color information representing a representative color of each of the plurality of areas overlapping with the subject area;
A weight determining step for determining a weight for each of the regions overlapping with the subject region;
An integrating step in which the integrating means applies the weights determined for the areas overlapping the subject area for which the color information was obtained to each of the color information,
White balance control value calculation means for calculating a white balance control value for calculating the white balance control value such that the representative color becomes the target color from the integration result of the color information and information on a predetermined target color And having a process
In the weight determination step, the size of the subject region detected by the subject detection unit is one region in which the entirety of the plurality of regions is included in the subject region regardless of the position of the subject region in the image. The white balance control method, wherein the weight is determined to be 0 if it is less than the threshold value Th of the existing size, and the weight is determined to be a value greater than 0 if it is equal to or greater than the threshold Th.   The program for functioning a computer as each means of the white balance control apparatus of any one of Claim 1 thru | or 7.

Images