JP2007312099A - Image-taking device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform a precise white balance process without erroneously taking a flesh color for a white color when having detected a face. <P>SOLUTION: A face contained in image data is detected using a face detection means, and a defect in white balance is calculated from a color evaluated value of a face region, thus calculating a desirable white balance correction coefficient by means of usual white balance calculation and weighting addition. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing method in an imaging apparatus such as a digital camera or a digital video camera, and more particularly to white balance control using face detection.

The signal output from the image sensor is digitized by AD conversion and divided into a plurality of blocks as shown in FIG. The block signal is composed of color signals including R, G, and B, and there is equation (1) as an example of calculating the color evaluation value of the block.
Cx [i] = (R [i] -B [i]) / Y [i] x 1024
Cy [i] = (R [i] + B [i]-G [i]) / Y [i] x 1024 (1)
(However, Y [i] = R [i] + 2G [i] + B [i])
When the color evaluation value (Cx [i], Cy [i]) is included in the preset white detection range, it is determined that the block is white, and the integrated value SumR of the color pixels that have entered the white detection range , SumG and SumB are calculated, and the white balance coefficient is calculated as shown in the following equation (2).
WBCo ₋ R = SumY × 1024 / sumR
WBCo _- G = SumY x 1024 / sumG (2)
WBCo ₋ B = SumY × 1024 / sumB
However, SumY = (sumR + 2 x sumG + SumB) / 4
However, the conventional white balance coefficient calculation has the following drawbacks. Under a light source such as sunlight, the white color evaluation value is distributed in the vicinity of the region A in FIG. 3, and the skin color distribution is distributed in the vicinity of the region B. The skin color evaluation values under a solar light source are distributed approximately the same as the white dot color evaluation values under a low color light source. Therefore, the color evaluation value of the screen is distributed in the region B in FIG. That is, there is a problem that the skin color is mistakenly identified as white under a low color temperature, and the human skin is made white. As a conventional countermeasure against this problem, when the subject illuminance is bright, it is determined that the light is outside, and the white detection range is narrowed so that the skin color is not erroneously determined to be white. However, the skin color distribution of low-illumination high color temperature fluorescent lamps and medium color temperature fluorescent lamps in the indoor light source is in the downward direction of the black body radiation axis, and there is a case where the skin color is misclassified as white.

Therefore, in Patent Document 1, a proposal has been made to exclude the face area detected by the face detection circuit from the white detection target.
JP 2003-189325 A

  However, in the above-described conventional white balance correction value calculation method, no countermeasure is taken into account when the human face is erroneously recognized or cannot be detected by the face detection circuit. For this reason, when an area that is not a human face is erroneously detected as a human face in the face detection circuit, the white detection target area is reduced, so that the accuracy of the output color temperature information is reduced. In addition, when a human face is not recognized even though it is a human face, white detection is performed in the face area, resulting in a disadvantage that white balance correction accuracy is lowered as a result. In addition, when a human face is photographed up, if the face area is excluded from the white detection target, there is almost no subject area where white detection is performed, resulting in low white balance correction accuracy.

An imaging device for imaging a subject and processing an obtained image signal,
First white balance correction means for determining whether each pixel of the image sensor is white and performing white balance correction; and face detection means for detecting a face area from the image signal;
Having a determination means for determining whether to estimate the light source from the photographing conditions and use the result of the face detection means;
The image processing apparatus includes a second white balance correction unit that corrects a defect in white balance due to a skin region based on information output by the first white balance correction unit and the face detection unit.

  According to the present invention, more stable white balance correction can be performed according to the face detection result.

  Next, details of the present invention will be described in accordance with the description of the embodiments.

  FIG. 1 is a schematic diagram of this embodiment. Embodiments will be described below with reference to the drawings.

  In FIG. 1, reference numeral 101 denotes a solid-state imaging device made of CCD, CMOS, or the like, and the surface thereof is covered with an RGB color filter such as a Bayer array so that color photography can be performed. Reference numeral 114 denotes a face detection unit that detects a face using an image signal stored in the memory 102. There are many face detection methods, but generally a method using learning typified by a neural network, a method using pattern recognition, extracting a region with physical shape features from an image area, skin color There are a number of methods such as a method that detects image feature quantities such as the shape of an eye and uses statistical analysis.

  Currently proposed products include a method of detecting faces using wavelet transform and image feature amounts, a method of combining template matching, and the like.

  Here, template matching, which is one of pattern recognition, will be described. Pattern recognition is a process of matching an observed pattern with one of predetermined concepts (classes). FIG. 4 is a flowchart of pattern recognition processing executed by the face information detection circuit in FIG. In this description, FIG. 5 is referred to in parallel.

  In FIG. 4, the image data 603 acquired from the memory 102 is preprocessed (step S501), the characteristic pattern is extracted from the preprocessed face image data (step S502), and the extracted pattern is used as a template ( The recognition pattern is acquired by making it correspond to (standard pattern) 601 (template matching) (step S503), the acquired recognition pattern is output (step S504), and the pattern recognition processing is terminated.

  The template matching is performed as follows.

  In FIG. 5, first, the center point 602 of the template 601 is placed at a certain coordinate point (i, j) of the acquired image data 603, and the overlapping portion between the template 601 and the image data 603 is scanned while scanning the center point 602. The degree of similarity is calculated and the position where the degree of similarity is maximized is determined. By matching a pattern extracted from the face image data 603 with a template 601 including shapes such as eyes and ears, it is possible to acquire eye position information and face regions (face coordinates).

  Based on the image signal sent from the face detection unit 114, the CPU 115 calculates a shutter speed and an aperture value so that the face has an optimum brightness if a face is detected, and focuses on the face. The driving amount of the focus lens is calculated as follows. On the other hand, if no face is detected, the CPU 115 calculates the shutter speed and aperture value so that the entire image becomes bright, and calculates the driving amount of the focus lens to focus on the subject in the in-focus area. To do. The exposure value (shutter speed and aperture value) calculated by the CPU 115 and the driving amount of the focus lens are sent to the control circuit 113 and controlled based on each value. A white balance control unit 103 calculates a WB correction value based on the image signal stored in the memory 102 and the face information obtained from the face detection unit 114, and stores it in the memory 102 using the calculated WB correction value. WB correction is performed on the processed image signal. The detailed configuration of the white balance control unit 103 and the method for calculating the WB correction value will be described later.

  104 is a color matrix circuit that converts the color difference signals RY and BY by applying color gain so that the image signal corrected by WB by the WB control unit 103 is reproduced in an optimal color, and 105 is a band of the color difference signals RY and BY. A low pass filter (LPF) circuit 106 for limiting, and a CSUP (Chroma Supress) circuit 106 for suppressing a false color signal in a saturated portion of the image signal band-limited by the LPF circuit 105. On the other hand, the WB corrected image signal by the WB control unit 103 is also output to the luminance signal (Y) generation circuit 111 to generate the luminance signal Y, and the edge enhancement circuit 112 performs edge processing on the generated luminance signal Y. Emphasis processing is performed.

  The color difference signals RY and BY output from the CSUP circuit 106 and the luminance signal Y output from the edge enhancement circuit 112 are converted into RGB signals by the RGB conversion circuit 107, and gradation correction is performed by the gamma correction circuit 108. Is done. Thereafter, it is converted into a YUV signal by the color luminance conversion circuit 109, further compressed by the compression circuit 110, and recorded as an image signal on an external recording medium or an internal recording medium.

First, the calculation method of the first white balance correction means will be described with reference to FIG. First, the image signal stored in the memory 102 is read, and the screen is divided into arbitrary m blocks as shown in FIG. 2 (step S101). Then, for each block (1 to m), the pixel value is added and averaged to each color to calculate a color average value (R [i], G [i], B [i],). The color evaluation values (Cx [i], Cy [i]) are calculated by using them (step S102).
Cx [i] = (R [i]-B [i]) / Y [i] x 1024
Cy [i] = (R [i] + B [i]-2G [i]) / Y [i] x 1024
However, Y [i] = R [i] + 2G [i] + B [i]
Next, it is determined whether or not the color evaluation value (Cx [i], Cy [i]) of the i-th block calculated in step S102 is included in the preset white detection range 301 shown in FIG. S103).

  The white detection range 301 is obtained by photographing white under different light sources in advance and plotting the calculated color evaluation values. In FIG. 3, the negative direction of the x coordinate (Cx) is a color evaluation value when white of a high color temperature subject is photographed, and the positive direction is a color evaluation value when white of a low color temperature subject is photographed. Further, the y coordinate (Cy) means the degree of the green component of the light source, and the G component increases as it goes in the negative direction, that is, it indicates a fluorescent lamp. If the calculated color evaluation values (Cx [i], Cy [i]) are included in this white detection range 301 (YES in step S103), it is determined that the block is white, and the color average of the block The values (R [i], G [i], B [i]) are accumulated (step S104), and if not included, the process proceeds to step S105 without adding. The processing of step S103 and step S104 can be expressed by equation (3).

ここで、式（3）において、色評価値（Cx[i]、Cy[i]）が白検出範囲（図3の301）に含まれる場合はSw[i]を1に、含まれない場合にはSw[i]を０とすることにより、ステップS103の判断により色評価値（R[i]、G[i]、B[i]）加算を行うか、行わないかの処理を実質的に行っている。

Here, in Equation (3), when the color evaluation value (Cx [i], Cy [i]) is included in the white detection range (301 in FIG. 3), Sw [i] is not included in 1. In this case, by setting Sw [i] to 0, the process of adding or not adding color evaluation values (R [i], G [i], B [i]) according to the determination in step S103 is substantially performed. Is going to.

  In step S105, it is determined whether or not the above process has been performed for all blocks. If there is an unprocessed block, the process returns to step S102 and the above process is repeated. If all the blocks have been processed, the process proceeds to step S106. .

In step 106, the first WB correction values (WBCol_R, WBCol_G, WBCol_B) are calculated from the integrated values (sumR, SumG, SumB) of the obtained color evaluation values using the following equation (4).
WBCol_R = sumY x 1024 / sumR
WBCol_G = sumY x 1024 / sumG (4)
WBCol_B = sumY × 1024 / sumB
However, sumY = (sumR + 2 x sumG + sumB) / 4
Next, a method for calculating the second WB correction value will be described with reference to FIG. In the second WB correction value calculation method in this embodiment, the first WB correction value and the color evaluation value of the detected face area are weighted and added according to the conditions described later, thereby suppressing WB defects caused by the face area. is there.

  First, the brightness is determined in S111. If the brightness is equal to or greater than the threshold, it is determined that the light source is an outdoor light source, and the WB calculation is performed without using the detection result obtained by the face detection unit. If it is determined that the light source is an outdoor light source, the face region is not recognized as white, and therefore the final WB correction coefficient uses the above-described first WB correction value as the final WB correction coefficient (S119). If the brightness is less than or equal to the threshold value, it is determined whether or not a face is detected by the face detection unit (S112). If no face is detected, the first WB correction value is used (S119). If it is determined that there is a face, the color average value of the human face area is calculated by averaging the pixel values in the area for each color of RGB in S113. If the color average value (Cx, Cy) of the skin area calculated in S113 is not within the white detection range, the first WB correction value is used. If it is determined in S114 that the skin area is a white detection range, a defect is estimated from the size of the face area in S115, the first correction value obtained in S116, and the color evaluation value of the skin area, and the first WB A correction value from the correction value is determined (S117). The correction amount is determined by weighting based on the detected face size (α) and weighting based on distance information between the first WB correction value and the skin area color evaluation value (β) (see FIG. 8).

That is, the correction amount (ΔHosei) that can eliminate the problem of white balance due to the face area is obtained as follows (see FIG. 9).
ΔHosei = α × β × K
(K is a correction amount constant)
By realizing the above, it is possible to eliminate a white balance problem when a face is present in the screen. In addition, even when a human face is misrecognized in the face detection circuit, it can be determined from the first WB correction result and the color evaluation value of the face detection area whether the white balance correction result is causing a problem. A white balance correction coefficient can be obtained.

The block diagram of the imaging device provided with the face detection function in the embodiment of the present invention. The figure which shows the example which divided | segmented the screen into several blocks. The figure which shows a white detection range. The flowchart of the pattern recognition process performed by the face detection part in FIG. The example of the template matching in the pattern recognition process of FIG. The flowchart which shows a 1st WB correction | amendment calculation process. The flowchart which shows a 2nd WB correction | amendment calculation process. The figure which showed the relationship between the size used in the 2nd WB correction | amendment calculation, distance, and a weight. The conceptual diagram which adds correction amount in the case of the 2nd WB correction calculation.

Explanation of symbols

101 CCD image sensor
102 memory
103 WB circuit
104 color conversion MTX circuit
105 LPF circuit
106 CSUP circuit
107 RGB conversion circuit
108 Gamma correction circuit
109 color luminance conversion circuit
110 Compression circuit
111 Y generator
112 Edge enhancement circuit
113 Control circuit
114 Face detector
115 CPU

Claims (7)

An imaging device for imaging a subject and processing an obtained image signal,
First white balance correction means for determining whether each pixel of the image sensor is white and performing white balance correction; and face detection means for detecting a face area from the image signal;
Having a determination means for determining whether to estimate the light source from the photographing conditions and use the result of the face detection means;
An image pickup apparatus comprising: a second white balance correction unit that corrects a defect in white balance due to a skin region based on information output by the first white balance correction unit and the face detection unit.   The imaging apparatus according to claim 1, wherein the face detection unit detects position information of an eye of a subject.   2. The imaging apparatus according to claim 1, wherein the face detection unit outputs a color evaluation value by averaging pixel values in the face area.   The imaging apparatus according to claim 1, wherein the estimation unit estimates based on brightness at the time of shooting and white balance correction coefficient information obtained by the white detection unit.   2. The white balance correction unit according to claim 1, wherein the second white balance correction unit performs white balance correction by estimating and weighting a defect in white balance based on the color evaluation value of the first white balance correction unit and the face area. Imaging device.   6. The imaging apparatus according to claim 5, wherein in the second white balance correction, a correction amount is variable depending on a face size.   6. The imaging apparatus according to claim 5, wherein in the second white balance correction, a correction amount is variable depending on a difference value between the first white balance correction result and the feature amount of the face area.

Images