JP2011139481A - Device, method and program for calculating auto white balance correction value, and imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent erroneous correction in auto white balance correction using a face detection function. <P>SOLUTION: In an S138, a difference (distance) Li between a representative color and a light source color of each area is calculated. In an S146, a weighted average L' of differences obtained by weighting each difference Li by a weight βi corresponding to each difference Li and averaging the differences Li is calculated. In an S147, the last WB correction value calculating part 52g selects a proper correction value calculating method on the basis of the difference weighted average L'. Thereby, the occurrence of a color failure can be prevented. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to white balance adjustment of an image using a face detection function.

  In Patent Document 1, a skin color area in an image is specified, a light source is specified according to the color of the skin area, and a white balance (WB) correction value is obtained.

  In Patent Literature 2, a face area in an image is specified, a representative skin color is extracted from the face area, a light source is specified according to the representative skin color, and a WB correction value is obtained.

Japanese Patent Laid-Open No. 11-283025 JP 2005-531189 A

  In Patent Documents 1 and 2, when a non-human face such as a sculpture or stone statue is detected, or when a non-skinned human face is detected by face painting, the correct light source cannot be specified and an appropriate white The balance correction value cannot be obtained. For example, suppose that it is estimated from the color distribution of the face of a white stone image that blue light hits the skin color of the face and becomes white. If the white balance adjustment is performed on the assumption that the blue light is the light source color, the image becomes yellow (so-called color failure phenomenon).

  An object of the present invention is to prevent erroneous correction in auto white balance correction using a face detection function.

  An auto white balance correction value calculation method according to the present invention includes a step of inputting image data, and a step of calculating a normal AWB correction value that is a first auto white balance (AWB) correction value based on the input image data. Identifying a face area from the image data, calculating a face AWB correction value that is a second AWB correction value based on the face area of the image data, and extracting first feature data from the image data And at least one of the face AWB correction value and the normal AWB correction value based on the step of extracting the second feature data from the image data in the face region and the comparison result between the first feature data and the second feature data. Calculating a final AWB correction value corresponding to one of them.

  The first feature data is the light source color specified based on the image data, and the second feature data is the light source color specified based on the face area.

  This method may further include a step of calculating a difference amount between the light source color of the image data and the light source color of the face region, and a step of calculating a final AWB correction value based on the difference amount.

  The method further includes the step of selecting either the face AWB correction value or the normal AWB correction value as the final AWB correction value based on the comparison result between the first feature data and the second feature data. But you can.

  Based on the comparison result between the first feature data and the second feature data, the final AWB correction is performed by weighted averaging the face AWB correction value and the normal AWB correction value with a predetermined weight α of 0% to 100%. A step of calculating a value may further be included.

  Based on the comparison result between the first feature data and the second feature data, each of the light source color of the image data and the light source color of the face area is weighted and averaged by a predetermined weight α of 0% or more and 100% or less. A step of obtaining a typical light source color and a step of calculating a final AWB correction value in accordance with the final light source color.

  An auto white balance correction value calculation method according to the present invention includes a step of inputting image data, and a step of calculating a normal AWB correction value that is a first auto white balance (AWB) correction value based on the input image data. A step of specifying a face area from the image data, a step of calculating a face AWB correction value as a second AWB correction value based on the face area of the image data, and dividing the image data into one or a plurality of areas. The step of extracting feature data from each area, the step of extracting feature data from the face area, and the face AWB correction based on the comparison result between the feature data extracted from each area and the feature data extracted from the face area Calculating a final AWB correction value corresponding to at least one of the value and the normal AWB correction value.

  A light source color is extracted as feature data from the face area, and a representative color of each area is extracted as feature data from each area.

  This method includes a step of calculating a difference amount between a representative color of a specific area among the representative colors of each area and a light source color of the face area, and a step of calculating a final AWB correction value based on the difference amount. Further, it may be included.

  The step of calculating the difference amount between the representative color of the specific area among the representative colors of each area and the light source color of the face area, and the difference amount for a predetermined number of areas in the vicinity of the face area of the difference amount And a step of calculating a final AWB correction value based on a value obtained by weighted averaging with a predetermined weight β of 0% or more and 100% or less corresponding to.

  A step of selecting either the face AWB correction value or the normal AWB correction value as the final AWB correction value based on the comparison result between the feature data extracted from each area and the feature data extracted from the face area. Further, it may be included.

  Based on the comparison result between the feature data extracted from each area and the feature data extracted from the face area, the face AWB correction value and the normal AWB correction value are weighted and averaged with a predetermined weight α of 0% to 100%. The method may further include a step of calculating a final AWB correction value.

  A step of obtaining a final light source color by performing a weighted average of each of the light source color of the image data and the light source color of the face region with a predetermined weight α of 0% or more and 100% or less, and the final according to the final light source color And calculating a typical AWB correction value.

  The method may further include displaying a final AWB correction value.

  The method may further include displaying an extracted area of the feature data compared with the feature data of the face region.

  The method may further include a step of receiving selection of face priority and a step of changing the weight of the final AWB correction value according to the face priority.

  An auto white balance correction value calculation program for causing a computer to execute the above auto white balance correction value calculation method is also included in the present invention.

  An auto white balance correction value calculation apparatus according to the present invention is an image data input unit that inputs image data, and a first auto white balance (AWB) correction value based on the image data input to the image data input unit. A normal AWB correction value calculating unit that calculates a normal AWB correction value, a face region specifying unit that specifies a face region from image data, and a face AWB correction value that is a second AWB correction value based on the face region of the image data A face AWB correction value calculation unit to be calculated; a feature data extraction unit that extracts first feature data from the image data and extracts second feature data from the image data in the face region; and first feature data; A final AW that calculates a final AWB correction value corresponding to at least one of the face AWB correction value and the normal AWB correction value based on the comparison result with the second feature data. Comprising a correction value calculating unit.

  The first feature data is the light source color specified based on the image data, and the second feature data is the light source color specified based on the face area.

  The apparatus further includes a difference amount calculation unit that calculates a difference amount between the light source color of the image data and the light source color of the face area, and the final AWB correction value calculation unit calculates a final AWB correction value based on the difference amount. It may be calculated.

  The final AWB correction value calculation unit selects one of the face AWB correction value and the normal AWB correction value as the final AWB correction value based on the comparison result between the first feature data and the second feature data. May be.

  The final AWB correction value calculation unit calculates a weighted average of the face AWB correction value and the normal AWB correction value with a predetermined weight α of 0% to 100% based on the comparison result between the first feature data and the second feature data. By doing so, the final AWB correction value may be calculated.

  Based on the comparison result between the first feature data and the second feature data, each of the light source color of the image data and the light source color of the face area is weighted and averaged by a predetermined weight α of 0% or more and 100% or less. A final light source color calculation unit for obtaining a specific light source color, and the final AWB correction value calculation unit may calculate a final AWB correction value according to the final light source color.

  An image data input unit that inputs image data, and a normal AWB correction value calculation unit that calculates a normal AWB correction value that is a first auto white balance (AWB) correction value based on the image data input to the image data input unit A face area specifying unit that specifies a face area from the image data, a face AWB correction value calculating unit that calculates a face AWB correction value that is a second AWB correction value based on the face area of the image data, and image data An area feature data extraction unit that extracts feature data from each area by dividing into one or a plurality of areas, a face region feature data extraction unit that extracts feature data from a face region, and feature data and a face extracted from each area Based on the comparison result with the feature data extracted from the region, a final AWB correction value corresponding to at least one of the face AWB correction value and the normal AWB correction value is calculated. It comprises a final AWB correction value calculation unit for, a.

  A light source color is extracted as feature data from the face area, and a representative color of each area is extracted as feature data from each area.

  The apparatus further includes a difference amount calculation unit that calculates a difference amount between the representative color of the specific area and the light source color of the face area among the representative colors of each area, and the final AWB correction value calculation unit is based on the difference amount, A final AWB correction value may be calculated.

  A difference amount calculation unit that calculates a difference amount between the representative color of the specific area of the representative colors of each area and the light source color of the face region is further provided, and the final AWB correction value calculation unit is located near the face region of the difference amount. Even if a final AWB correction value is calculated based on a value obtained by weighted averaging a difference amount for a certain number of areas with a predetermined weight β of 0% or more and 100% or less corresponding to the difference amount. Good.

  The final AWB correction value calculation unit finalizes either the face AWB correction value or the normal AWB correction value based on the comparison result between the feature data extracted from each area and the feature data extracted from the face area. You may select as an AWB correction value.

  The final AWB correction value calculation unit sets the face AWB correction value and the normal AWB correction value to a predetermined value of 0% or more and 100% or less based on the comparison result between the feature data extracted from each area and the feature data extracted from the face area. The final AWB correction value may be calculated by weighted averaging with the weight α.

  A final light source color calculation unit for obtaining a final light source color by weighted averaging each of the light source color of the image data and the light source color of the face area with a predetermined weight α of 0% or more and 100% or less, and final AWB correction The value calculation unit may calculate a final AWB correction value according to the final light source color.

  You may further provide the display part which displays a final AWB correction value.

  You may further provide the display part which displays the area from which the feature data compared with the feature data of the face area was extracted.

  A face priority selection unit that receives selection of face priority may be further provided, and the final AWB correction value calculation unit may change the weight of the final AWB correction value according to the face priority.

  An image pickup apparatus according to the present invention includes the above-described auto white balance correction value calculation apparatus, an image pickup device that receives a subject image via a photographing optical system, and outputs an analog image signal indicating the subject image, and an analog image signal. An image data output unit that converts the digital image data into an image data input unit; a correction unit that corrects the white balance of the image data based on the final AWB correction value calculated by the auto white balance correction value calculation device; .

  According to the present invention, when white balance correction is performed on the basis of the face area, even if a face having no normal skin color is specified, it is possible to make an erroneous white balance adjustment based on the face. Can be avoided.

Diagram showing the electrical configuration of a digital camera 1 is a block diagram of an image signal processing circuit according to a first embodiment. Flowchart of white balance correction processing according to the first embodiment Block diagram of an image signal processing circuit according to the second embodiment The figure which illustrated light source color L1 and L2 in color space (R / G, B / G) Flowchart of white balance correction processing according to the second embodiment Block diagram of an image signal processing circuit according to the third embodiment The figure which shows an example of the calculation formula of the difference amount (distance) of light source color L1 and L2. Flowchart of white balance correction processing according to the third embodiment Block diagram of an image signal processing circuit according to the fourth embodiment Flowchart of white balance correction processing according to the fourth embodiment The figure which illustrated the relationship between light source color L1 and L2, and a threshold value The figure which illustrated the concrete effect by white balance amendment of this embodiment Block diagram of an image signal processing circuit according to a fifth embodiment The figure which illustrated the numerical formula which carries out the weighted average of weight alpha specific function, normal AWB correction value, and face AWB correction value Flowchart of white balance correction processing according to the fifth embodiment Block diagram of an image signal processing circuit according to a sixth embodiment The figure which illustrated the numerical formula which carries out the weighted average of weight alpha specific function and normal AWB light source color, and face AWB light source color Flowchart of white balance correction processing according to the sixth embodiment Block diagram of an image signal processing circuit according to a seventh embodiment The figure which illustrated a mode that the whole image data was divided | segmented into the predetermined 1 or several area Flowchart of white balance correction processing according to the seventh embodiment The figure which illustrated the concrete effect by white balance amendment of this embodiment The figure which illustrated the image data which the light source estimation in normal AWB is far from the light source estimation in face AWB Block diagram of an image signal processing circuit according to the eighth embodiment Flowchart of white balance correction processing according to the eighth embodiment The figure which illustrated a mode that the light source color of the face area and the representative color of each area were plotted in the color space Block diagram of an image signal processing circuit according to the ninth embodiment The figure which illustrated the numerical formula which calculates the difference (distance) L between the representative color of each area in color space, and the light source color which the face AWB light source color extraction part 52c-1 obtained Flowchart of white balance correction processing according to the ninth embodiment The figure which illustrated minimum value Lmin among the difference Li with the light source color of the face area calculated | required for every representative color of each area Block diagram of an image signal processing circuit according to the tenth embodiment The figure which illustrated the position in the color space of the predetermined number of representative colors located near the light source color and the light source color The figure which illustrated the function which prescribes | regulates the relationship between the amount of difference and weight for specifying weight (beta) i The figure which illustrated weighted average L of distance Li Flowchart of white balance correction processing according to the tenth embodiment Block diagram of an image signal processing circuit according to the eleventh embodiment Flowchart of white balance correction processing according to the eleventh embodiment Block diagram of an image signal processing circuit according to a twelfth embodiment The figure which illustrated the function which prescribes | regulates the relationship between the amount of difference and weight for specifying the weight (alpha) corresponding to the value of the difference L Flowchart of white balance correction processing according to the twelfth embodiment Block diagram of an image signal processing circuit according to a thirteenth embodiment Diagram illustrating weight α determination function Flowchart of white balance correction processing according to the thirteenth embodiment 14 is a block diagram of an image signal processing circuit according to a fourteenth embodiment. The figure which shows an example which superimposed the weight (alpha) on image data, and displayed the postview Flowchart of white balance correction processing according to the fourteenth embodiment Block diagram of an image signal processing circuit according to the fifteenth embodiment The figure which shows an example which superimposed the frame which shows the division area where the light source color of the face AWB and the representative color were compared on the image data, and displayed it post-view Flowchart of white balance correction processing according to the fifteenth embodiment Block diagram of an image signal processing circuit according to the sixteenth embodiment Flowchart of white balance correction processing according to the sixteenth embodiment

<First Embodiment>
FIG. 1 shows an electrical configuration of the digital camera 2. A lens motor 30 is connected to the imaging lens 10. An iris motor 32 is connected to the diaphragm 31. These motors 30 and 32 are stepping motors, are controlled in operation by drive pulses transmitted from motor drivers 34 and 35 connected to the CPU 33, and perform photographing preparation processing in response to half-pressing of the release button 12.

  The lens motor 30 moves the zoom lens of the imaging lens 10 to the wide side or the tele side in conjunction with the operation of the zoom operation button 24. Further, the focus lens (not shown) of the imaging lens 10 is moved in accordance with zooming magnification of the zoom lens, etc., and focus adjustment is performed so that the photographing conditions are optimized. The iris motor 32 operates the diaphragm 31 to perform exposure adjustment.

  Behind the imaging lens 10 is a CCD 36 that captures a subject image that has passed through the imaging lens 10. A timing generator (TG) 37 controlled by the CPU 33 is connected to the CCD 36, and the shutter speed of the electronic shutter is determined by a timing signal (clock pulse) input from the TG 37.

  The imaging signal output from the CCD 36 is input to a correlated double sampling circuit (CDS) 38, and is output as R, G, B image data that accurately corresponds to the accumulated charge amount of each cell of the CCD 36. The image data output from the CDS 38 is amplified by an amplifier (AMP) 39 and converted into digital image data by an A / D converter (A / D) 40.

  The image input controller 41 is connected to the CPU 33 via the bus 42, and controls the CCD 36, CDS 38, AMP 39, and A / D 40 in accordance with a control command from the CPU 33. The image data output from the A / D 40 is temporarily recorded in the SDRAM 43.

  The image signal processing circuit 44 reads out image data from the SDRAM 43, performs various image processing such as gradation conversion, white balance correction, and γ correction processing, and records the image data in the SDRAM 43 again. The YC conversion processing circuit 45 reads out the image data subjected to various processes by the image signal processing circuit 44 from the SDRAM 43, and converts it into a luminance signal Y and color difference signals Cr and Cb.

  The VRAM 46 is a memory for outputting a through image to the LCD 22, and stores image data that has passed through the image signal processing circuit 44 and the YC conversion processing circuit 45. In the VRAM 46, two frames of memories 46a and 46b are secured so that image data can be written and read in parallel. The image data stored in the VRAM 46 is converted into an analog composite signal by the LCD driver 47 and displayed on the LCD 22 as a through image.

  The compression / decompression processing circuit 48 performs image compression on the image data YC converted by the YC conversion processing circuit 45 in a predetermined compression format (for example, JPEG format). The compressed image data is stored in the memory card 50 via the media controller 49.

  In addition to the release button 12, the receiving unit 20, and the operation unit 23 described above, an EEPROM 51 is connected to the CPU 33. The EEPROM 51 stores various control programs and setting information. The CPU 33 reads these pieces of information from the EEPROM 51 to the SDRAM 43, which is a working memory, and executes various processes.

  The bus 42 detects an exposure amount, that is, whether the shutter speed of the electronic shutter and the aperture value of the aperture 31 are appropriate for shooting, and detects whether the white balance is appropriate for shooting. 52, an AF detection circuit 53 that detects whether the focus adjustment of the imaging lens 10 is appropriate for shooting, and a strobe control circuit 55 that controls the operation of the strobe device 54 are connected.

  The AE / AWB detection circuit 52 detects the appropriateness of the exposure amount and the white balance based on the integrated value of the luminance signal Y of the image data YC converted by the YC conversion processing circuit 45 and the color difference signals Cr and Cb. This detection result is transmitted to the CPU 33. The CPU 33 controls the operations of the imaging lens 10, the diaphragm 31, and the CCD 36 based on the detection result transmitted from the AE / AWB detection circuit 52.

  The AF detection circuit 53 calculates a focus evaluation value representing the sharpness of the image from the image data digitized by the A / D 40 and transmits the calculation result to the CPU 33. The focus evaluation value is obtained by performing contour extraction processing on a specific area of the image, for example, image data of the central portion of the shooting angle of view by using a band pass filter or the like, and extracting the contour signal and image data of the central portion. Can be obtained by integrating the luminance values. Here, the larger the focus evaluation value is, the more high-frequency components are in that portion, indicating that the portion is in focus.

  The CPU 33 determines the search range of the focus lens focus position from the position of the zoom lens at the time of shooting preparation processing in response to half-pressing of the release button 12, and controls the operation of the lens motor 30 via the motor driver 34. Then, within the determined search range, the focus lens is moved, for example, from the near point side to the far point side, and the focus evaluation values sequentially transmitted from the AF detection circuit 53 at that time are compared, thereby obtaining a focus evaluation value. The focus lens is stopped at the position where is the maximum, that is, at the in-focus position.

  When the release signal is received from the remote controller 16 via the receiving unit 20 under the self-photographing mode, and the calculation result of the integrated value of luminance by the AE / AWB detection circuit 52 is smaller than a preset threshold value, The signal processing circuit 44 extracts image data representing light emitted from the light source 19 of the remote controller 16 from the image data read from the SDRAM 43 when the release signal is transmitted.

  Specifically, the extraction of the image data representing the light emitted from the light source 19 is performed by taking a difference between the image data of the previous frame and the current frame, and taking a frame in which the light emitted from the light source 19 is imaged. A part where light is imaged is specified from the difference between image data of no frames, and image data representing the part or the surrounding including the part is extracted. Note that the case where the calculation result of the integrated luminance value by the AE / AWB detection circuit 52 is smaller than a preset threshold means that the focus adjustment is difficult (such as in a dark place).

  The AF detection circuit 53 calculates a focus evaluation value for the image data representing the light extracted by the image signal processing circuit 44. The CPU 33 sets the point where the waveform of the focus evaluation value is a valley as the in-focus position, controls the operation of the lens motor via the motor driver 34, and stops the focus lens at this position.

  FIG. 2 shows a principal block configuration of the image signal processing circuit 44 according to the first embodiment. Detailed functions of each block disclosed in this figure will be described later.

  The face area specifying unit 52a specifies a face area that is an area including the face portion of a person from the digital image data (recording still image, through image, or moving image frame) of the SDRAM 43. As a face area detection method, for example, the technique disclosed in Japanese Patent Application Publication No. 2007-124112 by the present applicant can be applied.

  That is, the face area specifying unit 52a reads the image data P0 'of the photographed image and detects the face portion P0f' in the image P0 '. Specifically, as described in Japanese Patent Application Laid-Open No. 2005-108195, a first feature amount representing the direction of a gradient vector representing the direction and size of an edge in each pixel of the image P0 ′ is set to a plurality of features. By inputting to the first discriminator, it is determined whether or not a face candidate area exists in the image P0 ′. If a face candidate area exists, that area is extracted, and the extracted face candidate is extracted. The face candidate area extracted by normalizing the magnitude of the gradient vector in each pixel of the area and inputting the second feature amount indicating the magnitude and direction of the normalized gradient vector to the second classifier Is a true face area, and if it is a true face area, the area may be detected as a face portion P0f '. Here, the first / second discriminators are calculated for each of a plurality of images that are known to be faces as learning samples and a plurality of images that are known not to be faces. Each of them is obtained by a learning process using a machine learning method such as AdaBoost, which inputs 2 feature amounts.

  In addition, as a method for detecting the face portion P1f, in addition to the method using the correlation score between the unique face expression and the image itself as described in JP-T-2004-527863, a knowledge base, feature extraction, template matching, Various known methods such as graph matching and statistical methods (neural network, SVM, HMM) can be used. However, in order to enable face detection that does not depend on the color of the person's face (for example, to enable detection of a bronze face or face painted face), in each embodiment of the present specification, the face of the person It is assumed that a face detection method depending on the color, for example, a method using skin color detection is not adopted.

  The face AWB correction value calculation unit 52b determines a white balance correction value to be applied to the entire image based on the image data of the face region specified by the face region specifying unit 52a.

  The face AWB feature data extraction unit 52c extracts feature data based on the image data in the face area specified by the face area specification unit 52a. The feature data is, for example, the type of light source (sunlight, tungsten, fluorescence, etc.) estimated from the image data in the face area, or the color temperature of the light source (for example, the color temperature is estimated between 2500K to 9400K). . Alternatively, the representative color of the face area may be used.

  The normal AWB correction value calculation unit 52d is a normal AWB area that is a specific area for determining a normal white balance correction value (for example, the entire image data or the remaining part of the entire image data excluding a predetermined peripheral area. Based on the specific area is not the same as the face area), a white balance correction value to be applied to the entire image is determined.

  The normal AWB feature data extraction unit 52e extracts feature data based on the normal AWB area. The feature data includes a light source type, a color temperature, a representative color, and the like that are estimated from image data in a normal AWB area.

  The WB correction unit 52h adjusts the captured image to an appropriate white balance by increasing / decreasing the image data of each color at a ratio according to the type of light source or the color temperature of the light source, that is, color so that white does not have a tint. to correct.

  The feature data comparison unit 52f compares the feature data obtained by the face AWB feature data extraction unit 52c with the feature data obtained by the normal AWB feature data extraction unit 52e. As will be described later, for example, the distance between the light source color L1 and the light source color L2 in various color spaces is obtained.

  The final WB correction value calculation unit 52g selects a white balance correction value calculation method for the entire image based on the comparison result of the two feature data from the normal AWB feature data extraction unit 52e and the face AWB feature data extraction unit 52c. As will be described later, for example, based on whether the distance between the light source colors L1 and L2 is greater than or less than a predetermined threshold, the correction value calculation formula and the face AWB correction value used by the normal AWB correction value calculation unit 52d One of the correction value calculation formulas used by the calculation unit 52b is to be selected.

  The WB correction unit 52h corrects the white balance of the entire image using the white balance correction value calculated by the calculation method selected by the final WB correction value calculation unit 52g.

  FIG. 3 is a flowchart of white balance correction processing executed by the image signal processing circuit 44 according to the first embodiment.

  In S1, the normal AWB correction value calculation unit 52d calculates a correction value.

  In S2, the face area specifying unit 52a attempts to specify the face area.

  In S3, the face area specifying unit 52a determines whether the face area has been specified successfully. If the identification of the face area is successful, the process proceeds to S4. If the face area is unsuccessful, the process proceeds to S10.

  In S4, the face AWB correction value calculation unit 52b calculates a face AWB correction value.

  In S5, the face AWB feature data extraction unit 52c and the normal AWB feature data extraction unit 52e each extract feature data in the color space.

  In S6, the feature data comparison unit 52f compares the two feature data extracted in S5. For example, two representative colors in the normal AWB area and the face area are obtained, and the color differences (distances) between the two representative colors in a predetermined color space or chromaticity diagram are compared.

  In S7, the final WB correction value calculation unit 52g uses one suitable method for calculating the white balance correction value for the entire image based on the comparison result of the two feature data, for example, the face AWB correction value calculation unit 52b. Either the calculation method or the calculation method by the normal AWB correction value calculation unit 52d is selected. If the calculation method by the face AWB correction value calculation unit 52b is selected, the process proceeds to S8. If the calculation method by the normal AWB correction value calculation unit 52d is selected, the process proceeds to S9. The selection of the calculation method based on the comparison result of the two feature data is, for example, the calculation method of the face AWB if each color difference (distance) between the two representative colors is less than a predetermined threshold, and the normal AWB if the predetermined threshold is the predetermined threshold. Select the calculation method.

  In S8, the WB correction unit 52h corrects the white balance of the entire image by the final WB correction value = face AWB correction value obtained by the white balance correction value calculation method selected by the final WB correction value calculation unit 52g.

  In S9, the WB correction unit 52h corrects the white balance of the entire image by the final WB correction value = normal AWB correction value obtained by the white balance correction value calculation method selected by the final WB correction value calculation unit 52g.

  In S10, the WB correction unit 52h corrects the white balance of the entire image with the normal AWB correction value.

  That is, in this embodiment, the correction value calculation method is selected according to the comparison result between the feature data of the normal AWB area and the feature data of the face area.

Second Embodiment
FIG. 4 shows a detailed block configuration of the image signal processing circuit 44 according to the second embodiment. Here, the face AWB light source color extraction unit 52c-1 is an example of the face AWB feature data extraction unit 52c in FIG. 2, and the normal AWB light source color extraction unit 52e-1 is an example of the normal AWB feature data extraction unit 52e. A light source color comparison unit 52f-1 is shown as an example of the unit 52f.

  The face AWB light source color extracting unit 52c-1 extracts a light source color based on the image data in the face area specified by the face area specifying unit 52a. For example, various methods for estimating a light source color from a part of image data for white balance adjustment (for example, those disclosed in Japanese Patent Publication Nos. 2000-209598 and 2006-222928). Is used to estimate the light source color (R1, G1, B1), and the estimated light source color is converted to the coordinate L1 = (R1 / G1, B1 / G1) of the color space formed by the ratio of R / G and B / G. Convert (FIG. 5). Note that the color space for plotting the light source colors may be various types such as YCrCb.

  The normal AWB light source color extraction unit 52e-1 extracts the light source color based on the image data of the normal AWB area. For example, the light source color (R2, G2, B2) is estimated by using various methods for estimating the light source color from the entire image data for white balance adjustment, and the estimated light source color is set to R / G and A color space coordinate L2 = (R2 / G2, B2 / G2) having a B / G ratio is converted (FIG. 5).

  The light source color comparison unit 52f-1 compares the light source color L1 obtained by the face AWB light source color extraction unit 52c-1 with the light source color L2 obtained by the normal AWB light source color extraction unit 52e-1. As will be described later, for example, the color difference (distance in the color space) between the light source color L1 and the light source color L2 is obtained.

  The normal AWB correction value calculation unit 52d determines a white balance correction value to be applied to the entire image based on the image data of the normal AWB area.

  The WB correction unit 52h adjusts the captured image to an appropriate white balance by increasing / decreasing the image data of each color at a ratio according to the type of light source or the color temperature of the light source, that is, color so that white does not have a tint. to correct.

  The final WB correction value calculation unit 52g selects a white balance correction value calculation method for the entire image based on the comparison result between the light source colors L1 and L2 in the color space. As will be described later, for example, based on whether the distance between the light source colors L1 and L2 is greater than or less than a predetermined threshold, the correction value calculated by the normal AWB correction value calculation unit 52d and the face AWB correction value calculation One of the correction values calculated by the unit 52b is selected.

  The WB correction unit 52h corrects the white balance of the entire image using the white balance correction value calculated by the calculation method selected by the final WB correction value calculation unit 52g.

  FIG. 6 shows the flow of white balance correction processing executed by the image signal processing circuit 44.

  In S1, the normal AWB correction value calculation unit 52d calculates a correction value.

  In S2, the face area specifying unit 52a attempts to specify the face area.

  In S3, the face area specifying unit 52a determines whether the face area has been specified successfully. If the identification of the face area is successful, the process proceeds to S4. If the face area is unsuccessful, the process proceeds to S10.

  In S4, the face AWB correction value calculation unit 52b calculates a face AWB correction value.

  In S5-1, the face AWB light source color extraction unit 52c-1 and the normal AWB light source color extraction unit 52e-1 extract the light source colors L1 and L2 in the face area and the normal AWB area, respectively.

  In S6-1, the light source color comparison unit 52f-1 compares the light source colors L1 and L2.

  In S7-1, the final WB correction value calculation unit 52g uses one appropriate white balance correction value calculation method for the entire image based on the comparison result between the light source colors L1 and L2 by the light source color comparison unit 52f-1. For example, one of the calculation method by the face AWB correction value calculation unit 52b and the calculation method by the normal AWB correction value calculation unit 52d is selected. If the calculation method by the face AWB correction value calculation unit 52b is selected, the process proceeds to S8. If the calculation method by the normal AWB correction value calculation unit 52d is selected, the process proceeds to S9.

  In S8, the WB correction unit 52h corrects the white balance of the entire image by the final WB correction value = the face AWB correction value obtained by the white balance correction value calculation method selected by the final WB correction value calculation unit 52g.

  In S9, the WB correction unit 52h corrects the white balance of the entire image by the final WB correction value = normal AWB correction value obtained by the white balance correction value calculation method selected by the final WB correction value calculation unit 52g.

  In S10, the WB correction unit 52h corrects the white balance of the entire image with the normal AWB correction value.

  That is, in the present embodiment, the correction value calculation method is selected according to the comparison result between the light source color of the normal AWB area and the light source color of the face area.

<Third Embodiment>
FIG. 7 shows a block configuration of an image signal processing circuit 44 according to the third embodiment. Here, a light source color difference calculation unit 52f-2 is shown as an example of the light source color comparison unit 52f-1 in FIG.

  The light source color difference calculation unit 52f-2 calculates a difference between the light source color L1 obtained by the face AWB light source color extraction unit 52c-1 and the light source color L2 obtained by the normal AWB light source color extraction unit 52e-1. As will be described later, for example, as shown in FIG. 8, this is to obtain the distance L between the light source color L1 and the light source color L2 in the color space.

  FIG. 9 shows the flow of white balance correction processing executed by the image signal processing circuit 44.

  S11 to S15 are the same as S1 to S5 in FIG.

  In S16, the light source color difference calculation unit 52f-2 calculates a difference L between the light source colors L1 and L2. Specifically, the difference L indicates the distance (color difference) between the light source colors L1 and L2 in the color space, as shown in FIG. The value can be obtained by a mathematical formula as shown in FIG.

  In S17, the final WB correction value calculation unit 52g calculates an optimum correction value calculation method (for example, the first and second implementations) based on the difference L between the light source colors L1 and L2 calculated by the light source color difference calculation unit 52f-2. The same as the form). If the calculation method of the face AWB correction value is selected, the process proceeds to S18, and if the calculation method of the normal AWB correction value is selected, the process proceeds to S19.

  S18 to S20 are the same as S8 to S10.

<Fourth embodiment>
FIG. 10 shows a block configuration of an image signal processing circuit 44 according to the fourth embodiment. Here, a final WB selection unit 52g-1 is shown as an example of the final WB correction value calculation unit 52g in FIGS. This function will be described later.

  FIG. 11 shows the flow of white balance correction processing executed by the image signal processing circuit 44.

  S21 to S26 are the same as S11 to S16, respectively.

  In S27, the final WB correction value calculation unit 52g determines whether or not the difference L between the light source colors L1 and L2 is equal to or less than a predetermined threshold T1. If L ≦ T1, the process proceeds to S28, and if L> T1, the process proceeds to S29. FIG. 12 illustrates the case of L> T1. T1 is a value for distinguishing which one of the face AWB correction value and the normal AWB correction value is used for appropriate white balance correction, and therefore it may be determined empirically according to the shooting conditions.

  In S28, the final WB correction value calculation unit 52g determines the face AWB correction value as the final WB correction value. The WB correction unit 52h corrects the white balance of the entire image using the final WB correction value determined by the final WB correction value calculation unit 52g = the face AWB correction value.

  In S29, the final WB correction value calculation unit 52g determines the normal AWB correction value as the final WB correction value. The WB correction unit 52h corrects the white balance of the entire image by the final WB correction value = normal AWB correction value determined by the final WB correction value calculation unit 52g.

  S30 is the same as S10.

  FIG. 13 illustrates a specific effect by this white balance correction.

  For example, as shown in FIG. 13A, it is assumed that image data having a white stone image irradiated with sunlight as a subject is acquired.

  As shown in FIG. 13B, the normal AWB light source color extraction unit 52e-1 estimates the light source color from the color distribution of the entire image data. As shown in FIG. 13C, it is assumed here that sunlight is estimated.

  As shown in FIG. 13D, the face AWB light source color extraction unit 52c-1 estimates the light source color from the color distribution of the face area specified by the face area specifying unit 52a in the image data. As shown in FIG. 13 (e), it is assumed here that it is estimated from the color distribution of the face of the white stone image that blue light hits the skin color of the face and becomes white. If the white balance adjustment is performed on the assumption that the blue light is the light source color, the image becomes yellow.

  In this case, as shown in FIG. 13 (f), the distance L between the light source color L1 of the face AWB light source color extraction unit 52c-1 and the light source color L2 of the normal AWB light source color extraction unit 52e-1 tends to increase. . If this distance L exceeds a predetermined threshold T1, the difference between the two is large, the estimation of the light source color L1 of the face AWB light source color extraction unit 52c-1 is low, and the normal AWB light source color extraction unit 52e-1 It can be said that it is more appropriate to determine the WB correction value based on the light source color L2. Therefore, when L> T1, the white balance of the entire image is corrected by the normal AWB correction value.

  On the other hand, when L ≦ T1, the reliability of the estimation of the light source color L1 is ensured to a certain degree. In this case, the white balance of the entire image is corrected by the face AWB correction value.

  In this way, when performing white balance correction based on the face area, even if a face that does not have a normal skin color is identified, it is possible to make an incorrect white balance adjustment based on that face. Can be avoided.

<Fifth Embodiment>
FIG. 14 shows a block configuration of an image signal processing circuit 44 according to the fifth embodiment. Here, a weight α calculator 52i is added.

  For example, a function (weight α determination function) that defines the relationship between the difference L and the weight α as shown in FIG. 15A is stored in advance in the weight α calculation unit 52i, and the weight α calculation unit 52i By specifying the weight α corresponding to the value of the function from the function, the function weight α is calculated.

  The final WB correction value calculation unit 52g calculates the final WB correction value by a mathematical expression that weights and averages the normal AWB correction value and the face AWB correction value with the weight α as shown in FIG. 15B, for example.

  FIG. 16 is a flowchart showing white balance correction processing according to the fifth embodiment.

  S31 to S36 are the same as S11 to S16.

  In S37, it is determined whether or not the difference L is equal to or less than a predetermined threshold value T2. If L ≦ T2, the process proceeds to S38, and if L> T2, the process proceeds to S41.

  In S38, the weight α is set to 100 (%).

  In S40, the final WB correction value is set as the face AWB correction value. This value is a value in which α = 100 in the equation of FIG.

  In S41, it is determined whether or not the difference L is greater than or equal to a predetermined threshold T3. If L ≧ T3, the process proceeds to S42, and if L <T3, the process proceeds to S45.

  In S42, the weight α = 0 (%).

  In S44, the final WB correction value is set as the normal AWB correction value. This value is a value in which α = 0 in the equation of FIG.

  In S45, the weight α corresponding to the difference L is set to a value of 1 to 99 (%). For example, a function (weight α determining function) that defines the relationship between the difference L and the weight α as shown in FIG. 15A is stored in advance, and the weight α corresponding to the value of the difference L is specified from the function.

  In S47, for example, as shown in FIG. 15B, the final WB correction value is calculated by a mathematical expression that weights and averages the normal AWB correction value and the face AWB correction value with the weight α.

  S48 is the same as S10.

<Sixth Embodiment>
FIG. 17 shows a block configuration of an image signal processing circuit 44 according to the sixth embodiment. Here, a final light source color calculation unit 52j is added.

  For example, a function (weight α determination function) that defines the relationship between the difference L and the weight α as shown in FIG. 18A is stored in the weight α calculation unit 52i in advance, and the weight α calculation unit 52i By identifying the weight α corresponding to the value from the function, the light source color weight α is calculated.

  The final light source color calculation unit 52j calculates the final light source color by a mathematical expression that weights and averages the normal AWB light source color and the face AWB light source color with a weight α as shown in FIG. 18B, for example.

  FIG. 19 is a flowchart showing white balance correction processing according to the sixth embodiment.

  S51 to S56 are the same as S31 to S36.

  In S57, it is determined whether or not the difference L is equal to or less than a predetermined threshold T4. If L ≦ T4, the process proceeds to S58, and if L> T4, the process proceeds to S61.

  In S58, the weight α is set to 100 (%).

  In S59, the final light source color is the light source color extracted by the face AWB light source color extraction unit 52c-1.

  In S60, the light source color extracted by the face AWB light source color extraction unit 52c-1 that is the final light source color is used as a reference for calculating the correction value, and the final WB correction value is used as the face AWB correction value.

  In S61, it is determined whether or not the difference L is greater than or equal to a predetermined threshold T5. If L ≧ T5, the process proceeds to S62, and if L <T5, the process proceeds to S65.

  In S62, the weight α = 0 (%).

  In S63, the final light source color is the light source color extracted by the normal AWB light source color extraction unit 52e-1.

  In S64, the light source color extracted by the normal AWB light source color extraction unit 52e-1 which is the final light source color is used as a correction value calculation reference, and the final WB correction value is set as a normal AWB correction value.

  In S65, the weight α corresponding to the difference L is set to a value of 1 to 99 (%). For example, a function (weight α determination function) that defines the relationship between the difference L and the weight α as shown in FIG. 18A is stored in advance, and the weight α corresponding to the value of the difference L is specified from the function.

  In S66, the final light source color is calculated by a mathematical formula that weights and averages the two light source colors with the weight α as shown in FIG. 18B, for example.

  In S67, the final WB correction value is calculated using the final light source color as the correction value calculation reference.

  S68 is the same as S10.

<Seventh embodiment>
FIG. 20 shows a block configuration of an image signal processing circuit 44 according to the seventh embodiment. The same blocks as those in the above embodiment are denoted by the same reference numerals, and description thereof is omitted. Here, an area division unit 52k and an area feature data extraction unit 52l are added.

  The area dividing unit 52k divides the entire image data into predetermined one or a plurality of areas.

  FIG. 21 shows an example of dividing into a plurality of areas. The method of dividing into a plurality of areas is not limited to that shown in the figure, and may be divided into a larger number or a smaller number of small areas. In addition, the divided areas are not necessarily equal areas and equally spaced, and more precise area division may be performed according to the characteristics of the area (for example, whether or not the area is included in the face area) and importance. . Or you may divide | segment an image for every different hue. Note that dividing into one area is the same as obtaining the entire image data itself.

  FIG. 22 is a flowchart of white balance correction processing according to the seventh embodiment.

  S71 to S75 are the same as S1 to S5.

  In S76, the area dividing unit 52k divides the entire image data into one or a plurality of areas.

  In S77, the area feature data extraction unit 52l extracts feature data of each area. The feature data is, for example, a representative color of each area. As a method for obtaining a representative color of an area, for example, as described in Japanese Patent Application Publication No. 2007-36462 and Paragraph 0038 by the present applicant, the color of each pixel is set to R / G- The coordinates are converted into coordinates in the B / G space, the coordinates of the center of gravity of each of these coordinates are obtained, and this is used as the representative color coordinates.

  In S78, the feature data comparison unit 52f compares the feature data of each area with the feature data from the face area.

  In S79, an appropriate correction value calculation formula is selected according to the result of comparing the feature data of each area with the feature data from the face area. For example, as described above, the optimum value of either the correction value (normal AWB correction value) calculated by the normal AWB correction value calculation unit 52d or the correction value (face AWB correction value) calculated by the face AWB correction value calculation unit 52b is used. Select a correction value.

  S80 to S82 are the same as S8 to S10.

  FIG. 23 illustrates a specific effect by this white balance correction.

  For example, as shown in FIG. 23A, it is assumed that a person is illuminated with a red light source and a blue aquarium exists in the background in one image data.

  In this case, as shown in FIG. 23B, one screen is divided into a plurality of areas, and as shown in FIG. 23C, the representative color of each area is calculated as feature data of each area. In this case, blue is the representative color in the water tank side area, and red is the representative color in the light source side area.

  Further, as shown in FIG. 23D, the light source color is estimated from the detected face area. In this case, as shown in FIG. 23E, a red light source is estimated from the face color.

  That is, there is a discrepancy between the feature data of each area and the feature data of the face area, and when white balance correction is performed according to each feature data, the image data obtained also differs. Therefore, it is necessary to perform more appropriate white balance correction based on either one of the feature data.

  As shown in FIG. 23 (f), when the feature data (representative color) of the face area and the feature data (representative color) of each area are plotted in the R / GB / G color space, In the vicinity, feature data of an area near the light source where the red light source is dominant are concentrated, and in the area near the water tank, blue is concentrated at a dominant position.

  If feature data of a certain degree of divided area exists within the vicinity of the predetermined threshold T centered on the face area, it can be determined that this represents the correct light source color, and the blue color outside the vicinity can be determined. The feature data can be identified as a color that dominates the background, and it can be identified that white balance correction by the face AWB is a better correction method.

  As shown in FIG. 24, this method is particularly effective when the light source estimation in the normal AWB is far from the light source estimation in the face AWB because most of the image is dominated by colors that are not related to the original light source. is there.

<Eighth Embodiment>
FIG. 25 shows a block configuration of an image signal processing circuit 44 according to the eighth embodiment. The same reference numerals are given to the same blocks as those in the other embodiments.

  FIG. 26 is a flowchart of white balance correction processing according to the eighth embodiment.

  S91 to S94 are the same as S71 to S74.

  In S95, the face AWB feature data extraction unit 52c extracts a light source color from the face area. Then, the extracted light source color is plotted in a color space (see, for example, FIG. 27).

  S96 is the same as S76.

  In S97, the area feature data extraction unit 52l extracts a representative color of each area (for example, an average color of pixels for each area). Then, the extracted representative colors of each area are plotted in a color space (see, for example, FIG. 27).

  In S98, the feature data comparison unit 52f compares the representative color of each area with the light source color from the face area.

  S99 branches to S100 or S101 depending on the comparison result of S98.

  S100 to S102 are the same as S80 to S82.

<Ninth Embodiment>
FIG. 28 shows a block configuration of an image signal processing circuit 44 according to the ninth embodiment. The same reference numerals are given to the same blocks as those in the other embodiments.

  The difference calculation unit 52f-3 between the light source color and the representative color of the specific area calculates the representative color of each area in the color space and the light source color obtained by the face AWB light source color extraction unit 52 by a mathematical formula as illustrated in FIG. The difference (distance) L is calculated.

  FIG. 30 is a flowchart of white balance correction processing according to the ninth embodiment.

  S111 to S117 are the same as S91 to S97.

  In S118, the difference (distance) between the representative color of each area Ri (i is a subscript assigned to the area) in the color space and the light source color obtained by the face AWB light source color extraction unit 52, for example, using the formula illustrated in FIG. ) Calculate Li. Then, the minimum value Lmin is determined among the differences Li obtained for each area (see FIG. 31).

  In S119, the final WB correction value calculation unit 52g selects an appropriate correction value calculation method based on the difference Lmin. For example, as described above, the correction value (normal AWB correction value) calculated by the normal AWB correction value calculation unit 52d and the calculation by the face AWB correction value calculation unit 52b according to the magnitude relationship between the difference Lmin and a predetermined threshold value. One of the optimum correction values is selected from the corrected values (face AWB correction values). If the face AWB correction value is selected, the process proceeds to S120. If the normal AWB correction value is selected, the process proceeds to S121.

  S120 to S122 are the same as S100 to S102.

<Tenth Embodiment>
FIG. 32 shows a block configuration of an image signal processing circuit 44 according to the tenth embodiment. The same reference numerals are given to the same blocks as those in the other embodiments.

  The difference calculation unit 52f-4 between the light source color and the representative colors of the n areas calculates the difference between the representative color of each area and the light source color from the face AWB light source color extraction unit 52c-1.

  For example, as shown in FIG. 33, a distance between a predetermined number of representative colors (for example, about 5 to 20% of the total number of areas) located near the light source color and the light source color is obtained for each representative color.

  The weighting average unit 52p calculates a weighted average of differences by multiplying the weight (βi) corresponding to the difference (distance) Li between each area and the light source color (FIG. 35). The weight βi corresponding to each difference Li is specified from a weight βi determination function that defines the relationship between the difference amount and the weight, for example, as shown in FIG.

  FIG. 36 is a flowchart of white balance correction processing according to the tenth embodiment.

  S131 to 137 are the same as S111 to S117.

  In S138, a difference (distance) Li between the representative color of each area and the light source color is calculated.

  S139 to S145 are processing units that are repeated until all (n) weights βi corresponding to each difference Li in each area are calculated.

  In S140, it is determined whether or not the difference Li is equal to or less than a predetermined threshold T6. If the determination is “Y”, the process proceeds to S142, and if “N”, the process proceeds to S141.

  In S141, it is determined whether or not the difference Li is greater than or equal to a predetermined threshold T7. If the determination is “Y”, the process proceeds to S143, and if “N”, the process proceeds to S144.

  In S142, the weight βi = 100% is set according to the weight determination function of FIG.

  In S143, the weight βi = 0% according to the weight determination function of FIG.

  In S144, the weight βi is set to 1 to 99% according to the weight determination function of FIG.

  In S145, it is determined whether n weights have been calculated, and if calculated, the process proceeds to S146. If not calculated, the process returns to S140.

  In S146, a weighted average L ′ of differences obtained by weighting and averaging each difference Li with a weight βi corresponding to each difference Li is calculated (see FIG. 35).

  In S147, the final WB correction value calculation unit 52g selects an appropriate correction value calculation method based on the difference weighted average L '. A specific example of this is the same as described above. If the first calculation method (for example, face AWB correction value) is selected, the process proceeds to S148. If the second calculation method (for example, normal AWB correction value) is selected, the process proceeds to S149.

  S148 to S150 are the same as S120 to S122.

<Eleventh embodiment>
FIG. 37 shows a block configuration of the image signal processing circuit 44 according to the eleventh embodiment. The same reference numerals are given to the same blocks as those in the other embodiments.

  The difference calculation unit 52f-5 between the light source color and the representative color calculates the difference between the representative color of each area and the light source color from the face AWB light source color extraction unit 52c-1.

  FIG. 38 is a flowchart of white balance correction processing according to the eleventh embodiment.

  S161 to S167 are the same as S131 to S137.

  In S168, the difference L between the light source color of the face AWB and the representative color of each area is calculated. What kind of difference is used is arbitrary, and is, for example, the above-described difference minimum value Lmin, weighted difference average L ′, or the like.

  In S169, it is determined whether or not the difference L is equal to or less than a predetermined threshold T5. If L ≦ T5, the process proceeds to S170, and if L> T5, the process proceeds to S171.

  S170 to S172 are the same as S28 to S30.

<Twelfth embodiment>
FIG. 39 shows a block configuration of an image signal processing circuit 44 according to the twelfth embodiment. The same reference numerals are given to the same blocks as those in the other embodiments.

  For example, the weight α calculating unit 52i stores in advance a function that defines the relationship between the difference L and the weight α as shown in FIG. 40, and specifies the weight α corresponding to the value of the difference L from the function. The weight α is calculated.

  FIG. 41 is a flowchart of white balance correction processing according to the twelfth embodiment.

  S181 to 188 are the same as S161 to S168.

  S189 to S200 are the same as S37 to S48 (FIG. 16). However, the threshold values used in S189 and S193 are T6 and T7, respectively.

<13th Embodiment>
FIG. 42 shows a block configuration of an image signal processing circuit 44 according to the thirteenth embodiment. The same reference numerals are given to the same blocks as those in the other embodiments.

  For example, the final light source color calculation unit 52j stores in advance a weight determination function that defines the relationship between the difference L and the weight α as shown in FIG. 43, and specifies the weight α corresponding to the value of the difference L from the function. Thus, the weight α is calculated.

  FIG. 44 is a flowchart of white balance correction processing according to the thirteenth embodiment.

  S211 to S218 are the same as S181 to S188.

  S219 to S230 are the same as S57 to S68 (FIG. 19). However, the threshold values used in S219 and S223 are T8 and T9, respectively.

<Fourteenth embodiment>
FIG. 45 shows a block configuration of an image signal processing circuit 44 according to the fourteenth embodiment. The same reference numerals are given to the same blocks as those in the other embodiments.

  As illustrated in FIG. 46, the display icon creation unit 52q generates a video signal of the icon IC indicating the weight α (), superimposes it on the captured image data, outputs it to the LCD 22, and displays it together with the image. Let

  FIG. 47 is a flowchart of white balance correction processing according to the fourteenth embodiment.

  S241 to S257 are the same as S181 to S200 (FIG. 41).

  In S258, a video signal of the icon IC indicating the weight α is generated, superimposed on the captured image data, output to the LCD 22, and displayed as a postview (image view after imaging) together with the image. Thereby, the user can know what weight is added, and if it is judged visually that it is inappropriate, the user can redo the shooting.

<Fifteenth embodiment>
FIG. 48 shows a block configuration of an image signal processing circuit 44 according to the fifteenth embodiment. The same reference numerals are given to the same blocks as those in the other embodiments.

  As illustrated in FIG. 49, the display frame creation unit 52q generates a video signal of a frame F indicating an area having a representative color closest to the light source color of the face AWB, and superimposes the video signal on the captured image data. , Output to the LCD 22 and display it together with the image.

  FIG. 50 is a flowchart of white balance correction processing according to the fifteenth embodiment.

  S261 to S267 are the same as S181 to S200.

  In S278, a video signal of a frame F indicating an area having a representative color closest to the light source color of the face AWB is generated, superimposed on the captured image data, output to the LCD 22, and displayed as a postview together with the image. Let Thereby, the user can know which area's feature data is the standard of white balance, and can re-shoot the image if it is visually judged as inappropriate.

<Sixteenth Embodiment>
FIG. 51 shows a block configuration of an image signal processing circuit 44 according to the sixteenth embodiment. The same reference numerals are given to the same blocks as those in the other embodiments.

  The weight α ′ calculation unit 52 s calculates the weight α ′ based on the priority of the face AWB read from the operation unit 23 in accordance with a command from the CPU 33.

  Here, the priority of the face AWB is, for example, high (the setting priority is higher than the standard priority and the weight α ′ is larger than the weight α calculated by the image signal processing circuit 44), medium (setting The priority and the standard priority are the same, and the weight α and the weight α ′ calculated by the image signal processing circuit 44 are the same), low (the setting priority is lower than the standard priority, and the image signal processing circuit The weight α ′ is made smaller than the weight α calculated in 44), and the user can select one desired level from among the three levels. Of course, the high and low stages may have further subdivided stages to further increase or decrease the weight α ′ by a desired width.

  FIG. 52 is a flowchart of white balance correction processing according to the sixteenth embodiment.

  S281 to S295 are the same as S261 to S275 (FIG. 50).

  In S296, the priority setting (weight α ′) of the face AWB is read from the operation unit 23. Then, it is determined whether the priority (weight α ′) of the set face AWB matches the priority (weight α) of the reference face AWB. If they match, the process proceeds to S297, and if not, the process proceeds to S298.

  In S297, as in S47, the final WB correction value is calculated based on the weight α.

  In S298, it is determined whether the priority (weight α ′) of the set face AWB is higher than the priority (weight α) of the reference face AWB. If “Y”, the process proceeds to S299, and if “N”, the process proceeds to S301.

  In S299, a final WB correction value is calculated based on the weight α ′. In this case, since the weight of the face AWB increases as a result, a final correction value close to the face AWB correction value is obtained (S300).

  In S301, a final WB correction value is calculated based on the weight α ′. In this case, as a result, the weight of the face AWB is lowered, so that a final correction value close to the normal AWB correction value is obtained (S302).

  S303 is the same as S277.

  Thus, by changing the weights of the face AWB and the normal AWB according to the user's preference, image data as intended by the user can be obtained.

  44: Image signal processing circuit, 52a: Face area specifying unit, 52b: Face AWB correction value calculating unit, 52c: Face AWB feature data extracting unit, 52d: Normal AWB correction value calculating unit, 52e: Normal AWB feature data extracting unit, 52f: feature data comparison unit, 52g: final AWB correction value calculation unit, 52h: WB correction unit

Claims (22)

Inputting image data;
Calculating a normal AWB correction value that is a first auto white balance (AWB) correction value based on the input image data;
Identifying a face region from the image data;
Calculating a face AWB correction value that is a second AWB correction value based on the face area of the image data;
Dividing the image data into one or more areas and extracting feature data from each area;
Extracting feature data from the face region;
Based on the comparison result between the feature data extracted from each area and the feature data extracted from the face area, a final AWB correction value corresponding to at least one of the face AWB correction value and the normal AWB correction value Calculating steps,
Auto white balance correction value calculation method including From the face area, a light source color is extracted as the feature data,
The auto white balance correction value calculation method according to claim 1, wherein a representative color of each area is extracted as the feature data from each area. Calculating a difference amount between a representative color of a specific area among representative colors of each area and a light source color of the face area;
Calculating a final AWB correction value based on the difference amount;
The auto white balance correction value calculation method according to claim 2, further comprising: Calculating a difference amount between a representative color of a specific area among representative colors of each area and a light source color of the face area;
Based on a value obtained by performing a weighted average of a difference amount for a predetermined number of areas in the vicinity of the face area in the difference amount with a predetermined weight β corresponding to the difference amount of 0% or more and 100% or less. Calculating a final AWB correction value;
The auto white balance correction value calculation method according to claim 2, further comprising:   Based on the comparison result between the feature data extracted from each area and the feature data extracted from the face area, one of the face AWB correction value and the normal AWB correction value is selected as the final AWB correction value. The auto white balance correction value calculation method according to claim 1, further comprising a step.   Based on the comparison result between the feature data extracted from each area and the feature data extracted from the face area, the face AWB correction value and the normal AWB correction value are weighted average by a predetermined weight α of 0% to 100%. The auto white balance correction value calculation method according to claim 1, further comprising a step of calculating a final AWB correction value. Obtaining a final light source color by weighted averaging each of the light source color of the image data and the light source color of the face region with a predetermined weight α of 0% or more and 100% or less;
Calculating a final AWB correction value according to the final light source color;
The auto white balance correction value calculation method according to claim 2, further comprising:   The auto white balance correction value calculation method according to claim 1, further comprising a step of displaying the final AWB correction value.   The auto white balance correction value calculation method according to claim 1, further comprising a step of displaying an extracted area of feature data compared with the feature data of the face region. Receiving a face priority selection;
Changing the weight according to the face priority;
The auto white balance correction value calculation method according to claim 4, 6 or 7, further comprising: Inputting image data;
Calculating a normal AWB correction value that is a first auto white balance (AWB) correction value based on the input image data;
Identifying a face region from the image data;
Calculating a face AWB correction value that is a second AWB correction value based on the face area of the image data;
Dividing the image data into one or more areas and extracting feature data from each area;
Extracting feature data from the face region;
Based on the comparison result between the feature data extracted from each area and the feature data extracted from the face area, a final AWB correction value corresponding to at least one of the face AWB correction value and the normal AWB correction value Calculating steps,
Auto white balance correction value calculation program that causes a computer to execute. An image data input unit for inputting image data;
A normal AWB correction value calculation unit that calculates a normal AWB correction value that is a first auto white balance (AWB) correction value based on the image data input to the image data input unit;
A face area specifying unit for specifying a face area from the image data;
A face AWB correction value calculation unit that calculates a face AWB correction value that is a second AWB correction value based on the face area of the image data;
An area feature data extraction unit that divides the image data into one or a plurality of areas and extracts feature data from each area;
A face area feature data extraction unit for extracting feature data from the face area;
Based on the comparison result between the feature data extracted from each area and the feature data extracted from the face area, a final AWB correction value corresponding to at least one of the face AWB correction value and the normal AWB correction value A final AWB correction value calculation unit for calculating
An auto white balance correction value calculation device. From the face area, a light source color is extracted as the feature data,
The automatic white balance correction value calculation apparatus according to claim 12, wherein a representative color of each area is extracted as the feature data from each area. A difference amount calculating unit that calculates a difference amount between a representative color of a specific area and a light source color of the face area among the representative colors of each area;
The auto white balance correction value calculation apparatus according to claim 13, wherein the final AWB correction value calculation unit calculates a final AWB correction value based on the difference amount. A difference amount calculating unit that calculates a difference amount between a representative color of a specific area and a light source color of the face area among the representative colors of each area;
The final AWB correction value calculation unit weights a difference amount for a predetermined number of areas in the vicinity of the face area in the difference amount with a predetermined weight β of 0% or more and 100% or less corresponding to the difference amount. The auto white balance correction value calculation apparatus according to claim 13, wherein a final AWB correction value is calculated based on a value obtained by averaging.   The final AWB correction value calculation unit calculates one of a face AWB correction value and a normal AWB correction value based on a comparison result between the feature data extracted from each area and the feature data extracted from the face area. The auto white balance correction value calculation apparatus according to any one of claims 12 to 15, which is selected as a final AWB correction value.   The final AWB correction value calculation unit calculates a face AWB correction value and a normal AWB correction value from 0% to 100% based on a comparison result between the feature data extracted from each area and the feature data extracted from the face area. The auto white balance correction value calculation apparatus according to any one of claims 12 to 15, wherein a final AWB correction value is calculated by performing a weighted average with a predetermined weight α described below. A final light source color calculation unit for obtaining a final light source color by weighted averaging each of the light source color of the image data and the light source color of the face region with a predetermined weight α of 0% or more and 100% or less;
The auto white balance correction value calculation apparatus according to claim 13, wherein the final AWB correction value calculation unit calculates a final AWB correction value according to the final light source color.   The auto white balance correction value calculation apparatus according to claim 12, further comprising a display unit that displays the final AWB correction value.   The auto white balance correction value calculation apparatus according to claim 12, further comprising a display unit that displays an area where feature data compared with the feature data of the face region is extracted. A face priority selection unit for receiving selection of face priority;
The auto white balance correction value calculation device according to claim 15, 17 or 18, wherein the final AWB correction value calculation unit changes the weight according to the face priority. The auto white balance correction value calculation device according to any one of claims 12 to 21,
An image sensor that receives a subject image via a photographing optical system and outputs an analog image signal indicating the subject image;
An image data output unit for converting the analog image signal into digital image data and outputting the digital image data to the image data input unit;
A correction unit that corrects the white balance of the image data based on the final AWB correction value calculated by the auto white balance correction value calculation device;
An imaging apparatus comprising:

Images