JP5854716B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to a white balance control technique for image data.

  An imaging apparatus using an imaging element such as a digital camera or a digital video camera has a white balance control function for adjusting the color tone of image data obtained by shooting. White balance control includes manual white balance control and automatic white balance control. Manual white balance control is control in which a white subject is imaged in advance and a white balance coefficient is calculated, and the calculated white balance coefficient is applied to the entire screen. Auto white balance control is a control that automatically detects white parts from captured image data, calculates the white balance coefficient from the average value of each color component on the entire screen, and applies the calculated white balance coefficient to the entire screen. is there.

  In conventional auto white balance control, when a light source different from strobe light exists in image data in a strobe lighting scene, white balance control is performed by applying the white balance coefficient calculated as described above to the entire screen. . For this reason, it is difficult to perform white balance control in which each light source has an appropriate color. For example, if there is a low color temperature light source such as a light bulb color light source in the ambient light in a strobe lighting scene, the strobe light is a high color temperature light source. Will not fit. On the other hand, when the white balance is matched with the low color temperature light source, the white balance is not matched with the strobe light. Even if white balance control is performed by adjusting the white balance in the middle of both light sources, the white balance does not match both light sources, the area illuminated by the strobe light is bluish, and the low color temperature light source is used. The irradiated area has a reddish color.

  In contrast, Patent Document 1 discloses the following technique. That is, the technique disclosed in Patent Document 1 compares the image data captured when the flash is fired with the image data captured when the flash is not flashed for each subject area to obtain a data ratio. The contribution of strobe light is determined based on the value of. Then, a white balance control value is selected for each subject area according to the determined contribution, and white balance control is performed.

Japanese Patent No. 3540485

  In conventional automatic exposure control (AE), exposure control is performed so that the subject brightness is measured and an appropriate exposure is obtained. In order to measure the subject brightness correctly, it is necessary to correct the color of the light source. However, since the exposure is determined before photographing and the color temperature of the light source is determined after photographing, the color temperature of the photographed image data cannot be obtained at the time of photometry by AE. Therefore, in photometry, exposure control is performed by calculating the luminance using the white balance coefficient of the color temperature of sunlight. Thereafter, the color temperature of the light source is accurately obtained after still image shooting, and the difference from the white balance coefficient of sunlight temporarily used during photometry is calculated as a luminance correction coefficient.

  Luminance correction based on such variations in color temperature is particularly called color temperature correction. Conventionally, one white balance coefficient is applied to the entire screen for captured image data. However, when a different white balance coefficient is applied to each region, the color temperature correction coefficient cannot be uniquely obtained. There was a problem.

  FIG. 9 is a diagram showing the relationship between the color temperature and the luminance. When the color temperature of sunlight used for photometry is 5200K, the luminance changes as the color temperature of the light source changes. When calculating the luminance Y from the video signal RGB, the luminance composition ratio is calculated as a ratio of R: G: B = 3: 6: 1. If the color temperature of the light source is not corrected correctly, an accurate luminance is obtained. I can't ask for it. If the light source color is not corrected correctly, the brightness will be greatly different between a low color temperature light source such as a light bulb and a high color temperature light source such as strobe light. Is a general shooting scene. Therefore, optimum brightness correction under a plurality of light sources has been required.

  SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to more accurately correct a luminance component accompanying a change in color temperature between photometry for exposure control and photographing.

  The image processing apparatus according to the present invention calculates a first white balance correction value based on image data captured when the flash is fired, and a second white balance correction value based on image data captured when the flash is not flashing. Based on the first white balance correction value and the correction value calculation means for calculating the first image data from the image data photographed during the flash emission, and the second white balance correction value. A developing means for developing the second image data from the image data photographed when the flash is not emitted, the luminance value of the image data photographed when the flash is not emitted, and the image data captured when the flash is emitted. Component calculating means for calculating a luminance component of external light and a luminance component of strobe light based on the luminance value, and the luminance of the external light A composite ratio calculating means for calculating a composite ratio between the first image data and the second image data based on the minute and the luminance component of the strobe light; and the composite calculated by the composite ratio calculating means Based on the ratio, for correcting the light source color of the shooting environment at the time of photometry by the mixing means for mixing the first white balance correction value and the second white balance correction value and the photometry means for exposure control Correction coefficient calculation means for calculating a color temperature correction coefficient based on the white balance correction value used and the white balance correction value generated by the mixing means, and the composition ratio calculated by the composition ratio calculation means The first image data and the second image data are combined to generate third image data, and the correction coefficient calculation unit Using the calculated the color temperature correction coefficient, and having a correction means for correcting the luminance component of the third image data generated by the synthesizing means.

  According to the present invention, it is possible to more accurately correct the luminance component accompanying the variation in the color temperature between the photometry for exposure control and the shooting.

It is a figure which shows the structure of the imaging device which concerns on embodiment of this invention. It is a flowchart which shows the calculation process of the 1st WB correction value at the time of strobe light emission. It is a figure which shows the graph used in the case of white detection. It is the figure which showed photographing control side by side in time series. It is a figure which shows the WB correction value in still image photography drive mode, and the WB correction value in EVF photography drive mode. It is a figure for demonstrating the method to calculate a 2nd WB correction value so that a color may be left in consideration of the atmosphere of a scene. It is a flowchart which shows the synthetic | combination process of image data. It is a figure for demonstrating extraction of the to-be-photographed area | region for calculating an irradiation ratio. It is a figure which shows the relationship between color temperature and a brightness | luminance.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments to which the invention is applied will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a diagram illustrating a configuration of an imaging apparatus according to an embodiment of the present invention. Note that the imaging apparatus according to the present embodiment has a configuration as an application example of the image processing apparatus. In FIG. 1, reference numeral 101 denotes a solid-state imaging device composed of a CCD, a CMOS, or the like, and its surface is covered with, for example, an RGB color filter such as a Bayer array so that color photography can be performed.

  The CPU 114 calculates a shutter speed and an aperture value that brighten the entire image data, and calculates a driving amount of the focus lens so as to focus on a subject in the focus area. The exposure value (shutter speed, aperture value) and the driving amount of the focus lens calculated by the CPU 114 are output to the control circuit 113 and controlled based on each value. A white balance (hereinafter referred to as WB) control circuit 103 calculates a WB correction value based on the image data stored in the memory 102, and uses the calculated WB correction value to convert the image data stored in the memory 102 into image data. On the other hand, WB correction is performed. The details of the WB correction value calculation method by the WB control circuit 103 will be described later.

  A color matrix circuit 104 converts the color difference signals RY and BY by applying a color gain so that the image data corrected by the WB by the WB control circuit 103 is reproduced with an optimum color. Reference numeral 105 denotes a low-pass filter (LPF) circuit that limits the bands of the color difference signals RY and BY. Reference numeral 106 denotes a CSUP (Chroma Supress) circuit that suppresses a false color signal in a saturated portion of the image data band-limited by the LPF circuit 105. On the other hand, the image data subjected to the WB correction by the WB control circuit 103 is also output to the luminance signal (Y) generation circuit 111 to generate the luminance signal Y, and the edge enhancement circuit 112 outputs the generated luminance signal Y to the luminance signal Y. Apply edge enhancement.

  The color difference signals RY and BY output from the CSUP circuit 106 and the luminance signal Y output from the edge enhancement circuit 112 are converted into RGB signals by the RGB conversion circuit 107, and a gamma (γ) correction circuit. At 108, gradation correction is performed. The RGB signal subjected to the gradation correction is converted into a YUV signal by the color luminance conversion circuit 109, further compressed by the JPEG compression circuit 110, and recorded as image data on an external recording medium or an internal recording medium.

  Next, a method for calculating the WB correction value in the WB control circuit 103 will be described in detail. First, a calculation process of a first WB correction value (first white balance correction value) that is a WB correction value at the time of main exposure (during strobe light emission) will be described with reference to the flowchart of FIG. Note that the first WB correction value is a correction value calculated based on image data captured during flash emission (hereinafter, image data during flash emission).

In step S <b> 201, the WB control circuit 103 reads the strobe flashing image data stored in the memory 102 and divides the strobe flashing image data into arbitrary m blocks. In step S202, the WB control circuit 103 calculates a color average value (R [i], G [i], B [i]) by averaging pixel values for each color for each block (1 to m). Then, a color evaluation value (Cx [i], Cy [i]) is calculated using the following formula.
Cx [i] = (R [i] −B [i]) / Y [i] × 1024
Cy [i] = (R [i] + B [i] -2G [i]) / Y [i] × 1024
However, Y [i] = (R [i] + 2G [i] + B [i]) / 4

  The WB control circuit 103 performs white detection using a graph having coordinate axes as shown in FIG. The negative direction of the x coordinate (Cx) represents a color evaluation value when white of a high color temperature subject is photographed, and the positive direction represents a color evaluation value when a color of a low color temperature subject is photographed. The y-coordinate (Cy) means the degree of the green component of the light source, and the green component increases as it goes in the negative direction, that is, it indicates a fluorescent lamp.

  In step S203, the WB control circuit 103 includes the color evaluation values (Cx [i], Cy [i]) of the i-th block calculated in step S202 in the preset white detection range 301 shown in FIG. It is determined whether or not. The white detection range 301 is limited because the strobe light is a known light source. When the color evaluation value (Cx [i], Cy [i]) of the i-th block is included in the preset white detection range 301, the process proceeds to step S204. On the other hand, when the color evaluation value (Cx [i], Cy [i]) of the i-th block is not included in the preset white detection range 301, the process skips step S204 and proceeds to step S205. .

  In step S204, the WB control circuit 103 determines that the i-th block is white, and integrates the color average values (R [i], G [i], B [i]) of the block. Note that the processing of steps S203 and S204 can be expressed by the following equation.

  Here, in the above formula, when the color evaluation value (Cx [i], Cy [i]) is included in the white detection range 301, Sw [i] is set to 1, and the color evaluation value (Cx [i], Cy If [i]) is not included in the white detection range 301, Sw [i] is set to 0. Thereby, in steps S203 and S204, the color average values (R [i], G [i], B [i]) are substantially integrated or not integrated.

  In step S205, the WB control circuit 103 determines whether the above processing has been performed for all blocks. If there is an unprocessed block, the process returns to step S202 to repeat the above formula. On the other hand, when the process has been performed for all blocks, the process proceeds to step S206.

In step S206, the WB control circuit 103 calculates the first WB correction values (WBCol_R1, WBCol_G1, WBCol_B1) from the obtained integrated values (SumR1, SumG1, SumB1) of the average color values using the following equation. To do.
WBCol_R1 = SumY1 × 1024 / SumR1
WBCol_G1 = SumY1 × 1024 / SumG1
WBCol_B1 = SumY1 × 1024 / SumB1
However, SumY1 = (SumR1 + 2 × SumG1 + SumB1) / 4
The strobe light WB correction value may be set in advance as a known WB correction value.

  Next, a calculation process of a second WB correction value (second white balance correction value) that is a WB correction value when the flash is not emitted will be described. Note that the second WB correction value is a correction value calculated based on image data captured when the flash is not flashing (hereinafter, image data when the flash is not flashing). FIG. 4 is a diagram showing the imaging control arranged in time series. As shown in FIG. 4, before SW1 is pressed, live view image data is periodically taken, and when SW1 is pressed, AF lock and AE lock are performed. After pressing SW2, test light emission and main exposure are performed, but the image data exposed and photographed at the time indicated as “external light” before the test light emission, here, image data when no flash is emitted And As the image data when the flash is not emitted, captured image data that has been exposed and photographed after the main exposure may be used.

  The calculation process of the second WB correction value is performed by the same method as the calculation process of the first WB correction value described above. However, the calculation processing of the first WB correction value is different in that the processing is performed in a white detection range for external light as indicated by 302 in FIG. This is because strobe light is a known light source, whereas external light is not a known light source, and thus cannot be limited as in the white detection range 301 during strobe light emission. The white detection range 302 for external light is obtained by photographing white under a different light source in advance and plotting the calculated color evaluation values. The white detection range 302 can be set separately depending on the shooting drive mode. The second WB correction value can be applied even when the imaging drive mode is different. For example, the WB correction value of the EVF (Electronic View Finder) imaging drive mode calculated in the past can be used. However, when a spectral difference occurs due to the photographing drive mode, the WB correction circuit 103 corrects the WB correction value. FIG. 5 is a diagram illustrating a WB correction value in the still image shooting driving mode and a WB correction value in the EVF shooting driving mode. When calculating the WB correction value of the still image shooting drive mode using the WB correction value of the EVF shooting drive mode shot in the past, as shown in FIG. 5, the WB control circuit 103 uses the WB of the EVF shooting drive mode. Correction values (EvfCx, EvfCy) 501 are corrected by ΔCx and ΔCy. Thereby, the WB correction value (CapCx, CapCy) 502 in the still image shooting driving mode is calculated.

  Here, when the background is a low color temperature light source such as a light bulb color light source, the second WB correction value may be calculated so as to leave the color in consideration of the atmosphere of the scene. For example, as shown in FIG. 6, when the color temperature of the second WB correction value is low (for example, input T1), by controlling to leave the color (for example, output T'1), the light bulb color Image data that retains the redness of the light source can be generated. Note that the above-described calculation processing of the first WB correction value and the second WB correction value is a processing example of the correction value calculation means.

  Next, image data combining processing will be described with reference to the flowchart of FIG. In step S700, the WB control circuit 103 develops the image data Yuv1 from the image data during strobe light emission using the first WB correction value. In step S701, the WB control circuit 103 develops the image data Yuv2 from the non-flash image data using the second WB correction value. In step S702, the WB control circuit 103 divides the image data at the time of flash emission, the image data at the time of non-flash emission, image data Yuv1, and image data Yuv2 into n blocks. Note that the image data Yuv1 is an example of first image data, and the image data Yuv2 is an example of second image data.

  In step S703, the WB control circuit 103 adds and averages the pixel values for each color for each block (1 to n) of the image data when the flash is not emitted, and calculates the color average values (R2 [i], G2 [i], B2 [i]) is calculated, and the luminance value a [i] of each block is calculated. The formula for calculating the luminance value a [i] of each block is as shown in the following formula. Here, the calculated luminance value a [i] of each block is used as a luminance component of external light (hereinafter referred to as external light component) of each block.

In step S <b> 704, the WB control circuit 103 performs pixel processing for each block (1 to n) of image data during strobe light emission, similarly to the process of calculating the luminance value a [i] of each block from the image data during strobe non-light emission. The color average values (R1 [i], G1 [i], B1 [i]) are calculated by averaging the values for each color, and the luminance value b [i] of each block is calculated. As shown in the following equation, the WB control circuit 103 subtracts the luminance value a [i] of each block of the corresponding strobe non-flash image data from the calculated luminance value b [i] of each block. The luminance component (hereinafter, strobe component) c [i] of the strobe light for each block is calculated.
c [i] = b [i] -a [i]
Steps S703 and S704 are processing examples of the component calculation unit.

In step S705, the WB control circuit 103 calculates the image data Yuv1 and the image data Yuv2 based on the ratio of the strobe component c [i] and the external light component a [i] for each corresponding block according to the following equations. The mix rate α [i] is calculated for each block when combining.
α [i] = c [i] / (a [i] + c [i])
Note that step S705 is a processing example of the composition ratio calculation unit.

  In step S706, the WB control circuit 103 obtains the mix rate Sumα of the entire screen, which is the sum of the mix rate α [i] for each block obtained in step S705, by the following equation.

  In step S707, the WB control circuit 103 mixes the first WB correction value and the second WB correction value according to the value of Sumα. Here, Sumα is the sum of the Mix rate for each block, and the Mix rate α [i] indicates the ratio of the strobe component. Therefore, by calculating Sumα, the strobe component and the external light component occupy the entire screen. A ratio can be obtained. The WB control circuit 103 determines the WB correction value for the entire screen by mixing the first WB correction value and the second WB correction value using the ratio of the strobe component and the external light component as a mixing ratio. To do. The WB control circuit 103 stores the WB correction value obtained here as a post-Mix WB correction value.

  As another method, a method for obtaining a post-Mix WB correction value without using Sumα will be described. The WB control circuit 103 applies the white detection range indicated by the white detection range 302 in FIG. 3 to the subject area, and detects the color temperature of the subject area. The WB control circuit 103 uses the color temperature of the subject area as a post-Mix WB correction value.

In step S708, the WB control circuit 103 calculates a color temperature correction coefficient. In AE, exposure control is performed so that the subject brightness is measured and an appropriate exposure is obtained. In order to measure the subject brightness correctly, it is necessary to correct the color of the light source. However, since the exposure is determined before photographing and the color temperature of the light source is determined after photographing, the color temperature of the photographed image data cannot be obtained at the time of photometry by AE. Therefore, in photometry, exposure is controlled by calculating brightness using the WB coefficient of the color temperature of sunlight. After that, the WB control circuit 103 accurately obtains the color temperature of the light source after still image shooting, and calculates the difference from the WB coefficient of sunlight for correcting the light source color of the shooting environment temporarily used during metering, as a luminance correction coefficient. It is calculated as Color temperature correction refers to luminance correction due to variation in color temperature, and color temperature correction coefficient refers to a luminance correction coefficient due to variation in color temperature. The WB control circuit 103 calculates the color temperature correction coefficient k by the following equation based on the post-Mix WB correction value obtained in step S707 and the sunlight WB correction value used during photometry.
Color temperature correction coefficient k = After-mix WB correction value / sunlight WB correction value Note that step S708 is a processing example of the correction coefficient calculation means.

  In step S709, the WB control circuit 103 synthesizes the image data Yuv1 and the image data Yuv2 using the mix rate α [i] for each block, and generates composite image data Yuv3. The color evaluation values (Y3 [i], u3 [i], v3 [i]) in the composite image data Yuv3 are the same as the color evaluation values (Y1 [i], u1 [i], v1 [i]) in the image data Yuv1. The color evaluation values (Y2 [i], u2 [i], v2 [i]) in the image data Yuv2 are used to calculate the following equation.

  Here, in order to alleviate the color shift at the block boundary portion, the pixel interpolation process is performed in step S705 to calculate the pixel-to-pixel Mix rate α ′ [j] from the block-specific Mix rate α [i]. Good. For example, the WB control circuit 103 uses bilinear interpolation as the pixel interpolation process, and calculates the Mix rate α ′ [j] for each pixel from the Mix rate α [i] for each block. At this time, in step S709, the WB control circuit 103 synthesizes the image data Yuv1 and the image data Yuv2 using the mix rate α ′ [j] for each pixel, and generates composite image data Yuv3. Note that the composite image data Yuv3 is an example of third image data.

  The color evaluation values (Y3 [j], u3 [j], v3 [j]) in the composite image data Yuv3 are the same as the color evaluation values (Y1 [j], u1 [j], v1 [j]) in the image data Yuv1. The color evaluation values (Y2 [j], u2 [j], v2 [j]) in the image data Yuv2 are used to calculate the following equation.

  Further, in step S709, the processing may be made variable by determining whether or not to perform the image composition processing for each block according to the luminance of each block in the image data at the time of flash emission. That is, when the brightness of the block is low or high, the image data is developed using a WB correction value calculated by normal white balance control, which will be described later, without performing synthesis processing for the block. On the other hand, when the luminance of the block is other than that, the same processing as the above-described image data synthesis processing is performed.

  Furthermore, in step S709, the WB control circuit 103 performs luminance correction by applying the color temperature correction coefficient k calculated in step S708 only to the luminance component Y3 as shown in the following equation.

  Next, a normal white balance control method will be described in detail. First, the WB control circuit 103 calculates the first and second WB correction values in the same manner as the WB correction value calculation process described above. Next, the WB control circuit 103 performs Mix (weighted addition) processing of the first and second WB correction values. In the Mix (weighted addition) process, weighted addition is performed according to the irradiation ratio of the external light and strobe light applied to the subject.

  Here, with reference to FIG. 8, the extraction of the subject area for calculating the irradiation ratio will be described. The WB control circuit 103 acquires image data 801 photographed before the test light emission, and calculates a luminance block a 802 composed of an m × n matrix from the image data 801 photographed before the test light emission. Next, by performing the test light emission of the strobe under the same conditions as before the test light emission, the WB control circuit 103 acquires the image data 803 photographed at the time of the test light emission, and is similarly configured by an m × n matrix. Luminance block b804 to be acquired. The luminance block a802 and the luminance block b804 are temporarily stored in a RAM or the like provided in the imaging device. It is assumed that the background image of the image data 801 photographed before the test light emission and the background image of the image data 803 photographed during the test light emission are not substantially changed. Therefore, the difference data between the two luminance blocks a802 and b804 becomes reflected light of the subject area during the strobe test light emission, and the subject position information c805 can be acquired from the difference data.

  The WB control circuit 103 acquires a subject position block having a value of 1 from the subject position information c805 calculated in this way, and the brightness value Y1 when the strobe light is emitted and the brightness value Y2 when the strobe light is not emitted in the obtained subject position block. And calculate. Here, when the exposure conditions differ between strobe light emission and strobe non-light emission, the WB control circuit 103 calculates Y1 and Y2 by aligning the exposure conditions. The WB control circuit 103 uses the first WB correction value and the second WB correction value as the light quantity ratio with which the ratio of Y1 and Y2 calculated in this way is applied to the subject, and mix processing based on the light quantity ratio (weighted addition) )I do. Then, the WB control circuit 103 determines the WB correction value obtained by the Mix processing as the WB correction value used for the WB processing, develops the image data, and sets it as final image data.

  In the present embodiment, the color is corrected for each block, and the correction of the luminance component is applied to the entire screen. However, the present invention is not limited to this. For example, the first WB correction value and the second WB correction value are mixed using the Mix rate α [i] for each block, and the post-Mix WB correction value is obtained for each block, thereby obtaining a color for each block. It is possible to obtain the temperature correction coefficient k.

  According to the present embodiment, when the subject and the background are irradiated with different light sources, when the subject and the background have both proper colors, the color at the time of metering for exposure control and the time of shooting It becomes possible to correct the luminance signal accompanying the variation in temperature more accurately.

  The present invention can be applied to an apparatus (for example, a digital still camera, a digital video camera, etc.) composed of a single device even if a system composed of a plurality of devices (for example, a host computer, an interface device, a camera head, etc.) is applied. May be.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

  101: Individual imaging device, 102: Memory, 103: WB control circuit, 104: Color matrix circuit, 105: LPF circuit, 106: CSUP circuit, 107: RGB conversion circuit, 108: γ correction circuit, 109: Color luminance conversion circuit 110: JPEG compression circuit, 111: Y generation circuit, 112: Edge enhancement circuit, 113: Control circuit, 114: CPU

Claims (10)

Correction value calculating means for calculating a first white balance correction value based on image data shot when the flash is fired and calculating a second white balance correction value based on image data shot when the flash is not flashing; ,
Based on the first white balance correction value, the first image data is developed from the image data photographed when the flash is fired, and is photographed when the flash is not fired based on the second white balance correction value. Developing means for developing second image data from the processed image data;
Component calculating means for calculating a luminance component of external light and a luminance component of strobe light based on a luminance value of image data shot when the flash is not emitted and a luminance value of image data shot when the flash is emitted; ,
A composition ratio calculating means for calculating a composition ratio between the first image data and the second image data based on the luminance component of the external light and the luminance component of the strobe light;
A mixing unit that mixes the first white balance correction value and the second white balance correction value based on the combination ratio calculated by the combination ratio calculation unit;
Based on the white balance correction value used to correct the light source color of the shooting environment during metering by the metering means for exposure control, and the white balance correction value generated by the mixing means, the color temperature correction coefficient is calculated. Correction coefficient calculation means for calculating,
Synthesizing means for synthesizing the first image data and the second image data according to the synthesis ratio calculated by the synthesis ratio calculating means to generate third image data;
An image processing apparatus comprising: a correction unit that corrects a luminance component of the third image data generated by the synthesis unit using the color temperature correction coefficient calculated by the correction coefficient calculation unit. .   The synthesis ratio calculation means calculates a synthesis ratio of the first image data and the second image data for each block based on the luminance component of the external light and the luminance component of the strobe light for each block. The image processing apparatus according to claim 1, wherein:   The image processing apparatus according to claim 2, wherein the combination ratio calculation unit calculates a combination ratio for each pixel from a combination ratio for each block by pixel interpolation processing.   4. The image processing apparatus according to claim 1, wherein the synthesis unit controls whether or not to perform synthesis processing of a corresponding block according to a luminance value of each block. 5. .   5. The image processing according to claim 1, wherein the correction value calculation unit calculates the second white balance correction value so as to leave an arbitrary color for a light source. apparatus. First mixing ratio calculating means for calculating a mixing ratio of the first white balance correction value and the second white balance correction value based on the combining ratio calculated by the combining ratio calculating means. And
The mixing means mixes the first white balance correction value and the second white balance correction value according to the mixing ratio calculated by the first mixing ratio calculation means. The image processing apparatus according to any one of 1 to 5.   The image processing apparatus according to claim 6, wherein the first mixing ratio calculation unit calculates the mixing ratio by adding the synthesis ratio for each block for the entire screen. The detected color temperature is used as a substitute for the white balance correction value generated by the mixing unit by detecting the color temperature of the subject region. Image processing apparatus. An image processing method executed by an image processing apparatus,
A correction value calculating step of calculating a first white balance correction value based on image data photographed when the flash is emitted and calculating a second white balance correction value based on image data photographed when the flash is not emitted; ,
Based on the first white balance correction value, the first image data is developed from the image data photographed when the flash is fired, and is photographed when the flash is not fired based on the second white balance correction value. A developing step for developing second image data from the processed image data;
A component calculation step of calculating a luminance component of external light and a luminance component of strobe light based on a luminance value of image data shot when the flash is not emitted and a luminance value of image data shot when the flash is emitted; ,
A synthesis ratio calculating step of calculating a synthesis ratio of the first image data and the second image data based on the luminance component of the external light and the luminance component of the strobe light;
A mixing step of mixing the first white balance correction value and the second white balance correction value based on the combination ratio calculated by the combination ratio calculation step;
Based on the white balance correction value used for correcting the light source color of the shooting environment during metering by the metering means for exposure control, and the white balance correction value generated by the mixing step, the color temperature correction coefficient is calculated. A correction coefficient calculation step for calculating,
Combining the first image data and the second image data according to the composition ratio calculated in the composition ratio calculating step to generate third image data;
A correction step of correcting a luminance component of the third image data generated by the synthesis step using the color temperature correction coefficient calculated by the correction coefficient calculation step. . A correction value calculating step of calculating a first white balance correction value based on image data shot when the flash is fired and calculating a second white balance correction value based on image data shot when the flash is not flashed; ,
Based on the first white balance correction value, the first image data is developed from the image data photographed when the flash is fired, and is photographed when the flash is not fired based on the second white balance correction value. A developing step for developing second image data from the processed image data;
A component calculation step of calculating a luminance component of external light and a luminance component of strobe light based on a luminance value of image data shot when the flash is not emitted and a luminance value of image data shot when the flash is emitted; ,
A synthesis ratio calculating step of calculating a synthesis ratio of the first image data and the second image data based on the luminance component of the external light and the luminance component of the strobe light;
A mixing step of mixing the first white balance correction value and the second white balance correction value based on the combination ratio calculated by the combination ratio calculation step;
Based on the white balance correction value used for correcting the light source color of the shooting environment during metering by the metering means for exposure control, and the white balance correction value generated by the mixing step, the color temperature correction coefficient is calculated. A correction coefficient calculation step for calculating,
Combining the first image data and the second image data according to the composition ratio calculated in the composition ratio calculating step to generate third image data;
A program for causing a computer to execute a correction step of correcting a luminance component of the third image data generated in the synthesis step using the color temperature correction coefficient calculated in the correction coefficient calculation step.

Images