JP5932392B2 - Image processing apparatus and image processing method - Google Patents

Description

The present invention relates to an image processing apparatus and an image processing method, and more particularly to a technique for adjusting the brightness of a composite image.

  In an imaging device using an image sensor, by combining multiple images of the same scene taken at different exposures, an image with reduced white stripes and blackout can be obtained even in a scene having a luminance distribution wider than the dynamic range of the image sensor. Can be obtained. Such a photographing method is called HDR (High Dynamic Range) photographing, and an obtained image is called an HDR image.

  It is also known to analyze the luminance histogram of an image and subject information such as face information and perform tone correction so as to obtain an image with favorable brightness and contrast.

  For example, a so-called backlight scene in which the background is significantly brighter than the main subject is a typical example of a scene with a wide dynamic range. In a captured image in a backlight scene, a shadowed main subject portion is darkened, and a phenomenon in which the luminance value of the background is saturated and gradation is lost (referred to as white stripes) easily occurs. Therefore, HDR shooting and gradation correction for increasing the luminance of the main subject portion in the shot image are effective.

There has also been proposed a technique for correcting the main subject and the background to appropriate brightness by combining HDR shooting and gradation correction (Patent Document 1).
As a gradation correction method, a method using a correction table (tone curve) for converting each input luminance value into a predetermined output luminance value is well known. However, if tone correction is performed using a tone curve that increases the luminance of a dark region and decreases the luminance of a bright region in order to suppress blackout and white highlight, the contrast of the image is decreased.

  On the other hand, a method has been proposed in which a gradation correction is performed using a low-frequency component signal of a luminance component of an image to obtain an effect like so-called dodging (Patent Document 2). In this method, tone correction can be performed while suppressing a decrease in contrast of the image as compared with a method using a tone curve.

Japanese Patent No. 3424576 JP 2008-072604 A

  However, even if the dynamic range of an image is expanded by HDR shooting or tone correction using a tone curve, the dynamic range of an output device such as a display or a printer is fixed. Therefore, after all, it is necessary to compress to the dynamic range of the output device, and the image tends to have a reduced contrast.

  In addition, gradation correction using a low-frequency luminance component as described in Patent Document 2, for example, may be applied to a bright part, resulting in an unnatural image due to excessive contrast.

The present invention has been made in view of such a problem of the prior art. The present invention provides an image processing apparatus and an image processing method capable of performing gradation correction on an HDR image while suppressing a decrease in contrast and preventing an image after gradation correction from becoming unnatural. Objective.

The foregoing objects, an acquisition unit configured to acquire a plurality of images captured by a plurality of different exposure amounts, a plurality of images, by applying a gamma conversion processing for reducing the contrast as the exposure amount difference between the images is greater synthesizing means for generating a composite image by synthesizing from, based on the luminance information of the combined image, the target correction amount calculating means for calculating a target correction amount of gradation correction for the combined image, in order to achieve the target correction amount A correction amount determination means for determining a first correction amount of gradation correction using a luminance low-frequency component of a composite image and a second correction amount of gradation correction using a tone curve; A first gradation correction unit that applies gradation correction processing to the composite image based on the correction amount, and a second that applies gradation correction processing to the composite image based on the second correction amount. Gradation correction means, and the correction amount determination means The correction amount, after determined to be larger as the exposure amount difference between images is large, the image processing apparatus characterized by determining by correcting such brightness of the main subject of the composite image is decreased with increasing Achieved by:

  With such a configuration, according to the present invention, it is possible to perform gradation correction on an HDR image while suppressing a decrease in contrast and preventing an image after gradation correction from becoming unnatural.

1 is a block diagram illustrating a configuration of an imaging apparatus 100 that is an example of an image processing apparatus according to a first embodiment of the present invention. The flowchart for demonstrating the process of the exposure amount calculation part of FIG. The flowchart for demonstrating the process of the gradation correction amount calculating part of FIG. The figure which shows the example of the face luminance acquisition area in embodiment of this invention The figure for demonstrating the example of the production | generation operation | movement of the gradation correction characteristic in the correction amount calculation part in embodiment of this invention when a face is detected The figure for demonstrating the example of the production | generation operation | movement of the gradation correction characteristic in the correction amount calculation part in embodiment of this invention when a face is not detected The flowchart for demonstrating the correction amount calculation process in the gradation correction amount calculating part of FIG. The figure which shows the example of the correction amount of the correction | amendment using the luminance low frequency component in the embodiment of this invention, and the example of the difference in the correction amount by exposure amount difference The figure which shows the example of the relationship between the intensity | strength of the correction | amendment using a brightness | luminance low frequency component, and the representative brightness value of a face in embodiment of this invention. The flowchart for demonstrating the process of the gradation correction | amendment part of FIG. The figure which shows the example of the distribution method of the target correction amount in the 2nd Embodiment of this invention.

(First embodiment)
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.
FIG. 1 is a block diagram showing a configuration of an imaging apparatus 100 that is an example of an image processing apparatus according to the first embodiment of the present invention. The imaging apparatus 100 includes an apparatus that captures a subject and obtains image data, such as a digital camera or a digital video camera. In the present invention, it is sufficient if a plurality of images photographed with a predetermined exposure amount difference can be acquired, and a configuration relating to photographing is not essential.

  The operation of the imaging apparatus 100 is controlled by the control unit 102. In addition, when an external operation such as a user instruction is input from the interface (I / F) 101, the control unit 102 controls the operation of the imaging apparatus 100 according to the input.

  A subject optical image that has passed through an imaging lens (not shown) is formed on the photometric sensor 103. The photometric sensor 103 has a photometric area divided into a plurality of parts, and measures the subject brightness of each photometric area. The photometric sensor 103 may be a photoelectric conversion element having the same configuration as an imaging element 105 (described later) that captures an image for display or recording, except that the number of pixels is small. In this case, each photometric area may include a plurality of pixels.

  The exposure amount calculation unit 104 determines a plurality of exposure amounts to be used in high dynamic range (HDR) shooting from the subject luminance of each photometric area measured by the photometric sensor 103. In the present embodiment, one HDR image is generated by synthesizing three photographed images obtained by performing photographing with different exposure amounts three times. As for the exposure amount, first, one exposure amount is determined, and an exposure amount for photographing the subject brighter than this exposure amount and an exposure amount for photographing the subject darker are determined. For the sake of convenience, the exposure amount determined first is referred to as a reference exposure amount or an intermediate exposure amount. Also, the exposure amount for photographing the subject brighter than the intermediate exposure amount is called exposure amount (+), and the exposure amount for photographing the subject darker than the intermediate exposure amount is called exposure amount (−).

The operation of the exposure amount calculation unit 104 will be described with reference to the flowchart of FIG.
In S201, the exposure amount calculation unit 104 calculates an intermediate exposure amount. Specifically, the exposure amount calculation unit 104 generates a photometric value histogram based on the subject luminance (Bv value) of each photometric area acquired from the photometric sensor 103. Then, the exposure amount calculation unit 104 calculates an appropriate exposure amount using a histogram and sets it as an intermediate exposure amount (Ev value). The intermediate exposure amount may be specified by the user through the interface 101, but here, an appropriate exposure amount is determined by a known exposure amount calculation method (center-weighted metering, spot metering, multi-segment metering, etc.) based on the metering value. Calculate and use as intermediate exposure.

In Expression (1), subject brightness (Bv value), exposure amount (Ev value), exposure time (Tv value) in which incident light is accumulated in the image sensor, amount of incident light passing through the lens (Av value), and image sensor The relationship of the gain amount (Sv value) of the signal obtained from (1) is shown.
Ev = Sv + Bv = Tv + Av (1)

Note that the addition here is performed by converting subject brightness, exposure amount, exposure time, and gain amount into APEX (Additive System of Photographic Exposure) values. For details of the APEX value, refer to, for example, “Appendix C APEX” of “Image File Format Standard (Exif) Version 2.1 for Digital Still Camera”. In recent years, the Ev value (= Ev ₁₀₀ ) in the case where the Sv value is 5 (= ISO ₁₀₀ ) is often simply handled as the subject brightness. Therefore, Ev ₁₀₀ is also used in this embodiment to represent the subject brightness.

  In S202, the exposure amount calculation unit 104 calculates the luminance range of the subject. The exposure amount calculation unit 104 can calculate, for example, the difference between the maximum (brightest) photometric value and the minimum (darkest) photometric value in the histogram of photometric values as the luminance range of the subject. However, when considering the influence of isolated brightness values such as point light sources, the average of multiple photometric values in order from the maximum (lightest) photometric value and the multiple photometric values in sequence from the minimum (darkest) photometric value The difference from the averaged value may be calculated as the luminance range.

  In S203, the exposure amount calculation unit 104 uses the calculated luminance range to calculate at least one of the difference between the intermediate exposure amount and the exposure amount (+) and the difference between the intermediate exposure amount and the exposure amount (−). Determine as the difference.

  The exposure amount difference can be determined so as to increase as the luminance range increases. For example, it can be determined using a table that is prepared in advance and associates a luminance range with an exposure amount difference. Further, instead of using a table, a function having the same characteristics and having a luminance range as an argument may be defined in advance, and the exposure amount difference may be calculated according to the luminance range.

The relationship between the brightness range and the exposure amount difference is not limited to one. For example, in order to eliminate overexposure and underexposure due to insufficient dynamic range of the image sensor, a larger exposure amount difference is calculated, or a smaller exposure amount difference is calculated for the purpose of maintaining image contrast. It is good also as such a relationship. Further, the exposure amount difference may be designated by the user from the interface 101. Note that the exposure amount difference may be different between the exposure amount (+) and the exposure amount (−).
In this way, the exposure amount difference between the intermediate exposure amount and the exposure amount (+) and the exposure amount (−) is determined.

  Note that the setting of the exposure conditions for HDR shooting described here is performed in response to a shooting preparation instruction such as half-pressing the shutter button included in the interface 101. When the shutter button is changed from the half-pressed state to the fully-pressed state and a shooting start instruction is given, shooting is performed a plurality of times (here, three times) according to each shooting condition with a plurality of set exposure amounts. Here, for convenience, an image shot with an intermediate exposure amount is called an appropriate image, an image shot with an exposure amount (+) is called an over image, and an image shot with an exposure amount (−) is called an under image. To do.

  The setting of the exposure condition is not essential, and a plurality of images taken with a predetermined exposure amount difference may be acquired and the exposure amount difference may be detected from the images. The difference in exposure amount can be detected based on, for example, the photographing condition recorded in the attached data of the image (accompanying information such as Exif data). Further, since the image related to the composition is basically an image of the same scene, the exposure amount difference can also be detected by comparing the luminance of the images.

  Please refer to FIG. 1 again. In each photographing, the image sensor 105 is exposed by an object optical image that has passed through an imaging lens (not shown), and is converted into a charge corresponding to the amount of light received by each pixel of the image sensor 105. For example, the image sensor 105 includes a color filter in which R (red), G (green), and B (blue) regions are arranged. Each color region has spectral sensitivity around the wavelength band of each color, and a pixel (photoelectric conversion element) provided corresponding to each region photoelectrically converts light in the band that has passed through each color filter.

  The electric charge converted by each photoelectric conversion element is output as an electric signal from the imaging element 105 to the A / D conversion unit 106 and converted into a digital signal (image data) by A / D conversion processing. The digital signal output from the A / D conversion unit 106 is sent to the development processing unit 107.

  The development processing unit 107 performs development processing for each exposure amount image, and generates an intermediate image suitable for the HDR image. Here, known white balance processing, noise reduction processing, gamma conversion processing, and the like are performed. These processes may be performed by known methods, but the gamma conversion process is performed using a gamma curve having a shape corresponding to the exposure amount (for example, a gamma conversion table corresponding to the exposure amount is prepared). This is to improve the gradation of the dark part and the bright part by HDR imaging. Specifically, the gamma curve applied to the over image has a shape that makes the dark portion brighter, and the gamma curve applied to the under image has a shape that suppresses white highlight in the bright portion. Further, in order to produce an effect of extending the dynamic range as the exposure amount difference between the under image and the over image increases, the dark part is brightened and the bright part is darkened. That is, the gamma curve has a shape in which the contrast is reduced as the exposure amount difference between a plurality of images to be combined is increased.

  The image synthesizing unit 108 synthesizes a plurality of intermediate images generated by a plurality of shootings to generate an HDR image. For example, the image combining unit 108 combines the under image and the over image into the appropriate image so that the under image is used for the bright area of the appropriate image and the over image is used for the dark area of the appropriate image, and the HDR image is generated. Generate.

  The gradation correction amount calculation unit 109 performs gradation correction on the composite image. The gradation correction amount calculation unit 109 includes a luminance low frequency component detection unit 110, a histogram detection unit 111, a face detection unit 112, a correction amount calculation unit 113, and a gradation correction unit 114.

The tone correction processing performed by the tone correction amount calculation unit 109 will be described with reference to the flowchart of FIG.
In S301, the luminance low-frequency component detection unit 110 extracts a luminance component from the pixel value of the composite image and performs low-pass filter processing. In the present embodiment, three low-frequency component images are generated using three spatial low-pass filters having different filter sizes. Here, the sizes of the original image and the low-frequency component image do not have to match. For example, when extracting the luminance component, the input image data may be reduced. The reduction process may be any known method, but can be easily realized by, for example, a thinning process, a nearest neighbor method, an average value method, or the like.

In S302, the face detection unit 112 performs face detection preprocessing. This is a process that makes it easy to detect a face by performing reduction processing, gamma processing, or the like on the image data.
In S303, the face detection unit 112 performs face detection processing on the preprocessed image data using a known method. By the face detection process, an area that is considered to be a face (face area) and detection reliability are obtained.

  In S304, the face detection unit 112 determines that a face has been detected when there is a face area whose reliability is higher than a preset evaluation threshold among the face areas detected by the face detection process in S303. If it is determined that a face has been detected, the process proceeds to S305. If it is determined that no face has been detected, the process proceeds to S307.

  In step S <b> 305, the face detection unit 112 calculates a part of the face area whose reliability is higher than the evaluation threshold as a face luminance acquisition area. For example, as shown in FIG. 4, the face detection unit 112 is located at three locations in the face area 402 detected from the image data 401, the areas 403 and 405 below both eyes and the intermediate area 404. A square area having the same size is calculated as the face luminance acquisition area. The size of each area is determined by the size of the face area. The face luminance acquisition area is an area for acquiring the luminance of the bright part of the face, and there is no particular limitation on the number and position thereof.

  In step S306, the face detection unit 112 calculates a representative luminance value of the detected face. Here, for each face area whose reliability is higher than the evaluation threshold, the average brightness value of the face brightness acquisition areas 403 to 405 is calculated from the synthesized image, and the maximum value is set as the representative brightness value. Here, the reason why the maximum value of luminance is used is that importance is attached to prevention of overcorrection of correction using a low-frequency luminance component or a tone curve.

  If the face detection unit 112 determines that no face area has been detected in S304, the histogram detection unit 111 generates a luminance histogram of the image in S307. Although it is desirable to generate a luminance histogram of all pixels of the composite image, a histogram may be generated for the composite image reduced by thinning processing, low-pass filter processing, or the like in order to save processing time and memory capacity. It should be noted that the edge portion of the image is generally not important in many cases, and depending on the imaging lens, it is a portion that is affected by a decrease in the amount of peripheral light. Therefore, a histogram may be created excluding the peripheral pixels.

  In S308, the histogram detection unit 111 detects a histogram feature amount. The histogram feature amount may be, for example, a level (SD) to which a pixel having a cumulative frequency of 10% from the dark side in the histogram belongs.

  In step S309, the correction amount calculation unit 113 serving as a target correction amount calculation unit calculates a target correction amount based on the luminance information of the composite image. When it is determined that a face has been detected, the correction amount calculation unit 113 obtains a target luminance with respect to the representative luminance value of the face, and performs spline interpolation or the like from the target luminance and the minimum luminance of the maximum luminance of the pixels of the composite image. A target output luminance for each input luminance is calculated. The difference between the input luminance and the target output luminance is the target correction amount, and a tone curve formed by the target output luminance with respect to each input luminance represents the gradation correction characteristic. The gradation correction characteristic can be expressed by, for example, a look-up table indicating the correspondence between input luminance and target output luminance.

  An example of the gradation correction characteristic creation operation of the correction amount calculation unit 113 will be described with reference to FIG. FIG. 5A shows an example of the relationship between the representative luminance value of the face and the target luminance with respect to the three input luminance values (64LSB, 128LSB, and 224LSB). Note that LSB means resolution, and in the present embodiment, the full scale (minimum value to maximum value) of the luminance value is represented by 8-bit resolution (0 to 255 LSB).

  In FIG. 5A, the horizontal line corresponding to “64 LSB point” indicates the relationship between the target output luminance of the pixel having the input luminance of 64 LSB and the representative luminance value of the face. For example, if the representative luminance value of the face detected from the captured image is in the range of 255 LSB to 180 LSB, the target output luminance of a pixel having a luminance of 64 LSB in the image is 64 LSB (that is, no change). In addition, in an image whose face representative luminance value is in the range of 45 LSB to 0 LSB, the target output luminance of a pixel having a luminance of 64 LSB is 128 LSB. When the face representative luminance value is in the range of 180 to 45, the target output luminance of a pixel having a luminance of 64 LSB changes linearly between 64 LSB and 128 LSB.

  When the representative brightness value of the face is 110LSB, the target output brightness of the pixels having the brightness of 64LSB, 128LSB and 224LSB are 97LSB, 159LSB and 228LSB from FIG. 5 (a), respectively. Therefore, the gradation correction characteristic for an image having a face representative luminance value of 110 LSB is a spline of three points representing the input / output relationship of the three points and the point corresponding to the lowest value (0LSB) and the highest value (255LSB). By interpolating, it is obtained as shown in FIG. The correction table showing the input / output relationship of representative luminance values according to the representative luminance value of the face shown in FIG. 5A is a settable degree such as “weak”, “standard”, “strong”, etc. Each of them can be prepared in advance in the correction amount calculation unit 113.

  A method for creating gradation correction characteristics for an image in which no face has been detected will be described with reference to FIG. In FIG. 6, the input value in the gradation correction characteristic of FIG. 5A is replaced with the detected value of the level (SD) to which the pixel having a cumulative frequency of 10% from the dark side in the histogram from the representative luminance of the face. Is. Using this gradation correction characteristic, the correction amount can be calculated in the same manner as when it is determined that the face has been detected. This correction table can also be prepared in advance in the correction amount calculation unit 113 for each settable degree, such as “weak”, “standard”, and “strong”.

  In HDR shooting, the subject is shot brighter even in a backlight scene as compared to normal shooting. Therefore, the correction tables shown in FIG. 5 and FIG. 6 may have different characteristics for HDR shooting and other normal shooting, and the correction may be weakened in the HDR shooting correction table.

  Returning to FIG. 3, in step S <b> 310, the correction amount calculation unit 113 calculates a correction amount for each of the correction using the low-frequency luminance component and the correction using the tone curve, and sends the calculated correction amount to the gradation correction unit 114. Output.

The correction amount calculation process in S310 will be described using the flowchart shown in FIG.
In step S <b> 701, the correction amount calculation unit 113 serving as a correction amount determination unit determines the exposure amount difference between the exposure amount (+) and the intermediate exposure amount and the exposure amount difference between the exposure amount (−) and the intermediate exposure amount. A correction amount for correction using the luminance low frequency component as the first correction amount is determined. As shown in FIG. 8, a correction amount for correction using a low-frequency luminance component may be prepared in advance using a table or the like. In FIG. 8, the exposure amount difference between the exposure amount (+) and the exposure amount (−) with respect to the intermediate exposure amount is assumed to be equal, and three correction characteristic curves of ± 1 step, ± 2 steps, and ± 3 steps are shown. . The correction using the low-frequency luminance component has a characteristic that the correction amount increases as the exposure amount difference increases. This is because, as described above, the gamma conversion process is performed such that the contrast decreases as the exposure amount difference between the plurality of images to be combined increases. For the sake of convenience, the correction amount using the luminance low frequency component is set as a correction amount (A).

  In step S <b> 702, the correction amount calculation unit 113 determines whether the face detection unit 112 determines that a face has been detected. If it is determined that a face has been detected, the process proceeds to S703. If it is determined that no face has been detected, the process proceeds to S704.

  In step S <b> 703, the correction amount calculation unit 113 corrects the correction amount (A) according to the representative luminance value of the face. Specifically, the correction amount calculation unit 113 obtains a correction coefficient based on a predetermined relationship between the representative luminance value of the face and the intensity of correction using the luminance low frequency component, for example, as shown in FIG. The correction amount (A) is corrected. Basically, the higher the representative luminance of the face is, the lower the intensity of correction using the luminance low frequency component is. This is because a bright face included in a follow light scene originally has a low contrast, and therefore, if a contrast is applied to a bright face by correction using a low-frequency luminance component, an unnatural image is obtained.

  In step S <b> 704, the correction amount calculation unit 113 serving as a correction amount determination unit determines a correction amount (referred to as a correction amount (B)) using the tone curve as the second correction amount. Here, the correction amount calculation unit 113 calculates a difference between the target correction amount and the correction amount (A) corrected according to the representative luminance value of the face as the correction amount (B). Thereby, the target correction amount can be realized by the correction amount (A) and the correction amount (B).

  The gradation correction unit 114 in FIG. 1 applies gradation correction (correction amount A) using a low-frequency luminance component and correction (correction amount (B)) using a tone curve to the composite image. Process.

但し、Ｙ’（ｘ，ｙ）は座標値が（ｘ，ｙ）の補正後の輝度成分、Ｆｎ（ｘ，ｙ）は座標（ｘ，ｙ）におけるガウシアン関数、ｎは尺度を表すパラメータである。また、γ０およびγ１は、補正の度合いを調節する係数である。また、＊は積和演算、Ａｖｇは平均値演算を表す。 The gradation correction process will be described with reference to the flowchart of FIG.
In step S <b> 1001, the gradation correction unit 114 serving as a first gradation correction unit performs luminance correction processing using a luminance low-frequency component on the composite image. This brightness correction processing may be performed by the method described in Patent Document 2, for example. That is, logarithmic transformation of the scaled luminance component distribution is performed, an average value at a different scale (different resolution) is obtained, a difference from the logarithmic transformation of the original luminance component is output, and the original is obtained by inverse transformation (exp calculation). The process of returning to the luminance unit is performed. Here, in order to adjust the degree of correction according to the image, the logarithm conversion output of the scaled luminance component is multiplied by a coefficient for adjusting the degree. The luminance component after correction based on the processing described above is expressed by the following equation (2).

However, Y ′ (x, y) is a corrected luminance component with coordinate value (x, y), Fn (x, y) is a Gaussian function at coordinates (x, y), and n is a parameter representing a scale. . Γ0 and γ1 are coefficients for adjusting the degree of correction. * Represents a product-sum operation, and Avg represents an average value operation.

  When this luminance conversion is executed by hardware processing, it can be constituted by, for example, an average value calculation circuit, a circuit for creating a lookup table, a table storage unit, a table reference circuit (gamma conversion part), and a division circuit. Note that the average value calculation circuit may be provided in a portion for extracting the low-frequency luminance signal.

  In step S <b> 1002, the gradation correction unit 114 performs color adjustment processing (first color adjustment processing) associated with the luminance correction processing using the luminance low-frequency component. In the first color adjustment process, the color components are corrected according to the luminance correction amount so that the colors of the images before and after the luminance correction process are not changed as much as possible. For example, the color components X (x, y) and Z (x, y) are multiplied by the ratio Y ′ (x, y) / Y (x, y) before and after the change of the luminance component, respectively. Only Y (x, y) is changed to Y ′ (x, y), and the processing is simplified by not performing processing on the color components X (x, y) and Z (x, y). Also good.

  In step S1003, the gradation correction unit 114 as the second gradation correction unit performs luminance correction processing using a tone curve. The gradation correction unit 114 can perform correction using a look-up table indicating the relationship of output luminance data with respect to each input luminance.

  In step S1004, the gradation correction unit 114 performs color adjustment processing (second color adjustment processing) associated with the luminance correction processing using the tone curve. The second color adjustment process may be the same as the first color adjustment process performed in S1002. Further, without performing the first color adjustment process, only the second color adjustment process corresponding to the change of the luminance value by the luminance correction process using the luminance low frequency component and the correction process using the tone curve may be performed. Good.

  The image data output unit 115 in FIG. 1 outputs the composite image that has been subjected to the luminance correction processing by the gradation correction unit 114 in the form of a predetermined image file. For example, when the file format is converted to a JPEG file, the image data is converted into a JPEG file format by three-dimensional matrix conversion into YCrCb format and attached data. The image data output unit 115 can record the image file on a recording medium such as a memory card or output it to an external device.

  As described above, according to the present embodiment, both the correction using the luminance low frequency component and the correction using the tone curve are applied to the HDR image. Then, the correction amount for each correction is calculated in consideration of the exposure amount difference of the images to be combined, information on the main subject such as the face, and information on the histogram of the captured image. Therefore, it is possible to obtain an HDR image with good brightness and contrast that suppresses a decrease in contrast due to correction using a tone curve and an unnatural correction result due to correction using a luminance low-frequency component.

(Second Embodiment)
In the first embodiment, the correction amount (A) for correction using the low-frequency luminance component is determined according to the exposure amount difference of the images to be combined. On the other hand, in the present embodiment, the gradation correction amount for the HDR image is distributed to the correction amount (A) by the correction using the luminance low frequency component and the correction amount (B) of the correction using the tone curve.

  FIG. 11 is a diagram schematically illustrating an example of distribution of gradation correction amounts in the present embodiment. The horizontal axis of FIG. 11 is the representative luminance value of the face calculated by the face detection unit 112, and the vertical axis is the correction when the target gradation correction amount calculated by the correction amount calculation unit 113 is 100%. It is the ratio of the quantity (A). The result of subtracting the correction amount (A) ratio from 100% is the correction amount (B) ratio.

  For each exposure amount difference, the representative luminance value of the face and the ratio of the correction amount (A) and the correction amount (B) (the ratio of the correction amount (A) or the correction amount (B) to the target gradation correction amount) A table indicating the relationship of (good) is prepared, and the target gradation correction amount is distributed according to this ratio. FIG. 11 shows an example of distribution when the exposure amount difference is ± 1, ± 2, and ± 3.

  As can be seen from FIG. 11, as the exposure amount difference during HDR shooting (image to be combined) is larger, the ratio of the correction amount by the correction using the luminance low frequency component is increased. Further, the larger the representative luminance value of the face, the smaller the ratio of the correction amount by the correction using the luminance low frequency component. This balances the fact that the greater the difference in the exposure amount of a plurality of images, the lower the contrast due to gamma conversion, and that the image becomes unnatural if the contrast is too high for a bright face.

If it is determined that no face is detected, the same ratio as in the case where the representative luminance value of the face is sufficiently dark in FIG. 11 can be used.
The tone correction unit 114 applies the correction using the luminance low frequency component and the correction using the tone curve to the composite image with the correction amount determined in this way.

  In the present embodiment, the target gradation correction amount is distributed by correction using the low-frequency luminance component and correction using the tone curve at a ratio according to the luminance of the main subject (for example, the face). Thereby, appropriate gradation correction can be performed in consideration of the influence on the brightness and contrast by gradation correction.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (9)

Obtaining means for obtaining a plurality of images taken with different exposure amounts;
It said plurality of images, a synthesizing means for generating a composite image by synthesizing after applying gamma conversion processing for reducing the contrast as the exposure amount difference between images is large,
Target correction amount calculating means for calculating a target correction amount for gradation correction of the composite image based on luminance information of the composite image;
A first correction amount for gradation correction using a low-frequency luminance component of the composite image and a second correction amount for gradation correction using a tone curve for realizing the target correction amount are determined. Correction amount determining means;
First gradation correction means for applying gradation correction processing to the composite image based on the first correction amount;
Second gradation correction means for applying gradation correction processing to the composite image based on the second correction amount;
The correction amount determining means, said first correction amount, after determined to be larger as the exposure amount difference between the images is large is corrected so that the luminance of the main object of the synthesized image becomes smaller the higher An image processing apparatus that is determined by The image processing apparatus according to claim 1, wherein the acquisition unit acquires a plurality of images taken with an exposure amount difference determined by a user.   Determining means for determining a larger exposure amount difference as the luminance range of the subject is larger;
  Acquisition means for acquiring a plurality of images taken with the exposure amount difference determined by the determination means;
  Combining means for generating a combined image based on the plurality of images;
  Target correction amount calculating means for calculating a target correction amount for gradation correction of the composite image based on luminance information of the composite image;
  A first correction amount for gradation correction using a low-frequency luminance component of the composite image and a second correction amount for gradation correction using a tone curve for realizing the target correction amount are determined. Correction amount determining means;
  First gradation correction means for applying gradation correction processing to the composite image based on the first correction amount;
  Second gradation correction means for applying gradation correction processing to the composite image based on the second correction amount;
  The image processing apparatus according to claim 1, wherein the correction amount determination unit determines the first correction amount so that the first correction amount decreases as the luminance of the main subject of the composite image increases. It said synthesizing means synthesizes the plurality of images to which the gamma conversion processing in accordance with the exposure amount, the image processing apparatus according to 請 Motomeko 3 you and generating the composite image. The synthesizing unit applies a gamma conversion process that lowers the contrast as the exposure difference between the plurality of images increases.
The correction amount determination means determines the first correction amount so as to increase as the exposure difference between the plurality of images increases, and then corrects the first correction amount according to the luminance of the main subject. The image processing apparatus according to claim 4 . The correction amount determining means, wherein the second correction amount, to any one of claims 1 to 5, characterized in that to determine the difference between the target correction amount and the first correction amount Image processing apparatus. The target correction amount calculating means, according to any one of claims 1 to 6, characterized in that to calculate the target correction amount based on the luminance of the luminance histogram or the main object of the synthesized image Image processing device. An acquisition step of acquiring a plurality of images taken with a plurality of different exposure amounts;
It said plurality of images, a synthesizing step of generating a synthesized image by synthesizing after applying gamma conversion processing for reducing the contrast as the exposure amount difference between images is large,
A target correction amount calculating step of calculating a target correction amount of gradation correction of the composite image based on luminance information of the composite image;
A first correction amount for gradation correction using a low-frequency luminance component of the composite image and a second correction amount for gradation correction using a tone curve for realizing the target correction amount are determined. A correction amount determining step;
A first gradation correction step of applying a gradation correction process to the composite image based on the first correction amount;
A second gradation correction step of applying a gradation correction process using a tone curve to the composite image based on the second correction amount;
In the correction amount determination step, the first correction amount is determined so as to increase as the exposure amount difference between the images increases, and then corrected so as to decrease as the luminance of the main subject of the composite image increases. An image processing method characterized in that it is determined by   A determination step for determining a larger exposure amount difference as the luminance range of the subject is larger;
  An acquisition step of acquiring a plurality of images taken with the exposure amount difference determined in the determination step;
  Generating a composite image based on the plurality of images;
  A target correction amount calculating step of calculating a target correction amount of gradation correction of the composite image based on luminance information of the composite image;
  A first correction amount for gradation correction using a low-frequency luminance component of the composite image and a second correction amount for gradation correction using a tone curve for realizing the target correction amount are determined. A correction amount determining step;
  A first gradation correction step of applying a gradation correction process to the composite image based on the first correction amount;
  A second gradation correction step of applying a gradation correction process using a tone curve to the composite image based on the second correction amount;
  In the correction amount determination step, the first correction amount is determined so as to decrease as the luminance of the main subject of the composite image increases.

Images