JP6397261B2 - Image processing apparatus and method - Google Patents

Description

  The present invention relates to an image processing apparatus and method, and more particularly to an image processing apparatus and method for controlling gradation characteristics.

  Conventionally, there has been proposed a digital dodging process for controlling gradation characteristics by applying a locally different gain to an input image according to input luminance. This dodging process can be applied, for example, in a so-called backlight scene where the brightness of the main subject to be photographed is significantly darker than the brightness of the background. In a backlight scene, if the brightness of the entire image is controlled appropriately, the main subject is generally photographed dark. Therefore, by using the brightness-specific gain set so that a relatively large gain is applied to the dark part by the dodging process, the main subject photographed dark is controlled to have an appropriate brightness.

  In Patent Document 1, an input image is divided into a plurality of areas, and gradation conversion characteristics are obtained for each area, processing is performed, a main area is selected from the divided areas, and the selected main area is selected. With respect to the above, it is disclosed that high-precision processing is performed when the gradation conversion characteristics are calculated.

JP 2006-295582 A

  However, with the technique described in Patent Document 1, appropriate gradation processing can be performed in each area, but the relationship between the areas is not taken into consideration, and an unnatural impression appears when viewed as a whole. There is concern that an image will be generated.

  The present invention has been made in view of the above-described problems, and an object thereof is to suppress an unnatural image when performing gradation correction.

In order to achieve the above object, an image processing apparatus according to the present invention is based on a dividing unit that divides an input image into a plurality of subject areas according to a predetermined condition, and a luminance distribution of each of the divided subject areas. Then, the luminance of the image is divided into a plurality of luminance areas, and each of the luminance areas is determined based on a representative luminance value of one subject area corresponding to each of the divided luminance areas. calculating means for calculating a first gain, among the plurality of object regions, each front Kiteru level area, the distribution of the luminance of the corresponding one of the subject region, at least one luminance area other than the luminance area Correction means for correcting the first gain based on the corresponding luminance distribution of at least one other subject area to obtain a second gain for each of the luminance areas; and There are, and a converting means for converting the luminance of the image.

  According to the present invention, it is possible to suppress an unnatural image when performing gradation correction.

1 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention. The figure which shows an example of a backlight scene. The flowchart which shows the information acquisition process by the information acquisition part for every area | region in this embodiment. The figure for demonstrating the area | region determination process in this embodiment. The figure which shows an example of the brightness | luminance histogram of each to-be-photographed object area | region in this embodiment. 5 is a flowchart showing a tone conversion characteristic generation process by a tone conversion characteristic generation unit according to the present embodiment. The figure which shows the concept of the area division of the brightness | luminance in this embodiment. The figure which shows the calculation method of the boundary of the brightness | luminance in this embodiment. The figure which shows the calculation procedure of GAIN_HUMAN, GAIN_BACK, and GAIN_SKY in this embodiment. The flowchart which shows the gain amount adjustment process in this embodiment. The figure which shows the relationship between the difference Bv_HUMAN_OUT-Bv_BACK_OUT of the output Bv value of a person area | region and a background area | region in this embodiment, and the minimum value k_low_min of a gain adjustment coefficient. The figure which shows the low-intensity area | region and high-intensity area | region in the luminance histogram of the background area | region in this embodiment. The figure which shows the relationship between the low-intensity pixel rate RATIO_LOW and the gain adjustment amount k_low in this embodiment. The figure which shows the relationship between the difference Bv_BACK_OUT-Bv_SKY_OUT of the output Bv value of a background area | region and a sky area | region in this embodiment, and the minimum value k_hi_max of a gain adjustment coefficient. The figure which shows the relationship between the high-intensity pixel rate RATIO_HI and the gain adjustment amount k_hi in this embodiment. The figure for demonstrating the reason for adjusting gain amount.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention. In FIG. 1, the imaging unit 101 captures an image to be processed. In this embodiment, as described later, by applying a gain according to the luminance of the image, the subject of the low luminance part is brightened and output while suppressing saturation of the high luminance part. Therefore, it is desirable that the imaged exposure of the image is under the proper exposure. In the present embodiment, the method for determining the image exposure is not particularly limited, and any method may be used. As an example of how to determine the exposure of the image, set the shooting exposure so that the high-brightness area does not saturate based on the method of setting the shooting exposure under the preset step from the appropriate exposure or the brightness distribution of the image during live view The method etc. are mentioned. In the present embodiment, a case where an image to be processed is shot by the imaging unit 101 will be described. However, an image to be processed may be input from the outside via communication, a storage medium, or the like.

  The signal processing unit 102 performs predetermined signal processing on the image captured by the imaging unit 101. Here, examples of the predetermined signal processing include defective pixel correction of the image sensor, peripheral light amount drop correction, and noise reduction processing.

  The area-by-area information calculation unit 103 divides the image that has been subjected to predetermined signal processing by the signal processing unit 102 into a plurality of object areas, and acquires information for each object area. In the present embodiment, the priority for determining the representative luminance value, the luminance histogram, and the gradation conversion characteristics of each area is acquired as information for each subject area. The information acquisition process for each subject area in the area-by-area information calculation unit 103 will be described in detail later.

  The gradation conversion characteristic generation unit 104 determines the gradation conversion characteristic based on the information for each subject area obtained from the area information calculation unit 103. Here, as gradation conversion characteristics, a gain to be multiplied by the image to be processed is determined for each luminance. The gradation conversion characteristic generation processing in the gradation conversion characteristic generation unit 104 will be described in detail later.

  In the gradation conversion processing unit 105, gradation conversion processing (gain processing) is performed on the image signal of the image processed in the signal processing unit 102 using the gain for each luminance generated in the gradation conversion characteristic generation unit 104. I do. Note that the gradation conversion processing performed here is not particularly limited, and an output image is generated by multiplying each pixel value of the image by an appropriate gain based on the gain for each luminance and the luminance value of the image. As long as it is a process to do.

  Finally, the output image generated in the gradation conversion processing unit 105 is sent to the image display unit 106 and the image recording unit 107, where the image is displayed and recorded.

  Next, processing performed by the area information calculation unit 103 and the gradation conversion characteristic generation unit 104 will be described. In the present embodiment, tone conversion processing is performed using the processing results of the region-by-region information calculation unit 103 and the tone conversion characteristic generation unit 104. As an image to be subjected to tone conversion processing, a backlight scene as shown in FIG. Will be described as an example. As shown in FIG. 2, in the backlight scene, the background is significantly brighter than the brightness of the main subject 201.

  First, information acquisition processing for each subject area in the area-by-area information calculation unit 103 will be described with reference to the flowchart of FIG. In S301, the image to be processed is divided into a plurality of subject areas in accordance with predetermined conditions. In the example shown in FIG. 2, the subject area is divided into a person area, a background area, and an empty area, but may be changed according to the contents of the image. In this process, the area-by-area information calculation unit 103 first divides the block into blocks [i] [j] having a predetermined size as shown in FIG. Then, an area determination process is performed to determine whether each block Block [i] [j] is included in a person area, a background area, or an empty area. The method of region determination processing is not limited, but the region determination processing can be performed based on, for example, the representative luminance value or representative color information of each block Block [i] [j]. Further, the size of the block Block [i] [j] is not particularly limited, and for example, the area determination process may be performed on a pixel basis.

  Then, when it is determined whether each block Block [i] [j] is included in the person area, the background area, or the empty area, for example, as shown in FIG. Assign. Here, the determination result of the block Block [i] [j] is 0 if it is empty, 1 if it is the background, 2 if it is a person, and if it is not in any area (not applicable) Assigns 3. Here, “not applicable” refers to a portion where it is difficult to determine the region, for example, a boundary portion between the regions. Then, the blocks to which the same value is assigned can be divided into subject areas. That is, the blocks assigned 0 are grouped together into an empty area, the blocks allocated 1 are grouped together as a background area, and the blocks allocated 2 are grouped as a person area. In the present embodiment, blocks to which 3 is assigned are not used.

Next, in S302, a representative luminance value is calculated for each subject area. The representative luminance value is a weighted average value of the average luminance values of the blocks belonging to each subject area. Specifically, assuming that the representative luminance values of the sky area, the background area, and the person area are Y_SKY, Y_BACK, and Y_HUMAN, the respective values are calculated by the following equations (1) to (3).
Y_SKY = Σ (W_SKY [i] [j] × Block_Y [i] [j]) / ΣW_SKY [i] [j]
(Only blocks with Block_id [i] [j] = 0) (1)
Y_BACK = Σ (W_BACK [i] [j] × Block_Y [i] [j]) / ΣW_BACK [i] [j]
(Only blocks with Block_id [i] [j] = 1) (2)
Y_HUMAN = Σ (W_HUMAN [i] [j] × Block_Y [i] [j]) / ΣW_HUMAN [i] [j]
(Block_id [i] [j] = 2 block only) (3)
Here, Block_Y [i] [j] is an average luminance value of each block, and W_SKY [i] [j], W_BACK [i] [j], and W_HUMAN [i] [j] are weighting coefficients for each block. The weighting factor can be set arbitrarily.

  The reason why the weight can be set separately for each subject area is as follows. First, in a person or sky area that is a single subject area, the representative luminance value may be the average value of the average luminance values of the blocks included in the area, and thus the weight may be the same for all blocks. On the other hand, since there are subjects with various brightness values such as plants and buildings in the background area, it is necessary to calculate an accurate representative brightness value reflecting the subject intended by the photographer. In general, since the subject intended by the photographer is often in the center of the image, for example, the weight of the center of the image is set relatively large with respect to the background area. As described above, since the concept of calculating the representative luminance value may be different for each subject area, the weighting factor can be set separately for each subject area in the present embodiment.

  Next, in S303, a luminance histogram of each subject area is generated. Here, a histogram of the average luminance value of each block belonging to each subject area is generated. 5A and 5B are diagrams illustrating an example of the generated luminance histogram, in which FIG. 5A is a luminance histogram of a person region, FIG. 5B is a luminance histogram of a background region, and FIG. 5C is a luminance of a sky region. A histogram is shown.

  Next, in step S304, a priority for generating gradation conversion characteristics is determined for each subject area. Here, the reason for setting the priority is that the tone conversion processing performed in the present embodiment uses the luminance-specific gain with the same characteristics for the entire image area, so that a desired gain is not necessarily applied to all subject areas. This is because they are not multiplied. Therefore, in generating the gradation conversion characteristics, it is necessary to set the priority of the region to which a desired gain is applied in advance.

  The specific method of determining the priority is not particularly limited in this case. For example, a method of determining the priority from the evaluation value by using a value obtained by multiplying the area of each subject region by a predetermined coefficient is given. It is done. In addition, there is a method in which the priority of the region having the smallest representative luminance value is made highest. In the backlight scene shown in FIG. 2 assumed in the present embodiment, it is assumed that a person is present in the low brightness area, the sky is present in the high brightness area, and the background is unevenly distributed in the middle brightness area. In the form, the region with the highest priority is set as the background region.

  Through the processing described above, the priority for generating the representative luminance value, the luminance histogram, and the gradation conversion characteristic of each subject area is obtained as the output of the area-by-area information calculation unit 103.

  Next, the gradation conversion characteristic generation processing in the gradation conversion characteristic generation unit 104 will be described with reference to the flowchart of FIG. The gradation conversion characteristic generation unit 104 determines the gradation conversion characteristic based on the information on each subject area obtained from the area information calculation unit 103. Here, the determination of the gradation conversion characteristic is to determine a gain for each luminance to be multiplied to the image to be processed.

  First, in S501, the luminance is divided into sections based on the luminance histogram of each subject area obtained from the area-by-area information calculation unit 103. In S502, the gain for each luminance section is calculated. As described above, in a typical backlight scene targeted in the present embodiment, the luminance distribution tends to be biased for each subject area as shown in FIG. It can be seen that the brightness of the person area is biased toward the low brightness area as shown in FIG. 5A, and the brightness of the sky area is biased toward the high brightness area as shown in FIG. 5C. Therefore, as shown in FIG. 7, luminance sections corresponding to each subject area are calculated from the information for each subject area, and the characteristics of gradation conversion processing corresponding to each section, that is, the gain to be multiplied by the image are determined. .

The specific processing of S501 is to calculate the luminances Y0 and Y1 that are the luminance boundaries shown in FIG. An example of the calculation method is shown in FIG. As shown in FIG. 8 (a), the luminance histogram of the person region is sequentially accumulated in ascending order of luminance, and the luminance value when the cumulative frequency reaches a predetermined ratio TH_RATIO_HUMAN with respect to the region area N_human is Y0. . Further, as shown in FIG. 8B, the luminance histogram of the sky region is sequentially accumulated in descending order of luminance, and the luminance value when the cumulative frequency reaches a predetermined ratio TH_RATIO_SKY with respect to the region area N_sky is expressed as Y1. And The brightness is divided as shown in FIG. 7 based on the brightness Y0 and Y1 of the boundary thus obtained. That is, the area of luminance 0 to Y0 is classified as a low luminance area where the luminance of the person is unevenly distributed, and the area of luminance Y1 to 2 ^N −1 is divided as a high luminance area where the luminance of the sky is unevenly distributed. Then, areas of luminance Y0 to Y1 other than the low luminance area and the high luminance area are classified as medium luminance areas where the background luminance is unevenly distributed.

  Next, in S502, a gain to be multiplied by the luminance of each divided luminance region is determined so that the luminance of the output image approaches the photographer's impression.

First, an exposure amount (hereinafter referred to as “appropriate Bv value”) when each subject area is output with appropriate brightness is calculated. The appropriate Bv value of each subject area is a Bv value when the representative brightness value of each subject area becomes a preset target brightness value. Assuming that the exposure amount at the time of shooting the image to be processed is Bv_CAPTURE, the appropriate Bv value for each subject area is calculated from the representative brightness value of each subject area as shown in FIG. can do.
Bv_SKY = Bv_CAPTURE + ΔBv_SKY (4)
Bv_BACK = Bv_CAPTURE + ΔBv_BACK (5)
Bv_HUMAN = Bv_CAPTURE + ΔBv_HUMAN (6)
here,
ΔBv_SKY = log ₂ (Y_SKY_TARGET / Y_SKY) (7)
ΔBv_BACK = log ₂ (Y_BACK_TARGET / Y_BACK)… （8）
ΔBv_HUMAN = log ₂ (Y_HUMAN_TARGET / Y_HUMAN)… （9）
Bv_SKY, Bv_BACK, and Bv_HUMAN are appropriate Bv values for each subject area, and ΔBv_SKY, ΔBv_BACK, and ΔBv_HUMAN are the difference amounts from Bv_CAPTURE to the appropriate Bv value for each subject area. Y_SKY_TARGET, Y_BACK_TARGET, and Y_HUMAN_TARGET are target brightness values for each subject area, and Y_SKY, Y_BACK, and Y_HUMAN are brightness values for each subject area.

Here, if each subject area is output with brightness obtained with an appropriate Bv value, the overall appearance of the image becomes unnatural. Therefore, an exposure amount (hereinafter referred to as an “output Bv value”) that makes the appearance natural is calculated, and a gain is set so that the output luminance of each subject area becomes a brightness equivalent to each output Bv value. . The output Bv value is calculated by compressing the difference amount ΔBv between the appropriate Bv value of each subject area and the exposure amount at the time of shooting at a predetermined ratio with respect to the reference exposure amount. In the example shown in FIG. 9, the reference of the output Bv value is the output Bv value of the background area, and the predetermined ratio is 1/2. The formula is expressed as follows.
Bv_SKY_OUT = (Bv_SKY + Bv_BACK) / 2 ... (10)
Bv_BACK_OUT = Bv_BACK (11)
Bv_HUMAN_OUT = (Bv_HUMAN + Bv_BACK) / 2 ... (12)
Here, Bv_SKY_OUT, Bv_BACK_OUT, and Bv_HUMAN_OUT are output Bv values of each subject area.

As shown in FIG. 9, the gain multiplied to each subject area is calculated from the difference obtained by subtracting the output Bv value from Bv_CAPTURE. The formula is expressed as follows.
GAIN_SKY = 2 ^ (Bv_CAPTURE-Bv_SKY_OUT)… (13)
GAIN_BACK = 2 ^ (Bv_CAPTURE-Bv_BACK_OUT)… (14)
GAIN_HUMAN = 2 ^ (Bv_CAPTURE-Bv_HUMAN_OUT)… (15)
Here, GAIN_SKY, GAIN_BACK, and GAIN_HUMAN are gains multiplied by each subject area.

  Next, in S503, the gain amount is adjusted. Here, the reason for adjusting the gain amount will be described with reference to FIG. As shown in FIG. 16A, a case is assumed where many pixels having the same luminance value as the person area are distributed in the background area of the backlight scene. When a gain having the characteristics shown in FIG. 16B is used, a gain multiplied by the person area is applied to an area having a luminance as low as that of the person area existing in the background area. The brightness value in the brightness area increases. As a result, as shown in FIG. 16C, an unnatural image in which the dark part of the background area is floating is generated. This is due to the fact that the gradation conversion processing is realized using a uniform gain by brightness, and the brightness level of the dark area in the background area not desired to be brightly output and the person area desired to be brightly output are substantially the same. This is because these two areas cannot be distinguished from the luminance values.

  Conversely, especially when the sky is very bright against the background, the gain amount in the high luminance region is suppressed and the gradation is compressed so that the sky does not fly out. However, when many pixels having the same luminance as the high luminance region are distributed in the background region, the luminance is compressed in the same manner as in the sky region, resulting in an output image with lost contrast.

  In S503, the gain amount is adjusted to alleviate the above phenomenon. Note that the phenomenon described above becomes prominent when low luminance and / or high luminance pixels are distributed in the background area and the gain amount multiplied by the low luminance area is large, or when the level of the high luminance area is high. This is a case where the adjustment compression amount is large. In such a case, an adjustment is made to reduce the gain amount multiplied by the person area and increase the gain amount multiplied by the empty area. That is, the adjustment in the direction of reducing the desired effect is performed. However, if such an adjustment is always performed, the expected effect in the output image may not be obtained. Therefore, it is necessary to perform adaptive control according to the scene.

  FIG. 10 is a flowchart showing the gain amount adjustment processing performed in S503. As shown in FIG. 10, the gain adjustment processing performed in S503 includes low luminance side processing and high luminance side processing. The processing performed in S801 to S804 is low-luminance side processing, and the processing performed in S805 to S808 is high-luminance side processing. First, in S801, the minimum value k_low_min (lower limit) of the gain adjustment coefficient is obtained from the difference Bv_HUMAN_OUT−Bv_BACK_OUT between the output Bv values of the person area and the background area.

  FIG. 11 is a graph showing the relationship between the difference Bv_HUMAN_OUT−Bv_BACK_OUT between the output Bv values of the person area and the background area and the minimum value k_low_min of the gain adjustment coefficient. As can be seen from FIG. 11, the minimum value k_low_min of the gain adjustment coefficient takes a smaller value as the difference between the output Bv values of the person area and the background area increases. The reason for this is that the greater the difference in the output Bv value, the greater the degree of backlighting and the greater the gain applied to the person area. Therefore, the phenomenon described in FIG. 16 is likely to occur remarkably, and it is better to suppress the gain amount. Because there is. In the example illustrated in FIG. 11, when the output Bv value difference Bv_HUMAN_OUT−Bv_BACK_OUT is smaller than the threshold value TH1_dBv_low, the gain is not adjusted, and when the difference is larger than the threshold value TH2_dBv_low, the minimum value k_low_min of the gain adjustment coefficient. Is fixed to min.

  Next, in S802, the low luminance pixel rate RATIO_LOW is calculated from the luminance histogram of the background area. The low luminance pixel rate is the ratio of the number of pixels belonging to the low luminance area whose luminance value is equal to or lower than the luminance Y0 to the total number of pixels in the background area. FIG. 12 is a diagram illustrating a low luminance region and a high luminance region described later. The pixels belonging to the low luminance area shown in FIG. 12 are approximately the same as the luminance level of the pixel value of the person area, and gain of the same value as that of the person area is applied by the luminance-specific gain processing. Therefore, it is considered that the phenomenon described with reference to FIG. 16 becomes more remarkable as the low luminance pixel rate RATIO_LOW is larger.

  Next, in S803, a final gain adjustment amount k_low is obtained from the low luminance pixel rate RATIO_LOW. FIG. 13 is a graph showing the relationship between the low luminance pixel rate RATIO_LOW and the gain adjustment amount k_low. As described above, the value of the gain adjustment amount k_low is decreased as the low luminance pixel rate RATIO_LOW is increased. In the example shown in FIG. 13, when the low luminance pixel rate RATIO_LOW is smaller than the threshold value TH1_ratio_low, the gain is not adjusted. This is because when the number of low-luminance pixels in the background area is small, it is considered that the effect described in FIG. When the low luminance pixel rate RATIO_LOW is larger than the threshold value TH2_ratio_low, the gain adjustment amount k_low is fixed to the minimum value k_low_min of the gain adjustment coefficient. That is, when the low luminance pixel rate RATIO_LOW is larger than the threshold value TH1_ratio_low or the threshold value TH2_ratio_low, the gain adjustment coefficient is made smaller than that in the case of the threshold value TH1_ratio_low or less.

  However, as shown in FIG. 11, the minimum value k_low_min of the gain adjustment coefficient is between 1 and min depending on the difference Bv_HUMAN_OUT−Bv_BACK_OUT of the output Bv value. Therefore, for example, when the difference between the output Bv values Bv_HUMAN_OUT−Bv_BACK_OUT is small and the minimum gain adjustment coefficient value k_low_min is 1, the gain adjustment amount k_low is 1 regardless of the low luminance pixel rate RATIO_LOW, and the gain No adjustment will be made.

Next, in S804, the gain GAIN_HUMAN multiplied by the low luminance area (person area) is recalculated by the following equation (16).
GAIN_HUMAN = GAIN_HUMAN * k_low (16)

  The above is the content of the low luminance side processing of gain amount adjustment. Subsequently, the high luminance side processing will be described.

  First, in S805, the maximum value k_hi_max (upper limit) of the gain adjustment coefficient is calculated from the difference Bv_BACK_OUT−Bv_SKY_OUT between the output Bv values of the background area and the sky area. FIG. 14 is a graph showing the relationship between the difference Bv_BACK_OUT−Bv_SKY_OUT between the output Bv values of the background area and the sky area and the maximum value k_hi_max of the gain adjustment coefficient. As can be seen from FIG. 14, the maximum value k_hi_max of the gain adjustment coefficient takes a larger value as the difference between the output Bv values of the background area and the sky area increases. This is because, as the difference in output Bv value is larger, the gradation compression degree in the high luminance region is larger and the gain multiplied in the sky region is smaller. Therefore, the phenomenon described with reference to FIG. This is because it may be better. In the example illustrated in FIG. 14, when the output Bv value difference Bv_BACK_OUT−Bv_SKY_OUT is smaller than the threshold value TH1_dBv_hi, the gain is not adjusted, and when the output Bv value difference is larger than the threshold value TH2_dBv_hi, the maximum value k_hi_max Is fixed to max.

  In step S806, the high luminance pixel rate RATIO_HI is calculated from the luminance histogram of the background area. The high luminance pixel rate is the ratio of the number of pixels belonging to the high luminance area whose luminance value is equal to or higher than the luminance Y1 to the total number of pixels in the background area. The pixels belonging to the high luminance area shown in FIG. 12 are approximately the same as the luminance level of the pixel value in the sky area, and gain of the same value as that in the sky area is applied by the luminance-specific gain processing. Therefore, it is considered that the phenomenon described with reference to FIG. 16 becomes more remarkable as the high luminance pixel rate RATIO_HI is larger.

  Next, in S807, a final gain adjustment amount k_hi is obtained from the high luminance pixel rate RATIO_HI. FIG. 15 is a graph showing the relationship between the high luminance pixel rate RATIO_HI and the gain adjustment amount k_hi. From the above consideration, the value of k_hi is increased as RATIO_HI is larger. In the example illustrated in FIG. 15, when the high luminance pixel rate RATIO_HI is smaller than the threshold value TH1_ratio_hi, the gain is not adjusted. This is because when the number of high-luminance pixels in the background area is small, it is considered that the effect described in FIG. When the high luminance pixel rate RATIO_HI is larger than the threshold value TH2_ratio_hi, the gain adjustment amount k_hi is fixed to the maximum value k_hi_max of the gain adjustment coefficient. That is, when the high luminance pixel rate RATIO_HI is larger than the threshold value TH1_ratio_hi or the threshold value TH2_ratio_hi, the gain adjustment coefficient is made larger than when the threshold value TH1_ratio_hi is less than or equal to.

  However, as shown in FIG. 14, the maximum value k_hi_max of the gain adjustment coefficient is between 1 and max according to the difference Bv_BACK_OUT−Bv_SKY_OUT of the output Bv value. Therefore, for example, when the difference between the output Bv values Bv_BACK_OUT−Bv_SKY_OUT is small and the maximum value k_hi_max of the gain adjustment coefficient is 1, the gain adjustment amount k_hi is 1 regardless of the high luminance pixel rate RATIO_HI, and the gain No adjustment will be made.

Next, in S808, the gain GAIN_SKY multiplied by the high luminance area (sky area) is recalculated by the following equation (17).
GAIN_SKY = GAIN_SKY * k_hi (17)

  By the above process, the gain GAIN_HUMAN multiplied by the low luminance region and the gain GAIN_SKY multiplied by the high luminance region are readjusted so as to reduce the phenomenon described in FIG.

  After the above-described gain amount adjustment processing, returning to FIG. 5, in step S504, the gradation conversion characteristics are generated. Here, the gradation conversion characteristics are gain characteristics by luminance. When the gains GAIN_HUMAN, GAIN_BACK, and GAIN_SKY adjusted as described above are used in each luminance section, gradation inversion and tone jump may occur at the boundary of each luminance section. Therefore, a gain characteristic that does not cause such a problem is generated. The generation of the gain characteristic is not particularly limited in the present embodiment. Determine characteristics.

  As a result of the above processing, the brightness-specific gain characteristics that are adaptively generated for the image to be processed are generated as the output of the gradation conversion characteristic generation unit 104.

  As described above, according to the present embodiment, the gain amounts of the low luminance region and the high luminance region are determined in consideration of not only the luminance difference between the subject regions but also the luminance distribution of the intermediate luminance region. As a result, it is possible to suppress the dark portion of the background portion from floating unnaturally, or the bright portion from being excessively tone-compressed, resulting in an unnatural output image with a reduced contrast.

  In the above embodiment, the case where the highest priority area is set as the background area has been described. However, the highest priority area may be a person area or an empty area. For example, when the area with the highest priority is a person area, the person area is compared with the background area as intermediate luminance side processing instead of the low luminance side processing shown in FIG. And the empty area may be compared.

  Further, the divided subject areas do not include the three subject areas of the person area, the background area, and the sky area, and may include only two subject areas such as a person area and a sky area. In that case, only one of the processes corresponding to the two processes shown in FIG. 10 may be performed.

<Other embodiments>
Note that the present invention can be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, a scanner, a video camera, etc.), or a device (for example, a copier, a facsimile device, etc.) composed of a single device You may apply to.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

  101: Image processing unit, 102: Signal processing unit, 103: Information acquisition unit for each area, 104: Tone conversion characteristic generation unit, 105: Tone conversion processing unit, 106: Image display unit, 107: Image recording unit

Claims (12)

Dividing means for dividing the input image into a plurality of subject areas according to a predetermined condition;
Based on the luminance distribution of each of the divided subject areas, the luminance of the image is divided into a plurality of luminance areas, and a representative of one subject area corresponding to each of the divided luminance areas among the plurality of subject areas. Calculating means for calculating a first gain for each of the luminance regions based on a luminance value;
Of the plurality of subject areas, for each of the brightness areas, the brightness distribution of the corresponding one subject area and the brightness of at least one other subject area corresponding to at least one brightness area other than the brightness area. Correction means for correcting the first gain based on the distribution and obtaining a second gain for each of the luminance regions;
An image processing apparatus comprising: conversion means for converting the luminance of the image using the second gain.   The calculation means calculates a gain for converting a representative luminance value of the one subject area corresponding to each of the divided luminance areas into a predetermined luminance as the first gain. The image processing apparatus according to 1. The plurality of luminance regions include a low luminance region corresponding to a subject region having the lowest representative luminance value among the plurality of subject regions,
The correction means corrects the first gain based on a ratio of an image signal having a luminance of the low luminance region among image signals included in the different subject region,
The image processing apparatus according to claim 1, wherein when the ratio is higher than a predetermined threshold, the first gain is made smaller than when the ratio is equal to or lower than the threshold. The correction means further corrects the first gain based on a difference between a luminance of the one subject region corresponding to the low luminance region and a luminance of the other subject region,
The image processing apparatus according to claim 3, wherein when the difference is larger than a predetermined threshold value, the lower limit of the second gain is made smaller than when the difference is equal to or smaller than the threshold value. The plurality of luminance regions include a high luminance region corresponding to a subject region having the highest representative luminance value among the plurality of subject regions,
The correction means corrects the first gain based on a ratio of an image signal having the luminance of the high-luminance region among image signals included in the different subject region,
5. The image processing apparatus according to claim 1, wherein when the ratio is higher than a predetermined threshold value, the first gain is made larger than when the ratio is equal to or lower than the threshold value. . The correction means further corrects the first gain based on a difference between a luminance of the one subject region corresponding to the high luminance region and a luminance of the another subject region,
The image processing apparatus according to claim 5, wherein when the difference is larger than a predetermined threshold, the upper limit of the second gain is set larger than when the difference is equal to or smaller than the threshold.   The image processing apparatus according to claim 1, wherein the subject area includes at least a person area, a background area, and a sky area.   The said correction | amendment means further determines the priority of these subject area | regions, and calculates | requires a said 2nd gain based on this priority, The one of Claim 1 thru | or 7 characterized by the above-mentioned. Image processing apparatus.   The image processing apparatus according to claim 8, wherein the correction unit determines the priority based on an area of each of the plurality of subject regions. A dividing step in which the dividing means divides the input image into a plurality of subject areas according to a predetermined condition;
A dividing step of dividing the luminance of the image into a plurality of luminance areas based on a luminance distribution of each of the divided subject areas;
A calculating step in which the calculation means calculates a first gain for each of the luminance regions based on a representative luminance value of one subject region corresponding to each of the divided luminance regions among the plurality of subject regions;
The correction means includes, for each of the plurality of subject areas, a brightness distribution of the corresponding one subject area and at least one other subject corresponding to at least one brightness area other than the brightness area. A correction step of correcting the first gain based on the luminance distribution of the region and obtaining a second gain for each of the luminance regions;
An image processing method, comprising: a conversion step in which conversion means converts the luminance of the image using the second gain. The program for functioning a computer as each means of the image processing apparatus of any one of Claim 1 thru | or 9 . A computer-readable storage medium storing the program according to claim 11 .