JP6345430B2 - Multi-area white balance control device, multi-area white balance control method, multi-area white balance control program, computer recording multi-area white balance control program, multi-area white balance image processing device, multi-area white balance image processing method, multi-area White balance image processing program, computer recording multi-area white balance image processing program, and imaging apparatus provided with multi-area white balance image processing device - Google Patents

Description

  In the multi-area white balance image processing in an image obtained by photographing a scene illuminated with a plurality of light sources, the present invention narrows down the types of light sources in the image for each image divided into a plurality of areas, and Based on the result, a white balance correction gain map of the entire image is generated, and white balance correction is performed on the input image using the generated image, thereby making it possible to reproduce the color without any discomfort in the entire image. Multi-area white balance control device, multi-area white balance control method, multi-area white balance control program, computer recording multi-area white balance control program, multi-area white balance image processing device, multi-area white balance image processing Way, multi Rear white balance image processing program, an image pickup apparatus having recorded a computer and multi-area white balance image processing apparatus multi area white balance image processing program.

If the photographing light source is outdoors during the day, its color temperature changes depending on the altitude of the sun and the atmospheric conditions.
In the same scene, the color temperature differs between the sun and the shade.
Furthermore, there are various artificial lights such as fluorescent lamps and incandescent lamps indoors, and the color temperature of each light source is different.
In addition, when flash photography is taken outdoors in the daytime, mixed light of flash illumination and external light serves as a photographing light source.
Also, when flash photography is taken outdoors at night, the flash illumination light and the background artificial illumination light are not mixed, but the color temperature of the light source is different in different areas in the same scene.
In addition, when flash photography is performed indoors, light in which flash illumination light and artificial illumination light are mixed becomes a photographing light source.

  Under the various light sources described above, the human eye adapts to color. That is, the human eye feels a white subject in the attention area white regardless of whether the light source is sunlight, artificial illumination light, or a mixed light thereof. The camera, which is an input device in accordance with the characteristics of the human eye, is also configured to adjust so-called white balance by adjusting the RGB gain.

  However, for a camera, the illuminating light source is unknown until shooting, so in general, the type of illuminating illumination (color temperature of the light source) is estimated from the captured image, or the illumination is specified by the user. Correct the white balance of the entire image shot according to the type.

  When the subject scene is illuminated by multiple light sources, in order to perform white balance processing of the entire image with high accuracy, the color temperature of the light source is detected according to the area of the acquired image, and a different white balance for each area. It is necessary to perform processing.

Conventionally, for example, Patent Document 1 discloses an example of a method for performing white balance processing on an image obtained by photographing a scene illuminated with a plurality of light sources.
In the method described in Patent Document 1, an input color image is divided into a plurality of small screens, and a color temperature of a photographing light source when the color image is photographed is estimated for each of the divided small screens. , Create a histogram of the estimated color temperature for each small screen, group the small screens, again estimate the color temperature of the photographic light source for each group, and according to the color temperature estimation result for each group, A white balance correction amount is calculated for each group, and white balance correction is performed on each small screen in the group using the white balance correction amount calculated for each group.

JP 2002-271638 A

By the way, in the conventional so-called mixed light source image white balance correction technique in which shooting is performed with a plurality of different light sources, the input image is divided into a plurality of areas prior to the white balance correction, and each divided area is divided into a plurality of areas. For example, the color temperature of the photographing light source when the mixed light source image is photographed is estimated using a light source color temperature estimating method that is generally performed.
However, the color temperature of the light source is likely to be estimated at an incorrect color temperature in an area that hits a boundary between a plurality of light sources or an area that has many subjects of a specific color.
For example, as shown in FIG. 1, when an input image of a scene in which the lower left half of the screen is sunny and the upper right half is shaded is divided into 6 × 6 areas, the color temperature of the light source in each area is estimated. The color temperature of the light source is likely to be erroneously estimated in an area that is close to the diagonal line from the upper left to the lower right, where the shade and the sun are mixed, or in an area where the entire lower left is almost lawn.
When the color temperature of a light source is estimated by a light source color temperature estimation method that is generally performed on an area of the entire surface of a lawn, the color temperature of the light source is likely to be erroneously estimated as artificial lighting such as a fluorescent lamp. If the white balance is corrected using the corresponding white balance correction gain based on the color temperature of the light source that is erroneously estimated in this way, the balance between the color of the area and the color of the adjacent area is greatly lost, and the entire image May result in a very unnatural hue.

  However, in the method described in Patent Document 1, after estimating the color temperature of the light source for each small screen, the small screens with the estimated color temperature of the light source are sorted into a plurality of groups, and again for each sorted group. Since the color temperature of the light source is estimated, areas that belong to the same group are not estimated to have different color temperatures, and there is less change in color temperature between areas, and white that uses the re-estimated color temperature of the light source. The balance correction can prevent the entire image from having a very unnatural hue. However, when the estimation of the color temperature of the light source of the small screen is wrong and the estimated color temperature of the light source is significantly different from the color temperature of the original light source, the small screen is not sorted into the group to which it originally belongs. For this reason, in the method described in Patent Document 1, in an area where the color temperature of the light source of the small screen is wrongly estimated, the white balance correction gain corresponding to the color temperature of the light source estimated incorrectly is used. As a result of correcting the above, it is impossible to cope with the problem that the color balance of the area is greatly lost and the color becomes unnatural locally.

  The present invention has been made in view of the above problems in the prior art, and when dividing the entire captured image into a plurality of areas and estimating the color temperature of the light source in each area, the boundary portion between the plurality of light sources Multi-area white balance control device that can reduce the estimation error of the color temperature of the light source in individual areas such as areas where there are many objects of a specific color , Multi-area white balance control method, multi-area white balance control program, computer recording multi-area white balance control program, multi-area white balance image processing apparatus, multi-area white balance image processing method, multi-area white balance image processing program, Multi-area why And its object is to provide an imaging apparatus having recorded a computer and multi-area white balance an image processing apparatus to balance image processing program.

In order to achieve the above object, a multi-area white balance control apparatus according to the present invention estimates area color means for dividing an input image into a plurality of areas and the color temperature of the light source for each area divided by the area division means. The color temperature of the light source estimated by the first area light source color temperature estimation unit and the first area light source color temperature estimation unit is counted for each type, and the color of some types of light sources having the higher count number Among the areas divided by the color temperature estimation target light source extraction means and the area dividing means, the color temperature of the light source estimated by the first area light source color temperature estimation means is the color temperature estimation. targeting only areas that do not correspond to any of the color temperature of a part narrowed down the target light source extracting means types of light sources, the color temperature estimated color temperature of the light source Second area light source color temperature estimating means for estimating one of the color temperatures of some types of light sources narrowed down by the target light source extracting means; and each area estimated by the second area light source color temperature estimating means. And white balance correction gain map generation means for generating a white balance correction gain map of the entire input image based on the color temperature of the light source.

Further, the multi-area white balance control method according to the present invention divides the input image into a plurality of areas, estimates the color temperature of the light source for each divided area , counts the estimated color temperature of the light source for each type, Narrow down the color temperature of some types of light sources with higher counts , and the estimated color temperature of the light source in each divided area corresponds to any of the color temperatures of some of the narrowed types of light sources The target color temperature of the light source is re-estimated to one of the narrowed-down light source color temperatures, and the entire input image is based on the re-estimated color temperature of the light source of each area. The white balance correction gain map is generated.

Further, a computer that records a multi-area white balance control program according to the present invention is a computer provided in a multi-area white balance control device, wherein the computer divides an input image into a plurality of areas, the area division First area light source color temperature estimating means for estimating the color temperature of the light source for each area divided by the means, counting the color temperature of the light source estimated by the first area light source color temperature estimating means for each type, Of the respective areas divided by the color temperature estimation target light source extraction means and the area division means, which are narrowed down to the color temperatures of some types of light sources with higher counts , the first area light source color temperature estimation means. The estimated color temperature of the light source is a part of the color temperature estimation target light source extraction means narrowed down. Targeting only areas that do not correspond to any of the color temperature of the kind of light source, the color temperature of the light source, to estimate any of the color temperature estimation of target light source extracting means partially narrowed down type of light source color temperature White balance correction for generating a white balance correction gain map of the entire input image based on the color temperature of the light source in each area estimated by the second area light source color temperature estimation means and the second area light source color temperature estimation means A multi-area white balance control program for functioning as a gain map generating means is recorded.

A multi-area white balance control program according to the present invention is a computer-readable multi-area white balance control program provided in a multi-area white balance control device, wherein the computer divides an input image into a plurality of areas. dividing means, the first area light source color temperature estimating means for estimating the color temperature of the light source for each area divided by the area dividing means, the color temperature of the light source estimated by the first area light source color temperature estimating means The first area among the areas divided by the color temperature estimation target light source extraction unit and the area division unit that counts for each type and narrows down to the color temperatures of some types of light sources with higher counts. The color temperature of the light source estimated by the light source color temperature estimation means is the color temperature estimation. Targeting only areas that do not correspond to any of the color temperature of a part narrowed down the target light source extracting means type of light source, the color temperature of the light source, a portion narrowed down is the color temperature estimation target light source extracting means type A second area light source color temperature estimator for estimating one of the color temperatures of the light source, and a white color of the entire input image based on the color temperature of the light source in each area estimated by the second area light source color temperature estimator It is characterized by functioning as white balance correction gain map generation means for generating a balance correction gain map.

  According to the present invention, when the entire captured image is divided into a plurality of areas and the color temperature of each area is estimated, individual areas such as an area hitting the boundary of a plurality of light sources or an area with a large number of subjects of a specific color are used. A multi-area white balance control device, a multi-area white balance control method, a multi-area white balance control program capable of reducing the estimation error of the color temperature of the light source in the area, and realizing color reproduction without any sense of incongruity in the entire image, Computer recording multi-area white balance control program, multi-area white balance image processing apparatus, multi-area white balance image processing method, multi-area white balance image processing program, computer recording multi-area white balance image processing program, and An image pickup apparatus having a Ruchi area white balance image processing device can be obtained.

It is explanatory drawing which shows an example which divided | segmented the input image into the several area. 1 is a block diagram of a digital camera including a multi-area white balance control device according to a first embodiment of the present invention. 3 is a flowchart showing a processing procedure until correction of white balance of an input image in the multi-area white balance control device of FIG. 2. When estimating the color temperature of the light source from the color distribution of the subject in the (R / G, B / G) chromaticity diagram, all the pixels in the image data are plotted on the chromaticity diagram, and the chromaticity diagram in advance. It is a graph which shows an example of a chromaticity diagram and an area determination standard for determining the color temperature of the light source about the plotted point according to the set color temperature area determination standard of various light sources. In the multi-area white balance control device of FIG. 2, the input image is divided into areas and is an explanatory diagram showing an example of the color temperature of the light source estimated for each area by the first area light source color temperature estimation means. In the multi-area white balance control apparatus of FIG. 2, it is explanatory drawing which shows an example of the color temperature of the light source estimated for every area by the 2nd area light source color temperature estimation means. It is a block diagram of the digital camera provided with the multi-area white balance control apparatus concerning 2nd Embodiment of this invention. It is a flowchart which shows the process sequence until it correct | amends the white balance of the input image in the multi-area white balance control apparatus of FIG. FIG. 8 is an explanatory diagram showing an example in which the first area dividing unit divides an input image into a plurality of first areas in the multi-area white balance control device of FIG. 7. In the multi-area white balance control apparatus of FIG. 7, it is explanatory drawing which shows an example of the color temperature of the light source estimated for every 1st area shown in FIG. 9 by the 1st area light source color temperature estimation means. FIG. 8 is an explanatory diagram showing an example in which the second area dividing unit divides an input image into a plurality of second areas in the multi-area white balance control device of FIG. 7. In the multi-area white balance control apparatus of FIG. 7, it is explanatory drawing which shows an example of the color temperature of the light source estimated for every 2nd area shown in FIG. 11 by the 2nd area light source color temperature estimation means. It is a flowchart which shows the process sequence until it correct | amends the white balance of the input image concerning the modification of the multi-area white balance control apparatus of 2nd Embodiment. In the multi-area white balance control device according to the modification of the multi-area white balance control device of the second embodiment, an example of the color temperature of the light source estimated for each first area by the first area light source color temperature estimation means It is explanatory drawing which shows. In the multi-area white balance control device according to the modification of the multi-area white balance control device of the second embodiment, the color temperature of the light source estimated for each first area by the first area light source color temperature estimation means is shown in FIG. In such a case, it is explanatory drawing which shows the state which re-estimated the color temperature of the light source of the whole 1st area.

Prior to the description of the embodiments, the effects of the present invention will be described.
The present invention divides an input image into a plurality of areas, estimates the color temperature of the light source for each divided area , counts the estimated color temperature of the light source for each type, and counts the upper part of the count number . Only the areas where the estimated color temperature of the selected light source does not correspond to any of the color temperatures of some of the narrowed-down light sources among the divided areas are narrowed down to the color temperature of the type of light source. Re-estimate the color temperature to one of the narrowed-down types of light sources, and generate a white balance correction gain map for the entire input image based on the re-estimated color temperatures of the light sources in each area I have to.

As in the present invention, the estimated color temperature of the light source is counted for each type, the color temperature of some types of light sources with the highest count is narrowed down, and the estimated light source of each estimated area is divided . Estimate the color temperature of the light source to one of the narrowed types of light sources only in areas where the color temperature does not correspond to any of the narrowed types of light sources. If corrected, even if the color temperature of the area estimated at the beginning is wrong, the color temperature of the light source estimated in the area can be estimated again to a more appropriate color temperature. As a result, a white balance correction gain map of the entire input image can be generated based on the color temperature of the light source re-estimated to an appropriate color temperature of each area, and the entire input image can be generated based on the generated white balance correction map. When the white balance is corrected, the color balance of the area in the entire input image is properly maintained, and color reproduction without a sense of incongruity can be realized.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention according to the claims.

First Embodiment FIG. 2 is a block diagram of a digital camera provided with a multi-area AWB (auto white balance) control device according to a first embodiment of the present invention. In FIG. 2, a portion surrounded by a one-dot chain line corresponds to the multi-area white balance control device according to the present embodiment, and a portion surrounded by a two-dot difference line corresponds to the multi-area white balance image processing device. FIG. 3 is a flowchart showing a processing procedure until the white balance of the input image is corrected in the multi-area white balance control apparatus of FIG.

  As shown in FIG. 2, the multi-area white balance control device 1 according to the present embodiment includes an image area dividing unit 11 as an area dividing unit and a first area color temperature estimating unit as a first area light source color temperature estimating unit. Unit 12, color temperature estimation target light source extraction unit 13 as color temperature estimation target light source extraction unit, second area light source color temperature estimation unit second area color temperature estimation unit 14, and white balance correction gain map generation unit As a WB (white balance) gain map generation unit 15. In FIG. 2, 10 is a multi-area white balance image processing device, 16 is a gain map storage unit that stores a white balance correction gain map generated by a WB (white balance) gain map generation unit 15, and 17 is a white balance processing unit. A WB (white balance) gain correction unit, 21 is a lens, 22 is an image sensor, and 23 is an image storage unit that stores an image captured by the image sensor 22. Note that the digital camera including the multi-area white balance image processing apparatus 10 according to the present embodiment includes a control unit (not shown). The control unit is separately provided with a function of storing an image captured by the image sensor 22 based on a shooting instruction in the image storage unit 23 and a function of inputting an image stored in the image storage unit 23. In the example of FIG. 2, an image formed on the image sensor 22 through the lens 21 is picked up and stored as a digital signal in the image storage unit 23 in accordance with a shooting instruction from a control unit (not shown).

  The image area dividing unit 11 divides the input image into a plurality of blocks having a predetermined size.

  The first area color temperature estimation unit 12 estimates the color temperature of the light source for each block divided by the image area division unit 11. Details of the method for estimating the color temperature of the light source will be described later.

The color temperature estimation target light source extraction unit 13 narrows down the color temperatures of some types of light sources out of the color temperatures of the light sources estimated by the first area color temperature estimation unit 12. Specifically, the color temperature estimation target light source extraction unit 13 counts the color temperatures of the light sources of all blocks estimated by the first area color temperature estimation unit 12 for each type, and determines the main light sources included in the entire input image. As the types of color temperatures, the color temperatures of some types (here, the top two types) of light sources whose count number exceeds a predetermined threshold (for example, the count number is 15% of the total number of blocks) are extracted. Here, for convenience, the color temperature types of light sources extracted by the color temperature estimation target light source extraction unit 13 are the top two types, but the color temperature types of light sources extracted by the color temperature estimation target light source extraction unit 13 are When there are two or more types of color temperatures of light sources whose count number exceeds a predetermined threshold, the color temperatures of types of light sources exceeding two types may be extracted. In addition, the color temperature estimation target light source extraction unit 13 regards counting for each type of color temperature of the light source, and excludes blocks for which the color temperature of the light source cannot be estimated by the first area color temperature estimation unit 12 from being counted. It is configured as follows.
The second area color temperature estimator 14 includes a light source color temperature estimated by the first area color temperature estimator 12 among the blocks divided by the image area divider 11 , and a color temperature estimation target light source. For only blocks that do not correspond to any of the color temperatures of the top two types of light sources narrowed down by the extraction unit 13, any one of the color temperatures of the top two types of light sources narrowed down by the color temperature estimation target light source extraction unit 13 Estimate the color temperature of the light source.
The details of the method for extracting the color temperature of the main light source and the method for estimating the color temperature of the light source will be described later.

  The WB gain map generation unit 15 generates a white balance correction gain map of the entire input image based on the color temperature of the light source of each block estimated by the second area color temperature estimation unit 14.

  The WB gain correction unit 17 corrects the white balance of the entire input image based on the white balance correction gain map generated by the WB gain map generation unit 15.

  The flow of processing up to correcting the white balance of the input image using the multi-area white balance control device of this embodiment configured as described above will be described with reference to FIGS.

First, a control unit of a digital camera (not shown) inputs image data stored in the image storage unit 23 via an output from the image sensor 22 (step S1).
Next, the image area dividing unit 11 divides the input image data into a plurality of blocks having a predetermined size (step S2). For convenience, in the following description, the image data is divided into 6 × 6 blocks.

  Next, the first area color temperature estimation unit 12 estimates the color temperature of the light source for each block with respect to the image of each block divided by the image area division unit 11 by, for example, the following method. (Step S3).

  For example, it is assumed that the effective pixels of the image sensor are 4800 × 3600 pixels. The input Bayer pattern RAW image is divided into 6 × 6 blocks by the image area dividing unit 11, and the size of each block is 800 × 600 pixels.

  Prior to the estimation of the color temperature of the light source, for example, a 20 × 20 pixel range is set as one sampling point for the 800 × 600 pixel block, and R, G, By obtaining an average value of the pixel values of B, R, G, and B pixel data having a sampling point of 40 × 30 pixels down-sampled are obtained.

  The chromaticity values (R / G, B / G) are obtained for the sampling points of 40 × 30 pixels, and the chromaticity values at the respective sampling points are plotted on, for example, the chromaticity diagram shown in FIG. Then, for example, using the color temperature area determination criteria of various light sources on the chromaticity diagram as shown in FIG. 4, the area determination of the color temperature of the light source for the points where the chromaticity values of all the sampling points are plotted. And the number of plotted points belonging to the color temperature area of each light source is totaled. The color temperature of the light source of the block is estimated from the order of the number of plotted points belonging to the aggregated color temperature area of each light source, for example, according to the conditions shown in Table 1 below.

  The first area color temperature estimation unit 12 estimates the color temperature of the light source using the above method for all 36 blocks. Thereby, for example, the estimation result of the color temperature of the light source as shown in FIG. 5 is obtained for the entire image.

  After the area color temperature estimation unit 12 finishes estimating the color temperature of the light source for all the blocks, the color temperature estimation target light source extraction unit 13 performs the 36 blocks estimated by the first area color temperature estimation unit 12. The color temperature of the light source is counted by type. In the example of FIG. 5, regarding the type of color temperature of the light source counted by the color temperature estimation target light source extraction unit 13, 13 blocks estimated as “A” (Hyuga) were estimated as “B” (shade). There are 14 blocks, 4 blocks estimated as “C” (fluorescent lamp), and 5 blocks estimated as “D” (other). Therefore, the color temperature estimation target light source extraction unit 13 has some types (here, the top two types) in which the count number exceeds the predetermined threshold value (for example, the count number is 15% of the total number of blocks) in the above 36 blocks. "B" (shade) and "A" (Hyuga) are extracted as the color temperature types of the light source (step S4).

Subsequently, the second area color temperature estimation unit 14, among the 36 pieces of each block divided by the image area dividing portion 11, the color temperature of the light source estimated by the first area color temperature estimating part 12 The color temperature estimation target light source extraction unit 13 re-estimates the color temperature of the light source by, for example, the following method with respect to an image of a block that does not correspond to any of the color temperatures of the top two types of light sources. (Step S5). In this case, the color temperature of one of the top two types of light sources narrowed down by the color temperature estimation target light source extraction unit 13, that is, either “B” (shade) or “A” (Hyuga) in the example of FIG. To estimate.

  For example, assuming that the effective pixels of the image sensor are 4800 × 3600 pixels, the input RAW image of the Bayer pattern is divided into 6 × 6 blocks by the image area dividing unit 11, and the size of each block is 800 ×. It is 600 pixels.

  Prior to the estimation of the color temperature of the light source, for example, a 20 × 20 pixel range is set as one sampling point for the 800 × 600 pixel block, and R, G, By obtaining an average value of the pixel values of B, R, G, and B pixel data having a sampling point of 40 × 30 pixels down-sampled are obtained.

  The chromaticity values (R / G, B / G) are obtained for the sampling points of 40 × 30 pixels, and the chromaticity values at the respective sampling points are plotted on, for example, the chromaticity diagram shown in FIG.

Next, for example, as shown in FIG. 4, the color temperature areas of the shade (area 1: shade) and the sun (area 3: clear sky) are determined using the color temperature area determination criteria of various light sources on the chromaticity diagram. Aggregate the number of plotted points to which they belong. Determine whether the centroid of the plotted points belonging to the shaded and sunny color temperature areas is in the shaded or sunny color temperature area, and determine the color temperature of the area with the centroid as the color of the light source of the block Estimated as temperature.
Thereby, for example, the estimation result of the color temperature of the light source as shown in FIG. 6 is obtained for the entire image.

  When the color temperature of the light source of all the blocks is estimated by the first area color temperature estimation unit 12, the color of the light source in most blocks estimated by the first area color temperature estimation unit 12 compared with the actual visual result The temperature will be correct but may be incorrect for some blocks. For example, (0,0), (1,1), (2,2), (2,3), (3,3), (4,3), (4,4), (4) shown in FIG. , 5) and (5, 5) are blocks located at the boundary between “B” (shade) and “A” (Hyuga), the first area color temperature estimator 12 uses the light source It is easy to mistake the estimation of the color temperature for “C” (fluorescent lamp) or “D” (others).

However, in the multi-area white balance control device 1 according to the present embodiment, the color temperature estimation target light source extraction unit 13 counts the color temperature of the light source estimated by the first area color temperature estimation unit 12 for each type. The second area color temperature estimator 14 narrows down to the color temperatures of some types of light sources with higher numbers , and the second area color temperature estimator 14 is the first area color temperature estimator of the blocks divided by the area divider 11. The color temperature of the light source is calculated only for the area where the color temperature of the light source estimated by 12 does not correspond to any of the color temperatures of some types of light sources extracted by the color temperature estimation target light source extraction unit 13. The temperature estimation target light source extraction unit 13 estimates to one of the color temperatures of some types of light sources narrowed down, and as shown in FIG. 6, a block surrounded by a circle (that is, (0 0), (1,1), (2,2), (2,3), (3,3), (4,3), (4,4), (4,5), (5,5) The color temperature of the light source of each block) is the correct color of the light source such as “B” (shade) or “A” (Hyuga) from the wrong color temperature to “C” (fluorescent lamp) or “D” (other) Estimated to temperature.

The color temperature of the light source estimated by the first area color temperature estimation unit 12 by the second area color temperature estimation unit 14 is the color temperature of some types of light sources narrowed down by the color temperature estimation target light source extraction unit 13. After estimating the color temperature of the light source only for the block that does not correspond to any one of the color temperatures of some types of light sources narrowed down by the color temperature estimation target light source extraction unit 13, the WB gain map The generation unit 15 refers to R and B gain data corresponding to a predetermined color temperature of a light source (not shown) based on the color temperature of the light source of each block reestimated shown in FIG. A WB correction gain map corresponding to R and B is generated (step S6). The WB correction gain map generated by the WB gain map generation unit 15 is stored (saved) in the gain map storage unit 16.

  Then, the WB gain correction unit 17 corrects the white balance by applying the gain obtained by inputting the gain map storage unit 16 to the pixel value of each color corresponding to the position of the input image, thereby correcting the image. Output (step S7).

According to multi-area white balance control apparatus 1 and the multi-area white balance control method of the present embodiment, estimated, of the color temperature of the light source, narrowing the color temperature of some types of light sources count becomes higher, Of the divided areas , only the areas where the estimated color temperature of the light source does not correspond to any of the color temperatures of some types of light sources that have been narrowed down, and the color temperature of the light source is limited Since the color temperature of one type of light source is re-estimated, the color temperature of that area estimated in an individual area such as an area that hits the boundary between multiple light sources or an area with many subjects of a specific color Even if it is wrong, the color temperature of the light source estimated in the area can be estimated again to a more appropriate color temperature. As a result, a white balance correction gain map of the entire input image can be generated based on the color temperature of the light source re-estimated to an appropriate color temperature of each area, and the entire input image can be generated based on the generated white balance correction map. When the white balance is corrected, the color balance of the area in the entire input image is properly maintained, and color reproduction without a sense of incongruity can be realized.

In the above example, the second area color temperature estimator 14 selects some types of the 36 areas divided by the image area divider 11 and the color temperature estimation target light source extractor 13 narrows down. The color temperature of the light source is estimated to one of the color temperatures of the light source, and in FIG. 5, the light source of a different type from the color temperature of the main light source (“B” (shade), “A” (Hyuga)) color temperature ( "C" (fluorescent lamp), "D" (other)) with respect to only going on blocks, intends rows estimated color temperature again of the light source. That is, in the multi-area white balance control device 1 according to the present embodiment, the second area color temperature estimation unit 14 uses the first area color temperature estimation unit 12 among the blocks divided by the area division unit 11. The color temperature of the light source is estimated only for blocks whose color temperature of the estimated light source does not correspond to any of the color temperatures of some types of light sources extracted by the color temperature estimation target light source extraction means 13. you. In this way, overlapping processes can be omitted, and the speed of the entire multi-area white balance process can be increased.

Second Embodiment FIG. 7 is a block diagram of a digital camera provided with a multi-area AWB (auto white balance) control device according to a second embodiment of the present invention. In FIG. 7, a portion surrounded by a one-dot chain line corresponds to the multi-area white balance control device according to this embodiment, and a portion surrounded by a two-dot difference line corresponds to the multi-area white balance image processing device. FIG. 8 is a flowchart showing a processing procedure until the white balance of the input image is corrected in the multi-area white balance control device of FIG.

  As shown in FIG. 7, the multi-area white balance control apparatus 1 of the present embodiment includes an image area dividing unit 11 as a first area dividing unit and a first area as a first area light source color temperature estimating unit. Color temperature estimation unit 12, color temperature estimation target light source extraction unit 13 as color temperature estimation target light source extraction means, second area light source color temperature estimation means second area color temperature estimation unit 14, and white balance correction gain It has a WB (white balance) gain map generation unit 15 as a map generation unit and an image area division unit 18 as a second area division unit, and the area division unit is a first area division unit. A first image area dividing unit 11 and a second image area dividing unit 18 as second area dividing means are provided. In FIG. 7, 10 is a multi-area white balance image processing apparatus, 16 is a gain map storage unit that stores a white balance correction gain map generated by a WB (white balance) gain map generation unit 15, and 17 is a white balance processing unit. A WB (white balance) gain correction unit, 21 is a lens, 22 is an image sensor, and 23 is an image storage unit that stores an image captured by the image sensor 22.

The first image area dividing unit 11 divides the input image into a plurality of blocks having a predetermined first size (for example, 9 blocks of 3 × 3 as shown in FIG. 9).
The second image area dividing unit 18 divides the input image into a plurality of blocks having a predetermined second size smaller than the first size (for example, 36 blocks of 6 × 6 as shown in FIG. 11). To divide.

  The first area color temperature estimation unit 12 estimates the color temperature of the light source for each block divided by the first image area division unit 11.

The second area color temperature estimation unit 14 determines the color temperature of the light source estimated by the first area color temperature estimation unit 12 among the blocks divided by the second image area division unit 18. Which of the color temperatures of the top two types of light sources narrowed down by the color temperature estimation target light source extraction unit 13 is targeted only for blocks that do not correspond to any of the color temperatures of the top two types of light sources narrowed down by the estimation target light source extraction unit 13 In addition, the color temperature of the light source is estimated.

  Other configurations are substantially the same as those of the multi-area white balance control device 1 of the first embodiment.

  The flow of processing up to correcting the white balance of the input image using the multi-area white balance control device of the present embodiment configured as described above will be described with reference to FIGS.

First, a control unit of a digital camera (not shown) inputs image data stored in the image storage unit 23 via an output from the image sensor 22 (step S11).
Next, the first image area dividing unit 11 divides the input image data into 9 blocks of 3 × 3 having the first size as shown in FIG. 9 (step S12).

  Next, the first area color temperature estimation unit 12 performs the same processing as the first area color temperature estimation unit 12 in the first embodiment on the image of each block divided by the first image area division unit 11. The color temperature of the light source for each block is estimated by the method for estimating the color temperature of the light source (step S13). Thereby, for example, an estimation result of the color temperature of the light source as shown in FIG. 10 is obtained for the entire image.

  After the area color temperature estimation unit 12 finishes estimating the color temperature of the light source for all the blocks, the color temperature estimation target light source extraction unit 13 performs the above nine blocks estimated by the first area color temperature estimation unit 12. The color temperature of the light source is counted by type. In the example of FIG. 10, regarding the type of the color temperature of the light source counted by the color temperature estimation target light source extraction unit 13, four blocks estimated as “A” (Hyuga) were estimated as “B” (shade). There are three blocks, one block estimated as “C” (fluorescent lamp), and one block estimated as “D” (others). Therefore, the color temperature estimation target light source extraction unit 13 uses some types (here, the top two types) in which the count number exceeds the predetermined threshold value (for example, the count number is 15% of the total block number) in the nine blocks. "A" (Hyuga) and "B" (shade) are detected as the color temperature types of the light source (step S14).

  Subsequently, the second image area dividing unit 18 divides the input image data into 36 blocks of 6 × 6 of the second size as shown in FIG. 11 (step S15).

  Subsequently, the second area color temperature estimation unit 14 applies the second area color temperature estimation unit in the first embodiment to the 36 blocks of images divided by the second image area division unit 18. The color temperature of the light source for each block is estimated again by the same method for estimating the color temperature of the light source as in step 14 (step S16). In this case, the color temperature of one of the top two types of light sources narrowed down by the color temperature estimation target light source extraction unit 13, that is, either “A” (Hyuga) or “B” (shade) in the example of FIG. To estimate. Thereby, for example, an estimation result of the color temperature of the light source as shown in FIG. 12 is obtained for the entire image.

  When the color temperature of the light source of all the blocks is estimated by the first area color temperature estimation unit 12, the color of the light source in most blocks estimated by the first area color temperature estimation unit 12 compared with the actual visual result The temperature will be correct but may be incorrect for some blocks. For example, the blocks (1, 1) and (2, 2) shown in FIG. 10 are blocks located at the boundary between “A” (Hyuga) and “B” (shade). The color temperature estimation unit 12 is likely to mistake the estimation of the color temperature of the light source as “D” (others) or “C” (fluorescent lamp).

However, in the multi-area white balance control device 1 according to the present embodiment, the color temperature estimation target light source extraction unit 13 counts the color temperature of the light source estimated by the first area color temperature estimation unit 12 for each type. The second area color temperature estimation unit 14 narrows down to the color temperatures of some types of light sources with higher numbers , and the second area color temperature estimation unit 14 uses the first area color among the blocks divided by the second area dividing unit 17. The color temperature of the light source is estimated only for blocks in which the color temperature of the light source estimated by the temperature estimation unit 12 does not correspond to any of the color temperatures of some types of light sources extracted by the color temperature estimation target light source extraction unit 13. Is estimated to one of the color temperatures of some types of light sources narrowed down by the color temperature estimation target light source extraction unit 13, as shown in FIG. 12, the block (1, 1) in FIG. Corresponding blocks (2, 2), (2, 3), (3, 2), (3, 3), and blocks (4, 4), (4) corresponding to block (2, 2) in FIG. , 5), (5, 4), (5, 5), the color temperature of the light source from “D” (other) or “C” (fluorescent light) was changed from “A” (Hyuga) or It is re-estimated to the correct color temperature of the light source such as “B” (shade).

  After estimating the color temperature of the light source for each block by the second area color temperature estimation unit 14 to one of the color temperatures of some types of light sources narrowed down by the color temperature estimation target light source extraction unit 13, the WB gain Based on the color temperature of the light source of each reestimated block shown in FIG. 12, the map generation unit 15 refers to R and B gain data corresponding to a predetermined color temperature of the light source (not shown). , R and B corresponding to WB correction gain maps are generated (step S17). The WB correction gain map generated by the WB gain map generation unit 15 is stored (saved) in the gain map storage unit 16.

  Then, the WB gain correction unit 17 corrects the white balance by applying the gain obtained by inputting the gain map storage unit 16 to the pixel value of each color corresponding to the position of the input image, thereby correcting the image. Output (step S18).

According to the multi-area white balance control device 1 and multi-area white balance control method of the present embodiment, the first area dividing unit 11 increases the first size to be divided into a plurality of blocks, and the number of blocks to be divided is minimized. By reducing the number, it is possible to reduce the number of times the color temperature of the light source is estimated by the first area color temperature estimation unit 12 and to speed up the entire multi-area white balance process.
Other functions and effects are substantially the same as those of the multi-area white balance control device 1 and the multi-area white balance control method of the first embodiment.

  In the multi-area white balance control device of this embodiment, when the first area color temperature estimation unit 12 has approximately one type of color temperature of the light source estimated for each first area. It can be determined that the color temperature of the light source of the entire input image is one type. Therefore, in such a case, it is desirable to perform processing in the following procedure.

  FIG. 13 is a flowchart showing a processing procedure until the white balance of the input image according to the modification of the multi-area white balance control device of the second embodiment is corrected.

  FIG. 14 shows, for each type, the color temperature of the light source estimated by the first area color temperature estimation unit 12 by the color temperature estimation target light source extraction unit 13 in the multi-area white balance control device 1 of the modification of the second embodiment. The type of color temperature of the light source with the highest count is counted as “A” (Hyuga) only. Specifically, in the example of FIG. 14, regarding the type of color temperature of the light source counted by the color temperature estimation target light source extraction unit 13, seven blocks estimated as “A” (Hyuga) and “B” (shade) are used. There are one estimated block, one block estimated as “C” (fluorescent lamp), and zero block estimated as “D” (other), and a predetermined threshold (for example, count) The type of color temperature of the light source, the number of which exceeds 15% of the total number of blocks), is only one type of “A” (Hyuga).

In such a case, in this modification, the color temperature of the light source estimated by the first area color temperature estimation unit 12 is a color temperature of a light source other than “A” (Hyuga) (here, “B” (shade). ), It is determined that the estimation of the color temperature of the light source of the block estimated at “C” (fluorescent lamp)) is incorrect, and the first area color temperature estimation unit 12 is informed as shown in FIG. The color temperature of the light source of all blocks is re-estimated to “A” (Hyuga) (step S30).
Then, the WB gain map generation unit 15 performs the re-estimation of the light source of each block shown in FIG. 15 without undergoing the processing by the second image area dividing unit 18 and the second area color temperature estimation unit 14. WB correction gain map is generated with reference to the R and B gain data corresponding to the predetermined color temperature of the light source (not shown) based on the color temperature (step S28).

  Other configurations and processing procedures are substantially the same as those of the multi-area white balance control device of the second embodiment.

According to the modification of the second embodiment, the color temperature estimation target light source extraction unit 13 counts the color temperature of the light source estimated by the first area color temperature estimation unit 12 for each type. The second image area dividing unit 18 and the second area color temperature estimating unit when the type of the color temperature of the light source exceeds a predetermined threshold (for example, the count number is 15% of the number of blocks in the figure). 14 can be omitted, and the speed of the entire multi-area white balance process can be further increased.
Other functions and effects are substantially the same as those of the multi-area white balance control device 1 and the multi-area white balance control method according to the second embodiment described with reference to FIGS.

  The multi-area white balance control device 1 and the multi-area white balance control method of the present invention have been described above. In the multi-area white balance control device 1 of each of the above embodiments, the image area dividing unit 11 as the area dividing means, the first In the example described above, the image area dividing unit 18 divides the input image into blocks having the same shape and area divided by vertical and horizontal straight lines. However, the area dividing means in the multi-area white balance control device 1 of the present invention is described below. The shape and size of each area to be divided is not limited to this embodiment, and may be any shape such as a triangular shape or a curved shape, and the size of each area may be different.

Furthermore, the multi-area white balance image processing apparatus 10 of each of the above embodiments includes a computer (first) image area dividing unit 11 ((first) area dividing means) (and second image area dividing unit 18). (Second area dividing unit)), first area color temperature estimating unit 12 (first area light source color temperature estimating unit), color temperature estimating target light source extracting unit 13 (color temperature estimating target light source extracting unit), first 2 area color temperature estimator 14 (second area light source color temperature estimator), WB gain map generator 15 (white balance correction gain map generator), and WB gain corrector 17 (white balance processor). A multi-area white balance image processing program may be configured by a computer recorded on a hard disk.
Further alternatively, the multi-area white balance image processing apparatus 10 according to the present embodiment includes a computer (first) image area dividing unit 11 ((first) area dividing unit) (and second image area dividing unit). 18 (second area dividing means)), a first area color temperature estimation unit 12 (first area light source color temperature estimation means), a color temperature estimation target light source extraction unit 13 (color temperature estimation target light source extraction means), Functions as a second area color temperature estimation unit 14 (second area light source color temperature estimation unit), a WB gain map generation unit 15 (white balance correction gain map generation unit), and a WB gain correction unit 17 (white balance processing unit). A multi-area white balance image processing program for recording, for example, a computer-readable medium such as a CD-ROM Good.

  Multi-area white balance control apparatus, multi-area white balance control method, multi-area white balance control program, computer recording multi-area white balance control program, multi-area white balance image processing apparatus, multi-area white balance image processing method A multi-area white balance image processing program, a computer having a multi-area white balance image processing program recorded thereon, and an imaging apparatus having a multi-area white balance image processing apparatus are so-called mixed light source image white images captured with a plurality of different light sources. This is useful in fields where balance correction is required.

DESCRIPTION OF SYMBOLS 1 Multi area white balance control apparatus 10 Multi area white balance image processing apparatus 11 (1st) Area division means ((1st) image area division part)
12 1st area light source color temperature estimation means (1st area color temperature estimation part)
13 Color temperature estimation target light source extraction means (color temperature estimation target light source extraction unit)
14 2nd area light source color temperature estimation means (2nd area color temperature estimation part)
15 White balance correction gain map generation means (WB gain map generation unit)
16 Gain map storage unit 17 White balance processing means (WB gain correction unit)
18 Second area dividing means (second image area dividing unit)
21 Lens 22 Image sensor 23 Image storage unit

Claims (8)

Area dividing means for dividing the input image into a plurality of areas;
First area light source color temperature estimating means for estimating the color temperature of the light source for each area divided by the area dividing means;
A color temperature estimation target light source extraction unit that counts the color temperature of the light source estimated by the first area light source color temperature estimation unit for each type, and narrows down to the color temperature of some types of light sources with higher counts ; ,
Wherein within each of the areas divided by the area dividing means, the color temperature of the light source estimated by the first area light source color temperature estimating means, wherein the color temperature estimation target light source extracting means is part narrowed down type Secondly, the color temperature of the light source is estimated to one of the color temperatures of some types of light sources narrowed down by the color temperature estimation target light source extraction means , only for an area that does not correspond to any of the color temperatures of the light sources. Area light source color temperature estimation means,
White balance correction gain map generation means for generating a white balance correction gain map of the entire input image based on the color temperature of the light source in each area estimated by the second area light source color temperature estimation means. Multi-area white balance control device. The area dividing means includes first area dividing means for dividing an input image into a plurality of first areas, and second area dividing means for dividing the input image into a plurality of second areas,
The first area light source color temperature estimating means estimates the color temperature of the light source for each first area divided by the first area dividing means,
The second area light source color temperature estimator is a part of a type in which the color temperature estimation target light source extractor narrows down the color temperature of the light source for each second area divided by the second area divider. The multi-area white balance control apparatus according to claim 1, wherein the multi-area white balance control apparatus estimates the color temperature of any one of the light sources. The size of the first area, multi-area white balance control device according to claim 2, characterized in that different from the size of the second area. The multi-area white balance control device according to any one of claims 1 to 3 ,
Based on the white balance correction gain of each area generated by the white balance correction gain map generation unit provided in the multi-area white balance control device, the area division unit of the multi-area white balance control device sets the division target. A multi-area white balance image processing apparatus comprising: white balance processing means for performing white balance processing on the entire input image. An imaging apparatus comprising the multi-area white balance image processing apparatus according to claim 4 . Divide the input image into multiple areas,
Estimate the color temperature of the light source for each divided area,
Count the estimated color temperature of the light source for each type, narrow down to the color temperature of some types of light sources with higher counts ,
Of the divided areas , only the areas where the estimated color temperature of the light source does not correspond to any of the color temperatures of some types of light sources that have been narrowed down, and the color temperature of the light source is limited Reestimate one of the color temperatures of the different light sources,
A multi-area white balance control method, comprising: generating a white balance correction gain map of the entire input image based on the color temperature of the light source of each area re-estimated. A computer provided in a multi-area white balance control device,
The computer,
Area dividing means for dividing the input image into a plurality of areas;
First area light source color temperature estimating means for estimating the color temperature of the light source for each area divided by the area dividing means;
A color temperature estimation target light source extraction unit that counts the color temperature of the light source estimated by the first area light source color temperature estimation unit for each type, and narrows down to the color temperature of some types of light sources having a higher count number ;
Wherein within each of the areas divided by the area dividing means, the color temperature of the light source estimated by the first area light source color temperature estimating means, wherein the color temperature estimation target light source extracting means is part narrowed down type Secondly, the color temperature of the light source is estimated to one of the color temperatures of some types of light sources narrowed down by the color temperature estimation target light source extraction means , only for an area that does not correspond to any of the color temperatures of the light sources. Area light source color temperature estimation means,
A multi function for functioning as a white balance correction gain map generating unit that generates a white balance correction gain map of the entire input image based on the color temperature of the light source in each area estimated by the second area light source color temperature estimating unit. A computer that records the area white balance control program. A computer-readable multi-area white balance control program provided in the multi-area white balance control device,
The computer,
Area dividing means for dividing the input image into a plurality of areas;
First area light source color temperature estimating means for estimating the color temperature of the light source for each area divided by the area dividing means;
A color temperature estimation target light source extraction unit that counts the color temperature of the light source estimated by the first area light source color temperature estimation unit for each type, and narrows down to the color temperature of some types of light sources having a higher count number ;
Wherein within each of the areas divided by the area dividing means, the color temperature of the light source estimated by the first area light source color temperature estimating means, wherein the color temperature estimation target light source extracting means is part narrowed down type Secondly, the color temperature of the light source is estimated to one of the color temperatures of some types of light sources narrowed down by the color temperature estimation target light source extraction means , only for an area that does not correspond to any of the color temperatures of the light sources. Area light source color temperature estimation means,
A multi function for functioning as a white balance correction gain map generating unit that generates a white balance correction gain map of the entire input image based on the color temperature of the light source in each area estimated by the second area light source color temperature estimating unit. Area white balance control program.

Images