JP2008042644A - Color image pickup apparatus and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a color image pickup apparatus capable of enhancing a success rate of white balance control, and to provide an image processing program. <P>SOLUTION: This color image pickup apparatus includes: classification units (S16, S17) each for classifying a plurality of small regions in a color image photographing an object into a plurality of groups in accordance with their illumination colors; and a calculation unit which regards a weighted average color of the illumination color of each of the plurality of groups as an illumination color of the object and calculates a white balance controlled variable optimum for the illumination color as a white balance controlled variable for the color image. The calculation unit sets weighting of the illumination color of each of the plurality of groups in accordance with the region size and luminance of each of the plurality of groups (A', B', C'). <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a color imaging device such as a digital camera, and an image processing program for processing a color image taken by a digital camera or the like.

Automatic white balance control with a digital camera is considered difficult when the object is illuminated with two different types of illumination light, such as when a digital camera shoots the interior of a light bulb into which sunlight enters. Yes.
For this reason, Patent Document 1 discloses a technique for performing white balance control with a white balance control amount optimum for the illumination color, assuming that the weighted average color of the two types of illumination light is the illumination color of the entire field of view. Yes. The weight of each illumination color is set larger as the existence area of the object illuminated with the illumination color is wider. In this way, if the intermediate color of the two types of illumination light is regarded as the illumination color of the entire object field, color continuity can be provided between a plurality of images taken while gradually changing the framing.
JP-A-8-251620

However, in the technique described in Patent Document 1, if there are two subjects with different sizes and illumination colors in the field, even if the user places importance on the smaller subject, the large subject There is a high possibility that the lighting color will be emphasized. For this reason, white balance control that satisfies the user may not be performed.
SUMMARY An advantage of some aspects of the invention is that it provides a color imaging apparatus and an image processing program capable of increasing the success rate of white balance control.

  The color imaging device of the present invention includes a classification unit that classifies a plurality of small regions in a color image obtained by photographing an object scene into a plurality of groups according to the illumination color, and a weighted average color of each illumination color of the plurality of groups. A color imaging device including a calculation unit that regards the illumination color of the object scene and calculates a white balance control amount optimal for the illumination color as a white balance control amount for the color image, The weight of the illumination color of each of the plurality of groups is set according to the area size and brightness of each of the plurality of groups.

The calculation unit preferably sets the weight of each illumination color of the plurality of groups according to a determination result of the illumination type of the object scene that is performed before the color image is captured.
In addition, it is preferable that the calculation unit sets the weight of the illumination color of each of the plurality of groups according to the illumination type of the subject existing in the color image.
In addition, any one of the color imaging devices of the present invention further includes a limit unit that adjusts the calculated value so that the calculated value of the weighted average color or the calculated value of the white balance control amount falls within an allowable range, It is preferable that the limit unit weights and averages an allowable range for each of the plurality of groups with a weight equivalent to the weight, and performs the adjustment using a weighted average range obtained thereby.

In addition, it is preferable that any one of the color imaging devices of the present invention further includes a processing unit that performs white balance control on the color image with the calculated white balance control amount.
The image processing program of the present invention includes a classification procedure for classifying a plurality of small regions in a color image obtained by photographing an object scene into a plurality of groups according to the illumination color, and a weighted average of the illumination colors of each of the plurality of groups. A calculation procedure that regards a color as an illumination color of the object scene and calculates a white balance control amount optimal for the illumination color as a white balance control amount for the color image, the image processing program comprising: In the procedure, the weight of the illumination color of each of the plurality of groups is set according to the area size and brightness of each of the plurality of groups.

In the calculation procedure, it is preferable that the weight of the illumination color of each of the plurality of groups is set according to the determination result of the illumination type of the object scene that is performed before the color image is captured.
In the calculation procedure, it is preferable that the weight of the illumination color of each of the plurality of groups is set in accordance with the illumination type of the subject existing in the color image.
In addition, any one of the image processing programs of the present invention further includes a limit procedure for adjusting the calculated value so that the calculated value of the weighted average color or the calculated value of the white balance control amount falls within an allowable range, In the limit procedure, it is preferable that the tolerance range for each of the plurality of groups is weighted and averaged with a weight equivalent to the weight, and the adjustment is performed using a weighted average range obtained thereby.

  In addition, it is preferable that any one of the image processing programs of the present invention further includes a processing procedure for performing white balance control on the color image with the calculated white balance control amount.

  According to the present invention, a color imaging device and an image processing program capable of increasing the success rate of white balance control are realized.

[First Embodiment]
Hereinafter, a first embodiment of the present invention will be described. The present embodiment is an embodiment of a digital camera system.
First, the configuration of this system will be described.
FIG. 1 is a configuration diagram of the present system. As shown in FIG. 1, this system comprises a single-lens reflex digital camera body 1 and an interchangeable lens 2 attached thereto.

  The digital camera main body 1 stores a quick return mirror 11, a shutter 13, a color image pickup device 12 for taking an image, a photometry unit 20 for monitoring an object scene, a CPU 15 for controlling each part of the system, and an operation program for the CPU 15. ROM16, RAM17 used during operation of CPU15, operation buttons such as release button 19, frame memory 14 capable of storing color image signals for at least one frame, and reading / writing of data to / from card memory 18A attached to digital camera body 1 A recording unit 18 is provided.

  At the time of non-photographing, the light flux from the object scene enters the quick return mirror 11 through the interchangeable lens 2 and is reflected there, and is reflected by the pentagonal prism (not shown) or the lens 20A in the photometry unit 20. Is incident on the color image pickup device 20B, and an image of the object scene is formed in the vicinity of the image pickup surface. Since the color image sensor 20B is used for monitoring the object scene, the resolution is lower than that of the color image sensor 12 for photographing an image, and the number of pixels is 1005, for example. Normally, the color image pickup device 20B is continuously driven, and color image signals (hereinafter referred to as “pre-photographing image signals”) sequentially output from the color image pickup device 20B during the drive are sequentially sent to the CPU 15. Is done.

  At the time of shooting, when the quick return mirror 11 jumps up and the shutter 13 is opened, the light flux from the object scene enters the color image sensor 12 and forms an image of the object scene near the image pickup surface. The number of pixels of the color image sensor 12 is 12 million, for example. During this shooting, the color image sensor 12 is driven, and a color image signal for one frame (hereinafter referred to as “captured image signal”) output from the color image sensor 12 at this time is temporarily stored in the frame memory 14. Is done. The captured image signal is subjected to image processing such as white balance processing and compression processing by the CPU 15, and then recorded in the card memory 18 </ b> A via the recording unit 18.

In shooting, the CPU 15 captures various types of shooting information such as a spot photometry area and an AF range set by the user via the photometry unit 20 and the interchangeable lens 2. The shooting information is also recorded in the card memory 18A together with the shot image signal.
Moreover, although part or all of the above-described image processing and compression processing may be executed by an image processing circuit and compression processing circuit provided separately from the CPU 15, the following description will be made assuming that the CPU 15 executes them.

Next, the operation of the CPU 15 related to the white balance process will be described.
FIG. 2 is an operation flowchart of the CPU 15 related to the white balance processing. This operation program is written in the ROM 16 in advance.
As shown in FIG. 2, the CPU 15 takes in the pre-photographing image signal from the photometry unit 20 (step S11), extracts an evaluation value indicating the color scheme of the object scene from the pre-photographing image signal (step S12), and the evaluation value And a database prepared in advance, it is determined whether the illumination type of the object scene is highly likely to be natural light or is highly likely to be artificial light (fluorescent lamp or light bulb) (step S13). The determination of the illumination type in steps S11, S12, and S13 is repeated unless the release button 14 is fully pressed (NO in step S14). This determination is based on the assumption that there is only one type of illumination light that illuminates the object scene. The above-mentioned database contains information on various types of images with known illumination types. By examining the correlation between the color scheme of these images and the color scheme of the scene, the illumination type of the scene is determined. It becomes possible to do. This determination result (natural light / artificial light) is used in a later step S18.

  When the release button 14 is fully pressed (YES in step S14), the CPU 15 drives the shooting-related parts such as the quick return mirror 11, the shutter 13, and the color image sensor 12 to take a picture of the object scene, and outputs the taken image signal. Obtain (step S15). Further, the CPU 15 estimates the illumination color of the object scene at the time of photographing based on the photographed image signal (steps S16 to S18). This estimation is based on the premise that there are a plurality of types of illumination light for illuminating the object scene. Details of the estimation (steps S16 to S18) will be described later.

  Further, the CPU 15 calculates a white balance gain optimum for the estimated illumination color, that is, a white balance gain for eliminating the coloring of the achromatic object illuminated with the illumination color (step S19), and the white balance gain is calculated. White balance processing is performed by multiplying the captured image signal (step S20). The CPU 15 compresses the captured image signal after the white balance process, and further records it in the card memory 18A (step S21).

Next, the estimation procedure (steps S16 to S18) described above will be described in detail with reference to FIGS.
(Step S16)
The upper part of FIG. 3 shows a concept of an image of the object scene (photographed image I) indicated by the photographed image signal. CPU15 is small region A ₁ of equal area the captured image I, A _2, A _3, divided ..., they small areas A _1, A _2, A _3, of ... each color C _1, C _2, C ₃ ,... Are calculated. Color C _i of the small region A _i is the average color of the small region A _i. At this time, the luminances Y ₁ , Y ₂ , Y ₃ ... Of the small areas A ₁ , A ₂ , A ₃ ,. Luminance Y _i of the small region A _i is the average brightness of the small region A _i.

Subsequently, the CPU 15 determines whether each of the small areas A ₁ , A ₂ , A ₃ ,... Is an achromatic area based on the colors C ₁ , C ₂ , C ₃ ,. Is determined whether the illumination type corresponds to natural light, a fluorescent light, or a light bulb, and the small areas A ₁ , A ₂ , A ₃ ,. Natural light group, fluorescent light group, light bulb group). In that case, the color C _i of the small area A _i is the color range E _n of an achromatic object illuminated with natural light, the color range E _f of an achromatic object illuminated with a fluorescent light, and the color range E _f illuminated with a light bulb. It is checked whether it belongs to any one of the color ranges E ₁ of the chromatic object, and if it belongs to any one, it belongs to any one of the color ranges E _n , E _f , E ₁ .

(Step S17)
The CPU 15 classifies the small areas belonging to the natural light group (color range E _n ) into smaller sub-groups e ₁ , e ₂ , e ₃ ,. As a result of this classification, a color histogram of the natural light group (color range E _n ) is created as shown in FIG. The vertical axis in FIG.

Similarly, the CPU 15 classifies each small region belonging to the fluorescent lamp group (color range E _f ) into smaller sub-groups e ₁ , e ₂ , e ₃ ,. As a result of this classification, a color histogram of the fluorescent lamp group (color range E _f ) is created as shown in FIG.
Similarly, the CPU 15 classifies each small area included in the light bulb group (color range E ₁ ) into smaller groups e ₁ , e ₂ , e ₃ ,. As a result of this classification, a color histogram of the light bulb group (color range E ₁ ) is created as shown in FIG.

The classification into small groups in this step may be performed simultaneously with the classification into large groups in step S16, but here, for the sake of explanation, both classifications are performed in order.
Subsequently, the CPU 15 determines from the histogram of the natural light group (FIG. 3A) that the first small group en ₁ with the first region number F and the second small group _en 2 with the second region number F. And the color C _{n1 of} the first small group _{en 1} and the color C _{n2 of} the second small group _en 2 are calculated. Color C _n1 of the first small group e _n1 is the average color of the small areas belonging to the first small group e _n1, color C _n2 of the second small group e _n2 is small areas belonging to the second sub-group e _n2 Is the average color.

Then, the CPU 15 performs weighted averaging of the color C _{n1 of} the first small group _{en 1} and the color C _{n2 of} the second small group _{en 2} by the following expression (1n), and the weighted average color C _n obtained thereby is obtained. This is regarded as an illumination color of natural light that illuminates the object scene at the time of shooting (see FIG. 3 (A ′)).
C _n = W _n1 C _n1 + W _n2 C _n2 (1n)
However, in the formula (1n), the colors C _n , C _n1 and C _n2 are vectors each having two components, and the weights W _n1 and W _n2 are scalars. Hereinafter, the two components representing the color are a combination of a color temperature component (unit: mired) and a deviation component from the black body locus.

The weights W _n1 and W _n2 are set according to the luminance Y _n1 and the number of areas F _{n1 of} the first small group _{en 1} and the luminance Y _n2 and the number of areas F _{n2 of} the second small group _en 2.
Specifically, weighting W _n1 to be applied to the color C _n1 of the first small group e _n1, the more the luminance Y _n1 of the first small group e _n1 is high, also the number of regions of the first small group e _n1 F _n1 The larger the number, the larger the setting.

Further, weighting W _n2 to be applied to the color C _n2 of the second small group e _n2 are as luminance Y _n2 of the second small group e _n2 is high, also, often the number of areas F _n2 of the second small group e _n2 The larger it is set.
Therefore, the natural light illumination color C _n estimated in this step reflects the color of the small area of high brightness and majority in the natural light group.

Incidentally, the luminance Y _n1 of the first small group e _n1 mentioned above is the average brightness of the small areas belonging to the first small group e _n1, the number of areas F _n1 of the first small group e _n1 is the first sub-group e the total number of the small areas belonging to _n1, the luminance Y _n2 of the second small group e _n2 is the average brightness of the small areas belonging to the second sub-group e _n2, the number of areas F _n2 of the second small group e _n2 is , The total number of small areas belonging to the second small group _en 2.

Similarly, the CPU 15 determines from the fluorescent lamp group histogram (FIG. 3B) that the first small group _ef1 having the first area number F and the second small group _ef2 having the second area number F. found the door, and calculates the color C _f1 of the first small group e _f1, the second small group e _f2 and a color C _f2, respectively. Color C _f1 of the first small group e _f1 is the average color of the small areas belonging to the first small group e _f1, color C _f2 of the second small group e _f2 is small areas belonging to the second sub-group e _f2 Is the average color.

Then, CPU 15 has a color C _f1 of the first small group e _f1, a second small group e _f2 of the color C _f2 weighted by the following formula (1f), the weighted average color C _f which thereby obtained This is regarded as the illumination color of the fluorescent lamp that illuminated the object scene at the time of photographing (see FIG. 3B ′).
C _f = W _f1 C _f1 + W _f2 C _f2 (1f)
However, in the formula (1f), the colors C _f , C _f1 , and C _f2 are vectors each having two components, and the weights W _f1 and W _f2 are scalars.

Further, weighting W _f1, W _f2 is the luminance Y _f1 and the number of areas F _f1 of the first small group e _f1, is set according to the luminance Y _f2 and the number of areas F _f2 of the second small group e _f2.
Specifically, weighting W _f1 to be applied to the color C _f1 of the first small group e _f1 is, the more the luminance Y _f1 of the first small group e _f1 is high, also the number of regions of the first small group e _f1 F _f1 The larger the number, the larger the setting.

Further, weighting W _f2 to be applied to the color C _f2 of the second small group e _f2, the more the luminance Y _f2 of the second small group e _f2 is high, also, often the number of areas F _f2 of the second small group e _f2 The larger it is set.
Therefore, the illumination color C _f of the fluorescent lamp to be estimated in this step, of the fluorescent lamp groups, the color of the small area of the high luminance and the majority is reflected.

Incidentally, the luminance Y _f1 of the first small group e _f1 described above, the average luminance of the sub-areas belonging to the first small group e _f1, the number of areas F _f1 of the first small group e _f1, the first small group e the total number of the small areas belonging to _f1, the luminance Y _f2 of the second small group e _f2 is the average brightness of the small areas belonging to the second sub-group e _f2, the number of areas F _f2 of the second small group e _f2 is , The total number of small areas belonging to the second small group _ef2 .

Similarly, the CPU 15 determines from the histogram of the light bulb group (FIG. 3C) that the first small group e _l1 having the first area number F and the second small group e _l2 having the second area number F. heading, and color C _l1 of the first small group e _l1, calculates the color C _l2 of the second small group e _l2 respectively. Color C _l1 of the first small group e _l1 is the average color of the small areas belonging to the first small group e _l1, color C _l2 of the second small group e _l2 are small areas belonging to the second sub-group e _l2 Is the average color.

Then, CPU 15 has a color C _l1 of the first small group e _l1, and a color C _l2 of the second small group e _l2 weighted by the following formula (1l), a weighted average color C _l obtained thereby This is regarded as the illumination color of the light bulb that illuminated the object scene during shooting (see FIG. 3B ′).
C _l = W _l1 C _l1 + W _l2 C _l2 (1l)
However, in Expression (1l), the colors C _l , C _l1 , and C _l2 are vectors each having two components, and the weights W _l1 and W _l2 are scalars.

Further, weighting W _l1, W _l2 includes a luminance Y _l1 and the number of areas F _l1 of the first small group e _l1, is set according to the luminance Y _l2 and the number of areas F _l2 of the second small group e _l2.
Specifically, weighting W _l1 to be applied to the color C _l1 of the first small group e _l1, the more the luminance Y _l1 of the first small group e _l1 is high, also the number of regions of the first small group e _l1 F _l1 The larger the number, the larger the setting.

Further, weighting W _l2 to be applied to the color C _l2 of the second small group e _l2, the more the luminance Y _l2 of the second small group e _l2 is high, also, often the number of areas F _l2 of the second small group e _l2 The larger it is set.
Therefore, the illumination color C _l natural light estimated in this step, of the bulb groups, the color of the small area of the high luminance and the majority is reflected.

Incidentally, the luminance Y _l1 of the first small group e _l1 described above, the average luminance of the sub-areas belonging to the first small group e _l1, the number of areas F _l1 of the first small group e _l1 is first small group e the total number of the small areas belonging to _l1, luminance Y _l2 of the second small group e _l2 is the average brightness of the small areas belonging to the second sub-group e _l2, the number of areas F _l2 of the second small group e _l2 is , The total number of small areas belonging to the second small group _e12 .

(Step S18)
As shown in FIG. 4, the CPU 15 weights and averages the natural light illumination color C _n estimated in step S17, the fluorescent light illumination color C _f, and the light bulb illumination color _Cl by the following equation (2). Is regarded as the illumination color of the object scene at the time of shooting.
C = W _n C _n + W _f C _f + W _l C _l (2)
However, in Equation (2), the colors C, C _n , C _f , and C _l are vectors each having two components, and the weights W _n , W _f , and W _l are scalars.

The weights W _n , W _f , and W _l are the luminance Y _n and the number of regions F _n of the natural light group, the luminance Y _f and the number of regions F _{f of} the fluorescent lamp group, the luminance Y _l and the number of regions F of the bulb group _l and the latest determination result (artificial light / natural light) in step S13.
Specifically, weighting W _n natural light group, the higher the brightness F _n natural light group, also, the more the number of areas F _n natural light group, also as the determination result is "natural light" is set large Is done.

Further, the weight W _f of the fluorescent lamp groups, the higher the brightness F _f of the fluorescent lamp group, also, the more the number of areas F _f of the fluorescent lamp groups, also as the determination result is "artificial light", It is set large.
In addition, the weight W _l of the light bulb group is set to be larger as the brightness F _{l of the} light bulb group is higher, as the number of areas F _k of the light bulb group is larger, and as the determination result is “artificial light”. Here, the luminance Y _n of the natural light group is set as the luminance Y _n1 of the first small group en ₁ of the natural light group, and the number of regions F _n of the natural light group is the number of regions of the first small group _en 1 of the natural light group. _Let it be F _n1 . Further, the luminance Y _f of the fluorescent lamp group is set to the luminance Y _f1 of the first small group e _f1 of the fluorescent lamp group, and the number of areas F _f of the fluorescent lamp group is the number of areas of the first small group e _f1 of the fluorescent lamp group. _Let it be F _f1 . Further, the brightness Y _l of the light bulb group is set as the brightness Y _l1 of the first small group e _l1 of the light bulb group, and the number of areas F _l of the light bulb group is set as the number of areas F _l1 of the first small group e _l1 of the light bulb group. (End of description of step S18).

Above, CPU 15 of the system, the achromatic small area in the photographed image I, as well as classified into a plurality of groups by the color (step S16, S17), and illumination color C _n of natural light, illumination color of the fluorescent light and C _f, the illumination color C _l bulb were estimated respectively (step S17), their illumination color C _n, C _f, a weighted average color C of C _l, regarded as the illumination color of the object scene at the time of shooting ( Step S18). Then, the CPU 15 sets the weights W _n , W _f , W _l of the illumination colors C _n , C _f , C _l , the luminances Y _n , Y _f , Y _l of each group, and the number of areas F _n , F of each group. Set according to both _f and _Fl .

  FIG. 5 is a diagram illustrating an example of the degree of control of each illumination light that illuminates the object scene during shooting. The color plane in FIG. 5 indicates the illumination color of each illumination light. The shorter the distance on the color plane, the higher the correlation between illumination colors. In addition, the vertical axis in FIG. 5 indicates the total degree of control of each illumination light, that is, the size and brightness of the illuminated object. The higher the height in the vertical axis direction, the stronger the control.

The illumination color C of the object scene calculated in this system (hereinafter referred to as “calculated illumination color C”) is the illumination color C _n , C of each illumination light, as indicated by the mark “X” in FIG. _f, a neutral color of C _l, moreover, fluorescence illumination light (FIG correlation is highest, dominant degree 2-position of the illumination color C _n of the illumination light domination degree of 1-position (natural light in the figure) The correlation with the illumination color C _f of the lamp (light) is the next highest, and the correlation with the illumination color C ₁ of the illumination light (bulb in the figure) having the third dominant degree tends to be the lowest. That is, the degree of control of each of the plurality of types of illumination light is accurately reflected on the calculated illumination color C.

  For example, as shown in the upper part of FIG. 3, there are a relatively large white ship illuminated by sunset and a relatively large gray building illuminated by fluorescent light. It is assumed that the former is higher and the former has higher luminance. At this time, the ship is more dominant in the object scene. In this case, the white balance processing of the present system works in a direction to reproduce a white ship with a high degree of control whiter. If the degree of control of the white ship is high in this way, it is highly likely that the user attaches importance to the white ship, and therefore the user is highly likely to be satisfied with this white balance processing.

Further, in step S18, the CPU 15 of the present system also reflects the result of the illumination determination (step S13) immediately before photographing on the weights W _n , W _f , W _l of the illumination colors C _n , C _f , C _l . . Therefore, as long as there is no significant change (such as a large object crossing) in the field immediately before shooting, the probability of successful white balance control is further increased.
Incidentally, CPU 15 of the system, in step S18, the luminance Y _n natural light group has been the luminance Y _n1 of the first small group e _n1 belonging to natural light group, a luminance Y _n1 of the first small group e _n1 first 2 may be obtained by a weighted average luminance Y _n2 and the number of areas of the two F _n1, F _n2 small group e _n2. In that case, the luminance Y _f of the fluorescent lamp groups, and those weighted average luminance Y _f1 and the second small group e number of areas of both the luminance Y _f2 of _f2 F _f1, F _f2 of the first small group e _f1 Become. The luminance Y _l bulb group becomes to weighted average in the first small group e _l1 luminance Y _l1 and second small group e _l2 luminance Y _l2 number of regions of the both F _l1, F _l2.

Further, in step S18, the CPU 15 of the present system sets the number of areas F _n of the natural light group as the number of areas F _{n1 of} the first small group _en 1 belonging to the natural light group, but may be the total number of areas of the natural light group. In this case, the number of areas F _f of the fluorescent lamp group is the total number of areas of the fluorescent lamp group, and the number of areas F _l of the light bulb group is the total number of areas of the light bulb group.
Further, CPU 15 of the system, the illumination color C _n, C _f, was calculated white balance gain from the weighted average of C _l, the optimum white balance gain to each of the illumination color C _n, C _f, C _l May be calculated and then weighted averaged. The same effect can be obtained if the weighted average weighting is set in the same manner as described above.

Further, the CPU 15 of this system uses the information to be recorded in the card memory 18A as the photographic image signal after the white balance process, but it is also possible to add the white balance gain information to the photographic image signal before the white balance process. .
Further, the CPU 15 of this system may execute some of the following processes to further improve the success rate of white balance control.
(1) Face recognition processing is performed on the captured image I to determine whether or not a person exists in the captured image I. When a person exists, the illumination color of the person is determined based on the color of the existence area and the surrounding area, and further, a small group to which the illumination color belongs is examined. In creating the histogram in step S17, the number of areas in the small group is increased.
(2) The person's illumination color is determined in the same procedure as in (1), and further, a large group (natural light group, fluorescent lamp group, light bulb group) to which the illumination color belongs is examined. In the weight setting in step S18, the weight corresponding to the large group is set to be larger.
(3) The main subject position in the photographed image I is estimated based on the photographing information (spot metering area, AF ranging point position, etc.). Then, based on the color of the main subject and the surrounding area, the illumination color of the main subject is determined, and further, the small group to which the illumination color belongs is examined. In creating the histogram in step S17, the number of areas in the small group is increased.
(4) The illumination color of the main subject is determined by the same procedure as in (3), and a large group (natural light group, fluorescent lamp group, light bulb group) to which the illumination color belongs is further examined. In the setting of the weight in step S18, the weight corresponding to the large group is set larger.
(5) Recognizing the luminance of the main subject in the scene based on the shooting information. And it is determined whether the brightness | luminance is more than a predetermined threshold value. When the brightness is equal to or higher than the threshold, the number of small group areas having a high color temperature is counted a little in the creation of the natural light group histogram in step S17.
(6) The luminance of the main subject is recognized in the same procedure as (5). And in the setting of each weight in step S18, each weight is adjusted according to the brightness | luminance. Specifically, when the luminance of the main subject is equal to or higher than a predetermined threshold, the weight corresponding to the natural light group is set to be larger, and when the luminance is lower than the predetermined threshold, the artificial light group (fluorescent light group and Set a larger weight corresponding to the light bulb group.

[Second Embodiment]
Hereinafter, a second embodiment of the present invention will be described. This embodiment is also an embodiment of a digital camera system. Here, only differences from the first embodiment will be described.
FIG. 6 is an operation flowchart of the CPU 15 related to the white balance processing of the present embodiment. As shown by the dotted line in FIG. 6, the difference from the first embodiment is that a procedure (step S18 ′) for limiting the color temperature component of the calculated illumination color C is inserted between step S18 and step S19. In the point.

FIG. 7 is a conceptual diagram illustrating step S18 ′.
In general, when a limiter is applied to the color temperature component of the calculated illumination color and the illumination type is natural light, the limiter characteristic for natural light as shown in FIG. 7A (however, the limit range depends on the subject luminance). When the illumination type is a fluorescent lamp, a limiter characteristic for a fluorescent lamp as shown in FIG. 7B (however, the limit range varies depending on the subject brightness) is used. When the illumination type is a light bulb, a light bulb limiter characteristic (however, the limit range varies depending on the subject brightness) as shown in FIG. As apparent from FIGS. 7A, 7B, and 7C, even if the subject brightness is the same, the limit ranges (T _min , T _max) are limited between natural light, a fluorescent lamp, and a light bulb. ) Are different from each other.

The CPU 15 of this embodiment recognizes these three types of limiter characteristics in advance. In step S18 ′, the CPU 15 weights and averages these three types of limiter characteristics, and limits the color temperature component of the calculated illumination color C using the resulting characteristics as the limiter characteristics. In the weighted average, the limiter characteristic weight for natural light, the limiter characteristic weight for fluorescent lamps, and the limiter characteristic weight for bulbs are set to be equal to the weights W _n , W _f , and W _l set in step S18, respectively. To do.

However, since the amount of computation increases when all of the limiter characteristics are weighted average, the actual operation is performed as follows.
That is, the CPU 15 first refers to the limiter characteristic for natural light (FIG. 7A) according to the luminance Y ₀ of the main subject recognized from the photographing information, and limits the range (T _nmin , T nm corresponding to the luminance Y _{0. nmax} ). Also recognize that in accordance with the luminance Y ₀ refers to the limiter characteristic of the fluorescent lamp (Fig. 7 (B)), the limit range (T _fmin, T _fmax) corresponding to the luminance Y ₀ a. Further, referring to a limiter characteristic of the bulb (FIG. 7 (C)) in accordance with the luminance Y _0, the limit range (T _lmin, T _lmax) corresponding to the luminance Y ₀ recognize.

CPU15, they limit the range _{(T nmin, T nmax),} (T fmin, T fmax), weighted by (T _lmin, T _lmax) Equation (3), the limit range (T _min, T _max) the Obtained (see FIG. 7D).
T _max = W _n T _nmax + W _f T _fmax + W _l T _{lmax
T min = W n T nmin +} W f T fmin + W l T lmin ... (3)
Incidentally, Equation (3), the color temperature _{T max, T nmax, T lmax} , T min, T nmin, T fmin, the unit of T _lmin is, the color C in the equation (2), C _n, C _f, C _l It is assumed that the unit is the same as the unit of the color temperature component (here, mired). In this case, the values of the weights W _n , W _f , W _l in the equation (3) are the same as the values of the weights W _n , W _f , W _l in the equation (2). However, when the unit is changed, the values of the weights W _n , W _f , and W _l need to be changed.

The CPU 15 compares the acquired limit range (T _min , T _max ) with the color temperature component of the calculated illumination color C. If the color temperature component is below the lower limit value T _min , the color temperature If the component value is replaced (clipped) with the same value as the lower limit value T _min and exceeds the upper limit value T _max , the value of the color temperature component is replaced (clipped) with the same value as the upper limit value T _max ( End of description of step S18 ′).

As described above, according to this embodiment, the limiter characteristic for natural light (FIG. 7A), the limiter characteristic for fluorescent lamp (FIG. 7B), and the limiter characteristic for light bulb (FIG. 7C). And the weight is set to be the same as the weight set in step S18, so that the dominance of each of the plurality of types of illumination light is also limited to the limit ranges (T _min , T _max ). Reflected accurately. Therefore, the possibility that the white balance process will fail is further reduced.

In addition, although CPU15 of this embodiment made only the color temperature component of the calculation illumination color C the object of a limit, it is good also as both a color temperature component and a deviation component.
[Other Embodiments]
In the embodiment described above, a single-lens reflex digital camera (with at least a quick return mirror and a color imaging device for subject monitoring) is described. However, a compact digital camera (quick return mirror and subject monitoring) is described. The present invention can also be applied to a non-mounted color image sensor. In that case, both acquisition of the pre-shooting image signal and acquisition of the shot image signal are performed by the same color image pickup device (color image pickup device for image pickup).

It is a block diagram of this system. It is an operation | movement flowchart of CPU15 in connection with the white balance process of 1st Embodiment. It is a figure explaining step S16, S17. It is a figure explaining step S18. It is a figure which shows an example of the control degree of each illumination light which illuminates an object scene at the time of imaging | photography. It is an operation | movement flowchart of CPU15 in connection with the white balance process of 2nd Embodiment. It is a conceptual diagram explaining step S18 '.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Digital camera body, 2 ... Interchangeable lens, 11 ... Quick return mirror, 13 ... Shutter, 12 ... Color image sensor for image photography, 13 ... Shutter, 20 ... Photometry part, 15 ... CPU

Claims (10)

A classification unit that classifies a plurality of small areas in a color image obtained by photographing a scene into a plurality of groups according to the illumination color;
A calculation unit that regards a weighted average color of the illumination colors of each of the plurality of groups as the illumination color of the object scene and calculates an optimal white balance control amount for the illumination color as a white balance control amount for the color image;
A color imaging device including:
The calculation unit includes:
The color imaging device, wherein the weight of the illumination color of each of the plurality of groups is set according to the area size and brightness of each of the plurality of groups. The color imaging device according to claim 1,
The calculation unit includes:
The color imaging apparatus, wherein the weight of the illumination color of each of the plurality of groups is set according to a determination result of the illumination type of the object scene that is performed before the color image is captured. In the color imaging device according to claim 1 or 2,
The calculation unit includes:
A color imaging apparatus, wherein a weight of each illumination color of the plurality of groups is set in accordance with an illumination type of a subject existing in the color image. In the color imaging device according to any one of claims 1 to 3,
A limit unit for adjusting the calculated value so that the calculated value of the weighted average color or the calculated value of the white balance control amount falls within an allowable range;
The limit part is
A color imaging apparatus, wherein an allowable range for each of the plurality of groups is weighted and averaged with a weight equivalent to the weight, and the adjustment is performed using a weighted average range obtained thereby. In the color imaging device according to any one of claims 1 to 4,
A color imaging apparatus, further comprising: a processing unit that performs white balance control on the color image with the calculated white balance control amount. A classification procedure for classifying a plurality of small areas in a color image obtained by photographing a scene into a plurality of groups according to the illumination color;
A calculation procedure that regards a weighted average color of the illumination colors of each of the plurality of groups as the illumination color of the object scene, and calculates a white balance control amount optimal for the illumination color as a white balance control amount for the color image;
An image processing program including
In the calculation procedure,
An image processing program, wherein a weight of each illumination color of the plurality of groups is set according to a region size and luminance of each of the plurality of groups. The image processing program according to claim 6,
In the calculation procedure,
The image processing program characterized by setting the weight of the illumination color of each of the plurality of groups according to the result of determination of the illumination type of the object scene performed before photographing the color image. In the image processing program according to claim 6 or 7,
In the calculation procedure,
An image processing program, wherein a weight of each illumination color of the plurality of groups is set according to an illumination type of a subject existing in the color image. In the image processing program according to any one of claims 6 to 8,
A limit procedure for adjusting the calculated value so that the calculated value of the weighted average color or the calculated value of the white balance control amount falls within an allowable range;
In the limit procedure,
An image processing program for performing an adjustment using a weighted average range obtained by performing a weighted average on an allowable range for each of the plurality of groups with a weight equivalent to the weight. In the image processing program according to any one of claims 6 to 9,
An image processing program, further comprising: a processing procedure for performing white balance control on the color image with the calculated white balance control amount.

Images