JP5225137B2 - Imaging apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an imaging device, an image processing method, and a program, and is particularly suitable for use in white balance processing when a strobe light is emitted.

  In general, a signal output from an image sensor is digitized by AD conversion, and the digitized captured image is divided into a plurality of blocks as shown in FIG. The block signal is composed of color signals including R, G, and B, and the following formulas (1-1) and (1-2) are examples of color evaluation value calculation for the block.

  Next, when the color evaluation value (Cx [i], Cy [i]) is included in the preset white detection range, it is determined that the block is white. Then, the integral values SumR, SumG, and SumB of the color pixels that have entered the white detection range are calculated, and the white balance coefficients (WBCo_R, WBCo_G, WBCo_B) are calculated as in the following equations (2-1) to (2-3). ) Is calculated.

  Conventionally, when calculating the white balance coefficient during flash photography, the white balance coefficient of external light is calculated from an image signal irradiated with only external light, and the white balance coefficient of the flash is calculated from the image signal irradiated with the flash. . Then, the white balance coefficient of both is calculated by weighted averaging from the strobe contribution to the subject.

  At this time, the image signal read out in the field is used as the image signal irradiated with only external light. This is because the field readout can be processed in a shorter time than the frame readout by adding pixels. Therefore, the white balance coefficient of the image signal irradiated with only external light is calculated from the image data read out from the field, and the white balance coefficient is calculated from the image signal irradiated with the strobe light. Then, a final white balance coefficient is calculated by weighted addition of both.

JP 2007-300320 A JP 09-322191 A

However, for example, in the case of external light with a color temperature difference such as a strobe light such as A light source, high sensitivity, and long seconds, the white balance coefficient of external light and the white balance coefficient of strobe light are weighted and added. When it becomes, it becomes easy to be influenced by outside light. That is, there is a problem that a desired white balance coefficient cannot be obtained because the background light source is easily pulled.
The present invention has been made in view of the above-described problems, and makes it possible to perform a desired white balance correction when a strobe light is emitted.

The present invention is an imaging apparatus that performs image processing of captured image data, and first white balance correction calculation means for calculating a white balance correction value for external light from image data at the time of exposure when a flash is emitted. And a second white balance correction calculating means for calculating a white balance correction value for the strobe from the image data at the time of exposure, and a second white balance correction value for the external light from the past image data at the time of no strobe light emission. And a white balance correction value for performing image processing on the captured image data based on the first white balance correction value, the second white balance correction value, and the third white balance correction value. And calculating means for calculating.
The present invention is an image processing method of an imaging apparatus that performs image processing of captured image data, and is a first white that calculates a white balance correction value for external light from image data at the time of exposure when a strobe is emitted. A balance correction calculating step; a second white balance correction calculating step for calculating a white balance correction value for the strobe from the image data at the time of exposure; and a white balance correction value for the external light from the past image data when the strobe is not emitting light. Image processing is performed on the captured image data based on the third white balance correction calculation step for calculating the first white balance correction value, the second white balance correction value, and the third white balance correction value. And a calculating step for calculating a white balance correction value.
The present invention provides a first white balance correction calculating step for calculating a white balance correction value for external light from image data at the time of exposure when a strobe is emitted to a computer that performs image processing of captured image data; A second white balance correction calculation step for calculating a white balance correction value for strobe from image data at the time of exposure, and a third white for calculating a white balance correction value for external light from past image data at the time of non-flash emission Based on the balance correction calculating step and the first white balance correction value, the second white balance correction value, and the third white balance correction value, a white balance correction value for performing image processing on the captured image data is calculated. This is a program for executing the calculation step.

  According to the present invention, desired white balance correction can be performed when a strobe is emitted.

It is a block diagram which shows the structure of the imaging device which concerns on this embodiment. It is a figure which shows an example which divided | segmented the screen into several blocks. It is a conceptual diagram which shows the calculation method of the white balance correction value which concerns on 1st Embodiment. It is a flowchart which shows the calculation process of a 1st white balance correction value. It is a figure which shows an example of a white detection range. It is a flowchart which shows the calculation process of a 2nd white balance correction value. It is a figure which shows the example of correction | amendment of the spectral difference by a drive mode. It is a figure for demonstrating the process which extracts a to-be-photographed area | region. It is a figure which shows a response | compatibility of the gain (alpha) determined according to the difference of a photometric value. It is a conceptual diagram which shows the calculation method of the white balance correction value which concerns on 2nd Embodiment. It is a figure which shows a response | compatibility of the white balance correction value determined according to external light.

Hereinafter, embodiments according to the present invention will be described with reference to the drawings.
(First embodiment)
FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus according to the present embodiment.
In FIG. 1, reference numeral 101 denotes a solid-state imaging device made of a CCD, a CMOS, or the like. The surface of the fixed imaging element 101 is covered with an RGB color filter such as a Bayer array so that color imaging can be performed.

  The CPU 114 calculates the shutter speed and aperture value so that the entire image has a predetermined brightness, and calculates the driving amount of the focus lens so that the subject in the focusing area is focused. The exposure value (shutter speed, aperture value) calculated by the CPU 114 and the driving amount of the focus lens are sent to the control circuit 113 and controlled based on each value.

  Reference numeral 103 denotes a white balance (WB) control unit. The white balance control unit 103 calculates a white balance correction value based on information from the image signal stored in the memory 102, and uses the calculated white balance correction value for the image signal stored in the memory 102. Perform white balance correction. The detailed configuration of the white balance control unit 103 and the method for calculating the white balance correction value will be described later.

Reference numeral 104 denotes a color matrix circuit. The color matrix circuit 104 applies color gain to convert the image signal that has been subjected to white balance correction by the white balance control unit 103 to an optimum color and converts the image signal into color difference signals RY and BY. Reference numeral 105 denotes a low-pass filter (LPF) circuit that limits the bands of the color difference signals RY and BY.
Reference numeral 106 denotes a CSUP (Chroma Supress) circuit. The CSUP circuit suppresses the false color signal in the saturated portion of the image signal band-limited by the LPF circuit 105. On the other hand, the image signal subjected to white balance correction by the white balance control unit 103 is also output to the luminance signal (Y) generation circuit 111, and the luminance signal Y is generated. The generated luminance signal Y is subjected to edge enhancement processing by the edge enhancement circuit 112.

  The color difference signals RY and BY output from the CSUP circuit 106 and the luminance signal Y output from the edge enhancement circuit 112 are converted into RGB signals by the RGB conversion circuit 107. The RGB signal is subjected to gradation correction by the gamma correction circuit 108. Thereafter, the tone-corrected RGB signal is converted into a YUV signal by the color luminance conversion circuit 109. The YUV signal is compressed by the compression circuit 110 and recorded as an image signal on an external recording medium or an internal recording medium.

Next, a method for calculating a white balance correction value by the white balance control unit 103 will be described with reference to FIG. Note that the processing by the white balance control unit 103 is performed under the control of the CPU 114 when the CPU 114 executes a program stored in a nonvolatile memory or the like of the imaging apparatus.
FIG. 3 is a conceptual diagram showing a method for calculating a white balance correction value. The white balance control unit 103 calculates a first white balance correction value 601, a second white balance correction value 602, and a third white balance correction value 603 as shown in FIG.

First, a method for calculating the first white balance correction value (white light correction value for external light) 601 will be described with reference to the flowchart of FIG. FIG. 4 is a flowchart illustrating an example of a procedure for calculating the first white balance correction value 601.
In step S101, the white balance control unit 103 reads out an image signal stored in the memory 102 as image data at the time of exposure when the strobe is emitted, and displays the screen in arbitrary m blocks as shown in FIG. Divide into
In step S102, the white balance control unit 103 adds and averages the pixel values to the respective colors for each block (1 to m), and obtains the color average values (R [i], G [i], B [i]). calculate. Next, the white balance control unit 103 substitutes the calculated color average value into the equations (1-1) and (1-2) shown in the above-described equation 1 to obtain color evaluation values (Cx [i], Cy [I]) is calculated.

In step S103, the white balance control unit 103 includes the calculated color evaluation values (Cx [i], Cy [i]) of the i-th block in the preset white detection range 301 shown in FIG. To determine whether or not
Here, the white detection range 301 shown in FIG. 5A is obtained by photographing white under different light sources in advance and plotting the calculated color evaluation values. The negative direction of the x coordinate (Cx) shown in FIG. 5A is a color evaluation value when white of a high color temperature subject is photographed, and the positive direction is a color evaluation value when white of a low color temperature subject is photographed. It is. Further, the y coordinate (Cy) means the degree of the green component of the light source, and indicates that the light source has a G component that increases in the negative direction, that is, a fluorescent lamp.

  As a result of the determination in step S103, when the calculated color evaluation values (Cx [i], Cy [i]) are included in the white detection range 301 (the determination result in step S103 is YES), the block is white. Judge that there is. In this case, the process proceeds to step S104, and the color average values (R [i], G [i], B [i]) of the block are integrated. If the result of determination in step S103 is not included in the white detection range 301 (result of determination in step S103 is NO), the process proceeds to step S105 without integration. The processing of step S103 and step S104 can be expressed by equations (3-1) to (3-3).

  Here, in the expressions (3-1) to (3-3), when the color evaluation value (Cx [i], Cy [i]) is included in the white detection range 301, Sw [i] is included in 1. If not, Sw [i] is set to 0. Thereby, the process of whether or not to add the color average values (R [i], G [i], B [i]) is substantially performed according to the determination in step S103.

  In step S105, the white balance control unit 103 determines whether the above-described processing has been performed for all blocks. If there is an unprocessed block, the process returns to step S102, and the processes of steps S102 to S105 described above are repeated. If all the blocks have been processed, the process proceeds to step S106.

  In step S106, the white balance control unit 103 uses the following formulas (4-1) to (4-3) based on the calculated integrated values (SumR, SumG, SumB) of the color evaluation values, for the first white Balance correction values (WBCol_R, WBCol_G, WBCol_B) are calculated. This processing corresponds to an example of processing by the first white balance correction calculation unit.

Next, a method for calculating the second white balance correction value (strobe white balance correction value) 602 will be described with reference to the flowchart of FIG. FIG. 6 is a flowchart illustrating an example of a procedure for calculating the second white balance correction value 602. Here, the flowchart of FIG. 6 is the same process as the calculation method of the first white balance correction value 601 described above, and a description thereof will be omitted.
The difference from the first white balance correction value calculation method is that the white balance control unit 103 performs processing in the white detection range for strobe light in the white detection range 302 shown in FIG. This is because it is a known light source called strobe light and can be limited to the white detection range 302. The second white balance correction value can be calculated by the processing in steps S201 to S206 shown in the flowchart of FIG. This processing corresponds to an example of processing by the second white balance correction calculation unit.

Next, a method for calculating the third white balance correction value (external light white balance correction value) 603 will be described. Here, the white balance control unit 103 calculates a white balance correction value for external light from past image data when the flash is not emitted. Here, the past image data is, for example, image data captured in an EVF (Electronic View Finder) drive mode immediately before the actual shooting is performed.
The calculation of the third white balance correction value can be applied even when the imaging drive mode is different. For example, the white balance correction value of the EVF drive mode calculated in the past can be used as the white balance correction value of the still image drive mode. However, when a spectral difference occurs in the drive mode, it is necessary to correct the white balance correction value. Specifically, as shown in FIG. 7, the white balance correction value corresponding to the still image drive mode is calculated by correcting the spectral differences ΔCx and ΔCy in the EVF drive mode. As described above, the white balance control unit 103 corrects the spectral difference between the EVF drive mode and the still image drive mode for the image data captured in the EVF drive mode, and white balance corresponding to the still image drive mode. A correction value is calculated. This processing corresponds to an example of processing by the third white balance correction calculation unit.

Next, returning to the conceptual diagram of the method for calculating the white balance correction value shown in FIG. 3, the first white balance correction value, the second white balance correction value, and the third white balance correction value mix (weighted addition) method. Will be described.
First, in processing 610 shown in FIG. 3, weighted addition is performed in accordance with the irradiation ratio between external light and strobe light applied to the subject.
Here, a process of cutting out a subject area for calculating the irradiation ratio will be described. In the present embodiment, test light emission (pre-light emission) of the flash is performed before the main photographing is performed in order to cut out the subject area.

First, as shown in FIG. 8, the white balance control unit 103 obtains image data 81 before flash test light emission, and a luminance block composed of an m × n matrix from the image data 81 before test light emission. a (82) is calculated.
Next, the white balance control unit 103 performs flash test light emission under the same conditions as before the test light emission, acquires the image data 83 at the time of the test light emission, and similarly a luminance block configured by an m × n matrix. b (84) is acquired.
The white balance control unit 103 temporarily stores the acquired data of the luminance block a (82) and the luminance block b (84) in, for example, a RAM. Here, it is assumed that the background image of the subject image hardly changes between the image data before the test light emission and the image data at the time of the test light emission.
Accordingly, the difference data between the two luminance blocks a (82) and b (84) becomes the reflected light of the subject area during the flash test emission, and the white balance control unit 103 determines subject position information from the difference data. c (85) can be acquired.
A subject position block can be acquired from the subject position information calculated in this way.

  Next, the imaging apparatus captures the image by strobe light emission and acquires image data 86 shown in FIG. The white balance control unit 103 calculates the main exposure luminance (Y1) corresponding to the acquired subject position block in the acquired image data 86. Further, the white balance control unit 103 calculates the non-flash luminance (Y2) corresponding to the acquired subject position block among the past image data when the flash is not flashing. If the exposure conditions differ between the main exposure and the strobe non-emission, Y1 and Y2 are calculated by aligning the exposure conditions. The white balance control unit 103 mixes (weights and adds) the first and second white balance correction values using the calculated ratio of Y1 and Y2 as the ratio of the amount of light applied to the subject, and performs the second and third whites. The balance correction value is subjected to Mix processing (weighted addition). Here, for example, if the ratio of the strobe light luminance (Y1) during main exposure to the external light luminance (Y2) when no strobe light is emitted is 5: 1, the first white balance correction value and the second white balance correction value The ratio at which the white balance correction value is subjected to the Mix processing is 1: 5. Further, the ratio of the mix processing of the second white balance correction value and the third white balance correction value is 5: 1.

  Next, the white balance correction value mix processing in step 620 shown in FIG. 3 will be described. The white balance control unit 103 multiplies the white balance correction value obtained by mixing the first and second white balance correction values with a gain α. The white balance control unit 103 multiplies the white balance correction value obtained by mixing the second and third white balance correction values with a gain (1-α). Then, the white balance control unit 103 calculates a white balance correction value used for final image processing by adding the white balance correction value obtained by multiplying the gain α and the gain (1−α).

Here, for example, when a scene change such as a light source change occurs between the image data at the time of main exposure and the past image data at the time of non-flashing, the reliability of the third white balance correction value decreases. There is.
Therefore, when a scene change occurs, the white balance control unit 103 controls the gain α to change the ratio of using the third white balance correction value. Here, a method by which the white balance control unit 103 calculates the gain α will be described with reference to FIG. FIG. 9 is a graph showing the correspondence of the gain α determined according to the difference in Bv value (photometric value) between the main exposure shooting and the non-flash shooting.
As shown in FIG. 9, the difference ΔBv between the main exposure shooting and the non-flash shooting is ΔBv_T.
When it is smaller than h1, the gain α is 0, and when it is larger than ΔBv_Th2, the gain α is 1.

Therefore, as shown in FIG. 6, in the process 620, for example, when the difference between the Bv values (photometric values) is larger than ΔBv_Th2, the white balance control unit 103 determines that a scene change such as a light source change has occurred, and the gain α is Set to 1. Then, the white balance control unit 103 calculates the white balance correction value without using the third white balance correction value.
The white balance control unit 103 determines the white balance correction value obtained in this way as a white balance correction value to be used for image processing, and ends the processing.
As described above, according to the present embodiment, the white balance correction value for performing image processing is calculated using the first, second, and third white balance correction values, so that the desired white can be obtained even when the flash is emitted. Balance correction can be performed.

(Second Embodiment)
Next, a method for calculating the white balance correction value according to the second embodiment will be described. In the present embodiment, the limiter control is performed after performing the Mix process according to the light amount ratio. The configuration of the imaging device is the same as that of the first embodiment.
A method of calculating the white balance correction value by the white balance control unit 103 will be described with reference to FIG. Note that the processing by the white balance control unit 103 is performed under the control of the CPU 114 when the CPU 114 executes a program stored in a nonvolatile memory or the like of the imaging apparatus.

FIG. 10 is a conceptual diagram illustrating a white balance correction value calculation method according to the present embodiment.
In the present embodiment, the white balance control unit 103 performs mix processing based on the light amount ratio on the second white balance correction value and the third white balance correction value, and then performs limiter control in the limiter processing 901.
For example, when the external light is a low color temperature light source such as an A light source, the white balance control unit 103 is used in a shooting condition that is easily affected by external light such as high sensitivity or long seconds. The white balance correction value is controlled by the limiter.

Here, a method in which the white balance control unit 103 performs limit control of the white balance correction value will be described with reference to FIG. FIG. 11 is a graph showing the correspondence of white balance correction values determined according to external light.
As shown in FIG. 11A, for example, when the color temperature of the external light is lower than Cx0 (low color temperature), the white balance control unit 103 performs limit control to the correction value ΔCx0. In the limiter process 901, the limiter process is performed on the white balance correction values (Cx, Cy) obtained by the light quantity ratio MIX in the process 610. That is, in this example, the white balance correction value (Cx + Δ) is calculated by adding the limiter correction value ΔCx0 to the white balance correction value calculated in the process 610.
Cx0, Cy) is calculated.

As illustrated in FIG. 11B, for example, when the external light ratio is higher than Ratio1, the white balance control unit 103 performs limit control to the correction value ΔCx1. This makes it possible to calculate a desired white balance correction value even in a scene that is easily influenced by outside light and is pulled by a background light source. The other processes are the same as those in the first embodiment, and the same reference numerals are given and description thereof is omitted.

  As described above, according to the present embodiment, the white balance correction value for performing image processing is calculated using the first, second, and third white balance correction values, so that the desired white can be obtained even when the flash is emitted. Balance correction can be performed.

Note that the present invention can be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, a camera head, etc.), or a device composed of a single device (for example, a digital still camera, a digital video camera) You may apply to.
The object of the present invention can also be achieved as follows. First, a storage medium (or recording medium) that records a program code of software that realizes the functions of the above-described embodiments is supplied to a system or an apparatus. Then, the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also the following can be achieved. That is, when the operating system (OS) running on the computer performs part or all of the actual processing based on the instruction of the read program code, the functions of the above-described embodiments are realized by the processing. It is. Here, as the recording medium for storing the program code, for example, a flexible disk, hard disk, ROM, RAM, magnetic tape, nonvolatile memory card, CD-ROM, CD-R, DVD, optical disk, magneto-optical disk, MO, etc. Can be considered. A computer network such as a LAN (Local Area Network) or a WAN (Wide Area Network) can be used to supply the program code.

DESCRIPTION OF SYMBOLS 101 CCD image pick-up element 102 Memory 103 White balance control part 104 Color matrix circuit 105 LPF circuit 106 CSUP circuit 107 RGB conversion circuit 108 Gamma correction circuit 109 Color luminance conversion circuit 110 Compression circuit 111 Luminance signal generation circuit 112 Edge enhancement circuit 113 Control circuit 114 CPU

Claims (9)

An imaging device that performs image processing of captured image data,
First white balance correction calculating means for calculating a white balance correction value for external light from image data at the time of exposure when the strobe is emitted;
Second white balance correction calculating means for calculating a white balance correction value for a strobe from the image data at the time of exposure;
Third white balance correction calculating means for calculating a white balance correction value for external light from past image data when the strobe is not lit;
Calculating means for calculating a white balance correction value for performing image processing on the captured image data based on the first white balance correction value, the second white balance correction value, and the third white balance correction value; An imaging apparatus characterized by the above.   The calculation means calculates a white balance correction value for performing image processing by performing weighted addition using the first white balance correction value, the second white balance correction value, and the third white balance correction value. The imaging apparatus according to claim 1.   The imaging apparatus according to claim 2, wherein the calculation unit performs weighted addition according to a ratio between external light and strobe light irradiated on the subject.   The imaging apparatus according to any one of claims 1 to 3, wherein the past image data when the strobe is not lit is image data captured in an EVF drive mode.   The third white balance correction calculating unit corrects a spectral difference between the EVF drive mode and the still image drive mode for the image data captured in the EVF drive mode, and performs white corresponding to the still image drive mode. The imaging apparatus according to claim 4, wherein a balance correction value is calculated. A limiter control unit is further provided for performing limiter control after combining the second white balance correction value and the third white balance correction value based on the light amount ratio according to the external light irradiated to the subject. The imaging apparatus according to claim 1, wherein the imaging apparatus is characterized. The limiter control means uses the second white balance correction value and the third white balance correction value based on the light amount ratio according to the color temperature of the external light or the ratio of the external light and the strobe light applied to the subject. The image pickup apparatus according to claim 6, wherein the combined value is subjected to limiter control. An image processing method of an imaging apparatus that performs image processing of captured image data,
A first white balance correction calculating step for calculating a white balance correction value for external light from image data at the time of exposure when the flash is emitted;
A second white balance correction calculating step for calculating a white balance correction value for the strobe from the image data at the time of exposure;
A third white balance correction calculating step for calculating a white balance correction value for external light from past image data when the flash is not lighted;
And calculating a white balance correction value for performing image processing on the captured image data based on the first white balance correction value, the second white balance correction value, and the third white balance correction value. An image processing method characterized by the above. In a computer that performs image processing of captured image data,
A first white balance correction calculating step for calculating a white balance correction value for external light from image data at the time of exposure when the flash is emitted;
A second white balance correction calculating step for calculating a white balance correction value for the strobe from the image data at the time of exposure;
A third white balance correction calculating step for calculating a white balance correction value for external light from past image data when the flash is not lighted;
Based on the first white balance correction value, the second white balance correction value, and the third white balance correction value, a calculation step of calculating a white balance correction value for performing image processing on the captured image data is executed. Program for.

Images