JP2003324755A - Image processing apparatus and method, recording medium, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus for setting proper white balance. <P>SOLUTION: A monitoring image storage section 41 stores image data acquired just before image data stored in a capture image storage section 42. A calculation section 43 calculates an average of the image data respectively stored in the monitoring image storage section 41 and the capture image storage section 42 and calculates a gain used for a capture image by using the calculated average and a gain for the white balance set at that point of time. A setting section 44 controls the adjustment of the white balance on the basis of the calculated gain. This invention is applicable to cameras for adjusting white balance of a picked-up image and applying processing to reproducing an optimum color tone. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and method, a recording medium, and a program, and in particular, to an apparatus for reproducing an optimum color tone by adjusting white balance of a captured image. The present invention relates to a suitable image processing apparatus and method, recording medium, and program.

[0002]

2. Description of the Related Art When a digital camera or the like is used to shoot under artificial light, the captured image and the subject seen by the human eye may have different color tones. This is because the artificial light source has a light source with a strong red color and a light source with a strong green color depending on the color temperature, and the color balance of the reflected light from the subject that received the artificial light source is broken, and humans feel white. However, the color of the artificial light source is sensed in the digital camera using the light receiving element. As a result, a reddish image or a greenish image is obtained as a whole.

A digital camera has a function of adjusting the color tone of a subject image taken into a light receiving element so that the color balance is adjusted so that the color tone is the same as that of the subject seen by the human eye. This kind of color balance adjustment means that colors are reproduced under a white light source with an appropriate color balance.
It is generally called white balance adjustment, and white balance adjustment is performed with R (red) under an appropriate white light source.
R / G, which is the intensity ratio of the three primary colors of G (green) and B (blue),
It is performed using the value defined by B / G.

In a conventional digital camera, for example, the white balance value is adjusted to the reflected light of the room light when shooting with only the indoor light, and is adjusted to the reflected light of the strobe light when shooting with the strobe light. It was done. That is, the white balance value is calculated according to the reflected light of the light from the strongest light source.

The exposure of a digital camera is AE
(Auto Exposure: Automatic exposure mechanism). AE
Measures the brightness of each part of the screen to be photographed, and sets the exposure so that the average is a predetermined brightness. In addition, some digital cameras have an exposure correction function that corrects the exposure of the digital camera so that the user can shoot the image with the brightness that the user thinks of.

When a subject is imaged under illumination light of which brightness is not sufficient, a strobe emits light to take an image, or the exposure time is set longer than usual in order to obtain a sufficient amount of light for image pickup. An imaging mode called slow shutter is used. When the illumination is dark and the subject is at a short distance, a natural image can be obtained by shooting with a strobe, but for example, a distant subject that the light emitted from the strobe does not reach sufficiently is captured. In such cases, for example, when capturing an image of a night view, even if flash firing is performed, it is not possible to obtain a sufficient amount of light for imaging, and the slow shutter is used to capture a more natural image. It is possible to obtain a clear image.

For example, using a conventional digital camera,
When shooting a subject against a night view, if you focus on a subject, such as a person, and shoot with a flash, the white balance will be adjusted according to the reflected light reflected by the strobe light, so the night view of the resulting image There was a problem that the white balance was not correct.

As a technique for solving such a problem, there is a method disclosed in Japanese Patent Laid-Open No. 11-298908. In this method, the optimum white balance is set separately for the foreground to which the strobe light is emitted and the background to which the strobe light is not emitted (does not reach). In order to set the white balance separately, the foreground and the background are imaged separately, and the foreground and the background are combined in the subsequent processing to form one image.

According to this method, in the adjusted subject data, the image data of the subject portion is proper (focus and white balance are proper). On the other hand, the background portion has image data below the black level because the strobe light does not reach it, and the adjusted background data has image data below the black level for the subject portion and is appropriate for the background portion. Since the image data has proper focus and white balance, the final image data obtained by combining adjusted subject data and adjusted background data is appropriate for both subject and background (focus and white balance). Is appropriate).

[0010]

However, according to the method disclosed in Japanese Patent Laid-Open No. 11-298908, the foreground and the background are photographed twice. When the photographing is performed twice as described above, there is a problem that it takes a long time to perform all the photographing (to photograph one image).

Further, the time taken to capture one image becomes long, which means that the subject moves, the photographer moves, and camera shake occurs during the time required to capture the image. There is a high possibility that the position of the subject will be deviated due to such causes as the above. Then, if the displacement occurs, there is a problem that it is difficult to accurately perform a process of superimposing two images.

Further, when a person in the foreground is irradiated with constant light (light other than strobe light), it is difficult to obtain an optimum white balance.

The present invention has been made in view of such a situation, and by adjusting the white balance by using the image at the time of monitoring and the image at the time of capturing, the time required for image pickup can be shortened and the optimum The purpose is to set the white balance.

[0014]

An image processing apparatus according to the present invention includes an image pickup means for picking up an image, an average value of first image data of an image picked up by the image pickup means and instructed to be recorded on a recording medium. , Calculating means for calculating the average value of the second image data acquired at the time before the acquisition of the first image data, and calculating the average value of the second image data calculated by the calculating means, Setting for setting the white balance gain applied to the first image data by multiplying the value obtained by dividing the average value of the first image data by the gain of the white balance when the second image data is acquired. And means.

It is possible to further include dividing means for dividing the image picked up by the image pickup means into a plurality of areas, and the calculation by the calculation means and the setting by the setting means can be performed for each area.

The image pickup device further includes detection means for detecting an area having a high brightness value in the image picked up by the image pickup means, and the calculation by the calculation means and the setting by the setting means are performed on the area detected by the detection means. Can be done by.

According to the image processing method of the present invention, an image capturing step for capturing an image, an average value of the first image data of the image captured in the processing of the image capturing step and instructed to be recorded on the recording medium, and the first image data The calculation step of calculating the average value of the second image data acquired at the time before the image data is acquired and the average value of the second image data calculated by the processing of the calculation step A setting step of setting a white balance gain applied to the first image data by multiplying a value divided by the average value of the image data by a white balance gain when the second image data is acquired; It is characterized by including.

A recording medium program according to the present invention includes an image capturing step for capturing an image, an average value of first image data of an image captured in the processing of the image capturing step, and instructed to record on the recording medium, and a first value. Of the second image data obtained by the calculation step of calculating the average value of the second image data acquired at the time before the image data of Setting step of setting the gain of the white balance applied to the first image data by multiplying the value obtained by dividing the average value of the image data of 1) by the gain of the white balance when the second image data is acquired. It is characterized by including and.

The program of the present invention includes an image capturing step for capturing an image, an average value of the first image data of the image captured in the processing of the image capturing step and instructed to record on the recording medium, and the first image data. The calculation step of calculating the average value of the second image data acquired before the acquisition of the first image data and the average value of the second image data calculated by the processing of the calculation step. A setting step of setting a white balance gain to be applied to the first image data by multiplying a value obtained by dividing the average value by the white balance gain when the second image data is acquired. It is characterized by making it execute.

In the image processing apparatus and method and the program of the present invention, among the picked-up images, the average value of the image data of the image instructed to be recorded on the recording medium is obtained at the time before the image. The value obtained by dividing the average value of the image data is multiplied by the gain of the white balance set at that time to set the white balance gain to be applied next.

[0021]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing the configuration of an embodiment of an image processing apparatus to which the present invention is applied.
Light (that is, an image of a subject) is received by a CCD (Charge Coupled Devices) 12 via a lens 11.
The CCD 12 converts the intensity of the received light into an electric signal,
Output to CDS (Correlated Double Sampling) 13.

The CCD 12 is the timing generator 1
The accumulated electric signal is output to the CDS 13 in accordance with the timing signal supplied from the circuit 4. Most of the noise included in the output signal of the CCD 12 is reset noise (generated when the charge detection differential amplifier (not shown) inside the CCD 12 is reset), and the CDS 13 reduces the reset noise.

The output waveform of the CCD 12 has a reset period,
It is divided into three periods of a field through period and a signal period. In the signal period, there are a video signal and reset noise, and during the field through period, only reset noise exists. Therefore, the CDS 13 samples the signals in these two periods by using two different pulses, and then subtracts the two signals to obtain a video signal with reduced reset noise, and through the amplifier 15. , A / D (Analog to
digital) output to the conversion unit 16.

The A / D converter 16 converts the input RGB color analog signals into digital signals and outputs them to the selector 17. The selector 17 outputs the input signal to a RAM (Random Access Memory) 18 or a detection circuit 19. The selector 17 outputs a signal input to the RAM 18 when the image data is a captured image, and outputs a signal input to the detection circuit 19 when the image data is a monitored image.
Further, the selector 17 has a function of reading out a signal once stored in the RAM 18 and outputting it to the detection circuit 19.

The detection circuit 19 calculates the brightness value or the integrated value of each of the RGB signals from the input RGB color digital signal in accordance with the control signal from the control section 20, and outputs it to the control circuit 20. WB (White Balance)
The amplifier 21 multiplies the input image signal by a white balance coefficient in accordance with the control signal input from the control circuit 20, and outputs the image signal to the signal processing circuit 22.

The signal processing circuit 22 performs processing such as compression, and outputs the signal to either one or both of the recording medium 23 and the display circuit 24. The recording medium 23 is the image processing apparatus 1.
It may be removable with respect to 0 or may be built-in. The display circuit 24 is an LCD (Liquid
A user can view the image already displayed on the LCD and recorded in the recording medium 23, the image at the time of monitoring, etc., including a crystal display). By viewing the monitoring image, the user can confirm the brightness, focus, color tone, etc. of the image to be recorded before capturing.

The program ROM 25 includes the control circuit 20.
Stores the program to be executed by. EEPROM 26
Records parameters necessary for the control circuit 20 to control the timing generator 14, for example.

The charging / light emission control circuit 27 includes a strobe 28.
The power supply necessary for the light emission of the flash is charged, and the light emission of the strobe 28 is controlled based on the control signal input from the control circuit 20. The strobe 28 emits light under the control of the charging / light emission control circuit 27 by the power source supplied from the charging / light emission control circuit 27. The shutter button 29 is a button operated by the user when the imaged image is recorded on the recording medium 23.

Although not shown, the control circuit 20 outputs a control signal to each section of the image processing apparatus 10, and each section performs processing based on the control signal.

Next, referring to the flowchart of FIG.
The operation of the image processing apparatus 10 will be described. Step S
At 11, the image capturing in the image processing apparatus 10 is started. The image capturing is started, for example, when the power of the image processing apparatus 10 is turned on.

In step S12, it is determined whether or not the shutter button 29 for instructing image pickup and recording is operated. The control circuit 20 has a shutter button 29.
Is constantly monitored, and the determination in step S12 is performed based on the monitoring.

If it is determined in step S12 that the shutter button 29 has not been operated, step S12
In step 13, the captured image is treated as a monitoring image, and when it is determined that the shutter button 29 has been operated, the process proceeds to step S14, and the captured image is treated as a capture image.

The processing as the monitoring image in step S13 will be described. C through lens 11
The light received by the CD 12 is converted into an electric signal. C
The signal from the CD 12 is converted into a signal with noise reduced by the CDS 13 and output to the amplifier 15. Amplifier 15
Amplifies the level of the input signal to a signal of a magnitude required by the subsequent processing.

The signal that has been set to a predetermined level by the amplifier 15 is input to the A / D conversion circuit 16. The signals up to the A / D conversion circuit 16 are processed as analog signals, but are converted into digital signals by the A / D conversion circuit 16.
The digital signal converted by the A / D conversion circuit 16 is
It is output to the selector 17. The selector 17 selects the output destination of the input digital signal according to the control signal from the control circuit 20.

The control circuit 20 instructs the selector 17 to switch the output destination to the RAM 18 only when it is determined that the shutter button 29 has been operated. In this case, it is determined that the shutter button 29 has not been operated, and the imaged image has been determined to be handled as a monitoring image.
The input signal is detected by the detection circuit 19 and the WB amplifier 21.
Output to.

The selector 17 may output the signal to the detection circuit 19 while outputting the signal to the WB amplifier 21. Since the processing by the detection circuit 19 is not always necessary, it can be omitted. In a situation where it can be omitted, the processing by the detection circuit 19 is omitted, that is, without the detection circuit 19, The signal may be directly output to the WB amplifier 21.

However, in the present embodiment, as will be described later, since the gain used for the white balance adjustment performed by the WB amplifier 21 is set by using the data of the monitoring image, the result detected by the detection circuit 19 is obtained. Must be stored in the control circuit 20. Therefore, the detection circuit 19 to the control circuit 20 and W
If a signal is supplied to the B amplifier 21, the signal may not be directly exchanged from the selector 17 to the WB amplifier 21.

The RGB digital signals supplied from the detection circuit 19 to the control circuit 20 are further supplied to the WB amplifier 21, and the gain set at that time is used to adjust the white balance, and the signal processing circuit 22. Is output to. The signal processing circuit 22 performs processing such that an image corresponding to the input digital signal is displayed on the display circuit 24.

As described above, when the shutter button 29 is not operated, the captured image is the display circuit 2
Since it is displayed on the display unit composed of 4 LCD etc.,
The user can browse (monitor) the displayed image.

On the other hand, step S in the flowchart of FIG.
When it is determined in 12 that the shutter button 29 has been operated, the process proceeds to step S14, and the process of handling the captured image as a captured image is started. The processing as a captured image performed in step S14 will be described with reference to the flowchart in FIG.

Basically, the operation until the light input to the lens 11 is processed by each part and input to the selector 17 is the same as the above-described processing as the monitoring image, and therefore its description is omitted. . In step S21, the signal input to the selector 17 is output to the RAM 18 and stored therein. The signals stored in the RAM 18 are
In step S22, the signal is read out based on the instruction from the control circuit 20 and supplied from the selector 17 to the detection circuit 19.

The detection circuit 19 calculates a brightness value, an integrated value and the like from the input signal and outputs it to the control circuit 20.
The control circuit 20 has a storage unit that stores image data regarding each of the captured image and the monitoring image output from the detection circuit 19. Figure 4
3 is a functional block diagram of a control circuit 20. FIG. Control circuit 20
Is a monitoring image storage unit 4 for storing image data relating to the monitoring image output from the detection circuit 10.
1 and a capture image storage unit 42 that stores image data relating to the capture image.

The data stored in the monitoring image storage unit 41 is the data of the image taken except when the shutter button 29 is operated, and new data is sequentially input and updated. The data stored in the captured image storage unit 42 is data of an image captured when the shutter button 29 is operated. The data stored in the monitoring image storage unit 41 includes at least the data acquired immediately before the data stored in the capture image storage unit 42, and preferably the data from several seconds before to immediately before is stored. It is better to set it.

The data stored in the monitoring image storage unit 41 and the data stored in the captured image storage unit 42 are output to the WB amplifier 21 as needed. Further, the data stored in the monitoring image storage unit 41 and the data stored in the captured image storage unit 42 are also output to the calculation unit 43. The calculation unit 43
Using the acquired data, various numerical values that determine the gain of the white balance described later are calculated. When the gain is calculated by the calculation unit 43, the setting unit 44 sets the WB gain of the WB amplifier 21, and control based on the setting is started.

Returning to the explanation of the flow chart of FIG. 3, the RG stored in the RAM 18 in step S22.
The digital signal of B is detected by the detection circuit 1 via the selector 17.
9 is supplied to the captured image storage unit 42 of the control circuit 20, and the data obtained as a result of the processing of the detection circuit 19 is supplied to and stored in the control circuit 20. At this point of time, the processing of the detection circuit 19 is also performed, and the data acquired immediately before the data stored in the captured image storage unit 42 is stored in the monitoring image storage unit 41.

In step S23, the data stored in the monitoring image storage unit 41 is read by the calculation unit 43, and the average value of the data is calculated. Similarly, in step S24, the data stored in the captured image storage unit 42 is read by the calculation unit 43, and the average value of the data is calculated.

In step S25, the calculated two average values are used to calculate the WB gain used for the white balance adjustment by the calculation unit 43. Here, the calculation of the WB gain will be described. First, the definition of each character in the following formula is described. Horizontal coordinate of pixel i Vertical coordinate of pixel j Pixel data of monitoring image M (i, j) Pixel data of captured image C (i, j) White balance WB gain Ga during monitoring Ga
in White balance WB gain Gai at capture
n ′ Average value of pixel data of monitoring image Ma (i,
j) Average value Ca (i, j) of the pixel data of the captured image

Here, it is assumed that the monitoring image and the capture image are images obtained when the same subject is imaged. As described above, the data used to calculate the WB gain is the data of the captured image and the data of the monitoring image immediately before the capture image was captured. Therefore, it is generally considered that the captured image and the monitoring image have the same subject.

The data of the monitoring image used for calculating the WB gain is preferably the data of the image immediately before the time when the captured image is acquired. Therefore, for example, when the shutter button 29 is half pressed, the image is in focus. The image data acquired when the monitoring image storage unit 4 for monitoring image
1 may be stored.

As for the white balance, the R signal, the G signal, and the B signal when a white subject is imaged are R = G = B.
The WB gain is set so that It is considered that the values obtained by multiplying the pixel data in the monitoring image and the captured image by the WB gain are the same. That is,
The following formula holds. Gain × M (i, j) = Gain '× C (i, j) (1)

When shooting in a dark place such as night, normally,
The strobe 28 is fired. In such a case, the monitoring image is an image in a state where the strobe 28 is not lit, and the capture image is an image in a state where the strobe 28 is lit. The above formula (1) is the strobe 28
This is true when it is assumed that the output image does not change by appropriately setting the WB gain before and after the emission of light.

However, in the case of an image captured at such a night, for example, when a person is photographed against the background of a night view, the light from the strobe 28 does not always illuminate the entire subject, and the entire CCD 12 is illuminated. Equation (1) in pixels
Is considered to hold. Therefore, as an index showing the color reproducibility of the entire image, the formula (1) is rewritten as the following formula (2) using the average value of the image data. Gain × Ma (i, j) = Gain '× Ca (i, j) (2)

Further, the equation (2) can be rewritten as the following equation (3). Gain × (1 / i × j) × ΣM (i, j) = Gain ′ × (1 / i × j) × ΣC (i, j) (3) From both sides of the equation (3), (1 / i × j) is deleted to obtain the following expression (4). Gain × ΣM (i, j) = Gain ′ × ΣC (i, j) (4)

Further, from the equation (4), the WB gain (Gain ') for adjusting the white balance for the target captured image is obtained by the following equation (5). Gain ′ = (Gain × ΣM (i, j)) / ΣC (i, j) (5)

Equation (5) is used when it is difficult to calculate the optimum white balance gain in a situation in which it is difficult to obtain distance information regarding the distance to the subject or information regarding the amount of strobe light emission. , A situation in which the subject is at a long distance and the strobe light does not reach even though the strobe 28 has fired, or conversely, the subject is in a short distance and the strobe light cannot be completely emitted. Is an expression that holds in.

ΣM (i, j) and ΣC in equation (5)
(I, j) can be detected as integrated data of the detection circuit 19 at the time of monitoring and at the time of capturing, respectively. Thus, in the detection circuit 19, ΣM (i, j) and Σ
When C (i, j) is detected as integral data, the detected integral data is stored in the monitoring image storage unit 41 and the captured image storage unit 42, respectively. Then, the stored integration data is used in the calculation of the WB gain in the calculation unit 43.

When the integrated data from the detection circuit 19 is used for calculating the gain, the calculating section 43 does not need to calculate the average value of the pixel data of the monitoring image or the capture image, and therefore the calculating section 43. It is not necessary to provide the processing of step S23 or step S24 as the above processing, but it may be provided as the processing of the detection circuit 19. In this way, the steps may be omitted or the part that executes the process may be changed as necessary.

When the detection data by the detection circuit 19 at the time of capture or the detection data at the time of monitoring becomes "0", the strobe light from the strobe 28 is predominant as the light source irradiated to the subject. Therefore, the WB gain matched with the strobe light is WB amplifier 21.
Be set to. In order to enable such setting, the WB gain for the strobe light source measured in advance is stored in the EEPROM 26 and is read out and used as needed.

Returning to the explanation of the flowchart of FIG. 3, in step S25, the WB gain applied to the captured image is calculated based on the equation (5). As the WB gain (Gain) at the time of monitoring, for example, the setting unit 44 may store the WB gain set immediately before, and the stored WB gain may be supplied to the calculation unit 43. However, the WB gain set at that time may be supplied from the WB amplifier 21 to the calculation unit 43.

When the WB gain used for adjusting the white balance is calculated by the calculating unit 43 in this way, the setting unit 44 adjusts the white balance with the WB gain. The setting unit 44, step S2
6, the WB is adjusted so that the white balance is adjusted based on the WB gain calculated by the calculation unit 43.
The amplifier 21 is controlled.

In this way, the WB gain is set,
The image data of the captured image whose white balance has been adjusted is output to the signal processing circuit 22 in step S27, subjected to compression processing by a predetermined compression method, and recorded on the recording medium 23.

In this way, the white balance WB
When the gain is adjusted, it is not necessary to measure (obtain) information on the distance to the subject, information on the amount of light emitted from the strobe 28, etc., and optimal white balance adjustment can be made with one shot. It can be carried out.

In the above-described embodiment, the CCD 12
CCD with RGB mosaic pattern color filter
The case of using 12 will be described. The parameters required for calculating the gains of R, G, and B for white balance are shown below. Detection data of monitoring image Mr, Mg, Mb Detection data of captured image Cr, Cg, Cb WB gain during monitoring Rgain, Ggain, Bga
in WB gain at capture Rgain ', Ggain', B
gain '

Also in this case, the equation (5) is established and the following equation (6) can be obtained by applying the equation (5). Rgain '= Rgain * Mr / Cr Ggain' = Ggain * Mg / Cg Bgain '= Bgain * Mb / Cb (6)

In equation (6), the brightness value changes due to the change in the WB gain. Therefore, when the equation (6) is rewritten so that the WB gain for the G signal has the same value at the time of capturing and at the time of monitoring, the following equation (7) is obtained. Rgain "= Rgain '/ (Mg / Cg) = Rgain × (Mr / Mg) / (Cr / Cg) Ggain" = Ggain' / (Mg / Cg) = Ggain Bgain "= Bgain '/ (Mg / Cg) = Gain × (Mb / Mg) / (Cb / Cg) (7)

The WB gain is R for G (green) data.
Since it is the ratio of the (red) data and the B (blue) data, the WB gain for each data of each color can be calculated as in Expression (7).
In the case where the WB gain is calculated using the equation (7), if the detection data of any one of the R, G, and B color signals cannot be acquired, the strobe is used as the WB gain at the time of capture. WB gain for EE
It is read from the PROM 26 and set.

As described above, the present invention can be applied to the CCD 12 having the color filter of the mosaic pattern, and according to the present invention, the information about the distance to the subject and the information about the strobe emission amount. It is possible to set the optimum WB gain by one-time imaging without acquiring such information.

In the above-described embodiment, the white balance is adjusted using the same WB gain for the entire image by using the average value (integrated data) of the entire imaged image. WB for each area
The gain may be changed.

For example, when a person is photographed in the background of a night view, the strobe light is applied to the foreground but is hardly applied to the background. When the range where the strobe light reaches and the range where the strobe light does not reach are imaged on the same screen, it is better to set the optimum WB gain and adjust the white balance separately for a more optimal white balance adjustment. It is possible to provide the user with the image that has been subjected to.

Therefore, a case where a different WB gain is set for each area will be described below. FIG. 5 is a diagram for explaining the area. Lens 11 and CCD
One image acquired by the processing of 12 is divided into a plurality of regions as shown in FIG. 5 by the detection circuit 19, and the detection is performed for each region. In the example shown in FIG. 5, one image is 7
It is divided into x7 areas.

Numbers are assigned to the respective areas so that the respective areas can be uniquely distinguished. Also in this case,
One screen is divided into 7 × 7 areas, and one area is composed of a plurality of pixels.

The processing as the captured image in step S14 of FIG. 3 when the image processing apparatus 10 divides one image into a plurality of areas for processing will be described with reference to the flowchart of FIG. The processing in step S31 and step S32 is the same as the processing in step S21 and step S22 of FIG.

In step S33, the brightness value is calculated for each area. The detection circuit 19 performs detection for each 7 × 7 region, outputs the R, G, B signals to the control circuit 20, and also outputs the calculated luminance value. The output data is stored in the capture image storage unit 42 of the control circuit 20.
Memorized in. In this case as well, similar to the above-described embodiment, the data output from the detection circuit 19 immediately before the captured image is already stored in the monitoring image storage unit 41.

In step S34, the calculation section 43 of the control circuit 20 uses the luminance value stored in the captured image storage section 42 to set the area (processing target area) used to calculate the WB gain. . The processing target area is set, for example, by setting an area having a higher brightness value than other areas as the processing target area. This is because the area where the brightness value is judged to be higher than other areas is
This is because it can be considered as an image of a subject irradiated with strobe light, and it is considered that it is better to use a WB gain different from that of other regions in the region of such an image.

In the setting of the area where the brightness value is judged to be higher than the other areas, the area to be compared is set to be the area adjacent to the area of interest. You may stay. Then, when the difference between the brightness value of the region of interest and the brightness value of the adjacent region is equal to or more than a certain value,
The area of interest may be set as the processing target area.

In the case where the processing target area is set in this way, no restriction may be set and all the corresponding areas may be set as the processing target area, or a predetermined restriction, for example, a higher rank which has a large difference from other areas. It is also possible to set a limit of up to 5 regions and set the processing target region within the limit.

The area to be processed may be set in advance as an area located near the center of the image. Normal,
The main part of the captured image is considered to be the central part of the image, and the most appropriate white balance adjustment can be performed for that part.

When the processing target area is determined by any of the methods in step S34, the image data of the monitoring image corresponding to the set processing target area is stored in the monitoring image storage section 41 in step S35.
And the average value is calculated.

Similarly, in step S36, the image data of the captured image corresponding to the set processing target area is read from the captured image storage section 42, and the average value is calculated.

The WB gain is calculated in step S37 using the average value thus calculated.
The calculation of the WB gain in step S37 is performed by the equation (5).
Alternatively, it is performed based on the equation (7), and which equation is used is determined by the CCD 12 provided in the image processing apparatus 10.

The processing of steps S37 to S39 is shown in FIG.
Since it is similar to the processing of steps S25 to S27 of
The description is omitted.

As described above, when the area is fixed in advance, the average value is calculated for each area set as the processing target area, and the WB gain is calculated, as shown in FIG. 7, for example. In such a case, an appropriate WB gain may not be set. Although FIG. 7 shows an example in which the subject is imaged in 3 × 3 9 areas,
In this case, the area where the subject is imaged is six areas in total in the left side and the center in the figure, but in each area, the image of the subject does not necessarily exist in the entire area.

Even when a part of the image of the subject exists in only a part of the area, the area is W
There is a possibility that the optimum WB gain may not be set due to the influence of a portion that is not related to the image of the subject because it is set in the processing target area when the B gain is calculated.

Therefore, as shown in FIG. 8, only the portion where the subject exists may be set as the processing target area, and the WB gain may be calculated from the area. In other words, the processing target area may not have a plurality of areas of a preset size, but may have a variable size corresponding to the size of the subject.

The operation of the image processing apparatus 10 when setting such a variable area as the processing target area is basically the same as that described with reference to the flowchart of FIG. However, in the setting of the processing target area in step S34, the area is variably set. In order to set this variable area, for example, before the capture image is captured, the flash 28 pre-flashes, and the processing target area is set based on the monitoring image captured when the pre-flash is captured. You may do it.

As described above, by making the area to be processed variable corresponding to the size of the subject, it is possible to set the WB gain suitable for the subject. Also,
It is possible to set a WB gain different from that of the subject for the portion other than the subject, and such a different WB gain is set.
By setting the gain, it is possible to obtain an image in which the WB gains of both the subject and the background are appropriately adjusted, and in which the unnatural switching of the WB gains that occurs near the boundary of the region is reduced.

In other words, the above-mentioned series of processing is the same as that shown in FIG.
The process performed by each circuit of the image processing apparatus 10 as shown in FIG. 5, particularly the process relating to the setting of the WB gain may be executed by a circuit configured in hardware, or may be configured by software and executed. You may do it. When a series of processes is executed by software, it is possible to execute various functions by installing a computer in which the programs that make up the software are built into dedicated hardware, or by installing various programs. The recording medium is installed in a general-purpose personal computer, for example.

Before describing the recording medium, a general-purpose personal computer will be briefly described. Figure 9
FIG. 3 is a diagram showing an internal configuration example of a general-purpose personal computer. CPU of personal computer (Central
Processing Unit 111 is a ROM (Read Only Memor)
y) Various processes are executed according to the program stored in 112. RAM (Random Access Memory) 11
3, data and programs necessary for the CPU 111 to execute various processes are appropriately stored. The input / output interface 115 is connected to an input unit 116 including a keyboard and a mouse, and outputs a signal input to the input unit 116 to the CPU 111. An output unit 117 including a display and a speaker is also connected to the input / output interface 115.

Further, the input / output interface 115 has a storage unit 118 including a hard disk,
Also, a communication unit 119 for exchanging data with other devices via a network such as the Internet is also connected. The drive 120 includes a magnetic disk 131, an optical disk 132, a magneto-optical disk 133, and a semiconductor memory 13.
It is used when reading or writing data from a recording medium such as No. 4 or the like.

As shown in FIG. 9, the recording medium is a magnetic disk 131 (including a flexible disk) and an optical disk 132 on which the program is recorded, which is distributed in order to provide the program to the user separately from the personal computer. (CD-ROM (Compact Disc-Read Only Memor
y), a DVD (including a Digital Versatile Disc), a magneto-optical disc 133 (including an MD (Mini-Disc) (registered trademark)), a package medium including a semiconductor memory 134, and a computer. The hard disk includes a ROM 112 in which a program is stored, a hard disk including a storage unit 118, and the like, which is provided to a user in a state where the program is installed in advance.

In this specification, the steps for writing the program provided by the medium are not limited to the processing performed in time series according to the order described, but may be performed in parallel even if the processing is not necessarily performed in time series. Alternatively, it also includes processes that are individually executed.

[0092]

As described above, according to the image processing apparatus, the method, and the program of the present invention, among the picked-up images, the average value of the image data of the images instructed to be recorded on the recording medium is the average value Set the white balance gain to be applied next by dividing the average value of the image data acquired at the time before and multiplying that value by the white balance gain set at that time. As a result, both the subject and the background are images in which the white balance gains are appropriately adjusted, and the user is provided with an image in which the unnatural switching of the white balance gains that occurs near the boundary of the area is reduced. It becomes possible.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of an embodiment of an image processing apparatus to which the present invention has been applied.

FIG. 2 is a flowchart illustrating an operation of the image processing apparatus 10.

FIG. 3 is a flowchart illustrating detailed processing of step S14 of FIG.

FIG. 4 is a diagram showing an internal configuration example of a control circuit 20.

FIG. 5 is a diagram illustrating a region.

FIG. 6 is a flowchart illustrating another detailed process of step S14 of FIG.

FIG. 7 is a flowchart illustrating a relationship between a subject and a processing target area.

FIG. 8 is a flowchart illustrating a relationship between a subject and a processing target area.

FIG. 9 is a diagram illustrating a medium.

[Explanation of symbols]

10 image processing device, 11 lens, 12 CC
D, 13 CDS, 14 timing generator,
15 amplifiers, 16 A / D conversion circuits, 17 selectors, 18 RAM, 19 detection circuits, 20 control circuits, 21 WB amplifiers, 22 signal processing circuits,
23 recording medium, 24 display circuit, 25 program ROM, 26 EEPROM, 27 charge / light emission control circuit, 28 strobe, 29 shutter button

   ─────────────────────────────────────────────────── ─── Continued front page    F term (reference) 5C065 AA01 BB02 CC01 CC09 DD02                       DD17 EE12 FF02 GG17 GG18                       GG23 GG32                 5C066 AA01 BA01 CA23 EA14 GA01                       GA33 GB01 KA12 KE05 KE09                       KE19 KM02 LA02

Claims (6)

[Claims] 1. An image pickup unit for picking up an image, an average value of first image data of the image picked up by the image pickup unit and instructed to be recorded on a recording medium, and the first image data.
Calculating means for calculating an average value of the second image data acquired at a time before the acquisition of the second image data, and an average value of the second image data calculated by the calculating means, The gain of the white balance applied to the first image data is set by multiplying the value obtained by dividing the average value of the first image data by the gain of the white balance when the second image data is acquired. An image processing apparatus, comprising: 2. The image forming apparatus further includes a dividing unit that divides the image captured by the image capturing unit into a plurality of regions, and the calculation by the calculating unit and the setting by the setting unit are performed for each of the regions. The image processing apparatus according to claim 1. 3. The image pickup device further includes detection means for detecting a region having a high luminance value in the image picked up by the image pickup means, wherein the calculation by the calculation means and the setting by the setting means are detected by the detection means. The image processing apparatus according to claim 1, wherein the image processing apparatus is performed on the created area. 4. An image capturing step of capturing an image, an average value of first image data of the image captured in the processing of the image capturing step and instructed to be recorded on a recording medium, and the first image data The calculation step of calculating the average value of the second image data acquired at the time before the acquisition and the average value of the second image data calculated by the processing of the calculation step are the first Setting for setting the white balance gain to be applied to the first image data by multiplying the value divided by the average value of the image data by the gain of the white balance when the second image data is acquired An image processing method comprising: 5. An image capturing step of capturing an image, an average value of first image data of the image captured in the processing of the image capturing step and instructed to record on a recording medium, and the first image data The calculation step of calculating the average value of the second image data acquired at the time before the acquisition and the average value of the second image data calculated by the processing of the calculation step are the first Setting for setting the white balance gain to be applied to the first image data by multiplying the value divided by the average value of the image data by the gain of the white balance when the second image data is acquired A recording medium having a computer-readable program recorded thereon, comprising: 6. An image capturing step of capturing an image, an average value of first image data of the image captured by the processing of the image capturing step and instructed to be recorded on a recording medium, and the first image data. The calculation step of calculating the average value of the second image data acquired at the time before the acquisition and the average value of the second image data calculated by the processing of the calculation step are the first Setting for setting the white balance gain to be applied to the first image data by multiplying the value divided by the average value of the image data by the gain of the white balance when the second image data is acquired A program that causes a computer to perform steps and.