JPH09214985A - Image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain an image pickup device that detects an accurate color temperature independently of a luminance signal level by providing a prescribed color temperature information acquisition means or the like in the case of image pickup of a still image. SOLUTION: A signal generating means generates a chrominance signal representing a color, a saturation signal representing saturation, and a luminance signal representing luminance based on an output of an image pickup element having plural color filters whose spectral characteristics differ. A signal ratio generating means generates a chrominance/luminance ratio signal denoting a ratio of the chrominance signal to the luminance signal and a saturation/ luminance ratio signal denoting a ratio of the saturation signal to the luminance signal. A color temperature acquisition means obtains color temperature information of a picked-up image based on the luminance signal denoting luminance, the chrominance/luminance ratio signal and the saturation/luminance ratio signal. According to the processing method as above, even when a white color is comparatively small in an image, accurate white balance processing is conducted and even when no white color part is in existence in an image, a white balance adjustment value is obtained with simple processing.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an imaging device,
In particular, it is suitable for use in an image pickup device that picks up a still image.

[0002]

2. Description of the Related Art FIG. 12 is a block diagram schematically showing a conventional digital electronic camera. In FIG. 12, 300
Is a digital electronic camera, and 308 is a recording medium such as a memory card.

In the digital electronic camera 300, 30
Reference numeral 1 denotes an imaging lens whose focal length and angle of view can be adjusted, 302 denotes a shutter that also serves as an aperture and shutter function, and 304 denotes an imaging lens 301 and a shutter that also serves as aperture 3.
02 is a mechanical system drive circuit.

Reference numeral 101 is an image sensor for converting light reflected from a subject into an electric signal, 103 is a timing signal generating circuit (hereinafter referred to as TG) for generating a timing signal necessary for operating the image sensor, and 102 is a timing signal. An image pickup element drive circuit that amplifies a signal from the generation circuit to a level capable of driving an image pickup signal, 104 is a preprocessing circuit (preprocessing means) including a CDS circuit or an AGC circuit for removing output noise of the image pickup element 101, Reference numeral 105 is an A / D converter.

Reference numeral 303 denotes an A / D converter 105
An image pickup signal processing circuit having a circuit for processing an image pickup signal for processing a D-converted digital signal, 108 is a system control CPU, 116 is an operation display unit for displaying a display for assisting operation and the state of the camera, and 118 is a camera photographing A switch for controlling the start by an external photographer. The first stroke SW
Two-step operation of 1 and the second stroke SW2 is possible.

A recording medium I / F 307 connects the digital electronic camera 300 and the recording medium 308.
Reference numeral 305 is a device-supplied distance measuring circuit for measuring the distance between the image pickup device and the subject for focus adjustment of the optical system 301, and 306.
Is a photometric circuit that measures the brightness of the subject.

FIG. 13 is a block diagram showing the internal structure of the image pickup signal processing circuit 303 and its surroundings. In FIG. 13, reference numeral 101 denotes an image sensor (CC
D), the imaged signal is input to the A / D converter 105 via the pre-processing unit 104, converted into a digital signal in the A / D converter 105, and then input to the image pickup signal processing circuit 303. .

The signal input to the image pickup signal processing circuit 303 is once stored in the memory 410 or directly input to the 401 enhancer / CSEP circuit. Then, the luminance signal and the chrominance signal are separated by using the enhancer / C SEP circuit 401 and the delay device 402, and the luminance signal is emphasized / emphasized in the high frequency part.
Compensation is provided.

The enhancer / C SEP circuit 401
The color signal output from the CCD color filter (Mg, G, Cy, Y
e) are added signals (Wr, Wb, Gr, Gb), and the same color signals are converted into RGB signals in the CMTX 403.

Next, the first low-pass filter 404 filters the luminance signal Y and the color signal.
Then, the white balance adjustment circuit (WB) 405 is provided. The white balance adjustment circuit (WB) 40
5 controls each of the gains of the color signals RGB according to the AWB 408.

Reference numeral 406 denotes a luminance signal Y and a color signal RGB.
Is a gamma circuit that performs gamma processing on the. CY
The MTX 407 is a matrix circuit that converts the gamma-processed RGB signals into color difference signals (RY) and (BY).

The AWB 408 is an RGB signal or C-
Y MTX407 output (RY) signal, BY signal and Y
A control circuit for integrating a signal for a certain period of time and generating a control signal for white balance control by controlling the gains of the R signal and the B signal in the white balance adjusting circuit (WB) 405 according to the integrated value. Is.

The second low pass filter 409 is a filter for filtering color difference signals RGB. Then, the luminance signal Y and the color difference signals (RY) and (BY) are stored in the memory 4
After storing one image in 10, the compression circuit 411 performs data compression processing. Reference numeral 412 is a pixel addition circuit for performing pixel addition in the vertical direction when the frame is read from the CCD.

FIG. 14 is a detailed drawing of the control circuit (AWB) 408. When a color difference signal (RY, BY) is input, a large amplitude exceeding a predetermined value with respect to the amplitude of the color difference signal (RY, BY) is shown. It has a clipping circuit that limits the input of.

The color difference signal (RY) and the color difference signal (BY) that have passed through the clipping circuit and the luminance signal Y are
For example, one screen is divided into a plurality of blocks, and each divided block is integrated.

In the coordinates formed by the color difference signals (RY) and (BY), the change range when the color temperature of white changes is set as the white determination range (as shown in FIG. 25), and the integrated value of each block has the same coordinate. In the above, the system control CPU 108 determines whether or not it is within the white determination range, and determines the gains of the R signal and the B signal by using the integrated value within the white determination range, and controls according to this gain. The value adjusts the gain of each R / B.

[0017]

The white balance control according to the above-mentioned conventional example converges to a control state with the lapse of time for adapting to the shooting of a moving image. However, in order to apply the white balance control based on the above-mentioned conventional example in still image shooting, the image pickup device is once exposed before image shooting (main exposure).

Then, a color difference signal is created from the exposed signal by using a signal processing circuit, and the color temperature condition is detected from the position on the coordinates of the color difference signals (RY) and (BY) by the conventional white balance detection to detect the same color temperature. After getting the adjusted value in
The main exposure is performed, and the gain adjustment is performed using the previously obtained adjustment value to achieve white balance.

As described above, it is necessary to detect the accurate color temperature with one white balance detection, but as in the conventional method, coordinates are made with the color difference signals (RY) and (BY), and white discrimination is performed on the same coordinates. The method for setting the range is that when a certain color is photographed at low brightness and at a certain color temperature (especially within the white judgment range
(Color temperature that is far from the intersection of the RY and BY axes)
There is a drawback in that it is difficult to specify the color temperature because it is in the white discrimination range both when the white is photographed and the color temperature cannot be accurately detected.

For example, as shown in FIG. 16, the relationship between the color difference signal (RY) and the luminance signal Y is as follows. When the color temperature change is between α and β, white is 3 points a, b and o. It exists in the area surrounded by two. On the other hand, a certain color (color a) has points c, d, and o.
It shall exist in the area surrounded by the three points. When white is discriminated by the level (e) of the color difference signal (RY) as in the conventional case, (a color) in the area (F) is also included, which has a problem of hindering color temperature detection.

In view of the above problems, it is an object of the present invention to enable accurate color temperature detection independent of the luminance signal level.

[0022]

An image pickup apparatus according to the present invention represents a color signal representing a color, a saturation signal representing a saturation, and an illuminance on the basis of an output of an imaging device having a plurality of color filters having different spectral characteristics. A signal generating means for generating an illuminance signal, a color / illuminance ratio signal indicating a ratio between the color signal and the illuminance signal, and a saturation / representing a ratio between the saturation signal and the illuminance signal.
A signal ratio generating unit that generates an illuminance ratio signal, and a color temperature information acquiring unit that obtains color temperature information of a captured image from the illuminance signal representing the illuminance, the color / illuminance ratio signal, and the saturation / illuminance ratio signal. It is characterized by doing.

Another feature of the present invention is that
An image sensor having three or more types of color filters having different spectral characteristics, and a predetermined color representing a difference between signals output from the image sensors corresponding to a predetermined two-color filter of the three or more types of color filters A difference between signals output from a predetermined color filter output difference signal generating unit that generates a filter output difference signal and an image sensor corresponding to a predetermined external color filter that is a combination of color filters other than the predetermined two-color filter. A predetermined external color filter output difference signal generating means for generating a predetermined external color filter output difference signal,
Luminance signal value creating means for creating a brightness signal value, the predetermined color filter output difference signal, and the predetermined two-color filter that created the predetermined color filter output difference signal. First ratio signal generating means for generating a first ratio signal representing a ratio with the signal output from the element, the predetermined external color filter output differential signal, and the predetermined external color filter output differential signal are created 2 Of the two color filters, second ratio signal generating means for generating a second ratio signal representing a ratio of the signal output from the image sensor of one of the color filters; the first ratio signal; With respect to the second ratio signal, a white change range setting means for setting a white change range when the color temperature changes, a first ratio signal and a second ratio signal obtained from a plurality of pixels of the image sensor. Of the plurality of pixels determined to be within the white range when the white change range determination means determines that the white change range is determined to be within the white change range. An image pickup device corresponding to three or more different types of color filters is based on an integration unit that weights and integrates the output of the image pickup device corresponding to each color filter using a luminance signal, and an integrated value output from the integration unit. And a gain control value acquisition means for obtaining a gain control value for the output of the element.

Further, another feature of the present invention is that, in the image pickup apparatus according to the second aspect, when the white change range determining means determines that the white change range is present, the white change range is determined to be within the white range. Three or more different types of image pickup device outputs corresponding to the determined color filters of the plurality of pixels are integrated based on the integration value output from the integration unit and the integration unit that weights and integrates the outputs using the luminance signal. When it is determined by the first gain control value acquisition unit that obtains the first gain control value for the output of the image sensor having a color filter and the white change range determination unit, the gain control value acquisition unit determines that the white gain range is not within the white range,
From the image sensor output signals of a plurality of pixels which are determined not to be within the white range and the first ratio signal and the second ratio signal at the boundary of the white range, the plurality of image sensor outputs at the boundary of the white range are output. Pseudo signal calculating means for calculating a plurality of corresponding pseudo signals, and second gain control for outputs of an image pickup device having three or more types of color filters different from each other by integrating the plurality of pseudo signals respectively. Second gain control value acquisition means for obtaining a value, pixel number detection means for detecting the number of pixels within the white range and number of pixels not within the white range, and number of pixels detected by the pixel number detection means A hybrid ratio calculating means for calculating a hybrid ratio, and the first gain control value and the second gain control value are mixed according to the hybrid ratio calculated by the hybrid ratio calculating means, and the different And a third, further comprising a gain control value obtaining means for obtaining a third gain control value for the image sensor output having three or more types of color filters.

Another feature of the present invention is that it has an image pickup device having four types of color filters having different spectral characteristics, and outputs from the image pickup device corresponding to the above four types of color filters, respectively. A first / second difference signal for generating a first / second difference signal representing a difference between the first image sensor output signal and the second image sensor output signal from the first to fourth image sensor output signals Generating means, third / fourth difference signal generating means for generating a third / fourth difference signal representing a difference between the third image sensor output signal and the fourth image sensor output signal, and the first ~ Weighting signal generating means for generating a weighting signal using the fourth image sensor output signal, the first / second difference signal, and the first image sensor output signal or the second image sensor output signal Ratio of the first / second difference that represents the ratio of No., and a third / fourth difference ratio signal indicating the ratio of the third / fourth difference signal and the third image sensor output signal or the fourth image sensor output signal, when the color temperature changes Corresponding to each color filter of a plurality of pixels determined to be within the white range when the white change range determining means determines that the white change range is within the white range. Based on the four integration values output from the weighted integration means, weighted integration means for weighting and integrating the output of the image pickup element using the weighting signal, and the first to fourth imaging operations. A first gain control value generating means for generating a first gain control value for the element output signal.

Further, another feature of the present invention is that, in the image pickup device according to claim 4, the first or second pixel of a plurality of pixels judged by the white change range judging means not to be within the white range. Image sensor output signal, the third or fourth image sensor output signal, the ratio signal of the first / second difference at the boundary portion of the white range, and the third / fourth
A second / first pseudo signal corresponding to the second or first image sensor output signal at the boundary portion of the white range is calculated from the ratio signal of the difference, and the fourth or third pseudo signal is calculated.
Pseudo signal calculating means for calculating fourth / third pseudo signals corresponding to the image sensor output signal, the first or second image sensor output signal, and the third or fourth image sensor output signal, The second / first pseudo signal and the fourth / third
The integrating means for integrating the pseudo signal and the four integrated values output from the integrating means are used to determine the first to fourth values.
Second gain control value generation means for generating a second gain control value for the image pickup device output signal, pixel number detection means for detecting the number of pixels that fall within the white range and the number of pixels that do not fall within the white range, and A hybrid ratio calculating means for calculating a hybrid ratio according to the number of pixels detected by the pixel number detecting means, and the first gain control value and the second gain according to the hybrid ratio calculated by the hybrid ratio calculating means. It is further characterized by further comprising a third gain control value generating means for generating a third gain control value for the first to fourth image sensor output signals by mixing with the control value.

Another feature of the present invention is that, in the image pickup apparatus according to any one of claims 2 to 5, it is determined that the weighting means falls within the white range. When the above-mentioned first to fourth image sensor output signals are integrated, when the luminance signal value is higher than the luminance value calculated in advance, the four output signals are respectively multiplied by N, and the luminance signal value is higher than that of the lower luminance signal value. , The effect of the four output signal values on the integrated value is increased.

Another feature of the present invention is that it represents a color signal representing a color, a saturation signal representing a saturation, and an illuminance based on the output of an image sensor having a plurality of color filters having different spectral characteristics. A signal generating means for generating an illuminance signal, a color / luminance ratio signal indicating a ratio between the color signal and the luminance signal, and a signal generating a saturation / luminance ratio signal indicating a ratio between the chroma signal and the luminance signal. It is characterized by comprising a ratio generation means, and an illuminance signal representing the illuminance, a color temperature information acquisition means for obtaining color temperature information of a captured image from the color / luminance ratio signal and the saturation / luminance ratio signal.

Another feature of the present invention is that it has an image sensor having three or more types of color filters having different spectral characteristics, and among the three or more types of color filters described above,
A predetermined color filter output difference signal generating means for generating a predetermined color filter output difference signal representing a difference between signals output from the image pickup devices corresponding to the predetermined two color filters; and a color filter other than the predetermined two color filter. A predetermined external color filter output difference signal generating means for generating a predetermined external color filter output difference signal representing a difference between signals output from the image sensors corresponding to a combination of predetermined external color filters, and an exposure amount of the image sensor. It corresponds to one of the color filters of the exposure amount signal generating means that generates the exposure amount signal that represents, the predetermined color filter output difference signal, and the predetermined two-color filter that created the predetermined color filter output difference signal. First representing the ratio with the signal output from the image sensor
In the first ratio signal generating means for generating the ratio signal, the predetermined external color filter output difference signal, and the two color filters that have created the predetermined external color filter output difference signal. A second ratio signal for generating a second ratio signal representing a ratio with the signal output from the corresponding image sensor
Ratio signal generating means, the first ratio signal and the second ratio signal
With respect to the ratio signal, the white change range setting means for setting a change range of white when the color temperature changes, and whether or not the first ratio signal and the second ratio signal fall within the white change range. A white change range determining means for determining, and an image sensor output corresponding to each color filter of a plurality of pixels determined to be within the white range when it is determined to be within the white range by the white change range determining means, Based on the integrating means for weighting and integrating using the exposure amount signal and the integrated value output from the integrating means, gain control values for the outputs of the image pickup devices corresponding to three or more different color filters are obtained. And a gain control value acquisition means for obtaining the gain control value.

Another feature of the present invention resides in the white range in the image pickup device according to the eighth aspect, when the white change range determining means determines that the white change range is present. The output of the image pickup device corresponding to each color filter of the plurality of pixels determined to be different from each other based on the integrating means for weighting and integrating with the exposure amount signal and the integrated value output from the integrating means. First gain control value acquisition means for obtaining a first gain control value for the output of an image sensor having three or more types of color filters, and imaging of a plurality of pixels determined by the white change range determination means to be outside the white range. From the element output signal and the first ratio signal and the second ratio signal at the boundary of the white range, pseudo signals corresponding to the outputs of the plurality of image sensors at the boundary of the change range are calculated. A second gain control value that obtains a second gain control value for the output of the image sensor having three or more different color filters from the plurality of integration values by integrating each of the plurality of pseudo signals with the pseudo signal calculation means. An acquisition unit, a pixel number detection unit that detects the number of pixels that fall within the white range and a number that does not fall within the white range, and a blend ratio calculation that calculates a blend ratio according to the number of pixels detected by the pixel number detection unit. Means, and the first gain control value and the second gain control value according to the hybrid ratio calculated by the hybrid ratio calculating means.
And a third gain control value acquisition means for obtaining a third gain control value for the output of the image pickup device having three or more different types of color filters. .

Another feature of the present invention is that it has an image pickup device having color filters of four colors having different spectral characteristics, and outputs from the image pickup device corresponding to the color filters of the above four colors. ~ First / second difference signal generation for generating a first / second difference signal representing a difference between the first image sensor output signal and the second image sensor output signal from the fourth image sensor output signal Means, and third / fourth representing the difference between the third image sensor output signal and the fourth image sensor output signal
Third / fourth difference signal generating means for generating a difference signal, exposure amount signal generating means for generating an exposure amount signal representing the exposure amount of the image pickup device, the first / second difference signal and the first A ratio signal of the first / second difference representing the ratio with the image sensor output signal or the second image sensor output signal, and the third signal
/ For the ratio signal of the third / fourth difference representing the ratio of the fourth difference signal and the output signal of the third image sensor or the output signal of the fourth image sensor, the change range of white when the color temperature changes is set. White change range setting means for setting, white change range determining means for determining whether or not the ratio signal of the first / second difference and the ratio signal of the third / fourth difference are within the white change range, When the white change range determining means determines that the white change range is present, the first to fourth image sensor output signals of the plurality of pixels that are determined to be within the white range are weighted using the exposure amount signal. And a gain control value generation means for generating a gain control value for the first to fourth image sensor output signals based on the four integrated values output from the weighted integration means. Specially equipped with It is set to.

Another feature of the present invention is that, in the image pickup device according to claim 10, the first or second pixel of a plurality of pixels judged by the white change range judging means to be outside the white range. Image sensor output signal of the
Alternatively, from the fourth image sensor output signal and the ratio signal of the first / second difference and the ratio signal of the third / fourth difference at the boundary portion of the white range, the second or first ratio signal at the boundary portion of the change range is obtained. 2 / corresponding to the image sensor output signal of
Pseudo signal calculating means for calculating the first pseudo signal and also for calculating a fourth / third pseudo signal corresponding to the fourth or third image sensor output signal, and the first or second image sensor output signal An integrating means for integrating the third or fourth image sensor output signal, the second / first pseudo signal, and the fourth / third pseudo signal, and the above-mentioned 4 output from the integrating means. Second gain control value generation means for generating a second gain control value for the first to fourth image sensor output signals from one integrated value, and the number of pixels that fall within the white range and the number of pixels that do not fall within the white range. Detecting the number of pixels, a hybrid ratio calculating unit for calculating a hybrid ratio according to the number of pixels detected by the pixel number detecting unit, and the first ratio according to the hybrid ratio calculated by the hybrid ratio calculating unit.
Third gain control value generating means for generating a third gain control value for the first to fourth image sensor output signals by mixing the second gain control value and the second gain control value. It is characterized by doing.

Another feature of the present invention is that, in the image pickup apparatus according to any one of claims 8 to 11, the weighting means is determined to fall within the white range. When integrating the first to fourth image sensor output signals, the color temperature of the subject is predicted in advance using the exposure amount signal to divide the white change region, and the first / second
If it is determined that the difference ratio signal and the third / fourth difference ratio signal fall within the previously predicted white change region,
It is characterized in that the output signals of the four image pickup devices are multiplied by N to increase the influence of the four output signal values on the integrated value.

[0034]

In order to solve the above problems, in the present invention, the slopes of the color difference signals (RY) and (BY) with respect to the luminance signal Y, that is, (RY) / Y and (BY) / Y are the luminance signal Y. Since the value is determined from the color temperature and the shooting color regardless of the level, the coordinates are created from (RY) / Y, (BY) / Y to determine the white judgment range, and the (RY) / Y of the shot image, (BY) / Y and luminance signal Y,
Alternatively, the exposure amount of the image pickup device is used for weighting and the color temperature is detected, so that accurate detection independent of the luminance signal level can be performed.

Further, by weighting the signal value in the white discrimination using the luminance signal Y or the exposure amount of the image pickup device, even if the ratio of white in the image is low, the influence of other colors is reduced. Therefore, it is possible to obtain a more accurate white balance signal.

According to another feature of the present invention, the gradients (RY) / Y and (BY) / Y of the color difference signals are replaced with (Mg-G) / (Mg +
G), (Ye-Cy) / (Ye + Cy) or (Mg-G) / (Mg + G + Cy + Ye),
By making coordinates by (Ye-Cy) / (Mg + G + Ye + Cy) and detecting the color temperature, a simple signal can be obtained from the read signal of the image sensor with the (Mg, G, Cy, Ye) complementary color filter. Since the color temperature can be directly detected by the processing, the exposure and signal processing for the color temperature detection before the main exposure can be omitted, and the simple and accurate color temperature detection can be performed.

Here, by defining the molecules as (Mg-G) and (Ye-Cy), the color temperature change direction is substantially perpendicular to the (Mg-G) axis and substantially parallel to the (Ye-Cy) axis. Therefore, the white determination range can be easily set. Also, in the case of an image sensor with a (R, G, B) pure color filter, (RB) / (R + G + B), (R
+ BG) / (R + G + B) creates coordinates to detect color temperature, thus omitting exposure and signal processing for color temperature detection before main exposure, similar to an image sensor with a complementary color filter Therefore, it is possible to perform accurate color temperature detection by simple processing from the image sensor output signal.

According to another feature of the present invention, the gradients (RY) / Y and (BY) / Y of the color difference signals are replaced by (Mg-G) / G,
Or by making coordinates by (Mg-G) / Mg and (Ye-Cy) / Cy, or (Ye-Cy) / Ye and detecting the color temperature, (Mg, G,
(Cy, Ye) Since the color temperature can be detected directly from the read signal of the image sensor having the color filter, the exposure and signal processing for detecting the color temperature before the main exposure can be omitted,
A simple and accurate color temperature detection is possible.

Further, even when photographing an object containing only a small amount of white portion, by giving coordinates within the white discrimination range in the form of (Mg-G) / G = a, (Ye-Cy) / Cy = b , It is possible to replace the image sensor output by simple processing in the form of Mg ′ = (1 + a) * G, Ye ′ = (1 + b) * Cy within the white discrimination range, even under the above shooting conditions. It is possible to adjust the white balance so that the white balance does not shift significantly.

Further, in the case of an image pickup device having a (R, G, B) pure color filter, the coordinates are made by (RB) / G and (R + BG) / G to detect the color temperature, and thereby the complementary color filter is obtained. Similarly to the image sensor having the above, it is possible to omit the exposure and the signal processing for the color temperature detection before the main exposure, and it is possible to perform the accurate color temperature detection by the simple process from the image sensor output signal, Even when shooting a subject that contains only a small portion, by giving coordinates within the white discrimination range in the form of (RB) / G = a, (R + BG) / G = b, R'in the white discrimination range
= (1 + a + b) * G / 2, B '= (1-a + b) * G / 2 The image sensor output can be replaced by simple processing, even under the above shooting conditions. The white balance can be adjusted so that the white balance does not shift significantly.

[0041]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of an image pickup apparatus of the present invention will be described below with reference to the drawings. 1 and 2 are flowcharts for explaining the operation of the image pickup apparatus of the present embodiment, and FIG. 3 is a block diagram showing the flow of signal processing of the electronic camera according to the first embodiment of the image pickup apparatus of the present invention. Is.

Since the image sensor in this embodiment performs frame reading, output signals M, G, C, Y from the image sensor are output.
Appears according to the filter attached to the image sensor. First, each filter output in the optical black region is integrated by the integrator 31, and the integrated value is given to the black level adjusting circuit 32 to adjust the black level.

Next, the white balance detection circuit 33 performs white balance detection, the white balance detected output is given to the gain adjustment circuit 34, and the output gains of Ma, Gr, Cy, and Ye at the white points are the same. The gain is adjusted so that

On the other hand, the luminance signal (Y) is subjected to aperture correction after passing through the low-pass filter 35 in order to remove moire due to the color filter. Here, the high-pass filters 36a and 36b are passed through horizontally and vertically, respectively.

After that, the gain is adjusted by the gain adjusting circuits 37a and 37b. After that, the amplitude adjusting circuits 38a, 3
In 8b, a small amplitude signal of the aperture signal is subjected to a process for increasing the amplitude, and a large amplitude signal is subjected to a Peak To Peak process for decreasing the amplitude.

Next, the base clip (BC) circuit 39
After the processing of cutting a signal having a low noise level is performed in a and 39b, horizontal and vertical are added in the adding circuit 40. Further, after the gain adjustment is performed in the gain adjustment circuit 41, it is added to the luminance component (Yn) in the addition circuit 42.

In order to generate the color difference signals (RY, BY), first, the synchronizing circuit 50 synchronizes the four color signals M, G, C, Y. Next, in the pixel interpolation circuit 51, pixel interpolation is performed so that four color signals M, G, C, Y are assigned to the entire area of the image pickup surface.

Next, the 4 × 3 first matrix circuit 52
To convert the MGYS signal into an RGB signal. At this time,
The first matrix circuit 52 uses a matrix that determines the color temperature of the subject from the white balance information and maximizes color reproduction at that color temperature.

Next, in the second matrix circuit 53, the RGB signal is converted into a (RY) signal and a (BY) signal and is passed through a low pass filter 54 to perform low pass filter processing for reducing moire. Next, the color processing circuit 5
In 5, the saturation achromatic process is performed with reference to the Y signal.

Luminance signal (Y) produced in this way
And the color difference signals (RY, BY) are converted into RGB signals by the color decoder 56, and then subjected to gamma correction processing in the gamma processing circuit 57 and then recorded.

Next, the white balance processing, which is the main point of this embodiment, will be described in detail. When the CCD complementary color filters are arranged as shown in FIG.
The pixel array is used as a unit block, and the entire array is formed by repeating this unit block.

The outputs of these eight pixels are matched with the respective color filters by Ma1, Ma2, Gr1, Gr2, Cy1 and Cy2.
, Ye1 and Ye2. Note that the outputs of the same color filters are two colors (Ma1 and Ma2, Gr1 and Gr2, Cy1 and Cy2, Ye1).
And Ye2) are treated separately because the output levels of pixels of the same color filter are different from each other due to the colors of adjacent pixels being different.

The white balance processing will be described in detail below with reference to FIGS. 1 and 2. First, the first step S
In 11, the sample data is extracted for each sample.

Next, in step S12, it is determined whether the sample data extracted in step S11 is valid as a WB sample point. As a result of the determination, if the extracted sample data is valid as the WB sample point, the process proceeds to step S13, and if it is not valid, the process starts from step S11.

In step S13, it is determined whether or not there is brightness that can be calculated as WB data for all of the 8 sampled sample points, and is performed using the following formula. DarkThreshold <PixelValue <BrightThreshold where DarkThreshold: predetermined value BrightThreshold:-"-PixelValue: sample point 8 pixels

As a result of the judgment in step S13,
If all eight pixels of the sample points satisfy the following equation, the process proceeds to step S14, and if none of them satisfies the following equation,
Move the sample point to the next step S
Start over from 11.

In step S14, the threshold value of the edge is determined. This checks whether all 8 pixels of the sample point satisfy the following equation. EdgeThreshold> | Ma1-Ma2 | EdgeThreshold> | Gr1-Gr2 | EdgeThreshold> | Cy1-Cy2 | EdgeThreshold> | Ye1-Ye2 | Where EdgeThreshol: is a predetermined value.

As a result of the determination in step S14, if all eight pixels are satisfied, the process proceeds to step S15. If none of them is satisfied, it is determined that the sample point includes an edge, and the next sample point is determined. The process shifts to step S11 and starts over.

In step S15, the value of (X1, Y1, X2, Y2) is obtained from the following equations (1) to (4) at each sample point. X1 = (Ye1-Cy1) / Cy1… (1) Y1 = (Ma1-Gr1) / Gr1… (2) X2 = (Ye2-Cy2) / Cy2… (3) Y2 = (Ma2-Gr2) / Gr2… ( 4)

Next, in step S16, as shown in FIG. 4, taking (Ye-Cy) / Cy as the X axis and (Ma-Gr) / Gr as the Y axis, equations (1) to (4) It is determined whether or not it is within a predetermined white area provided on the XY axes.

As a result of this judgment, when it is within a predetermined white area, that is, the following conditional expressions, a <X1 <b, c <Y1 <d and a <X2 <b, c <Y2 <d (5) If the above condition is satisfied, the sample point is regarded as valid as data for performing the white balance WB setting, and the process proceeds to step S17. If the above equation (5) is not satisfied, the above data is regarded as invalid and the process proceeds to step S27.

In step S17, first, at the sample point, using the following equation, the average value of the XY axis values of the sample point in the WB coordinate, and
Calculate the brightness value. AverageX = (X1 + X2) / 2 AverageY = (Y1 + Y2) / 2 AverageBrightness = (Ma1 + Ma2 + Gr1 + Gr2 + Cy1 + Cy2 + Ye1 + Ye
2) / 8

Next, in step S18, it is determined whether the sample data is valid data for the first white balance WB setting. Then, if the sample point is the first sample point regarded as valid data for setting, the process proceeds to step S19, and if not, the process proceeds to step S21.

In step S19, a sample point group having a value as a parameter is created as shown in FIG. 5, and the value obtained in step S17 and the pixel value of the same point are substituted. That is, the following calculation is performed. GpX = AverageX GpY = AverageY GpBrightness = AverageBrightness GpMa1 = Ma1 GpMa2 = Ma2 GpGr1 = Gr1 GpGr2 = Gr2 GpCy1 = Cy1 GpCy2 = Cy2 GpYe1 = Ye1 GpYe2 = Ye2 GpNumber = 1 GpWeight

After that, the process proceeds to step S20, and it is determined whether or not the extraction of all sample data is completed. And
If extraction of all sample data has not been completed yet, the process returns to step S11 and the above-described processing is repeated.

On the other hand, if the result of determination in step S18 is that the sample point is not the first sample point regarded as setting valid data, step S21.
Proceed to and determine whether it is included in the sample point group.

As a result of the above judgment, if it is included, the process proceeds to step S22, and the process of including it in the sample point group is performed. If the result of determination in step S21 is that the sample point group is not included, the process proceeds to step S23, it is determined whether or not the sample point group is finished, and if it is not finished, the process returns to step S21, and the sample point group is finished. Then, the process proceeds to step S19.

Steps S21 to S22 described above
If the same sample point is not the first sample point that is considered to be valid WB setting data, the values of GpX, GpY, and GpBrightness of the existing sample point group are used to describe When the conditional expression of is satisfied, it is determined that the same sample point is included in the sample point group and included in the sample point group.

If | GpX-AverageX | <DistanceX and | GpY-AverageY | GpCy1 + Cy1 GpCy2 = GpCy2 + Cy2 GpYe1 = GpYe1 + Ye1 GpYe2 = GpYe2 + Ye2 GpNumber = GpNumber + 1

If it cannot be included in all the sample point groups currently created, a new sample point group is created with the same sample point as a reference.

When the processing of steps S17 to S19 is completed for all the sample points, the true white group is searched for from the one or a plurality of sample point groups created in step S17. I don't really know which is white.

Therefore, next, first, in step S2 of FIG.
In 4, the sample point groups are sorted in descending order of brightness. Next, assuming that the one with high brightness is white, and using the brightness information GpBrightness of the plurality of sample point groups, samples from the one with the highest brightness to the Nth are considered to have a high probability of being white Weight point groups. Weighting is performed by the following formula (steps S25 and S26).

GpWeight = GpWeight * i where i is a weighting coefficient. For example, i = 2 N is 1 or more, and is calculated by the following equation. if (AllGpNumber * 10%)> 1 AllGpNumber: Total number of sample point groups created in step5 then N = AllGpNumber * 10% else N = 1

On the other hand, if it is determined in step S16 that the white points (X1, Y1) (X2, Y2) obtained in step S15 and represented in the coordinates are not within the predetermined range, step S in FIG.
27, the same coordinate point is moved onto the predetermined range frame as shown in FIG. 5, and from the coordinates (X ', Y') on the range frame, the color filter outputs Ma1 ', Ma2', Request new Ye1 'and Ye2'. Ma1 '= (1 + Y') * Gr1 Ma2 '= (1 + Y') * Gr2 Ye1 '= (1 + X') * Cy1 Ye2 '= (1 + X') * Cy2

However, the method of moving the coordinate point is to move to the coordinate on the predetermined range frame closest to the coordinate point as shown in FIG. This coordinate movement processing is called color clip processing. The total number of sample points that have been subjected to this color clip processing is counted in the next step S28 to be SN2.
Next, in step S29, the integrated values obtained by the color clip are calculated as Ma1 ', Ma2', Gr1 ', Gr2', Cy1 ', Cy2',
Ye1 'and Ye2'.

After the weighting is finished, next, step S
30, the weighting is performed using the following formula for all the sample point groups obtained in step S17 to step S19 and step S25 to step S26, and the integrated value (SMa1, SMa2, SGr1, SGa1, SMa2, SGr1, SGr2, SCy1,
SCy2, SYe1, SYe2) is calculated.

SMa1 = SMa1 + GpMa1 * GpWeight SMa2 = SMa2 + GpMa2 * GpWeight SGr1 = SGr1 + GpGr1 * GpWeight SGr2 = SGr2 + GpGr2 * GpWeight SCy1 = SCy1 + GpCy1 * GpWeySCy2 * GyG + Cy1 * GpWeight SCy2 * GpWeight SYe2 = SYe2 + GpYe2 * GpWeight

Next, in step S31, the number SN1 of sample points simultaneously regarded as white is obtained. SN1 = SN1 + GpNumber

Next, in step S32, it is judged whether or not the processing is executed for all the sample point groups,
If not executed, the process returns to step S30. In addition, when it is executed for all, the process proceeds to step S33.

In step S33, the number of samples (S
N1 + SN2) is 0, or it is determined whether it is obtained above a predetermined value. If it is not obtained above a predetermined value, the process proceeds to step S34, and the white balance of a predetermined color temperature (for example, 5000K) is used as white balance adjustment data. Use the adjusted value.

If the number of samples (SN1 + SN2) is not less than the predetermined value as a result of the determination in step S33, the process proceeds to step S35, where SY = (SMa1 +) as the average luminance level during white photographing. Calculate SMa2 + SGr1 + SGr2 + SCy1 + SCy2 + SYe1 + SYe2) / 8 and calculate the ratio of the integral value of each color to this average brightness level. gmg1 = SY / (2 * SMa1) gmg2 = SY / (2 * SMa2) gg1 = SY / (2 * SGr1) gg2 = SY / (2 * SGr2) gcy1 = SY / (2 * SCy1) gcy2 = SY / ( 2 * SCy2) gye1 = SY / (2 * SYe1) gye2 = SY / (2 * SYe2)

Next, proceeding to step S36, SY '= (SMa1' + SMa2 '+ SGr1' + SGr2 '+ SCy1' + SCy2 '+ SYe1' + SYe as the average luminance level at the time of clipping.
2 '/ 8 is calculated and the ratio of the integrated value of each color to this average luminance level is calculated. gmg1 '= SY' / (2 * SMa1 ') gmg2' = SY '/ (2 * SMa2') gg1 '= SY' / (2 * SGr1 ') gg2' = SY '/ (2 * SGr2') gcy1 ' = SY '/ (2 * SCy1') gcy2 '= ST' / (2 * SCy2 ') gye1' = SY '/ (2 * SYe1') gye2 '= SY' / (2 * SYe2 ')

Then, the process proceeds to step S37, and step S
35 and white balance adjustment data (wmg1, wg1, wcy1, wye1) based on the ratio of the integrated value of each color obtained in step 35 and step S36, the number SN1 of integrated samples in the white discrimination range, and the number SN2 of integrated samples by the color clip. , wmg2, wg2, wcy2, wy
Make e2).

For example, Such a process is performed.

These values are stored as white balance adjustment data. White balance can be obtained by multiplying the signal value for each pixel obtained from the image sensor by the white balance adjustment data obtained by the above processing.

As described above, in the present embodiment, the coordinates for white balance are created by the signal values from the image pickup device, and the white ones are relatively bright with respect to the sample points in the white area at the coordinates. Weighting is performed under the assumption that the value is high.

After that, since the integration is performed at the sample point, it is possible to prevent the white balance from being greatly deteriorated by being influenced by other colors even if the image has a small amount of white. .

(Second Embodiment) Next, a second embodiment of the present invention will be described. In the above-described first embodiment, the signal reading of the electronic camera is frame reading, but in the second embodiment, the signal of the electronic camera is field reading.

Since the image pickup device in the present embodiment performs field reading, as shown in FIG. 9, the output from the image pickup device is read by adding two pixels in the vertical direction. In this FIG. 9, Wb = Ma + Cy Gr = G + Ye Gb = Cy + G Wr = Ye + Ma.

In the case of this embodiment, the color filters are (Ma, g, C
Considering that it changed from (y, Ye) to (Wb, Gr, Gb, Wr),
Since almost the same processing as that of the frame reading described above is performed and the hardware configuration is the same as that of FIG. 3 in the above-described first embodiment, illustration is omitted.

In this embodiment, first, the black level is adjusted using the integrated value of each filter output in the optical black area. Next, white balance processing is performed so that the output gains of Wb, Gr, Gb, and Wr at the white points are the same.

To produce the luminance signal (Y), aperture correction is performed after a horizontal low-pass filter is applied to remove moire due to the color filters. Here, horizontal,
A high-pass filter is applied vertically, and the gain is adjusted.
Peak To Peak processing is performed so that U becomes large, and that having a large amplitude becomes small, horizontal and vertical are added, and gain is added to the luminance component (Yn). .

In order to generate the color difference signals (RY, BY), first, the signals of four colors are synchronized by the synchronizing circuit, and pixel interpolation is performed so that four color signals are assigned to the image pickup area. Then 4x
Wb signal, Gr signal, Gb signal, using the matrix circuit of 3
The Wr signal is converted into an RGB signal. At this time, as the matrix circuit, a matrix circuit is used that determines the color temperature of the subject from the white balance information and maximizes color reproduction at that color temperature.

Next, the RGB signals are converted into RY and BY signals, a low-pass filter for reducing moire is applied, and saturation achromatic processing is performed with reference to the Y signals. The luminance signal (Y) and the color difference signals (RY, BY) created in this way are RG
It is converted into a B signal, and after being subjected to gamma correction processing, it is recorded.

Next, the white balance processing, which is the main point of this embodiment, will be described in detail. When the CCD complementary color filters are arranged as shown in FIG.
The pixel array is used as a unit block, and the entire array is formed by repeating this unit block. Here, since field reading is performed, in one field, as shown in FIG.
The signal as shown in (b) is read.

Next, the processing procedure of the second embodiment will be described with reference to FIGS. First, the first step S
At 61, sample data is extracted for each sample.

Next, in step S62, it is determined whether the sample data extracted in step S61 is valid as a WB sample point. As a result of the determination, if the extracted sample data is valid as the WB sample point, the process proceeds to step S63, and if it is not valid, the process starts from step S61.

In step S63, it is checked whether or not the sample point has a brightness that can be calculated as WB data, and all four pixels (Wb, Gr, Gb, Wr) of the sample point are checked.
However, if the following equation is satisfied, the process proceeds to the next step S64.
If even one is not satisfied, move to the next sample point and start over from step 1. DarkThreshold <PixelValue
<BrightThreshold

Next, in step S64, the value of the following equation is obtained at the sample point. X1 = ((Wr-Gb)-(Wb-Gr)) / (Wr + Gb)… (6) Y1 = ((Wr-Gb) + (Wb-Gr)) / (Wb + Gr)… (7)

Next, in step S65, it is determined whether or not the data is valid as WB balance setting data. That is, as shown in FIG. 5, ((Wr-Gb)-(Wb-Gr)) / (Wr + Gb)
Is the X axis, and ((Wr-Gb) + (Wb-Gr)) / (Wb + Gr) is the Y axis, and the above formulas (6) and (7) are given regions on the X and Y axes. If the following conditional expression, a <X1 <b, c <Y1 <d (8), is satisfied, it is considered that the sample point is valid as data for setting WB, and the process proceeds to step S66. .
If the above expression (8) is not satisfied, step S76.
Proceed to.

In step S66, the average value of the XY axis values and the brightness value of the sample point at the WB coordinate are calculated at the same sample point using the following formula. AverageX = X1 AverageY = Y1 AverageBrightness = (Wb + Gr + Gb + Wr) / 2

Next, in step S67, it is determined whether the sample point is the first WB setting valid data. As a result of the above judgment, the same sample point is W
If it is the first sample point regarded as B setting valid data, the process proceeds to step S68.

In step S68, a sample point group having a value as shown in FIG. 6 as a parameter is created, and the value obtained in step S66 and the pixel value of the same point are substituted. GpX = AverageX GpY = AverageY GpBrightness = AverageBrightness GpWb = Wb GpGr = Gr GpGb = Gb GpWr = Wr GpNumber = 1 GpWeight = 1

If the sample point is not the first sample point regarded as WB setting valid data, the process proceeds to step S69, and it is determined whether it is included in the sample point group.

The above determination uses the values of GpX, GpY, and GpBrightness of the existing sample point group and determines that the sample point group is included in the sample point group when the following conditional expression is satisfied. In this case, the process proceeds to step S70 to be included in the sample point group.

That is, if | GpX-AverageX | <DistanceX and | GpY-AverageY | GpNumber = GpNumber + 1

If it cannot be included in all the sample point groups, the process proceeds to step S71, and it is determined whether or not the sample point group ends. If the result of the above judgment is not the end, step S
Returning to 69, and in the case of the end, the process proceeds to step S68, and as described above, a new sample point group with the same sample point as a reference is created.

Step S68 or step S70
When the processing of (1) is completed, the process proceeds to step S72, and it is determined whether all sample data extraction is completed. Then, as a result of the above determination, when it is not completed, the process returns to step S61 and the above-described processing is repeated.

When the extraction of all sample data is completed, the processes of steps S73 to S75 of FIG. 8 are performed.
This is the step S for all sample points.
When the processing from 67 to step S71 is completed, step 6
A process of searching for a true white group is performed from one or a plurality of sample point groups created in 7 to step S71.

However, the true white group does not really know which is white. Therefore, assuming that the one with high brightness is white, and using the brightness information GpBrightness of the plurality of sample point groups, from the one with the highest brightness to the Nth one is a true white group,
The sample points are weighted. That is, the processes of steps S73 to S75 are sequentially performed.

The weighting process is performed by the following formula. GpWeight = GpWeight * 2 However, N is 1 or more, and is calculated by the following equation, for example.

[0112]

On the other hand, if it is determined in step S65 that the white points (X1, Y1) and (X2, Y2) expressed in the obtained coordinates in step S64 are not within the predetermined range, the same coordinate points are set in the predetermined range frame. Moving up, Wb 'and Wr' are newly calculated (step S76).

Next, in step S77, after the total number of the color clipped sample points is set to SN2,
In step S78, a process of adding Wb ', Gr', Gb ', Wr' to the integrated value for each color is performed.

After the weighting is completed, next, step S
Proceeding to 79, the integrated values (SWb, SGr, SGb, SWr) within the white discrimination range (SWb, SGr, SGb, SWr) are used for all the sample point groups obtained in steps S67 to S71 and steps S74 to S75 using the following formula. ). SWb = SWb + GpWb * GpWeight SGr = SGr + GpGr * GpWeight SGb = SGb + GpGb * GpWeight SWr = SWr + GpWr * GpWeight

Next, in step S80, the number SN1 of sample points simultaneously regarded as white is obtained. SN1 = SN1 + GpNumber

Next, in step S81, it is determined whether or not the processing has been executed for all the sample point groups. If not executed, the processing returns to step S79 and the above-described operation is repeated.

If all the sample point groups have been executed, the process proceeds to step S82,
It is determined whether or not the number of samples (SN1 + SN2) is 0 or more than a predetermined value.

As a result of the judgment in step S82, the number of samples
If (SN1 + SN2) is not more than the predetermined value, step S
In step 83, the white balance adjustment value of a predetermined color temperature (for example, 5000K) is used as the white balance adjustment data.

On the other hand, if the result of determination in step S82 is that the number of samples (SN1 + SN2) is greater than or equal to a predetermined value, the process proceeds to step S84, where SY = (SWb + SGr +) as the average brightness level during white photography. SGb + SWr) / 4 is calculated, and the ratio of the integrated value of each color to this average luminance level is calculated. gwb = SY / (2 * SWb) ggr = SY / (2 * SGr) ggb = SY / (2 * SGb) gwr = SY / (2 * SWr)

Next, in step S85, SY '= (SWb' + SGr '+ SGb' + SWr ') / 4 is calculated as the average luminance level at the time of clipping, and the integrated value of each color with respect to this average luminance level. Find the ratio of. gwb '= SY' / (2 * SWb ') ggr' = SY '/ (2 * SGr') ggb '= SY' / (2 * SGb ') gwr' = SY '/ (2 * SWr')

Then, the process proceeds to step S86, and step S
The white balance adjustment data (wwb, wgr, wgb, wwr) is obtained from the ratio of the integrated value of each color obtained in step 84 and step S85, the number SN1 of integrated samples in the white discrimination range and the number SN2 of integrated samples by the color clip. create.

For example, Such a process is performed. Then, these values are stored as white balance adjustment data.

White balance can be obtained by multiplying the signal value of each pixel obtained from the image pickup device by the white balance adjustment data obtained by the above processing.

(Third Embodiment) FIG. 10 is a part of a flowchart for explaining a part of the processing procedure of white balance according to the third embodiment of the present invention. Steps S25 to S25 in FIG. This is the portion corresponding to S26. This part is a part for weighting the group after creating the white balance group, but the present embodiment is characterized in that the illuminance information at the time of shooting is used for this weighting.

The illuminance information is, for example, an EV value,
The processing of steps S91 to S95 is performed. That is, first, the EV value is received from the camera side as information, and E
If the V value is large, it is often taken outside (sunlight), so the white coordinate range is limited as shown in FIG.

On the contrary, the EV value is limited to the sunset or artificial light, and the white coordinate range is limited as shown in FIG. 11 (b). Here, weighting is performed as follows for all sample point groups. That is, the weight of the white balance group in the limited coordinate range is increased.

[0128]

By performing such weighting, more preferable white balance can be performed even when the brightness of white is low. Even if the limited coordinate range falls within a sample point that is not originally white and a large number of sample point groups are formed, the white group is selected by weighting, so that the white balance is not greatly deteriorated.

Further, the weighting in the present embodiment is performed by the steps S74 to S75 in the second embodiment.
May be used for. Moreover, you may use it combining with the weighting method of 1st Embodiment and 2nd Embodiment.

[0131]

As described above, according to the first aspect of the present invention, color temperature information can be obtained by performing a simple process on the output of the image pickup device when the image pickup device is frame read driven. Obtainable. Further, since the processing for increasing the weight of the sample points with high brightness is performed under the assumption that white has a relatively high brightness value, accurate white balance processing is performed even when there is relatively little white in the image. It becomes possible. Even if there is no white portion in the image, the white balance adjustment value can be obtained by simple processing.

Further, according to the second invention of the present invention, when the image pickup device is field read driven, the same effect as that of the first invention can be obtained and a good white balance adjustment value can be obtained. .

According to the third aspect of the present invention, it is assumed that the sample point entered in the white coordinates is indoors if it is dark from the exposure condition of the image sensor, and the coordinates near the indoor color temperature are set. By increasing the weight of the entered sample points, assuming that it is outdoors if it is bright, and increasing the weight of the sample points entering the coordinates near the outside,
A more accurate white balance adjustment value can be obtained.

[Brief description of drawings]

FIG. 1 is a flowchart showing a process of a first embodiment of the present invention.

FIG. 2 is a flowchart showing a process of the first embodiment.

FIG. 3 is a block diagram for explaining a state of signal processing of an electronic camera to which the present invention has been applied.

FIG. 4 is a diagram showing an example of a color filter pattern of an image sensor.

FIG. 5 is a diagram showing a concept of white balance.

FIG. 6 is a diagram showing each parameter of a sample point group.

FIG. 7 is a flowchart showing a process of the second embodiment.

FIG. 8 is a flowchart showing a process of the second embodiment.

FIG. 9 is a diagram showing how field reading is performed by the image sensor.

FIG. 10 is a flowchart showing a process of the third embodiment.

FIG. 11 is a diagram showing a weighting area of white balance coordinates in the third embodiment.

FIG. 12 is a block diagram showing an example of a conventional electronic camera.

13 is a block diagram showing a configuration of a signal processing circuit of the electronic camera of FIG.

FIG. 14 is a block diagram showing a configuration of a white balance control circuit.

FIG. 15 is a diagram showing a white determination range.

FIG. 16 is a diagram showing a relationship between a color difference signal and a luminance signal.

[Explanation of symbols]

 31 integrator 32 black level adjusting circuit 33 white balance detecting circuit 34 gain adjusting circuit 35 low pass filter 36a, 36b high pass filter 37a, 37b gain adjusting circuit 38a, 38b amplitude adjusting circuit 39a, 39b base clip circuit 40 adding circuit 41 gain adjusting circuit 42 adder circuit 50 synchronization circuit 51 pixel interpolation circuit 52 first matrix circuit 53 second matrix circuit 54 low-pass filter 55 color processing circuit 56 color decoder 57 gamma processing circuit

Claims (12)

[Claims] 1. A signal generating means for generating a color signal indicating a color, a saturation signal indicating a saturation, and an illuminance signal indicating an illuminance on the basis of an output of an image sensor having a plurality of color filters having different spectral characteristics, and A color / illuminance ratio signal indicating a ratio between a color signal and the illuminance signal, a signal ratio generating means for generating a saturation / illuminance ratio signal indicating a ratio between the saturation signal and the illuminance signal, and an illuminance indicating the illuminance. An image pickup apparatus, comprising: a signal, the color / illuminance ratio signal, and the color temperature information acquisition means for obtaining color temperature information of a captured image from the saturation / illuminance ratio signal. 2. An image pickup device having three or more types of color filters having different spectral characteristics, and a signal output from each of the image pickup devices corresponding to a predetermined two-color filter among the three or more types of color filters. A predetermined color filter output difference signal generating means for generating a predetermined color filter output difference signal representing a difference and an image sensor corresponding to a predetermined external color filter which is a combination of color filters other than the predetermined two-color filter are output. A predetermined external color filter output difference signal generating means for generating a predetermined external color filter output difference signal representing a signal difference; a luminance signal value generating means for generating a luminance signal value; the predetermined color filter output difference signal; Of the two predetermined color filters that created the color filter output difference signal,
First ratio signal generating means for generating a first ratio signal representing a ratio with a signal output from the image sensor corresponding to one of the color filters; the predetermined external color filter output difference signal; Second ratio signal generating means for generating a second ratio signal indicating a ratio of one of the two color filters that has created the external color filter output difference signal to the signal output from the image sensor And with respect to the first ratio signal and the second ratio signal,
White change range setting means for setting a white change range when the color temperature changes, and whether or not the first ratio signal and the second ratio signal obtained from a plurality of pixels of the image pickup device fall within the white change range. A white change range determining means for determining, and an image sensor output corresponding to each color filter of a plurality of pixels determined to be within the white range when it is determined to be within the white range by the white change range determining means, A gain that obtains a gain control value for the output of the image sensor corresponding to three or more different types of color filters based on the integrating means for weighting and integrating each using a luminance signal and the integrated value output from the integrating means. An image pickup apparatus comprising: a control value acquisition unit. 3. When the white color change range determination means determines that the color signal is within the white range, the image sensor output corresponding to each color filter of a plurality of pixels determined to be within the white range is output as a luminance signal. A first gain control value for the output of the image pickup device having three or more different color filters based on the integration value output from the integration means When the gain control value acquisition unit and the white change range determination unit determine that the pixel is not within the white range, the image sensor output signals of a plurality of pixels that are determined not to be within the white range and the boundary portion of the white range Pseudo signal calculation means for calculating a plurality of pseudo signals corresponding to a plurality of image sensor outputs at the boundary of the white range from the first ratio signal and the second ratio signal; Second gain control value acquisition means for respectively integrating the similar signals to obtain a second gain control value for the output of the image sensor having three or more kinds of color filters different from the plurality of integrated values, and within the white range. A pixel number detecting means for detecting the number of pixels and a number of pixels that do not fall within the white range; a hybrid ratio calculating means for calculating a hybrid ratio according to the pixel number detected by the pixel number detecting means; and a hybrid ratio calculating means The first gain control value and the second gain control value are mixed according to the calculated mixing ratio, and the third gain control value for the output of the image pickup device having the three or more different color filters is obtained. The image pickup apparatus according to claim 2, further comprising: a third gain control value acquisition unit that obtains the gain control value. 4. An image pickup device having four types of color filters having different spectral characteristics, wherein the first to fourth image pickup device output signals respectively output from the image pickup devices corresponding to the four types of color filters, First / second difference signal generation means for generating a first / second difference signal representing a difference between the first image sensor output signal and the second image sensor output signal, and the third image sensor output Third generation of a third / fourth difference signal representing a difference between the signal and the output signal of the fourth image sensor
/ Fourth difference signal generation means, weighting signal generation means for generating a weighting signal using the first to fourth image sensor output signals, the first / second difference signal and the first The 1/1 / which represents the ratio of the image sensor output signal or the second image sensor output signal
With respect to the ratio signal of the second difference and the ratio signal of the third / fourth difference representing the ratio between the third / fourth difference signal and the third image sensor output signal or the fourth image sensor output signal. A white change range setting means for setting a white change range when the color temperature changes, and a plurality of pixels determined to be in the white range when the white change range determining means determines that the white change range is set. Based on the four integrated values output from the weighting integration means, and the first integration based on the weighting integration means for weighting and integrating the output of the image sensor corresponding to each color filter using the weighting signal. An image pickup apparatus comprising: a first gain control value generation unit that generates a first gain control value for a fourth image sensor output signal. 5. The first or second image sensor output signal of a plurality of pixels, which is determined by the white change range determiner to be out of the white range, the third or fourth image sensor output signal, and the white color. Based on the ratio signal of the first / second difference and the ratio signal of the third / fourth difference in the boundary part of the range, the second / second corresponding to the output signal of the second or first image sensor in the boundary part of the white range. A pseudo signal calculating means for calculating one pseudo signal and a fourth / third pseudo signal corresponding to the fourth or third image sensor output signal; and the first or second image sensor output signal. , The third above
Alternatively, from the fourth image sensor output signal, the second / first pseudo signal, and the fourth / third pseudo signal, the integrating means, and the four integrated values output from the integrating means. Second gain control value generation means for generating a second gain control value for the first to fourth image sensor output signals, and a pixel for detecting the number of pixels that fall within the white range and the number of pixels that do not fall within the white range. Number detecting means, a hybrid ratio calculating means for calculating a hybrid ratio according to the number of pixels detected by the pixel number detecting means, and the first gain control value according to the hybrid ratio calculated by the hybrid ratio calculating means. And a second gain control value, and third gain control value generating means for generating a third gain control value for the first to fourth image sensor output signals. The imaging device according to claim 4. . 6. The weighting means has a luminance signal value higher than a luminance value calculated in advance when integrating the output signals of the first to fourth image pickup elements determined to fall within the white range. Multiply each of the above four output signals by N,
The image pickup apparatus according to claim 2, wherein the influence of the four output signal values on the integrated value is made larger than that of the one having a low luminance signal value. 7. A signal generating means for generating a color signal representing a color, a saturation signal representing a saturation, and an illuminance signal representing an illuminance on the basis of an output of an image pickup device having a plurality of color filters having different spectral characteristics, and A color / luminance ratio signal indicating a ratio between a color signal and a luminance signal, and a signal ratio generating means for generating a saturation / luminance ratio signal indicating a ratio between the saturation signal and the luminance signal, and an illuminance signal indicating the illuminance. An image pickup apparatus comprising: a color temperature information acquisition unit that obtains color temperature information of a captured image from the color / luminance ratio signal and the saturation / luminance ratio signal. 8. An image pickup device having three or more types of color filters having different spectral characteristics, and a signal output from each of the image pickup devices corresponding to a predetermined two-color filter among the three or more types of color filters. A predetermined color filter output difference signal generating means for generating a predetermined color filter output difference signal representing a difference and an image sensor corresponding to a predetermined external color filter which is a combination of color filters other than the predetermined two-color filter are output. A predetermined external color filter output difference signal generating means for generating a predetermined external color filter output difference signal representing a signal difference; an exposure amount signal generating means for generating an exposure amount signal representing the exposure amount of the image sensor; Of the filter output difference signal and the predetermined two-color filter that created the predetermined color filter output difference signal,
First ratio signal generating means for generating a first ratio signal representing a ratio with a signal output from the image sensor corresponding to one of the color filters; the predetermined external color filter output difference signal; A second ratio signal for generating a second ratio signal representing a ratio with a signal output from the image sensor corresponding to one of the two color filters that has created the external color filter output difference signal Generating means, and for the first ratio signal and the second ratio signal,
White change range setting means for setting a white change range when the color temperature changes, and white change range determination means for determining whether or not the first ratio signal and the second ratio signal fall within the white change range. When the white change range determining means determines that the image is within the white range, the image sensor outputs corresponding to the color filters of the plurality of pixels that are determined to be within the white range are weighted using the exposure amount signal. And an gain control value acquisition unit that obtains a gain control value for the output of the image sensor corresponding to three or more different types of color filters based on the integrated value output from the above-mentioned integrating unit. An image pickup apparatus comprising: 9. When the white change range determining means determines that the color change is within the white range, the output of the image pickup device corresponding to each color filter of a plurality of pixels determined to be within the white range is set as the exposure amount. A first gain control value for the output of the image sensor having three or more different color filters based on the integration value output from the integration means 1 gain control value acquisition means, the image sensor output signals of a plurality of pixels determined by the white change range determination means not to be within the white range, the first ratio signal and the second ratio at the boundary part of the white range. Pseudo signal calculating means for calculating pseudo signals corresponding to outputs of the plurality of image pickup elements at the boundary of the change range from the signal, and the plurality of pseudo signals respectively, and the plurality of integrated values. Second gain control value acquisition means for obtaining a second gain control value for the output of the image sensor having three or more different color filters, and detecting the number of pixels that fall within the white range and the number of pixels that do not fall within the white range. A pixel number detecting means, a hybrid ratio calculating means for calculating a hybrid ratio according to the number of pixels detected by the pixel number detecting means, and the first ratio according to the hybrid ratio calculated by the hybrid ratio calculating means. Third gain control value acquisition means for mixing the gain control value and the second gain control value to obtain a third gain control value for the output of the image sensor having three or more different types of color filters is further provided. The imaging apparatus according to claim 8, further comprising: 10. An image pickup device having color filters of four colors having different spectral characteristics, wherein the first to fourth image pickup device output signals output from the image pickup device corresponding to the color filters of the four colors First / second difference signal generating means for generating a first / second difference signal representing a difference between the first image sensor output signal and the second image sensor output signal, and the third image sensor output signal A third / fourth difference signal representing a difference between the output signal of the fourth image sensor and the output signal of the fourth image sensor
/ A fourth difference signal generating means, an exposure amount signal generating means for generating an exposure amount signal representing the exposure amount of the image pickup device, the first / second difference signal and the first image pickup device output signal or the second Of the image sensor output signal of 1 /
With respect to the ratio signal of the second difference and the ratio signal of the third / fourth difference representing the ratio between the third / fourth difference signal and the third image sensor output signal or the fourth image sensor output signal. Whether the white change range setting means for setting the white change range when the color temperature changes and the ratio signal of the first / second difference and the ratio signal of the third / fourth difference fall within the white change range. White change range determining means for determining whether or not the plurality of pixels are determined to be within the white range when the white change range determining means determines that the white change range is within the white range. Based on the four integration values output from the weighted integration means, the output signals are weighted and integrated using the exposure amount signal, and the output signals for the first to fourth imaging element output signals are obtained. The gain to generate the gain control value Imaging apparatus characterized by and a control value generating unit. 11. The first or second pixel of a plurality of pixels determined by the white change range determination means to be not within the white range.
Of the image sensor output signal, the third or fourth image sensor output signal, and the ratio signal of the first / second difference and the ratio signal of the third / fourth difference at the boundary part of the white range,
The second / first pseudo signal corresponding to the second or first image sensor output signal at the boundary of the change range is calculated, and the fourth / third pseudo signal corresponding to the fourth or third image sensor output signal is calculated. Pseudo signal calculation means for calculating a signal, the first or second image sensor output signal, and the third signal
Alternatively, from the fourth image sensor output signal, the second / first pseudo signal, and the fourth / third pseudo signal, the integrating means, and the four integrated values output from the integrating means. Second gain control value generation means for generating a second gain control value for the first to fourth image sensor output signals, and a pixel for detecting the number of pixels that fall within the white range and the number of pixels that do not fall within the white range. Number detecting means, a hybrid ratio calculating means for calculating a hybrid ratio according to the number of pixels detected by the pixel number detecting means, and the first gain control value according to the hybrid ratio calculated by the hybrid ratio calculating means. And a second gain control value, and third gain control value generating means for generating a third gain control value for the first to fourth image sensor output signals. The imaging device according to claim 10. Place. 12. The weighting means uses the exposure amount signal to determine the color temperature of the subject when integrating the output signals of the first to fourth image pickup devices that are determined to fall within the white range. If it is determined in advance that the white change area is divided, and the ratio signal of the first / second difference and the ratio signal of the third / fourth difference are determined to fall within the white change area predicted in advance. 12. The output signals of the four image pickup devices are each multiplied by N to increase the influence of the four output signal values on the integrated value.
An imaging device according to item 13.