JP2003324754A - Color signal processing method, color signal processing circuit, imaging apparatus, program, and storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a color signal processing circuit for preventing feedback type auto white balance control from excessively following a change in an object or a change in a color of a light source. <P>SOLUTION: An area setting section 178 sets a restart area to restart white balance control in a way of including a stop area for deciding a converging condition of the white balance control for including a complete dead zone surrounding the origin on a color difference plane. A gain control section 180 repetitively adjusts a gain signal Sr2 or the like given to a white balance amplifier 152R so as to converge an evaluation value to the origin by restarting the white balance control when the evaluation value exceeds the restart area without restarting the auto white balance control until the evaluation value exceeds the restart area after the white balance is stabilized. However, when the evaluation value reaches the inside of the stop area, the auto white balance control is immediately stopped even when the evaluation value does not reach the origin. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a color signal processing method, a color signal processing circuit for carrying out this processing method, and an image pickup apparatus equipped with the color signal processing circuit. The present invention also relates to a program for implementing the color signal processing method by a computer such as a microprocessor, and a computer-readable recording medium storing the program. In particular, the present invention relates to a color signal processing that automatically obtains a white balance of a color signal obtained as a result of imaging using an image pickup device such as a CCD (Charge Coupled Device) image pickup device or a CMOS type image pickup device, which is a so-called auto white balance function. .

[0002]

2. Description of the Related Art When a solid-state image pickup device such as a CCD image pickup device is used to pick up a subject image, the white color contained in the subject becomes red when taken in an environment where the color temperature is low, such as indoors. When the image is picked up in an environment with a high color temperature, the phenomenon that it becomes bluish occurs. This
It is said that the white balance will be lost. The color temperature is the temperature (K) of a black body having the same chromaticity as the test light source.
Say.

FIG. 7 shows a black body radiation curve (black body locus) on a color difference coordinate plane (specifically, a vector scope) indicated by color difference signal axes R-Y and B-Y, and automatic white balance processing. It is a figure explaining the relationship of. In an image pickup apparatus using a solid-state image pickup device as an image pickup device, in the image pickup result of a white image of a subject, since the white color is projected as an achromatic white color, when the color temperature of the light source changes, as shown in FIG. In addition, according to the change in color temperature, the white color moves along the black body radiation curve, and the white color that appears to be colored is adjusted to the achromatic white color by the automatic white balance processing. In FIG. 7, the horizontal axis represents the color difference signal B-
The vertical axis represents the amplitude (gain) of Y and the vertical axis represents the amplitude of the color difference signal RY, and the origin of the color difference coordinate plane is achromatic white.

There are various methods for this automatic white balance processing. For example, in a color signal processing system, the integrated value of the color difference signal for each field is obtained, and based on this integrated value, the subject is achromatic. The white balance is finally determined by determining how much color is present, and operating the gain of the current white balance amplifier by a certain amount so that the subject approaches white, and repeating this procedure every field from the result. There is a feedback control system.

In this feedback control system, if the integrated value of the color difference signals for each field is apart from the achromatic state and the state is maintained for a certain period of time, the white balance of the entire screen is maintained. Is determined to have collapsed, it is determined that it is necessary to start the gain operation of the white balance amplifier, and the gain operation control is restarted (started).

Further, when a general subject is imaged, various color components are randomly present on the entire screen. Therefore, if all the color components of the entire screen are integrated, color signals (for example, R, G, B) Considering that each element will be extracted approximately equally, or the integrated value of the color difference component will be zero, the gain operation of the white balance amplifier is performed to bring it closer to the control target, that is, an achromatic color. .

[0007]

However, for example, when the ratio of the object having a color close to the black body radiation curve to the image pickup screen becomes high, the object actually changes, and the white balance changes. It is possible that it is determined that the light source around the subject, that is, the color temperature has changed, even though it is not.

In this case, even though the color is the original color of the subject, it is determined that the color is influenced by the color temperature of the light source, and the color of the white balance amplifier is moved toward the achromatic color. The gain may be operated immediately, and the user may feel discomfort in the output image data.

Further, when a part of the image pickup screen is changed, for example, when a new subject enters a part of the image pickup screen once the white balance is obtained, the data used for the white balance control is the entire screen. Since it is the integral value of, even if the subject has a dark color, it will be averaged into the overall integral value. Then, it is determined that the entire screen is light-colored and the white balance is lost, and the white balance control is immediately executed. In this case as well, the user may feel discomfort in the output image data.

On the other hand, when the light source color changes greatly, for example, when reddish light enters the entire screen, the automatic white balance function immediately provides white balance in the direction in which the color becomes completely achromatic. When the gain of the amplifier is operated, the user may feel a sense of discomfort. This is considered to be due to human adaptation to color (memory for color).

As described above, the conventional auto white balance function still has a problem that the phenomenon of excessively following the change of the subject and the change of the light source color occurs, which gives a feeling of strangeness to the user.

The present invention has been made in view of the above circumstances, and in color signal processing for an image pickup signal obtained by using an image pickup device, an automatic white balance is performed without excessively following a subject change or a light source color change. It is an object to provide a method capable of realizing a function.

Another object of the present invention is to provide a color signal processing circuit to which such a color signal processing method is applied, or an image pickup apparatus equipped with this color signal processing circuit.

A further object of the present invention is to provide a program for implementing the color signal processing method by a computer such as a microprocessor, and a computer-readable recording medium storing this program.

[0015]

That is, a first color signal processing method according to the present invention relates to at least one color signal of a plurality of color signals acquired based on an image pickup signal from an image pickup device. A color signal processing method for achieving white balance by performing gain adjustment. First, in a color difference space indicated by a color difference axis corresponding to a plurality of color difference signals generated based on gain adjusted color signals. A restart area indicating a reference value for restarting the gain adjustment for white balance is set so as to surround (include) a substantially white reference point.

Then, the coordinate values in the color difference space are calculated based on the plurality of integrated signals integrated over a predetermined period for each of the plurality of color difference signals, and the calculated coordinate values and the set restart area are set. The gain value for adjusting the gain is controlled based on the positional relationship of.

In the first color signal processing method, the coordinate value to be obtained next is moved in the direction of the white reference point on the condition that the coordinate value in the color difference space exists outside the set restart area. Moreover, it is desirable to change the gain value for performing gain adjustment from the gain value at that time.

On the other hand, in the second color signal processing method according to the present invention, first, the reference for stopping the gain adjustment for the white balance so as to surround the substantially white reference point on the color difference space. Set the stop area that shows the value. Then, the coordinate value on the color difference space is obtained based on the plurality of integrated signals integrated over a predetermined period for each of the plurality of color difference signals, and based on the positional relationship between the obtained coordinate value and the set stop area. Control the gain value for adjusting the gain.

In the second color signal processing method, the gain value for performing the gain adjustment is maintained at the gain value at that point on the condition that the coordinate value in the color difference space exists inside the set stop area. Alternatively, the gain value for gain adjustment is adjusted so that the coordinate value to be calculated next moves in the direction of the white reference point at each predetermined timing that is slower than the normal gain adjustment cycle. It is desirable to change the value sequentially.

Further, in the color signal processing method according to the present invention, it is more preferable to combine the above-mentioned first and second color signal processing methods. In this case, the restart area is set so as to surround the stop area. Then, the gain value for performing the gain adjustment is controlled based on the obtained positional relationship between the coordinate value and the stop region and the restart region.

A first color signal processing circuit according to the present invention is a circuit for implementing the first color signal processing method, wherein a restart area is set so as to surround a substantially white reference point on a color difference space. Then, the coordinate value on the color difference space is obtained based on the plurality of integrated signals integrated over a predetermined period for each of the plurality of color difference signals, and the positional relationship between the obtained coordinate value and the restart area is set. Based on this, a white balance controller for controlling a gain value for adjusting the gain is provided.

A second color signal processing circuit according to the present invention is a circuit for implementing the second color signal processing method, wherein a stop area is set so as to surround a substantially white reference point on the color difference space. , A coordinate value in the color difference space is calculated based on the plurality of integrated signals integrated over a predetermined period for each of the plurality of color difference signals, and the gain is calculated based on the positional relationship between the calculated coordinate value and the stop area. It was equipped with a white balance controller that controls the gain value for adjustment.

In addition to the configuration of the first color signal processing circuit, a third color signal processing circuit according to the present invention has a gain adjustment amplifier section for performing gain adjustment of color signals, and a gain adjustment by the gain adjustment amplifier section. And an integration processing unit that acquires a plurality of integrated signals for each of the plurality of color difference signals over a predetermined period based on the color signals. Further, as a specific configuration of the white balance controller, a restart area setting unit that sets a restart area so as to surround a substantial white reference point on the color difference space, and a plurality of integrated signals generated by the integration processing unit. The coordinate value in the color difference space is calculated based on the above, and the gain value to be set in the gain adjustment amplifier unit is controlled based on the positional relationship between this coordinate value and the restart area set by the restart area setting unit. And a gain control section.

The fourth color signal processing circuit according to the present invention has, in addition to the configuration of the second color signal processing circuit, a gain adjustment amplifier section for performing gain adjustment of a color signal and a gain adjustment by the gain adjustment amplifier section. And an integration processing unit that acquires a plurality of integrated signals for each of the plurality of color difference signals over a predetermined period based on the color signals. In addition, as a specific configuration of the white balance controller, a stop region setting unit that sets a stop region so as to surround a substantially white reference point on the color difference space, and a plurality of integrated signals generated by the integration processing unit. A gain control unit that determines a coordinate value in the color difference space based on the coordinate value and controls the gain value to be set in the gain adjustment amplifier unit based on the positional relationship between the coordinate value and the stop region set by the stop region setting unit. Equipped with.

In the color signal processing circuit according to the present invention,
As in the color signal processing method according to the present invention, a configuration in which first and second color signal processing circuits are combined, or
It is more desirable to have a configuration in which the third and fourth color signal processing circuits are combined.

In the color signal processing method and the color signal processing circuit according to the present invention, the plurality of color signals are R (red) and G.
It is desirable to use (green) and B (blue) primary color signals.
For this reason, it is desirable that the color signal processing circuit includes a primary color separation unit that separates the R, G, and B primary color signals from the image pickup signal.

Further, in the color signal processing method and the color signal processing circuit according to the present invention, the color difference axis corresponding to the plurality of color difference signals corresponds to the R-G signal which is a component obtained by subtracting the G primary color signal from the R primary color signal. Color difference axis and B for the R primary color signal
It is desirable to use a color difference axis corresponding to the R + B-2G signal, which is a component obtained by subtracting twice the component of the G primary color signal from the component of the primary color signal.

In addition to the color signal processing circuit according to any one of the first to fourth aspects of the present invention,
The image pickup device outputs an image pickup signal according to an image pickup result.

The program according to the present invention is suitable for realizing the color signal processing circuit according to the present invention by software using an electronic computer (computer). The program may be provided by being stored in a computer-readable storage medium, or may be distributed via wired or wireless communication means.

[0030]

The first color signal processing method according to the present invention, the first
Alternatively, in the third color signal processing circuit or the image pickup apparatus including this color signal processing circuit, first, a white reference point (coordinate origin) is included in the color difference space represented by a predetermined color difference axis. , Set a restart area indicating a reference value for restarting the gain adjustment for white balance.

Next, a coordinate value as an evaluation value is obtained based on the integrated value of the color difference signals, and the gain value is controlled based on the positional relationship between this coordinate value and the restart area. For example, only when the coordinate value after the white balance is taken is outside the restart area, the white balance gain value is sequentially changed so that the coordinate value to be obtained next moves toward the white reference point. That is, white balance is achieved by feedback control.

On the other hand, in the second color signal processing method, the second or fourth color signal processing circuit according to the present invention, or the image pickup apparatus equipped with this color signal processing circuit, first, a predetermined color difference signal axis is used. In the represented color difference space, a stop area indicating a reference value for stopping the gain adjustment for taking the white balance is set so as to include the white reference point (coordinate origin). In addition, when combined with the first color signal processing method, the restart area is set so as to include the stop area.

Next, a coordinate value as an evaluation value is obtained based on the integrated value of the color difference signals, and the gain value is controlled based on the positional relationship between this coordinate value and the stop area. For example, in the process of taking the white balance by the feedback control, when the obtained coordinate value reaches the inside of the stop area, the white balance gain value is maintained at the gain value at that time, that is, the gain change is performed. Stop. Alternatively, the white balance gain value is sequentially changed so that the coordinate value to be calculated next moves in the direction of the white reference point at each predetermined timing that is slower than the repeated cycle of normal gain adjustment, that is, slower than usual. White balance is achieved by feedback control of the response speed.

[0034]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a block diagram showing a first embodiment of an image pickup apparatus according to the present invention. The image pickup apparatus 10 according to the first embodiment includes a lens 11, an image pickup element 12, and a preamplifier 1.
3, an A / D converter 14, and an optical detector 1
60 and a digital signal processing unit 15 having a white balance controller 170.

The digital signal processing unit 15 is an example of a color signal processing circuit according to the present invention, and the optical detector 160 is an example of an integration processing unit according to the present invention. The optical detector 160 and the white balance controller 170 may be provided outside the digital signal processing unit 15.

In the image pickup apparatus 10 having the above structure, the lens 11 forms an image of a subject (not shown) on the image pickup surface of the image pickup element 12. As the image sensor 12, a solid-state image sensor such as a CCD image sensor or a CMOS image sensor is used. The image pickup device 12 converts the image formed on the image pickup surface into an electric signal pixel by pixel and supplies the electric signal to the preamplifier 13 as an image pickup signal.

The preamplifier 13 samples and holds the image pickup signal output from the image pickup device 12 to take out necessary data, and also performs gain control in order to adjust to an appropriate level. The output signal of the preamplifier 13 is supplied to the A / D conversion unit 14. A / D converter 1
Reference numeral 4 converts the output signal of the preamplifier 13 from an analog signal to a digital signal and supplies it to the digital signal processing unit 15.

The digital signal processing section 15 includes a primary color separation section 1
51, white balance amplifiers 152R, 152G, 1
52B, gamma correction unit 153, color difference matrix circuit 15
4, encoder section 155, and D / A converter 15
Have six. The A / D converter 14 is replaced by the digital signal processor 1
It is also possible to adopt a configuration in which the D / A converter 156 is provided inside the digital signal processing unit 15.

In the digital signal processing section 15, the primary color separation section 151 converts the digital image signal supplied from the A / D conversion section 14 into R (red), G (green) and B (blue) primary color signals. The signal Sr1, the G signal Sg1, and the B signal Sb1 are separated, and these are separated by a white balance (WB) amplifier 152.
Supply to R, 152G, 152B. The white balance amplifier 152R for R (red) adjusts the gain of the R signal Sr1 supplied from the primary color separation unit 151 based on the R gain signal Sr2 supplied from the white balance controller 170, and performs gamma correction as the R signal Sr3. Part 15
Supply to 3.

Similarly, the white balance amplifier 152G for G (green) is the white balance controller 1
Based on the G gain signal Sg2 supplied from 70, the gain of the G signal Sg1 supplied from the primary color separation unit 151 is adjusted and supplied to the gamma correction unit 153 as a G signal Sg3. The white balance amplifier 152B for B (blue) is
B supplied from the white balance controller 170
Based on the gain signal Sb2, the gain of the B signal Sb1 supplied from the primary color separation unit 151 is adjusted and supplied to the gamma correction unit 153 as the B signal Sb3.

The gamma correction unit 153 uses the R signals Sr3, G
Gamma (γ) correction for faithful color reproduction is performed based on the signals Sg3 and B signals Sb3, and the gamma (γ) corrected output signals R, G, and B for each color are input to the color difference matrix circuit 154. . The color difference matrix circuit 154 performs color difference matrix processing to obtain color difference signals RY and BY.
Is input to the encoder unit 155.

The encoder section 155 digitally modulates the color difference signals RY and BY with a digital signal corresponding to the color signal subcarrier, and then synthesizes the color difference signals RY and BY with the brightness signal Y generated by a brightness signal generation section (not shown). Digital video signal VD (=
Y + S + C; S is a synchronizing signal, C is a chroma signal, and is input to the D / A converter 156. The D / A converter 156 converts the digital video signal VD into the analog video signal V.

In addition, the white balance amplifier 152R,
152G and 152B are R signal Sr3, G signal Sg3,
The B signal Sb3 is also supplied to the optical detector 160.

2A and 2B are views for explaining an example of the optical detector 160 and the white balance controller 170. FIG. 2A is a functional block diagram showing a first configuration example, and FIG. 2B is a first block diagram. FIG. 2C is a functional block diagram illustrating the operation of the second configuration example, and FIG. 2C is a functional block diagram illustrating the second configuration example.

As shown in FIG. 2A, the optical detector 160 of the first configuration example has a subtractor 161a for subtracting the color signal Sg3 from the color signal Sr3 and outputting a color difference signal (RG) a, and a subtractor 161a. And a subtractor 161b that subtracts the color signal Sg3 from the signal Sb3 and outputs a color difference signal (BG) a.

The optical detector 160 includes an integrating circuit 163a for integrating the color difference signal (RG) a, which is the output signal of the subtractor 161a, for each field, and the color difference signal (BG, which is the output signal of the subtractor 161b. ) A has an integrating circuit 163 b for integrating each field. The color difference signals (R−G) a and (B−G) a are other examples of optical detector values.

The integrating circuits 163a and 163b are shown in FIG.
As shown in (B), there is a different integration range divided into a high-luminance portion and a normal-luminance portion according to the integration slice level based on the luminance level, and in the high-luminance portion, the color difference data ( RG) a, (B-
G) a only is integrated, and in the normal luminance part, the color difference data (RG) a, (B-
G) Integrate only a.

However, when the brightness is extremely high, it is determined that the brightness is saturated, and the data above the high brightness limiter level is not integrated. In addition, data whose brightness is too low is regarded as noise, and data below the low brightness limiter level is not integrated. In addition, the optical detector 1
60 is a special condition (for example, a condition of full-color single color)
, Or if the integrated value is too low or too high,
Various limiters and special processing are applied to prevent the white balance processing from malfunctioning.

The integrating circuit 163a integrates the integrated value data (R-G) b and (R-G) c (b having high brightness in the high brightness part / normal brightness part in different integration ranges for each field). Side) to the next white balance controller 170
Supply to. Similarly, the integration circuit 163b integrates the integrated value data (B-G) b and (B-G) c obtained by integrating in different integration ranges of the high brightness part / normal brightness part for each field.
(B is on the high brightness portion side) is supplied to the white balance controller 170 at the next stage.

The white balance controller 170 of the first configuration example has integrated value data (R-G) b, (B-G) of the high-luminance portion supplied from the optical detector 160.
b and the integrated value data (R-G) c, (R-
G) It has a comparison circuit 171 which compares with c and outputs the integrated value data closer to “0; zero”.

The white balance controller 170 of the first configuration example has the RG selected by the comparison circuit 171.
Adder 173a that adds the integrated value data (R−G) d and the integrated value data (B−G) d of B−G to generate the data of R + B−2G, and the R selected by the comparison circuit 171.
A subtractor 173b for generating RB data by subtracting BG integrated value data (BG) d from -G integrated value data (RG) d.

Further, the white balance controller 17
0 is the data of R + B-2G output from the adder 173a, the data of RB output from the subtractor 173b,
And the integrating circuit 163 of the optical detector 160
Based on the integrated signal Sg5 (= G) output from G,
It has a gain setting unit 175 that generates an R gain signal Sr2, a G gain signal Sg2, and a B gain signal Sb2.

On the other hand, as shown in FIG. 2C, the optical detector 160 of the second configuration example has integration circuits 163R, 163G, 163B prepared for R, G, B colors.
have. Each integrating circuit 163R, 163G, 163
B integrates the input R signal Sr3, G signal Sg3, and B signal Sb3 for each field, and integrates signals Sr5, Sg5, Sb5, which are examples of optical detector values.
Is input to the white balance controller 170.

The integrating circuits 163R, 163G, 16
Similar to the integrating circuits 163a and 163b of the first example, 3B has different integration ranges divided into a high-luminance portion and a normal-luminance portion according to the integral slice level based on the luminance level, and in the high-luminance portion, 3B is higher than the integral slice level. It is also possible to integrate only the data R, G, B of the high-luminance part and integrate only the data R, G, B of the low-luminance part below the integration slice level in the normal brightness part. In this case, it is also possible not to integrate the data above the high brightness limiter and the data below the low brightness limiter.

The optical detector 160 of the second configuration example has a subtracter 165a for subtracting the integrated signal Sg5 from the integrating circuit 163G from the integrated signal Sr5 from the integrating circuit 163R and outputting a color difference signal RG, and an integrating circuit 163.
A subtractor 165b that subtracts the integrated signal Sg5 from the integration circuit 163G from the integrated signal Sb5 from B to output the color difference signal BG.

The white balance controller 170 of the second configuration example has the color difference signal R-G from the subtractor 165a.
And the color difference signal BG from the subtractor 165b are added to obtain R
An adder 173a that generates + B-2G data, and a subtracter 173b that subtracts the color difference signal BG from the subtractor 165b from the color difference signal RG from the subtractor 165a to generate RB data. Have.

Further, the white balance controller 17
0 is the data of R + B-2G output from the adder 173a, the data of RB output from the subtractor 173b,
And the integrating circuit 163 of the optical detector 160
Based on the integrated signal Sg5 (= G) output from G,
It has a gain setting unit 175 that generates an R gain signal Sr2, a G gain signal Sg2, and a B gain signal Sb2.

FIG. 3 is a diagram for explaining an example of the gain setting section 175, FIG. 3 (A) is a functional block diagram showing a configuration example, and FIG. 3 (B) is a diagram for explaining the operation in this configuration example. Is.

As shown in FIG. 3A, the gain setting unit 1
75 is the data R + B- input from the adder 173a.
2G is Sg5 (= G) input from the integrating circuit 163G
Data normalized by dividing by (R + B-2G)
The data (R-B) / normalized by dividing the data R-B input from the divider 176a that generates / G and the subtractor 173b by Sg5 (= G) input from the integration circuit 163G. And a divider 176b for generating G.

The gain setting section 175 also includes a divider 176.
The normalized data (R + B-2G) / G output from a and the normalized data (R- output from the divider 176b).
B) / G (both of which are collectively referred to as an evaluation value) is a stop portion determination unit 177a as a functional portion for determining whether or not a stop area on the coordinate plane shown in FIG. Inside or inside the restart area to the pull-in restriction frame, the restart area determination unit 177b as a functional part for determining whether or not it is set, and various area information used for the determination processing in the gain setting unit 175 are set. The area setting unit 178 is provided.

The area setting section 178 includes a non-volatile memory such as a flash memory. The area setting unit 178
The area information may be switched and set in response to an instruction from the user via a user operation (operation unit) not shown.

Further, the gain setting section 175 has a pull-in determination section 179 for determining the pull-in range based on the normalized (R + B-2G) / G and the data RB / G, a stop area determination section 177a, and a restart. When the determination results by the area determination unit 177b and the pull-in determination unit 179 satisfy the predetermined condition (see later), the R gain signal Sr2 and the G gain signal Sg.
2. It has a gain control unit 180 that generates the B gain signal Sb2 and activates the automatic white balance control.

The attraction determination unit 179 is the stop area determination unit 17
7a and the restart area determination unit 177b may be provided in the preceding stage. That is, the gain setting unit 175
It is determined in advance whether the evaluation value is within the pull-in restriction frame, and only when the evaluation value is within the pull-in restriction frame, the evaluation value is further within the stop region, whether within the stop region to the restart region, or It may be determined whether or not it is within the restart area to the pull-in restriction frame.

The gain controller 180 controls the R gain signal Sr.
2, G gain signal Sg2, B gain signal Sb2 are converted to corresponding white balance amplifiers 152R, 152G, 15
2B (see FIG. 1) to input the R gain signals Sr1, G
The white balance is automatically adjusted (adjusted) by feedback controlling each gain so that the ratio of the gain signal Sg1 and the B gain signal Sb1 becomes equal.

When taking this white balance,
Based on the two axes of RB and R + B-2G, the white balance amplifiers 152R and 15R are controlled by feedback control.
Each gain of 2G and 152B is operated. This gives R
Signal Sr1 × R gain signal Sr2 = G signal Sg1 × G gain signal Sg2 = B signal Sb1 × B gain signal Sr2,
That is, (R signal Sr1 × R gain signal Sr2) − (G
Signal Sg1 × G gain signal Sg2) = (B signal Sb1 ×
The relational expression of B gain signal Sb2) − (G signal Sg1 × G gain signal Sg2) = 0 holds.

At this time, the white balance controller 170 performs a process for adjusting the white balance, and based on the white balance gain and the evaluation value data from the optical detector 160, the color temperature in the state where the white balance is adjusted and the color temperature after that. In this way, the size of the pull-in limit frame that limits the range of white balance is changed according to the change in the color temperature of the light source.

The data R- output from the subtractor 173b
B is for the color temperature change of the subject (or light source),
It is close to the locus where the white color changes (blackbody radiation curve). on the other hand,
The data R + B-2G output from the adder 173a is
This is the direction in which the white color changes when the subject is under fluorescent lighting (or when the light source is fluorescent lighting).

The white balance controller 170 is
When performing auto white balance processing by feedback control based on each data of R + B-2G and RB, white balance is performed based on a white reference point (corresponding to the origin of the coordinate axis) under a certain color temperature (for example, 3200K). By setting a pull-in limit frame that limits the range of taking, the movement amount of the evaluation value data (integrated value data) from the optical detector 160 from the reference point (0, 0) is constantly monitored, and the evaluation value data after the movement is set. Is present within the withdrawal restriction frame (withdrawal determination by the withdrawal determination unit 179), and if there is R, G,
Control each gain of B.

In order to adjust the white balance, first, the integrated value data of the high luminance part and the integrated value data of the normal luminance part are compared, and the origins (R−G = 0, B−G) on the color difference plane are compared.
= 0, R + B-2G = 0), that is, the one closer to “0” is adopted. The evaluation value data is (R + B-2G) /
A coordinate plane represented by the G axis and the (RG) / G axis (see FIG.
(See (B)) It is determined which coordinate position is on, and if the coordinate position is outside the pull-in limit frame, the pull-in operation for adjusting the white balance is not performed.

The reason why the pull-in limit frame is set is to prevent a malfunction such as pulling in an object that is not originally white. That is, the pull-in limit frame is not a color on the black body radiation curve (for example, yellow, green, magenta).
It is to prevent you from pulling in.

On the other hand, if it is within the pull-in restriction frame, the normalized data (R + B-2G) / G and (RB) / G are further added.
Whether the (collectively referred to as an evaluation value obtained from the optical detector value) is within the stop area on the coordinate plane shown in FIG. 3B, is the stop area determination unit 177a within the stop area to the restart area? Alternatively, the restart area determining unit 177b determines whether the restart area is within the pull-in restriction frame.

Then, the white balance gain is operated according to the coordinate position (quadrant or on the axis) to bring the evaluation value closer to the origin (R-G = 0, R + B-2G = 0) direction. That is, the gain setting unit 175 has the stop area determining unit 177.
When the determination result of “a” and the restart region determination unit 177b satisfies the predetermined condition, the automatic white balance control is activated.

Here, in the state where the white balance is maintained, the evaluation value obtained from the optical detector value is shown in the complete dead zone region including the coordinate origin of FIG. 3 (B). In addition to the complete dead zone area, a stop area, which is another dead zone, is provided so as to include the complete dead zone area, and a restart area is provided so as to include this stop area.

The complete dead zone is provided for the purpose of preventing oscillation when the focus does not completely converge on the origin. That is, 2
Two orthogonal color difference axes (R + B-2G) / G and (R-
The inside of the complete dead zone surrounded by the origin on the color difference plane represented by B) / G is a region where the white balance is substantially taken.

When the object or the light source changes in this state, the evaluation value obtained from the optical detector value is
It comes out of the complete dead zone and further out of the stop zone. This indicates that the relationship of the color signal components of the entire image pickup screen is broken. When it is determined that the Hawaiian balance has been lost immediately when it goes out of the complete dead zone and the gain operation of the white balance amplifiers 152R, 152G, 152B is restarted, the user feels strange.

Therefore, a stop region indicating a reference value for determining the end (convergence point) of the predefined white balance control and a reference value for determining the start (restart) of the predefined white balance control are set. The start and end of white balance control are controlled by comparing and determining the restart area shown and a predetermined index value. Specifically, it is as follows.

First, when the white balance control is once stable, that is, when the evaluation value is in a state of being located within the complete dead zone region, the evaluation value changes from that state, and the evaluation value changes outside the complete dead zone region. However, if the evaluation value is within the restart area, the white balance control is not restarted. If the optical detector value further changes from this state and the evaluation value is in a state of being located outside the restart region, and if it is within the pull-in limit frame, white balance control is performed, that is, the white balance amplifier 152.
The R, 152G, and 152B gain operations are restarted, and feedback control is performed so that the evaluation value converges toward the origin. That is, based on the operation direction and the operation amount of the white balance amplifier 152 determined as described above, the system controller 170 causes the white balance amplifier 152 to operate.
A series of controls such as setting each gain to each field is performed.

Then, when the evaluation value finally reaches the convergence point of the white balance control, that is, the complete dead zone including the origin of the coordinates shown in FIG. Stop the operation. In this way, once the state reaches the complete dead zone or the position within the stop region, the white balance control is not restarted until the evaluation value comes out of the restart region again.

As a result, white balance can be achieved without hypersensitivity to changes in the image pickup screen, that is, changes in the optical detector. The complete dead zone and the stop area may be completely the same.

Here, the gain control section 180 adjusts the white balance by performing the gain operation as follows. First, in FIG. 3B, when the evaluation value is in the first quadrant, the gain of the white balance amplifier 152R (R gain signal Sr2) is lowered, and when it is in the second quadrant, the gain of the white balance amplifier 152B (B gain). Signal Sb2) is raised, the R gain signal Sr2 is raised when in the third quadrant, and the B gain signal Sb is raised when in the fourth quadrant.
Lower 2.

When the evaluation values are on the axes R-B and R + B-2G, the R gain signal Sr2 and the B gain signal Sb2 are simultaneously operated. That is, the evaluation value (R +
B-2G) / G is positive (> 0) and (R + B-2G) /
When it is on the G axis, both the R gain signal Sr2 and the B gain signal Sb2 are lowered. Evaluation value (R + B-2G)
/ G is negative (<0) and is on the (R + B-2G) / G axis, the R gay signal Sr2 and the B gain signal Sb
Raise 2 together.

The data (RB) / G is positive (>0;
When the color temperature is on the low color temperature side and is on the (RB) / G axis, R
The gain signal Sr2 is lowered and the B gain signal Sb2 is raised. When the evaluation value (RB) / G is negative (<0; high color temperature side) and is on the (RB) / G axis, the R gain signal Sr
2 is raised and the B gain signal Sb2 is lowered.

Here, in the case where the complete dead zone and the stop area are different, if the white balance control is performed until the evaluation value falls within the complete dead zone, almost perfect white balance can be achieved. However, when the light source color changes significantly, for example, when reddish light enters the entire screen, the user may feel a sense of discomfort due to adaptation to human color. In such a case, this problem can be solved by stopping the white balance control when the evaluation value falls within the stop area.

The complete dead zone area, the stop area, and the restart area constitute a hysteresis area related to the start of the white balance control. However, the sizes of the complete dead zone area (or the stop area) and the restart area are different from each other. Regarding 1), the complete dead zone area (or stop area) has at least the size that can prevent oscillation when it does not completely converge to the origin. 2) The restart area includes the complete dead zone area (or stop area). There is no particular condition as long as the following two conditions (hereinafter referred to as region definition conditions), which have the minimum included size, are satisfied, but set a large difference between the complete dead zone (or stop area) and the restart area. Due to
It is possible to reduce the chances of restarting the white balance control, and it is possible to realize the white balance control that does not react hypersensitively to changes in the imaging screen, that is, changes in the optical detector value.

In addition, by providing the stop area between the complete dead zone and the restart area, it is possible to realize white balance control that does not give the user a feeling of discomfort even when the light source color changes significantly. .

When the evaluation value is within the stop area, the white balance control is stopped, and after a predetermined time has elapsed, the white balance control is stabilized, and then the white balance control is continued until the evaluation value falls within the complete dead zone area. If implemented, even if the light source color changes significantly, it is possible to achieve almost perfect white balance without giving a sense of discomfort to the user.

If the repetitive cycle at this time is made slower than the normal repetitive cycle of the gain adjustment, the repetitive cycle slowly converges to the perfect white, so that there is almost no possibility of giving the user a feeling of strangeness due to the white balance control. In addition, without stopping once when the evaluation value falls within the stop area,
Immediately, the white balance control may be performed in a repetition cycle slower than the normal gain adjustment repetition cycle until the evaluation value falls within the complete dead zone region.

By the white balance control as described above, for example, even when the ratio of the object having a color close to the black body radiation curve to the image pickup screen becomes high, the light source around the object, that is, the color temperature changes. Therefore, it is possible to prevent an erroneous operation such as controlling the gain of the white balance amplifier in the direction of making the color closer to an achromatic color.

Further, even if a new subject enters a part of the image pickup screen in a state where white balance is once taken, it is possible to prevent the inconvenience of immediately executing the white balance control.

In other words, under a constant light source, while the white balance control was once stable, a new subject entered the imaging screen range or the angle of view of the camera changed,
Even if the subject is displaced as a whole, it is possible to suppress re-execution of the white balance control, and it is possible to improve the visual performance felt by the user in the image pickup apparatus having the white balance control. .

If the range of the stop region is set to be too large, it is determined that the convergence is completed before the optical detector value approaches the white-balanced relationship, which may cause a visual discomfort to the user. Conceivable. Also, if the range of the restart area is set too large,
It is conceivable that the relationship of the optical detector values, which is the restart condition of the white balance control, becomes more unbalanced, which may cause a visual discomfort for the user.

Therefore, the stop area and the restart area should be set in appropriate ranges such that the white balance control does not give the user a visually uncomfortable feeling. Since the level at which a visually uncomfortable feeling is generated varies depending on the user, the subject, the state of the light source, and the like, it is extremely significant to have a configuration in which the stop region and the restart region can be changed appropriately.

Each functional unit in the white balance controller 170 may be implemented by software, for example. In this case, programs for implementing these functions are stored in a program memory (for example, a non-volatile memory) not shown.

Then, the white balance controller 1
In 70, by converting the respective integrated value data of the color difference data RB and BG into the respective data of R + B-2G and RB by the addition / subtraction processing, the data can be obtained by simple arithmetic processing such as addition and subtraction. Since the conversion can be performed, the processing load can be reduced. Further, by configuring each functional block of the white balance controller 170 with hardware, it is possible to reduce the load on software.

As described above, in the color signal processing circuit of the image pickup apparatus, the evaluation value from the reference point is determined according to each condition when performing the operation of automatically adjusting the white balance by the feedback control. Then, it is determined whether the coordinates of the evaluation value are in any of the stop area, the restart area, and the pull-in restriction frame. Then, the white balance pull-in operation is performed according to the determination result, that is, the restart condition of the white balance control is controlled, or the convergence completion condition is controlled.
As a result, the user does not feel uncomfortable, and
It is possible to realize automatic white balance control that does not deviate from the originally set pull-in restriction frame.

That is, when the white balance is taken under a light source of a certain color temperature and then the color temperature changes next, if the color temperature is within the standard pull-in limit frame and the color temperature is adjusted to white, the coordinates of the evaluation value are shown. White balance depending on whether it is in the stop area or the restart area,
By not taking anything that is not so, it is possible to perform the white balance operation without malfunction or discomfort. For non-white objects, it is possible to perform an operation that does not change the color (achromatic color).

In addition, from the idea that when the colors of images including various colors are added together, white is obtained.
Color signals R-G and B-G are generated after integrating the color signals of, or the color difference signals R-G and B-G are generated and then integrated, and the integration range is set to the high brightness portion and the normal brightness. 2 settings are made, and it is determined whether the brightness is higher or the normal brightness is closer to white depending on the condition of the subject, and the white balance gain control is performed based on the integrated value data closer to white. As a result, more accurate auto white balance can be achieved.

In particular, the optical detector 1 of the first example
60, and the white balance controller 170, use each data of RG and BG as a parameter when deciding which data of the high brightness part of the integration by brightness or the normal brightness part is adopted. By doing so, it is not necessary to perform arithmetic processing such as multiplication and division, so that the load on hardware and software can be reduced.

Furthermore, when the white balance is adjusted, the respective data of RB and R + B-2G are used, so that RB is a locus in which white changes with respect to the color temperature of the subject (or the light source). Since it is close to (black body radiation curve), accurate control can be performed, and when R + B-2G is the subject under fluorescent light (or the light source is fluorescent light).
Since white is in the direction of change, accurate control can be performed as in RB. Moreover, the color signals Sr3, Sg3 before passing through the gamma correction unit 153 which is a non-linear circuit.
Since gain information is obtained using Sb3, there is no concern about color misregistration.

In the image pickup apparatus according to the present invention, most of the white balance function of the feedback control system can be realized by a simple addition / subtraction process, so that the hardware and software are less burdened and the subject (or light source) is also reduced.
It is possible to perform highly accurate control with respect to changes in the color temperature of.

Therefore, when it is applied to a consumer, commercial or industrial camera having an automatic white balance function, it can contribute to the improvement of image quality. This allows
Since there is no malfunction and more accurate auto white balance adjustment can be realized, it can greatly contribute to the image quality improvement of these cameras.

FIG. 4 is a flowchart summarizing the processing procedure of white balance control in the image pickup apparatus of the first embodiment. The processing of this routine is called and executed when the color temperature of the subject changes.
Before starting this process, a reference point (0, 0) is set at a predetermined color temperature (for example, 3200K), and the stop area and the restart area are set based on the reference point (0, 0). , Or set the pull-in limit frame and set the reference point (0,
0) may be set as the operation starting point of the automatic white balance.

When there is a change in the color temperature of the subject, first, in the optical detector 160, the subtractor 161
a and 161b are white balance amplifiers 152R and 1
Color difference signals (R-G) a and (R-G) a are acquired based on the color signals input from 52G and 152B (S10).
0). Next, the integrator circuits 163a and 163b eliminate each color difference signal in a portion below the low-luminance limiter level and above the high-luminance limiter level, and perform integration in different integration ranges of the high-luminance portion / normal luminance portion for each field. , Integrated color difference signals (R-G) b, (R-G) c, (B-G)
b, (BG) c are acquired (S102).

Then, the white balance controller 1
In 70, the comparison circuit 171 first determines whether or not the integrated color difference signal (integral value data) is data under special conditions such as full-face monochromatic color, full-face low brightness, or full-face high brightness (S104). ). If the data has special conditions, the process ends (S104-YES). On the other hand, if the data is not under special conditions, it is determined whether the data is outside the proper range, that is, whether the integrated value data is too low or too high (S106).

If the integrated value data is out of the proper range, the process ends (S106-YES). If the integrated value data is within the proper range, the integrated value data of RG and BG separately integrated in the high luminance part and the normal luminance part are compared, and the data closer to "0" is adopted. , The selected color difference signal (R-
G) d and (BG) d are output (S108).

The adder 173a and the subtractor 173b are
By adding and subtracting using the integrated value data of the adopted one, the integrated value data of RG and BG can be converted into R + B.
-2G, RB data (evaluation value) is converted and input to the gain setting unit 175 (S110).

Next, in the gain setting section 175, the stop area determination section 177a, the restart area determination section 177b, and the pull-in determination section 179 determine the sign of R + B-2G (S112), and then the R-B. The sign is determined (S11
4). By this code determination, it is determined where the evaluation value is on the R-B and R + B-2G color difference coordinate planes.

For example, the stop area determination unit 177a determines whether or not the evaluation value is within the stop area (S120).
If it is not within the stop area, the restart area determination unit 177b
It is determined whether the evaluation value is within the restart area (S12).
0-NO, S122). If it is not within the restart area, the attraction-in determination unit 179 determines whether or not the evaluation value is within the attraction-in restriction frame (S122-NO, S124). If the evaluation value is outside the pull-in restriction frame, the process is terminated (S124-N).
O).

On the other hand, if the evaluation value is within the pull-in limit frame, the gain control section 180 restarts the white balance control, and the white balance amplifiers 152R, 152G, 15
By controlling (adjusting) each gain of 2B, a pull-in operation for white balance is performed (S12).
6).

On the other hand, during the determination processing of step S120,
If the evaluation value is within the stop area, the gain controller 18
For 0, first, it is determined whether or not the white balance control is being restarted, that is, the gain adjustment is being performed (S120-YE).
S, S130). If the gain is not being adjusted, the process ends as it is (S130-NO). If the gain is being adjusted, the gain adjustment is stopped and then the process ends (S1).
32).

In any case, when the processing is stopped (end), the gain control section 180 maintains the gain value at that time. In this case, the white balance is normally achieved.

FIG. 5 is a diagram for explaining another operation example of the gain setting section 175. Here, in FIG. 5A, the restart area is divided into two, and the restart area 1 is provided inside the pull-in restriction frame.
3 is that the restart area 2 is provided so as to include
Different from that shown in (B).

Restart region determination unit 1 of gain setting unit 175
77b determines whether the evaluation value is within the stop region to the restart region 1. Further, the restart area determination unit 177b determines whether the evaluation value is within the restart area 1 to restart area 2 or within the restart area 2 to the attraction-in restriction frame.

Then, the gain control section 180 operates the white balance gain according to the coordinate position (quadrant or on-axis), and sets the evaluation value to the origin (RG =
0, R + B-2G = 0) direction.

First, in the same way as shown in FIG.
Once the evaluation value is in the complete dead zone area, the evaluation value changes from that state, and even if the evaluation value changes outside the complete dead zone area, the evaluation value remains in the restart area 1. If so, the white balance control is not restarted.

If the optical detector value further changes from this state and the evaluation value is located outside the restart area 1, and if it is within the pull-in limit frame, white balance control is performed, that is, the white balance amplifier 152R. , 152G, 152B are restarted so that the evaluation value converges toward the origin.

At this time, for example, if the evaluation value is in the restart area 1 to restart area 2, the gain operation is performed until the evaluation value is in the complete dead zone.
Adjust the white balance almost completely. On the other hand, if the evaluation value is within the restart area 2 to the pull-in restriction frame, the gain operation is stopped when the evaluation value is in the stop area.

Even in this case, if the white balance control is performed until the evaluation value falls within the complete dead zone region after the gain operation is stabilized for a predetermined time after the gain operation is stopped, the light source color becomes large. Even when the white balance changes, the white balance can be almost completely adjusted without giving the user a feeling of strangeness. When the repetition cycle at this time is set to be slower than the normal gain adjustment repetition cycle, the white balance is slowly converged to completely white, so that there is almost no fear of giving the user a feeling of strangeness due to the white balance control.

In this way, once the state reaches the complete dead zone or within the stop area, the white balance control is not restarted until the evaluation value is outside the restart area.

As described above, according to the embodiment shown in FIG. 5A, the white balance control is not restarted when the light source color change is relatively small within the restart region, and the restart regions 1 to 1 are restarted. The white balance control is restarted when the change is moderate within 2 and the white balance is taken almost completely, and the white balance control is restarted when the change exceeds the restart range 2, but the convergence By limiting the range to the stop region, it is possible to realize white balance control that does not give the user a feeling of strangeness.

That is, since the stop area and the restart area shown in FIG. 3B are subdivided into the stop area, the restart area 1 and the restart area 2, the range of the stop area is secured too large. In addition, the starting point of restart (restart area 1) is not secured too much. As a result, the convergence is not determined before the optical detector value approaches the white-balanced relationship, and in addition, the relationship of the optical detector values that is the restart condition of the white balance control becomes unbalanced. In addition, smoother white balance control can be realized without giving the user a visual discomfort.

On the other hand, FIG. 5B is different from that shown in FIG. 3B in that the restart area is defined by a quadrangle other than the rectangular shape. It should be noted that not only the restart area but also the stop area may be defined by a quadrangle other than the rectangular shape. By doing so, the restart condition of the white balance control and the convergence completion condition of the white balance control can be made different depending on the hue.

For example, when a human face is the main subject, warm white images can be created by stopping the white balance control in a slightly reddish state, while fruits and vegetables are the main subjects. In this case, it is possible to realize white balance control according to the subject image without visually discomforting the user, such as creating an image that gives a fresh atmosphere by stopping the white balance control in a slightly bluish state. You can

The stop area and the restart area are not limited to being defined by a quadrangle, but may be another polygon, a circle, an ellipse, or a locally recessed or formed area as long as the above area definition conditions are satisfied. For example, a convex portion may be formed. Further, the restart area need only include the stop area, and may be a part of the restart area. Further, the area data such as the dead zone, the stop area, the restart area, or the pull-in restriction frame may be appropriately changed according to individual user specifications, for example, through user operation.

FIG. 6 is a block diagram showing a second embodiment of the image pickup apparatus according to the present invention. It differs from the first embodiment in that a recording medium such as a memory card can be inserted and removed and that the recording medium can be connected to a communication network such as the Internet.

That is, the image pickup apparatus 1 according to the second embodiment.
0 is the CPU 9 in addition to the image pickup apparatus 10 of the first embodiment.
02, ROM (Read Only Memory) 904, RAM 90
6, a memory reading unit 907, and a communication I / F (interface) 908.

The recording medium 924 is, for example, the CPU 90.
It is used for registering program data for causing 2 to perform software processing and area data such as a complete dead zone, a stop area, a restart area, or a pull-in restriction frame. The memory reading unit 907 stores (installs) the data read from the recording medium 924 in the RAM 906.

The communication I / F 908 mediates the transfer of communication data with a communication network such as the Internet.

The image pickup apparatus 10 having such a configuration can have the same basic configuration and operation as the image pickup apparatus 10 shown in the first embodiment. A program that causes a computer to execute the above-described processing is distributed through a recording medium 924 such as a flash memory, an IC card, or a nonvolatile semiconductor memory card such as a miniature card.

Further, the program may be downloaded and acquired from a server or the like via a communication network such as the Internet, or may be updated.

A semiconductor memory such as an IC card or a miniature card, which is an example of the recording medium 924, is used in the image pickup apparatus 10 described in the first embodiment (particularly, the optical detector 160 and the white balance controller 170 which are the main parts thereof). Some or all of the functionality of the process can be stored. Therefore, it is possible to provide the following program and a storage medium storing the program.

For example, the optical detector 160
The program for the program, that is, the software installed in the RAM 906 or the like, is the same as the optical detector 160 shown in the above embodiment, and the subtracter 161.
a, 161b, 165a, 165b and the integrating circuit 163
Each functional unit such as R, 163G, 163B, 163a, 163b is provided as software.

Similarly, the white balance controller 1
The program for 70, that is, the software installed in the RAM 906 or the like is similar to the white balance controller 170 shown in the above embodiment, and includes a comparison circuit 171, an adder 173a, a subtractor 173b, dividers 176a and 176b, and pull-in determination. Section 179, area setting section 1
78, stop area determination unit 177a, restart area determination unit 17
7b, or each functional unit such as the gain control unit 180 is provided as software.

Then, the memory reading section 907 stores the storage medium 9
The program data or the area data such as the complete dead zone, the stop area, the restart area, or the pull-in restriction frame is read from 24 and passed to the CPU 902. Then, the software is installed in the RAM 906 from the storage medium 924. It should be noted that a part of the program data or a part of the area data such as the dead zone, the stop area, the restart area, or the pull-in restriction frame may be installed in the ROM 904.

The RAM 906 stores various area data or programs read by the memory reading unit 907 and data created by the CPU 902 executing the programs, and reads the stored data or programs to the CPU 902. hand over.

The software is read by the RAM 906 and then executed by the CPU 902. For example CP
The U902 can realize the function for executing the above processing by software by executing the above-described auto white balance processing based on the programs stored in the ROM 904 and the RAM 906 which are examples of the recording medium. That is, the automatic white balance processing can be realized by digital signal processing using a computer.

Each storage medium 924 has a complete dead zone, a stop area, B, and
It is possible to retain area data such as C or the pull-in restriction frame, and a program in which the appropriate auto white balance processing based on the area data is described without being affected by the setting contents of other users.

By doing so, for example, the complete dead zone,
Area data such as a stop area, a restart area, or a pull-in restriction frame, and the above-mentioned automatic white balance processing based on the area data can be appropriately set in accordance with user specifications. Further, by simply exchanging the storage medium 924, it is possible to easily switch to a process according to individual user specifications.

Although the present invention has been described using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various changes or improvements can be added to the above-described embodiment, and a mode in which such changes or improvements are added is also included in the technical scope of the present invention. Also,
The above embodiments do not limit the claimed invention, and all combinations of the features described in the embodiments are not necessarily essential to the solution of the invention.

For example, in the above-described embodiment, the restart area that determines the restart condition of the white balance control and the stop area that determines the convergence condition of the white balance control are set, and the white balance control is performed according to the change of the optical detector value. Was trying not to react too sensitively,
The control may be performed only on one of the stop region and B.

In the above embodiment, the primary color separation section 151 for separating the image pickup signal obtained from the image pickup device 12 into the R (red), G (green) and B (blue) primary color signals is provided. B
The predetermined color difference signal or its integrated signal is obtained based on each primary color signal of, or the gain adjustment for white balance control is performed on each of the R, G, and B primary color signals. Not necessarily required.

For example, a predetermined color difference signal may be directly obtained from the image pickup signal from the image pickup device 12 without obtaining the primary color signal, or gain adjustment for white balance control may be performed on the color difference signal. . It is well known that the color difference signal directly obtained from the image pickup signal varies depending on the color arrangement of the color filters arranged on the image pickup element, the method of color separation processing, and the like.

In the above embodiment, the integrated values of the R-G and B-G color difference signals are used, but the color difference values are not limited to these, and R-Y, B-Y, or Cr, Cb are used.
Etc. can also be used.

In the above embodiment, the normalized data (R + B-2) is used as the color difference coordinate plane represented by the color difference axis corresponding to the R-G signal and the color difference axis corresponding to the R + B-2G signal.
G) / G axis and (RB) / G axis are used to achieve accurate white balance control, but the present invention is not limited to this, and the R + B-2G axis before normalization is not limited to this. And those shown on the R-B axis may be utilized. For example, a pair of RY, BY, a pair of RB, BG, etc.
It is also possible to use another color difference axis.

Further, Japanese Patent Laid-Open No. 11-41 filed by the applicant of the present application
619, 11-331854, or JP 20
No. 00-125309 or the like, for example, the pull-in limit frame in a white balance state is dynamically moved according to the color temperature, or the integral value is multiplied by a coefficient according to the white balance gain. The method of controlling the white balance gain based on the multiplication result and the method of the above embodiment may be combined. In this case, as described above, the reference point (0, 0) is set at a predetermined color temperature, and the stop area, the restart area, or the pull-in restriction frame is set based on the reference point, and the reference point is set. Should be used as the starting point of the automatic white balance operation.

[0147]

As described above, according to the present invention, in the automatic white balance adjustment using the feedback control, first, the stable point with the automatic white balance (the origin on the color difference plane or the complete dead zone surrounding the origin) is obtained. ) Outside of), the restart area which sets the restart condition of the white balance control so as to include this stable point is set. After stabilization, the auto white balance control is not restarted until the evaluation value obtained from the optical detector value exceeds the restart range, so white that does not react hypersensitively to changes in the imaging screen, that is, changes in the optical detector value. Balance control can be realized.

As another aspect, first, outside the stable point with the automatic white balance (same as above), a stop region for setting the convergence condition of the white balance control is set so as to include this stable point. After restarting, if the evaluation value obtained from the optical detector value reaches the stop range, automatic white balance control (gain change)
The control can be stopped before the color becomes completely achromatic, for example, even if the light source color changes significantly, that is, it is hypersensitive to changes in the optical detector value. White balance control can be realized.

As described above, according to the present invention, it is possible to prevent the phenomenon of excessively following changes in the subject and changes in the color of the light source, and it is possible to realize white balance control that does not give the user a feeling of strangeness.

[Brief description of drawings]

FIG. 1 is a block diagram showing a first embodiment of an image pickup apparatus according to the present invention.

FIG. 2 is a diagram illustrating an example of an optical detector and a white balance controller.

FIG. 3 is a diagram illustrating an example of a gain setting unit.

FIG. 4 is a flowchart summarizing a processing procedure of white balance control in the image pickup apparatus according to the first embodiment.

FIG. 5 is a diagram illustrating another operation example of the gain setting unit.

FIG. 6 is a block diagram showing a second embodiment of an image pickup apparatus according to the present invention.

FIG. 7 is a diagram illustrating a relationship between a blackbody radiation curve on a color difference coordinate plane and automatic white balance processing.

[Explanation of symbols]

10 ... Imaging device, 12 ... Imaging element, 13 ... Preamplifier,
14 ... A / D converter, 15 ... Digital signal processor, 15
1 ... Primary color separation part, 152R, 152G, 152B ... White balance amplifier, 153 ... Gamma correction part, 154 ...
Color difference matrix section, 155 ... Encoder section, 156 ... D
/ A converter, 160 ... Optical detector, 161
a, 161b, 165a, 165b, 173b ... Subtractor, 163R, 163G, 163B, 163a, 163
b ... Integrator circuit, 170 ... White balance controller, 171 ... Comparison circuit, 173a ... Adder, 175 ... Gain setting section, 176a, 176b ... Divider, 177a ...
Stop region determination unit, 177b ... Restart region determination unit, 178
... area setting section, 179 ... pull-in determination section, 180 ... gain control section

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Katsumi Kato             134, Kobe-cho, Hodogaya-ku, Yokohama-shi, Kanagawa               Sony LSI Design Stock Association             In-house F-term (reference) 5C065 AA01 BB02 CC01 CC03 CC09                       DD02 DD15 DD17 GG15 GG18                 5C066 AA01 CA17 EA03 EA13 EA14                       EC01 GA01 GA02 HA01 KA12                       KM02

Claims (26)

[Claims] 1. A color signal processing method for achieving white balance by performing gain adjustment on at least one color signal of a plurality of color signals acquired based on an image pickup signal from an image pickup device, The gain in order to achieve the white balance so as to surround a substantial white reference point on a color difference space indicated by a color difference axis corresponding to a plurality of color difference signals generated based on the gain adjusted color signals. Setting a restart region showing a reference value for restarting the adjustment, based on a plurality of integrated signals integrated over a predetermined period for each of the plurality of color difference signals, to determine the coordinate value on the color difference space A color signal processing method, wherein a gain value for performing the gain adjustment is controlled based on a positional relationship between the obtained coordinate value and the set restart area. . 2. The gain is adjusted so that the coordinate value to be obtained next moves toward the white reference point, provided that the coordinate value in the color difference space exists outside the set restart area. The color signal processing method according to claim 1, wherein the gain value for adjustment is changed from the gain value at that time. 3. A color signal processing method for achieving white balance by performing gain adjustment on at least one color signal of a plurality of color signals acquired based on an image pickup signal from an image pickup device, The gain for taking the white balance so as to surround a substantial white reference point on a color difference space indicated by a color difference axis corresponding to a plurality of color difference signals generated based on the gain adjusted color signals. A stop region indicating a reference value for stopping the adjustment is set, and a coordinate value on the color difference space is calculated based on a plurality of integrated signals integrated over a predetermined period for each of the plurality of color difference signals, A color signal processing method, wherein a gain value for performing the gain adjustment is controlled based on a positional relationship between the coordinate values and the set stop region. 4. The gain value for performing the gain adjustment is maintained at the gain value at that point on the condition that the coordinate value on the color difference space exists inside the set stop region. The color signal processing method according to claim 3. 5. The coordinate value to be obtained next at every predetermined timing slower than a normal repeating cycle of the gain adjustment, provided that the coordinate value in the color difference space exists inside the set stop region. 4. The color signal processing method according to claim 3, wherein the gain value for performing the gain adjustment is changed from the gain value at that time so that the value moves toward the white reference point. 6. A restart area is set so as to surround the stop area, the restart area indicating a condition for restarting the gain adjustment to obtain the white balance, and the calculated coordinate value, the set stop area, and the The color signal processing method according to claim 3, wherein the gain value for performing the gain adjustment is controlled based on the positional relationship with the set restart region. 7. The coordinate value to be obtained next is moved in the direction of the white reference point on the condition that the obtained coordinate value in the color difference space exists outside the set restart area. , The gain value for performing the gain adjustment is changed from the gain value at that time, and the gain adjustment is performed on condition that the obtained coordinate value on the color difference space exists inside the set stop region. The gain value to be performed is maintained at the current gain value, or the coordinate value to be obtained next is moved in the direction of the white reference point at every predetermined timing that is slower than the normal repetition cycle of the gain adjustment. 7. The color signal processing method according to claim 6, wherein the gain value for performing the gain adjustment is changed from the gain value at that time. 8. A color signal processing circuit for achieving white balance by performing gain adjustment on at least one color signal of a plurality of color signals acquired based on an image pickup signal from an image pickup device, The gain in order to achieve the white balance so as to surround a substantial white reference point on a color difference space indicated by a color difference axis corresponding to a plurality of color difference signals generated based on the gain adjusted color signals. Setting a restart region indicating a reference value for restarting the adjustment, to determine the coordinate value on the color difference space based on a plurality of integrated signals integrated over a predetermined period for each of the plurality of color difference signals, A white balance controller is provided for controlling the gain value for performing the gain adjustment based on the positional relationship between the obtained coordinate value and the set restart area. Color signal processing circuit, characterized in that the. 9. A color signal processing circuit for achieving white balance by performing gain adjustment on at least one color signal of a plurality of color signals acquired based on an image pickup signal from an image pickup device, The gain for taking the white balance so as to surround a substantial white reference point on a color difference space indicated by a color difference axis corresponding to a plurality of color difference signals generated based on the gain adjusted color signals. A stop region indicating a reference value for stopping the adjustment is set, and a coordinate value on the color difference space is obtained based on a plurality of integrated signals integrated over a predetermined period for each of the plurality of color difference signals, and the obtained value is obtained. A white balance controller that controls the gain value for performing the gain adjustment based on the positional relationship between the coordinate values and the set stop area. Color signal processing circuit according to claim. 10. The white balance controller sets a restart area indicating a reference value for restarting the gain adjustment for the white balance so as to surround the stop area, and the white balance controller sets a restart area for the plurality of color difference signals. A coordinate value on the color difference space is obtained based on a plurality of integrated signals integrated over a predetermined period for each, and the positional relationship between the obtained coordinate value and the set stop region and the set restart region. The color signal processing circuit according to claim 9, wherein a gain value for performing the gain adjustment is controlled based on the above. 11. A color signal processing circuit for achieving white balance by performing gain adjustment on at least one color signal of a plurality of color signals obtained based on an image pickup signal from an image pickup device, A gain adjustment amplifier unit that adjusts the gain of the color signal, and a plurality of integrated signals that are integrated over a predetermined period for each of the plurality of color difference signals based on the color signal that has been gain adjusted by the gain adjustment amplifier unit. And an integration processing unit that obtains, and restarts the gain adjustment for the white balance so as to surround a substantial white reference point on the color difference space indicated by the color difference axes corresponding to the plurality of color difference signals. Based on a plurality of integration signals generated by the integration processing unit and a restart region setting unit that sets a restart region indicating a reference value, the color difference sky Gain control for obtaining the upper coordinate value and controlling the gain value to be set in the gain adjustment amplifier section based on the positional relationship between the coordinate value and the restart area set by the restart area setting section. And a color signal processing circuit. 12. A color signal processing circuit for achieving white balance by performing gain adjustment on at least one color signal of a plurality of color signals acquired based on an image pickup signal from an image pickup device, A gain adjustment amplifier unit that adjusts the gain of the color signal, and a plurality of integrated signals that are integrated over a predetermined period for each of the plurality of color difference signals based on the color signal that has been gain adjusted by the gain adjustment amplifier unit. And an integration processing unit that obtains, and stops the gain adjustment for the white balance so as to surround a substantially white reference point on a color difference space indicated by color difference axes corresponding to the plurality of color difference signals. A stop area setting unit that sets a stop area indicating a reference value, and a plurality of integrated signals generated by the integration processing unit based on the color difference space. A gain control unit for obtaining a standard value and controlling a gain value to be set in the gain adjustment amplifier unit based on a positional relationship between the coordinate value and the stop region set by the stop region setting unit. A color signal processing circuit characterized by the above. 13. A restart area setting for setting a restart area indicating a reference value for restarting the gain adjustment for the white balance so as to surround the stop area set by the stop area setting unit. A gain control unit, based on the positional relationship between the coordinate values and the restart area set by the restart area setting unit and the stop area set by the stop area setting unit, the gain control unit, 13. The color signal processing circuit according to claim 12, which controls a gain value to be set in the gain adjustment amplifier unit. 14. The primary color separation unit that separates primary color signals of R (red), G (green), and B (blue) from the image pickup signal as the plurality of color signals. Item 13. The color signal processing circuit according to any one of items 1 to 13. 15. A color difference axis corresponding to the RG signal, which is a component obtained by subtracting the G (green) primary color signal from the R (red) primary color signal, as a color difference axis corresponding to the plurality of color difference signals. , A color difference corresponding to an R + B-2G signal, which is a component obtained by subtracting a double component of the G (green) primary color signal from a component obtained by adding the B (blue) primary color signal to the R (red) primary color signal. 15. The color signal processing circuit according to claim 14, wherein the axis is used. 16. An image pickup device which outputs an image pickup signal according to an image pickup result, and a gain adjustment for at least one color signal of a plurality of color signals acquired based on the image pickup signal from the image pickup device. A gain adjustment amplifier section to perform, and an integration processing section to obtain a plurality of integrated signals integrated over a predetermined period for each of the plurality of color difference signals, based on the color signals gain-adjusted by the gain adjustment amplifier section. , A restart indicating a reference value for restarting the gain adjustment to achieve the white balance so as to surround a substantially white reference point on the color difference space indicated by the color difference axes corresponding to the plurality of color difference signals. Based on a restart area setting unit that sets a region and a plurality of integration signals generated by the integration processing unit, a coordinate value on the color difference space is obtained, and the coordinate value and the previous value are calculated. An imaging device, comprising: a gain control unit that controls a gain value to be set in the gain adjustment amplifier unit based on a positional relationship with the restart region set by the restart region setting unit. . 17. An image pickup device for outputting an image pickup signal according to an image pickup result, and a gain adjustment for at least one color signal of a plurality of color signals acquired based on the image pickup signal from the image pickup device. A gain adjustment amplifier section to perform, and an integration processing section to obtain a plurality of integrated signals integrated over a predetermined period for each of the plurality of color difference signals, based on the color signals gain-adjusted by the gain adjustment amplifier section. , A stop region indicating a reference value for stopping the gain adjustment for the white balance so as to surround a substantially white reference point on the color difference space indicated by the color difference axes corresponding to the plurality of color difference signals. Based on the stop region setting unit to be set and a plurality of integration signals generated by the integration processing unit, a coordinate value in the color difference space is obtained, and the coordinate value and the stop Based on the positional relationship between the stop area set by frequency setting section, the imaging device being characterized in that a gain control unit for controlling a gain value for setting the gain adjusting amplifier unit. 18. A restart area setting for setting a restart area indicating a reference value for restarting the gain adjustment for the white balance so as to surround the stop area set by the stop area setting unit. A gain control unit, based on the positional relationship between the coordinate values and the restart area set by the restart area setting unit and the stop area set by the stop area setting unit, the gain control unit, The image pickup apparatus according to claim 17, wherein a gain value to be set in the gain adjustment amplifier unit is controlled. 19. The primary color separation unit that separates primary color signals of R (red), G (green), and B (blue) from the image pickup signal as the plurality of color signals. 19. The image pickup apparatus according to any one of items 1 to 18. 20. A color difference axis corresponding to the plurality of color difference signals, the color difference axis corresponding to an RG signal which is a component obtained by subtracting the G (green) primary color signal from the R (red) primary color signal. , A color difference corresponding to an R + B-2G signal, which is a component obtained by subtracting a double component of the G (green) primary color signal from a component obtained by adding the B (blue) primary color signal to the R (red) primary color signal. The image pickup device according to claim 19, wherein an axis is used. 21. Color signal processing for achieving white balance is performed by performing gain adjustment on at least one color signal of a plurality of color signals acquired based on an image pickup signal from an image pickup device. A program, wherein the computer is configured to surround a substantially white reference point on a color difference space indicated by a color difference axis corresponding to a plurality of color difference signals generated based on the gain-adjusted color signal, A restart area setting unit that sets a restart area indicating a reference value for restarting the gain adjustment for white balance, and a plurality of integrals integrated over a predetermined period for each of the plurality of color difference signals. A coordinate value on the color difference space is obtained based on a signal, and based on the positional relationship between the obtained coordinate value and the restart area set by the restart area setting unit. And a program that causes a gain control unit to control a gain value for performing the gain adjustment. 22. Color signal processing for achieving white balance is performed by performing gain adjustment on at least one color signal of a plurality of color signals acquired based on an image pickup signal from an image pickup device. A program, wherein the computer is configured to surround a substantially white reference point on a color difference space indicated by a color difference axis corresponding to a plurality of color difference signals generated based on the gain-adjusted color signal, Based on a plurality of integration signals integrated over a predetermined period for each of the plurality of color difference signals, a stop area setting unit that sets a stop area indicating a reference value for stopping the gain adjustment for white balance. Then, the coordinate value on the color difference space is obtained, and based on the positional relationship between the obtained coordinate value and the stop area set by the stop area setting unit,
A program that functions as a gain control unit that controls a gain value for performing the gain adjustment. 23. Color signal processing for achieving white balance is performed by performing gain adjustment on at least one color signal of a plurality of color signals acquired based on an image pickup signal from an image pickup device. A program, wherein the computer is configured to surround a substantially white reference point on a color difference space indicated by a color difference axis corresponding to a plurality of color difference signals generated based on the gain-adjusted color signal, A restart area setting unit that sets a restart area indicating a reference value for restarting the gain adjustment for white balance, and a plurality of color difference signals generated based on the gain adjusted color signals. A reference value for stopping the gain adjustment for the white balance is set so as to surround a substantially white reference point on the color difference space indicated by the color difference axis. A stop area setting unit that sets a stop area shown, and obtains coordinate values on the color difference space based on a plurality of integrated signals that are integrated over a predetermined period for each of the plurality of color difference signals, and the obtained coordinates Functions as a gain control unit that controls a gain value for performing the gain adjustment based on a positional relationship between a stop region set by the stop region setting unit and the stop region set by the stop region setting unit. A program characterized by: 24. Color signal processing for achieving white balance is performed by performing gain adjustment on at least one color signal of a plurality of color signals acquired based on an image pickup signal from an image pickup device. A computer-readable storage medium storing a program, wherein the computer is configured to operate on a color difference space indicated by color difference axes corresponding to a plurality of color difference signals generated based on the gain-adjusted color signals. A restart area setting unit that sets a restart area indicating a reference value for restarting the gain adjustment so as to take the white balance so as to surround a white reference point, and a predetermined period for each of the plurality of color difference signals. The coordinate value on the color difference space is calculated based on a plurality of integrated signals integrated over the range, and the calculated coordinate value and the restart area setting unit are A storage medium, which stores a program for functioning as a gain control unit that controls a gain value for performing the gain adjustment, based on the set positional relationship with the restart region. 25. A color signal process for achieving white balance is performed by performing gain adjustment on at least one color signal of a plurality of color signals acquired based on an image pickup signal from an image pickup device. A computer-readable storage medium storing a program, wherein the computer is configured to operate on a color difference space indicated by color difference axes corresponding to a plurality of color difference signals generated based on the gain-adjusted color signals. A stop area setting unit that sets a stop area indicating a reference value for stopping the gain adjustment for taking the white balance so as to surround a white reference point, and a predetermined period for each of the plurality of color difference signals. The coordinate values in the color difference space are calculated based on the integrated signals that have been integrated, and the calculated coordinate values and the stop area setting unit set the values. Based on the positional relationship with the stop area,
A storage medium storing a program for functioning as a gain control unit for controlling a gain value for performing the gain adjustment. 26. A color signal process for achieving white balance is performed by performing gain adjustment on at least one color signal of a plurality of color signals acquired based on an image pickup signal from an image pickup device. A computer-readable storage medium storing a program, wherein the computer is configured to operate on a color difference space indicated by color difference axes corresponding to a plurality of color difference signals generated based on the gain-adjusted color signals. A restart area setting unit that sets a restart area indicating a reference value for restarting the gain adjustment so as to surround the white reference point, and is generated based on the gain-adjusted color signal. The white balance so as to surround a substantially white reference point on the color difference space indicated by the color difference axes corresponding to the plurality of color difference signals. A stop area setting unit that sets a stop area indicating a reference value for stopping the gain adjustment, and the color difference based on a plurality of integrated signals that are integrated over a predetermined period for each of the plurality of color difference signals. Obtaining coordinate values in space, and performing the gain adjustment based on the positional relationship between the obtained coordinate values and the stop area set by the stop area setting unit and the stop area set by the stop area setting unit. A storage medium having stored therein a program for functioning as a gain control unit for controlling a gain value.