JPH0411494A - White balance adjusting device - Google Patents

Abstract

PURPOSE:To realize stable white balance adjusting by executing fuzzy inference based on some rules set in advance and determining the representative value of picture color information. CONSTITUTION:An integrator 23 receives a switching signal S2, adds the A/D converting value of a selective circuit 21 output for every area over for one field period, digitally integrates for every 64 areas and holds an integrated value as luminance or color evaluating value in a memory 26. consequently, the digital integrated value of a luminance signal Y is obtained as the 64 luminance evaluating values. the digital integrated value for every area of a color difference signal R-Y is obtained as the 64 color evaluating values in a next field. Thus, the values of luminance evaluating value and the color evaluating value are held in the memory 26 at the time when the multiplying of the luminance signal Y and a color difference signal (R-Y) (B-Y) is finished. then, the representative value of picture color information is determined by fuzzy inference.

Description

[Detailed description of the invention] (a) Industrial application field The present invention is based on a captured video signal obtained from an image sensor.
The present invention relates to a mill balance adjustment device for a color video camera that controls mill balance.

(b) Conventional technology In color video cameras, it is necessary to control white balance in order to correct for differences in the wavelength distribution of light depending on the light source.

This control includes red (hereinafter referred to as R), blue (hereinafter referred to as B), green (hereinafter referred to as G).
) by adjusting the gain of each color signal so that the ratio of the three primary color signals becomes 1:1:1. Generally, a method is used in which the gain is adjusted so that the integral value of the screen color difference signals R-Y and B-Y becomes zero, as shown in, for example, Japanese Patent Application Laid-Open No. 62-35792 (HO4N9/73). ing.

FIG. 2 is a block diagram of a mill balance adjustment circuit using this method.

The light that has passed through the lens (1) is transferred to an imaging device (CCD) (2).
), the color separation circuit (3) converts R, G,
The B signal is extracted as the three primary color signals, the G signal is directly output, and the R and B signals are output to the R amplifier circuit (4) and the B amplifier circuit (5).
The luminance signal Y and the red and blue color difference signals R are then input to the color process and matrix circuit (6).
-Y, B-Y are created and sent to the video circuit. At the same time, the two color difference signals are respectively input to the integrating circuit (17) (
1g), the gain control circuits (13) and (14) adjust the gains of the R and B amplifier circuits (4) and (5) so that the integration is performed for a sufficiently long time and the result becomes zero. .

In this method, the screen shot by a video camera,
If we take measures such as increasing the time constants of the integrating circuits (17) and (18) for various color distributions and average these color distributions, the color components that make up the color distribution will cancel each other out, It is assumed that a substantially white screen state can be approximated.

However, when the color of the subject itself is biased, for example when green grass or blue sky occupies a large area on the screen, the gain changes in the direction of canceling out the biased color and the white balance shifts to its complementary color. It shifts.

As a countermeasure against this, color information signals from parts of the screen that are not included in a predetermined range on the color plane or parts whose brightness level exceeds a certain value should not be input to the integrating circuits (17) and (18). Therefore, many methods are used to eliminate the influence of special parts within the screen.

(c) Problems to be Solved by the Invention In the above-mentioned method, in order to perform proper mill balance adjustment for various subjects, it is necessary to determine whether or not predetermined conditions are met based on color information signals including brightness levels. The color information signal selected here is used to adjust the mill balance.

However, when determining whether or not the conditions are satisfied, a small change in the subject has a large effect on the evaluation of the entire screen near the boundary of the conditions, making it difficult to obtain a stable image. On the other hand, subdividing conditions and setting intermediate processing in order to prevent this instability is subject to severe restrictions in terms of system capacity and processing ability.

(d) Means for Solving the Problems The present invention provides a representative value determining means for determining a representative value of the color information signal of the entire imaging screen based on the color information signal in the captured video signal, and a representative value determining means based on the representative value. The mill balance adjustment device has a gain control means for controlling the gain of each color signal by using fuzzy inference in determining the representative value in the representative value determining means. color evaluation value detection means for obtaining color information signal levels for each of a plurality of areas set by dividing the area as a color evaluation value for each color; and weighting the color evaluation value for each area using a weighting amount for each area; A screen color evaluation value calculation means for calculating a screen color evaluation value representative of the entire screen from the weighted color evaluation value, and a gain control means for controlling the amplification gain of each color signal based on the screen color evaluation value, The screen color evaluation value calculation means is characterized in that at least a value based on the color evaluation value of each area and a luminance level of each area are used as input variables, and fuzzy inference is performed using the weighting amount as a conclusion part.

(e) Operation The present invention is configured as described above, and performs inference based on a number of rules set in advance using fuzzy inference without setting detailed conditions corresponding to various conditions. It becomes possible to determine the representative value of screen color information,
Easily and stably adjust the mill balance.

(f) Examples An embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a circuit block diagram of an automatic mill balance adjustment circuit according to this embodiment.

The light that has passed through the lens (1) is imaged on the CCD (2) and subjected to photoelectric conversion, after which it is sent to the color separation circuit (3) into R,
It is extracted as three primary color signals of G and H. The R and B signals among these three primary color signals are processed by R and B amplification circuits (4
)(5), it is input to the camera process and matrix circuit (6) together with the G signal, and based on these, the luminance signal (
Y) and color difference signals of red and blue flames (RY), (B-Y)
is created, supplied to the video circuit (7), and subjected to well-known processing. Further, the signals (Y), (RY), and (B-Y) are simultaneously supplied to the selection circuit (21).

The selection circuit (21) selects the luminance signal (Y) and color difference signal (R) by the selection signal (Sl) from the timing circuit (25).
-Y), (B-Y), one of the three signals is selected sequentially for each field, and (Y) → (R-Y)
→ (B-Y) → (Y) → (R-Y) → ... is outputted to the subsequent A/D converter (22) for each field. Note that the selection signal (Sl) is sent to the synchronous separation circuit (24) as described later.
) is created based on the vertical synchronization signal obtained from
The A/D converter (22) receives the signal (Y) (RY) (selected by the selection circuit (21) at a predetermined sampling period).
B-Y) is sampled and converted into a digital value, and this value is output to the integrator (23). By the way, the timing circuit (25) receives vertical and horizontal synchronization signals from the camera process and matrix circuit (6) and the CCD (2).
Based on the fixed oscillator output that drives
(A12), (A13)...(A88), that is, (A
ij) (i, j = integer of 1 to 8), and a switching signal (S2) for time-divisionally extracting the output of the selection circuit (21) in these regions for each region is sent to the integrator (23). ).

The integrator (23) receives the switching signal (S2) and adds the A/D converted values of the output of the selection circuit (21) for each region over one field period, that is, digitally integrates each of the 64 regions. When the integration for one field is completed, this integrated value is held in the memory (26) as a luminance or color evaluation value. As a result, the digital integral value of the luminance signal (Y) corresponding to 64 areas in an arbitrary field can be obtained as 64 luminance evaluation values (yij) (i, j: 1 to 8). Become. In addition, in the next field, the color difference signal (RY) is selected by the selection circuit (21), so as a result of the integration in each region of the integrator (23), the color difference signal (R-Y) is Digital integral values are obtained as 64 color estimates ffi (r i j). Furthermore, in the next field, the selection circuit (21) selects the color difference signal (
Since B-Y) is selected, as a result of the integration in each region of the integrator (23), the digital integral value for each region of the color difference signal (B-Y) is obtained as 64 color evaluation values (bij). It will be done. In this way, the luminance signal (Y) and the color difference signal (R
-Y) (B-Y), the luminance evaluation value (yij) and color evaluation value (r ij)
The 64×3 value of (b i j) will be held in the memory (26). After this, the same operation as above is repeated, and in the next field, the brightness evaluation value (yij)
However, in the next field, the color evaluation value (r ij) is updated sequentially.

FIG. 11 shows the internal structure of this integrator (23) in more detail. Each A/D converted data is supplied to a switching circuit (61). This switching circuit (61) receives the switching signal (S2) and converts each A/D converted value into an adder (
Fl t) (Fl 2) (F88), it has the role of supplying data to the adder for the area where the sampling point of the corresponding data exists. That is, if a certain arbitrary data sampling point is included in the area (All), this data is supplied to the adder (Fll) for the area (All). Note that, in the following, the adder (Fij) ('+ J
='~8) are set for the area (Aij), and there are 64 in total.
Adders are provided. A holding circuit (Q i j ) is provided at the subsequent stage of each adder, and each added value is temporarily held in each holding circuit. The data held in each holding circuit is input to the adder again and added to the next input data. Further, each holding circuit is reset for each field based on a vertical synchronization signal, and only the data held immediately before this reset is supplied to the memory (26). Therefore, one set of adder and holding circuit constitutes one digital integration circuit, and a total of 64 integration circuits are connected to the integrator (2
3), the digital integral value is stored in memory (2
6).

When the integration for one field is completed, this integrated value is held in the memory (26) as a brightness evaluation value or color evaluation value.

Note that the reference level, that is, the zero level, of the internal difference signal input to the A/D converter (22) is set in advance to the level obtained when photographing a completely achromatic surface, and therefore,
It goes without saying that each A/D conversion value can take not only positive values but also negative values.

The latest evaluation value (yij) (ri
j) (bij) is input to the weighting determination circuit (27). This weighting determination circuit (27) determines the weighting amount of each area based on each evaluation value.

The weighting determination circuit (27) uses so-called fuzzy inference, which treats information with ambiguous boundaries as ambiguous, and uses the following rules for each area.

[Rule 1] andrij− where rifrij+bij is quite close to zero
"bij is close to zero then the weighting amount is large" [Rule 2] andri where rif ri j+bi j is slightly negative
``If j-bij is close to zero, then the weighting amount is somewhat small.'' [Rule 3] If rif y ij is quite large, then the weighting amount is quite small.'' These rules are as shown in Figures 4 to 6. Expressions such as “close” and “large” are expressed as “r1j+bij,
It is defined by membership functions for input variables such as ``1, rr i j-b ij'' and ``yij'' and output variables such as ``weighting amount''.

[Rule 1] is defined by membership functions as shown in FIGS. 4(a), (b), and (c). li diagram (a) is rr
This is to find the membership value of condition (1) of rule (1) that ij + bij is quite close to zero.
Flesh color evaluation value (rij) (bi
It is a membership function for the sum of j), and is a mountain-shaped function that takes the maximum value "1" when the sum of the flesh color evaluation values is zero.In order to satisfy the condition of "considerable", the slope of the mountain is relatively It is set to be steep.

From this function, a membership value (u, . . . ) corresponding to the sum of flesh color evaluation values is determined.

Figure 4(b) is for determining the membership value of condition (2) of rule (1) that ``rrij-bij is close to zero'', and the color evaluation value (r
It is a membership function for the subtracted value obtained by subtracting the color evaluation Vi (bij) from i j), and is a mountain-shaped function that takes the maximum value "l" when the subtracted value is zero. The ship value (u, ,) is found.

Figure 4 (C) shows the rule (the weighting amount is thick) (
It is a membership function for the weighting amount of each region which is the output variable of 2), and it is a function that includes a simple increasing straight line, and the membership value (U, , ) (U
, , ) is determined, this value is specified on the vertical axis ``2'' that indicates the membership value, and the diagonally shaded part obtained by cutting off the head of the membership function is determined by the rule (
This is the inference result obtained by multiplying 1) by 1!6. For example, if the sum of the flesh color evaluation values and the subtracted value of a certain area are the values shown by (△) in FIGS. 4(a) and (b), each membership value becomes u 11 < u l *, U is specified on the vertical axis of FIG.
) corresponds to the amount of weighting depending on the degree to which the equation holds true.

[Rule 2] is defined by membership functions as shown in FIGS. 5(a), (b), and (c). Figure 5(a) shows rr
This is to find the membership value of condition (1) of rule (2) that ij + bij is slightly negative.
), and is a mountain-shaped function that takes the maximum value "1" when the sum of the connotative evaluation values is slightly smaller than zero, and from this function, the membership according to the sum of the connotative evaluation values is The value (utt) is found.

Figure 5(b) is for finding the membership value of condition (2) of rule (2) that ``rrij-bij is close to zero'', and the condition (1) of rule (1) in Figure 4(b) is calculated. 2)
The membership value (U■) corresponding to the subtraction value is determined from this function.

Figure 5 (C) shows the membership function for the weighting amount of each area, which is the output variable of rule (2) that says "the weighting amount is rather small", and the maximum value is "1" when the weighting amount is around 0.25. When the smaller of the membership values (u,,) (utt) of both conditions of rule (2) is determined by the chevron-shaped function, this value is specified on the vertical axis showing the membership value. However, the shaded part obtained by cutting off the head of the membership function is the inference result obtained by comparing the facts to rule (2). For example, the sum and subtraction values of the connotative evaluation values of a certain area are shown in Fig. 5 (a) and (b) (△
), each membership value is ut
t > utt, utt is specified on the vertical axis in Figure 5 (C), the head of the membership function is removed, and the center of gravity of the diagonal line is weighted according to the degree to which rule (2) holds true. This corresponds to the amount.

[Rule 3] is defined by membership functions as shown in FIGS. 6(a) and 6(b). Figure 6(a) shows ry i
It is used to find the membership value of condition (1) of rule (3) that ``j is quite large.'' It is a membership function for the brightness evaluation value (yij) in each area, which is an input variable, and is a monotonically increasing straight line. A membership value (u, , ) corresponding to the luminance evaluation value is determined from this function.

Figure 6(b) shows the membership function for the weighting amount of each area, which is the output variable of rule (3) that says, ``The weighting amount is quite small.'' It is a function that includes a monotonically decreasing straight line, and is a function that satisfies the condition of rule (3). (1) Membership value (u
, , ) on the vertical axis, and the hatched part obtained by cutting off the head of the membership function is the inference result obtained by comparing the facts to rule (3). For example, if the luminance evaluation value of a certain area is the value indicated by (△) in FIG. 6(a), the head of the membership function is cut off as shown in FIG. 6(b), and the shaded area is The center of gravity corresponds to the weighting amount depending on the degree of fulfillment of rule (3).

Next, a method for determining the weighting amount of a corresponding area after considering all three rules will be explained. The hatched rectangles in Figures 4(C), 5(C), and 6(b) have the same coordinates as in each of these figures, that is, the weighting amount is on the horizontal axis and the membership value is on the vertical axis. The function in Figure 7 obtained by superimposing the obtained coordinates becomes the membership function that indicates the final inference result, and the position of the center of gravity of this function is 3.
This is the weighting amount (wij) of the corresponding area after considering all the conditions of the rule. Furthermore, instead of determining the weighting amount based on the position of the center of gravity as described above, it is also possible to set the weighting amount to a position on the horizontal axis that halves the area of the area surrounded by the membership function on the left and right. Needless to say.

Next, the influence of each rule on the white balance adjustment operation will be explained, but before that, let's consider the relationship between the color temperature change of the light source and each color difference signal. Generally, the color difference signal when the color temperature of the light source illuminating a white subject changes is
It changes like the light source color temperature axis in FIG. Therefore, the color difference signals appearing in the first and third quadrants that do not include the light source color temperature axis do not reflect the color temperature of the light source, and are preferably not considered as information when performing mill balance adjustment.

By the way, the color evaluation value (bi
j), and the color evaluation value (rij) of (RY) is plotted on the vertical axis (Y axis).
) and approximate the light source color temperature axis, it can be expressed as a straight line with a slope of -1 (rij=-bij as shown in the formula), as shown in (Ll) in FIGS. 9 and 10.

Therefore, in [Rule 1], regarding screen evaluation during mill balance adjustment, if the color evaluation value of the relevant area deviates from the origin, the component of the deviation in the direction perpendicular to the light source color temperature axis (Ll) is Even if the amount of deviation is small,
The weighting amount of the corresponding area is reduced, and at the same time, the degree of reduction of the weighting amount with respect to the deviation amount is adjusted compared to the orthogonal direction so that the component in the direction of the light source color temperature axis (Ll) is allowed even if the deviation amount is somewhat large. It is intended to be gradual.

In other words, if condition (1) of [Rule 1] is expressed as a formula, [margin] (k) Kaisho ↓ City J, and Kl (V intercept) is set to a value quite close to zero ± a (a
: positive value), the degree to which condition (1) is met (membership value) occurs on the straight lines (L2) (L3) in Figure 9.
In the range surrounded by , the probability of establishment increases as it approaches l1a (Ll). That is, as shown in FIG. 4(a), the weighting amount is set so as to increase rapidly only when a approaches zero, that is, when the straight lines (L2) and (L3) approach the origin considerably.

Furthermore, when condition (2) of [Rule 1] is expressed as a formula, (K21
' is 1 叡, = ゝ', and K2 (y-intercept) is a value close to zero, c (c: positive value), then equation (2) can be calculated directly in the range surrounded by a (L4) (L5). Moreover, the closer you get to the origin, the higher the probability of success. That is, C approaches zero, that is, straight line (L4) (L5)
The weighting amount is set to gradually increase as it approaches the origin, as shown in FIG. 4(b). Furthermore, the fourth
In figure (b), it takes extremely C for the membership value to become zero.
is the time when In this way, if the color evaluation value of the corresponding area is at the origin, the weighting amount is naturally maximized, and even if the color evaluation value deviates from the origin, the deviation is in the direction of the red or blue color temperature axis, and the light source color When the weighting amount is small in the direction perpendicular to the temperature axis (Ll) (in the direction of extremely high saturation), the weighting amount is increased to increase the degree of contribution to the screen evaluation.

Also, if we express the condition (1) of [Rule 2] as a formula, we get (,, iΔ2 soldiers 3°, and if K3 is a slightly negative value - d (d: positive value), then the members of condition (1) The ship value occurs in the 10th
As shown in the figure, the range is directly surrounded by at(Ll) and (L6).

Furthermore, since condition (2) is the same as condition (2) of [Rule 1], the range is surrounded by straight lines (L4) and (L5).

Therefore, when -d is the top of the mountain in Figure 5(a), ri
The weighting amount is increased as the value of j+brj approaches, and as C approaches zero, that is, the straight lines (L4) and (L5) approach the origin, the weighting amount is gradually increased as shown in FIG. 5(b). As a result, the color of the corresponding area becomes the 10th
If it deviates slightly toward the edge of the figure (the straight line indicated by rij-bij=o in the third quadrant), it is likely a greenish light source such as a fluorescent lamp, although it is off the color temperature axis. The amount of weighting is made to remain slightly.

[Rule 3] is when the brightness of an area is extremely high, and the saturation of the image sensor shifts the color balance, biasing it towards a specific color, so the weighting amount is reduced to reduce the contribution of this area. .

In the weighting determination circuit (27), when each evaluation value is input, inference is performed according to the above-mentioned rules, and the weighting amount (wi]) of each area is determined in the range of O to 1.

On the other hand, color evaluation value (rij) (bij) (i, j: 1-8
) is input to the screen evaluation circuit (28), and using the weighting amount (wij) obtained by the weighting determination circuit (27), the color difference signal (R-Y) is calculated based on the following equations (1) and (2). (B
-Y) for each entire screen are calculated as screen color evaluation values (Vr) (Vb).

The gain control circuits (29) and (30) control each of the R and B amplifier circuits (4) and (5) so that the screen color evaluation values (Vr) (Vb), which are the color evaluation values of the entire screen, are both zero. Gain is controlled. When the screen color evaluation values (Vr) (Vb) become zero in this way, it means that the white balance adjustment is completed.

As mentioned above, fuzzy inference is performed based on several preset rules to determine the representative value of screen color information, so there is no need to make detailed condition settings corresponding to various conditions, and the result is stable. control can be achieved.

Note that in this embodiment, the A/D converter (22) and the integrator (
23) as a luminance signal (Y), a color difference signal (RY) (B-Y
) is used to digitally integrate and extract the levels of the three signals for each region, and the integrated value of each signal could only be updated in three field cycles, but the A/D converter and integrator It goes without saying that if the signal level is provided exclusively for each signal, each signal level can be updated every l field.

Furthermore, a weighting determination circuit (27) and a screen evaluation circuit (
The operation 28) can be processed by software using a microcomputer.

(g) Effects of the Invention As described above, according to the present invention, fuzzy inference is performed based on several preset rules to determine the representative value of screen color information.

This makes it possible to achieve stable white balance adjustment without having to make detailed condition settings corresponding to various conditions.

[Brief explanation of drawings]

1, 3 to 11 relate to an embodiment of the present invention, in which FIG. 1 is an overall circuit block diagram, FIG. 3 is an explanatory diagram of screen division, and FIG. 4 is a membership diagram of rule 1. A diagram showing the function, Figure 5 is a diagram showing the membership function of rule 2,
Figure 6 is a diagram showing the membership function of rule 3, Figure 7
Figure 8 shows the membership function of the final inference result.
The figure is a diagram for explaining the light source color temperature axis, FIGS. 9 and 10 are diagrams for explaining Rule 1.2, and FIG. 11 is a block diagram of the main circuit. Further, FIG. 2 is a circuit block diagram of a conventional example. (27)...Weighting determination circuit, (28)...Screen evaluation circuit, (4)(5)...R and B amplification circuit.

Claims (3)

[Claims] (1) representative value determining means for determining a representative value of the color information signal of the entire imaging screen based on the color information signal in the captured video signal;
A white balance adjustment device having a gain control means for controlling the gain of each color signal based on the representative value, characterized in that the representative value determining means uses fuzzy inference to determine the representative value. Adjustment device. (2) color evaluation value detection means for obtaining color information signal levels for each of a plurality of areas set by dividing the imaging screen as color evaluation values for each color; and weighting amounts for each area on the color evaluation values for each area. a screen color evaluation value calculating means for calculating a screen color evaluation value representative of the entire screen from the weighted color evaluation value; and controlling the amplification gain of each color information signal based on the screen color evaluation value. The screen color evaluation value calculation means uses at least a value based on the color evaluation value of each region as an input variable, and performs fuzzy inference using the weighting amount as a conclusion part. White balance adjustment device. (3) The white balance adjustment device according to item 2, characterized in that a luminance signal level for each of a plurality of regions is used as an input variable for the fuzzy inference.