JP2001160976A - Image processing method, image processor and computer- readable recording medium with image processing program recorded thereon - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent white balance correction to an object in a chromatic color from all the more causing unnatural color correction and to correctly control color tone by preventing an RB gain from being locked, even when the color of a light source is changed rapidly or an object is changed to cause a rapid changes in the color of a reflected light. SOLUTION: An RB gain denoted by a point M on a cross line between an area consisting of points within a prescribed distance from a straight line 'ON' and a border line closest to a point N among border lines within the RB gain movable range is used for a new RB gain.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method for controlling white balance of a color image, an image processing apparatus, and a computer-readable recording medium storing an image processing program.

[0002]

2. Description of the Related Art In order to obtain high satisfaction with an image captured by a color camera or an image scanner using a solid-state image sensor such as a CCD or C-MOS imager as an image input device, the brightness, contrast, It is important that the color reproduction is proper. Among these, with regard to color reproduction, it is fundamental to reproduce white portions included in a subject illuminated by sunlight in white. Sunlight is a red light source in the morning or dusk, a white light source in the sunny daytime, a blue light source in the shade, time and season,
It can be a light source of various colors by direct, indirect, etc.

[0003] In this light source color change, image correction for reproducing a white object white is white balance control. In the case of a normal color camera, white balance correction is performed with a configuration as shown in FIG. Reference numeral 101 denotes a solid-state imaging device, which photoelectrically converts image information of a subject. The analog data photoelectrically converted by the A / D converter 102 is converted into digital data, and the output signal of a portion that blocks light provided on the photoelectric conversion surface of the image sensor 101 by the OB offset clamp 103 becomes zero. Clamped.

An R amplifier 104 and a B amplifier 105 are amplifiers for output signals of the R and B pixels, respectively, and adjust the gains (R gain and B gain) of these amplifiers so that the R and G of the white portion of the subject can be adjusted. , B are controlled so that the output values of the respective pixels become equal. The white pixel integration unit 106 calculates how many color components are included in a white portion in a digital image signal obtained by imaging a subject when white balance is not achieved. There are various methods for extracting a white portion included in a subject. For example, in Japanese Patent Application Laid-Open No. H11-205812,
A method of dividing a color difference domain into several meshes has been proposed. The next frame RB gain calculation unit 107 performs RB calculation based on the color components obtained by the white pixel integration unit 106.
Calculate the gain correction amount, add it to the current RB gain,
The RB gain of the next frame is calculated.

Usually, a time constant is multiplied by the RB gain, and the RB gain is calculated so that the white portion of the subject smoothly changes from the color of the light source to white. Movable range determination unit 10
In step 8, it is determined whether the RB gain calculated by the next frame RB gain calculator 107 is within a certain range defined on the RB plane. This is because if the color component of the subject is a color that is not originally included in the change in sunlight, the white balance correction is performed on the chromatic subject to prevent unnatural color correction. This is because the RB gain is only slightly changed.

That is, when the RB gain movable range is not particularly provided, for example, when the subject has a greenish tone, (1) the cause of the subject having a greenish tone is caused by the greenish tone. If it was due to the light source,
The subject is subjected to white balance control and becomes gray. On the other hand, on the other hand, (2) if the subject becomes greenish in this way because the subject is a green object, if the subject is a white (accurately gray) pixel If the subject is incorrectly determined to be white balance control of the subject is not properly performed, and as a result of being controlled as if a green subject is being photographed, the green subject is originally gray and the background is, for example, gray. The image becomes magenta, which results in a very unnatural image. As another example, if white balance control is performed so that a purple object becomes gray, the background becomes green.

The above-mentioned conventional white balance control is performed based on a YIQ cube R shown in FIG. 2 for explaining the present invention, and the YIQ cube R is usually slightly larger in order to allow a margin for control. Is set to
When the YIQ cube R is made larger in this way, it is possible to prevent the white balance control from being stopped when the light source color changes significantly as described above. However, colors not included in the change in sunlight, such as green and purple, are grayed by white balance control, and the color tone becomes unnatural. Therefore, an RB gain movable range that is an allowable movable range is provided in the RB gain resulting from the white balance control calculation.

When the RB gain is out of the movable range, the RB gain is only slightly changed within the RB gain movable range. Accordingly, when the color component of the subject is a color that is not included in sunlight, such as green or purple, the color tone is limited to a slight change to such an extent that the color becomes slightly lighter.

Further, if the green and purple colors are made closer to gray, the RB gain is out of the movable range, so that the RB gain is not changed.
Beyond that, this green or purple thing doesn't look gray. That is, the chromatic colors are output as chromatic colors.

The processing procedure of the RB gain determination portion will be described with reference to FIG. In steps 101 (hereinafter, steps are simply described as S) to S103,
The RB gain to be set next is calculated from the color components of the white pixels. In S104, it is determined whether or not the RB gain is within the movable range. If the RB gain is within the movable range, the calculated RB gain is set as it is in the RB gain amplifier, and if not, the RB gain is not set. That is, the current RB
The gain is also applied as the RB gain of the next frame.

[0011]

As described above, it is determined whether or not the RB gain is within the movable range as in the above-described prior art, and the setting of the gain is restricted to correct the white balance of the chromatic object. , It is possible to prevent an unnatural color correction.

However, in the RB gain limiting method according to the prior art, a problem occurs when the color of the light source changes abruptly, or when the subject changes and the color of the reflected light changes abruptly.

For example, an outdoor shade portion is photographed from the indoor through a window (the shade is a blue light source). As a result of white balance correction, the R gain is large and the B gain is small. In this state, when the camera is pointed at a subject suddenly illuminated by an indoor incandescent lamp (an incandescent lamp is a red light source), correction is performed so that the R gain is small and the B gain is large. At this time, due to the imbalance of the RB gain correction amount, the coordinates (B gain, R gain) are outside the RB movable range, and the RB gain does not change according to the flow shown in FIG. Since the RB gain does not change, the color component of the image captured from the camera does not change even in the next frame.
As a result, the same RB gain is calculated, but the RB gain does not change because it is outside the RB movable range. Hereinafter, this is repeated, and the RB gain is locked (locked out).
It is the same as what was done.

For example, in a situation where a blue light source is photographed as a large image such as a CRT of a television, the white balance control works in a direction to make the screen red. As a result, gains of large R gain and small B gain are calculated and set in the amplifier. Here, it is assumed that the camera is suddenly pointed at a white object illuminated by an incandescent lamp, and a red object is suddenly photographed temporally.
Then, the R gain is large and the B gain is small at the moment because the blue object was photographed last time, but since the subject has rapidly changed to a red object in time, the device that performs white balance control is: It is determined that it is necessary to make a large correction, and a fairly small R gain and a fairly large B gain are calculated as values that make the color fairly blue. If these values exceed the RB gain movable range,
Based on the determination that the RB gain has been exceeded, the previous RB gain is set as it is. That is, to return the red state to white,
Although it is necessary to set the gain to be small and the B gain to be large, the gain of large R gain and small B gain when a blue object is photographed remains set. As a result, the screen remains unchanged with a red tone. That is, the RB gain is locked.

That is, the reason that the RB gain exceeds the movable range is not only when a color other than sunlight such as green or purple is to be grayed (this needs to be prevented), but also in other cases, There are times when it is desired to immediately turn gray that is included in sunlight but changed rapidly, such as from blue to red or vice versa (this is desirable), so the RB gain movable range RB just exceeded
If the new setting of the gain is stopped, appropriate white balance control is not performed in the latter case, and a normal color tone is not obtained.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and has as its object to perform white balance correction on a chromatic subject, thereby preventing unnatural color correction from occurring. An image processing method, an image processing apparatus, and an image processing method capable of correctly controlling a color tone without locking the RB gain even when the color changes rapidly or the subject changes and the color of the reflected light changes rapidly. An object of the present invention is to provide a computer-readable recording medium on which a program is recorded.

[0017]

In order to solve the above-mentioned problems, an image processing method according to the present invention considers a coordinate plane in which the origin is set to O, the R gain and the B gain are set to coordinate axes, and a light source of sunlight is used. The trajectory of the RB gain necessary for obtaining the white balance according to the color is represented by a curve W, and a region representing the RB gain movable range for preventing chromatic colors from being drawn is defined by a boundary line based on the curve W. , The point representing the current RB gain is designated as point C, the point representing the RB gain obtained from the color component of the target pixel such that the color component is zero is designated as point N, and the pixel white In the image processing method for setting the RB gain for the process of adjusting the balance, when the point N is outside the RB gain movable range, the point R on the boundary line represents
It is characterized in that the B gain is set as a new RB gain.

With the above configuration, the RB gain represented by the point on the boundary line is set as a new RB gain. That is, the RB gain represented by a point on the boundary line that is the outer edge of the RB gain movable range is set as a new RB gain. Therefore, each value of the calculated RB gain remains within the RB gain movable range to the last, and does not go out of the range. Therefore, the control of the RB gain is not locked, and the white balance control is continued. Thereafter, the white balance is controlled as usual, and the color tone is correctly adjusted.

As described above, even when the color of the light source changes abruptly, or when the subject changes and the color of the reflected light changes abruptly, the lock of the RB gain can be prevented. In other words, by performing white balance correction on a chromatic subject, it is possible to prevent unnatural color correction from occurring, and when the light source color changes abruptly or when the subject changes, the reflected light color changes. Even in the case of a sudden change, the RB gain does not lock and the color tone can be correctly controlled.

According to the image processing method of the present invention, in addition to the above configuration, the area representing the RB gain movable range is formed by one or more line segments as boundary lines.
Is located outside the RB gain movable range, a point on the intersection line between the boundary line closest to the point N among the boundary lines and the region S including points within a predetermined distance from the straight line ON is determined. The RB gain to be represented is set as a new RB gain.

With the above configuration, the RB gain represented by the point on the intersection of the region S consisting of points within a predetermined distance from the straight line ON and the boundary line closest to the point N among the boundary lines is newly set. RB gain. Therefore, the calculated point N representing the RB gain to be originally set and the point representing the RB gain to be actually set are close to each other. That is, the R gain and the B gain are close to each other. Then, as compared with the movement from the point C to the point N, the increasing / decreasing direction of each gain is the same, and the white balance control direction such as the direction of adding bluish and reducing reddish or the direction of adding reddish and reducing bluish is increased. Do not deviate.
Therefore, in addition to the effect of the above configuration, when the color of the light source changes abruptly, or when the subject changes and the color of the reflected light changes abruptly, the color of the subject changes to an extremely different color. Can be suppressed.

Further, in addition to the above configuration, the image processing method according to the present invention further comprises a boundary line closest to the point N among the boundary lines and a straight line ON having the predetermined distance of 0. The RB gain represented by the point M is set as a new RB gain.

With the above configuration, the RB gain represented by the point M on the intersection of the straight line ON having the predetermined distance of 0 and the boundary line closest to the point N among the boundary lines is calculated by a new RB gain.
Set as gain. Therefore, the calculated point N representing the RB gain to be set and the point N actually set
The points representing the B gain are close to each other. That is, R
The gain and the B gain are close to each other. Then, as compared with the movement from the point C to the point N, the increasing / decreasing direction of each gain is the same, and deviates from the white balance control direction such as the direction of adding bluish to reduce reddish or the direction of adding reddish to reduce bluish. do not do. Therefore, in addition to the effect of the above configuration, when the color of the light source changes abruptly, or when the subject changes and the color of the reflected light changes abruptly, the color of the subject changes to an extremely different color. Can be suppressed more effectively.

Further, in the image processing method of the present invention, in the above configuration, the area representing the RB gain movable range is formed by one or more line segments as boundary lines.
Is located outside the RB gain movable range, a point on the intersection line between the boundary line closest to the point N among the boundary lines and the region T including points within a predetermined distance from the straight line CN is determined. The RB gain to be represented is set as a new RB gain.

With the above configuration, the RB gain represented by the point on the intersection of the region T consisting of points within a predetermined distance from the straight line CN and the boundary line closest to the point N among the boundary lines is newly set. RB gain. Therefore, the calculated point N representing the RB gain to be originally set and the point representing the RB gain to be actually set are close to each other. That is, the R gain and the B gain are close to each other. Then, as compared with the movement from the point C to the point N, the increasing / decreasing direction of each gain is the same, and the white balance control direction such as the direction of adding bluish and reducing reddish or the direction of adding reddish and reducing bluish is increased. Do not deviate.
Therefore, in addition to the effect of the above configuration, when the color of the light source changes abruptly or when the subject changes and the color of the reflected light changes abruptly, the color of the subject changes to an extremely different color. Can be suppressed.

Further, in addition to the above configuration, the image processing method according to the present invention further comprises an intersection between the boundary line closest to the point N among the boundary lines and the straight line CN having the predetermined distance of 0. The RB gain represented by the point P is set as a new RB gain.

According to the above configuration, the RB gain represented by the point P on the intersection of the straight line CN having the predetermined distance of 0 and the boundary line closest to the point N among the boundary lines is calculated by a new RB gain.
Set as gain. Therefore, the calculated point N representing the RB gain to be set and the point N actually set
The points representing the B gain are close to each other. That is, R
The gain and the B gain are close to each other. Then, as compared with the movement from the point C to the point N, the increasing / decreasing direction of each gain is the same, and deviates from the white balance control direction such as the direction of adding bluish to reduce reddish or the direction of adding reddish to reduce bluish. do not do. Therefore, in addition to the effect of the above configuration, when the color of the light source changes abruptly or when the subject changes and the color of the reflected light changes abruptly, the color of the subject changes to an extremely different color. Can be suppressed more effectively.

The image processing method according to the present invention, in addition to the above configuration, further comprises, when the point N is located outside the RB gain movable range, a point representing a newly set RB gain as a point A. Then, in the above-described boundary line, a portion on the same side as the point N with respect to the curve W, and the signs of the horizontal axis components and the vertical axis components of the vector CN and the vector CA are equal to each other. The RB gain represented by the point A on the portion is set as a new RB gain.

According to the above configuration, within the boundary line,
An RB gain represented by a point A on a portion on the same side as the point N with respect to the curve W and having the same sign between the horizontal axis components and the vertical axis components of the vector CN and the vector CA. Is set as a new RB gain. Therefore, the calculated point N representing the RB gain to be originally set and the point representing the RB gain to be actually set are close to each other. That is, the R gain and the B gain are close to each other. And point C
As compared to the movement from the point to the point N, the direction of increase / decrease of each gain is the same, and the control direction of the white balance such as the direction of adding bluish and reducing reddish or the direction of adding reddish and reducing bluish can greatly deviate. Absent. Therefore, in addition to the effect of the above configuration, when the color of the light source changes abruptly or when the subject changes and the color of the reflected light changes abruptly, the color of the subject changes to an extremely different color. Can be suppressed.

Further, in the image processing method according to the present invention, in the above configuration, the point A is represented by R
It is characterized in that the B gain is set as a new RB gain.

With the above configuration, as the point A, an RB gain represented by a point P on an intersection of the boundary line closest to the point N and the straight line CN among the boundary lines is set as a new RB gain. . Therefore, the calculated point N representing the RB gain to be originally set and the point representing the RB gain to be actually set are close to each other. That is, the R gain and the B gain are close to each other. Then, as compared with the movement from the point C to the point N, the increasing / decreasing direction of each gain is the same, and deviates from the white balance control direction such as the direction of adding bluish to reduce reddish or the direction of adding reddish to reduce bluish. do not do. Therefore, in addition to the effect of the above configuration, when the color of the light source changes abruptly or when the subject changes and the color of the reflected light changes abruptly, the color of the subject changes to an extremely different color. Can be suppressed more effectively.

Further, in order to solve the above-mentioned problems, the image processing apparatus of the present invention considers a coordinate plane in which the origin is set to O, and the R gain and the B gain are respectively set on the coordinate axes, and is set according to the light source color of sunlight. The trajectory of the RB gain necessary for obtaining the white balance is defined as a curve W, and a region representing the RB gain movable range for preventing chromatic colors from being drawn is formed by a boundary line provided based on the curve W. , The current R
A point representing a B gain is defined as a point C, a point representing an RB gain obtained from a color component of a target pixel and having a color component of zero is defined as a point N, and processing for adjusting the white balance of the pixel. An RB gain setting unit that sets an RB gain represented by a point on the boundary line as a new RB gain when the point N is outside the RB gain movable range. It is characterized by having.

The image processing apparatus includes, for example, R gain amplifying means for amplifying the output signal of the R pixel, B gain amplifying means for amplifying the output signal of the B pixel, and a color component of the white pixel. RB gain calculating means for calculating an RB gain such that the RB gain becomes zero, and a movable range determining means for determining whether the RB gain is within the RB gain movable range.

Further, the image processing apparatus includes, for example, an image pickup means for taking in information of a subject, a white pixel extraction means for extracting white pixels from R, G, and B pixels, and a color component of white pixels. Color component calculation means can be provided.

In order to solve the above-mentioned problem, a computer-readable recording medium on which an image processing program of the present invention is recorded is considered as a coordinate plane in which the origin is O and the R gain and the B gain are each coordinate axes. The trajectory of the RB gain necessary for obtaining the white balance according to the light source color of sunlight is defined as a curve W, and a region representing the RB gain movable range for preventing chromatic colors from being drawn is defined based on the curve W. The current R
A point representing a B gain is defined as a point C, a point representing an RB gain obtained from a color component of a target pixel and having a color component of zero is defined as a point N, and processing for adjusting the white balance of the pixel. In a computer-readable recording medium on which an image processing program for setting an RB gain is set, when the point N is outside the RB gain movable range, the RB gain represented by the point on the boundary line is set to a new RB. It is characterized in that it is set as a gain.

[0036]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [Embodiment 1] An embodiment of the present invention will be described with reference to FIGS.
It is as follows.

As shown in FIG. 1, a video signal correction device as an image processing device includes a solid-state image pickup device 1, an A / D converter 2, an OB offset clamp 3, an R amplifier 4, a B amplifier 5, a white pixel integrator 6, RB gain calculating section 7, movable range determining section 8, RB gain recalculating section 9 outside the movable range, and post-processing section 10. The white pixel integrator 6, the next frame RB gain calculator 7, the movable range determination unit 8, and the RB gain recalculator 9 outside the movable range constitute an RB gain setting unit.

The solid-state image pickup device 1 is an image pickup means for taking in information of a subject, and photoelectrically converts image information of the subject.
The A / D converter 2 converts the photoelectrically converted analog image electrical information into digital data, and the OB offset clamp 3 reduces the output signal of the portion of the solid-state imaging device 1 where light is blocked to zero. It is clamped to become.

The R amplifier 4 and the B amplifier 5 are respectively
A color signal amplifier (R gain amplifying means, B gain amplifying means) for amplifying output signals of the R pixel and the B pixel and outputting the amplified signal to the post-processing unit 10. Note that the R amplifier 4 and the B amplifier 5 are generically called an RB amplifier.

In this embodiment, the white balance control is performed by adjusting the gains (R gain and B gain) of these color signal amplifiers. That is, RB is set so that the output values of the R, G, and B pixels of the white portion included in the subject are equal.
By controlling the gain of the amplifier, control is performed so that the white portion of the subject is reproduced white.

The white pixel accumulator 6 includes an R pixel, a G pixel,
White pixel extraction means for extracting white pixels from B pixels, and color component calculation means for calculating the color components of the white pixels. A white portion in a digital image signal obtained by imaging a subject when white balance is not achieved. Calculate how many color components are included in. The white portion of the subject reflects the sunlight with various color changes from red to blue, and shows a change from a reddish color to a bluish color like the light source color.

Next frame RB as RB gain calculating means
The gain calculator 7 calculates the color component from the RGB values of the white portion. That is, the RB gain is calculated from the color components of the white pixels so that the color components become zero.

Next, the mechanism of white pixel extraction will be described with reference to FIG. Using the RGB values (R, G, B) of each pixel,
The color difference components I and Q and the luminance value Y of each pixel are calculated by the equation shown in the equation (1). That is, Y = 0.3R + 0.59G + 0.11B I = 0.6R−0.28G−0.32B Q = 0.21R−0.52G + 0.31B (1)

Next, Y, I, Q obtained by the equation (1)
For each value of, the RGB values of the pixels that simultaneously satisfy the condition of Expression (2) are separately integrated (added), and divided by the total number of pixels that satisfy the condition of Expression (2) to obtain an image of one frame. Color evaluation value (RGB evaluation value, RGB average value) r,
Find g and b. That is, Imin ≦ I ≦ Imax Qmin ≦ Q ≦ Qmax Ymin ≦ Y ≦ Ymax (2)

This means that, in FIG. 2, the RGB average value of only the pixels inside the YIQ cube R is obtained. Usually, the white portion of the subject reflects the light source color almost as it is. Therefore, four straight lines I = Ima
Consider a rectangle formed by x, Q = Qmax, I = Imin, Q = Qmin, and Imax, Qmax, Imi such that the distribution of the color of the sun light source is completely included in the rectangle.
By selecting n and Qmin, the RGB values of only the white pixels can be integrated, and the RGB average value of the frame can be obtained.

Here, the size of the YIQ cube R, that is, the above four boundary values (Imax, Qmax, Imin, Qm
in) and the RB gain movable range will be described.

That is, only pixels that are inside the YIQ cube R as shown in FIG. 2 are to be calculated, so that dark pixels can be prevented from being extracted as white pixels.

However, if the YIQ cube R is too small, white pixels cannot be extracted when the light source changes suddenly, and the white balance control stops there. For this reason, the YIQ cube R is set to be slightly larger so that even pixels having a certain degree of color are calculated as white pixels, more precisely, gray pixels. For example, in a situation where a blue light source such as a CRT (cathode ray tube) of a television is photographed in a close-up, and the camera is suddenly pointed at a white object illuminated by an incandescent lamp, before turning the camera, Since the white balance is controlled so as to make the blue light source of the CRT white, the parts other than the CRT have a complementary color of orange. When the camera is pointed at a white object illuminated by an incandescent lamp, which is an orange light source, as described above, a considerably red object appears on the camera. Originally, this is a white object, so we had to control it to make it white, but in this case, because the subject was too red,
This orange color is completely off the YIQ cube R,
As a result of the white pixel extraction, zero pixels pass through the YIQ cube R, and the white balance control stops. To prevent this, the YIQ cube R is set slightly larger. When the YIQ cube R is made large in this way, it is possible to prevent the white balance control from being stopped when the light source color changes greatly as described above.

However, as a result, green or purple
If the color not included in the change in sunlight is changed to gray by the white balance control, the color tone becomes unnatural. Therefore, an allowable movable range (RB gain movable range) is provided in the RB gain of the calculation result of the white balance control, and it is checked whether the RB gain of the calculation result is within this range.

Since the purpose of selecting the specified values Ymax and Ymin is to extract white pixels, ideally, a chart of seven colors is illuminated with a constant illuminance, and the yellow color is obtained in a shooting state. This is a value higher than the brightness and up to the upper limit. However, in reality, there is no state in which the chart of seven colors is illuminated with a constant illuminance. Therefore,
The setting is performed according to the sensor and the like to be used at that time by excluding a portion having a very low luminance.

Next, the next frame RB gain calculator 7 calculates the RB gain of the next frame based on the color evaluation value obtained by the white pixel accumulator 6. Before the white balance correction, the white portion of the subject is colored by the light source color, so that the color evaluation values r, g, and b are not equal. After white balance correction, the white portion should be white, so that r = g = b. Therefore, the respective coefficients of the RB pixels set in the R amplifier 4 and the B amplifier 5 necessary for obtaining the white balance are as follows: Gr = g / r Gb where Gr is the coefficient of the R pixel and Gb is the coefficient of the B pixel. = G / b (3)

Normally, the RB obtained by the equation (3) is corrected so that white is corrected smoothly over several frames.
The gain is set in the RB amplifier at a fixed rate. The proportion is d
(%), The RB gain of the next frame is
When the B gains are cGr and cGb, respectively, Gr = (g / r−cGr) × d / 100 + cGr Gb = (g / b−cGb) × d / 100 + cGb (4)

The movable range determining section 8 constitutes a movable range determining means for determining whether or not the RB gain is within the RB gain movable range. The movable range determination section 8 determines whether or not the Gr and Gb are within the RB gain movable range that is set so as not to draw the chromatic object color white. In this embodiment, as shown by hatching in FIG. 3, the region (RB gain) that can be set as an RB gain is set on the next six straight lines as a boundary line and inside a portion surrounded by the six straight lines. Gain movable range). That is, Gr = Grmin Gr = Grmax Gr = p × Gb + q Gb = Gbmin Gb = Gbmax Gr = p × Gb + q ′ (5)

In FIG. 3, a curve W at the center is an RB gain set for obtaining a white balance when the color of the light source of the sun light changes variously depending on the lighting conditions such as season, time, shade, and sunshine. Shows the trajectory of
The RB gain movable range is also set such that each boundary line includes the curve W and has a certain width (margin) so that white balance control is performed when sunlight is a light source. The curve W is a finite curve having both ends corresponding to the absence of the color not included in the sunlight. The curve W is precisely based on the blackbody radiation locus. That is, the black body radiation locus is a locus of a color generated when a black body (complete radiator) is heated, and changes from red to orange, yellow, and white. This color is represented by the temperature (color temperature) of a black body when a certain color is generated.

The limit value between Gr and Gb will be described. As described above, the red color has a low color temperature, and the blue color has a high color temperature. Therefore, when designing a camera, consider how much white light can be controlled from red light to blue light to a certain degree.
From 200K, the blue one is decided to 9100K. R corresponding to these 3200K and 9100K
B gains are (Grmin, Gbmax),
(Grmax, Gbmin). Thus, G
Limit values of r and Gb are set respectively. Note that these limit values can be extended with some margin.

In the present embodiment, the curve W is regarded as a substantially straight line and the RB gain movable range is set by the straight line shown in the equation (5). However, the boundary of the RB gain movable range does not need to be a straight line. The curve W is moved up and down as it is, and the upper and lower two curves and four straight lines Gr = Grmi
n, Gr = Grmax, Gb = Gbmin, Gb = Gb
It may be a range surrounded by max. Alternatively, the curve W
May be approximated by several broken lines.

The RB gain recalculation unit 9 outside the movable range recalculates the RB gain as described later when Gr and Gb are outside the RB gain movable range in the movable range determination unit 8.

The operation of the movable range determination section 8 will be described with reference to FIG. Point C is the current RB gain. C indicates that the current light source is blue light because the B gain is low and the R gain is high. For example, there is a case where a subject in the shade is photographed under fine weather, a case where the sun is hidden by clouds, and only the blue sky is a light source. Now, assuming the following photographic scene, a change in the RB gain at that time is considered.

It is assumed that an image of an outdoor shaded area is taken from a room illuminated by an incandescent lamp through a window. In this case, the coordinates (Gb, Gr) of the RB gain are at a position like point C in a state where the white balance is maintained. At this time, it is assumed that the camera is suddenly turned to the room. Then, the light incident on the solid-state imaging device 1 turns into red light, and the R gain is reduced to obtain white balance, and
Increase B gain. However, depending on the light source in the room and the color of the object reflected in the camera, the change in the RB gain is
The RB gain moves out of the RB gain movable range shown by oblique lines in FIG. In other words, if the white balance is to be completely controlled immediately when the light source suddenly changes from blue to red, a value out of the RB gain movable range, such as the point N, is calculated.

As described above, in addition to the case where the light source suddenly changes from blue to red and is affected by the chromatic color, the reflection by the light source having a color tone such as green or purple, which greatly deviates from the blackbody radiation locus. When white balance control is performed on light,
Also, when white balance control is performed on reflected light of an object (green or purple paper, clothes, or the like) of a color that deviates greatly from the blackbody radiation locus, such as green or purple, as the subject. The position is like point N.

Conventionally, the calculated RB gain is R
Since the control is stopped when the B gain is out of the movable range, as described above, if the color tone of the subject changes rapidly, the RB gain does not change in the future, and the white balance control is locked. I will.

On the other hand, in the present embodiment, the RB gain movable range is determined.
Even when the gain deviates from the RB gain movable range, the white balance control is performed up to a point such as a point M in FIG. 4 as described later. That is, when the value obtained by the calculation falls outside the RB gain movable range, the RB gain recalculating unit 9 outside the movable range performs control so as to set the RB gain indicated by a point on the boundary of the RB gain movable range. ing. As a result, no lockout occurs, and the red screen gradually returns to white.

Further, in the present embodiment, the calculated R
When the B gain deviates from the RB gain movable range as described above, a point (coordinate) indicating the RB gain, that is, in the example of the above-described rapid change from the CRT to the incandescent lamp, the red screen is returned to white. A region including a straight line connecting a point N indicating a very small R gain and a very large B gain for the purpose and an origin O is considered. Then, an RB gain indicated by a point on the intersection of this region and the boundary line of the RB gain movable range, for example, an intersection (point M) between the straight line ON and the boundary line is set. The white balance control is performed in the direction of decreasing the gain and increasing the B gain. Therefore, it is possible to smoothly return to white.

Note that when the subject itself is regarded as a green pixel as a white (accurately, gray) pixel, the RB gain on the boundary line is set as described above, but the white balance control is continued. Therefore, even in this case, the image gradually changes to an image having a correct color tone.

Next, details of the RB gain setting process will be described. The point N is outside the RB gain movable range originally provided to prevent the drawing in of a chromatic color, and in the present embodiment, the RB gain of the point N cannot be set for the purpose. In the present embodiment, the origin O and the point N
Is set as a predetermined range (hereinafter, referred to as S), which includes points within a predetermined distance from a straight line ON formed by connecting. That is, as shown in FIG.
It is shifted by ΔB in the positive direction on the horizontal axis (Gb). Let J be the intersection of the resulting straight line and the straight line Gr = p × Gb + q which is the boundary line closest to the point N among all the boundary lines as described later. Note that a similar intersection with the straight line Gr = p × Gb + q ′ is F. Similarly, the straight line ON is shifted by Δ in the negative direction on the horizontal axis.
Shift by B. Let K be the intersection of the resulting straight line and the straight line Gr = p × Gb + q. Note that the same intersection with the straight line Gr = p × Gb + q ′ is set to H.

The two straight lines shifted by ΔB and R
A portion surrounded by the G gain movable range, that is, a straight line JF
A band-shaped portion surrounded by and the straight line KH is a region S.

The amount of shift in the positive direction and the amount of shift in the negative direction may be different values.

The above-mentioned predetermined range, and thus the amount of shift (ΔB), gives the operator the desire to obtain what color tone after correction, and the magnitude of ΔB gives other factors. The operator can arbitrarily determine the influence according to the influence and the like. That is, the variable step of the RB gain is the basis for determining the white balance control speed. ΔB for determining the predetermined range is preferably set to a value smaller than the variable step of the RB gain. At this time, increasing ΔB decreases the white balance control speed, and decreasing ΔB increases the white balance control speed.

The value of the RB gain obtained after the adjustment is preferably the same as that of the RB gain indicated by the point N in the same control direction. That is, for example, in the case shown in FIG. 4, the control from point C to point N is "B gain increase, R gain decrease". Therefore, as the value of the RB gain obtained after the adjustment, it is preferable to similarly set a value that becomes “B gain increase, R gain decrease”. The above-mentioned predetermined range is preferably set so that the RB gain obtained after the adjustment in this way has the same control direction as the RB gain indicated by the point N.

Also, as shown in FIG.
And the vertical axis are parallel to each other, but are not limited thereto.
By making the straight line FK and the vertical axis (Gr axis) parallel to each other, two diagonal lines (the straight line Gr = p × Gb + q and the straight line G) out of the six straight lines that determine the RB gain movable range are set.
r = p × Gb + q ′) can be easily reflected in ΔB. That is, if the interval between the oblique lines is wide, the RB gain movable range is wide, and in this case, ΔB may be increased. On the other hand, if the interval between the oblique lines is narrow, it means that the RB gain movable range is narrow. At this time, ΔB should be small. By making the straight line FK and the vertical axis parallel to each other, ΔB,
Consequently, the RB gain to be set can be obtained.

The position of the point N can be various. For example, one of the straight lines constituting the movable range of the RB gain, Gr = p × Gb + q, or Gr = p
× Gb + q ′ in some cases. Alternatively, a straight line Gb = Gbmin, Gr = Grmax, Gb =
Gbmax or Gr = Grmin may be outside the range.

In the present embodiment, since the point N is above one of the straight lines constituting the RB gain movable range, Gr = p × Gb + q, the straight line Gr = p × Gb + q is more straight line Gr = p × Gb + q. Is closer to point N than '. Also, a straight line as all other boundary lines, that is, a straight line Gb = Gbm
in, Gr = Grmax, Gb = Gbmax, Gr = G
The straight line Gr = p × Gb + q is closest to the point N as compared with rmin. Therefore, the region S and the straight line Gr = p × G
A point on the line of intersection with b + q is determined, and Gr and Gb at the coordinates are set in the RB amplifier as the RB gain of the next frame.
This intersection is a line segment JK in FIG.

That is, straight lines as all boundary lines, ie, straight lines Gr = p × Gb + q, Gr = p × Gb + q ′, G
b = Gbmin, Gr = Grmax, Gb = Gbma
A straight line closest to the point N in x, Gr = Grmin is obtained each time. Then, a point on the line of intersection of the area S and the straight line is obtained, and Gr and Gb of the coordinates are converted to R of the next frame.
The RB amplifier is set as the B gain.

As described above, in the present embodiment, a region (region S) including point N and formed on the basis of point N is closest to point N among the boundaries of the RB gain movable range. Set to a point on the line of intersection with the boundary line. In particular, here, an intersection M between the straight line ON and the straight line Gr = p × Gb + q is determined, and Gr and Gb of the coordinates are set in the RB amplifier as the RB gain of the next frame. That is, this is the case where the above-mentioned predetermined distance is 0, and therefore, the shift amount (Δ
B) is 0.

For example, a straight line Gr = p × Gb + q ′
An upper point, for example, a point near an intersection L of the straight line and the straight line ON, and particularly, a coordinate Gr of an intersection of the straight line and a straight line passing through the point C (see FIG. 4) and parallel to the vertical axis, When Gb is set as the RB gain of the next frame in the RB amplifier, the R gain decreases as compared to before the adjustment, that is, at the point C,
Since the B gain remains unchanged, the color becomes magenta.

The coordinates (Gb_G) of the intersection of the straight line ON and the boundary line closest to the point N among the boundary lines of the RB gain movable range.
m, Gr_m) is obtained by the following equation. That is, when the point N is located above the straight line Gr = p × Gb + q as shown in the figure, (Gb_m, Gr_m) = (q / (ap), q / (ap) × a) (6) (Point M), while the point N is a straight line Gr = p × Gb +
When it is below q ′ (not shown), (Gb_m, Gr_m) = (q ′ / (ap), q ′ / (ap) × a) (7) In the figure, it corresponds to the point L.

By setting the RB gain corresponding to the coordinates of the intersections M and L in the R amplifier and the B amplifier respectively, even when the target RB gain is out of the RB gain movable range, the RB gain can be adjusted. Since the value on the straight line (boundary line) that determines the range is set as the RB gain,
It is possible to realize a smooth white balance control operation without drawing in a chromatic color and without generating an RB gain lockout.

The change in the color tone of the image after the RB gain of the point M and the like is set as described above also changes depending on how the color of the image captured by the camera changes. In the example, it calms down to the extent that it moves slightly from the point M). However, since a new RB gain is set every time, when another subject is photographed, the RB gain changes so as to draw a portion corresponding to a white pixel in the subject to white.

The present invention is particularly useful when an actual subject is photographed using the solid-state imaging device 1 or the like, because it can cope with a sudden change in the color of the subject or the color of the light source. However, the present invention is not limited to this, and the solid-state imaging device 1, A
Instead of using the / D converter 2 and the OB offset clamp 3, an image signal as moving image data similar to that obtained by processing by these members is recorded on an external storage medium, and the moving image is recorded from the recording medium. It is also possible to read out the data one after another and perform the white balance adjustment process.

Next, a case where the above processing is performed by computer software will be described. Arbitrary members of the blocks shown in FIG. 1, for example, the next frame RB gain calculation unit 7, the movable range determination unit 8, and the RB gain recalculation unit 9 outside the movable range are not controlled by dedicated hardware but by a microcomputer. It may be realized by software to be executed. This software may be recorded on a recording medium such as a floppy disk, MO, or memory card together with a program for providing an automatic exposure adjustment method, read into a camera device, and executed by a microcomputer. Since the integration of pixel values requires a high-speed operation, it is often realized by hardware in general.

The processing under software control will be described with reference to FIGS. 6 and 7. The contents of the white balance control performed here are the same as those described with reference to the configuration in FIG. 1 except that the white balance control is performed by software control. Therefore, in the following, the point that the control is performed mainly by software will be described.

Here, the solid-state image pickup device 21, A / D converter 22, OB offset clamp 23, R amplifier 2
4, the B amplifier 25, and the white pixel accumulator 26 are respectively a solid-state image sensor 1, an A / D converter 2, an OB offset clamp 3, an R amplifier 4, a B amplifier 5, and a white pixel accumulator 6 shown in FIG. The same is used. That is, reference numeral 21 denotes a solid-state imaging device, which photoelectrically converts image information of a subject. The analog image electrical information subjected to the photoelectric conversion is converted into digital data by the A / D converter 22, and the OB offset clamp 23 cuts off the light provided on the solid-state imaging device 21 and outputs the signal at the portion where the photoelectric conversion is not performed. Is clamped to be zero.

The R amplifier 24 and the B amplifier 25 amplify the output signals of the R and B pixels, respectively, and
Color signal amplifiers (R gain amplifying means and B gain amplifying means) for outputting the gains (R
Gain and B gain) to perform white balance control. That is, by controlling the gain of the RB amplifier so that the output values of the R, G, and B pixels of the white portion of the subject become equal, the white portion of the subject is controlled to be reproduced white. Note that the R amplifier 24 and the B amplifier 25
It is generically called B amplifier.

The white pixel accumulating section 26 calculates how many color components are included in a white portion in a digital image signal obtained by photographing a subject when white balance is not achieved. The white portion of the subject reflects the sunlight accompanied by various color changes from red to blue, and shows a change from red to bluish as well as the light source color.

The microcomputer 27 has a role of the next frame RB gain calculation section 7, the movable range determination section 8, and the RB gain recalculation section 9 outside the movable range.
OM is built in, and software shown in the flowchart of FIG. 7 and software for performing automatic exposure adjustment are written.

Next, the processing procedure of the software built in the microcomputer 27 will be described with reference to FIG. In step 1 (hereinafter, the step is simply referred to as S), first, the white pixel integrator 26 extracts white pixels and calculates the color components I and Q under the conditions shown in the above equations (1) and (2). Output the integrated value. Then, the software built in the microcomputer 27 is
By dividing this integrated value by the total number of pixels, the color evaluation values r, g, and b of the image of one frame are obtained.

In S2, the color evaluation value (r, r,
The RB gain of the next frame is calculated based on g and b). Before the white balance correction, the white portion of the subject is colored by the light source color, so that the color evaluation values r, g, and b are not equal. After white balance correction, the white portion should be white, so that r = g = b. Therefore, the R amplifier 24 necessary for white balance is required.
And each coefficient of the RB pixel set in the B amplifier 25 is
If the coefficient of the R pixel is Gr and the coefficient of the B pixel is Gb, the above equation (3) is obtained.

Normally, the RB gain obtained by the above equation (3) is set in the RB amplifier at a constant rate so that the white is corrected smoothly over several frames. Assuming that the ratio is d (%), the RB gain of the next frame is expressed by the above equation (4), where the current RB gain is cGr and cGb, respectively (S3).

The microcomputer 27 also has a function as the movable range setting section 8 as described above, and
It is determined whether or not r and Gb are within the RB gain movable range which is set so as not to draw the chromatic object color white (S4). In the present embodiment, as shown in FIG. 3, only the area on the six straight lines and the area surrounded by the six straight lines shown in the above equation (5) is a region that can be set as the RB gain. If Gr and Gb are within the RB gain movable range, Gr is set to the R amplifier, and Gb is set to the B amplifier (S5). On the other hand, if Gr and Gb are not within the RB gain movable range, the process proceeds to S6, S7 and S8.

As described above, in FIG. 3, the curve W at the center is used to balance the white light when the color of the light source of the sun varies variously depending on the lighting conditions such as the season, time, shade, and sunshine. Shows the locus of the RB gain set in the curve W. The movable range of the RB gain also indicates the curve W so that white balance control is performed when sunlight is the light source.
Is set to have a certain width.

In the present embodiment, the curve W is regarded as a substantially straight line, and the RB movable range is set by the straight line represented by the equation (5). However, the movable range of the RB gain does not need to be a straight line. W is moved up and down respectively, and two upper and lower curves, Gr = Grmin, Gr = Grmax,
The range formed by Gb = Gbmin and Gb = Gbmax may be used. Alternatively, the curve W may be approximated by several broken lines.

As described above, the microcomputer 27 also has a function as the out-of-movement-range RB gain recalculation unit 9, and as a result of operating as the movable-range determination unit 8, Gr,
When Gb is out of the RB gain movable range, the RB gain is recalculated.

Referring to FIG.
The operation as the movable range determination unit 8 will be described. That is, as described above, point C is the current RB gain. C indicates that the current light source is blue light because the B gain is low and the R gain is high. For example, there is a case where there is a shade on a sunny day, or a case where the sun is hidden by clouds and the blue sky is a light source. Now, assuming the following photographic scene, a change in the RB gain at that time is considered.

It is assumed that an image of an outdoor shaded area is taken from a room illuminated by an incandescent lamp through a window. In this case, the coordinates (Gb, Gr) of the RB gain are at a position like point C in a state where the white balance is maintained. At this time, it is assumed that the camera is suddenly turned to the room. Then, the light incident on the solid-state imaging device 21 turns into red light,
The R gain is reduced and the B gain is increased to achieve white balance. However, when the light source in the room is different from the sunlight, the change in the RB gain moves out of the gray range in FIG. The point N is outside the RB gain movable range originally provided to prevent the drawing in of a chromatic color, and for that purpose, the RB gain at the point N cannot be set.

In this embodiment, as described above,
The point N is one of the straight lines Gr constituting the RB gain movable range.
= P × Gb + q, the intersection M of the straight line ON connecting the origin O and the straight line Gr = p × Gb + q is determined, and Gr and Gb of the coordinates are set as R and B gains of the next frame.
Set to B amplifier (S6, S7, S8 in FIG. 7). When the point N is below Gr = p × Gb + q ′, the intersection L of the straight line ON and Gr = p × Gb + q ′ is set in the RB amplifier as the RB gain of the next frame. At this time, the coordinates (Gb_m, Gr_m) of the intersection M (or L) are obtained by the following equation. That is, the point N is a straight line Gr = p × Gb
When it is higher than + q, the above equation (6) is obtained.
When the point N is below the straight line Gr = p × Gb + q ′, the above equation (7) is obtained.

By setting the RB gain corresponding to the coordinates of the intersections M and L in the R amplifier 24 and the B amplifier 25 respectively, even when the target RB gain is out of the movable range of the RB gain, Since the value on the straight line (boundary line) that determines the gain movable range is set as the RB gain, it is possible to realize a smooth white balance control operation without drawing in chromatic colors.

[Second Embodiment] The following will describe another embodiment of the present invention with reference to FIG. For convenience of explanation, members having the same functions as the members shown in the drawings of the above-described embodiment are denoted by the same reference numerals, and description thereof will be omitted.

When a point N outside the RB gain movable range is obtained in the same manner as in the first embodiment, in the present embodiment, the point N is formed based on the point N as in the first embodiment.
Is set to a point on the intersection of a certain region including the boundary and the boundary line closest to the point N among the boundary lines of the RB gain movable range. The details are described below.

First, a straight line C connecting point C and point N
An area (referred to as T) consisting of points within a predetermined distance from N is set. That is, as shown in FIG.
The straight line CN is shifted by ΔB in the positive direction along the horizontal axis (Gb). The resulting straight line and the point N of the whole boundary line as described later
The intersection with the straight line Gr = p × Gb + q, which is the boundary line closest to, is defined as J ′. Similarly, the straight line CN is shifted by ΔB in the negative direction along the horizontal axis. Straight line and straight line Gr = p × Gb
The intersection with + q is K ′. A band-shaped portion surrounded by the two straight lines shifted by ΔB and the RG gain movable range is a region T.

The method of determining the shift amount (ΔB) is the same as in the first embodiment. Therefore, the description is omitted.

In this embodiment, since the point N is above one of the straight lines constituting the RB gain movable range, Gr = p × Gb + q, the straight line Gr = p × Gb + q is more straight line Gr = p × Gb + q. Is closer to point N than '. Also, a straight line as all other boundary lines, that is, a straight line Gb = Gbm
in, Gr = Grmax, Gb = Gbmax, Gr = G
The straight line Gr = p × Gb + q is closest to the point N as compared with rmin. Therefore, the region T and the straight line Gr = p × G
A point on the line of intersection with b + q is determined, and Gr and Gb at the coordinates are set in the RB amplifier as the RB gain of the next frame.
This intersection is a line segment J'K 'in FIG.

That is, straight lines as all boundary lines, that is, straight lines Gr = p × Gb + q, Gr = p × Gb + q ′, G
b = Gbmin, Gr = Grmax, Gb = Gbma
A straight line closest to the point N in x, Gr = Grmin is obtained each time. Then, a point on the line of intersection between the region T and the straight line is determined, and Gr and Gb of the coordinates are calculated as R and R of the next frame.
The RB amplifier is set as the B gain.

As described above, a certain region (region T) including the point N formed on the basis of the point N and the boundary line closest to the point N among the boundary lines of the RB gain movable range are shown in FIG. Set to a point. Therefore, the direction of control matches the case of setting the RB gain indicated by point N.

In particular, here, a straight line CN and a straight line Gr =
An intersection P with p × Gb + q is obtained, and Gr, Gb of the coordinates are obtained.
Is set in the RB amplifier as the RB gain of the next frame. That is, when the above-mentioned predetermined distance is 0,
Therefore, this is a case where the shift amount (ΔB) is 0.
That is, a straight line CN including a point C which is the previous RB gain coordinate and is within the RB gain movable range or on the boundary line, and a point N which is the current RB gain coordinate which is out of the RB gain movable range, and An intersection P with a line is set. Actually, the range of possible values of the RB gain is 0 to 2047, whereas the variable step of the RB gain is a small value of 1 to 16. Therefore, it can be considered that the distance between the point N and the point C is considerably small. As described above, the method of setting the intersection of the straight line CN and the boundary line also includes a method in which the direction of control indicated by the point after the setting is
Since the control direction indicated by the point N coincides with the control direction, better control can be performed.

Note that, as described in the first embodiment, the straight line O
When the intersection (point M) between N and the boundary line is adopted, the amount of calculation is smaller than in this embodiment, and the degree of compression of the ROM capacity can be suppressed.

[Embodiment 3] The following will describe another embodiment of the present invention with reference to FIG. For convenience of explanation, members having the same functions as the members shown in the drawings of the above-described embodiment are denoted by the same reference numerals, and description thereof will be omitted.

If a point N outside the RB gain movable range is obtained in the same manner as in the first embodiment, the following processing is performed in the present embodiment. That is, considering a vector starting from the point C, its end point is the area α,
It is located at any one of four of β, γ, and δ. That is, if the vector is expressed as (u, v), the area α is u
> 0, v <0. Similarly, the region β is u>
0, v> 0, region γ is u <0, v> 0, and region δ is u <0, v <0. Since the coordinate plane is divided into these four regions, the RB gain movable range is also divided according to the four regions. Note that four regions on a line (indicated by a dashed line in the figure) forming a boundary between the regions α to δ, that is, u> 0, v = 0
, An area of u = 0 and v> 0, an area of u <0 and v = 0, and an area of u = 0 and v <0.

Next, it is checked in which of the above four areas the point N is located. Here, as shown in the figure, the point N is located in the area α (hereinafter, referred to as a point N existence area).

Next, of the boundary lines forming the RB gain movable range (here, six line segments as in the first embodiment), the portion of the curve W on the same side as the point N (hereinafter, the point N Side part). In the case of the figure, (A) all of the straight lines Gr = p × Gb + q, (A)
Straight line Gr = part of Grmax, and (c) straight line Gb =
Gbmax.

An area included in both the point N existence area and the point N side portion (hereinafter, referred to as an overlap area)
Is determined. In this embodiment, Gr and Gb of the coordinates indicated by the point on the overlapping area are set in the RB amplifier as the RB gain of the next frame. As points on this area, points A and P are illustrated in FIG. In other words, the point P is an intersection of the straight line CN and the portion (A), and the point A is an example of a point different from the point P on the portion (A).

That is, all the boundaries forming the RB gain movable range, that is, the straight line Gr = p × Gb +
q, Gr = p × Gb + q ′, Gb = Gbmin, Gr =
Among the areas Grmax, Gb = Gbmax, and Gr = Grmin, a part that is the same as the point N among the areas α to δ and that is on the same side of the curve W as the point N is obtained. Gr and Gb of the coordinates of the points within are set in the RB amplifier as the RB gain of the next frame.

In other words, assuming that the point representing the RB gain to be newly set is point A, it is a part on the same side as point N with respect to curve W in the boundary line forming the RB gain movable range, and , The RB gain represented by the point A on the portion where the signs are equal between the horizontal axis components and the vertical axis components of the vector CN and the vector CA,
That is, a new RB gain is set. That is, in this example, assuming that the vector CN is (Δx, Δy), Δx> 0 and Δy <0, and the vectors CA and CP are also (Δxa, Δya), respectively.
(Δxp, Δyp), Δxa> 0, Δya <
0, Δxp> 0 and Δyp <0.

Since the point on the overlap area is on the area where the point N exists, the control direction is the same as the case where the RB gain is set according to the point N. That is, R gain · B
The direction in which the gain is increased or decreased is the same. In addition, since the point on the overlapping area is on the point N side, the value of the RB gain does not greatly differ from the point N as compared with the point on the side where the point N does not exist.

In particular, here, a straight line CN and a straight line Gr =
An intersection P with p × Gb + q is obtained, and Gr, Gb of the coordinates are obtained.
Is set in the RB amplifier as the RB gain of the next frame. That is, this is a case where the direction of the vector starting from the point C and ending at the point after setting matches the direction of the straight line CN, particularly the direction of the vector CN.

In the first and second embodiments, the boundary line closest to the point N is selected from the plurality of boundary lines, but instead of doing so, as in the present embodiment, It may be selected as a boundary line.

Note that the image processing apparatus according to the present invention may be configured as follows. That is, the image processing device
An imaging unit that captures information of a subject, an amplifying unit that amplifies an output signal of an R pixel, an amplifying unit that amplifies an output signal of a B pixel, and a unit that extracts a white pixel from an R pixel, a G pixel, and a B pixel; Means for calculating a color component of a white pixel; means for calculating an R gain (R amplifier gain) and a B gain (B amplifier gain) such that the color component becomes zero from the color component of the white pixel; B gain on both axes and origin at O
Means for judging whether the R gain and the B gain are within the movable ranges of the R gain and the B gain set so as not to draw in chromatic colors, and R and B gains outside the movable range. In some cases, when a point indicated by the R gain and the B gain outside the movable range is set to N, a straight line O
A point on the intersection of an area consisting of points within a predetermined distance from N and a boundary line closest to the point N on the boundary line of the movable range is defined as a new R gain and B gain, respectively.
You may comprise so that it may set to an amplifier and a B amplifier.

Further, the image processing apparatus according to the present invention may be configured as follows. That is, the image processing device
An imaging unit that captures information of a subject, an amplifying unit that amplifies an output signal of an R pixel, an amplifying unit that amplifies an output signal of a B pixel, and a unit that extracts a white pixel from an R pixel, a G pixel, and a B pixel; Means for calculating a color component of a white pixel; means for calculating an R gain and a B gain such that the color component becomes zero from the color component of the white pixel; and the R gain and the B gain do not draw a chromatic color. Means for judging whether the R gain and the B gain are within the movable ranges set as described above, and, when the R gain and the B gain are outside the movable range, the R gain and the B gain outside the movable range are determined. The R gain and the B gain that are near and within the movable range may be set to the R amplifier and the B amplifier, respectively.

According to the above configuration, even if the calculated RB gain is out of the RB gain movable range, the RB gain is within the RB gain movable range and is closest to the RB gain calculated result. RB gain is set,
A new color component different from the color component of the previous frame is obtained for each frame. Therefore, the RB gain is not locked. That is, even when the illumination changes suddenly, the RB gain movable range boundary line and the value inside the RB gain movable range boundary are always set in the RB amplification unit, so that lockout does not occur.

Further, since the processing is carried out by a simple linear expression, software for performing the processing according to the present method can be easily described, and a microcomputer or the like to execute the processing can be a simple 4b.
There is an advantage that an it microcomputer can be used.

The white balance processing can be performed by a computer such as a personal computer or a handheld computer.

The image processing apparatus according to the present invention may be configured as follows. That is, the image processing device
When the new R gain and B gain are out of the movable range,
A straight line connecting the coordinates (B gain, R gain) and the origin on the coordinates of the R gain and the B gain as the R gain and the B gain close to the R gain and the B gain outside the movable range and within the movable range; The coordinates of the intersection with the boundary of the movable range are
The gain and the B gain may be set to the R amplifier and the B amplifier, respectively.

According to the above configuration, it is possible to continue smooth white balance correction with a simple linear equation without imposing a computational burden on the processing device.

[0123]

As described above, according to the image processing method of the present invention, when the point N is located outside the RB gain movable range, the RB gain represented by the point on the boundary line is set to a new R
In this configuration, the gain is set as B gain.

Further, the image processing apparatus of the present invention has the above-mentioned point N
Has an RB gain setting means for setting an RB gain represented by a point on the boundary line as a new RB gain when the RB gain is outside the RB gain movable range.

The computer-readable recording medium on which the image processing program of the present invention is recorded is characterized in that
Is located outside the RB gain movable range, the RB gain indicated by the point on the boundary line is set as a new RB gain.

Thus, by performing white balance correction on a chromatic subject, it is possible to prevent unnatural color correction from occurring, and when the light source color changes suddenly or when the subject changes, the Even when the color of light changes abruptly, the RB gain is not locked, and the color tone can be controlled correctly.

Further, in the image processing method according to the present invention, in addition to the above configuration, the area representing the RB gain movable range is formed by one or more line segments as boundaries, and the point N When located outside the RB gain movable range, a point on the intersection line between the boundary line closest to the point N among the boundary lines and the region S including points within a predetermined distance from the straight line ON is represented. In this configuration, the RB gain is set as a new RB gain.

As a result, since the control direction of the white balance does not greatly deviate, in addition to the effect of the above configuration, when the color of the light source changes abruptly or the subject changes, the color of the reflected light changes abruptly. In this case, it is possible to prevent the color of the subject from changing to an extremely different color.

Further, in addition to the above configuration, the image processing method according to the present invention further comprises a line on the intersection of the boundary line closest to the point N among the boundary lines and the straight line ON having the predetermined distance of 0. In this configuration, the RB gain indicated by the point M is set as a new RB gain.

As a result, since the control direction of the white balance is not deviated, in addition to the effect of the above configuration, when the color of the light source changes rapidly or when the subject changes and the color of the reflected light changes rapidly, It is possible to more effectively suppress the color of the subject from changing to an extremely different color.

Further, in the image processing method of the present invention, in addition to the above configuration, the area representing the RB gain movable range is formed by one or more line segments as boundaries, and the point N When located outside the RB gain movable range, a point on the intersection of a boundary line closest to the point N and a region T including points within a predetermined distance from the straight line CN among the boundary lines represents the boundary line. In this configuration, the RB gain is set as a new RB gain.

As a result, the control direction of the white balance does not greatly deviate, and in addition to the effects of the above-described configuration, when the color of the light source changes abruptly or the subject changes, the color of the reflected light changes abruptly. In this case, it is possible to prevent the color of the subject from changing to an extremely different color.

Further, in addition to the above configuration, the image processing method according to the present invention further comprises an intersecting line between the boundary line closest to the point N among the boundary lines and the straight line CN having the predetermined distance of 0. In this configuration, the RB gain indicated by the point P is set as a new RB gain.

As a result, since the control direction of the white balance is not deviated, in addition to the effect of the above configuration, when the color of the light source changes abruptly or when the color of the reflected light changes abruptly when the subject changes. It is possible to more effectively suppress the color of the subject from changing to an extremely different color.

Further, in addition to the above configuration, when the point N is located outside the movable range of the RB gain, the image processing method of the present invention sets the point representing the newly set RB gain to the point A. Then, in the above-described boundary line, a portion on the same side as the point N with respect to the curve W, and the signs of the horizontal axis components and the vertical axis components of the vector CN and the vector CA are equal to each other. In this configuration, the RB gain represented by the point A on the portion is set as a new RB gain.

As a result, the control direction of the white balance does not greatly deviate. In addition to the effects of the above-described configuration, when the color of the light source changes abruptly or the subject changes, the color of the reflected light changes abruptly. In this case, it is possible to prevent the color of the subject from changing to an extremely different color.

Further, the image processing method of the present invention has a configuration in which, in addition to the above configuration, the RB gain represented by the point P on the straight line CN is set as a new RB gain as the point A.

As a result, since the control direction of the white balance is not deviated, in addition to the effect of the above configuration, when the color of the light source changes abruptly or when the color of the reflected light changes abruptly when the subject changes. It is possible to more effectively suppress the color of the subject from changing to an extremely different color.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration example of a video signal correction device that executes an image processing method according to the present invention.

FIG. 2 is an explanatory diagram illustrating a method of extracting a color component from a video signal.

FIG. 3 is an explanatory diagram showing a movable range of an R gain and a B gain.

FIG. 4 is an explanatory diagram showing a recalculation method when the R gain and the B gain are out of a movable range in the present invention.

FIG. 5 is an explanatory diagram showing a recalculation method when the R gain and the B gain are out of a movable range in the present invention.

FIG. 6 is a block diagram illustrating a configuration example of a video signal correction device that executes an image processing method according to the present invention by a microcomputer.

FIG. 7 is a flowchart illustrating a video signal correction process according to the present invention.

FIG. 8 is an explanatory diagram showing a recalculation method when an R gain and a B gain are out of a movable range in another configuration example of the image processing method according to the invention.

FIG. 9 is an explanatory diagram showing a recalculation method when an R gain and a B gain are out of a movable range in still another configuration example of the image processing method according to the present invention.

FIG. 10 is a block diagram illustrating a configuration example of a conventional video signal correction device.

FIG. 11 is a flowchart showing a conventional video signal correction process.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Solid-state image sensor 2 A / D converter 3 OB offset clamp 4 R amplifier 5 B amplifier 6 White pixel integration part (RB gain setting means) 7th-order frame RB gain calculation part (RB gain setting means) 8 Movable range determination part (RB Gain setting means) 9 RB gain recalculation unit outside movable range (RB gain setting means) 10 post-processing unit 21 solid-state imaging device 22 A / D converter 23 OB offset clamp 24 R amplifier 25 B amplifier 26 white pixel integration unit (RB gain Setting means) 27 microcomputer (RB gain setting means) 30 post-processing unit

 ──────────────────────────────────────────────────続 き Continuing on the front page F term (reference) 5C065 AA01 BB02 CC01 CC02 CC03 CC09 DD01 DD17 GG15 GG17 GG18 GG22 GG31 GG32 5C066 AA01 BA20 CA08 DD07 EA08 EA14 EA19 GA01 GA02 GA05 HA03 KA09 KA12 KD02 KD17 KG04 KM01

Claims (9)

[Claims] 1. Consider a coordinate plane in which the origin is O and the R and B gains are coordinate axes, respectively. An area representing an RB gain movable range for preventing chromatic colors from being drawn is formed by a boundary line provided based on the curve W, and a point representing a current RB gain is set as a point C and is set as an object. In the image processing method for setting an RB gain for a process of adjusting a white balance of a pixel, a point representing an RB gain obtained from a color component of a pixel such that the color component becomes zero is set as a point N, When N is outside the RB gain movable range, the RB gain represented by the point on the boundary line is changed to a new R
An image processing method characterized by setting as a B gain. 2. An area representing the RB gain movable range is formed by one or more line segments as boundaries, and when the point N is located outside the RB gain movable range, the boundary is determined. An RB gain represented by a point on an intersection of a boundary line closest to the point N among the lines and an area S including points within a predetermined distance from the straight line ON is set as a new RB gain. The image processing method according to claim 1. 3. An RB gain represented by a point M on an intersection of the boundary line closest to the point N and the straight line ON having the predetermined distance of 0 is set as a new RB gain. 3. The image processing method according to claim 2, wherein: 4. An area representing the RB gain movable range is formed by one or more line segments as boundaries, and when the point N is located outside the RB gain movable range, the boundary is determined. An RB gain represented by a point on an intersection of a boundary line closest to the point N among the lines and an area T including a point within a predetermined distance from the straight line CN is set as a new RB gain. The image processing method according to claim 1. 5. An RB gain represented by a point P on an intersection of the boundary line closest to the point N among the boundary lines and the straight line CN having the predetermined distance of 0 is set as a new RB gain. 5. The image processing method according to claim 4, wherein: 6. When the point N is outside the RB gain movable range and a point representing a newly set RB gain is point A, the point W of the curve W in the boundary line An RB gain represented by a point A on a part on the same side and having the same sign between the horizontal axis components and the vertical axis components of the vector CN and the vector CA is represented by a new RB gain. The image processing method according to claim 1, wherein: 7. The image processing method according to claim 6, wherein, as the point A, an RB gain represented by a point P on the straight line CN is set as a new RB gain. 8. A coordinate plane in which the origin is O and the R gain and the B gain are each on a coordinate axis is considered, and the locus of the RB gain required for white balance according to the light source color of sunlight is defined as a curve W. An area representing an RB gain movable range for preventing chromatic colors from being drawn is formed by a boundary line provided based on the curve W, and a point representing a current RB gain is set as a point C and is set as an object. In the image processing apparatus for setting an RB gain for a process of adjusting a white balance of a pixel, a point representing an RB gain obtained from a color component of the pixel such that the color component becomes zero is set as a point N. When N is outside the RB gain movable range, the RB gain represented by the point on the boundary line is changed to a new R
An image processing apparatus comprising RB gain setting means for setting as a B gain. 9. Consider a coordinate plane in which the origin is O and the R and B gains are coordinate axes respectively, and the locus of the RB gain required for white balance according to the light source color of sunlight is a curve W. An area representing an RB gain movable range for preventing chromatic colors from being drawn is formed by a boundary line provided based on the curve W, and a point representing a current RB gain is set as a point C and is set as an object. A computer which records an image processing program for setting an RB gain for a process of adjusting a white balance of a pixel, where a point representing an RB gain obtained from a color component of the pixel such that the color component becomes zero is set as a point N. In a readable recording medium, when the point N is outside the RB gain movable range, the RB gain represented by the point on the boundary line is changed to a new R
A computer-readable recording medium having recorded thereon an image processing program set as a B gain.