JP4447520B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus that detects a light source color.

2. Description of the Related Art A white balance control device that adjusts white balance in a video camera or the like is known (see Patent Documents 1, 2, and 3). In this type of conventional white balance control device, it is common to detect a white balance deviation by integrating signals within a white region of an imaging subject.
Japanese Patent Laid-Open No. 2-94890 Japanese Patent Laid-Open No. 5-344530 JP-A-11-205806

  However, an actual imaging subject is not completely white even if the color is close to white, and the color temperature of the light source varies from light source to light source, so it is difficult to specify a complete white region. For example, in Patent Document 3 described above, white balance is based on the color distribution of a subject obtained by dividing a color space composed of color signals and luminance signals into a plurality of regions and detecting the amount of signal included in each region. This is the same as other known documents in that white balance deviation is detected with reference to the white region.

  As described above, since the conventional white balance control device detects and adjusts the white balance deviation based on the white region, it is difficult to adjust the white balance correctly by using an imaging subject having a small white portion. is there. In addition, as described above, when white balance adjustment is performed in consideration of the color temperature of the light source or the color of the subject itself, complicated processing must be performed, and there is a problem that processing takes time. .

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an image processing apparatus and an image processing method capable of predicting the light source color of an input image easily and accurately.

According to one aspect of the present invention, a first pixel value that detects a pixel value at a first pixel position provided at an arbitrary position in the image that is brighter than an average of the pixel values of the image in the image. Detection means;
Second pixel value detection means for detecting a pixel value at a second pixel position that is close to the first region and darker than the first pixel position;
More difference between the detected pixel values at the detected pixel values and the second pixel value detection means at the first pixel value detection means, the light source color for specifying a light source color of the light source to be irradiated on the image An image processing apparatus comprising: a specifying unit.

  According to the present invention, the light source color of the input image can be predicted easily and accurately, and white balance adjustment can be performed with accuracy by using the prediction result.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a schematic configuration of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus in FIG. 1 includes a light source color detection unit 1 that detects a light source color irradiated on an image, and a white balance calculation unit that calculates a white balance based on each color value of the light source color detected by the light source color detection unit 1. 2 and a white balance correction unit 3 that adjusts the white balance based on the calculated white balance.

  First, the operation principle of the light source color detection unit 1 will be described. It is known that two different reflections occur when light strikes an object. One is a reflection called diffuse reflection (diffuse reflection) that is reflected without directivity, and reflects the color of the object. The other is a reflection called specific direction scattering (specular reflection), which has the color of the light source instead of reflecting the color of the object (http: //www.rsch.tuis.ac.jp~naka/naka /scola/member/chapter5_html/chapter5.html).

  Here, the color of a portion with high luminance detected from a certain image (more specifically, the portion with higher luminance than the average luminance of the image) is A, and the color of a portion with a little luminance decrease in the vicinity thereof is B. To do. In this case, since the position of A and the position of B are close to each other, there is a high possibility that they are the same object. Even if they are not the same object, errors can be reduced by detecting many portions from the image and performing processing such as averaging.

Assuming that the diffuse light is D and the specific direction scattered light is S, the positions of A and B are close to each other, so D is substantially the same. On the other hand, S is known to cause a rapid change in the vicinity where the incident angle and the reflection angle of light are equal. For this reason, A and B can be approximately expressed by the following equations (1) and (2).
A = fa * S + D (1)
B = fb * S + D (2)
Here, fa represents the ratio of specular reflection to irregular reflection at the position A, and fb represents the ratio of specular reflection to irregular reflection at the position B.

When the light beam irradiated to the object is sampled with tristimulus values R, G, and B, the A value, B value, D value, and S value are three-dimensional vectors. These vectors are expressed by the following equations (3) to (6).
A = (AR, AG, AB) (3)
B = (BR, BG, BB) (4)
D = (DR, DG, DB) (5)
S = (SR, SG, SB) (6)
When the light beam is sampled with the four stimulus values C, M, Y, and G, the A value, the B value, the D value, and the S value are four-dimensional vectors.

  Assume that the color (A, B, C) of an object is (10, 20, 40). At this time, if this object is irradiated with stronger light (for example, 10 times the intensity of light), (100, 200, 400) is obtained. (10, 20, 40) and (100, 200, 400) represent the same color with different light intensities.

  Thus, in a linear RGB space, a certain color (R, G, B) and a color (f * R, f * G, f * B) obtained by multiplying that color by f are the same color.

The white balance gains GainR and GainB of the colors (R, G, and B) are expressed by the following equations (7) and (8).
GainR = 20/10 = 2.0 (7)
GainB = 20/40 = 0.5 (8)

  The white balance can be adjusted by multiplying the R value by the gain GainR and multiplying the B value by the gain GainB. The white balance adjustment value for the color (10, 20, 40) of an object is (20, 20, 20) It becomes. This represents a gray color. Similarly, the white balance adjustment value for strong light (100, 200, 400) is (200, 200, 200), which is also gray.

More precisely, the pixel values A and B acquired (sampled) in a bright place of the object are expressed by the following expressions (9) and (10).
A = fs * S + fd * D (9)
Here, if normalization is performed so that fd = 1 when fa = fs / fd, the same expression as expression (1) is obtained. The B value is also normalized in the same way to obtain the equation (2).

Taking the difference between the above-described equations (1) and (2), the following equation (11) is obtained.
A−B = (fa−fb) * S (11)

  From the equation (11), it can be seen that the (AB) value is a multiple of the specular reflection light S. This indicates that the (AB) value represents the same color as the specular reflection light (that is, the color of the light source color) and does not depend on the color of the object at all. As described above, when the brightness is a problem and only the color (specially, the color coordinate or chromaticity coordinate) is handled, the color difference (AB) of the object and the specular reflected light S are the same. Can be treated as a color.

The white balance gains GainR and GainB in this case are expressed by equations (12) and (13), respectively.
GainR = (fa−fb) * SG / (fa−fb) * SR (12)
GainB = (fa−fb) * SG / (fa−fb) * SB (13)
In both the expressions (12) and (13), the denominator and the numerator (fa−fb) are canceled, and the expressions (14) and (15) are obtained.
GainR = SG / SR (14)
GainB = SG / SB (15)
These expressions (14) and (15) are equivalent to the following expressions (16) and (17).
GainR = (AG−BG) / (AR−BR) (16)
GainB = (AG−BG) / (AB−BB) (17)
As can be seen from these equations (16) and (17), since the brightness of the light source color is not related to the white balance gain, the above-mentioned (fa−fb) can be ignored.

  As described above, the light source color can be estimated by detecting the difference between two adjacent points A and B of a certain object, and the white balance gain can be calculated using the result.

  Hereinafter, a specific processing operation of the present embodiment will be described. FIG. 2 is a flowchart illustrating an example of a processing procedure of the light source color detection unit 1. First, in order to detect a high-luminance portion A in the image and a portion B in the vicinity of which the luminance is slightly reduced, the entire image is divided into blocks composed of a plurality of pixels (step S1). In the following, an example is shown in which an image is divided into a total of 30 blocks of 6 × 5 pixels as shown in FIG. For example, when the processing of FIG. 2 is performed using an image taken with a digital camera, the number of pixels in one block is 50,000 to 500,000 pixels. However, the block size is not particularly limited, and an optimal value may be set as appropriate.

Next, the entire area in the scan result accumulation buffer 11 for classifying and storing the pixel values of each pixel in the image is initialized (zero cleared) (step S2). In this buffer, pixel values of each pixel in the image are classified and stored for each block. FIG. 4 is a diagram showing a data structure of the accumulation buffer 11. As shown in the figure, the accumulation buffer 11 is roughly divided into 30 blocks, each block having 16 stages, and Sum and Cnt are stored in each stage. For example, when each pixel value is composed of an R value, a G value, and a B value, Sum is a third-order vector quantity, and the R value, the G value, and the B value are stored. Alternatively, when each pixel value is composed of a C value, an M value, a Y value, and a G value, Sum is a fourth-order vector quantity, and the C value, the M value, the Y value, and the G value are stored. Cnt is the number of pixels stored in each stage.

When the initialization of the scan result accumulation buffer 11 is completed, the image is next scanned to obtain the pixel values of the respective pixels in order (step S3). Then, the block number of the acquired pixel is detected (step S4). Next, a pixel value indicating the brightness of the acquired pixel is calculated (step S5), and a stage where the pixel value is to be stored is determined (step S6). Then, the calculated pixel value is added to Sum in the determined stage, and 1 is added to Cnt (step S7).

  Next, it is determined whether or not all the pixels in the image have been scanned (step S8). If there is a pixel that has not been scanned yet, the processing from step S3 is repeated.

  When the scan is completed, for each block, Sum / Cnt at the brightest stage among the stages where Cnt is not 0 is set as A, and Sum / Cnt at the second brightest stage where Cnt is not 0 is set as B. A and B are stored in the AB value buffer 12 (step S9).

Since the A value and the B value are actually vectors, the AB value buffer 12 stores values shown in the following equations (18) to (23) in more detail.
AR = Sum.R / Cnt at the brightest stage (18)
AG = Sum.G / Cnt at the brightest stage (19)
AB = Sum.B / Cnt at the brightest stage (20)
BR = Sum.R / Cnt at the second brightest stage (21)
BG = Sum.G / Cnt at the second brightest stage (22)
BB = Sum.B / Cnt at the second brightest stage (23)

  FIG. 5 is a diagram showing an example of the data structure of the AB value buffer 12. As shown in the figure, the AB value buffer 12 has an area for storing the A value and the B value for each block.

  When the process of step S9 is completed, it is determined whether or not the storage in the AB value buffer 12 has been completed for all blocks (step S10). If the process has not been completed, the process of step S9 is repeated. The difference C between the average value of all A values and the average value of all B values in the AB value buffer 12 is calculated (step S11). This step S11 is equivalent to performing the calculation of the above-described equation (11), and this difference C becomes the light source color.

  When performing the processing of step S10 described above, instead of calculating the average of the A values and the average of the B values of all the blocks, each of the A value and the B value stored in the AB value buffer 12 is increased from the largest to n. An average of (1 ≦ n <30) may be calculated and the difference may be calculated.

Alternatively, the weight of (A value−B value) weights may be averaged by the following method. Assume that the gradation value of the actual light C is (CR, CG, CB), and (A value−B value) is S = (SR, SG, SB). At this time, the color distance k is expressed by the following equation (24).

  It can be considered that k shown in the equation (24) is a deviation and the inverse 1 / k is weighted and averaged as a weight of (A value−B value) of each block. Here, when k = 0, the weight is infinite, so that it may be approximated as weight = 1 / (k + 1) in implementation.

  The light source color detection unit 1 stores the difference C calculated in step S10 described above as a light source color. Thus, the processing of the light source color detection unit 1 is completed.

  After the processing of the light source color detection unit 1 is finished, the white balance calculation unit 2 in FIG. 1 calculates a white balance gain according to the processing procedure of the flowchart in FIG. First, the white balance gain GainR of the R value is calculated according to the above equation (16) (step S21). Specifically, the G value of the light source color / the R value of the light source color detected by the light source color detection unit 1 is calculated as GainR.

  Next, a B-value white balance gain GainB is calculated according to equation (17) (step S22). Specifically, the G value of the light source color / the B value of the light source color detected by the light source color detection unit 1 is calculated as GainB. Thus, the processing of the white balance calculation unit 2 ends.

In the expressions (16) and (17) described above, the light source color is white (the R value, the G value, and the B value are the same value), but the light source color is not necessarily white. For example, in the case of a light source with a low color temperature that is difficult for humans to adapt to color, it may look natural if there is a slight yellow component. In this case, for example, when the light source color <3500 K (Kelvin), the white balance gain may be calculated according to the following equations (25) to (27).
White balance gain of R value = 0.9 * Gc * Rc (25)
White balance gain of G value = Gc / Gc = 1 (26)
B value white balance gain = 1.1 * Gc / Bc (27)

On the other hand, when the light source color ≧ 3500K, the white balance gain may be calculated according to the following equations (28) to (30).
White balance gain of R value = Gc / Rc (28)
White balance gain of G value = Gc / Gc = 1 (29)
B value white balance gain = Gc / Bc (30)

  When the calculation of the white balance gain is completed, the white balance correction unit 3 in FIG. 1 uses the white balance gains GainR and GainB calculated by the white balance calculation unit 2 in accordance with the processing procedure of the flowchart in FIG. , Adjust the white balance of the image.

First, the image is scanned to obtain pixels in order (step S31), and the R value and B value of each pixel are corrected according to the following equations (31) and (32), respectively (step S32).
R value = R value * (white balance gain of R value) (31)
B value = B value * (white balance gain of B value) (32)
Thereby, final image data subjected to white balance adjustment is obtained.

  Thus, in this embodiment, since the light source color is detected by taking the difference between the pixel values of two adjacent points A and B in the input image and the white balance is adjusted using the light source color, Regardless of the color, the white balance can be adjusted very easily and accurately.

  In step S9 of FIG. 2 described above, the A value and the B value are determined for the stage where the Cnt value is not 0 in the AB value buffer 12, but Cnt is, for example, a predetermined ratio of the total number of pixels in the target block (for example, 1 %), The brightest stage Sum may be the A value, and the second brightest Sum may be the B value. In this way, the influence of image noise can be reduced by narrowing down the candidates for the A value and the B value in advance.

  In step S9 described above, the brightest stage Sum is A value and the second brightest Sum is B value, but the average of Sum from the brightest stage to several stages is A value and the next brightest. The average of Sum from several stages may be used as the B value.

  In step S9 described above, among the sums stored in the AB value buffer 12, Sums that do not satisfy the predetermined reference value may be excluded in advance from the target of the A value and the B value.

  The image processing apparatus shown in FIG. 1 can realize all the components by software. Therefore, for example, it can be realized in the form of application software activated on an operating system of a personal computer, or can be implemented as firmware in a digital camera. In any case, the processing procedures of FIGS. 2 and 6 described above can be executed by a microprocessor.

  Specific implementation forms of the present invention are, for example, a printer, a digital developing machine, a digital camera photo printer on a street corner, a print club (registered trademark), and the like.

  Further, at least a part of the configuration requirements of the present invention may be configured by hardware such as a semiconductor chip. By configuring with hardware, processing can be speeded up. Further, the present invention may be implemented inside the system LSI.

1 is a block diagram showing a schematic configuration of an image processing apparatus according to an embodiment of the present invention. 6 is a flowchart illustrating an example of a processing procedure of the light source color detection unit 1. The figure which shows the example which divided | segmented the image into 30 blocks. The figure which shows the data structure of the storage buffer 11. The figure which shows an example of the data structure of the AB value buffer. The flowchart which shows an example of the process sequence of a white balance calculation part. The flowchart which shows an example of the process sequence of a white balance correction | amendment part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Light source color detection part 2 White balance calculation part 3 White balance correction | amendment part 11 Scan result accumulation | storage buffer 12 AB value buffer

Claims (5)

First pixel value detection means for detecting a pixel value at a first pixel position provided at an arbitrary position in the image that is brighter than an average of the pixel values of the image in the image ;
Second pixel value detection means for detecting a pixel value at a second pixel position that is close to the first region and darker than the first pixel position;
More difference between the detected pixel values at the detected pixel values and the second pixel value detection means at the first pixel value detection means, the light source color for specifying a light source color of the light source to be irradiated on the image An image processing apparatus comprising: an identification unit; A luminance classification means for dividing an image into a plurality of blocks each consisting of a plurality of pixels and classifying the luminance of each pixel in each block into a plurality of stages,
The first pixel value detecting means detects an average pixel value of pixels classified in the brightest stage for each block,
The second pixel value detecting means detects an average pixel value of pixels classified in the second brightest stage for each block,
The light source color specifying means includes at least a part of an average value of pixel values for each block detected by the first pixel value detecting means and a pixel value for each block detected by the second pixel value detecting means. The image processing apparatus according to claim 1, wherein the light source color is specified based on a difference from at least a part of an average value of the light source. A luminance classification means for dividing an image into a plurality of blocks each consisting of a plurality of pixels and classifying the luminance of each pixel in each block into a plurality of stages,
A first pixel average for detecting a predetermined average value corresponding to each of the predetermined blocks in descending order of the average value among the average values of the pixel values detected by the first pixel value detecting means. A value detection means;
A second pixel average for detecting a predetermined average value corresponding to each of the predetermined blocks in descending order of the average value among the average values of the pixel values for each block detected by the second pixel value detecting means; A value detection means,
The light source color specifying means is based on a difference between the average value detected by the first pixel average value detecting means and the average value detected by the second pixel average value detecting means. the image processing apparatus according to claim 1, wherein the identifying the. White balance adjustment means for performing white balance adjustment based on the light source color detected by the light source color specifying means;
White balance calculating means for calculating a white balance gain for performing white balance adjustment based on a plurality of color components constituting the light source color detected by the light source color specifying means,
The white balance adjusting means, the image processing apparatus according to any one of claims 1 to 3, characterized in that the white balance adjustment is performed based on the white balance gain. Detecting a pixel value at a first pixel position provided at an arbitrary position in the image that is brighter than an average of the pixel values of the image in the image ;
Detecting a pixel value at a second pixel position proximate to the first region and darker than the first pixel position;
Identifying a light source color of a light source irradiated on the image based on a difference between a pixel value detected at the first pixel position and a pixel value detected at the second pixel position. A featured image processing method.

Images