JP2005190473A - System and method for light source model estimation - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To estimate a light source model of a given image. <P>SOLUTION: A set of colors as candidates is mapped on a chromaticity space, and distances to the image are calculated to determine the most suitable light source. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

Detailed Description of the Invention

(Technical field)
The present invention relates generally to digital image processing, and more particularly to a method and system for illuminant model estimation.

(Background technology)
The color visible in an image is determined by the light that illuminates the subject of that image. If the light source is different, the light reflected from the surface of the subject of the image is also different. The human visual system roughly corrects the difference due to the reflected light so that the perceived surface color is substantially constant. However, when an image is captured on a medium and viewed under a light source that is different from the light source in the scene of the image, correction according to the above natural law does not occur. Therefore, it is preferable to adjust the color balance to a standard, reference light source so that the recorded image looks as if it is visible to the natural eye. This balancing or color correction can be performed once the scene light source is identified.

  In the known method, a fixed set of known light sources is used for light source estimation. The known light sources are each characterized by a color gamut of chromaticity that can be under the light source. A chromaticity histogram is calculated for each image and compared with each of the model histograms.

(Summary of Invention)
Each embodiment according to the present invention implements a method and system for estimating a light source in an image by model matching using a distance criterion.

(Brief description of figure)
Each embodiment of the present application will become more fully apparent in the following description and appended claims, taken in conjunction with the accompanying drawings. It should be understood that the above drawings are merely representative of exemplary embodiments and therefore do not limit the scope of the invention. Further, in each embodiment, further specialities and details are described by using attached drawings.
FIG. 1 is a graph showing a set of candidate light sources in xy chromaticity coordinates.
FIG. 2 is a diagram showing a typical ordering scheme.
FIG. 3 is a graph showing a typical matched curved surface for an image and the light source graph of FIG.

(Detailed description of the invention)
The estimation of the light source may be addressed by a model matching strategy. In such a scheme, a fixed set of light sources is modeled. Modeling may be performed with parameterization by means of sample-based statistics or other methods. The model that is evaluated to be optimal with respect to the original image data is selected as the scene light source with the highest certainty that it would have produced the image. The determination process includes calculating the same parameterized description or the same description based on statistics from the image, and then describing the image with respect to the model base to select an “optimal” model. And the step of performing the matching procedure.

As shown in FIG. 1, a typical model set consists of 81 light sources plotted in an xy chromaticity space. The chromaticity coordinates are shown as point 2. In this particular example, the coordinates are regularly sampled on CIE-Lab coordinates centered on D65 white point 4, which is the daylight reference light source. The coordinates are then mapped into the xy chromaticity space as illustrated in FIG. In a similar manner, each pixel of the input image is converted to CIE-xyY space. (This CIE-xyY space is an xy chromaticity space including luminance information.)
In another embodiment, color values may be represented in another color space or other chromaticity space as follows:
r = R / (R + G + B)
g = G / (R + G + B)
b = B / (R + G + B)
In some embodiments of the invention, a distance criterion may be used to determine which model best fits the light source of the image. The term “distance” is often used to describe a length in space (eg, the length of a line segment in a two-dimensional or three-dimensional space). However, in the present application, the term distance may refer to the same criterion as described above even in a three-dimensional or higher space, or may refer to a difference between two values that can be represented by a point in a multidimensional space.

The measurement value (measure) d (x, y) is a distance reference when and only when the following three conditions are satisfied (if; if and only if).
1. d (x, y) = 0 iffx = y [0 when they are the same]
2. d (x, y) = d (y, x) [Symmetry]
3. d (x, y) + d (y, z) <= d (x, z) [triangle inequality]
The condition of non-negativity (ie d (x, y)> = 0 for all x, y) is usually added (but often unconditionally).

  A distance criterion may also refer to a difference metric or a dissimilarity metric. In the present specification and claims, the terms distance, difference, and dissimilarity are synonymous.

  In some embodiments of the invention, operations can be performed on a subset of the complete model set. The complete model set may be narrowed down to a subset by an initial selection process. The final light source selection will then occur within a subset of the model.

In another embodiment, the evaluation may be performed in a “greedy” manner. Examples of “greedy” evaluation methods are shown below.
(A) Evaluating the first guess model as the prior value.
(B) Evaluating the second optimal estimation model as the current value.
(C) A difference vector between the current value and the prior value is calculated.
(D) While the difference vector is decreasing and larger than the reference value,
(1) The minimum value of the prior value and the current value is set as the prior value.

  (2) A new optimum estimated value is evaluated as the current value. The difference vector is usually used as a guide when selecting a new model.

  (3) The difference in score between the prior value and the current value is calculated.

  In each embodiment of the present invention, it is possible to use distance / difference / dissimilarity measurement using any vector for each element. Such a measurement method is suitable for vectors or distributions in the same space.

  In some embodiments, distances under Minkowski norms, such as L1, L2, L∞, etc. may be used as a reference. The above parameters are represented by the following equation:

  In another embodiment of the present invention, a Chi-squared statistical difference may be used as a criterion. This difference criterion is expressed by the following equation.

  In yet another embodiment, a Jeffrey divergence metric may be used. This criterion is represented by the following equation:

  In some of the above-described embodiments, a quadratic criterion may be used. This criterion is represented by the following equation:

  In yet another embodiment, a cumulative match distance may be used. This criterion is represented by the following equation:

  In another embodiment, a projective match distance may be used as a reference. This criterion is represented by the following equation:

  Further complicating the estimation of the light source is when the image contains a non-reflective or self-luminous object that does not directly reflect light from the light source. Conventional correction algorithms assume that every image pixel represents a reflective surface. If the image contains self-luminous objects, such as the sky or other illuminants, the assumption of surface pixels is disturbed. If the image contains an important part that does not reflect light, i.e., a self-luminous body, the conventional method will fail and the light source of the image will be erroneously determined. For example, if the image contains a blue sky and the color balance algorithm assumes that all pixels are reflective objects, the blue pixels will be perceived as having a blue illumination of the scene. Will. Since color correction is roughly to reverse the estimated hue of the light source, correction for a bluish light source changes the image in the yellow direction. This correction results in an overly yellowish ground / surface area and a desaturated sky area.

  The above color correction, color balance, or color constancy algorithms generally answer the question of how to deal with images containing illuminants (hereinafter also referred to as self-illuminators). Not paying attention. Rather, they have focused on images that satisfy the assumption of surface pixels (eg, a Mondrian-like image that is illuminated uniformly).

  In some embodiments of the invention, a function is calculated for each pixel, image element, or image region. This function is for estimating the probability (that is, the probability of not self-luminous) p corresponding to the reflective surface of the pixel, image element, or image area. This function is necessary because almost all illuminant estimation procedures assume that image pixels are created by light sources that are reflected from the surface of the scene. Thus, including self-luminous pixels makes the estimation invalid. The above calculation is not necessary for the model histogram. This is because the model histogram is based on a definition constructed only from data corresponding to the reflection surface.

Details of the function embodiment are described in detail in another previously filed patent application.
US patent application no. 10 / 677,034 (Title of the invention; system and method for calculating the presence of self-luminous elements in an image, inventor; John M. Spiegle, John E. Dolan, filed September 30, 2003)
US patent application no. 10 / 676,306 (Title of the invention; image color balance correction system and method, inventor; John M. Spiegle, John E. Dolan, filed September 30, 2003)
US patent application no. 10 / 677,009 (Title of Invention; Light Source Estimation System and Method, Inventor; John E. Dolan, John M. Spiegle, filed September 30, 2003)
The above is incorporated herein by reference below.

  In another embodiment, where the image color histogram is calculated in three dimensions (or more than three dimensions), the function for estimating the probability that a pixel element or region corresponds to a reflective surface is a color coordinate and pixel coordinate argument. Receive.

  In another embodiment, the function for estimating the probability that an element or region corresponds to a reflective surface may operate in the vicinity of the image pixel element.

  In some of the embodiments described above, the histogram encodes the degree of evidence in the image for the presence of each chromaticity arising from the reflective surface.

  In an embodiment of the present invention, the image histogram is matched with a model histogram, and an optimal light source at this time is selected. This fitting criterion used in some embodiments is the chi-squared statistic. This chi-square statistic is a procedure for testing the variance of a sample group with respect to the base population. In embodiments of the present invention, the image may represent a sample population, and the light source model may be represented as an assumed base population. The formula representing the above relationship may be formed as follows.

The fitting criteria are calculated for each of the models, and the light source that yields the minimum of the function is the optimal light source for the image data. The optimum light source is a light source that minimizes the difference between the value observed in the image and the value expected under the light source in the equation (1). In a rough statistical term, as the χ ² reference value decreases, the proportion of data that supports the hypothesis that the corresponding model is the true light source of the scene increases.

  FIG. 3 shows examples of match curved surfaces for an image and the light source graph of FIG. 1 under various matching criteria (correlation, L1, L2, and chi-square). The index of the optimal light source is also shown for each. The correlation and L2 criteria are the same, and the L1 and chi-square criteria are slightly different choices.

  The algorithm in the embodiment of the present invention can be executed on software of a general-purpose computer or a special arithmetic device such as a DSP. Embodiments may also be implemented by dedicated circuitry such as ASICs. The processes of the embodiments can be performed in any image processing pipeline that outputs images for display, information retrieval, indexing, or other purposes.

It is the graph which showed the set of the light source used as a candidate by xy chromaticity coordinate. It is a figure which shows a typical sequence diagram. 2 is a graph showing a typical match curved surface for an image and the light source graph of FIG. 1.

Claims (16)

Calculating a color gamut model for a plurality of candidate light sources;
Calculating an image color gamut;
Determining a distance matching criterion for each of the candidate light sources for the image color gamut;
Selecting a light source of the image from the plurality of candidate light sources based on the distance matching criteria;
A method for estimating a light source in an image, comprising: Calculating a color gamut for a plurality of candidate light sources;
Calculating a self-luminous characteristic that includes a characteristic that indicates a degree of similarity between the image element and the self-luminous image element or the reflective image element;
Separating a self-luminous image-like element from a reflective-image-like element;
The reflective image element is considered separate from the self-luminous image element and calculating an image color gamut;
Determining a fitness criterion for each of the candidate light sources;
Selecting a light source of the image from the plurality of candidate light sources based on the matching criteria;
A method for estimating a light source in an image, comprising:   The method according to claim 2, wherein the step of calculating the self-luminous property uses a binary value indicating whether the element is reflective or self-luminous.   The method of claim 1, wherein the color gamut for a plurality of candidate light sources is a histogram of color values for a set of color chips calculated under each candidate light source.   The method of claim 2, wherein separating the self-luminous image element comprises determining a proximity of the image element to an image boundary.   The method of claim 2, wherein separating the self-luminous image element comprises comparing the color characteristics of the image element with the color characteristics of a reflective surface under a known light source.   The method of claim 2, wherein the step of separating the self-luminous image elements comprises comparing the luminance characteristics of the image elements with the luminance characteristics of known self-luminous elements.   The method of claim 2, wherein the step of calculating the image chromaticity gamut calculates the image chromaticity gamut based on the reflective image element alone.   The step of calculating the image chromaticity range is a weighted distribution of the reflected image element and the self-luminous image element, and the reflected image element has a greater influence on the color gamut. The method of claim 2, wherein the calculation is based on a weighted distribution.   The method of claim 1, wherein determining the fitness criteria comprises calculating a chi-square statistic associated with a variance of an image related to candidate light sources. Calculating a color gamut for a plurality of candidate light sources;
Identifying image elements according to likelihood of being self-luminous;
Calculating an image color gamut from the image element, wherein the image element having a higher likelihood of being self-luminous is less weighted than the image element having a higher likelihood of being a reflective element;
Determining a match criterion for matching a gamut histogram for each of the candidate light sources to a gamut histogram for the image element;
Selecting a light source of the image from the plurality of candidate light sources based on the matching criteria;
A method for estimating a light source in an image, comprising: Selecting a set of known light sources;
Establishing, for each of the known light sources, a color gamut represented by a sample distribution histogram of color values for the set of color chips calculated under each of the known light sources;
Estimating a weighting value associated with the probability that an image element in the image corresponds to a reflective surface;
Establishing a gamut histogram for the image, wherein the weighting value is used to increase the accumulator of the corresponding histogram interval; and
Calculating a match criterion between the image color gamut histogram and the histogram of at least one known light source;
Selecting an estimated image light source that best fits the image gamut histogram from the set of known light sources;
A method for estimating a light source in an image, comprising:   The method of claim 12, wherein estimating the weighting value comprises using a function having an element color value and two image position values.   The step of calculating the adaptation criterion includes the step of using a chi-square statistic for measuring the difference between the image chromaticity histogram and the chromaticity gamut histogram of the known light source, normalized and squared. The method of claim 12 comprising. A first calculator for calculating a color gamut for a plurality of candidate light sources;
A second calculator for calculating the degree to which the element is self-luminous or reflected;
A third calculator for calculating an image color gamut, wherein the reflective image element is considered different from the self-luminous image element;
A matching section for determining a matching criterion for each of the candidate light sources;
A selection unit for selecting a light source of the image from the plurality of candidate light sources based on the matching criteria;
A system for estimating a light source in an image. A set of executable instructions for estimating the light source in the image,
The above instruction
Calculating the color gamut for multiple candidate light sources,
An operation for calculating a self-luminous characteristic for estimating a degree of the element to be self-luminous or reflected;
An operation for calculating an image color gamut, wherein the reflective image element is considered different from the self-luminous image element;
An operation for determining a matching criterion for each of the candidate light sources;
Selecting an image light source from the plurality of candidate light sources based on the matching criteria;
Including the above instruction set.

Images