TWI601091B - Multiple light sources color balance algorithm - Google Patents

Description

Multi-source color balance algorithm

The invention relates to a color balance method, in particular to a multi-source color balance algorithm.

Color balance is a type of image processing algorithm developed to compensate for the effects of light. Generally, an image that has not been color-balanced will have an overall color biased to a certain color, that is, the image has a color cast. The color balance algorithm can adjust the values of the three basic color layers of red, green and blue of the image to make the various colors in the color-shifted image return to normal, that is, remove the color cast. The general algorithm basically consists of two major steps: (1) estimating the lighting conditions of the captured image, and (2) adjusting the image, as further explained below.

In a general digital camera, a color balance calculation method is usually built in, and the main color balance algorithm is selected by the user to estimate the color temperature of the light source, and then a set of color balance correction values corresponding to the selected color temperature is used for the image. Perform color balance processing, and also provide a reference object (such as a white color card) for the user to align the camera with the reference color under the ambient light source, and then estimate a set of color balance correction values according to the captured color value, and then use the color set. The balance correction value is used for color balance processing. Generally speaking, a certain color temperature is used as a standard (for example, D65 standard light source), and the R ₆₅ \G ₆₅ \B ₆₅ value of the standard light source is divided by R _{w of} various reference color temperatures. G _w \B _w value, the color balance correction value at various color temperatures is obtained, and is built as a comparison table. After the user sets the reference color temperature, the color balance correction value corresponding to the reference color temperature can be obtained by looking up the table mode, and each pixel is obtained. The R, G, and B values are multiplied by the color balance correction value.

The above-mentioned color balance processing method is based on a single light source as an evaluation basis. After calculating a single color balance correction value, all the pixels in the image are corrected according to the same color balance correction value. However, the actual photography environment is mostly Multi-light source environment. Therefore, after estimating the color temperature of the overall image, the prior art usually only takes the average value or the peak value as the color temperature of the entire image, and then calculates the color balance correction value of the group, and uses all the pixels in the image to have the same color. Balance correction value for color balance correction will shift the pixels of some color groups in the whole image to warm color or cool color. The effect of color balance is not good. At present, most of the problems of solving multiple light sources are currently only It can be processed by manual retouching, and the program is complicated and time consuming.

In view of the fact that the existing color balance cannot handle the multi-light source environment and must rely on the technical defects of manual processing, the present invention proposes a multi-source color balance algorithm, which can group the pixels in the image according to the color temperature and correct the colors according to the grouping result. The values are color balanced to replace manual processing.

To achieve the above objectives, the multi-source color balance algorithm includes the following steps:

Inputting the original image, inputting an original image in a color space, and the original image includes a plurality of pixels;

Performing a light source estimation calculation to calculate an initial estimated color temperature of the plurality of pixels;

Performing a light source grouping calculation, setting a light source grouping number, and performing grouping according to the initial color temperature of the complex pixel and the number of the light source grouping, and calculating the corresponding complex number to determine the color temperature, and then determining the color temperature according to the complex number, and estimating by the table method a complex array color balance correction value corresponding to the plurality of pixels in the original image;

The image compensation operation is performed by multiplying the plurality of pixels by a corresponding color balance correction value to output a corrected image after color balance.

The multi-source color balance algorithm calculates the initial estimated color temperature of each pixel by using the light source estimation calculation, and then calculates the determined color temperature of each pixel by the grouping of the light source grouping, wherein the number of the light source group is the number of ambient light sources, and the light source is grouped by the light source. The calculus can effectively group the initial estimated color temperature of the pixels according to the ambient light source. After that, the color balance correction value is corrected by the look-up table method to correct each pixel one by one, so as to avoid the color-shift phenomenon of the corrected pixels being overheated or too cold. Achieve the color balance of multi-source image.

The multi-source color balance algorithm described above can further perform an over-saturation processing, including:

Set a supersaturation condition;

Marking the pixel whose initial color temperature meets the supersaturation condition is a supersaturation point;

An expansion process is performed to reset the initial estimated color temperature of the supersaturation point.

The above multi-source color balance algorithm, performing expansion processing includes:

Setting an expansion matrix;

Centering on the marked supersaturation point, extending a search range according to the expansion matrix;

Search for the highest color temperature in the search range;

The initial estimated color temperature value for setting the supersaturation point is the highest color temperature.

In the multi-source color balance algorithm described above, the step of performing the source group calculus includes:

Setting the number of clusters of the light source and a spatial function;

Estimating the central value and the attribution function, calculating a complex center value corresponding to the number of the light source groups according to the number of the light source groups, and calculating a complex attribution function according to the central value, the number of the light source group, and the initial estimated color temperature of the complex pixel;

Performing a cyclotron integral process, performing a convolution integral operation on the complex attribution function and the spatial function according to the number of the light source group;

Calculating the determined color temperature of each pixel, multiplying the result of the home function and the spatial function by the convolution integral with the center value, and summing according to the number of the light source groups, and obtaining the determined color temperature corresponding to each pixel;

The color balance correction value is calculated, and the color balance correction value of the complex array is obtained according to the determined color temperature of each pixel.

The color space described above is an RGB color space or a YCbCr color space.

The above-mentioned light source estimation calculus may be a Type-1 fuzzy inference calculation system, or may be a Type-2 TSK fuzzy inference calculation system.

The Type-1 fuzzy inference calculation system includes an input layer, a fuzzification layer, a regular layer and an output layer, wherein: the input layer respectively inputs a color space of the original image, and transmits the pixel to the fuzzification a layer; the fuzzification layer performs a fuzzification operation on the input pixel of the original image to calculate the center value and the attribution function; the rule layer performs an AND operation on the result of the fuzzification layer; and the output layer performs A de-fuzzification operation outputs the initial estimated color temperature of the complex number corresponding to the complex pixel.

The Type-2 TSK fuzzy inference calculation system includes an input layer, a fuzzification layer, a rule layer, a TSK function layer, a KM function layer and an output layer, wherein: the input layer respectively inputs the color of the original image Space, and transmitting the pixel to the fuzzification layer and the TSK function layer; the fuzzification layer performs a fuzzification operation on the input pixel of the original image to calculate the center value and the attribution function, Type-2 TSK There are two attribution functions and the central value, so the output of the layer can output two central values and the attribution function respectively above and below the lower bound; the rule layer performs an AND operation on the result of the fuzzification layer, in this embodiment, and the fuzzification The layers are the same, and the output is also the upper and lower bounds; the TSK function layer converts the color space into a single output; the KM computing layer sorts the output of the TSK function layer from small to large, and then outputs the upper and lower bounds of the third layer. The calculation of the KM algorithm; the output layer performs a defuzzification operation, and outputs the complex initial color temperature corresponding to the complex pixels.

The invention relates to a multi-source color balance algorithm, which mainly inputs an original image into a pixel-to-color balance algorithm one by one in a color space format, and estimates the corresponding correspondence by the source group calculus included in the color balance algorithm. Color temperature of complex pixels . If the color temperature of 6500K is used as the standard, the R ₆₅ \G ₆₅ \B ₆₅ value of the standard light source, in addition to the color temperature R\G\B value, get the corresponding color temperature, a total of n color balance correction values The R _I , G _I , and B _I values of the pixels of the original image are multiplied by the color balance correction value to obtain an output image, as shown in Equation 1. 【Formula 1】

Referring to FIG. 1 , the multi-source color balance algorithm of the present invention mainly includes the following steps:

Inputting the original image, inputting an original image in a color space, and the original image includes a plurality of pixels;

Performing a light source estimation calculation to calculate the initial color temperature of the complex pixel, please further refer to FIG. 2A, taking a color temperature of 5*5 pixels as an example, this step preliminarily estimates the initial estimated color temperature of each pixel. However, the results show that the initial estimated color temperature estimated by this step does not have several obvious peaks, that is, the initial estimated color temperature of the complex pixels is still scattered, so further steps are needed;

Performing a light source grouping calculation, setting a number of light source groups, and grouping according to the initial color temperature of the plurality of pixels and the number of the light source groups, and calculating a corresponding complex number to determine the color temperature, and then determining the color temperature according to the complex number, and estimating the table by the table method The complex array color balance correction value corresponding to the complex pixels in the original image, the detailed steps, please explain later;

The image compensation operation is performed by multiplying the plurality of pixels by a corresponding color balance correction value to output a corrected image after color balance.

The multi-source color balance algorithm color balance method may further perform an over-saturation processing, set an over-saturation condition, and perform the light source estimation calculation to obtain the initial estimated color temperature corresponding to each pixel:

The pixel whose initial color temperature is consistent with the supersaturation condition is a supersaturation point, as shown in FIG. 2A, the setting 6300 is a supersaturation point, and the pixel corresponding to the initial estimated color temperature 6385 will be marked as a supersaturation point;

Perform an expansion process to reset the initial estimated color temperature of the supersaturation point. The detailed steps are explained later.

In this embodiment, the supersaturation condition may be set to: in the RGB space, the R value is greater than 225 and the G value is greater than 225 and the B value is greater than 225, or the R value in the RGB space is less than 40 and the G value is less than 40 and the B value is less than 40. .

In the multi-source color balance algorithm described above, the expansion process is performed to set an expansion matrix. As described below with reference to FIGS. 2A and 2B, this embodiment sets a unique 5*5 matrix of the expansion matrix, and performs the following after marking the supersaturation point. step:

Centering on the marked supersaturation point, extending a search range according to the expansion matrix, that is, setting the 5*5 matrix as a search range;

Searching for a highest color temperature in the search range. In the embodiment of FIG. 2A, 4385 is found as the highest color temperature;

The initial estimated color temperature value of the supersaturation point is set to the highest color temperature, as shown in Fig. 2B, that is, the initial estimated color temperature of the supersaturation point is set to 4385.

In the multi-source color balance algorithm described above, the steps of performing the source group calculus include:

A light source grouping number C and a spatial function h _{ij are set} , wherein the light source grouping number C represents the number of ambient light sources, and the spatial function is a matrix, wherein N _ij is a matrix of 5*5 range centered on the ij pixel, k a attribution function other than the center point in the range, as shown in the following formula 2; [Equation 2]

Estimating the central value v _i and the attribution function u _ij , performing light source grouping according to the number of clusters of the light source by fuzzy grouping method, which mainly calculates a complex center value v _i corresponding to the number of clusters of the light source, and according to the central value, the light source The complex number attribute function u _ij calculated by the number of clusters and the initial estimated color temperature of the complex pixel is represented by Equations 3 and 4, respectively; n is the total number of pixels of the input image [Equation 3] [Formula 4]

Performing a cyclotron integral process, performing a convolution integral operation on the complex number attribution function and the spatial function h _ij according to the number of the light source group, and the complex attribution function is represented by the following formula 5 after the convolution integral processing; [Equation 5]

Calculating the determined color temperature L _{e of} each pixel, multiplying the result of the rotation of the attribution function u _ij and the spatial function h _ij by the center value v _i , and summing according to the number of light source groups, and obtaining the determined color temperature L _e corresponding to each pixel Determining the color temperature L _e as expressed by the following formula 6; [Equation 6]

The color balance correction value is calculated, and the color balance correction value of the complex array is obtained according to the determined color temperature of each pixel.

The above-mentioned light source estimation calculus may be a Type-1 fuzzy inference calculation system, or may be a Type-2 TSK fuzzy inference calculation system.

Please further refer to FIG. 3, the Type-1 fuzzy inference calculation system includes an input layer, a fuzzification layer, a regular layer and an output layer, wherein: the input layer respectively inputs the color space of the original image, and The pixel is transmitted to the fuzzification layer; the fuzzification layer performs a fuzzification operation on the input pixel of the original image to calculate the center value and the attribution function; the rule layer performs an AND on the result of the fuzzification layer And the output layer performs a defuzzification operation, and outputs the complex initial color temperature corresponding to the complex pixel respectively.

The parameters indicated in Figure 3 are as follows: R _i , G _i , B _i are the R, G, and B values of the input pixel, and β is the rule number. The color temperature is initially estimated for the plural of the image to be processed.

Please further refer to FIG. 4, the Type-2 TSK fuzzy inference calculation system includes an input layer, a fuzzification layer, a rule layer, a TSK function layer, a KM function layer and an output layer, wherein: the input layer Inputting the color space of the original image separately, and transmitting the pixel to the fuzzification layer and the TSK function layer; the fuzzification layer performs a fuzzification operation on the input pixel of the original image to calculate the center value and The attribution function, Type-2 TSK has two attribution functions and center values, so the output of this layer can output two center values and the attribution function respectively above and below the lower bound; the rule layer performs an AND operation on the result of the fuzzification layer. In the embodiment, the same as the fuzzification layer, the output is also the upper and lower bounds; the TSK function layer converts the color space into a single output; the KM computing layer sorts the output of the TSK function layer from small to large, and then The lower bound output of the three layers performs the calculation of the KM algorithm; the output layer performs a defuzzification operation, and outputs the complex estimated initial color temperature corresponding to the complex pixels.

The parameters indicated in Figure 4 are as follows: R _(m,n) , G _(m,n) , B _(m,n) are the R, G, B values of the coordinates (m,n) pixels, and q ⁽ⁱ⁾ is The output, superscript indicates the output at the ith layer, and T _{(m, n)} is the initial estimated color temperature of the coordinates (m, n) pixels.

Please further refer to Figure 5, through the implementation, the original image is captured under the environment of the color light 6500K and 3000K mixed light source. Through the multi-light source color balance algorithm processing, the light source grouping number C is set to 2 After the grouping and estimation by the light source grouping calculation, the complex color determining color temperature value output is shown in FIG. 4, which is obviously concentrated on two peaks, which is close to the actual shooting environment light source, and is confirmed by the above light source. The group calculus can accurately estimate the various ambient light sources of the actual captured image, and the number of ambient light sources can be set by the user, not limited to two.

According to the multi-source color balance algorithm of the present invention for image correction, the user may first set the number of light source groups to the number of ambient light sources. For example, in this embodiment, two ambient light sources are set as the number of light source groups, and the first and the light source grouping calculation are performed. The method estimates the color temperature corresponding to the complex pixel as shown in FIG. 5, and then uses the look-up table method to obtain the complex color balance correction values corresponding to the complex pixels, and finally, multiplies the complex pixel by the corresponding color balance correction value. The color-balanced image can be obtained, so that the corrected pixel can be caused by excessive or too cold color shift phenomenon, and the color balance of the multi-light source image can be achieved.

In summary, the multi-source color balance algorithm of the present invention has the following advantages:

The user can set the number of ambient light sources to estimate the color temperature of the pixels, and individually perform color balance according to the color temperature of the pixels to achieve the color balance of the multi-source image.

Can be applied to a variety of digital image capture tools (such as digital cameras, cameras, etc.), can also be used for offline operations (post-production images), instead of manual retouching.

The use of spatial functions can further improve the accuracy of color temperature grouping.

no.

Figure 1 is a schematic flow chart of the present invention. Fig. 2A is a schematic view showing the preliminary estimation of color temperature of the present invention. Figure 2B is a schematic diagram of the processing of the saturation point of Figure 2A. Figure 3 is a schematic illustration of one preferred embodiment of a light source estimation algorithm architecture. 4 is a schematic diagram of another preferred embodiment of a light source estimation algorithm architecture. Figure 5: Schematic diagram of the distribution of color temperature after the clustering of the light source.

Claims (8)

A multi-source color balance algorithm includes the following steps: inputting an original image, inputting an original image in a color space, and the original image includes a plurality of pixels; performing a light source estimation calculation to calculate an initial estimated color temperature of the plurality of pixels; Performing a light source estimation calculation to obtain an initial estimated color temperature corresponding to each pixel, and performing an oversaturation point processing, the supersaturation point processing includes: setting a supersaturation condition; marking a pixel whose initial color temperature is consistent with the supersaturation condition is a supersaturation point; Performing an expansion process to reset the initial estimated color temperature of the supersaturation point; wherein performing the expansion process comprises: setting an expansion matrix; extending a search range according to the expansion matrix centered on the marked supersaturation point; Searching for a highest color temperature in the search range; and setting an initial color temperature of the supersaturation point to the highest color temperature; performing a light source grouping calculation, setting a light source grouping number according to the initial color temperature of the plurality of pixels and the number of the light source grouping Grouping and calculating the corresponding complex numbers to determine the color temperature, and then determining according to the complex number Temperature, the complex array color balance correction value corresponding to the plurality of pixels in the original image is estimated by the look-up table method; the image compensation operation is performed, and the plurality of pixels are respectively multiplied by the corresponding color balance correction values to output the color balanced A corrected image. The multi-source color balance algorithm according to claim 1, the step of performing the source group calculus includes: setting the number of the source group and a spatial function; Estimating the central value and the attribution function, calculating a complex center value corresponding to the number of clusters of the light source according to the number of clusters of the light source, and calculating a complex attribution function according to the central value, the number of the light source clusters, and the initial estimated color temperature of the complex pixel; performing a swing integral processing According to the number of the light source group, the complex attribute function and the spatial function are subjected to a convolution integral operation; calculating the determined color temperature of each pixel, multiplying the result of the home function and the spatial function by the convolution integral with the center value, and grouping according to the light source The number is summed to obtain a determined color temperature corresponding to each pixel; the color balance correction value is calculated, and the color balance correction value of the complex array is obtained by comparing the determined color temperature value of each pixel with a lookup table. The multi-source color balance algorithm as claimed in claim 1, wherein the color space is an RGB color space or a YCbCr color space. The multi-source color balance algorithm as claimed in claim 2, wherein the color space is an RGB color space or a YCbCr color space. The multi-source color balance algorithm according to claim 1, wherein the light source estimation algorithm is a Type-1 fuzzy inference calculation system, which comprises an input layer, a fuzzification layer, a rule layer and an output layer, wherein: The input layer respectively inputs the color space of the original image, and transmits the pixel to the fuzzification layer; the fuzzification layer performs a fuzzification operation on the input pixel of the original image to calculate the center value and the attribution function; The rule layer performs an AND operation on the result of the fuzzification layer; and the output layer performs a defuzzification operation to output the initial estimated color temperature corresponding to the complex pixels. The multi-source color balance algorithm according to claim 4, wherein the light source estimation algorithm is a Type-1 fuzzy inference calculation system, comprising an input layer, a fuzzification layer, a rule layer and an output layer, wherein: The input layer respectively inputs the color space of the original image, and transmits the pixel to the fuzzification layer; the fuzzification layer performs a fuzzification operation on the input pixel of the original image to calculate the The central value and the attribution function; the rule layer performs an AND operation on the result of the fuzzification layer; and the output layer performs a defuzzification operation to output the initial estimated color temperature corresponding to the complex pixel respectively. The multi-source color balance algorithm according to claim 1, wherein the light source estimation algorithm is a Type-2 TSK fuzzy inference calculation system, comprising: an input layer, a fuzzification layer, a rule layer, and a TSK function layer. a KM function layer and an output layer, wherein: the input layer respectively inputs a color space of the original image, and transmits the pixel to the fuzzification layer and the TSK function layer; the fuzzification layer inputs the original The pixel of the image is subjected to a fuzzification operation to calculate the center value and the attribution function. The Type-2 TSK has two attribution functions and a central value, so the output of the layer can output two center values and a attribution function respectively above and below the lower bound; The rule layer performs an AND operation on the result of the fuzzification layer. In this embodiment, the same as the fuzzification layer, which is also an upper and lower bound output; the TSK function layer converts the color space into a single output; the KM computing layer system The output of the TSK function layer is sorted from small to large, and then the KM algorithm is calculated with the lower bound output of the third layer; the output layer performs a defuzzification operation, and outputs the complex pixel corresponding to the complex Preliminary estimates suggest that the color temperature. The multi-source color balance algorithm according to claim 4, wherein the light source estimation algorithm is a Type-2 TSK fuzzy inference calculation system, comprising: an input layer, a fuzzification layer, a rule layer, and a TSK function layer a KM function layer and an output layer, wherein: the input layer respectively inputs a color space of the original image, and transmits the pixel to the fuzzification layer and the TSK function layer; the fuzzification layer inputs the original The pixel of the image is subjected to a fuzzification operation to calculate the center value and the attribution function. The Type-2 TSK has two attribution functions and a central value, so the output of the layer can output two center values and a attribution function respectively above and below the lower bound; The rule layer performs an AND operation on the result of the fuzzification layer. In this embodiment, the same as the fuzzification layer, which is also an upper and lower bound output; the TSK function layer converts the color space into a single output; the KM computing layer system The output of the TSK function layer is sorted from small to large, and then the KM algorithm is calculated with the lower bound output of the third layer; the output layer performs a defuzzification operation, and outputs the complex pixel corresponding to the complex Preliminary estimates suggest that the color temperature.