CN106295679B - A kind of color image light source colour estimation method based on category correction - Google Patents

Abstract

The color image light source colour estimation method based on category correction that the invention discloses a kind of, the edge feature of image is extracted first on the image of one group of known luminaire color, then learnt by least square method, obtain the correction matrix between edge feature and light source, edge feature is extracted to test image to be processed again and is multiplied with correction matrix, rough light source estimation is obtained；Accurately light source estimation is obtained to relearn with one kind training image similar in test image feature to be processed finding in such a way that feature space finds K width adjacent image later.It is few that the present invention relates to parameters, and since the feature of extraction is simple and negligible amounts, so also possessing the features such as calculating is simple, speed is fast；In addition, the present invention is the method based on study, so high treating effect, accuracy is high, is very suitable for the relatively high occasion of the accuracy of estimation requirement to light source colour.

Description

A Color Image Light Source Color Estimation Method Based on Classification Correction

technical field

The invention belongs to the technical field of computer vision and image processing, in particular to the design of a color image light source color estimation method based on classification correction.

Background technique

In the natural environment, the same object will show different colors under the illumination of different colors of light. For example, green leaves are yellowish in the morning light, but bluish in the evening. The human visual system can resist the color change of the light source, so as to constantly perceive the color of the object, that is, the visual system has color constancy. However, due to technical limitations, machines do not have this capability, and pictures captured by physical devices, such as cameras, will have serious color shifts due to changes in the color of the light source. Therefore, how to accurately estimate the color of the light source in the scene and remove it according to the existing image information to obtain the color of the object under the illumination of standard white light is particularly important.

Computational color constancy is dedicated to solving this problem. Its main purpose is to calculate the color of the unknown light source contained in any image, and then use the calculated light source color to perform light source color correction on the original input image. Displayed under standard white light, a so-called standard image is obtained. Since the standard image removes the influence of the color of the light source, for subsequent computing tasks, such as color-based scene classification, there is no mis-classification or mis-retrieval caused by color casts in image retrieval.

Computational color constancy methods can be divided into two categories: learning-based methods and traditional static methods. Traditional static methods extract simple features from images for light source estimation, but these methods cannot meet engineering needs well due to large estimation errors. Based on this requirement, the learning-based method was born on the basis of the traditional non-learning-based method. The more typical learning-based method is the method proposed by G.D.Finlayson in 2013. Reference: G.D.Finlayson, "Corrected-moment illuminant estimation "in Proc.Comput.Vis.IEEEInt.Conf., 2013, pp.1904–1911, this method finds the relationship between features and light sources through feature extraction and regression. Since this method uses regression, the accuracy of estimating the light source is relatively high, but this method uses the same correction matrix for all images, resulting in a large error in the light source estimated for some images, so it cannot satisfy the estimation of the color of the light source. In applications where high accuracy is required, such as the front end of the device that receives images, such as intelligent robots or autonomous driving. Therefore, it is particularly important to implement a method of learning different correction matrices for different images.

SUMMARY OF THE INVENTION

The purpose of the present invention is to solve the problem that the prior art method for estimating the light source color of an image scene cannot meet the situation where the accuracy of the estimated light source color is very high, and proposes a color image light source color estimation method based on classification correction.

The technical scheme of the present invention is: a color image light source color estimation method based on classification correction, comprising the following steps:

S1. Extract the edge features of the training image: Take N color images of known light sources as the original training set T, and perform convolution operations with the template G after the derivation of the Gaussian distribution to obtain the edge value corresponding to each pixel of the image, Extract edge features to obtain edge feature matrix M of N training images;

S2, learning the correction matrix: through the least squares method, learn the correction matrix C between the feature matrix M calculated in step S1 and the standard light source L of the N training images;

S3, rough light source estimation: using the method in step S1 to extract the edge feature of the test image, and multiplying it with the correction matrix C learned in step S2 to obtain a rough light source estimation result L1;

S4. Find the training image corresponding to the test image: remove the light source from the test image and the original training set T respectively, and then use the method in S1 to extract the edge features respectively to form a feature space; find the features similar to the test image in the feature space The K training images are used as the new training set TN;

S5. Accurate light source estimation: Repeat steps S1-S4, each time the training set T in step S1 is replaced with the new training set TN obtained in step S4, and the number of training images is correspondingly changed from N to K, until step S5. The TN obtained in S4 is the same as the TN obtained in step S4 in the previous operation, and the light source estimation result L1 obtained in step S3 in the last operation is used as the final light source estimation result.

Further, the template G obtained from the derivation of the Gaussian distribution in step S1 is a Gaussian gradient operator.

Further, the calculation formula for extracting edge features in step S1 is:

In the formula, R _i , G _i , and B _i represent the edge values of each pixel in the R, G, and B channels, respectively, N ₁ represents the number of pixels in the image, and M _xyz is the difference between x, y, and z. The values of the corresponding edge features, x, y, and z are all combinations that satisfy x≥0, y≥0, z≥0 and x+y+z=3.

Further, the value range of K in step S4 is

Further, step S4 specifically includes the following sub-steps:

S41, removing the standard light source L from the original N training images, and using the method in step S1 to extract edge features;

S42, remove the light source L1 roughly estimated in step S3 to the test image, and adopt the method in step S1 to extract edge features, and form a feature space together with the edge features extracted from N pieces of training images in step S41;

S43 , find out the K images that are closest to the feature distance of the test image in the feature space, and use them as a new training image set TN of the test image.

Further, the feature distance in step S43 is the Euclidean distance.

The beneficial effects of the present invention are as follows: the present invention first extracts the edge features of the images from a group of images with known light source colors, and then learns through the least squares method to obtain the correction matrix between the edge features and the light source, and then tests the to-be-processed test. The edge features are extracted from the image and multiplied by the correction matrix to obtain a rough light source estimate; then, a class of training images similar to the characteristics of the test image to be processed is found by searching for K adjacent images in the feature space, so as to relearn and obtain accurate Light source estimation. Since the distance between the test image to be processed and the training image in the feature space is different, by appropriately adjusting the value of the number K corresponding to the training image, a better result suitable for different types of training images can be obtained, where K is the only parameter. The present invention involves few parameters (only one parameter K), and because the extracted features are simple and small in number, it also has the characteristics of simple calculation and fast speed; in addition, the present invention is based on a learning method, so the processing effect is good, High accuracy, it is very suitable for occasions where the estimation accuracy of light source color is relatively high, such as built-in intelligent robots or the front end of automatic driving image receiving equipment.

Description of drawings

FIG. 1 is a flow chart of a color image light source color estimation method based on classification correction provided by the present invention.

FIG. 2 is a to-be-processed test image tools_ph-ulm.tif according to Embodiment 1 of the present invention.

FIG. 3 is a schematic diagram of the error value between the estimated light source and the real light source in each step of Embodiment 1 of the present invention.

FIG. 4 is a schematic diagram showing the comparison between the final light source estimation result and the real light source according to the first embodiment of the present invention.

FIG. 5 is a schematic diagram of the result of performing tone correction on the original test image by using the color value of the light source calculated in step S5 according to the second embodiment of the present invention.

Detailed ways

The embodiments of the present invention will be further described below with reference to the accompanying drawings.

The present invention provides a color image light source color estimation method based on classification correction, as shown in FIG. 1 , including the following steps:

S1. Extract the edge features of the training image: Take N color images of known light sources as the original training set T, and perform convolution operations with the template G after the derivation of the Gaussian distribution to obtain the edge value corresponding to each pixel of the image, Extract edge features to obtain edge feature matrix M of N training images.

Among them, the template G after the derivation of the Gaussian distribution is the Gaussian gradient operator.

The calculation formula for extracting edge features is:

In the formula, R _i , G _i , and B _i represent the edge values of each pixel in the R, G, and B channels, respectively, N ₁ represents the number of pixels in the image, and M _xyz is the difference between x, y, and z. The values of the corresponding edge features, x, y, z are all combinations that satisfy x≥0, y≥0, z≥0 and x+y+z=3, the total number of combinations is 19, so 19 can be obtained here edge features.

S2. Learning the correction matrix: learn the correction matrix C between the feature matrix M calculated in step S1 and the standard light source L of the N training images through the least square method.

S3. Rough light source estimation: use the method in step S1 to extract the edge features of the test image, and multiply with the correction matrix C learned in step S2 to obtain a rough light source estimation result L1.

S4. Find the training image corresponding to the test image: remove the light source from the test image and the original training set T respectively, and then use the method in S1 to extract the edge features respectively to form a feature space; find the features similar to the test image in the feature space K training images (the value range of K is ) as the new training set TN.

This step specifically includes the following sub-steps:

S41 , remove the standard light source L from the original N training images, and extract edge features by using the method in step S1 .

S42, remove the light source L1 roughly estimated in step S3 from the test image, and extract edge features using the method in step S1, and form a feature space together with the edge features extracted from the N training images in step S41.

S43 , find out the K images that are closest to the feature distance of the test image in the feature space, and use them as a new training image set TN of the test image.

Wherein, the characteristic distance in step S43 is the Euclidean distance.

S5. Accurate light source estimation: Repeat steps S1-S4, each time the training set T in step S1 is replaced with the new training set TN obtained in step S4, and the number of training images is correspondingly changed from N to K, until step S5. The TN obtained in S4 is the same as the TN obtained in step S4 in the previous operation, and the light source estimation result L1 obtained in step S3 in the last operation is used as the final light source estimation result.

The final light source estimation result L1 of the image calculated after step S5 can be directly used for subsequent computer vision applications. For example, dividing the component of each color channel of the input primary color image by L1 can remove the color of the light source in the color image. the goal of. In addition, the white balance and color correction of the image also need to use the final light source estimation result L1 calculated by S5.

A method for estimating light source color of a color image based on classification correction provided by the present invention is further described below with a specific embodiment:

Example 1:

Download all the images (321 in total) of the currently internationally recognized image library SFU object for estimating the color of the scene light source and their corresponding real light source color (standard light source) L. The image size is 468×637, and the first 214 images in the image library are selected. tif (as shown in Figure 2) is selected as the test image to be processed for testing, and all images have not undergone any preprocessing of the camera itself (such as tone correction) , gamma correction). Then the detailed steps of the present invention are as follows:

S1. Extract the edge features of the training image: Take 214 color images of known light sources as the original training set T, and perform convolution operations with the template G (Gaussian gradient operator) after the derivation of the Gaussian distribution to obtain each pixel of the image. The edge value corresponding to the point, and then extract the 19-dimensional edge features, and finally obtain the edge feature matrix M of the training set image with a size of 214*19.

S2. Learning the correction matrix: Through the least squares method, learn the correction matrix between the feature matrix M calculated in step S1 and the standard light source L of the 214 training images, and obtain a correction matrix C with a size of 19*3:

C＝[-150.0689，-30.1462，-21.5186；-96.5582，-196.1642，-348.5298；52.6551，76.4461，115.5982；-200.5289，-240.3650，-179.6495；-79.6311，72.4539，125.1126；-56.1276，-130.2963，- 226.1518；683.9180，552.9035，366.8781；214.1444，-15.5379，-52.8198；-149.3407，138.0260，397.1888；154.6218，240.3336，128.2161；156.5752，-50.6503，69.4182；22.7103，90.3730，274.5781；-65.9786，-384.7642，-66.2556 ；-112.7044，-104.0913，-12.8868；-349.7427，81.5115，-215.8972；-79.0109，-48.0727，-32.2072；-98.2723，-22.7039，-51.2091；108.6481，-52.0896，-265.9989；172.6056，171.2726，95.1991] .

S3. Rough light source estimation: The method in step S1 is used to extract the 19-dimensional edge features of the test image, and the edge feature matrix M1 of the test image with a size of 1*19 is obtained:

M1 = [0.0002, 0.0004, 0.0002, 0.0014, 0.0017, 0.0012, 0.0015, 0.0013, 0.0014, 0.0036, 0.0040, 0.0031, 0.0037, 0.0034, 0.0039, 0.0037, 0.0032, 0.0].

Then multiply M1 by the correction matrix C learned in step S2 to obtain a rough light source estimation result L1=[0.1985, 0.2151, 0.2360].

S4. Find the training image corresponding to the test image: remove the light source from the test image and the original training set T with 214 images respectively, and then use the method in S1 to extract 19-dimensional edge features to form a feature space. Find K training images with similar characteristics to the test image in the feature space, so as to obtain a class of images with similar characteristics, and use these K images as a new training set TN. In the embodiment of the present invention, K=100 is selected.

This step specifically includes the following sub-steps:

S41 , remove the standard light source L from the original 214 training images, and extract 19-dimensional edge features by using the method in step S1 to obtain a feature matrix M0 with a size of 214*19.

S42, remove the light source L1 roughly estimated in step S3 from the test image, and use the method in step S1 to extract edge features to obtain a feature matrix M2 with a size of 1*19:

M2 = [0.0060, 0.0089, 0.0038, 0.0366, 0.0371, 0.0224, 0.0365, 0.0282, 0.0285, 0.0937, 0.0900, 0.0581, 0.0921, 0.0790, 0.0909, 0.0768, 0.0673, 0.0664, 0.0673].

Then, M2 and the edge features M0 extracted from the 214 training images in step S41 together form a feature space.

S43 , find out 100 images with the closest feature distance to the test image in the feature space, and use them as a new training image set TN of the test images.

S5. Accurate light source estimation: Repeat steps S1-S4, each time the training set T in step S1 is replaced with the new training set TN obtained in step S4, and the number of training images is correspondingly changed from N to K, until step S5. The TN obtained in S4 is the same as the TN obtained in step S4 in the previous operation, and the light source estimation result L1 obtained in step S3 in the last operation is used as the final light source estimation result.

In the embodiment of the present invention, in order to save time, the operation may be repeated twice. The light source estimation result obtained after repeating the operation once is L1=[0.3412, 0.3591, 0.3168], and the light source estimation result obtained after repeating the operation twice is L1=[0.3312, 0.3365, 0.3430]. The light source estimation result L1=[0.3312, 0.3365, 0.3430] obtained after executing twice is used as the final light source estimation result.

As shown in Figure 3, the first column represents the angle error value between the roughly estimated light source and the real light source in step S3, and the second column is the angle error between the estimated light source and the real light source after repeating the operation in step S5 once value, the third column is the angle error value between the estimated light source and the real light source after repeating the operation twice in step S5. The broken line connecting the three columns reflects the decreasing trend of estimation error, indicating that the estimated light source is getting more and more accurate.

Figure 4 shows the direction of the response of the red and green components in the three primary color space finally calculated in step S5 and the direction of the response of the red and green components of the real light source. Figure 4 shows the response value calculated in step S5 and the real scene light source. The color information is close.

The following is a simple demonstration of the actual application of the light source estimation result finally obtained by the present invention, taking the tone correction of an image as an example with a specific embodiment:

Embodiment 2:

The pixel value of each color component of the original input image is corrected respectively by using the light source color value under each color component calculated in step S5. Taking a pixel (0.335, 0.538, 0.601) of the test image input in step S3 as an example, the corrected result is (0.335/0.3312, 0.538/0.3365, 0.601/0.3430)=(1.0115, 1.5988, 1.7522), After normalization, it becomes (0.2319, 0.3665, 0.4016), and then the corrected value is multiplied by the standard white light coefficient Obtain (0.1339, 0.2116, 0.2319) as the pixel value of the final output corrected image, and do similar calculations for other pixels of the original input image, and finally obtain the corrected color image, as shown in Figure 5.

Those of ordinary skill in the art will appreciate that the embodiments described herein are intended to assist readers in understanding the principles of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations without departing from the essence of the present invention according to the technical teaching disclosed in the present invention, and these modifications and combinations still fall within the protection scope of the present invention.

Claims (6)

1. a kind of color image light source colour estimation method based on category correction, which comprises the following steps:

S1, extract training image edge feature: using the color image of N width known luminaire as original training set T, respectively with height Template G after this distribution derivation does convolution algorithm, obtains the corresponding marginal value of each pixel of image, extracts edge feature, obtain To the edge feature matrix M of N width training image；

S2, learning correction matrix: by least square method, study is schemed by the step S1 eigenmatrix M being calculated and the training of N width Correction matrix C between the standard sources L of picture；

S3, rough light source estimation: the edge feature of test image is extracted using the method in step S1, is learnt with step S2 The correction matrix C arrived is multiplied, and obtains rough light source estimated result L1；

S4, searching training image corresponding with test image: removing light source to test image and original training set T respectively, then Edge feature is extracted using the method in S1 respectively, forms feature space；It is found out in feature space with test image feature most Similar K width training image, as new training set TN；

S5, accurately light source are estimated: repeating step S1-S4, replace in step S4 the training set T in step S1 obtain every time New training set TN, training image number also becomes K from N accordingly, in TN and last operation obtained in the step S4 Until the TN that step S4 is obtained is identical, using the light source estimated result L1 that step S3 is obtained in last time operation as final light source Estimated result.

2. the color image light source colour estimation method according to claim 1 based on category correction, which is characterized in that institute Stating the template G in step S1 after Gaussian Profile derivation is Gauss gradient operator.

3. the color image light source colour estimation method according to claim 1 based on category correction, which is characterized in that institute State the calculation formula that edge feature is extracted in step S1 are as follows:

R in formula_i、G_i、B_iEach pixel is respectively indicated in the marginal value in tri- channels R, G, B, N₁Indicate pixel in image Number, M_xyzFor different x, the value of corresponding edge feature under y, z, x, y, z is to meet x >=0, y >=0, z >=0 and x+y+z=3's All combinations.

4. the color image light source colour estimation method according to claim 1 based on category correction, which is characterized in that institute The value range for stating K in step S4 is

5. the color image light source colour estimation method according to claim 1 based on category correction, which is characterized in that institute State step S4 specifically include it is following step by step:

S41, standard sources L is removed to original N width training image, and edge feature is extracted using the method in step S1；

S42, the light source L1 that rough estimate in step S3 is removed to test image, and edge spy is extracted using the method in step S1 Feature space is collectively formed in sign, the edge feature extracted with N width training image in step S41；

S43, it is found out in feature space with the most similar K width image of test image characteristic distance, new instruction as test image Practice image collection TN.

6. the color image light source colour estimation method according to claim 5 based on category correction, which is characterized in that institute Stating the characteristic distance in step S43 is Euclidean distance.