CN112150563B - Method and device for determining light source color, storage medium and electronic equipment - Google Patents
Method and device for determining light source color, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN112150563B CN112150563B CN201910576317.2A CN201910576317A CN112150563B CN 112150563 B CN112150563 B CN 112150563B CN 201910576317 A CN201910576317 A CN 201910576317A CN 112150563 B CN112150563 B CN 112150563B
- Authority
- CN
- China
- Prior art keywords
- light source
- determining
- class
- target
- dimension
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 78
- 238000012549 training Methods 0.000 claims abstract description 62
- 238000010801 machine learning Methods 0.000 claims abstract description 26
- 239000006185 dispersion Substances 0.000 claims description 35
- 238000013507 mapping Methods 0.000 claims description 20
- 230000008569 process Effects 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 239000003086 colorant Substances 0.000 abstract description 12
- 230000000694 effects Effects 0.000 abstract description 7
- 230000002708 enhancing effect Effects 0.000 abstract description 5
- 239000002609 medium Substances 0.000 description 13
- 238000001028 reflection method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000012896 Statistical algorithm Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 239000012120 mounting media Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Abstract
The embodiment of the application discloses a method and a device for determining the color of a light source, a storage medium and electronic equipment. The method comprises the following steps: inputting the target image into a machine learning model, and determining the spatial position of the target image in a prediction space; the number of dimensions of the prediction space is determined according to the number of selectable values of each parameter in the light source color algorithm; determining a matching sample class according to the spatial position of the target image in the prediction space and the class center position of a predetermined sample class; obtaining a target dimension of a matching sample class, and determining a target parameter value corresponding to the target dimension according to the target dimension; substituting the target parameter value into a light source color algorithm to calculate the light source color of the target image. By executing the technical scheme, the training can be performed under the condition of unknown light source colors of sample images by adopting an unsupervised learning mode, and the effects of improving the accuracy and usability of the light source colors and enhancing the adaptability of the light source color determining mode are achieved.
Description
Technical Field
The embodiment of the application relates to the technical field of image processing, in particular to a method and a device for determining the color of a light source, a storage medium and electronic equipment.
Background
In natural environments, the human visual system has the ability to resist color changes of light sources in a scene. For example, the ability of our vision system to perceive a scene that is always constant in color, whether in the morning yellow-tinted sun or in the other evening red-tinted sun, is also known as the color constancy of the vision system.
Currently common light source color estimation algorithms include Gray World method (Gray World), perfect reflection method (White-Patch), and the like, which are simple in principle and easy to implement, but have limited use scenes due to the fact that the assumption based on the methods is too strong. For example, for gray world methods, which assume that the statistical average of all pixel values in a color image is gray, when some scene such as a large area of solid color (e.g., large area yellow, blue, etc.) is encountered, the assumption of such algorithms is clearly not true, resulting in a serious deviation in the estimation of the light source color; the assumption of perfect reflection is that the color of the highlight in the picture represents the light source color, which also fails in some evenly lit scenes (no apparent highlights). Therefore, how to determine the light source color of the picture more accurately and to increase the applicable scenes of the determining process has become a technical problem to be solved by those skilled in the art.
Disclosure of Invention
The embodiment of the application provides a method, a device, a storage medium and electronic equipment for determining light source colors, which can realize training under the condition of unknown light source colors of sample images by adopting an unsupervised learning mode, and improve the accuracy and usability of determining high light source colors and the effect of enhancing the adaptability of the light source color determining mode.
In a first aspect, an embodiment of the present application provides a method for determining a color of a light source, where the method includes:
inputting the target image into a machine learning model, and determining the spatial position of the target image in a prediction space; the number of dimensions of the prediction space is determined according to the number of optional values of each parameter in a light source color algorithm;
determining a matching sample class according to the spatial position of the target image in the prediction space and the class center position of a predetermined sample class;
obtaining a target dimension of a matching sample class, and determining a target parameter value corresponding to the target dimension according to the target dimension;
substituting the target parameter value into a light source color algorithm to calculate the light source color of the target image.
Further, the machine learning model is configured to process training samples in the following manner:
Mapping all training samples in the training sample set into a prediction space;
according to the spatial position distribution of the training samples in the prediction space, carrying out clustering operation to obtain at least two sample classes and class centers of each sample class;
and determining the target dimension of each sample class according to a preset rule.
Further, determining the target dimension of each sample class according to a preset rule includes:
and determining the target dimension of each sample class according to the inter-class dispersion and/or intra-class dispersion of each sample class in the current dimension of the prediction space, traversing all dimensions and the inter-class dispersion and/or intra-class dispersion in each dimension.
Further, inputting the target image into a pre-trained machine learning model, and determining the spatial position of the target image in a prediction space includes:
inputting the target image into a light source color algorithm of a machine learning model, and substituting the light source color algorithm with optional values of parameters to obtain a light source calculated value which is used as a projection value of the target image in each dimension of a prediction space;
based on the projection values of the target image in each dimension in the prediction space, the spatial position in the prediction space is determined.
Further, the light source color algorithm includes: generalized gray edge algorithm.
In a second aspect, an embodiment of the present application provides a device for determining a color of a light source, where the device includes:
the target image input module is used for inputting the target image into a machine learning model and determining the spatial position of the target image in a prediction space; the number of dimensions of the prediction space is determined according to the number of optional values of each parameter in a light source color algorithm;
the matching sample class determining module is used for determining a matching sample class according to the spatial position of the target image in the prediction space and the class center position of the predetermined sample class;
the target parameter value determining module is used for obtaining the target dimension of the matching sample class and determining a target parameter value corresponding to the target dimension according to the target dimension;
and the light source color calculation module is used for substituting the target parameter value into a light source color algorithm to calculate the light source color of the target image.
Further, the device also comprises a model training module for:
the training sample mapping unit is used for mapping all training samples in the training sample set into a prediction space;
The sample class determining unit is used for carrying out clustering operation according to the spatial position distribution of the training samples in the prediction space to obtain at least two sample classes and class centers of each sample class;
and the target dimension determining unit is used for determining the target dimension of each sample class according to a preset rule.
Further, according to a preset rule, the target dimension determining unit is specifically configured to:
and determining the target dimension of each sample class according to the inter-class dispersion and/or intra-class dispersion of each sample class in the current dimension of the prediction space, traversing all dimensions and the inter-class dispersion and/or intra-class dispersion in each dimension.
In a third aspect, embodiments of the present application provide a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method for determining a color of a light source according to embodiments of the present application.
In a fourth aspect, an embodiment of the present application provides an electronic device, including a memory, a processor and a computer program stored on the memory and capable of being executed by the processor, where the processor executes the computer program to implement a method for determining a color of a light source according to an embodiment of the present application.
According to the technical scheme provided by the embodiment of the application, the target image is input into the machine learning model, and the spatial position of the target image in the prediction space is determined; the number of dimensions of the prediction space is determined according to the number of optional values of each parameter in a light source color algorithm; determining a matching sample class according to the spatial position of the target image in the prediction space and the class center position of a predetermined sample class; obtaining a target dimension of a matching sample class, and determining a target parameter value corresponding to the target dimension according to the target dimension; substituting the target parameter value into a light source color algorithm to calculate the light source color of the target image. By adopting the technical scheme provided by the application, the effect of improving the accuracy of determining the color of the light source and enhancing the adaptability of the light source color determination mode can be realized.
Drawings
Fig. 1 is a flowchart of a method for determining a color of a light source according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a light source color determining device according to a third embodiment of the present application;
fig. 3 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present application.
Detailed Description
The present application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the application and not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present application are shown in the drawings.
Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart depicts steps as a sequential process, many of the steps may be implemented in parallel, concurrently, or with other steps. Furthermore, the order of the steps may be rearranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, and the like.
Color constancy is a capability of eliminating interference of light source colors to achieve accurate perception of scene colors. Obtaining color constancy is of great importance for current machine vision applications such as image retrieval, image classification, object recognition and tracking, and the like.
The key to achieving color constancy capability for machine vision is the accurate estimation of scene illuminant color. However, in most applications, it is difficult to obtain the color information of the scene light source directly or in real time, and the available information is mainly the scene image captured by the camera in real time. However, as known from the Lambertian reflection model, the color of an image imaged by a camera is mainly determined by three factors, namely, the color of a light source, the reflection characteristic of a scene and the response characteristic of the camera, and although the response characteristic of the camera can be obtained through the calibration in advance, the back-deduction of the color of the light source by only image information still belongs to an ill-posed problem mathematically. To solve this problem, some constraints or assumptions need to be externally added.
Currently common light source color estimation algorithms include Gray World method (Gray World), perfect reflection method (White-Patch), and the like, which are simple in principle and easy to implement, but have limited use scenes due to the fact that the assumption based on the methods is too strong. For example, for gray world methods, which assume that the statistical average of all pixel values in a color image is gray, when some scene such as a large area of solid color (e.g., large area yellow, blue, etc.) is encountered, the assumption of such algorithms is clearly not true, resulting in a serious deviation in the estimation of the light source color; the assumption of perfect reflection is that the color of the highlight in the picture represents the light source color, which also fails in some evenly lit scenes (no apparent highlights). Essentially, the gray world method and the perfect reflection method are statistical algorithms based on image pixel values, and the algorithms have high sensitivity to scenes due to higher dependence on the image pixel values, and are reflected to have poor adaptability to the scenes. A generalized expression for such algorithms is the Shades of Gray (SoG) algorithm, expressed in terms of the minkowski norm:
in equation (1), f (x) is the pixel value of the image at coordinate x, p is the norm parameter, Is an estimated value of the color of the light source, k is a constant; when p is 1, the above method is gray world method, and when p is +.The above equation corresponds to a perfect reflection method. Therefore, the gray world method and the perfect reflection method are special cases of the SoG algorithm. The value of p in the SoG algorithm often has a larger influence on the effect of the algorithm, and the optimal value of p is generally considered to be 6.
On the other hand, the Gray Edge method (Gray Edge, GE) proposed by weijer et al in 2007, proposed an optimization direction at a higher level, i.e., extending the reference to image information from the original 0 th order statistical information to 1 st order or higher. Here, the 0 th order information represents the pixel value itself, the 1 st order information represents the first derivative value of the pixel value in the spatial domain, and the higher order information is the same. Research shows that the GE algorithm calculated by using 1-order or higher-order information of the image has higher precision and adaptability in most scenes compared with the SoG algorithm only using 0-order information; but for scenes with weaker details or textures, the GE algorithm tends to be less effective. Thus, both the SoG algorithm and the GE algorithm have their own shortcomings and advantages.
Example 1
Fig. 1 is a flowchart of a method for determining a light source color according to an embodiment of the present application, where the embodiment may be adapted to determine a light source color of an image so as to implement white balance correction of the image, and the method may be performed by a device for determining a light source color according to an embodiment of the present application, where the device may be implemented by software and/or hardware, and may be integrated into an electronic device such as an intelligent terminal.
As shown in fig. 1, the method for determining the color of the light source includes:
s110, inputting a target image into a machine learning model, and determining the spatial position of the target image in a prediction space; the number of dimensions of the prediction space is determined according to the number of selectable values of each parameter in the light source color algorithm.
The machine learning model may be pre-trained, and in this embodiment, the machine learning model may be obtained by using unsupervised training. I.e. a certain number of training samples can be used and the model can be obtained by training without knowing the light source color of the training samples. It will be appreciated that the training samples selected herein are color images.
The machine learning model may be input with a target image or features extracted from the target image in a preset manner, and the machine learning model may be output with a position of the target image in a prediction space. Wherein in the present embodiment, the prediction space may be two-dimensional, three-dimensional or even more dimensions. In this embodiment, the number of dimensions of the prediction space may be determined according to the number of selectable values of each parameter in the light source color algorithm. For example, in the light source color algorithm, two parameters are provided, the number of optional values of one parameter is 2, and the number of optional values of the other parameter is 3, so that the dimension in the prediction space can be determined to be six dimensions; similarly, if three parameters exist in the light source color algorithm, the selectable value of one parameter is 2, the selectable value of one parameter is 3, and the selectable value of the other parameter is 5, the number of dimensions of the prediction space can be determined to be thirty dimensions.
The spatial position of the target image in the prediction space is determined according to the light source color value obtained by the target image in each dimension, and a unique position in the space can be determined according to the projection values of all dimensions in the space, namely the light source color value in each dimension, and the unique position is taken as the spatial position of the target image in the prediction space.
S120, determining a matched sample class according to the spatial position of the target image in the prediction space and the class center position of the predetermined sample class.
The sample class can be obtained by clustering sample images in a prediction space, and a class center of each sample class can be obtained at the same time. The class center is the center position of the spatial position of the sample class in prediction space.
In the prediction space, after the spatial position of the target image is obtained, the matching sample class may be determined according to a predetermined distance between the class center position of the sample class and the spatial position of the target image. The matching sample class is one or more sample classes that are the most matched to the target image.
S130, obtaining a target dimension of the matching sample class, and determining a target parameter value corresponding to the target dimension according to the target dimension.
The target dimension of the matching sample class may be determined by the projection distribution characteristics of the samples in the sample class on each dimension, for example, a dimension with minimum projection distribution dispersion of the samples may be determined as the target dimension of the sample class. It will be appreciated that one or more target dimensions of the sample class may be determined, whether or not the sample class is determined to be a target sample class. In this embodiment, the matching sample class may be determined first and then the target dimension of the sample class may be determined, or the target dimensions of the sample classes may be determined while the sample class is determined. The target dimension of each sample class may be one dimension of the prediction space or may be multiple dimensions in the prediction space. When the target dimension is a plurality of dimensions, a weight may be set for each target dimension for subsequent computation.
And S140, substituting the target parameter value into a light source color algorithm to calculate the light source color of the target image.
Since each dimension in the prediction space corresponds to a set of parameter values, after determining the matching sample class of the target image, a set of target parameter values may be determined according to the target dimension of the matching sample class, and the target parameter values may be substituted into the light source color algorithm to calculate the light source color of the target image.
According to the technical scheme provided by the embodiment, a target image is input into a machine learning model, and the spatial position of the target image in a prediction space is determined; the number of dimensions of the prediction space is determined according to the number of optional values of each parameter in a light source color algorithm; determining a matching sample class according to the spatial position of the target image in the prediction space and the class center position of a predetermined sample class; obtaining a target dimension of a matching sample class, and determining a target parameter value corresponding to the target dimension according to the target dimension; substituting the target parameter value into a light source color algorithm to calculate the light source color of the target image. By adopting the technical scheme provided by the application, the effect of improving the accuracy of determining the light source color and enhancing the adaptability of the light source color determination mode can be realized.
Based on the above aspects, optionally, the machine learning model is configured to process the training samples in the following manner: mapping all training samples in the training sample set into a prediction space; according to the spatial position distribution of the training samples in the prediction space, carrying out clustering operation to obtain at least two sample classes and class centers of each sample class; and determining the target dimension of each sample class according to a preset rule. The specific mapping mode can input the sample image into a light source color algorithm, and substitutes each optional value of each parameter in the light source color algorithm into the light source color algorithm, so that a light source color value on each dimension in a prediction space, namely, projection on each dimension, can be obtained, and further, the spatial position of the sample image in the prediction space can be obtained by back-calculation. And clustering operation is carried out according to the position distribution of each training sample in the prediction space, so that a plurality of sample classes and class centers of each sample class can be obtained. The target dimension of each sample class is determined according to a preset rule, and can be a dimension with minimum dispersion of projection distribution of samples in the sample class, and the dimension is taken as the target dimension. According to the scheme, the training sample sets are clustered from the angle of the prediction space, so that each sample class can be obtained, and further the method can be used for determining the sample class which is most suitable for the space position of the target image in the prediction space, so that the light source color of the target image is estimated, and the method has the advantages that the application range of the technical scheme is wider, and the method can be suitable for the target image shot under any light source color.
Based on the above technical solutions, optionally, determining the target dimension of each sample class according to a preset rule includes: and determining the target dimension of each sample class according to the inter-class dispersion and/or intra-class dispersion of each sample class in the current dimension of the prediction space, traversing all dimensions and the inter-class dispersion and/or intra-class dispersion in each dimension. The inter-class dispersion can be the dimension with the largest projection distribution dispersion between classes after the sample class is divided, and the dimension is determined as the target dimension; the intra-class dispersion can be a dimension with minimum projection distribution dispersion among samples in the sample class after the sample class is divided, and the dimension is determined as a target dimension; in addition, the dimension having the largest ratio of the inter-class dispersion to the intra-class dispersion may be determined as the target dimension. It follows that the partitioning of the sample classes may depend on the distribution of all sample images in the prediction space. The method has the advantages that the division of the sample class can be more objective, the influence of subjective factors is avoided, and the accuracy of the determination of the sample class is improved.
On the basis of the above technical solutions, optionally, inputting the target image into a pre-trained machine learning model, and determining a spatial position of the target image in a prediction space includes: inputting the target image into a light source color algorithm of a machine learning model, and substituting the light source color algorithm with optional values of parameters to obtain a light source calculated value which is used as a projection value of the target image in each dimension of a prediction space; based on the projection values of the target image in each dimension in the prediction space, the spatial position in the prediction space is determined. Wherein the light source color algorithm may comprise one, two or more parameters, and the number of selectable values for each parameter may be one or more. The method can be used for substituting each optional value of each parameter into the target image, so that the projection value of each dimension of the target image on the prediction space can be determined, and the coordinate position of the target image in the prediction space can be obtained. The technical scheme has the advantages that the spatial position of the target image in the prediction space can be objectively obtained, and the accuracy of determining the light source color of the target image is further improved.
On the basis of the above technical solutions, optionally, the light source color algorithm includes: generalized gray edge algorithm. The generalized gray edge algorithm (Generalized Gray Edge, GGE) is formulated as follows:
in the formula (2), n is the derivative order of the image in the spatial domain, p is a norm parameter, and σ is a gaussian filter parameter used for image preprocessing. When n takes a value of 1 or greater, the above formula is a first or higher order GE algorithm. The performance of the GGE algorithm is jointly determined by three parameters of n, p and sigma, so that a three-dimensional parameter space formed by the range of the values of the n, p and sigma is defined as the parameter space of the GGE algorithm; wherein each set of parameters (n, p, σ) corresponds to a point in the parameter space. GGE is a general light source color estimation algorithm, covering most of the currently known statistical algorithms, the effect of which depends on the choice of parameters n, p, σ. Within the processing framework of the GGE, there theoretically exists an optimal combination of parameters (n, p, σ) corresponding to each type of scene; i.e. the scene has a mapping relation with the parameter space of the algorithm. In practical application, the theoretically optimal self-adaptive light source color estimation algorithm can be realized under the GGE algorithm framework only by accurately quantifying scene features and establishing an effective mapping relation between the scene features and the algorithm parameter space.
Example two
The present application further provides a preferred embodiment in order to enable those skilled in the art to more accurately understand the technical solutions provided in the present application.
The scheme provides a light source color estimation method based on unsupervised learning. The premise of the method is that a training set of a large number of samples for different scenes and different light source types is acquired. The samples in the training set must be color images, but do not need to be annotated with the true light source colors.
The flow of the method is divided into two parts: firstly, a training process and secondly, a prediction process.
Training process:
firstly, inputting a training set;
secondly, setting a GGE prediction space, and mapping training set samples into the GGE prediction space; the GGE prediction space is a space formed by estimating the light source color of a sample when different parameters are taken by a GGE algorithm, and the prediction of the GGE under each parameter point corresponds to one dimension to form a multidimensional prediction space;
thirdly, carrying out clustering operation on training samples in a GGE prediction space to obtain a plurality of sample classes;
fourth, GGE optimal parameters of each sample class are calculated.
The prediction flow is as follows:
the method comprises the steps of firstly, inputting an image to be predicted;
Secondly, mapping the image to be predicted into a GGE prediction space;
thirdly, searching the best matching sample class of the image to be predicted in the GGE prediction space;
fourth, based on GGE optimal parameters of the best matching sample class, predicting the illuminant color value of the input image.
The training set sample collection can be performed in the following two ways:
first, all training samples are required to be color images; since only color images have the information required for the estimation of the color of the light source, the color image format is in principle not limited and may be RGB, YUV or other format. For convenience of description, the present patent assumes that the format of the training sample and the image to be predicted is RGB format; since even an input color image is not in RGB format, an image in RGB format is generally obtained by linear or nonlinear color space conversion.
Second, the number of samples of the training set is required to be as large as possible, covering as many scenes and light source types as possible. For example, the training set sample number suggests not less than 1000, and the samples should cover various factors of different light source types, different lighting conditions, different scene types, different geographic locations, different climate conditions, and the like more uniformly.
1. GGE prediction space is mapped with training samples.
The GGE algorithm is determined by three parameters of n, p and sigma, so that a three-dimensional parameter space formed by the range of the values of the n, p and sigma is defined as the parameter space of the GGE algorithm; wherein each set of parameters (n, p, σ) corresponds to a point in the parameter space.
GGE prediction space is a space formed by estimating the light source color of a sample when GGE algorithm takes different parameters, and the prediction of GGE under each parameter point corresponds to one dimension to form a multidimensional prediction space.
The process of mapping the training samples to the GGE prediction space is to calculate the estimated value of the light source color of each training sample when the GGE takes different parameters. Assuming that the GGE parameter space has L parameter points, it indicates that the GGE prediction space is an L-dimensional space, and for each training sample, L light source predictors are generated that describe the coordinate position of the sample in the L-dimensional space.
One specific embodiment is provided below:
first, determining the parameter space of GGE. The parameter space of the GGE is composed of the range of values of n, p, and σ, which are mainly determined empirically, for example, n=0, 1,2, p=0, 3,6,9, σ=0, 2,4 are taken in the present embodiment, and 36 (n, p, σ) parameter points are taken in total to form the GGE parameter space.
And secondly, establishing a prediction space of the GGE. The prediction space of the GGE is composed of all parameter points of the GGE parameter space, and each parameter point corresponds to one dimension of the prediction space.
Third, all samples in the training set are mapped into the predictive space of the GGE. Traversing each sample in the training set, calculating predicted values of the samples in all dimensions of the GGE predicted space, and marking the samples to corresponding positions of the GGE predicted space according to the predicted values to finish mapping of the samples; this operation is repeated until the mapping of all samples of the training set to the GGE prediction space is completed.
2. And training sample clustering.
After mapping of training samples to GGE prediction space is completed, distribution of the samples in the GGE prediction space is obtained, and based on the distribution, a plurality of sample classes in the GGE prediction space can be obtained by combining clustering operation.
Here, the clustering operation may be implemented by using classical algorithms such as K-Means, mean-Shift, DBSCAN, gaussian Mixture Model (GMM), and the like, which is not limited in this patent.
For example, in a specific embodiment, a DBSCAN algorithm is used to perform clustering on sample points distributed in the GGE prediction space, so as to obtain a plurality of sample classes and class centers.
3. GGE optimal parameters of each sample class are calculated.
After the clustering of the training samples is completed, a plurality of sample classes and class centers are obtained. The next step is to determine the optimal parameters of the GGE for each sample class. The selection of the optimal parameters of the GGE is to select one or more optimal dimensions in the GGE prediction space, and parameter points corresponding to the optimal dimensions are the optimal parameters of the GGE. For a certain class, the selection basis of the optimal dimension corresponding to the class depends on the projection distribution of the sample points in the class on the dimensions, and on the other hand, also depends on the projection distribution of the class and other classes on the dimensions; in principle, when the intra-class distribution is more concentrated on a certain dimension, and the dispersion of the inter-class distribution is larger, the description of the corresponding sample class by the dimension is more stable and distinguishable, and the probability that the dimension is selected as the best dimension is higher.
A specific example is provided below:
first, traversing each sample class, and calculating the matching degree of each sample class in each dimension of the GGE prediction space. The calculation formula is as follows:
in the formulas (3), (4) and (5), L is the total number of dimensions of the GGE prediction space, and j represents the j-th dimension; k represents the number of sample classes, D k Represents the kth sample class, N k Represents D k Is a sample number of (a); f (f) i Represents D k I-th sample of (e) i j Representative sample f i Projection distribution value on GGE prediction space dimension j, i.e. sample f i At GGE parameter point (n) j ，p j ，σ j ) A predicted value of the light source at that time; mu (mu) k j For D k Projection value of class center on dimension j, mu m j (m=1, 2,., K) represents the projection values of class centers of other classes on dimension j; p (P) k j For D k The matching degree of the dimension j can be calculated by using a formula (4) or a formula (5), wherein a numerator term represents the inter-class dispersion degree, and a denominator term represents the intra-class dispersion degree.
And secondly, calculating GGE optimal parameters of each sample class based on the calculation result of the last step. Specifically, one or several optimal dimensions are selected based on the matching degree of each sample class in each dimension of the GGE prediction space, and the parameter points corresponding to the optimal dimensions are GGE optimal parameters. In sample class D k For example, the best dimension screening methods herein include, but are not limited to: (1) Based on D k The matching degrees in L dimensions of the GGE prediction space are ordered from large to small, and the first J dimensions are reserved as the optimal dimensions, so that J is more than or equal to 1 and less than or equal to L; as a special case, when j=1 is taken, the dimension with the largest matching degree is reserved as the optimal dimension; (2) Setting a lower limit threshold T of the matching degree 1 Satisfy the matching degree greater than T 1 As the best dimension. Here, parameters J and T 1 The values of (a) are all empirically determined, e.g., J=3 (provided that the total number of dimensions L.gtoreq.3) T 1 Taking 1/3 of the maximum matching degree (the matching degree value corresponding to the dimension with the maximum matching degree).
Thirdly, determining GGE optimal parameters and weights of the sample classes. In sample class D k For example, when the selected GGE optimal parameter has only one parameter point, the parameter point is the sample class D k The weight of the unique optimal parameter point of (a) is not required to be set; when the selected GGE optimal parameters comprise a plurality of parameter points, the parameter points form a sample class D k The weights of these optimal parameters can be calculated by:
in the formula (6), J represents a sample class D k J is the number of the optimal parameter points; omega k j The weight of the j-th optimal parameter point representing the sample class is in positive correlation with the matching degree; α is an adjustment parameter, and is empirically set, for example, α=1 is desirable.
4. And estimating the light source color value of the image to be predicted.
After the algorithm training is completed, a plurality of sample classes are obtained, and each sample class contains the following information: (1) Class center coordinates of each sample class in the GGE prediction space; (2) The GGE optimal parameters of each sample class comprise the values of the optimal parameter points and the weights of the optimal parameter points.
The next step is to apply the light source color estimation to the image to be predicted, and the specific flow is as follows:
the first step, mapping the image to be predicted to GGE prediction space, and the specific method is consistent with the mapping process of training samples. And obtaining the coordinate position of the image to be predicted in the GGE prediction space.
And secondly, selecting the best matching sample class of the image to be predicted in the GGE prediction space. Since the coordinates of the class center of each sample class in the GGE prediction space are known, the best matching class is found, that is, the sample class center closest to the coordinate point of the image to be predicted is found, and the sample class is defined as the best matching sample class of the image to be predicted. Wherein, the distance in GGE prediction space can be calculated by the following formula:
in the formula (7), P 1 And P 2 Two points in the L-dimensional GGE prediction space, p Min shape Is a Minkowski norm parameter; p is p Min shape Typically 2 is taken, i.e. corresponds to the euclidean distance.
Thirdly, after finding the best matching sample class of the image to be predicted, using GGE optimal parameters of the sample class to perform image processingAnd estimating the color of the row light source. The best matching class is provided with J GGE parameter points (n j ，p j ，σ j ) And the weights of the parameter points are omega j J=1, …, J, then the light source color estimation value of the image f is calculated with the following formula (8):
As a special case, when j=1, i.e. the best matching class has only one GGE parameter point (n, p, σ), the illuminant color estimation value of the image f is directly calculated with the following formula (9):
the technical scheme provides a light source color estimation algorithm based on unsupervised learning, which is improved in the aspects of light source color prediction precision, scene adaptability and the like compared with the existing similar method.
Example III
Fig. 2 is a schematic structural diagram of a light source color determining device according to a third embodiment of the present application. As shown in fig. 2, the device for determining the color of the light source includes:
a target image input module 210, configured to input the target image into a machine learning model, and determine a spatial position of the target image in a prediction space; the number of dimensions of the prediction space is determined according to the number of optional values of each parameter in a light source color algorithm;
a matching sample class determining module 220, configured to determine a matching sample class according to a spatial position of the target image in the prediction space and a class center position of a predetermined sample class;
a target parameter value determining module 230, configured to obtain a target dimension of the matching sample class, and determine a target parameter value corresponding to the target dimension according to the target dimension;
The light source color calculation module 240 is configured to substitute the target parameter value into a light source color algorithm to calculate a light source color of the target image.
According to the technical scheme provided by the embodiment of the application, the target image is input into the machine learning model, and the spatial position of the target image in the prediction space is determined; the number of dimensions of the prediction space is determined according to the number of optional values of each parameter in a light source color algorithm; determining a matching sample class according to the spatial position of the target image in the prediction space and the class center position of a predetermined sample class; obtaining a target dimension of a matching sample class, and determining a target parameter value corresponding to the target dimension according to the target dimension; substituting the target parameter value into a light source color algorithm to calculate the light source color of the target image. By adopting the technical scheme provided by the application, the training can be performed under the condition of unknown light source colors of sample images in an unsupervised learning mode, the accuracy and usability of determining the high light source colors are improved, and the adaptability of the light source color determining mode is enhanced.
Based on the above technical solutions, optionally, the apparatus further includes a model training module, configured to:
The training sample mapping unit is used for mapping all training samples in the training sample set into a prediction space;
the sample class determining unit is used for carrying out clustering operation according to the spatial position distribution of the training samples in the prediction space to obtain at least two sample classes and class centers of each sample class;
and the target dimension determining unit is used for determining the target dimension of each sample class according to a preset rule.
Based on the above technical solutions, optionally, according to a preset rule, the target dimension determining unit is specifically configured to:
and determining the target dimension of each sample class according to the inter-class dispersion and/or intra-class dispersion of each sample class in the current dimension of the prediction space, traversing all dimensions and the inter-class dispersion and/or intra-class dispersion in each dimension.
The product can execute the method provided by any embodiment of the application, and has the corresponding functional modules and beneficial effects of the execution method.
Example IV
The present embodiments also provide a storage medium containing computer executable instructions, which when executed by a computer processor, are for performing a method of determining a color of a light source, the method comprising:
Inputting the target image into a machine learning model, and determining the spatial position of the target image in a prediction space; the number of dimensions of the prediction space is determined according to the number of optional values of each parameter in a light source color algorithm;
determining a matching sample class according to the spatial position of the target image in the prediction space and the class center position of a predetermined sample class;
obtaining a target dimension of a matching sample class, and determining a target parameter value corresponding to the target dimension according to the target dimension;
substituting the target parameter value into a light source color algorithm to calculate the light source color of the target image.
Storage media-any of various types of memory devices or storage devices. The term "storage medium" is intended to include: mounting media such as CD-ROM, floppy disk or tape devices; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, lanbas (Rambus) RAM, etc.; nonvolatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. In addition, the storage medium may be located in a computer system in which the program is executed, or may be located in a different second computer system connected to the computer system through a network (such as the internet). The second computer system may provide program instructions to the computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations (e.g., in different computer systems connected by a network). The storage medium may store program instructions (e.g., embodied as a computer program) executable by one or more processors.
Of course, the storage medium containing the computer executable instructions provided in the embodiments of the present application is not limited to the above-described determination operation of the light source color, and may also perform the related operation in the determination method of the light source color provided in any embodiment of the present application.
Example five
The embodiment of the application provides electronic equipment, and the electronic equipment can integrate the light source color determining device provided by the embodiment of the application. Fig. 3 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present application. As shown in fig. 3, the present embodiment provides an electronic device 300, which includes: one or more processors 320; a storage device 310, configured to store one or more programs that, when executed by the one or more processors 320, cause the one or more processors 320 to implement a method for determining a color of a light source provided by an embodiment of the present application, the method includes:
inputting the target image into a machine learning model, and determining the spatial position of the target image in a prediction space; the number of dimensions of the prediction space is determined according to the number of optional values of each parameter in a light source color algorithm;
Determining a matching sample class according to the spatial position of the target image in the prediction space and the class center position of a predetermined sample class;
obtaining a target dimension of a matching sample class, and determining a target parameter value corresponding to the target dimension according to the target dimension;
substituting the target parameter value into a light source color algorithm to calculate the light source color of the target image.
Of course, those skilled in the art will appreciate that the processor 320 may also implement the technical solution of the method for determining the color of the light source provided in any embodiment of the present application.
The electronic device 300 shown in fig. 3 is only an example and should not be construed as limiting the functionality and scope of use of the embodiments herein.
As shown in fig. 3, the electronic device 300 includes a processor 320, a storage device 310, an input device 330, and an output device 340; the number of processors 320 in the electronic device may be one or more, one processor 320 being taken as an example in fig. 3; the processor 320, the storage device 310, the input device 330, and the output device 340 in the electronic device may be connected by a bus or other means, which is illustrated in fig. 3 as being connected by a bus 350.
The storage device 310 is a computer readable storage medium, and may be used to store a software program, a computer executable program, and program instructions corresponding to a method for determining a color of a light source in an embodiment of the present application.
The storage device 310 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for functions; the storage data area may store data created according to the use of the terminal, etc. In addition, storage 310 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some examples, storage device 310 may further include memory located remotely from processor 320, which may be connected via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 330 may be used to receive input numeric, character information, or voice information, and to generate key signal inputs related to user settings and function control of the electronic device. The output device 340 may include a display screen, a speaker, etc.
The electronic equipment provided by the embodiment of the application can realize training under the condition of unknown light source colors of sample images by adopting an unsupervised learning mode, and has the effects of improving the accuracy and usability of the high light source colors and enhancing the adaptability of the light source color determining mode.
The light source color determining device, the storage medium and the electronic device provided in the above embodiments can execute the light source color determining method provided in any embodiment of the present application, and have the corresponding functional modules and beneficial effects of executing the method. Technical details not described in detail in the above embodiments may be found in the method for determining the color of a light source provided in any embodiment of the present application.
Note that the above is only a preferred embodiment of the present application and the technical principle applied. Those skilled in the art will appreciate that the present application is not limited to the particular embodiments described herein, but is capable of numerous obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the present application. Therefore, while the present application has been described in connection with the above embodiments, the present application is not limited to the above embodiments, but may include many other equivalent embodiments without departing from the spirit of the present application, the scope of which is defined by the scope of the appended claims.
Claims (9)
1. A method for determining a color of a light source, comprising:
inputting the target image into a machine learning model, and determining the spatial position of the target image in a prediction space; the number of dimensions of the prediction space is determined according to the number of optional values of each parameter in a light source color algorithm;
Determining a matching sample class according to the spatial position of the target image in the prediction space and the class center position of a predetermined sample class;
obtaining a target dimension of a matching sample class, and determining a target parameter value corresponding to the target dimension according to the target dimension;
substituting the target parameter value into a light source color algorithm to calculate the light source color of the target image;
the step of inputting the target image into a pre-trained machine learning model and determining the spatial position of the target image in a prediction space comprises the following steps:
inputting the target image into a light source color algorithm of a machine learning model, and substituting the light source color algorithm with optional values of parameters to obtain a light source calculated value which is used as a projection value of the target image in each dimension of a prediction space;
based on the projection values of the target image in each dimension in the prediction space, the spatial position in the prediction space is determined.
2. The method of claim 1, wherein the machine learning model is configured to process training samples in the following manner:
mapping all training samples in the training sample set into a prediction space;
according to the spatial position distribution of the training samples in the prediction space, carrying out clustering operation to obtain at least two sample classes and class centers of each sample class;
And determining the target dimension of each sample class according to a preset rule.
3. The method of claim 2, wherein determining the target dimension for each sample class according to a preset rule comprises:
and determining the target dimension of each sample class according to the inter-class dispersion and/or intra-class dispersion of each sample class in the current dimension of the prediction space, traversing all dimensions and the inter-class dispersion and/or intra-class dispersion in each dimension.
4. The method of claim 1, wherein the light source color algorithm comprises: generalized gray edge algorithm.
5. A light source color determining apparatus, comprising:
the target image input module is used for inputting the target image into a machine learning model and determining the spatial position of the target image in a prediction space; the number of dimensions of the prediction space is determined according to the number of optional values of each parameter in a light source color algorithm;
the target image input module includes:
the space position determining unit is used for inputting the target image into a light source color algorithm of the machine learning model, substituting the light source color algorithm with the optional values of all parameters to obtain a light source calculated value which is used as a projection value of the target image in each dimension of a prediction space;
Determining a spatial position in a prediction space based on projection values of each dimension of the target image on the prediction space;
the matching sample class determining module is used for determining a matching sample class according to the spatial position of the target image in the prediction space and the class center position of the predetermined sample class;
the target parameter value determining module is used for obtaining the target dimension of the matching sample class and determining a target parameter value corresponding to the target dimension according to the target dimension;
and the light source color calculation module is used for substituting the target parameter value into a light source color algorithm to calculate the light source color of the target image.
6. The apparatus of claim 5, further comprising a model training module to:
the training sample mapping unit is used for mapping all training samples in the training sample set into a prediction space;
the sample class determining unit is used for carrying out clustering operation according to the spatial position distribution of the training samples in the prediction space to obtain at least two sample classes and class centers of each sample class;
and the target dimension determining unit is used for determining the target dimension of each sample class according to a preset rule.
7. The apparatus according to claim 6, wherein the target dimension determining unit is specifically configured to:
And determining the target dimension of each sample class according to the inter-class dispersion and/or intra-class dispersion of each sample class in the current dimension of the prediction space, traversing all dimensions and the inter-class dispersion and/or intra-class dispersion in each dimension.
8. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements a method of determining a color of a light source as claimed in any one of claims 1-4.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of determining the color of a light source according to any one of claims 1-4 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910576317.2A CN112150563B (en) | 2019-06-28 | 2019-06-28 | Method and device for determining light source color, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910576317.2A CN112150563B (en) | 2019-06-28 | 2019-06-28 | Method and device for determining light source color, storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112150563A CN112150563A (en) | 2020-12-29 |
| CN112150563B true CN112150563B (en) | 2024-03-26 |
Family
ID=73869427
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910576317.2A Active CN112150563B (en) | 2019-06-28 | 2019-06-28 | Method and device for determining light source color, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112150563B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007013415A (en) * | 2005-06-29 | 2007-01-18 | Ichikawa Soft Laboratory:Kk | Image processor and image processing method |
| CN106651795A (en) * | 2016-12-03 | 2017-05-10 | 北京联合大学 | Method of using illumination estimation to correct image color |
| CN109166170A (en) * | 2018-08-21 | 2019-01-08 | 百度在线网络技术（北京）有限公司 | Method and apparatus for rendering augmented reality scene |
-
2019
- 2019-06-28 CN CN201910576317.2A patent/CN112150563B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007013415A (en) * | 2005-06-29 | 2007-01-18 | Ichikawa Soft Laboratory:Kk | Image processor and image processing method |
| CN106651795A (en) * | 2016-12-03 | 2017-05-10 | 北京联合大学 | Method of using illumination estimation to correct image color |
| CN109166170A (en) * | 2018-08-21 | 2019-01-08 | 百度在线网络技术（北京）有限公司 | Method and apparatus for rendering augmented reality scene |
Non-Patent Citations (1)
| Title |
|---|
| 光照色调颜色恒常性算法研究;蔡珣, 孟祥旭, 向辉;中国图象图形学报(第08期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112150563A (en) | 2020-12-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3579187B1 (en) | Facial tracking method, apparatus, storage medium and electronic device | |
| US9805293B2 (en) | Method and apparatus for object recognition in image processing | |
| CN111460968B (en) | Unmanned aerial vehicle identification and tracking method and device based on video | |
| JP6438403B2 (en) | Generation of depth maps from planar images based on combined depth cues | |
| Hwang et al. | Context-based automatic local image enhancement | |
| US7283645B2 (en) | Object recognition using binary image quantization and Hough kernels | |
| Soriano et al. | Using the skin locus to cope with changing illumination conditions in color-based face tracking | |
| JP5719230B2 (en) | Object recognition device, method for controlling object recognition device, and program | |
| US9460521B2 (en) | Digital image analysis | |
| CN109711443A (en) | Floor plan recognition methods, device, equipment and storage medium neural network based | |
| Lu et al. | Color constancy using 3D scene geometry | |
| CN103886013A (en) | Intelligent image retrieval system based on network video monitoring | |
| Finlayson et al. | A curious problem with using the colour checker dataset for illuminant estimation | |
| US9940543B2 (en) | Control of computer vision pre-processing based on image matching using structural similarity | |
| WO2022206729A1 (en) | Method and apparatus for selecting cover of video, computer device, and storage medium | |
| CN110111341B (en) | Image foreground obtaining method, device and equipment | |
| CN112150563B (en) | Method and device for determining light source color, storage medium and electronic equipment | |
| CN109074643B (en) | Orientation-based object matching in images | |
| WO2021022820A1 (en) | White balance abnormality determination method and apparatus, storage medium, and electronic device | |
| CN110807375A (en) | Human head detection method, device and equipment based on depth image and storage medium | |
| JP2004054947A (en) | Object collation system, object collation method and object collation program | |
| CN116091831A (en) | Method and system for performing scene adaptation for target model | |
| CN112200207B (en) | Method and device for determining light source color, storage medium and electronic equipment | |
| CN112926552A (en) | Remote sensing image vehicle target recognition model and method based on deep neural network | |
| Zulkifley et al. | On improving CAMSHIFT performance through colour constancy approach |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |