WO2024027090A1 - 一种颜色校正矩阵优化方法、装置、电子设备及介质 - Google Patents

一种颜色校正矩阵优化方法、装置、电子设备及介质 Download PDF

Info

Publication number
WO2024027090A1
WO2024027090A1 PCT/CN2022/142609 CN2022142609W WO2024027090A1 WO 2024027090 A1 WO2024027090 A1 WO 2024027090A1 CN 2022142609 W CN2022142609 W CN 2022142609W WO 2024027090 A1 WO2024027090 A1 WO 2024027090A1
Authority
WO
WIPO (PCT)
Prior art keywords
color
matrix
color correction
saturation
loss function
Prior art date
Application number
PCT/CN2022/142609
Other languages
English (en)
French (fr)
Inventor
林凯
吴泼
孙岳
Original Assignee
浙江宇视科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浙江宇视科技有限公司 filed Critical 浙江宇视科技有限公司
Publication of WO2024027090A1 publication Critical patent/WO2024027090A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/64—Circuits for processing colour signals

Definitions

  • Embodiments of the present application relate to the field of image processing, for example, to a color correction matrix optimization method, device, electronic equipment, and media.
  • the current color correction scheme simply optimizes the color correction matrix based on the Euclidean distance between the coordinates of the image to be corrected and the corrected image.
  • the Euclidean distance cannot fully represent the difference between the color perceived by the human eye and the color of the image collected by the camera. Therefore, the color of the image after color correction cannot be close to the color perceived by the human eye.
  • an initialized color correction matrix is obtained through the least squares method. In the subsequent optimization process, the optimization result may easily fall into a local minimum trap and the global optimal solution cannot be obtained.
  • This application provides a color correction matrix optimization method, device, electronic equipment and media to determine the loss function based on the actual perception of hue and saturation by the human eye, optimize the color correction matrix, and improve the image color after color correction. Compatibility with human eye perception of color.
  • the embodiment of the present application provides a color correction matrix optimization method, including:
  • the global hue error and the saturation constraint regular term are determined, and a loss is constructed based on the global hue error and the saturation constraint regular term.
  • Lab coordinates are coordinates in Lab color space;
  • the color correction matrix is updated according to the loss function value until the loss function value converges, and an optimal color correction matrix is obtained to perform color correction on the image to be processed collected by the image collector based on the optimal color correction matrix.
  • the embodiment of the present application provides a color correction matrix optimization device, including:
  • the correction processing module is set to perform correction processing on the color source matrix of the collected color card image based on the color correction matrix to obtain the color output matrix;
  • the loss function construction module is configured to determine the global hue error and the saturation constraint regular term according to the Lab coordinates of the color output matrix and the Lab coordinates of the color target matrix of the target color card image, and determine the global hue error and the saturation constraint regular term according to the global hue error and the Lab coordinates of the color target matrix of the target color card image.
  • the saturation constraint regular term constructs the loss function; where, the Lab coordinates are the coordinates in the Lab color space;
  • the optimization module is configured to update the color correction matrix according to the loss function value until the loss function value converges to obtain the optimal color correction matrix to process the image collected by the image collector based on the optimal color correction matrix.
  • the image is color corrected.
  • the embodiment of the present application provides an electronic device, including:
  • processors one or more processors
  • a storage device configured to store one or more programs
  • the one or more processors are caused to implement the color correction matrix optimization method as described in any of the above embodiments.
  • the embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored.
  • the program is executed by a processor, the color correction matrix optimization as described in any of the above embodiments is implemented. method.
  • Figure 1 is a flow chart of a color correction matrix optimization method provided by an embodiment of the present application.
  • Figure 2 is a schematic diagram of color block pixels provided by an embodiment of the present application.
  • Figure 3 is a flow chart of a color correction matrix optimization method provided by another embodiment of the present application.
  • Figure 4 is a flow chart of a color correction matrix optimization method provided by yet another embodiment of the present application.
  • Figure 5 is a schematic diagram of an iterative optimization process provided by yet another embodiment of the present application.
  • Figure 6 is a schematic structural diagram of a color correction matrix optimization device provided by an embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • example embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts depict various steps as a sequential process, many of the steps may be performed in parallel, concurrently, or simultaneously. Additionally, the order of multiple steps can be rearranged. The process may be terminated when its operation is completed, but may also have additional steps not included in the figures. The processing may correspond to a method, function, procedure, subroutine, subroutine, or the like.
  • FIG. 1 is a flow chart of a color correction matrix optimization method provided by an embodiment of the present application.
  • the embodiment of the present application may be applicable to optimizing a color correction matrix.
  • the color correction matrix optimization method can be executed by a color correction matrix optimization device, which can be implemented in software and/or hardware and integrated in an electronic device.
  • the method includes the following steps:
  • the collected color card image is an image obtained by collecting the image of the 24-color standard color card by the image collector. Before performing color correction, you can also perform preprocessing such as black level correction, Bayer interpolation, and white balance correction on the collected color card images.
  • the color source matrix of the collected color card image is constructed based on the intensity values of the R, G, and B channels of each color patch in the collected color card image. For example, in the entire pixel area of each color block, the pixels in the partial pixel area closest to the center pixel are taken.
  • Figure 2 is a color block in the color card image
  • select the partial pixel area closest to the center pixel that is, the 25 pixels indicated by the shade in Figure 2.
  • Calculate the average intensity value of the R channel in 25 pixels, average the intensity value of the G channel, and average the intensity value of the B channel to obtain the corresponding color block. and Perform the above processing on all 24 color blocks, and the color source matrix of the collected color card image can be expressed as Among them, the first column represents the intensity values of the R, G, and B channels of the first color block, the second column represents the intensity values of the R, G, and B channels of the second color block, and the last column represents the 24th color block.
  • the intensity values of the R, G, and B channels of the color patch In the embodiment of this application, the number of pixels in the entire pixel area and the number of pixels in part of the pixel area of the color block in Figure 2 are just examples. The specific number is determined based on the collected color card image and is not limited here.
  • the color correction matrix can be Parameters in CCM can be adjusted during subsequent optimization processes.
  • the color source matrix is linearly transformed by CCM to realize the correction of the color source matrix.
  • the target color card image is a predetermined color card image that reflects the color perceived by the human eye.
  • the color target matrix is a matrix obtained by sampling and averaging the three-channel intensity values of each color block pixel in the target color card image.
  • the process of determining the color target matrix of the target color card image is the same as determining the color of the collected color card image.
  • the process of the source matrix is the same, see S110 for details.
  • the resulting color target matrix is In the actual color correction process, after the image collected by the image collector is color corrected through the color correction matrix, it is also necessary to use gamma correction to perform non-linear operations to obtain the final corrected image.
  • the color target matrix in order to compare the color output matrix after color correction with the color target matrix, the color target matrix is subjected to inverse gamma transformation, so that the color output matrix and the color target matrix are in Comparative analysis of results under the same processing dimension.
  • Lab color space is a color space in which brightness and chroma are relatively separated. It is a uniform color space recommended by the International Illuminating Institute. This space is a three-dimensional Cartesian coordinate system, which is currently the most widely used color measurement system. Among them, L is the brightness component, which represents the brightness. The two components (a, b) together form the chromaticity plane, and a and b represent the color opposite dimensions. To evaluate the color difference between the color output matrix and the color target matrix, it is necessary to convert the color output matrix and the color target matrix from data in other color spaces to data in the Lab color space for comparison.
  • the hue information and saturation information of the color output matrix can be characterized according to the coordinates of the color output matrix
  • the hue information and saturation information of the color target matrix can be characterized according to the coordinates of the color target matrix to determine the color output matrix and color target.
  • Global hue error and saturation constrained regularization terms of the matrix are the difference in chroma. If the loss function is determined only based on the global hue error, the saturation in the chroma space will expand and contract at will, and the ideal color correction effect will not be achieved.
  • a saturation constraint regular term is constructed to limit the random expansion and contraction of saturation during the loss function optimization process, thereby improving the color correction effect.
  • the saturation constraint regular term can be determined based on the saturation difference.
  • the specific determination method can be determined based on the actual situation. For example, the Lab coordinates of the color output matrix and the Lab coordinates of the color target matrix can be used to represent the respective saturations.
  • the saturation difference can be determined based on the saturation.
  • the degree difference constructs the saturation constraint regular term.
  • the global hue error and saturation constraint regular term can be calculated to construct a loss function, and the specific values of the elements in the color output matrix and color target matrix can be brought into the loss function to obtain the loss function value.
  • coefficients can be introduced according to the actual situation to adjust the weight of the global hue error and saturation constraint regularization term in the loss function.
  • the solution in the embodiment of the present application takes into account the global hue difference and saturation difference that the human eye is sensitive to, and then determines the loss function value to update the color correction matrix according to the global hue error and saturation constraint regular term, so that After color correction is performed on the collected images based on the optimized color correction matrix, the color can be closer to the color perceived by the human eye.
  • the color correction matrix is iteratively optimized until the loss function value converges and the minimum value of the loss function value is obtained, and the color correction matrix corresponding to the minimum value of the loss function value is used as the optimal color correction matrix.
  • the relevant parameters of the optimal color correction matrix can be input into the image collector, so that the image collector generates the optimal color correction matrix based on the relevant parameters, performs color correction on the collected image to be processed, and The color-corrected image to be processed is subjected to gamma transformation to obtain the final output image. Due to the color correction and gamma transformation, the color of the output image can be close to the color perceived by the human eye, improving the user's viewing experience.
  • the electronic device that implements the above solution in the embodiment of the present application may be the same as the device that subsequently uses the optimal color correction matrix to perform color correction on the image to be processed, or may be different.
  • the relevant parameters of the optimal color correction matrix are input to the image collector, so that the image collector can apply relevant parameters to generate an optimal color correction matrix to process the collected images.
  • Embodiments of the present application provide a color correction matrix optimization method, device, electronic equipment and media.
  • the implementation plan is to perform correction processing on the color source matrix of the collected color card image based on the color correction matrix to obtain a color output matrix; according to the color
  • the global hue error and saturation constraint regular term of the output matrix and the color target matrix of the target color card image in the Lab color space are used to determine the loss function value; the color correction matrix is updated according to the loss function value until the loss function value converges, and we get
  • the optimal color correction matrix is used to perform color correction on the image to be processed collected by the image collector based on the optimal color correction matrix.
  • the loss function is determined taking into account the human eye's perception of hue and saturation deviations, so that the image collected by the image collector is color corrected based on the optimized color correction matrix. Can be closer to the color perceived by the human eye.
  • Figure 3 is a flow chart of a color correction matrix optimization method provided by another embodiment of the present application. This embodiment is optimized based on the above embodiment. It should be noted that technical details that are not described in detail in this embodiment may be referred to any of the above embodiments.
  • the method includes the following steps:
  • each column element in the color output matrix and the color target matrix represents a color patch.
  • the i-th color block in the color output matrix uses the i-th column element and Represents, converted to Lab color space, the i-th color block in the color output matrix is represented by and express, That is, the coordinates of the i-th color patch in the Lab color space.
  • the matrix in the Lab color space composed of the coordinates of 24 color patches is In the same way, the matrix of the color target matrix in the Lab color space is
  • the chromaticity coordinates are the coordinates of the chromaticity plane in the Lab color space, that is, the a coordinate and b coordinate representing the color. Since the evaluation standard of color error will directly affect the visual effect of color correction, the global hue error and saturation constraint regularization terms of the image are analyzed according to the chromaticity coordinates.
  • the Lab color space is a color space in which brightness and chromaticity are relatively separated, the two components (a, b) together form the chromaticity plane, and a and b represent the opposite dimensions of color. Therefore, the hue difference and saturation difference between the color output matrix and the color target matrix in the Lab color space can be characterized according to the chromaticity coordinates.
  • determining the global hue error based on the chromaticity coordinates corresponding to each color patch includes: for the color patch pairs represented by the same column elements in the color output matrix and the color target matrix, based on the color patch pair chromaticity coordinates The angle between the vectors determines the hue error of the color patch pair; among them, the chromaticity coordinate vector is a vector with the origin as the starting point and the chromaticity coordinate as the end point; based on the weighted average of the hue errors of multiple color patch pairs, the global Hue error.
  • the color patch pair is two color patches represented by the same column elements in the color output matrix and the color target matrix.
  • the color patch represented by the element in the 5th column of the color output matrix and the color patch represented by the element in the 5th column of the color target matrix form a color patch pair.
  • the color patch represented by the element in the 24th column of the color output matrix and the color target matrix The color blocks represented by the elements in column 24 form a color block pair.
  • the chromaticity coordinate of the color patch is a point in the chromaticity plane, the origin of the chromaticity plane is used as the starting point, the chromaticity coordinate is used as the end point, and the vector from the starting point to the end point is used as the color A vector of chromaticity coordinates of the block. Since the chromaticity coordinates of the same color patch pair are different, the chromaticity coordinate vectors are different. There is a certain angle between the chromaticity coordinate vectors of the color patch pair, and the angle represents the hue error of the color patch pair.
  • the global hue error of the color output matrix and the color target matrix is the weighted average of the global hue error of the 24 color patches.
  • the formula is: For color patch pair i, the chromaticity coordinates of the color patches belonging to the color output matrix are The chromaticity coordinates of the color patch belonging to the color target matrix are The hue error of the color patch for i is The global hue error of the color output matrix and the color target matrix can be expressed by ⁇ 1 , ⁇ 2 ,..., ⁇ 24 , for example:
  • ⁇ Hue is the global hue error of the color output matrix and the color target matrix
  • w i is the weight value, which can be determined according to the actual situation. If w i are all equal, ⁇ Hue is the arithmetic mean of ⁇ 1 , ⁇ 2 ,..., ⁇ 24 .
  • the human eye has a small perception difference between black, white and gray, so the weight of the hue error of the corresponding color block pairs of black, white and gray can be set to 0 to avoid the corresponding The impact of the color patch on the color evaluation of the color patch, thereby improving the accuracy of the global hue error.
  • determining the saturation constraint regular term based on the chromaticity coordinates corresponding to each color patch includes: determining the first relationship between the chromaticity coordinates of multiple color patches represented by multiple column elements in the color output matrix and the origin. The average value of the distance, and determine the average value of the second distance between the chromaticity coordinates of the multiple color patches represented by the multi-column elements in the color target matrix and the origin; determine the ratio of the average value of the first distance to the average value of the second distance , and determine the saturation constraint regular term based on the difference between the ratio and 1. For example, in the chromaticity plane, the distance between the chromaticity coordinate of the color patch and the origin can be used to characterize the saturation of the color patch.
  • the saturation constraint regular term is determined.
  • “1" refers to the ratio of the second distance to the second distance.
  • the first distance Characterizes its saturation Since the human eye has a small sensory difference between black, white and gray, it is possible to ignore the saturation of the corresponding color blocks of black, white and gray and only consider the saturation of the remaining 18 color blocks, so as to use the third average of a distance Characterizes the saturation of the color output matrix.
  • the second distance Characterizes its saturation Since the human eye has a small sensory difference between black, white and gray, it is possible to ignore the saturation of the corresponding color blocks of black, white and gray and only consider the saturation of the remaining 18 color blocks, so as to use the third average of two distances Characterizes the saturation of the color target matrix.
  • the ratio of the average of the first distance to the average of the second distance is The difference between the ratio and 1 is As a saturation constraint regular term, it is used to characterize the saturation difference between the color output matrix and the color target matrix. Since the saturation constraint regular term of concern in practical applications is a positive value, it can take the absolute value, that is as a saturation constraint regular term.
  • constraint parameters can also be introduced to adjust the weight of ⁇ Chroma.
  • the constraint parameter may be a parameter determined based on the actual situation. For example, it may be determined based on the tolerance level for the relative saturation difference, and may take a value such as 0.1.
  • E the loss function
  • ⁇ Hue the global hue error
  • ⁇ Chroma the saturation constraint regular term
  • ⁇ the constraint parameter
  • the sensitivity to the global hue error is much higher than the saturation constraint regular term, so the global hue error construction loss is introduced
  • the function calculates the loss function value to correct for human eye perception.
  • the saturation constraint regular term is used to constrain the saturation in the chromaticity space to expand and contract arbitrarily without affecting the optimization results of the global hue error, so that the color of the corrected image can be closer to the color perceived by the human eye.
  • the correction effect is closer to human eye perception.
  • a color correction matrix optimization method provided by embodiments of the present application is optimized based on the above embodiments. For multiple color patches represented by multiple column elements in the color output matrix and the color target matrix, each The chromaticity coordinates corresponding to the color block; where the chromaticity coordinates are the coordinates in the chromaticity plane after the color output matrix and the color target matrix are converted to the Lab color space; according to the chromaticity coordinates corresponding to each color block, Determine the global hue error and saturation constraint regular term of the color output matrix and the color target matrix in the Lab color space; construct a loss function based on the sum of the saturation constraint regular term and the global hue error, fully taking into account the human eye
  • the color correction matrix is optimized according to the global hue error, and the saturation constraint regular term is used to constrain the saturation during the optimization process to avoid arbitrary expansion and contraction of saturation, so that the optimized image color can follow Colors that are close to what the human eye perceives improve the user experience.
  • Figure 4 is a flow chart of a color correction matrix optimization method provided by yet another embodiment of the present application. This embodiment is optimized based on the above embodiment. It should be noted that technical details that are not described in detail in this embodiment can be referred to any of the above embodiments.
  • the method includes the following steps:
  • YUV is a color encoding method.
  • YIQ, Y'UV, YUV, YCbCr, YPbPr, etc. can all be called YUV.
  • "Y” represents brightness, which is the grayscale value
  • "U” and "V” represent chroma, which are used to describe the color and saturation of the image and are used to specify the color of the pixel.
  • YIQ color space is a color space in which brightness and chroma are relatively independent.
  • YIQ color space and RGB color space have a linear relationship with each other, so the linear transformation of YIQ color space can be linearly mapped to the linear transformation of RGB space.
  • one element in each row of the initialized third-order color correction matrix is represented by two other elements to obtain a color correction matrix containing six variables.
  • a third-order matrix composed of nine parameters can be
  • S340 Determine the global hue error and saturation constraint regular term according to the Lab coordinate of the color output matrix and the Lab coordinate of the color target matrix of the target color card image, and determine the global hue error and the saturation constraint regular term according to the global hue error and the saturation constraint regular term. Construct a loss function; where, Lab coordinates are coordinates in Lab color space.
  • the implementation process of the embodiment of the present application is shown in Figure 5.
  • the color source matrix is color corrected based on the color correction matrix CCM to obtain a color output matrix.
  • the update process of the color correction matrix is actually the update and optimization of six variables, and the optimization process is an unconstrained optimization process.
  • the optimization algorithm is not limited. For example, it can be the gradient descent method or Newton's method. , quasi-Newton method, conjugate gradient method, etc.
  • the process of optimizing a color correction matrix containing nine variables based on white balance constraints is a constrained optimization process, while the solution of the embodiment of the present application is simplified into an unconstrained optimization process, which speeds up the optimization convergence.
  • the initialization process of the color correction matrix includes: determining the hue rotation factor according to the value range of the hue rotation factor in the hue rotation matrix and the value range of the saturation scaling factor in the saturation scaling matrix. and the saturation scaling factor; determining at least two color correction matrices based on the hue rotation factor and the saturation scaling factor; or, based on the probability distribution model of the hue rotation factor and the saturation scaling factor, Randomly generate at least two sets of hue rotation factors and saturation scaling factors; determine at least two color correction matrices based on at least two sets of hue rotation factors and saturation scaling factors.
  • two solutions for determining the color correction matrix in batches are provided as examples.
  • the solution for batch generation of color correction matrices can be selected according to the actual situation, and is not limited. As long as the idea of batching is used, it is within the scope of protection of this application.
  • the hue rotation factor is ⁇ in the hue rotation matrix M R
  • the saturation stretching factor is s in the saturation stretching matrix M S.
  • the value range of the hue rotation factor and the saturation scaling factor can be determined in advance according to the actual situation, ⁇ [ ⁇ min , ⁇ max ], s ⁇ [s min ,s max ], and a certain step size can be set in advance, starting from the value Select the hue rotation factor and the saturation stretching factor in the range.
  • the hue rotation factor For example, select the hue rotation factor as ⁇ 1 and ⁇ 2 and the saturation stretching factor as s 1 and s 2. Combine the hue rotation factor and the saturation stretching factor to get four Combine ( ⁇ 1 , s 1 ), ( ⁇ 2 , s 1 ), ( ⁇ 1 , s 2 ) and ( ⁇ 1 , s 2 ). Substituting the four combinations into T respectively and converting from YUV color space to RGB color space, four color correction matrices are obtained.
  • the number of selected hue rotation factors and saturation scaling factors is not limited and can be determined according to the number of initial color matrices that need to be determined.
  • the solution of the embodiment of the present application can also be to establish a probability distribution model of ⁇ and s, such as a uniform distribution model, randomly select at least two groups ( ⁇ , s) in the probability distribution model, bring them into T, and use the YUV color Convert the space to RGB color space to obtain at least two initial color matrices.
  • a probability distribution model of ⁇ and s such as a uniform distribution model
  • At least two ( ⁇ , s) can also be determined in other ways, such as random generation, etc., to obtain at least two color correction matrices, thereby achieving batch generation of initial color correction matrices. It avoids the situation of easily falling into a local minimum when optimizing a single initialized color correction matrix, so that at least two initialized color correction matrices can be optimized to find a global minimum and improve the optimization accuracy.
  • the number of selected hue rotation factors and saturation scaling factors can be at least in the order of hundreds of digits, so that enough initialized color correction matrices can be generated in batches to find the optimal solution more accurately and avoid falling into local Minimal case.
  • performing correction processing on the color source matrix based on the color correction matrix to obtain a color output matrix includes: performing correction processing on the color source matrix based on at least two color correction matrices to obtain at least two color output matrices; Update the color correction matrix according to the loss function value until the loss function value converges, and obtain the optimal color correction matrix, including: optimizing at least two color correction matrices according to the loss function value until the loss function value converges to the minimum value, and obtain at least two minimum values; determine the minimum value among the at least two minimum values, and use the updated color correction matrix corresponding to the minimum value as the optimal color correction matrix.
  • At least two color output matrices can be obtained.
  • the color correction processing can be executed in parallel without affecting each other. It is assumed that the number of at least two color correction matrices CCM j is N. Traverse the color correction matrix, perform correction processing on the color source matrix, and obtain N color output matrices. According to the global hue error and saturation constraint regular terms of the color output matrix and color target matrix in the Lab color space, determine the loss function value, and Update and optimize N color correction matrices until N minimum values MinE j and corresponding optimization matrices MinCCM j are obtained.
  • at least two color correction matrices are determined to generate color correction matrices in batches, which disperses the optimization starting points to the greatest extent, so as to increase the probability of obtaining the global minimum and avoid optimizing a single color correction matrix. This leads to the problem of falling into a local minimum and being unable to continue to converge to the global minimum.
  • FIG. 6 is a schematic structural diagram of a color correction matrix optimization device provided by an embodiment of the present application.
  • the color correction matrix optimization device provided in this embodiment includes:
  • the correction processing module 410 is configured to perform correction processing on the color source matrix of the collected color card image based on the color correction matrix to obtain a color output matrix;
  • the loss function construction module 420 is configured to determine the global hue error and the saturation constraint regular term according to the Lab coordinates of the color output matrix and the Lab coordinates of the color target matrix of the target color card image, and determine the global hue error and the saturation constraint regular term according to the global hue error and the Lab coordinates of the target color card image.
  • the above-mentioned saturation constraint regular term constructs the loss function; among them, the Lab coordinates are the coordinates in the Lab color space;
  • the optimization module 430 is configured to update the color correction matrix according to the loss function value until the loss function value converges and obtains the optimal color correction matrix, so as to evaluate the image to be collected by the image collector based on the optimal color correction matrix. Process the image for color correction.
  • the loss function construction module 420 includes:
  • a coordinate determination unit configured to determine the chromaticity coordinates corresponding to each color block for multiple color blocks represented by multiple column elements in the color output matrix and the color target matrix; wherein the chromaticity coordinates are the color output matrix The coordinates of the matrix and the color target matrix in the chromaticity plane after being converted to the Lab color space;
  • the difference data determination unit is configured to determine the global hue error and saturation constraint regularization terms based on the chromaticity coordinates corresponding to each color block;
  • a numerical value determination unit is configured to construct a loss function based on the sum of the saturation constraint regularization term and the global hue error.
  • the difference data determination unit includes:
  • the hue error determination subunit is configured to determine the hue error of the color block pair according to the angle between the color block and the chromaticity coordinate vector for the color block pair represented by the same column element in the color output matrix and the color target matrix;
  • the chromaticity coordinate vector is a vector with the origin as the starting point and the chromaticity coordinate as the end point;
  • the global hue error calculation unit is configured to calculate the global hue error based on the weighted average of the hue errors of multiple color patch pairs.
  • the difference data determination unit includes:
  • the distance data determination subunit is configured to determine the average value of the first distance between the chromaticity coordinates of multiple color patches represented by multiple column elements in the color output matrix and the origin, and determine the multiple color values represented by multiple column elements in the color target matrix. The average of the second distance of the patch's chromaticity coordinates from the origin;
  • the relative saturation difference determination subunit is configured to determine the ratio of the average value of the first distance to the average value of the second distance, and determine the saturation constraint regular term based on the difference between the ratio and 1.
  • the device further includes:
  • the matrix building module is set as a color correction matrix composed of a hue rotation matrix and a saturation scaling matrix, left multiplied by the mapping matrix from YUV color space to RGB color space, and right multiplied by the mapping matrix from RGB color space to YUV color space to obtain the initialized Third-order color correction matrix;
  • the transformation module is set to be based on the white balance constraint, and represents one element in each row of the initialized third-order color correction matrix with two other elements to obtain a color correction matrix containing six variables.
  • Optimization module 430 set to:
  • the device further includes:
  • the first factor selection unit is configured to determine the hue rotation factor and the saturation expansion factor based on the value range of the hue rotation factor in the hue rotation matrix and the value range of the saturation expansion factor in the saturation expansion matrix. ;
  • a first matrix building unit is configured to determine at least two color correction matrices based on the hue rotation factor and the saturation scaling factor. or,
  • the second factor unit is configured to randomly generate at least two sets of hue rotation factors and saturation scaling factors according to the probability distribution model of the hue rotation factor and the saturation scaling factor;
  • the second matrix building unit is configured to determine at least two color correction matrices based on at least two sets of hue rotation factors and saturation scaling factors.
  • the correction processing module 410 is configured as:
  • Optimization module 430 including:
  • a minimum value determination unit is configured to optimize at least two color correction matrices according to the loss function value until the loss function value converges to a minimum value, and obtain at least two minimum values.
  • the optimal color correction matrix determination unit is configured to determine the minimum value of at least two minimum values, and use the updated color correction matrix corresponding to the minimum value as the optimal color correction matrix.
  • the color correction matrix optimization device provided by the embodiments of the present application can be configured to execute the color correction matrix optimization method provided by any of the above embodiments, and has corresponding functions and beneficial effects.
  • FIG. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
  • Electronic device 10 is intended to represent many forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers.
  • Electronic device 10 may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, user equipment, wearable devices (eg, helmets, glasses, watches, etc.), and other similar computing devices.
  • the components shown herein, their connections and relationships, and their functions are examples only and are not intended to limit the implementation of the present application as described and/or claimed herein.
  • the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores There is a computer program that can be executed by at least one processor.
  • the processor 11 can perform the operation according to the computer program stored in the read-only memory (ROM) 12 or loaded from the storage unit 18 into the random access memory (RAM) 13. Performs a variety of appropriate actions and processes.
  • RAM 13 various programs and data required for the operation of the electronic device 10 can also be stored.
  • the processor 11, the ROM 12 and the RAM 13 are connected to each other via the bus 14.
  • Input/output (I/O) interface 15 is also connected to bus 14.
  • the I/O interface 15 Multiple components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc. etc.; and communication unit 19, such as network card, modem, wireless communication transceiver, etc.
  • the communication unit 19 allows the electronic device 10 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunications networks, wireless networks.
  • Processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processing processor (DSP), and any appropriate processor, controller, microcontroller, etc.
  • the processor 11 performs a plurality of methods and processes described above, such as a color correction matrix optimization method.
  • the color correction matrix optimization method may be implemented as a computer program, which is tangibly embodied in a computer-readable storage medium, such as the storage unit 18 .
  • part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19.
  • the processor 11 may be configured to perform the color correction matrix optimization method in any other suitable manner (eg, by means of firmware).
  • Various implementations of the systems and techniques described above may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip implemented in a system (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or a combination thereof.
  • FPGAs field programmable gate arrays
  • ASICs application specific integrated circuits
  • ASSPs application specific standard products
  • SOC system
  • CPLD load programmable logic device
  • computer hardware firmware, software, and/or a combination thereof.
  • These various embodiments may include implementation in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor
  • the processor which may be a special purpose or general purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
  • An output device may be a special purpose or general purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
  • An output device may be a special purpose or general purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
  • Computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that the computer program, when executed by the processor, causes the functions/operations specified in the flowcharts and/or block diagrams to be implemented.
  • a computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
  • a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device.
  • Computer-readable storage media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing.
  • the computer-readable storage medium may be a machine-readable signal medium.
  • machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • the computer-readable storage medium may be a non-transitory computer-readable storage medium.
  • the systems and techniques described herein may be implemented on an electronic device 10 having a display device (eg, a cathode ray tube (CRT) or a liquid crystal display (LCD)) for displaying information to a user. LCD) monitor); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the electronic device 10 .
  • a display device eg, a cathode ray tube (CRT) or a liquid crystal display (LCD) monitor
  • LCD liquid crystal display
  • keyboard and pointing device eg, a mouse or a trackball
  • Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and may be provided in any form, including Acoustic input, voice input or tactile input) to receive input from the user.
  • the systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., A user's computer having a graphical user interface or web browser through which the user can interact with implementations of the systems and technologies described herein), or including such backend components, middleware components, or any combination of front-end components in a computing system.
  • the components of the system may be interconnected by any form or medium of digital data communication (eg, a communications network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.
  • Computing systems may include clients and servers.
  • Clients and servers are generally remote from each other and typically interact over a communications network.
  • the relationship of client and server is created by computer programs running on corresponding computers and having a client-server relationship with each other.
  • the server can be a cloud server, also known as a cloud computing server or cloud host. It is a host product in the cloud computing service system to avoid the existence of physical hosts and virtual private servers (Virtual Private Servers, VPS) in related technologies.
  • VPS Virtual Private Servers

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Color Image Communication Systems (AREA)
  • Facsimile Image Signal Circuits (AREA)

Abstract

本申请实施例公开了一种颜色校正矩阵优化方法、装置、电子设备及介质。该方法包括：基于颜色校正矩阵，对采集色卡图像的颜色源矩阵进行校正处理，得到颜色输出矩阵；根据颜色输出矩阵的Lab坐标和目标色卡图像的颜色目标矩阵的Lab坐标，确定全局色相误差和饱和度约束正则项，并构造损失函数；根据损失函数值对颜色校正矩阵进行更新，直到损失函数值收敛，得到最优颜色校正矩阵，以基于最优颜色校正矩阵对待处理图像进行颜色校正。

Description

一种颜色校正矩阵优化方法、装置、电子设备及介质
本申请要求在2022年8月4日提交中国专利局、申请号为202210932339.X的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及图像处理领域，例如涉及一种颜色校正矩阵优化方法、装置、电子设备及介质。
背景技术
数字摄像机拍摄得到的图像，与人眼感知的实际图像往往存在一定的颜色偏差。导致图像颜色偏差的因素包括光源色温、镜头透光特性、图像传感器感光特性等。在数字颜色校正矩阵优化中，自动白平衡算法(Automatic white balance，AWB)被用于纠正色温带来的偏色问题，来模拟人眼对灰色感知的恒常性。彩色图像经过AWB校正后，仍存在较严重的偏色现象。此时偏色的主要原因是图像传感器与人眼对光谱和亮度的响应存在明显差异。因此，在AWB恢复灰色物体颜色后，还需要通过颜色校正矩阵(Color Correction Matrix，CCM)对摄像机采集的图像进行校正，以纠正彩色物体的偏色问题。
但是，目前的颜色校正方案，单纯以待校正图像和校正后图像坐标的欧氏距离对颜色校正矩阵进行优化，但是欧式距离并不能完全代表人眼感知的颜色与摄像机采集图像的颜色的差异，因此颜色校正后图像的颜色并不能贴近人眼感知的颜色。另外，目前的颜色校正矩阵优化的方案中，通过最小二乘法得到一个初始化颜色校正矩阵，在后续的优化过程中容易导致优化结果落入局部最小陷阱，无法得到全局最优解。
发明内容
本申请提供了一种颜色校正矩阵优化方法、装置、电子设备及介质，以基于人眼对色相和饱和度的实际感知情况确定损失函数，对颜色校正矩阵进行优化，提高颜色校正后的图像颜色和人眼感知颜色的贴合度。
在一个实施例中，本申请实施例提供了一种颜色校正矩阵优化方法，包括：
基于颜色校正矩阵，对采集色卡图像的颜色源矩阵进行校正处理，得到颜色输出矩阵；
根据所述颜色输出矩阵的Lab坐标和目标色卡图像的颜色目标矩阵的Lab坐标，确定全局色相误差和饱和度约束正则项，并根据所述全局色相误差和所述饱和度约束正则项构造损失函数；其中，Lab坐标为在Lab颜色空间中的坐标；
根据损失函数值对所述颜色校正矩阵进行更新，直到所述损失函数值收敛，得到最优颜色校正矩阵，以基于所述最优颜色校正矩阵对图像采集器采集的待处理图像进行颜色校正。
在一个实施例中，本申请实施例提供了一种颜色校正矩阵优化装置，包括：
校正处理模块，设置为基于颜色校正矩阵，对采集色卡图像的颜色源矩阵进行校正处理，得到颜色输出矩阵；
损失函数构造模块，设置为根据所述颜色输出矩阵的Lab坐标和目标色卡图像的颜色目标矩阵的Lab坐标，确定全局色相误差和饱和度约束正则项，并根据所述全局色相误差和所述饱和度约束正则项构造损失函数；其中，Lab坐标为在Lab颜色空间中的坐标；
优化模块，设置为根据损失函数值对所述颜色校正矩阵进行更新，直到所述损失函数值收敛，得到最优颜色校正矩阵，以基于所述最优颜色校正矩阵对图像采集器采集的待处理图像进行颜色校正。
在一个实施例中，本申请实施例提供了一种电子设备，包括：
一个或多个处理器；
存储装置，设置为存储一个或多个程序；
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如上述任一实施例中所述的颜色校正矩阵优化方法。
在一个实施例中，本申请实施例还提供了一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现如上述任一实施例中所述的颜色校正矩阵优化方法。
附图说明
贯穿附图中，相同或相似的附图标记表示相同或相似的元素。应当理解附图是示意性的，原件和元素不一定按照比例绘制。
图1为本申请一种实施例提供的颜色校正矩阵优化方法的流程图；
图2为本申请一种实施例提供的色块像素点示意图；
图3为本申请另一实施例提供的颜色校正矩阵优化方法的流程图；
图4为本申请又一实施例提供的颜色校正矩阵优化方法的流程图；
图5为本申请又一实施例提供的迭代优化流程示意图；
图6为本申请一种实施例提供的颜色校正矩阵优化装置的结构示意图；
图7为本申请一种实施例提供的电子设备的结构示意图。
具体实施方式
在更加详细地讨论示例性实施例之前应当提到的是，一些示例性实施例被 描述成作为流程图描绘的处理或方法。虽然流程图将多个步骤描述成顺序的处理，但是其中的许多步骤可以被并行地、并发地或者同时实施。此外，多个步骤的顺序可以被重新安排。当其操作完成时所述处理可以被终止，但是还可以具有未包括在附图中的附加步骤。所述处理可以对应于方法、函数、规程、子例程、子程序等等。
为了更好地理解本申请实施例，下面对相关技术进行介绍。
图1为本申请实施例提供的一种颜色校正矩阵优化方法的流程图，本申请实施例可适用于对颜色校正矩阵进行优化的情况。例如，该颜色校正矩阵优化方法可以由颜色校正矩阵优化装置执行，该颜色校正矩阵优化装置可以通过软件和/或硬件的方式实现，并集成在电子设备中。
如图1所示，该方法包括如下步骤：
S110、基于颜色校正矩阵，对采集色卡图像的颜色源矩阵进行校正处理，得到颜色输出矩阵。
其中，采集色卡图像为图像采集器对24色标准色卡进行图像采集得到的图像。在进行颜色校正前，还可以对采集色卡图像进行黑电平校正、拜尔插值和白平衡校正等预处理。根据采集色卡图像中每个色块的R、G、B三通道的强度值构建采集色卡图像的颜色源矩阵。例如，在每个色块的整个像素区域中，取最靠近中心像素点的部分像素区域的像素点。对取出像素点的R通道的强度值求均值，G通道的强度值求均值，B通道的强度值求均值，作为该色块R、G、B三通道的强度值，并作为矩阵的一列，对于24个色块，得到24列R、G、B三通道的强度值，构造颜色源矩阵。将颜色校正矩阵与颜色源矩阵对应的矩阵进行运算，实现对颜色源矩阵的颜色校正。
示例性的，如图2所示，假设图2为采集色卡图像中的其中一个色块，取最靠近中心像素点的部分像素区域，即图2中阴影表示的25个像素点。计算25个像素点中R通道的强度值求均值，G通道的强度值求均值，B通道的强度值求均值，得到该色块对应的
Figure PCTCN2022142609-appb-000001
和
Figure PCTCN2022142609-appb-000002
对24个色块均做上述处理，得到采集色卡图像的颜色源矩阵可以表示为
Figure PCTCN2022142609-appb-000003
其中，第一列表示第1个色块的R、G、B三通道的强度值，第二列表示第2个色块的R、G、B三通道的强度值，最后一列表示第24个色块的R、G、B三通道的强度值。在本申请实施例中，图2中色块的整个像素区域的像素点数量和部分像素区域的像素点数量只是举例，具体数量根据采集色卡图像确定，在此不做限定。
颜色校正矩阵可以为
Figure PCTCN2022142609-appb-000004
CCM中的参数可以在后续的优化过程中进行调整。将颜色源矩阵经过CCM的线性变换，实现对颜色源矩阵 的校正。具体表达式为C o＝CCM·C s，其中，C o即为颜色输出矩阵。
S120、根据所述颜色输出矩阵的Lab坐标和目标色卡图像的颜色目标矩阵的Lab坐标，确定全局色相误差和饱和度约束正则项，并根据所述全局色相误差和所述饱和度约束正则项构造损失函数；其中，Lab坐标为在Lab颜色空间中的坐标。
其中，目标色卡图像为预先确定的体现人眼感知颜色的色卡图像。颜色目标矩阵为对目标色卡图像中每个色块像素点的三通道强度值进行采样、求均值处理得到的矩阵，确定目标色卡图像的颜色目标矩阵的过程与确定采集色卡图像的颜色源矩阵的过程一致，详见S110。得到的颜色目标矩阵的为
Figure PCTCN2022142609-appb-000005
Figure PCTCN2022142609-appb-000006
在实际的颜色校正过程中，对图像采集器采集的图像经过颜色校正矩阵进行颜色校正后，还需要用过伽马校正进行非线性运算，得到最终的校正后图像。因此在本申请实施例中，为了将经过颜色校正矩阵进行颜色校正后的颜色输出矩阵与颜色目标矩阵进行比对，将颜色目标矩阵进行反伽马变换，从而使颜色输出矩阵和颜色目标矩阵在同一处理维度下的结果进行对比分析。
Lab颜色空间是是一个亮度与色度相对分离的颜色空间，是由国际照明学会推荐的均匀色空间。该空间是三维直角坐标系统，是目前最受广泛使用的测色系统。其中L为亮度分量，表示亮度，(a,b)两个分量共同构成色度平面，a和b表示颜色对立维度。对颜色输出矩阵和颜色目标矩阵进行色差评价，需要将颜色输出矩阵和颜色目标矩阵从其他颜色空间的数据转换为Lab颜色空间的数据进行对比。在Lab颜色空间中，可以根据颜色输出矩阵的坐标表征颜色输出矩阵的色相信息和饱和度信息，根据颜色目标矩阵的坐标表征颜色目标矩阵的色相信息和饱和度信息，确定颜色输出矩阵和颜色目标矩阵的全局色相误差和饱和度约束正则项。其中，全局色相误差为色度的差异，如果仅根据全局色相误差确定损失函数，则色度空间上饱和度随意伸缩，同样无法达到理想的颜色校正效果。在本申请实施例中，构造饱和度约束正则项，从而限制损失函数优化过程中饱和度随意伸缩，进而提高颜色校正效果。饱和度约束正则项可以根据饱和度差异确定，具体确定方式可以根据实际情况确定，例如可以以颜色输出矩阵的Lab坐标和颜色目标矩阵的Lab坐标表征各自的饱和度，确定饱和度差异，根据饱和度差异构造饱和度约束正则项。
例如，可以将全局色相误差和饱和度约束正则项进行运算构造损失函数，将颜色输出矩阵和颜色目标矩阵中元素的具体值带入至损失函数中得到损失函数值。在运算过程中可以根据实际情况引入系数以调节全局色相误差和饱和度约束正则项在损失函数中的权重。
本申请实施例中的方案，考虑到了人眼感官上比较敏感的全局色相差异和饱和度差异，进而根据全局色相误差和饱和度约束正则项，确定损失函数值对颜色校正矩阵进行更新，以使基于优化后的颜色校正矩阵对采集的图像进行颜色校正后，能够更加贴近人眼感官上的颜色。
S130、根据损失函数值对所述颜色校正矩阵进行更新，直到所述损失函数值收敛，得到最优颜色校正矩阵，以基于所述最优颜色校正矩阵对图像采集器采集的待处理图像进行颜色校正。
示例性的，对颜色校正矩阵进行迭代优化，直至损失函数值收敛，得到损失函数值的极小值，将损失函数值的极小值对应的颜色校正矩阵作为最优颜色校正矩阵。在后续的应用中，可以将最优颜色校正矩阵的相关参数输入至图像采集器中，以使图像采集器基于相关参数生成最优颜色校正矩阵，对采集的待处理图像进行颜色校正，并对颜色校正后的待处理图像进行伽马变换，得到最终的输出图像，输出图像由于经过颜色校正以及伽马变换，其颜色能够贴近人眼感官上的颜色，提高用户的观看体验。
需要说明的是，本申请实施例执行上述方案的电子设备，与后续采用最优颜色校正矩阵对待处理图像进行颜色校正的设备可以一致，也可以不同。一般情况下，由于本申请实施例中方案的处理数据量较大，因此可以由处理性能较高的电子设备执行，在得到最优颜色校正矩阵之后，再将最优颜色校正矩阵的相关参数输入至图像采集器中，以供图像采集器应用相关参数生成最优颜色校正矩阵对采集的图像进行处理。
本申请实施例提供了一种颜色校正矩阵优化方法、装置、电子设备及介质，实现方案为，基于颜色校正矩阵，对采集色卡图像的颜色源矩阵进行校正处理，得到颜色输出矩阵；根据颜色输出矩阵和目标色卡图像的颜色目标矩阵在Lab颜色空间中的全局色相误差和饱和度约束正则项，确定损失函数值；根据损失函数值对颜色校正矩阵进行更新，直到损失函数值收敛，得到最优颜色校正矩阵，以基于最优颜色校正矩阵对图像采集器采集的待处理图像进行颜色校正。上述技术方案中，对颜色校正矩阵进行优化时，考虑到人眼对色相和饱和度偏差的感知情况确定损失函数，以使图像采集器采集的图像基于优化后的颜色校正矩阵进行颜色校正后，能够更加贴近人眼感知的颜色。
图3为本申请另一实施例提供的颜色校正矩阵优化方法的流程图。本实施例是在上述实施例的基础上进行优化。需要说明的是，未在本实施例中详尽描述的技术细节可参见上述任意实施例。
例如，如图3所示，该方法包括如下步骤：
S210、基于颜色校正矩阵，对采集色卡图像的颜色源矩阵进行校正处理，得到颜色输出矩阵。
S220、针对所述颜色输出矩阵和所述颜色目标矩阵中多列元素代表的多个色块，确定每个色块对应的色度坐标；其中，色度坐标为所述颜色输出矩阵和所述颜色目标矩阵转换到Lab颜色空间后在色度平面中的坐标。
示例性的，颜色输出矩阵和颜色目标矩阵中的每一列元素，都代表一个色块。将颜色输出矩阵和颜色目标矩阵转换至Lab颜色空间，即可得到颜色输出矩阵和颜色目标矩阵在Lab颜色空间中的多个元素，进而确定每个色块对应在Lab颜色空间中的色度坐标。示例性的，在RGB颜色空间中，颜色输出矩阵中第i个色块用第i列元素
Figure PCTCN2022142609-appb-000007
和
Figure PCTCN2022142609-appb-000008
表示，转换到Lab颜色空间中，颜色输出矩阵中第i个色块用
Figure PCTCN2022142609-appb-000009
和
Figure PCTCN2022142609-appb-000010
表示，
Figure PCTCN2022142609-appb-000011
即为第i个色块在Lab颜色空间中的坐标。由24个色块的坐标构成Lab颜色空间中的矩阵为
Figure PCTCN2022142609-appb-000012
Figure PCTCN2022142609-appb-000013
同理，颜色目标矩阵在Lab颜色空间中的矩阵为
Figure PCTCN2022142609-appb-000014
在本申请实施例中，色度坐标为在Lab颜色空间内的色度平面的坐标，即表示颜色的a坐标和b坐标。由于颜色误差的评价标准，将直接影响颜色校正的视觉效果，因此根据色度坐标对图像的全局色相误差和饱和度约束正则项进行分析。
S230、根据每个色块对应的色度坐标，确定全局色相误差和饱和度约束正则项。
由于Lab颜色空间是一个亮度与色度相对分离的颜色空间，(a,b)两个分量共同构成色度平面，a和b表示颜色对立维度。因此可以根据色度坐标，表征颜色输出矩阵和颜色目标矩阵在Lab颜色空间中的色相差异和饱和度差异。
在本申请实施例中，根据每个色块对应的色度坐标，确定全局色相误差，包括：针对颜色输出矩阵和颜色目标矩阵中同一列元素表示的色块对，根据色块对色度坐标向量的夹角，确定色块对的色相误差；其中，色度坐标向量为以原点为起点，以色度坐标为终点的向量；根据多个色块对的色相误差的加权平均值，计算全局色相误差。
其中，色块对为颜色输出矩阵和颜色目标矩阵中同一列元素表示的两个色块。例如颜色输出矩阵中第5列元素表示的色块，与颜色目标矩阵中第5列元素表示的色块，构成色块对，颜色输出矩阵中第24列元素表示的色块，与颜色目标矩阵中第24列元素表示的色块构成色块对。示例性的，在色度平面中，色块的色度坐标为色度平面中的一个点，将色度平面的原点作为起点，将色度坐标作为终点，将起点指向终点的向量作为该色块的色度坐标向量。由于同一色 块对的色度坐标不同，因此色度坐标向量不同，色块对的色度坐标向量存在一定的夹角，以该夹角表征该色块对的色相误差。颜色输出矩阵和颜色目标矩阵的全局色相误差即为24个色块对全局色相误差的加权平均值。
例如，对任意两个色度坐标(a 1,b 1)和(a 2,b 2)，使用它们在坐标系中的夹角来表示色相误差，公式为
Figure PCTCN2022142609-appb-000015
对于色块对i，属于颜色输出矩阵的色块的色度坐标为
Figure PCTCN2022142609-appb-000016
属于颜色目标矩阵的色块的色度坐标为
Figure PCTCN2022142609-appb-000017
色块对i的色相误差为
Figure PCTCN2022142609-appb-000018
颜色输出矩阵和颜色目标矩阵的全局色相误差可以用θ 1,θ 2,…,θ 24进行表示，例如：
Figure PCTCN2022142609-appb-000019
其中，ΔHue为颜色输出矩阵和颜色目标矩阵的全局色相误差，w i为权重值，可以根据实际情况确定。如果w i均相等，则ΔHue即为θ 1,θ 2,…,θ 24的算数平均值。在本申请实施例中，人眼对于黑色、白色和灰色的感知差别较小，因此可以将黑色、白色和灰色对应色块对的色相误差的权重设置为0，以避免黑色、白色和灰色对应的色块对彩色色块的颜色评估带来的影响，进而提高全局色相误差的精准度。
在本申请实施例中，根据每个色块对应的色度坐标，确定饱和度约束正则项，包括：确定颜色输出矩阵中多列元素表示的多个色块的色度坐标与原点的第一距离的平均值，并确定颜色目标矩阵中多列元素表示的多个色块的色度坐标与原点的第二距离的平均值；确定第一距离的平均值与第二距离的平均值的比值，并根据所述比值与1的差值，确定饱和度约束正则项。示例性的，在色度平面中，可以用色块的色度坐标与原点的距离表征该色块的饱和度。针对颜色输出矩阵中多列元素表示的多个色块，计算色块的色度坐标与原点的第一距离，并计算多个色块对应的第一距离的平均值，表征颜色输出矩阵的饱和度。针对颜色目标矩阵中多个色块，计算色块的色度坐标与原点的第二距离，并计算多个色块对应的第二距离的平均值，表征颜色目标矩阵的饱和度。根据第一距离的平均值与第二距离的平均值的比值，并计算比值与1的差值，确定饱和度约束正则项。其中，“1”是指第二距离与第二距离的比值。例如，对于颜色输出矩阵中的色块i，用第一距离
Figure PCTCN2022142609-appb-000020
表征其饱和度，由于人眼对于黑色、白色和灰色的感官差异较小，因此可以不考虑黑色、白色和灰色对应色块的饱和度，只考虑其余18个色块的饱和度，从而以第一距离的平均值
Figure PCTCN2022142609-appb-000021
表征颜色输出矩阵的饱和度。对于颜色目标矩阵中的色块i，用第二距离
Figure PCTCN2022142609-appb-000022
表征其饱和度，由于人眼对于黑色、白色和灰色的感官差异较小，因此可以不考虑黑色、白色和灰色对应色块的饱和度，只考虑其余18个色块的饱和度，从而以第二距离的平均值
Figure PCTCN2022142609-appb-000023
表征颜色目标矩阵的饱和度。第一距离的平 均值与第二距离的平均值的比值为
Figure PCTCN2022142609-appb-000024
比值与1的差值为
Figure PCTCN2022142609-appb-000025
作为饱和度约束正则项，用于表征颜色输出矩阵与颜色目标矩阵的饱和度差异由于在实际应用中关注的饱和度约束正则项是正值，因此可以取绝对值，即
Figure PCTCN2022142609-appb-000026
作为饱和度约束正则项。在本申请实施例中，还可以引入约束参数，来调节ΔChroma的权重。其中，约束参数可以为根据实际情况确定的参数，例如可以根据对相对饱和度差异的容忍程度确定，取0.1等数值。
S240、根据所述饱和度约束正则项与所述全局色相误差的和，构造损失函数。
例如，损失函数可以表示为E＝ΔHue+λ·ΔChroma，其中，E为损失函数，ΔHue为全局色相误差，ΔChroma为饱和度约束正则项，λ为约束参数。将颜色输出矩阵和颜色目标矩阵在Lab颜色空间内的坐标带入计算得到损失函数值。
在本申请实施例中，考虑到人眼对全局色相误差和饱和度约束正则项的认知情况不同，对全局色相误差的敏感度远高于饱和度约束正则项，因此引入全局色相误差构造损失函数计算损失函数值，以从人眼感知的方面进行校正。以饱和度约束正则项进行约束，限制色度空间中的饱和度任意伸缩，并且不会影响全局色相误差的优化结果，从而使校正后的图像的颜色能够更贴近于人眼感官的颜色，颜色校正效果更接近与人眼感知。
S250、根据损失函数值对所述颜色校正矩阵进行更新，直到所述损失函数值收敛，得到最优颜色校正矩阵，以基于所述最优颜色校正矩阵对图像采集器采集的待处理图像进行颜色校正。
本申请实施例提供的一种颜色校正矩阵优化方法，在上述实施例的基础上进行优化，针对所述颜色输出矩阵和所述颜色目标矩阵中多列元素代表的多个色块，确定每个色块对应的色度坐标；其中，色度坐标为所述颜色输出矩阵和所述颜色目标矩阵转换到Lab颜色空间后在色度平面中的坐标；根据每个色块对应的色度坐标，确定颜色输出矩阵和颜色目标矩阵在Lab颜色空间中的全局色相误差和饱和度约束正则项；根据所述饱和度约束正则项与所述全局色相误差的和，构造损失函数，充分考虑到人眼感知色相误差的敏感度较高的情况，根据全局色相误差对颜色校正矩阵进行优化，并在优化过程中以饱和度约束正则项进行约束避免饱和度任意伸缩，从而使优化后的图像颜色能够跟贴近与人眼感知的颜色，提高了用户的体验。
图4为本申请又一实施例提供的颜色校正矩阵优化方法的流程图。本实施例是在上述实施例的基础上进行优化。需要说明的是，未在本实施例中详尽描 述的技术细节可参见上述任意实施例。
例如，如图4所示，该方法包括如下步骤：
S310、对于色相旋转矩阵和饱和度伸缩矩阵构成的颜色校正矩阵，左乘YUV颜色空间到RGB颜色空间的映射矩阵，右乘RGB颜色空间到YUV颜色空间的映射矩阵，得到初始化的三阶颜色校正矩阵。
YUV是一种颜色编码方法，YIQ、Y'UV、YUV、YCbCr、YPbPr等都可以称为YUV。“Y”表示明亮度，也就是灰阶值，“U”和“V”表示色度，作用是描述图像色彩以及饱和度，用于指定像素的颜色。以YIQ颜色空间为例进行详细说明。YIQ颜色空间是一个亮度与色度相对独立的颜色空间。YIQ颜色空间与RGB颜色空间互为线性关系，因此YIQ颜色空间的线性变换可以线性映射为RGB空间的线性变换。在YIQ颜色空间，设计色相旋转参数α，构造色相旋转矩阵
Figure PCTCN2022142609-appb-000027
设计饱和度伸缩参数s，构造饱和度伸缩矩阵
Figure PCTCN2022142609-appb-000028
YIQ色域的线性变换最终表示为T＝M S·M R。根据T确定初始化的颜色校正矩阵BaseCCM＝M YIQ2RGB·T·M RGB2YIQ，其中，M YIQ2RGB为YIQ颜色空间到RGB颜色空间的映射矩阵，M RGB2YIQ为RGB颜色空间到YIQ颜色空间的映射矩阵，得到初始化的三阶颜色校正矩阵
Figure PCTCN2022142609-appb-000029
S320、基于白平衡约束，将初始化的三阶颜色校正矩阵的每一行中的其中一个元素用另外两个元素进行表示，得到包含六个变量的颜色校正矩阵。
示例性的，由九个参数构成的三阶矩阵可以为
Figure PCTCN2022142609-appb-000030
参数集S1＝{c 11,c 12,c 13,c 21,c 22,c 23,c 31,c 32,c 33}中包括九个变量。如果将该矩阵作为颜色校正矩阵，则多个参数间需要遵循白平衡约束条件，即矩阵中每一行参数的和等于1，c r1+c r2+c r3＝1，因此可以基于白平衡约束条件，将三阶颜色校正矩阵中的每一行中的其中一个元素用另外两个元素进行表示，得到变量集例如为S2＝{c 11,c 13,c 22,c 23,c 31,c 32}，c 12＝1-c 11-c 13。c 21＝1-c 22-c 22，c 33＝1-c 31-c 32。也可以选取非对角元素设计变量集S2＝{c 12,c 13,c 21,c 23,c 31,c 32}，将九个变量的三阶矩阵表达为六个变量的三阶矩阵。如果以非对角元素对对角元素进行表示，得到六个变量的三阶矩阵为
Figure PCTCN2022142609-appb-000031
S330、基于颜色校正矩阵，对采集色卡图像的颜色源矩阵进行校正处理， 得到颜色输出矩阵。
S340、根据所述颜色输出矩阵的Lab坐标和目标色卡图像的颜色目标矩阵的Lab坐标，确定全局色相误差和饱和度约束正则项，并根据所述全局色相误差和所述饱和度约束正则项构造损失函数；其中，Lab坐标为在Lab颜色空间中的坐标。
S350、根据所述损失函数值对六个变量进行更新，以对所述颜色校正矩阵进行更新，得到最优颜色校正矩阵，以基于所述最优颜色校正矩阵对图像采集器采集的待处理图像进行颜色校正。
示例性的，本申请实施例的实现流程如图5所示，基于颜色校正矩阵CCM对颜色源矩阵进行颜色校正，得到颜色输出矩阵。根据颜色输出矩阵与颜色目标矩阵计算损失函数值，根据损失函数值对颜色校正矩阵进行更新迭代直至损失函数值收敛，得到损失函数值的极小值以及对应的最优颜色校正矩阵。在本申请实施例中，颜色校正矩阵的更新过程实际为对六个变量的更新优化，对其优化的过程为一个无约束优化过程，优化算法不做限定，例如可以为梯度下降法，牛顿法，拟牛顿法，共轭梯度法等。对包含九个变量的颜色校正矩阵基于白平衡约束条件进行优化的过程，为有约束的优化过程，而本申请实施例的方案简化为无约束优化过程，加快了优化收敛的速度。
在本申请实施例中，颜色校正矩阵的初始化过程包括：根据色相旋转矩阵中的色相旋转因子的取值范围，以及饱和度伸缩矩阵中饱和度伸缩因子的取值范围，确定所述色相旋转因子和所述饱和度伸缩因子；根据所述色相旋转因子和所述饱和度伸缩因子，确定至少两个颜色校正矩阵；或者，根据所述色相旋转因子和所述饱和度伸缩因子的概率分布模型，随机生成至少两组色相旋转因子和饱和度伸缩因子；根据至少两组色相旋转因子和饱和度伸缩因子，确定至少两个颜色校正矩阵。在本申请实施例中，示例性的提供了两种批量化确定颜色校正矩阵的方案。实际应用过程中，批量化生成颜色校正矩阵的方案可以根据实际情况进行选取，不做限定，只要是进行批量化的思想均在本申请的保护范围内。例如，色相旋转因子为色相旋转矩阵M R中的α，饱和度伸缩因子为饱和度伸缩矩阵M S中的s。可以预先根据实际情况确定色相旋转因子和饱和度伸缩因子的取值范围，α∈[α min,α max]，s∈[s min,s max]，并预先设置一定的步长，从取值范围中选取色相旋转因子和饱和度伸缩因子，例如选取色相旋转因子为α 1和α 2，饱和度伸缩因子为s 1和s 2，对色相旋转因子和饱和度伸缩因子进行组合，得到四个组合(α 1，s 1)、(α 2，s 1)、(α 1，s 2)和(α 1，s 2)。将四个组合分别代入T中，并从YUV颜色空间转换至RGB颜色空间，得到四个颜色校正矩阵。例如选取的色相旋转因子的数量和饱和度伸缩因子的数量不做限制，可以根据需要确定的初始颜色矩阵数量确定。
本申请实施例的方案还可以为，建立α和s的概率分布模型，例如均匀分布模型，在概率分布模型中随机选取至少两组(α，s)，带入到T中，并从YUV颜色空间转换至RGB颜色空间，得到至少两个初始颜色矩阵。
在本申请实施例中，还可以通过其他方式确定至少两个(α，s)，例如随机生成的方式等，以得到至少两个颜色校正矩阵，实现初始的颜色校正矩阵的批量化生成，从而避免了对单一的初始化颜色校正矩阵进行优化时容易陷入局部最小值的情况，从而能够对至少两个初始化的颜色校正矩阵进行优化寻找全局最小值，提高优化准确度。本申请实施例中，选取的色相旋转因子的数量和饱和度伸缩因子可以至少为百位数量级，以批量化生成足够多的初始化的颜色校正矩阵，更加精准地寻找到最优解，避免陷入局部最小的情况。
在确定至少两个颜色校正矩阵之后，提取颜色校正矩阵中的对应位置元素的值，对变量集S2进行初始化赋值，进行迭代求解S2的最优解。
在本申请实施例中，基于颜色校正矩阵对颜色源矩阵进行校正处理，得到颜色输出矩阵，包括：基于至少两个颜色校正矩阵分别对颜色源矩阵进行校正处理，得到至少两个颜色输出矩阵；根据损失函数值对所述颜色校正矩阵进行更新，直到所述损失函数值收敛，得到最优颜色校正矩阵，包括：根据所述损失函数值对至少两个颜色校正矩阵进行优化直至损失函数值收敛到极小值，得到至少两个极小值；确定至少两个极小值中的最小值，将该最小值对应的更新后的颜色校正矩阵，作为最优颜色校正矩阵。示例性的，基于至少两个颜色校正矩阵分别对颜色源矩阵进行校正处理，能够得到至少两个颜色输出矩阵。颜色校正处理过程可以是并行执行的，互不影响。假设至少两个颜色校正矩阵CCM j的个数为N。遍历颜色校正矩阵，对颜色源矩阵进行校正处理，得到N个颜色输出矩阵，根据颜色输出矩阵和颜色目标矩阵在Lab颜色空间中的全局色相误差和饱和度约束正则项，确定损失函数值，并对N个颜色校正矩阵进行更新优化，直到得到N个极小值MinE j与相应优化矩阵MinCCM j。通过选取极小值中的最小值opt＝argmin j＝1,2,…,N(MinE j)，获得全局最小值OptE＝MinE opt及相应全局最优解OptCCM＝MinCCM opt作为最优颜色校正矩阵。在本申请实施例中，通过确定至少两个颜色校正矩阵，从而批量化生成颜色校正矩阵，最大程度的分散了优化起点，以提高求得全局最小值的概率，避免对单一颜色校正矩阵进行优化时导致陷入局部最小值，而无法继续收敛得到全局最小值的问题。
图6为本申请一种实施例提供的颜色校正矩阵优化装置的结构示意图。本实施例提供的颜色校正矩阵优化装置包括：
校正处理模块410，设置为基于颜色校正矩阵，对采集色卡图像的颜色源矩阵进行校正处理，得到颜色输出矩阵；
损失函数构造模块420，设置为根据所述颜色输出矩阵的Lab坐标和目标色 卡图像的颜色目标矩阵的Lab坐标，确定全局色相误差和饱和度约束正则项，并根据所述全局色相误差和所述饱和度约束正则项构造损失函数；其中，Lab坐标为在Lab颜色空间中的坐标；
优化模块430，设置为根据损失函数值对所述颜色校正矩阵进行更新，直到所述损失函数值收敛，得到最优颜色校正矩阵，以基于所述最优颜色校正矩阵对图像采集器采集的待处理图像进行颜色校正。
在本申请实施例中，损失函数构造模块420，包括：
坐标确定单元，设置为针对所述颜色输出矩阵和所述颜色目标矩阵中多列元素代表的多个色块，确定每个色块对应的色度坐标；其中，色度坐标为所述颜色输出矩阵和所述颜色目标矩阵转换到Lab颜色空间后在色度平面中的坐标；
差异数据确定单元，设置为根据每个色块对应的色度坐标，确定全局色相误差和饱和度约束正则项；
数值确定单元，设置为根据所述饱和度约束正则项与所述全局色相误差的和，构造损失函数。
在本申请实施例中，差异数据确定单元，包括：
色相误差确定子单元，设置为针对所述颜色输出矩阵和所述颜色目标矩阵中同一列元素表示的色块对，根据色块对色度坐标向量的夹角，确定色块对的色相误差；其中，色度坐标向量为以原点为起点，以色度坐标为终点的向量；
全局色相误差计算单元，设置为根据多个色块对的色相误差的加权平均值，计算全局色相误差。
在本申请实施例中，差异数据确定单元，包括：
距离数据确定子单元，设置为确定颜色输出矩阵中多列元素表示的多个色块的色度坐标与原点的第一距离的平均值，并确定颜色目标矩阵中多列元素表示的多个色块的色度坐标与原点的第二距离的平均值；
相对饱和度差异确定子单元，设置为确定第一距离的平均值与第二距离的平均值的比值，并根据所述比值与1的差值，确定饱和度约束正则项。
在本申请实施例中，所述装置还包括：
矩阵构建模块，设置为对于色相旋转矩阵和饱和度伸缩矩阵构成的颜色校正矩阵，左乘YUV颜色空间到RGB颜色空间的映射矩阵，右乘RGB颜色空间到YUV颜色空间的映射矩阵，得到初始化的三阶颜色校正矩阵；
变换模块，设置为基于白平衡约束，将初始化的三阶颜色校正矩阵的每一行中的其中一个元素用另外两个元素进行表示，得到包含六个变量的颜色校正矩阵。
优化模块430，设置为：
根据所述损失函数值对六个变量进行更新，以对所述颜色校正矩阵进行更 新。
在本申请实施例中，所述装置还包括：
第一因子选取单元，设置为根据色相旋转矩阵中的色相旋转因子的取值范围，以及饱和度伸缩矩阵中饱和度伸缩因子的取值范围，确定所述色相旋转因子和所述饱和度伸缩因子；
第一矩阵构建单元，设置为根据所述色相旋转因子和所述饱和度伸缩因子，确定至少两个颜色校正矩阵。或者，
第二因子单元，设置为根据所述色相旋转因子和所述饱和度伸缩因子的概率分布模型，随机生成至少两组色相旋转因子和饱和度伸缩因子；
第二矩阵构建单元，设置为根据至少两组色相旋转因子和饱和度伸缩因子，确定至少两个颜色校正矩阵。
在本申请实施例中，校正处理模块410设置为：
基于至少两个颜色校正矩阵分别对颜色源矩阵进行校正处理，得到至少两个颜色输出矩阵；
优化模块430，包括：
极小值确定单元，设置为根据所述损失函数值对至少两个颜色校正矩阵进行优化直至损失函数值收敛到极小值，得到至少两个极小值。
最优颜色校正矩阵确定单元，设置为确定至少两个极小值中的最小值，将该最小值对应的更新后的颜色校正矩阵，作为最优颜色校正矩阵。
本申请实施例提供的颜色校正矩阵优化装置可以设置为执行上述任意实施例提供的颜色校正矩阵优化方法，具备相应的功能和有益效果。
图7为本申请一种实施例提供的电子设备的结构示意图。电子设备10旨在表示多种形式的数字计算机，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备10还可以表示多种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、用户设备、可穿戴设备(如头盔、眼镜、手表等)和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本申请的实现。
如图7所示，电子设备10包括至少一个处理器11，以及与至少一个处理器11通信连接的存储器，如只读存储器(ROM)12、随机访问存储器(RAM)13等，其中，存储器存储有可被至少一个处理器执行的计算机程序，处理器11可以根据存储在只读存储器(ROM)12中的计算机程序或者从存储单元18加载到随机访问存储器(RAM)13中的计算机程序，来执行多种适当的动作和处理。在RAM 13中，还可存储电子设备10操作所需的多种程序和数据。处理器11、ROM 12以及RAM 13通过总线14彼此相连。输入/输出(I/O)接口15也连接 至总线14。
电子设备10中的多个部件连接至I/O接口15，包括：输入单元16，例如键盘、鼠标等；输出单元17，例如多种类型的显示器、扬声器等；存储单元18，例如磁盘、光盘等；以及通信单元19，例如网卡、调制解调器、无线通信收发机等。通信单元19允许电子设备10通过诸如因特网的计算机网络和/或多种电信网络、无线网络与其他设备交换信息/数据。
处理器11可以是多种具有处理和计算能力的通用和/或专用处理组件。处理器11的一些示例包括但不限于中央处理单元(CPU)、图形处理单元(GPU)、多种专用的人工智能(AI)计算芯片、多种运行机器学习模型算法的处理器、数字信号处理器(DSP)、以及任何适当的处理器、控制器、微控制器等。处理器11执行上文所描述的多个方法和处理，例如颜色校正矩阵优化方法。
在一些实施例中，颜色校正矩阵优化方法可被实现为计算机程序，其被有形地包含于计算机可读存储介质，例如存储单元18。在一些实施例中，计算机程序的部分或者全部可以经由ROM 12和/或通信单元19而被载入和/或安装到电子设备10上。当计算机程序加载到RAM 13并由处理器11执行时，可以执行上文描述的方法的一个或多个步骤。备选地，在其他实施例中，处理器11可以通过其他任何适当的方式(例如，借助于固件)而被配置为执行颜色校正矩阵优化方法。
本文中以上描述的系统和技术的多种实施方式可以在数字电子电路系统、集成电路系统、场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、芯片上系统的系统(SOC)、负载可编程逻辑设备(CPLD)、计算机硬件、固件、软件、和/或它们的组合中实现。这些多种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。
用于实施本申请的方法的计算机程序可以采用一个或多个编程语言的任何组合来编写。这些计算机程序可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理器，使得计算机程序当由处理器执行时使流程图和/或框图中所规定的功能/操作被实施。计算机程序可以完全在机器上执行、部分地在机器上执行，作为独立软件包部分地在机器上执行且部分地在远程机器上执行或完全在远程机器或服务器上执行。
在本申请的上下文中，计算机可读存储介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备 结合地使用的计算机程序。计算机可读存储介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。备选地，计算机可读存储介质可以是机器可读信号介质。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。计算机可读存储介质可以为非暂态计算机可读存储介质。
为了提供与用户的交互，可以在电子设备10上实施此处描述的系统和技术，该电子设备10具有：用于向用户显示信息的显示装置(例如，阴极射线管(CRT)或者液晶显示器(LCD)监视器)；以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给电子设备10。其它种类的装置还可以用于提供与用户的交互；例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。
可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：局域网(LAN)、广域网(WAN)、区块链网络和互联网。
计算系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。服务器可以是云服务器，又称为云计算服务器或云主机，是云计算服务体系中的一项主机产品，以避免相关技术中的物理主机与虚拟专用服务器(Virtual Private Server，VPS)中，存在的管理难度大，业务扩展性弱的情况。
应该理解，可以使用上面所示的多种形式的流程，重新排序、增加或删除步骤。例如，本申请中记载的多个步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本申请的技术方案所期望的结果，本文在此不进行限制。

Claims (10)

  1. 一种颜色校正矩阵优化方法，包括：
    基于颜色校正矩阵，对采集色卡图像的颜色源矩阵进行校正处理，得到颜色输出矩阵；
    根据所述颜色输出矩阵的Lab坐标和目标色卡图像的颜色目标矩阵的Lab坐标，确定全局色相误差和饱和度约束正则项，并根据所述全局色相误差和所述饱和度约束正则项构造损失函数；其中，Lab坐标为在Lab颜色空间中的坐标；
    根据损失函数值对所述颜色校正矩阵进行更新，直到所述损失函数值收敛，得到最优颜色校正矩阵，以基于所述最优颜色校正矩阵对图像采集器采集的待处理图像进行颜色校正。
  2. 根据权利要求1所述的方法，其中，所述根据所述颜色输出矩阵的Lab坐标和目标色卡图像的颜色目标矩阵的Lab坐标，确定全局色相误差和饱和度约束正则项，并根据所述全局色相误差和所述饱和度约束正则项，构造损失函数，包括：
    针对所述颜色输出矩阵和所述颜色目标矩阵中多列元素代表的多个色块，确定每个色块对应的色度坐标；其中，色度坐标为所述颜色输出矩阵和所述颜色目标矩阵转换到Lab颜色空间后在色度平面中的坐标；
    根据每个色块对应的色度坐标，确定全局色相误差和饱和度约束正则项；
    根据所述饱和度约束正则项与所述全局色相误差的和，构造损失函数。
  3. 根据权利要求2所述的方法，其中，所述根据每个色块对应的色度坐标，确定全局色相误差，包括：
    针对所述颜色输出矩阵和所述颜色目标矩阵中同一列元素表示的色块对，根据所述色块对的色度坐标向量的夹角，确定所述色块对的色相误差；其中，色度坐标向量为以原点为起点，以色度坐标为终点的向量；
    根据多个色块对的色相误差的加权平均值，计算所述全局色相误差。
  4. 根据权利要求2所述的方法，其中，所述根据每个色块对应的色度坐标，确定饱和度约束正则项，包括：
    确定颜色输出矩阵中多列元素表示的多个色块的色度坐标与原点的第一距离的平均值，并确定颜色目标矩阵中多列元素表示的多个色块的色度坐标与原点的第二距离的平均值；
    确定所述第一距离的平均值与所述第二距离的平均值的比值，并根据所述比值与1的差值，确定所述饱和度约束正则项。
  5. 根据权利要求1所述的方法，所述根据损失函数值对所述颜色校正矩阵进行更新之前，还包括：
    对于色相旋转矩阵和饱和度伸缩矩阵构成的所述颜色校正矩阵，左乘YUV颜色空间到RGB颜色空间的映射矩阵，右乘RGB颜色空间到YUV颜色空间的 映射矩阵，得到初始化的三阶颜色校正矩阵；
    基于白平衡约束，将所述初始化的三阶颜色校正矩阵的每一行中的其中一个元素用另外两个元素进行表示，得到包含六个变量的颜色校正矩阵；
    所述根据损失函数值对所述颜色校正矩阵进行更新，包括：
    根据所述损失函数值对六个变量进行更新，以对所述颜色校正矩阵进行更新。
  6. 根据权利要求1或5所述的方法，其中，所述颜色校正矩阵的初始化过程包括以下之一：
    根据色相旋转矩阵中的色相旋转因子的取值范围，以及饱和度伸缩矩阵中饱和度伸缩因子的取值范围，确定所述色相旋转因子和所述饱和度伸缩因子；根据所述色相旋转因子和所述饱和度伸缩因子，确定至少两个颜色校正矩阵；以及
    根据所述色相旋转因子和所述饱和度伸缩因子的概率分布模型，随机生成至少两组色相旋转因子和饱和度伸缩因子；根据至少两组色相旋转因子和饱和度伸缩因子，确定至少两个颜色校正矩阵。
  7. 根据权利要求6所述的方法，其中，所述基于颜色校正矩阵，对采集色卡图像的颜色源矩阵进行校正处理，得到颜色输出矩阵，包括：
    基于至少两个颜色校正矩阵分别对采集色卡图像的颜色源矩阵进行校正处理，得到至少两个颜色输出矩阵；
    所述根据损失函数值对所述颜色校正矩阵进行更新，直到所述损失函数值收敛，得到最优颜色校正矩阵，包括：
    根据所述损失函数值对所述至少两个颜色校正矩阵进行优化直至损失函数值收敛到极小值，得到至少两个极小值；
    确定所述至少两个极小值中的最小值，将所述最小值对应的更新后的颜色校正矩阵，作为最优颜色校正矩阵。
  8. 一种颜色校正矩阵优化装置，包括：
    校正处理模块，设置为基于颜色校正矩阵，对采集色卡图像的颜色源矩阵进行校正处理，得到颜色输出矩阵；
    损失函数构造模块，设置为根据所述颜色输出矩阵的Lab坐标和目标色卡图像的颜色目标矩阵的Lab坐标，确定全局色相误差和饱和度约束正则项，并根据所述全局色相误差和所述饱和度约束正则项构造损失函数；其中，Lab坐标为在Lab颜色空间中的坐标；
    优化模块，设置为根据损失函数值对所述颜色校正矩阵进行更新，直到所述损失函数值收敛，得到最优颜色校正矩阵，以基于所述最优颜色校正矩阵对图像采集器采集的待处理图像进行颜色校正。
  9. 一种电子设备，包括：
    至少一个处理器；以及
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的计算机程序，所述计算机程序被所述至少一个处理器执行，以使所述至少一个处理器能够执行如权利要求1-7中任一所述的颜色校正矩阵优化方法。
  10. 一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现如权利要求1-7中任一所述的颜色校正矩阵优化方法。
PCT/CN2022/142609 2022-08-04 2022-12-28 一种颜色校正矩阵优化方法、装置、电子设备及介质 WO2024027090A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210932339.XA CN117575954A (zh) 2022-08-04 2022-08-04 一种颜色校正矩阵优化方法、装置、电子设备及介质
CN202210932339.X 2022-08-04

Publications (1)

Publication Number Publication Date
WO2024027090A1 true WO2024027090A1 (zh) 2024-02-08

Family

ID=89848461

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/142609 WO2024027090A1 (zh) 2022-08-04 2022-12-28 一种颜色校正矩阵优化方法、装置、电子设备及介质

Country Status (2)

Country Link
CN (1) CN117575954A (zh)
WO (1) WO2024027090A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111861922A (zh) * 2020-07-21 2020-10-30 浙江大华技术股份有限公司 颜色校正矩阵的调整方法及装置、存储介质
CN112073703A (zh) * 2020-08-31 2020-12-11 深圳市景阳科技股份有限公司 一种色彩校正矩阵的调整方法、装置、终端设备及介质
US20210407365A1 (en) * 2020-06-26 2021-12-30 Magic Leap, Inc. Color uniformity correction of display device
CN113947533A (zh) * 2020-07-16 2022-01-18 浙江宇视科技有限公司 图像颜色校正方法、装置、设备和存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210407365A1 (en) * 2020-06-26 2021-12-30 Magic Leap, Inc. Color uniformity correction of display device
CN113947533A (zh) * 2020-07-16 2022-01-18 浙江宇视科技有限公司 图像颜色校正方法、装置、设备和存储介质
CN111861922A (zh) * 2020-07-21 2020-10-30 浙江大华技术股份有限公司 颜色校正矩阵的调整方法及装置、存储介质
CN112073703A (zh) * 2020-08-31 2020-12-11 深圳市景阳科技股份有限公司 一种色彩校正矩阵的调整方法、装置、终端设备及介质

Also Published As

Publication number Publication date
CN117575954A (zh) 2024-02-20

Similar Documents

Publication Publication Date Title
KR102605528B1 (ko) 근적외선 이미지 생성 방법, 생성 네트워크의 트레이닝 방법 및 장치
US10791310B2 (en) Method and system of deep learning-based automatic white balancing
WO2022012085A1 (zh) 人脸图像处理方法、装置、存储介质及电子设备
WO2023016320A1 (zh) 图像处理方法、装置、设备及介质
KR20080113950A (ko) 감마 보정 곡선의 자동 산출 장치 및 방법
CN112991180B (zh) 图像拼接方法、装置、设备以及存储介质
WO2021136391A1 (zh) 图像处理方法、图像处理装置和显示装置
WO2017101534A1 (zh) 液晶显示屏的色彩信号转换方法及装置
US11380284B2 (en) Color space mapping method, equipment, computer readable storage medium, and device
WO2019062633A1 (zh) 颜色阴影校正的方法和装置
US10540811B2 (en) Radial gradient module
JP2023552538A (ja) 画像処理方法及び装置、電子機器、記憶媒体ならびにコンピュータプログラム
CN113436105A (zh) 模型训练和图像优化方法、装置、电子设备及存储介质
US8249363B2 (en) Image comparison system and method
WO2022120799A1 (zh) 图像处理方法、装置、电子设备及存储介质
WO2024027090A1 (zh) 一种颜色校正矩阵优化方法、装置、电子设备及介质
CN113344213A (zh) 知识蒸馏方法、装置、电子设备及计算机可读存储介质
CN113590071A (zh) 基于抖动处理的图像处理方法、装置、计算机设备及介质
CN113947179A (zh) 白平衡校正方法、装置、设备和存储介质
CN116843566A (zh) 色调映射方法、装置、显示设备和存储介质
WO2023015993A9 (zh) 一种色度信息的确定方法及相关电子设备
CN110009676B (zh) 一种双目图像的本征性质分解方法
US8630488B2 (en) Creating a duotone color effect using an ICC profile
CN106791757B (zh) 影像校正系统及其方法
TWI749821B (zh) 圖像特徵比對處理方法與系統

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22953878

Country of ref document: EP

Kind code of ref document: A1