CN111565306A - Automatic white balance method and storage device for three-dimensional space white point detection and brightness weighting white point - Google Patents
Automatic white balance method and storage device for three-dimensional space white point detection and brightness weighting white point Download PDFInfo
- Publication number
- CN111565306A CN111565306A CN202010229027.3A CN202010229027A CN111565306A CN 111565306 A CN111565306 A CN 111565306A CN 202010229027 A CN202010229027 A CN 202010229027A CN 111565306 A CN111565306 A CN 111565306A
- Authority
- CN
- China
- Prior art keywords
- white point
- white
- point
- light source
- axis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000001514 detection method Methods 0.000 title claims abstract description 25
- 238000004364 calculation method Methods 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 235000019557 luminance Nutrition 0.000 description 28
- 239000013598 vector Substances 0.000 description 16
- 239000011159 matrix material Substances 0.000 description 12
- 238000010586 diagram Methods 0.000 description 9
- 230000004044 response Effects 0.000 description 5
- 230000007423 decrease Effects 0.000 description 4
- 230000009977 dual effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 230000001788 irregular Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 239000002023 wood Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
- H04N23/88—Camera processing pipelines; Components thereof for processing colour signals for colour balance, e.g. white-balance circuits or colour temperature control
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Color Television Image Signal Generators (AREA)
- Processing Of Color Television Signals (AREA)
Abstract
The invention relates to the technical field of image processing, in particular to an automatic white balance method and storage equipment for three-dimensional space white point detection and brightness weighting white point. The automatic white balance method for the three-dimensional space white point detection and the brightness weighting white point comprises the following steps: calibrating 2D white point intervals of different light sources on a UV plane, and primarily judging points falling in the 2D white point intervals as white points corresponding to the light sources; further detecting the white point which is preliminarily determined to be the white point corresponding to the light source through a Y-axis white point interval to obtain a final white point; and carrying out color temperature estimation on the final white point through brightness weighting to finish automatic white balance operation. The operation has double guarantee for the detection of the white point, the white point is determined as the final white point only when the white point is in the 2D white point interval and the Y-axis white point interval, the white point detection accuracy is greatly improved, the color temperature of the detected white point is estimated through brightness weighting, and the error rate of color temperature estimation can be greatly reduced.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an automatic white balance method and storage equipment for three-dimensional space white point detection and brightness weighting white point.
Background
In real life, more and more people start to use the image pickup function, and people demand higher and higher image pickup quality. Wherein the perception of the same color is substantially the same under different light when one views the natural world with the eye. This ability to eliminate or mitigate the effects of light sources and achieve "seeing" the color of the surface of an actual object is called color constancy.
However, various digital cameras do not have the capability that under different illumination, target objects of the same actual color may exhibit imaging colors that differ widely. This is because the image sensor on the digital camera simply records all the light projected onto it, and cannot itself distinguish whether the color light projected onto the image sensor is due to color reflections from the object itself, or due to color-shifting ambient light. By utilizing the automatic white balance technology, the digital camera can automatically estimate the environmental color light and remove the influence of the scene illumination color, so that the digital camera can have the same color constancy as the human visual system.
The conventional automatic white balance algorithm detects a gray white point by setting a white point interval of a two-dimensional CbCr plane, the deviation between a real gray white point and an ideal gray white point under each light source is large, in order to detect more real gray white points, the 3D white point plane may need to be enlarged, but the probability that other non-white points are detected as white points is also increased.
In addition, there is a problem in estimating color temperature by using RGB components commonly used in the prior art, for example, the RGB responses of a wood color under a cold fluorescent lamp and a white surface under a candle light may be relatively close, and it is difficult for the existing white balance algorithm to correctly estimate the two color temperatures, and color cast may be caused once estimation error occurs.
Disclosure of Invention
Therefore, it is necessary to provide an automatic white balance method for white point detection and brightness weighting white point in three-dimensional space, so as to solve the problems of low white point detection accuracy and low color temperature estimation accuracy of the existing white balance detection method. The specific technical scheme is as follows:
an automatic white balance method for three-dimensional space white point detection and brightness weighting white point comprises the following steps:
calibrating 2D white point intervals of different light sources on a UV plane, and primarily judging points falling in the 2D white point intervals as white points corresponding to the light sources;
fitting a 3D white point plane, rotating the 3D white point plane to enable the 3D white point plane to be vertical to a Y axis, and calibrating Y-axis white point intervals corresponding to different light sources to obtain a 3D white point space formed by the 3D white point plane and the Y-axis white point intervals;
further detecting the white point which is preliminarily determined to be the white point corresponding to the light source through a Y-axis white point interval to obtain a final white point;
and carrying out color temperature estimation on the final white point through brightness weighting to finish automatic white balance operation.
Further, the "preliminarily determining a point falling within the 2D white point interval as the white point corresponding to the light source" further includes:
and carrying out RGB-to-YUV conversion operation on the input image, judging whether the point is positioned in a white point interval of a preset convex quadrilateral, and if the point is positioned in the white point interval of the preset convex quadrilateral, preliminarily judging that the point is the white point of the light source corresponding to the quadrilateral.
Further, the "further detecting a final white point for the white point preliminarily determined to correspond to the light source by the Y-axis white point interval" further includes:
projecting the white point which is preliminarily determined as the light source corresponding to the quadrangle to the 3D white point space;
and judging whether the deviation of the Y component between the white point of the light source corresponding to the quadrangle and the ideal white point is smaller than a preset threshold value or not, and if the deviation is smaller than the preset threshold value, judging that the white point of the light source corresponding to the quadrangle is the final white point.
Further, the "estimating the color temperature of the last white point by luminance weighting to complete the automatic white balance operation" further includes the steps of:
setting a brightness weight curve, and calculating the weight of a white point according to the brightness Y interpolation of the white point;
and performing white balance gain calculation by combining the weights to complete automatic white balance operation.
Furthermore, different luminances correspond to different luminance weight curves;
different white point numbers account for different proportions of the total white point number and correspond to different brightness weight curves.
In order to solve the technical problem, the storage device is further provided, and the specific technical scheme is as follows:
a storage device having stored therein a set of instructions for performing:
calibrating 2D white point intervals of different light sources on a UV plane, and primarily judging points falling in the 2D white point intervals as white points corresponding to the light sources;
fitting a 3D white point plane, rotating the 3D white point plane to enable the 3D white point plane to be vertical to a Y axis, and calibrating Y-axis white point intervals corresponding to different light sources to obtain a 3D white point space formed by the 3D white point plane and the Y-axis white point intervals;
further detecting the white point which is preliminarily determined to be the white point corresponding to the light source through a Y-axis white point interval to obtain a final white point;
and carrying out color temperature estimation on the final white point through brightness weighting to finish automatic white balance operation.
Further, the set of instructions is further for performing:
the "preliminarily determining a point falling within the 2D white point interval as a white point corresponding to the light source" further includes:
and carrying out RGB-to-YUV conversion operation on the input image, judging whether the point is positioned in a white point interval of a preset convex quadrilateral, and if the point is positioned in the white point interval of the preset convex quadrilateral, preliminarily judging that the point is the white point of the light source corresponding to the quadrilateral.
Further, the set of instructions is further for performing:
the method further comprises the following steps of further detecting the final white point for the white point which is preliminarily determined to be corresponding to the light source through the Y-axis white point interval:
projecting the white point which is preliminarily determined as the light source corresponding to the quadrangle to the 3D white point space;
and judging whether the deviation of the Y component between the white point of the light source corresponding to the quadrangle and the ideal white point is smaller than a preset threshold value or not, and if the deviation is smaller than the preset threshold value, judging that the white point of the light source corresponding to the quadrangle is the final white point.
Further, the set of instructions is further for performing:
the method comprises the following steps of carrying out color temperature estimation on the final white point through brightness weighting to finish automatic white balance operation, and further comprises the following steps:
setting a brightness weight curve, and calculating the weight of a white point according to the brightness Y interpolation of the white point;
and performing white balance gain calculation by combining the weights to complete automatic white balance operation.
Furthermore, different luminances correspond to different luminance weight curves;
different white point numbers account for different proportions of the total white point number and correspond to different brightness weight curves.
The invention has the beneficial effects that: the method comprises the steps of initially judging a point falling in a 2D white point interval as a white point corresponding to a light source by calibrating the 2D white point interval of different light sources on a UV plane, wherein the point is the initial judgment of the white point corresponding to the light source, and then fitting a 3D white point plane; and rotating the 3D white point plane to be vertical to the Y axis, and calibrating Y-axis white point intervals corresponding to different light sources to obtain a 3D white point space formed by the 3D white point plane and the Y-axis white point intervals. Further detecting the white point which is preliminarily determined to be the corresponding white point of the light source through a Y-axis white point interval to obtain a final white point; the detection of the white point is a dual guarantee, the final white point is determined only by the white point which is in the 2D white point interval and the Y-axis white point interval, the white point detection accuracy is greatly improved, and after the white point is detected, the color temperature of the detected white point is estimated by luminance weighting, so that the influence of color temperature estimation errors caused by the fact that RGB values sensed by a sensor are the same and the color temperatures are different can be greatly reduced.
Drawings
FIG. 1 is a flow chart of a method for automatic white balance of a three-dimensional space white point detection and luminance weighted white point according to an embodiment;
FIG. 2 is a schematic diagram of the numbers of X-Rite24 color chips and color blocks in the detailed description;
FIG. 3 is a color chip diagram of various light sources according to one embodiment;
FIG. 4 is a schematic diagram of UV spatial white point detection according to an embodiment;
FIG. 5 is a diagram of a color card in YUV space according to an embodiment;
FIG. 6 is a schematic view of an embodiment of the present invention shown rotated about the U-axis;
FIG. 7 is a schematic view of the embodiment shown rotated about the V-axis;
FIG. 8 is a schematic view of an embodiment shown rotated about the Y axis;
FIG. 9 illustrates a RotateYUV space in accordance with an embodiment;
FIG. 10 is a schematic diagram of a Y-axis white point space according to an embodiment;
FIG. 11 is a schematic view of the vertical calculation according to the embodiment;
FIG. 12 is a diagram illustrating a luminance weight curve according to an embodiment;
fig. 13 is a block diagram of a storage device according to an embodiment.
Description of reference numerals:
1300. a storage device.
Detailed Description
To explain technical contents, structural features, and objects and effects of the technical solutions in detail, the following detailed description is given with reference to the accompanying drawings in conjunction with the embodiments.
Referring to fig. 1 to 12, in the present embodiment, an automatic white balance method for three-dimensional space white point detection and brightness weighted white point can be applied to a storage device, including but not limited to: personal computer, server, general-purpose computer, special-purpose computer, network equipment, embedded equipment, programmable equipment, intelligent mobile terminal, intelligent home equipment, wearable intelligent equipment, vehicle-mounted intelligent equipment and the like, wherein the storage equipment can have the function of camera shooting. The following is a detailed description:
an automatic white balance method for three-dimensional space white point detection and brightness weighting white point comprises the following steps:
step S101: and calibrating 2D white point intervals of different light sources on the UV plane, and primarily determining points falling in the 2D white point intervals as white points corresponding to the light sources.
Step S102: and fitting a 3D white point plane, rotating the 3D white point plane to enable the 3D white point plane to be vertical to the Y axis, and calibrating Y-axis white point intervals corresponding to different light sources to obtain a 3D white point space formed by the 3D white point plane and the Y-axis white point intervals.
Step S103: and further detecting the final white point for the white point which is preliminarily determined to be corresponding to the light source through the Y-axis white point interval.
Step S104: and carrying out color temperature estimation on the final white point through brightness weighting to finish automatic white balance operation.
In the present embodiment, the 2D white point interval and the Y-axis white point interval may be first calibrated, and then the determination operation of whether or not the point falls within the 2D white point interval and the Y-axis white point interval may be performed on the input image. Namely: whether the points of the input image fall in the 2D white point interval or not and the Y-axis white point interval are determined according to the calibration operation steps.
Specifically, the method further comprises the following steps before the step S101: as shown in FIG. 2, X-Rite24 color cards under different light sources were photographed in Judge3 light box. Which is demosaiced as shown in figure 3.
Specifically, the step S101 of "preliminarily determining a point falling within the 2D white point interval as the white point corresponding to the light source" further includes the steps of:
and carrying out RGB-to-YUV conversion operation on the input image, judging whether the point is positioned in a white point interval of a preset convex quadrilateral, and if the point is positioned in the white point interval of the preset convex quadrilateral, preliminarily judging that the point is the white point of the light source corresponding to the quadrilateral.
The following method can be specifically adopted:
as shown in fig. 4, seven quadrangles are the white point intervals of the calibrated seven light sources. UV plan 4 represents different light sources, and the numbers 1-24 represent the color patches of the color chart. It can be seen that the white points (blocks 19 to 24) of the individual light sources are substantially distributed in a straight line, the white points of all light sources enclosing an approximately fan-shaped area, from cyan to magenta (from top to bottom) indicating a change in color temperature from HZ to D75. Since the large sector area also contains some non-white dots, the subdivision into a plurality of quadrilateral areas in combination with the subsequent Y-axis white dot area can eliminate a part of the non-white dots. The area enclosed by the cyan quadrangle represents the white point range of HZ, and the points falling within this interval are the white points. While such subdivision supports assigning different weights to the white points of different light sources, the weights may be controlled by other factors such as brightness.
The point falling within the area of each of the nominally convex 4-sided polygons is the white point of the corresponding light source.
The matrix from 8bitRGB to 8bitYUV is
The method is also suitable for other spaces, such as YIQ and the like.
Step S102 may specifically adopt the following manner:
extracting each color block of the color card, and fitting a three-dimensional white point plane based on the gray blocks of each color card, as shown in fig. 5:
the straight line is the white point straight line fitted to each light source, and it can be seen that there is also a non-white point on the white point plane, and there are fewer non-white points on the white point plane for locations with large RGB response (corresponding to locations of 19 color patches on the graph, top left), and overlapping portions of white points and non-white points for locations with small RGB response (corresponding to locations of 24 color patches on the graph, bottom right).
The white point fitting white point planes of A, D50, D65, D75 were selected. The white point under the CWF, TL84 and HZ light sources has some deviation on the plane from the white point of other light sources, so the white point does not participate in the white point plane fitting.
The general expression of the spatial plane equation is: au + Bv + Cy + D ═ 0; then there are: the normal vector of the plane is
. The fitting of discrete points in space results in a plane, which is in fact an optimization process. I.e. the problem of minimizing the sum of the distances of these points to a certain plane. According to SVD transformation of the covariance matrix, the singular vector corresponding to the minimum singular value is the direction of the plane.
In order to make the rotated white point plane parallel to the UV plane, i.e. the white point plane normal vector is parallel to the Y-axis. The normal vectors (A, B, C) are rotated around the U axis to obtain vectors (A,0, K) and then rotated around the V axis to obtain vectors (0,0, L) parallel to the Y axis. Finally, translation and scaling are needed to ensure that the whole value range is not negative and the fixed range is not negative.
1. Wherein the schematic view of rotation about the U axis is shown in figure 6.
A rotation matrix of
Wherein
2. And then rotated around the V-axis, which is schematically shown in fig. 7, with the normal vector of the white point plane parallel to the Y-axis.
A rotation matrix of
Wherein,
3. rotating around Y axis to make white point line parallel to U axis as much as possible, and taking experimental value theta 3 of 100 deg. as shown in FIG. 8
A rotation matrix of
4. Of space after rotation (u)min,vmin,ymin) Move to the origin (make the value non-negative) and scale the value range of Y, U, V to [0,255]
The translation scaling matrices are respectively
5. From RGB to rotated YUV space matrix is
MRGB2RotateYUV＝MscaleMmove*My*Mv*Mu*MRGB2YUV
Finally from RGB to rotated YUV space
The rotated space rotayuv (i.e., the 3D white point space) is shown in fig. 9.
From the rotated yuv, it can be seen that the non-white block is far from the white point plane compared with the white block, but the distance from the white point to the white point plane is different under different luminances of different light sources, so the light source and luminance factors are considered when identifying the white point through the Y component.
Further, step S103 "further detects a final white point for the white point primarily determined to correspond to the light source through the Y-axis white point interval", further includes the steps of: projecting the white point which is preliminarily determined as the light source corresponding to the quadrangle to the 3D white point space; and judging whether the deviation of the Y component between the white point of the light source corresponding to the quadrangle and the ideal white point is smaller than a preset threshold value or not, and if the deviation is smaller than the preset threshold value, judging that the white point of the light source corresponding to the quadrangle is the final white point. The following method can be specifically adopted:
1. projecting white points of all light sources to the 3D white point space;
2. fitting a white point straight line of each light source, wherein the white point straight line is represented by two points on the straight line; as can be seen from the white point line fitted to the different light sources in fig. 9: the distances of the white points to the white point plane are different for different light sources and the distance of the white point to the white point plane under one light source increases with increasing component (brightness) at rtU.
3. The white point Y component deviation threshold line rtU-Th is calibrated for different luminances (U components) of different light sources.
The rtY component falls inside the irregular polyhedron as shown in fig. 10 as a white point space. The polyhedron decreases in the rtY direction as the rtU component decreases.
And (3) detecting a white point of a Y axis:
(1) the white point x is known by the white point identification algorithm through the UV space0(rtU0,rtV0,rtY0) Belongs to a light source i;
(2) calculating x0The vertical point p (rtU) of the point to the ith light source white point linep,rtVp,rtYp) The vertical point p is the ideal white point. Known point x0Two points x on a straight line1，x2(ii) a Find point x0In a straight line the foot p is
The schematic diagram of the calculation of the vertical point is shown in fig. 11. Where ● represents the vector dot product.
(3) Calculating a Y component deviation threshold Th
Will make the point rtU0And (5) carrying in an rtU-Th straight line under the ith calibrated light source to obtain Th.
(4) If the actual white point x0Deviation from the ideal white point Y component is less than a threshold Th, the last white point is identified, otherwise the non-white point.
I.e. | rtY0-rtYpIf | < Th, the last white point is identified, otherwise, no white point.
Further, step S104 "perform color temperature estimation on the final white point by luminance weighting to complete the automatic white balance operation", further includes the steps of:
setting a brightness weight curve, and calculating the weight of a white point according to the brightness Y interpolation of the white point;
and performing white balance gain calculation by combining the weights to complete automatic white balance operation.
Wherein different luminances correspond to different luminance weight curves; different white point numbers account for different proportions of the total white point number and correspond to different brightness weight curves.
The method specifically comprises the following steps: and setting a brightness weight curve, and interpolating according to the brightness Y of the white point to obtain the weight of the point. The number of white points is enabled when the total number of white points exceeds a certain threshold and the scene brightness exceeds a certain threshold.
In practical applications, different luminance weight curves can be applied when the brightness or the ratio of the white point number of the proper luminance to the total white point number is different, as shown in fig. 12.
The method of luminance weighting may also be used in conjunction with the white point detected by the method.
The white balance gain applied is:
WBGainG＝1
wherein SumR, SumG, and SumB are white point weighted sums,
SumR＝∑RiWi
SumG＝∑GiWi
SumB＝∑BiWi
the method comprises the steps of initially judging a point falling in a 2D white point interval as a white point corresponding to a light source by calibrating the 2D white point interval of different light sources on a UV plane, wherein the point is the initial judgment of the white point corresponding to the light source, and then fitting a 3D white point plane; and rotating the 3D white point plane to be vertical to the Y axis, and calibrating Y-axis white point intervals corresponding to different light sources to obtain a 3D white point space formed by the 3D white point plane and the Y-axis white point intervals. Further detecting the white point which is preliminarily determined to be the corresponding white point of the light source through a Y-axis white point interval to obtain a final white point; the detection of the white point is a dual guarantee, the final white point is determined only by the white point which is in the 2D white point interval and the Y-axis white point interval, the white point detection accuracy is greatly improved, and after the white point is detected, the color temperature of the detected white point is estimated by luminance weighting, so that the influence of color temperature estimation errors caused by the fact that RGB values sensed by a sensor are the same and the color temperatures are different can be greatly reduced.
Referring to fig. 2 to fig. 13, in the present embodiment, an embodiment of a memory device 1300 is as follows:
the storage device 1300 includes, but is not limited to: personal computer, server, general-purpose computer, special-purpose computer, network equipment, embedded equipment, programmable equipment, intelligent mobile terminal, intelligent home equipment, wearable intelligent equipment, vehicle-mounted intelligent equipment and the like, wherein the storage equipment can have the function of camera shooting.
The following is a detailed description:
a storage device 1300 having stored therein a set of instructions for performing:
calibrating 2D white point intervals of different light sources on a UV plane, and primarily judging points falling in the 2D white point intervals as white points corresponding to the light sources;
fitting a 3D white point plane, rotating the 3D white point plane to enable the 3D white point plane to be vertical to a Y axis, and calibrating Y-axis white point intervals corresponding to different light sources to obtain a 3D white point space formed by the 3D white point plane and the Y-axis white point intervals;
further detecting the white point which is preliminarily determined to be the white point corresponding to the light source through a Y-axis white point interval to obtain a final white point;
and carrying out color temperature estimation on the final white point through brightness weighting to finish automatic white balance operation.
Specifically, the method further comprises the following steps before the command is executed: as shown in FIG. 2, X-Rite24 color cards under different light sources were photographed in Judge3 light box. Which is demosaiced as shown in figure 3.
Specifically, the step S101 of "preliminarily determining a point falling within the 2D white point interval as the white point corresponding to the light source" further includes the steps of:
and carrying out RGB-to-YUV conversion operation on the input image, judging whether the point is positioned in a white point interval of a preset convex quadrilateral, and if the point is positioned in the white point interval of the preset convex quadrilateral, preliminarily judging that the point is the white point of the light source corresponding to the quadrilateral.
The following method can be specifically adopted:
as shown in FIG. 4, the UV plan 4 represents different light sources, and the numbers 1-24 represent color patches of the color chart. It can be seen that the white points (blocks 19 to 24) of the individual light sources are substantially distributed in a straight line, the white points of all light sources enclosing an approximately fan-shaped area, from cyan to magenta (from top to bottom) indicating a change in color temperature from HZ to D75. Since the large sector area also contains some non-white dots, the subdivision into a plurality of quadrilateral areas in combination with the subsequent Y-axis white dot area can eliminate a part of the non-white dots. The area enclosed by the cyan quadrangle represents the white point range of HZ, and the points falling within this interval are the white points. While such subdivision supports assigning different weights to the white points of different light sources, the weights may be controlled by other factors such as brightness.
The point falling within the area of each of the nominally convex 4-sided polygons is the white point of the corresponding light source.
The matrix from 8bitRGB to 8bitYUV is
The method is also suitable for other spaces, such as YIQ and the like.
Step S102 may specifically adopt the following manner:
extracting each color block of the color card, and fitting a three-dimensional white point plane based on the gray blocks of each color card, as shown in fig. 5:
the straight line is the white point straight line fitted to each light source, and it can be seen that there is also a non-white point on the white point plane, and there are fewer non-white points on the white point plane for locations with large RGB response (corresponding to locations of 19 color patches on the graph, top left), and overlapping portions of white points and non-white points for locations with small RGB response (corresponding to locations of 24 color patches on the graph, bottom right).
The white point fitting white point planes of A, D50, D65, D75 were selected. The white point under the CWF, TL84 and HZ light sources has some deviation on the plane from the white point of other light sources, so the white point does not participate in the white point plane fitting.
The general expression of the spatial plane equation is: au + Bv + Cy + D ═ 0; then there are: the normal vector of the plane is
. The fitting of discrete points in space results in a plane, which is in fact an optimization process. I.e. the problem of minimizing the sum of the distances of these points to a certain plane. According to SVD transformation of the covariance matrix, the singular vector corresponding to the minimum singular value is the direction of the plane.
In order to make the rotated white point plane parallel to the UV plane, i.e. the white point plane normal vector is parallel to the Y-axis. The normal vectors (A, B, C) are rotated around the U axis to obtain vectors (A,0, K) and then rotated around the V axis to obtain vectors (0,0, L) parallel to the Y axis. Finally, translation and scaling are needed to ensure that the whole value range is not negative and the fixed range is not negative.
1. Wherein the schematic view of rotation about the U axis is shown in figure 6.
A rotation matrix of
Wherein
2. And then rotated around the V-axis, which is schematically shown in fig. 7, with the normal vector of the white point plane parallel to the Y-axis.
A rotation matrix of
Wherein,
3. rotating around Y axis to make white point line parallel to U axis as much as possible, and taking experimental value theta 3 of 100 deg. as shown in FIG. 8
A rotation matrix of
4. Of space after rotation (u)min,vmin,ymin) Move to the origin (make the value non-negative) and scale the value range of Y, U, V to [0,255]
The translation scaling matrices are respectively
5. From RGB to rotated YUV space matrix is
MRGB2RotateYUV＝MscaleMmove*My*Mv*Mu*MRGB2YUV
Finally from RGB to rotated YUV space
The rotated space rotayuv (i.e., the 3D white point space) is shown in fig. 9.
From the rotated yuv, it can be seen that the non-white block is far from the white point plane compared with the white block, but the distance from the white point to the white point plane is different under different luminances of different light sources, so the light source and luminance factors are considered when identifying the white point through the Y component.
Further, step S103 "further detects a final white point for the white point primarily determined to correspond to the light source through the Y-axis white point interval", further includes the steps of: projecting the white point which is preliminarily determined as the light source corresponding to the quadrangle to the 3D white point space; and judging whether the deviation of the Y component between the white point of the light source corresponding to the quadrangle and the ideal white point is smaller than a preset threshold value or not, and if the deviation is smaller than the preset threshold value, judging that the white point of the light source corresponding to the quadrangle is the final white point. The following method can be specifically adopted:
1. projecting white points of all light sources to the 3D white point space;
2. fitting a white point straight line of each light source, wherein the white point straight line is represented by two points on the straight line; as can be seen from the white point line fitted to the different light sources in fig. 9: the distances of the white points to the white point plane are different for different light sources and the distance of the white point to the white point plane under one light source increases with increasing component (brightness) at rtU.
3. The white point Y component deviation threshold line rtU-Th is calibrated for different luminances (U components) of different light sources.
The rtY component falls inside the irregular polyhedron as shown in fig. 10 as a white point space. The polyhedron decreases in the rtY direction as the rtU component decreases.
And (3) detecting a white point of a Y axis:
(1) the white point x is known by the white point identification algorithm through the UV space0(rtU0,rtV0,rtY0) Belongs to a light source i;
(2) calculating x0The vertical point p (rtU) of the point to the ith light source white point linep,rtVp,rtYp) The vertical point p is the ideal white point. Known point x0Two points x on a straight line1，x2(ii) a Find point x0In a straight line the foot p is
The schematic diagram of the calculation of the vertical point is shown in fig. 11. Where · represents the vector dot product.
(3) Calculating a Y component deviation threshold Th
Will make the point rtU0And (5) carrying in an rtU-Th straight line under the ith calibrated light source to obtain Th.
(4) If the actual white point x0Deviation from the ideal white point Y component is less than a threshold Th, the last white point is identified, otherwise the non-white point.
I.e. | rtY0-rtYpIf | < Th, the last white point is identified, otherwise, no white point.
Further, step S104 "perform color temperature estimation on the final white point by luminance weighting to complete the automatic white balance operation", further includes the steps of:
setting a brightness weight curve, and calculating the weight of a white point according to the brightness Y interpolation of the white point;
and performing white balance gain calculation by combining the weights to complete automatic white balance operation.
Wherein different luminances correspond to different luminance weight curves; different white point numbers account for different proportions of the total white point number and correspond to different brightness weight curves.
The method specifically comprises the following steps: and setting a brightness weight curve, and interpolating according to the brightness Y of the white point to obtain the weight of the point. The number of white points is enabled when the total number of white points exceeds a certain threshold and the scene brightness exceeds a certain threshold.
In practical applications, different luminance weight curves can be applied when the brightness or the ratio of the white point number of the proper luminance to the total white point number is different, as shown in fig. 12.
The method of luminance weighting may also be used in conjunction with the white point detected by the method.
The white balance gain applied is:
WBGainG＝1
wherein SumR, SumG, and SumB are white point weighted sums,
SumR＝∑RiWi
SumG＝∑GiWi
SumB＝∑BiWi
executing the commands by executing the set of instructions on the storage device 1300: calibrating 2D white point intervals of different light sources on a UV plane, primarily judging points falling in the 2D white point intervals as white points corresponding to the light sources, primarily judging the white points corresponding to the light sources, and then fitting a 3D white point plane; and rotating the 3D white point plane to be vertical to the Y axis, and calibrating Y-axis white point intervals corresponding to different light sources to obtain a 3D white point space formed by the 3D white point plane and the Y-axis white point intervals. Further detecting the white point which is preliminarily determined to be the corresponding white point of the light source through a Y-axis white point interval to obtain a final white point; the detection of the white point is a dual guarantee, the final white point is determined only by the white point which is in the 2D white point interval and the Y-axis white point interval, the white point detection accuracy is greatly improved, and after the white point is detected, the color temperature of the detected white point is estimated by luminance weighting, so that the influence of color temperature estimation errors caused by the fact that RGB values sensed by a sensor are the same and the color temperatures are different can be greatly reduced.
It should be noted that, although the above embodiments have been described herein, the invention is not limited thereto. Therefore, based on the innovative concepts of the present invention, the technical solutions of the present invention can be directly or indirectly applied to other related technical fields by making changes and modifications to the embodiments described herein, or by using equivalent structures or equivalent processes performed in the content of the present specification and the attached drawings, which are included in the scope of the present invention.
Claims (10)
1. An automatic white balance method for detecting a white point in a three-dimensional space and weighting the white point by brightness is characterized by comprising the following steps:
calibrating 2D white point intervals of different light sources on a UV plane, and primarily judging points falling in the 2D white point intervals as white points corresponding to the light sources;
fitting a 3D white point plane, rotating the 3D white point plane to enable the 3D white point plane to be vertical to a Y axis, and calibrating Y-axis white point intervals corresponding to different light sources to obtain a 3D white point space formed by the 3D white point plane and the Y-axis white point intervals;
further detecting the white point which is preliminarily determined to be the white point corresponding to the light source through a Y-axis white point interval to obtain a final white point;
and carrying out color temperature estimation on the final white point through brightness weighting to finish automatic white balance operation.
2. The automatic white balancing method of three-dimensional space white point detection and luminance weighted white point of claim 1,
the "preliminarily determining a point falling within the 2D white point interval as a white point corresponding to the light source" further includes:
and carrying out RGB-to-YUV conversion operation on the input image, judging whether the point is positioned in a white point interval of a preset convex quadrilateral, and if the point is positioned in the white point interval of the preset convex quadrilateral, preliminarily judging that the point is the white point of the light source corresponding to the quadrilateral.
3. The automatic white balancing method of three-dimensional space white point detection and luminance weighted white point of claim 2,
the method further comprises the following steps of further detecting the final white point for the white point which is preliminarily determined to be corresponding to the light source through the Y-axis white point interval:
projecting the white point which is preliminarily determined as the light source corresponding to the quadrangle to the 3D white point space;
and judging whether the deviation of the Y component between the white point of the light source corresponding to the quadrangle and the ideal white point is smaller than a preset threshold value or not, and if the deviation is smaller than the preset threshold value, judging that the white point of the light source corresponding to the quadrangle is the final white point.
4. The automatic white balancing method of three-dimensional space white point detection and luminance weighted white point of claim 1,
the method comprises the following steps of carrying out color temperature estimation on the final white point through brightness weighting to finish automatic white balance operation, and further comprises the following steps:
setting a brightness weight curve, and calculating the weight of a white point according to the brightness Y interpolation of the white point;
and performing white balance gain calculation by combining the weights to complete automatic white balance operation.
5. The automatic white balancing method of three-dimensional space white point detection and luminance weighted white point of claim 4,
different brightness corresponds to different brightness weight curves;
different white point numbers account for different proportions of the total white point number and correspond to different brightness weight curves.
6. A storage device having a set of instructions stored therein, the set of instructions being operable to perform:
calibrating 2D white point intervals of different light sources on a UV plane, and primarily judging points falling in the 2D white point intervals as white points corresponding to the light sources;
fitting a 3D white point plane, rotating the 3D white point plane to enable the 3D white point plane to be vertical to a Y axis, and calibrating Y-axis white point intervals corresponding to different light sources to obtain a 3D white point space formed by the 3D white point plane and the Y-axis white point intervals;
further detecting the white point which is preliminarily determined to be the white point corresponding to the light source through a Y-axis white point interval to obtain a final white point;
and carrying out color temperature estimation on the final white point through brightness weighting to finish automatic white balance operation.
7. The storage device of claim 6, wherein the set of instructions is further configured to perform:
the "preliminarily determining a point falling within the 2D white point interval as a white point corresponding to the light source" further includes:
and carrying out RGB-to-YUV conversion operation on the input image, judging whether the point is positioned in a white point interval of a preset convex quadrilateral, and if the point is positioned in the white point interval of the preset convex quadrilateral, preliminarily judging that the point is the white point of the light source corresponding to the quadrilateral.
8. The storage device of claim 7, wherein the set of instructions is further configured to perform:
the method further comprises the following steps of further detecting the final white point for the white point which is preliminarily determined to be corresponding to the light source through the Y-axis white point interval:
projecting the white point which is preliminarily determined as the light source corresponding to the quadrangle to the 3D white point space;
and judging whether the deviation of the Y component between the white point of the light source corresponding to the quadrangle and the ideal white point is smaller than a preset threshold value or not, and if the deviation is smaller than the preset threshold value, judging that the white point of the light source corresponding to the quadrangle is the final white point.
9. The storage device of claim 6, wherein the set of instructions is further configured to perform:
the method comprises the following steps of carrying out color temperature estimation on the final white point through brightness weighting to finish automatic white balance operation, and further comprises the following steps:
setting a brightness weight curve, and calculating the weight of a white point according to the brightness Y interpolation of the white point;
and performing white balance gain calculation by combining the weights to complete automatic white balance operation.
10. A storage device according to claim 9,
different brightness corresponds to different brightness weight curves;
different white point numbers account for different proportions of the total white point number and correspond to different brightness weight curves.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010229027.3A CN111565306B (en) | 2020-03-27 | 2020-03-27 | Automatic white balance method and storage device for three-dimensional space white point detection and brightness weighting white point |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010229027.3A CN111565306B (en) | 2020-03-27 | 2020-03-27 | Automatic white balance method and storage device for three-dimensional space white point detection and brightness weighting white point |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111565306A true CN111565306A (en) | 2020-08-21 |
| CN111565306B CN111565306B (en) | 2022-05-24 |
Family
ID=72069912
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010229027.3A Active CN111565306B (en) | 2020-03-27 | 2020-03-27 | Automatic white balance method and storage device for three-dimensional space white point detection and brightness weighting white point |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111565306B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW548509B (en) * | 2002-11-12 | 2003-08-21 | Novatek Microelectronics Corp | Estimation method for white-point offset of the white balance point |
| US20080143844A1 (en) * | 2006-12-15 | 2008-06-19 | Cypress Semiconductor Corporation | White balance correction using illuminant estimation |
| CN105472365A (en) * | 2015-12-30 | 2016-04-06 | 浙江宇视科技有限公司 | Automatic white balance method and device |
| CN105959662A (en) * | 2016-05-24 | 2016-09-21 | 深圳英飞拓科技股份有限公司 | Self-adaptive white balance adjusting method and device |
-
2020
- 2020-03-27 CN CN202010229027.3A patent/CN111565306B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW548509B (en) * | 2002-11-12 | 2003-08-21 | Novatek Microelectronics Corp | Estimation method for white-point offset of the white balance point |
| US20080143844A1 (en) * | 2006-12-15 | 2008-06-19 | Cypress Semiconductor Corporation | White balance correction using illuminant estimation |
| CN105472365A (en) * | 2015-12-30 | 2016-04-06 | 浙江宇视科技有限公司 | Automatic white balance method and device |
| CN105959662A (en) * | 2016-05-24 | 2016-09-21 | 深圳英飞拓科技股份有限公司 | Self-adaptive white balance adjusting method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111565306B (en) | 2022-05-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106845455B (en) | Image processing method, system and server based on skin color detection | |
| Van De Weijer et al. | Edge-based color constancy | |
| CN107454345B (en) | White balancing treatment method, device and the terminal device of image | |
| Funt et al. | Is machine colour constancy good enough? | |
| CN111899182B (en) | Color enhancement method and device | |
| CN106878695A (en) | Method, device and computer equipment that white balance is processed | |
| CN112884682B (en) | Stereo image color correction method and system based on matching and fusion | |
| CN109688396B (en) | Image white balance processing method and device and terminal equipment | |
| CN113301318B (en) | Image white balance processing method and device, storage medium and terminal | |
| CN108616700B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN109147005A (en) | It is a kind of for the adaptive colouring method of infrared image, system, storage medium, terminal | |
| US10055824B2 (en) | Image correction device, image correction method and storage medium | |
| CN105681775A (en) | White balance method and device | |
| CN108234971A (en) | White balance parameter determines method, white balance adjustment method and device, storage medium, terminal | |
| JP2001148863A (en) | White balance adjustment method and adjustment device | |
| CN115100240A (en) | Method and device for tracking object in video, electronic equipment and storage medium | |
| Mayer et al. | Towards autonomous vision self-calibration for soccer robots | |
| US5995662A (en) | Edge detecting method and edge detecting device which detects edges for each individual primary color and employs individual color weighting coefficients | |
| CN111565306B (en) | Automatic white balance method and storage device for three-dimensional space white point detection and brightness weighting white point | |
| CN112036253B (en) | Face key point positioning method based on deep learning | |
| CN114511894A (en) | System and method for acquiring pupil center coordinates | |
| KR102618580B1 (en) | Nighttime low-light image enhancement method based on retinex and atmospheric light estimation, apparatus and computer program for performing the method | |
| TWI715075B (en) | Skin tone image gamut weight detecting method and device thereof | |
| US12067658B1 (en) | Method, apparatus and device for automatically making up portrait lips, storage medium and program product | |
| CN105761282B (en) | The detection method and device of image color cast |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 350003 building 18, No.89, software Avenue, Gulou District, Fuzhou City, Fujian Province Applicant after: Ruixin Microelectronics Co.,Ltd. Address before: 350003 building 18, No.89, software Avenue, Gulou District, Fuzhou City, Fujian Province Applicant before: FUZHOU ROCKCHIP ELECTRONICS Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |