US20120032970A1 - Setting the white balance of a video frame - Google Patents
Setting the white balance of a video frame Download PDFInfo
- Publication number
- US20120032970A1 US20120032970A1 US13/274,398 US201113274398A US2012032970A1 US 20120032970 A1 US20120032970 A1 US 20120032970A1 US 201113274398 A US201113274398 A US 201113274398A US 2012032970 A1 US2012032970 A1 US 2012032970A1
- Authority
- US
- United States
- Prior art keywords
- color
- sub
- colors
- ambiguous
- illuminant
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 239000003086 colorant Substances 0.000 claims abstract description 42
- 230000007935 neutral effect Effects 0.000 claims abstract description 42
- 238000000034 method Methods 0.000 claims abstract description 28
- WFKWXMTUELFFGS-UHFFFAOYSA-N tungsten Chemical compound [W] WFKWXMTUELFFGS-UHFFFAOYSA-N 0.000 claims description 3
- 229910052721 tungsten Inorganic materials 0.000 claims description 3
- 239000010937 tungsten Substances 0.000 claims description 3
- 238000004891 communication Methods 0.000 description 9
- 238000005286 illumination Methods 0.000 description 7
- 238000012360 testing method Methods 0.000 description 7
- 238000001914 filtration Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 4
- 241000023320 Luma <angiosperm> Species 0.000 description 2
- 210000004556 brain Anatomy 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
- H04N23/88—Camera processing pipelines; Components thereof for processing colour signals for colour balance, e.g. white-balance circuits or colour temperature control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/64—Circuits for processing colour signals
- H04N9/73—Colour balance circuits, e.g. white balance circuits or colour temperature control
Definitions
- the present invention is related generally to video processing and, more particularly, to setting the white balance of a video frame.
- the colors in an image vary depending upon the nature of the light illuminating the image.
- a white sheet of paper for example, should look yellow under a yellow light bulb. In general, this is because different sources of illumination (e.g., sunlight on a clear day, overcast sunlight, fluorescent light, incandescent light) produce light with different “power spectra,” that is, with light of different colors having different intensities.
- sources of illumination e.g., sunlight on a clear day, overcast sunlight, fluorescent light, incandescent light
- the colors in the image must be adjusted to match the human brain's white-balancing procedures. Colors in the image are adjusted by adjusting the gains of the three primary colors in the camera's light sensors. Adjusting white-balance semi-manually is relatively easy: A pure white card is placed before the camera, and the camera zooms in until the white card occupies the camera's full field of view. A button is pushed which tells the camera to adjust its color gain until the image is pure white.
- the white balance of a video frame is automatically set.
- the video frame is first divided into sub-regions. Those sub-regions determined to be possibly dominantly monochromatic are eliminated from consideration, and the white balance is set by adjusting the color gain of the video frame, the adjusting based on the other sub-regions in the video frame.
- a sub-region is determined to be possibly dominantly monochromatic if the difference between its averaged color and each of a set of neutral reference colors is greater than a set threshold. If the difference between the averaged color and at least one of the neutral reference colors is less than or equal to the threshold, then the sub-region is still determined to be possibly dominantly monochromatic if that neutral reference color, along with an illuminant type of a previous frame in the video, match closely enough to an element in an ambiguous reference color list.
- the color values in the neutral reference color set are defined under a plurality of possible lighting conditions.
- the ambiguous color reference list is created by comparing a neutral reference color with several reference colors under several different illumination conditions. An ambiguous color point is found wherever the difference between a reference color under a first illumination condition and the color value of the neutral reference color under a second illumination condition is less than a threshold.
- the illuminant type of a video frame is set to be the illuminant type of a neutral reference color that is closest to the greatest number of non-dominantly monochromatic sub-regions in the frame.
- FIG. 1 is an overview of a representational environment in which the present invention may be practiced
- FIGS. 2 a , 2 b , and 2 c are simplified schematics of a personal communication device that supports video white balancing
- FIGS. 3 a and 3 b together form a flowchart of an exemplary method for white balancing a video frame
- FIG. 4 is a table showing the values of an exemplary set of neutral reference colors
- FIG. 5 is a flowchart of an exemplary method for creating an ambiguous color reference list
- FIG. 6 is a table of the well known Macbeth colors.
- a communications environment 100 supports a user 102 working with his personal communication device 104 .
- the user 102 is shooting video with a camera on the device 104 .
- the video images captured by the device 104 are white-balanced in order to look natural.
- Specific aspects of the present invention are used to prevent the white-balancing from color-twisting those video images that have an intrinsic color cast or that contain a large area that is substantially monochromatic.
- the methods of the present invention can be performed locally on the personal communication device 104 itself, or on a remote computing server 106 , or in cooperation between the two.
- the server 106 can calculate data structures (such as the ambiguous color reference list, see FIG. 5 and accompanying discussion below) and make them available to the device 104 .
- the scope of the present invention is not limited and can be applied to any device that collects or processes digital video images.
- FIGS. 2 a , 2 b , and 2 c show a personal communication device 104 (e.g., a cellular telephone, personal digital assistant, personal computer, digital camera, or anything containing a digital camera) that incorporates an embodiment of the present invention.
- FIGS. 2 a and 2 b show the device 104 as a cellular telephone in an open configuration, presenting its main display screen 200 to the user 102 .
- the main display 200 is used for most high-fidelity interactions with the user 102 .
- the main display 200 is used to show video or still images, is part of a user interface for changing configuration settings, and is used for viewing call logs and contact lists.
- the main display 200 is of high resolution and is as large as can be comfortably accommodated in the device 104 .
- a device 104 may have a second and possibly a third display screen for presenting status messages. These screens are generally smaller than the main display screen 200 . They can be safely ignored for the remainder of the present discussion.
- the typical user interface of the personal communication device 104 includes, in addition to the main display 200 , a keypad 202 or other user-input devices.
- the user interface is oriented toward taking still pictures and videos.
- FIG. 2 b illustrates some of the more important internal components of the personal communication device 104 .
- the device 104 includes a communications transceiver 204 , a processor 206 , and a memory 208 .
- a microphone (or two) and a speaker are usually present.
- FIG. 2 c shows the other or “outside” face of the personal communication device 104 .
- the device 104 includes a camera 210 .
- regions of a video frame that are predominantly monochromatic are ignored during a process of white balancing that video frame. By ignoring these predominantly monochromatic areas, the white-balancing procedure is not skewed.
- FIGS. 3 a and 3 b together present an exemplary method for white-balancing a video frame.
- the video frame is captured prior to the beginning of this method. Also prior to the beginning of this method, some data structures are created. These data structures are described where appropriate in the following discussion.
- step 300 of FIG. 3 a the video frame is divided into a number of sub-regions.
- Various techniques, of varying complexity, are available for performing this step. At one extreme, for example, the color values of the pixels are examined and compared, and sub-regions are defined as contiguous areas that are mostly monochromatic. However, this method involves a great deal of processing. Experiments have shown that a much simpler method, e.g., dividing the video frame into a number of fixed-size rectangles, works very well. In a current preferred embodiment, the video frame is divided into twelve columns by sixteen rows, yielding 192 fixed-size rectangles. While other embodiments can use a greater number of fixed-size rectangles, the preferred embodiment has proven to be a good trade-off between acceptable white-balancing results and the increased processing required for an increased numbers of rectangles.
- steps 302 , 304 , and 306 of FIG. 3 a attempt to discover which of the sub-regions (if any) are predominantly monochromatic so that they can be ignored during the white-balancing process in step 308 of FIG. 3 b.
- Step 302 begins by calculating an averaged color for each sub-region of the video frame.
- Many ways are known in the art for calculating an averaged color. For example, the pixels in the sub-region are examined, and statistical information regarding the Red, Green, and Blue values of the pixels is gathered. By averaging the statistics, a point in YCbCr color space is found that represents the statistical averaged color of the pixels in the sub-region.
- YCbCr is a well known three-dimensional color space. It is an alternative to the R(ed)G(reen)B(lue) color space. Y is the brightness or luma value; Cb is Blue minus Y; Cr is Red minus Y.
- YCbCr over RGB is that it allows the luma component Y to be stored with high resolution, while the chroma components Cb and Cr are compressed to lower resolution. This closely matches the acuity of the human visual system which is much more sensitive to levels of black and white than to color distinctions.
- the calculated averaged color of the sub-region of the video frame is then compared with a set of neutral reference color values.
- the neutral reference color values are a small set of neutral colors, each standing for a gray color value taken under a different illuminant.
- a set of sixteen neutral gray reference colors under four illumination groups day lights, cloudy lights, fluorescent lightings, tungsten lightings
- These sixteen colors and their illuminant types are listed in the table of FIG. 4 .
- a first filtering test is performed on all of the sub-regions to see which of them may be dominantly monochromatic.
- the Euclidean distances in the color space from the sub-region's averaged color to each of the neutral reference colors are calculated and compared against a threshold. If at least one of the distances is less than the threshold (that is, if the averaged color of the sub-region is “close” to one of the neutral reference colors), then this sub-region is probably not dominantly monochromatic (but see step 306 below). If, however, all of the distances are greater than the threshold (that is, if the averaged color of the sub-region is not “close” to any of the neutral reference colors), then this sub-region is marked as possibly dominantly monochromatic.
- This reasoning is based on the observation that if the averaged color of the sub-region is close to one of the neutral reference colors, then the pixels in the sub-region probably contain a variety of colors that average out to a value near a neutral reference color, and thus, the sub-region is probably not dominantly monochromatic.
- step 306 a second filtering test is performed on those sub-regions of the video frame that were not marked as possibly dominantly monochromatic in step 304 . Because these sub-regions were not “caught” in the first filtering test of step 304 , it is known that the averaged color of each of these sub-regions is “close” to (that is, within the first threshold of) at least one of the neutral reference colors. Step 306 tests each neutral reference color found to be close to the averaged color of the sub-region, along with an illuminant assigned to the previous frame in the video, against the elements of an “ambiguous color reference list.” (Step 310 of FIG.
- step 308 of FIG. 3 b the video frame is white balanced.
- the sub-regions of the video frame that were marked as possibly dominantly monochromatic either under the first filtering test (step 304 of FIG. 3 a ) or under the second filtering test (step 306 ) are not considered in this white balancing.
- the color values of the remaining sub-regions are considered, and the color gains for the video frame are adjusted according to any well known technique for white balancing (e.g., the gray-world assumption or the white-world assumption).
- the video frame is not white balanced at all, and the color gains for the video frame are left unchanged.
- FIGS. 3 a and 3 b applies standard white balancing techniques to a video frame, but the white balancing is based only on those regions of the video frame that are not marked as possibly dominantly monochromatic. Experience has shown that this markedly reduces “color twist” caused by a naive application of white balancing to video frames that contain large monochromatic areas.
- An illuminant type is assigned to the video frame in step 310 . To do this, the averaged colors of all of the sub-regions that were not marked as possibly dominantly monochromatic are considered and compared against the neutral reference colors. The neutral reference color that is “closest” to the greatest number of averaged colors of these sub-regions is found. Then the illuminant type of that closest neutral reference color is assigned as the illuminant type of the video frame.
- FIG. 5 presents one way to create the ambiguous color reference list used in step 306 of FIG. 3 a .
- the ambiguous color reference list is expected to be created in an off-line process, possibly on the remote computing server 106 .
- a neutral reference color is compared against a number of reference colors under a number of illuminants. (In most embodiments, the set of neutral reference colors is the same set as used in step 302 of FIG. 3 a .)
- An ambiguous color is found in step 502 whenever the color value of a monochromatic color in one illuminant (call it Illuminant A) “closely” matches the color value of the neutral reference color in a different illuminant (call it Illuminant B).
- Illuminant A is stored along with the reference neutral color in Illuminant B.
- the well known Macbeth color set includes twenty-four colors that are very commonly seen.
- FIG. 6 is a table of the Macbeth colors, giving their names. Using the Macbeth colors and the neutral reference color set shown in the Table of FIG. 4 , the ambiguous color list includes:
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Processing Of Color Television Signals (AREA)
- Image Processing (AREA)
Abstract
Disclosed is a method for setting the white balance of a video frame. The frame is first divided into sub-regions. Those sub-regions determined to be possibly dominantly monochromatic are eliminated from consideration, and the white balance is set by adjusting the color gain of the frame, the adjusting based on the other sub-regions in the frame. A sub-region is determined to be possibly dominantly monochromatic if the difference between its averaged color and each of a set of neutral reference colors is greater than a threshold. If the difference between the averaged color and at least one of the neutral reference colors is less than or equal to the threshold, then the sub-region is still determined to be possibly dominantly monochromatic if that neutral reference color, along with an illuminant type of a previous frame in the video, match closely enough to an element in an ambiguous reference color list.
Description
- The present invention is related generally to video processing and, more particularly, to setting the white balance of a video frame.
- The colors in an image vary depending upon the nature of the light illuminating the image. A white sheet of paper, for example, should look yellow under a yellow light bulb. In general, this is because different sources of illumination (e.g., sunlight on a clear day, overcast sunlight, fluorescent light, incandescent light) produce light with different “power spectra,” that is, with light of different colors having different intensities.
- Human visual perception, however, does not slavishly follow variations in illumination. The human eyes and brain, knowing that a sheet of paper is white, automatically adjust the “white balance” of the perceived image of the paper. Due to this white balancing, the human consciously perceives the paper as white even under a wide variety of illumination types.
- To make a camera image look “natural,” the colors in the image must be adjusted to match the human brain's white-balancing procedures. Colors in the image are adjusted by adjusting the gains of the three primary colors in the camera's light sensors. Adjusting white-balance semi-manually is relatively easy: A pure white card is placed before the camera, and the camera zooms in until the white card occupies the camera's full field of view. A button is pushed which tells the camera to adjust its color gain until the image is pure white.
- User-friendly cameras, of course, perform white-balancing automatically. To summarize some very complicated procedures, the camera makes an assumption about the colors in an image. For example, the camera often assumes that the average intensities of the primary color channels (e.g., Red, Green, and Blue) are equal in the image. These automatic white-balancing procedures have difficulty, however, with images that violate the camera's assumptions, for example, images that have an intrinsic color cast or that have a large monochromatic area, such as a close-up of a human face or a landscape with a wide stretch of blue sky. When an image violates the camera's assumptions about the colors in the image, the camera's white-balancing operations are thrown off and give an undesirable “color twist” to the image.
- The above considerations, and others, are addressed by the present invention, which can be understood by referring to the specification, drawings, and claims. According to aspects of the present invention, the white balance of a video frame is automatically set. The video frame is first divided into sub-regions. Those sub-regions determined to be possibly dominantly monochromatic are eliminated from consideration, and the white balance is set by adjusting the color gain of the video frame, the adjusting based on the other sub-regions in the video frame.
- In some embodiments, a sub-region is determined to be possibly dominantly monochromatic if the difference between its averaged color and each of a set of neutral reference colors is greater than a set threshold. If the difference between the averaged color and at least one of the neutral reference colors is less than or equal to the threshold, then the sub-region is still determined to be possibly dominantly monochromatic if that neutral reference color, along with an illuminant type of a previous frame in the video, match closely enough to an element in an ambiguous reference color list.
- The color values in the neutral reference color set are defined under a plurality of possible lighting conditions.
- In some embodiments, the ambiguous color reference list is created by comparing a neutral reference color with several reference colors under several different illumination conditions. An ambiguous color point is found wherever the difference between a reference color under a first illumination condition and the color value of the neutral reference color under a second illumination condition is less than a threshold.
- In some embodiments, the illuminant type of a video frame is set to be the illuminant type of a neutral reference color that is closest to the greatest number of non-dominantly monochromatic sub-regions in the frame.
- While the appended claims set forth the features of the present invention with particularity, the invention, together with its objects and advantages, may be best understood from the following detailed description taken in conjunction with the accompanying drawings of which:
-
FIG. 1 is an overview of a representational environment in which the present invention may be practiced; -
FIGS. 2 a, 2 b, and 2 c are simplified schematics of a personal communication device that supports video white balancing; -
FIGS. 3 a and 3 b together form a flowchart of an exemplary method for white balancing a video frame; -
FIG. 4 is a table showing the values of an exemplary set of neutral reference colors; -
FIG. 5 is a flowchart of an exemplary method for creating an ambiguous color reference list; and -
FIG. 6 is a table of the well known Macbeth colors. - Turning to the drawings, wherein like reference numerals refer to like elements, the invention is illustrated as being implemented in a suitable environment. The following description is based on embodiments of the invention and should not be taken as limiting the invention with regard to alternative embodiments that are not explicitly described herein.
- In
FIG. 1 , acommunications environment 100 supports auser 102 working with hispersonal communication device 104. Theuser 102 is shooting video with a camera on thedevice 104. According to aspects of the present invention, the video images captured by thedevice 104 are white-balanced in order to look natural. Specific aspects of the present invention are used to prevent the white-balancing from color-twisting those video images that have an intrinsic color cast or that contain a large area that is substantially monochromatic. - The methods of the present invention can be performed locally on the
personal communication device 104 itself, or on aremote computing server 106, or in cooperation between the two. Specifically, theserver 106 can calculate data structures (such as the ambiguous color reference list, seeFIG. 5 and accompanying discussion below) and make them available to thedevice 104. The scope of the present invention is not limited and can be applied to any device that collects or processes digital video images. -
FIGS. 2 a, 2 b, and 2 c show a personal communication device 104 (e.g., a cellular telephone, personal digital assistant, personal computer, digital camera, or anything containing a digital camera) that incorporates an embodiment of the present invention.FIGS. 2 a and 2 b show thedevice 104 as a cellular telephone in an open configuration, presenting itsmain display screen 200 to theuser 102. Typically, themain display 200 is used for most high-fidelity interactions with theuser 102. For example, themain display 200 is used to show video or still images, is part of a user interface for changing configuration settings, and is used for viewing call logs and contact lists. To support these interactions, themain display 200 is of high resolution and is as large as can be comfortably accommodated in thedevice 104. Adevice 104 may have a second and possibly a third display screen for presenting status messages. These screens are generally smaller than themain display screen 200. They can be safely ignored for the remainder of the present discussion. - The typical user interface of the
personal communication device 104 includes, in addition to themain display 200, akeypad 202 or other user-input devices. When thedevice 104 is a digital camera, the user interface is oriented toward taking still pictures and videos. -
FIG. 2 b illustrates some of the more important internal components of thepersonal communication device 104. Thedevice 104 includes acommunications transceiver 204, aprocessor 206, and amemory 208. A microphone (or two) and a speaker are usually present. -
FIG. 2 c shows the other or “outside” face of thepersonal communication device 104. On this side, thedevice 104 includes acamera 210. - According to some aspects of the present invention, regions of a video frame that are predominantly monochromatic are ignored during a process of white balancing that video frame. By ignoring these predominantly monochromatic areas, the white-balancing procedure is not skewed.
- To apply this insight,
FIGS. 3 a and 3 b together present an exemplary method for white-balancing a video frame. The video frame is captured prior to the beginning of this method. Also prior to the beginning of this method, some data structures are created. These data structures are described where appropriate in the following discussion. - In
step 300 ofFIG. 3 a, the video frame is divided into a number of sub-regions. Various techniques, of varying complexity, are available for performing this step. At one extreme, for example, the color values of the pixels are examined and compared, and sub-regions are defined as contiguous areas that are mostly monochromatic. However, this method involves a great deal of processing. Experiments have shown that a much simpler method, e.g., dividing the video frame into a number of fixed-size rectangles, works very well. In a current preferred embodiment, the video frame is divided into twelve columns by sixteen rows, yielding 192 fixed-size rectangles. While other embodiments can use a greater number of fixed-size rectangles, the preferred embodiment has proven to be a good trade-off between acceptable white-balancing results and the increased processing required for an increased numbers of rectangles. - Having defined the sub-regions of the video frame in
step 300,steps FIG. 3 a attempt to discover which of the sub-regions (if any) are predominantly monochromatic so that they can be ignored during the white-balancing process instep 308 ofFIG. 3 b. - Step 302 begins by calculating an averaged color for each sub-region of the video frame. Many ways are known in the art for calculating an averaged color. For example, the pixels in the sub-region are examined, and statistical information regarding the Red, Green, and Blue values of the pixels is gathered. By averaging the statistics, a point in YCbCr color space is found that represents the statistical averaged color of the pixels in the sub-region. (YCbCr is a well known three-dimensional color space. It is an alternative to the R(ed)G(reen)B(lue) color space. Y is the brightness or luma value; Cb is Blue minus Y; Cr is Red minus Y. An advantage of YCbCr over RGB is that it allows the luma component Y to be stored with high resolution, while the chroma components Cb and Cr are compressed to lower resolution. This closely matches the acuity of the human visual system which is much more sensitive to levels of black and white than to color distinctions.)
- The calculated averaged color of the sub-region of the video frame is then compared with a set of neutral reference color values. In a preferred embodiment, the neutral reference color values are a small set of neutral colors, each standing for a gray color value taken under a different illuminant. In a preferred embodiment, a set of sixteen neutral gray reference colors under four illumination groups (day lights, cloudy lights, fluorescent lightings, tungsten lightings) has been defined. These sixteen colors and their illuminant types are listed in the table of
FIG. 4 . Experiments have shown that embodiments of the present invention work very well with this small set of neutral reference colors, making these embodiments easy to implement in a video processing system of limited capabilities. Other sets of reference colors can also be used. - In
step 304, a first filtering test is performed on all of the sub-regions to see which of them may be dominantly monochromatic. The Euclidean distances in the color space from the sub-region's averaged color to each of the neutral reference colors are calculated and compared against a threshold. If at least one of the distances is less than the threshold (that is, if the averaged color of the sub-region is “close” to one of the neutral reference colors), then this sub-region is probably not dominantly monochromatic (but seestep 306 below). If, however, all of the distances are greater than the threshold (that is, if the averaged color of the sub-region is not “close” to any of the neutral reference colors), then this sub-region is marked as possibly dominantly monochromatic. This reasoning is based on the observation that if the averaged color of the sub-region is close to one of the neutral reference colors, then the pixels in the sub-region probably contain a variety of colors that average out to a value near a neutral reference color, and thus, the sub-region is probably not dominantly monochromatic. - The method proceeds to step 306 where a second filtering test is performed on those sub-regions of the video frame that were not marked as possibly dominantly monochromatic in
step 304. Because these sub-regions were not “caught” in the first filtering test ofstep 304, it is known that the averaged color of each of these sub-regions is “close” to (that is, within the first threshold of) at least one of the neutral reference colors. Step 306 tests each neutral reference color found to be close to the averaged color of the sub-region, along with an illuminant assigned to the previous frame in the video, against the elements of an “ambiguous color reference list.” (Step 310 ofFIG. 3 b, discussed below, assigns the illuminant value to this previous frame of the video.) (Embodiments of a method for creating the ambiguous color reference list are discussed below in reference toFIG. 5 .) When the combination of that “close” neutral reference color and the illuminant of the previous video frame is “close” enough to an element of the ambiguous color reference list (that is, within a second threshold), then this sub-region of the video frame is marked as possibly dominantly monochromatic. The reasoning here is that the averaged color of a dominantly monochromatic sub-region can “accidentally” look like a neutral reference color under specific lighting situations. Those regions are “caught” in the filtering test ofstep 306 and are marked as possibly dominantly monochromatic. - After all of the sub-regions of the video frame have been examined for possibly dominant monochromaticity, then in
step 308 ofFIG. 3 b the video frame is white balanced. The sub-regions of the video frame that were marked as possibly dominantly monochromatic either under the first filtering test (step 304 ofFIG. 3 a) or under the second filtering test (step 306) are not considered in this white balancing. The color values of the remaining sub-regions are considered, and the color gains for the video frame are adjusted according to any well known technique for white balancing (e.g., the gray-world assumption or the white-world assumption). In some embodiments, if the number of sub-regions (or the total area of the sub-regions if they are not all of the same size) that were not marked as possibly dominantly monochromatic is too small, then the video frame is not white balanced at all, and the color gains for the video frame are left unchanged. - Thus, the method of
FIGS. 3 a and 3 b applies standard white balancing techniques to a video frame, but the white balancing is based only on those regions of the video frame that are not marked as possibly dominantly monochromatic. Experience has shown that this markedly reduces “color twist” caused by a naive application of white balancing to video frames that contain large monochromatic areas. - An illuminant type is assigned to the video frame in
step 310. To do this, the averaged colors of all of the sub-regions that were not marked as possibly dominantly monochromatic are considered and compared against the neutral reference colors. The neutral reference color that is “closest” to the greatest number of averaged colors of these sub-regions is found. Then the illuminant type of that closest neutral reference color is assigned as the illuminant type of the video frame. -
FIG. 5 presents one way to create the ambiguous color reference list used instep 306 ofFIG. 3 a. The ambiguous color reference list is expected to be created in an off-line process, possibly on theremote computing server 106. Instep 500, a neutral reference color is compared against a number of reference colors under a number of illuminants. (In most embodiments, the set of neutral reference colors is the same set as used instep 302 ofFIG. 3 a.) An ambiguous color is found instep 502 whenever the color value of a monochromatic color in one illuminant (call it Illuminant A) “closely” matches the color value of the neutral reference color in a different illuminant (call it Illuminant B). A variety of techniques are available for storing the ambiguous color point. In one embodiment, Illuminant A is stored along with the reference neutral color in Illuminant B. - The well known Macbeth color set includes twenty-four colors that are very commonly seen.
FIG. 6 is a table of the Macbeth colors, giving their names. Using the Macbeth colors and the neutral reference color set shown in the Table ofFIG. 4 , the ambiguous color list includes: -
- “Orange” (
color 7 on the standard Macbeth color chart) under the 6500 k illuminant group is “close to” (that is, is ambiguous with respect to) the (Cb=−43; Cr=67) neutral reference color inFIG. 4 under the below 3000 k illuminant group; - “Light Skin” (Macbeth color 2) and “Dark Skin” (Macbeth color 1) under the 6500 k illuminant group are close to the (Cb=−43; Cr=67) neutral reference color under the below 3000 k illuminant group; and
- “Blue Sky” (Macbeth color 3) and “Blue Flower” (Macbeth color 5) under the 4000 k illuminant group are close to the (Cb=54; Cr=−35) and (Cb=45; Cr=−30) neutral reference colors under the 6500 k illuminant group.
Different embodiments of the neutral reference color set, the standard reference colors, and the thresholds lead to a different ambiguous reference color set.
- “Orange” (
- In view of the many possible embodiments to which the principles of the present invention may be applied, it should be recognized that the embodiments described herein with respect to the drawing figures are meant to be illustrative only and should not be taken as limiting the scope of the invention. For example, different neutral reference colors may be called for in various environments. Therefore, the invention as described herein contemplates all such embodiments as may come within the scope of the following claims and equivalents thereof.
Claims (8)
1-14. (canceled)
15. A method for creating an ambiguous color reference list, the method comprising:
comparing a neutral reference color with a plurality of reference colors under a plurality of illuminants; and
based, at least in part, on the comparing, creating a list of ambiguous color points, wherein each element of the list indicates a reference color under a first illuminant that has a color value whose difference from a color value of the neutral reference color under a second illuminant is less than a threshold.
16. The method of claim 15 wherein the reference colors comprise Macbeth colors.
17. The method of claim 15 wherein illuminants comprise lighting conditions selected from the group consisting of: day lighting, cloudy lighting, fluorescent lighting, and tungsten lighting.
18. A computing server comprising:
a memory configured for storing an ambiguous color reference list; and
a processor operatively coupled to the memory and configured for creating the ambiguous color reference list, the creating comprising:
comparing a neutral reference color with a plurality of reference colors under a plurality of illuminants; and
based, at least in part, on the comparing, creating a list of ambiguous color points, wherein each element of the list indicates a reference color under a first illuminant that has a color value whose difference from a color value of the neutral reference color under a second illuminant is less than a threshold.
19. The computing server of claim 18 wherein the reference colors comprise Macbeth colors.
20. The computing server of claim 18 wherein illuminants comprise lighting conditions selected from the group consisting of: day lighting, cloudy lighting, fluorescent lighting, and tungsten lighting.
21. The computing server of claim 18 further comprising:
a transceiver operatively coupled to the memory and to the processor and configured for transmitting the ambiguous color reference list.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/274,398 US20120032970A1 (en) | 2008-12-18 | 2011-10-17 | Setting the white balance of a video frame |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/337,839 US8068147B2 (en) | 2008-12-18 | 2008-12-18 | Setting the white balance of a video frame |
US13/274,398 US20120032970A1 (en) | 2008-12-18 | 2011-10-17 | Setting the white balance of a video frame |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/337,839 Division US8068147B2 (en) | 2008-12-18 | 2008-12-18 | Setting the white balance of a video frame |
Publications (1)
Publication Number | Publication Date |
---|---|
US20120032970A1 true US20120032970A1 (en) | 2012-02-09 |
Family
ID=42265507
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/337,839 Active 2030-02-02 US8068147B2 (en) | 2008-12-18 | 2008-12-18 | Setting the white balance of a video frame |
US13/274,398 Abandoned US20120032970A1 (en) | 2008-12-18 | 2011-10-17 | Setting the white balance of a video frame |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/337,839 Active 2030-02-02 US8068147B2 (en) | 2008-12-18 | 2008-12-18 | Setting the white balance of a video frame |
Country Status (1)
Country | Link |
---|---|
US (2) | US8068147B2 (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9860507B2 (en) | 2015-06-08 | 2018-01-02 | Qualcomm Incorporated | Dynamic frame skip for auto white balance |
US11095865B2 (en) | 2017-07-24 | 2021-08-17 | Hewlett-Packard Development Company, L.P. | Color balance in electronic devices |
CN111953955B (en) * | 2020-08-26 | 2022-01-04 | 维沃移动通信有限公司 | White balance compensation method and device and electronic equipment |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5798943A (en) * | 1995-03-21 | 1998-08-25 | Light Source Computer Images | Apparatus and process for a digital swatchbook |
US7184080B2 (en) * | 2001-06-25 | 2007-02-27 | Texas Instruments Incorporated | Automatic white balancing via illuminant scoring |
US7683966B2 (en) * | 2003-01-21 | 2010-03-23 | Fujifilm Corporation | Digital camera having an electronic flash device using LED as a flash light source |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4399129B2 (en) * | 2001-06-14 | 2010-01-13 | シャープ株式会社 | Image forming apparatus |
US7190394B2 (en) * | 2002-06-04 | 2007-03-13 | Micron Technology, Inc. | Method for statistical analysis of images for automatic white balance of color channel gains for image sensors |
US7688468B2 (en) * | 2004-07-12 | 2010-03-30 | Canon Kabushiki Kaisha | Method of illuminant adaptation |
US7953274B2 (en) * | 2005-03-18 | 2011-05-31 | Valspar Sourcing, Inc. | Digital method for matching stains |
US20070041064A1 (en) * | 2005-08-16 | 2007-02-22 | Micron Technology, Inc. | Image sampling method for automatic white balance |
JP4529888B2 (en) * | 2005-12-07 | 2010-08-25 | ブラザー工業株式会社 | Image processing apparatus, image processing method, and image processing program |
-
2008
- 2008-12-18 US US12/337,839 patent/US8068147B2/en active Active
-
2011
- 2011-10-17 US US13/274,398 patent/US20120032970A1/en not_active Abandoned
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5798943A (en) * | 1995-03-21 | 1998-08-25 | Light Source Computer Images | Apparatus and process for a digital swatchbook |
US7184080B2 (en) * | 2001-06-25 | 2007-02-27 | Texas Instruments Incorporated | Automatic white balancing via illuminant scoring |
US7683966B2 (en) * | 2003-01-21 | 2010-03-23 | Fujifilm Corporation | Digital camera having an electronic flash device using LED as a flash light source |
Also Published As
Publication number | Publication date |
---|---|
US20100157160A1 (en) | 2010-06-24 |
US8068147B2 (en) | 2011-11-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3648459B1 (en) | White balance adjustment method and apparatus, camera and medium | |
US6995791B2 (en) | Automatic white balance for digital imaging | |
US11277595B2 (en) | White balance method for image and terminal device | |
US6788813B2 (en) | System and method for effectively performing a white balance operation | |
JP4266461B2 (en) | Color illumination color temperature detection method and apparatus | |
US20090009525A1 (en) | Color Adjustment Device and Method | |
US20070085911A1 (en) | Apparatus for color correction of subject-image data, and method of controlling same | |
US8243164B2 (en) | Method, apparatus, and system for selecting pixels for automatic white balance processing | |
TW201106685A (en) | Foreground image separation method | |
CN111064942B (en) | Image processing method and apparatus | |
US20140292616A1 (en) | Computer monitor equalization using handheld device | |
EP3849179B1 (en) | Method and system for adjusting white balance, and display | |
US8068147B2 (en) | Setting the white balance of a video frame | |
KR101143834B1 (en) | Aparatus and metod for controlling auto white balance | |
WO2017074076A1 (en) | Apparatus and method for controlling contrast ratio of content in electronic device | |
EP2209298A1 (en) | Method for adjusting white balance | |
KR20210093658A (en) | A method for auto white balance of image and an electronic device to process auto white balance method | |
US20230388465A1 (en) | Chroma adjustment method, electronic device and readable storage medium | |
JP2001057680A (en) | Method and device for adjusting white balance and recording medium | |
WO2023134235A1 (en) | Image processing method and electronic device | |
Gofaizen et al. | Problem questions of TV colorimetry | |
JP2009278227A (en) | Image correction apparatus and method | |
Hui | Real-Time Color Enhancement Method Used for Intelligent Mobile Terminals | |
Suzuki et al. | Adaptive Image Quality Improvement Method for Moving Pictures Using Smoothing Parameter Transition and Scene Cut Detection | |
Kwak et al. | Skin color reproduction algorithm for portrait images shown on the mobile display |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |