US20150363912A1

US20150363912A1 - Rgbw demosaic method by combining rgb chrominance with w luminance

Info

Publication number: US20150363912A1
Application number: US14/304,227
Authority: US
Inventors: Candice Hellen Brown Elliott; Anthony Botzas; Michel Francis HIGGINS
Original assignee: Samsung Electronics Co Ltd
Current assignee: Samsung Electronics Co Ltd
Priority date: 2014-06-12
Filing date: 2014-06-13
Publication date: 2015-12-17
Also published as: KR20150142600A; US20150363913A1; KR20150142599A; US9418398B2; US20150363944A1; US20150363916A1; US20150363922A1; KR20150142601A

Abstract

An apparatus and method for demosaicing sampled color values are provided. The method includes capturing color information using an RGBW image detector, demosaicing red color values, green color values, and blue color values at subpixels respectively corresponding to red photosites of the image detector, green photosites of the image detector, blue photosites of the image detector, and white photosites of the image detector, and changing one or more of the demosaiced red color value, the demosaiced blue color value, and the demosaiced green color value at a particular subpixel based at least in part on color information sampled at one or more white subpixels corresponding to the white photosites.

Description

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims the benefit under 35 U.S.C. §119(e) of a U.S. Provisional application filed on Jun. 12, 2014 in the U.S. Patent and Trademark Office and assigned Ser. No. 62/011,311, the entire disclosure of which is hereby incorporated by reference.

TECHNICAL FIELD

The present disclosure relates to an apparatus and method for demosaicing color information. More particularly, the present disclosure relates to an apparatus and method for demosaicing color information by combining chrominance with luminance.

BACKGROUND

Mobile terminals are developed to provide wireless communication between users. As technology has advanced, mobile terminals now provide many additional features beyond simple telephone conversation. For example, mobile terminals are now able to provide image and video capture. As a result of the ubiquity of mobile terminals, image capture and/or video capture have become increasingly popular. Consequently, various image processing techniques are used to provide a user with an accurate representation of the image intended to be captured.
In order to provide a more accurate representation of the color of the image intended to be captured, a technique commonly referred to as demosaicing may be performed. Demosaicing refers to the digital imaging process used to reconstruct a full color image from color samples output from an image detector.
Accordingly, there is a need for an apparatus and method for providing an improved representation of the image intended to be captured. Further, there is a need for an apparatus and method for demosaicing color values sampled during image capture.
The above information is presented as background information only to assist with an understanding of the present disclosure. No determination has been made, and no assertion is made, as to whether any of the above might be applicable as prior art with regard to the present disclosure.

SUMMARY

Aspects of the present disclosure are to address at least the above-mentioned problems and/or disadvantages and to provide at least the advantages described below. Accordingly, an aspect of the present disclosure is to provide an apparatus and method for demosaicing color information.
In accordance with an aspect of the present disclosure, a method for demosaicing color information is provided. The method includes capturing color information using an RGBW image detector, demosaicing red color values, green color values, and blue color values at subpixels respectively corresponding to red photosites of the image detector, green photosites of the image detector, blue photosites of the image detector, and white photosites of the image detector, and changing one or more of the demosaiced red color value, the demosaiced blue color value, and the demosaiced green color value at a particular subpixel based at least in part on color information sampled at one or more white subpixels corresponding to the white photosites.
In accordance with another aspect of the present disclosure, an apparatus for demosaicing color information is provided. The apparatus includes an RGBW image detector configured to capture color information, a storage unit configured to store color information, and at least one processor configured to capture color information using the RGBW image detector; to demosaic red color values, green color values, and blue color values at subpixels respectively corresponding to red photosites of the image detector, green photosites of the image detector, blue photosites of the image detector, and white photosites of the image detector, and to change one or more of the demosaiced red color value, the demosaiced blue color value, and the demosaiced green color value at a particular subpixel based at least in part on color information sampled at one or more white subpixels corresponding to the white photosites.
In accordance with another aspect of the present disclosure, a method for demosaicing color information is provided. The method includes capturing color information using an RGBW image detector, demosaicing the captured color information using chrominance values included in the captured color information; and
modifying the demosaiced color information using luminance values included in the captured color information.
Other aspects, advantages, and salient features of the disclosure will become apparent to those skilled in the art from the following detailed description, which, taken in conjunction with the annexed drawings, discloses various embodiments of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of various embodiments of the present disclosure will be more apparent from the following description taken in conjunction with the accompanying drawings, in which:

FIGS. 1A and 1B illustrate an array of photosites of a photo detector according to the related art;

FIG. 2 illustrates a flowchart of a method of reconstructing a color representation of an image according to an embodiment of the present disclosure;

FIG. 3 illustrates a flowchart of a method of reconstructing a color representation of an image according to an embodiment of the present disclosure;

FIG. 4 illustrates pseudo code for a method of reconstructing a color representation of an image according to an embodiment of the present disclosure;

FIG. 5 illustrates a flowchart of a method of reconstructing a color representation of an image according to an embodiment of the present disclosure;

FIG. 6 illustrates pseudo code for a method of reconstructing a color representation of an image according to an embodiment of the present disclosure;

FIG. 7 illustrates a flowchart of a method of reconstructing a color representation of an image according to an embodiment of the present disclosure;

FIG. 8 illustrates pseudo code for a method of reconstructing a color representation of an image according to an embodiment of the present disclosure;

FIG. 9 illustrates pseudo code for a method of reconstructing a color representation of an image according to an embodiment of the present disclosure; and

FIG. 10 illustrates a block diagram schematically illustrating a configuration of an electronic device according to an embodiment of the present disclosure.

Throughout the drawings, it should be noted that like reference numbers are used to depict the same or similar elements, features, and structures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of various embodiments of the disclosure as defined by the claims and their equivalents. It includes various specific details to assist in that understanding but these are to be regarded as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. In addition, descriptions of well-known functions and constructions are omitted for clarity and conciseness.
The terms and words used in the following description and claims are not limited to the bibliographical meanings, but, are merely used by the inventor to enable a clear and consistent understanding of the disclosure. Accordingly, it should be apparent to those skilled in the art that the following description of various embodiments of the present disclosure are provided for illustration purpose only and not for the purpose of limiting the disclosure as defined by the appended claims and their equivalents.
It is to be understood that the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to “a component surface” includes reference to one or more of such surfaces.
By the term “substantially” it is meant that the recited characteristic, parameter, or value need not be achieved exactly, but that deviations or variations, including for example, tolerances, measurement error, measurement accuracy limitations and other factors known to those of skill in the art, may occur in amounts that do not preclude the effect the characteristic was intended to provide.
According to various embodiments of the present disclosure, an electronic device may include communication functionality. For example, an electronic device may be a smart phone, a tablet Personal Computer (PC), a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop PC, a netbook PC, a Personal Digital Assistant (PDA), a Portable Multimedia Player (PMP), an mp3 player, a mobile medical device, a camera, a wearable device (e.g., a Head-Mounted Device (HMD), electronic clothes, electronic braces, an electronic necklace, an electronic appcessory, an electronic tattoo, or a smart watch), and/or the like.
According to various embodiments of the present disclosure, an electronic device may be a smart home appliance with communication functionality. A smart home appliance may be, for example, a television, a Digital Video Disk (DVD) player, an audio, a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave oven, a washer, a dryer, an air purifier, a set-top box, a TV box (e.g., Samsung HomeSync™, Apple TV™, or Google TV™), a gaming console, an electronic dictionary, an electronic key, a camcorder, an electronic picture frame, and/or the like.
According to various embodiments of the present disclosure, an electronic device may be a medical device (e.g., Magnetic Resonance Angiography (MRA) device, a Magnetic Resonance Imaging (MRI) device, Computed Tomography (CT) device, an imaging device, or an ultrasonic device), a navigation device, a Global Positioning System (GPS) receiver, an Event Data Recorder (EDR), a Flight Data Recorder (FDR), an automotive infotainment device, a naval electronic device (e.g., naval navigation device, gyroscope, or compass), an avionic electronic device, a security device, an industrial or consumer robot, and/or the like.
According to various embodiments of the present disclosure, an electronic device may be furniture, part of a building/structure, an electronic board, electronic signature receiving device, a projector, various measuring devices (e.g., water, electricity, gas or electro-magnetic wave measuring devices), and/or the like that include communication functionality.
According to various embodiments of the present disclosure, an electronic device may be any combination of the foregoing devices. In addition, it will be apparent to one having ordinary skill in the art that an electronic device according to various embodiments of the present disclosure is not limited to the foregoing devices.
Various embodiments of the present disclosure include an apparatus and method for demosaicing color information. Various embodiments of the present disclosure include an apparatus and method for demosaicing color information using chrominance values and luminance values.
An image is represented by a number of areas called pixels. Each pixel is associated with a color that should be substantially reproduced by a set of subpixels in a display. According to the related art, each subpixel displays a primary color. For example, each subpixel according to the related art is associated with some hue and saturation. Other colors may be obtained by mixing primary colors. Each pixel is mapped into a set of one or more subpixels which are to display the color of the pixel.
In some displays, each repeating set of subpixels includes a subpixel for each primary color. The subpixels are small, and are spaced closely together, to provide a desired resolution. This structure is not cost-effective however because the structure does not match the resolution of human vision. Humans are more perceptive to luminance differences than to chromatic differences. Therefore, some displays map an input pixel into a subpixel repeating set that does not include only the subpixels of each primary color. The chromatic resolution is reduced, but the luminance resolution remains high. One such display may be an RGBW display.
An image detector has a plurality of photo detectors used to sample an image. Each of the plurality of photo detectors may sample (e.g., capture) a value for a single color (e.g., a subpixel corresponding to a white photosite is assumed to sample a white color which may be a combination of all red, green, and blue). For example, each of the plurality of photo detectors may be configured with a color filter. According to the related art, a Color Filter Array (CFA) or a Color Filter Mosaic (CFM) is an array or mosaic of color filters disposed above the plurality of photo detectors.
Each of the plurality of photo detectors may be located at a photosite of the image detector. The photosite refers to the spatial location at which a color may be sampled by a photo detector. The array or mosaic of color filters may be disposed above the plurality of photo detectors such that each photo detector has a single corresponding color filter. Accordingly, each photosite may have a corresponding sampled value for a single color (e.g., a subpixel corresponding to a white photosite is assumed to sample a white color which may be a combination of all red, green, and blue).
Each of the photosites may be mapped or otherwise correspond to a subpixel of the image (e.g., when displayed on a display). Accordingly, each subpixel of the image may have a corresponding sampled value for a single color (e.g., a subpixel corresponding to a white photosite is assumed to sample a white color which may be a combination of all red, green, and blue). Because each subpixel of the image does not have sampled values for all colors, an image represented by the sampled color values at each subpixel may appear pixelated with a disjointed color representation of the intended image. In other words, the image represented by the sampled color values at each subpixel may be an inaccurate representation of the image intended to be captured.
In order to provide a more accurate representation of the color of the image intended to be captured, a technique commonly referred to as demosaicing may be performed. Demosaicing refers to the digital imaging process used to reconstruct a full color image from sampled color values output from an image detector. Because each photo detector has a spatial footprint in the image detector, the image detector is unable to capture a color value for every color at each respective photosite thereof. The reconstruction of the full color image using color samples output from the respective photo detectors constituting the image detector may use interpolation or other numerical methods to determine values for each color (e.g., red, green, and blue) at each subpixel.
FIGS. 1A and 1B illustrate an array of photosites of a photo detector according to the related art.
Referring to FIG. 1A, an example of an array of photosites comprising red, green, blue, and white photosites is illustrated. As an example, the red, green, blue, and white photosites may be created by placing an RGBW filter over the photo detectors respectively corresponding to the photosites. The filter disposed over the white photosites is a clear filter that allows all light. According to the related art, mosaic created by placing the RGBW filter over the red, green, blue, and white photosites may be referred to as RGBW mosaics. However, according to various embodiments of the present disclosure, the photosites corresponding to the photo detectors (e.g., sensors) corresponding to the white photosites may be thought to be better represented by Q rather than W because the response of such photosites represents the total light flux. The total light flux may be measured in units of photon quanta.
Referring to FIG. 1B, another example of an array of photosites comprising red, green, blue, and white photosites is illustrated. As an example, the red, green, blue, and white photosites may be created by placing an RGBW filter over the photo detectors respectively corresponding to the photosites.
According to the related art, color values of a sampled image may be reconstructed using chrominance values measured at the photosites. Specifically, according to the related art, the color values sampled at a plurality of photosites of an image detector are demosaiced. The related art demosaics the chrominance values of the sampled color values so as to reconstruct the color representation of the sampled image in relation to chrominance values. In other words, the RGB demosaicing algorithms according to the related art (e.g., a Bayer RGB demosaicing algorithm) have been developed to reconstruct the red (R), green (G), and blue (B) planes. However, the demosaicing algorithms according to the related art do not address how to deal with a fourth Q-color plane and how to optimally leverage the high frequency luminance information available in the Q-plane. The related art focuses on demosaicing the red, green, and blue color values because most image detectors typically only have red, green, and blue photosites. The related art thus reconstructs the color representation of sampled color values by interpolating the red color values, the blue color values, and the green color values across the plurality of subpixels.
According to various embodiments of the present disclosure, color information is demosaiced using chrominance information and luminance information. According to various embodiments of the present disclosure, color values sampled across a plurality of photosites is demosaiced using chrominance values and luminance values sampled at the plurality of photosites.
According to various embodiments of the present disclosure, the chrominance information derived from red, green, and blue color planes is combined with luminance information derived from the Q color plane. The Q values are typically sampled at a higher spatial frequency and thus have more detail in relation to the luminance information than the color information sampled at the red, green, and blue photosites.
According to various embodiments of the present disclosure, the color representation of the image is reconstructed using the chrominance values sampled at low spatial frequencies (e.g., the chrominance values respectively sampled at the red photosites, the blue photosites, and the green photosites), and using the luminance values sampled at high spatial frequencies (e.g., the luminance values sampled a the Q photosites). According to various embodiments of the present disclosure, the luminance value of the Q photosite is assumed to be a more accurate representation or measure of the true luminance than the luminance values sampled at the red photosites, the blue photosites, and the green photosites.
According to various embodiments of the present disclosure, the chrominance color values (e.g., a red color value, a green color value, and a blue color value) are demosaiced at a subpixel. For example, the chrominance color values are demosaiced at each subpixel. The chrominance color values may be demosaiced using an interpolation, another numerical method, and/or the like. For example, the chrominance color values may be demosaiced using, pixel replication, bilinear interpolation and median interpolation.
In pixel replication, each missing value is taken from the neighbor to the left, above, or diagonally above and left, whichever is nearest. Bilinear interpolation offers some improvement over pixel replication with a moderate increase in complexity. In the bilinear interpolation method, each missing value is calculated based on an average of the neighboring pixel values, horizontally, vertically and/or diagonally. Median interpolation, which is a nonlinear interpolation method, offers the best results among these three algorithms (pixel replication, bilinear and median), especially when there are defective pixels, but has the maximum complexity. Median interpolation has two steps. First, missing values having four diagonal neighbors are interpolated using the median of those four values. Second, the remaining missing pixels are interpolated by the median of north, south, east, and west neighbors.
According to various embodiments of the present disclosure, after the chrominance color values are demosaiced at each subpixel, the luminance value of the reconstructed color values (e.g., based on the demosaicing of the chrominance color values at each subpixel) is replaced with the sampled luminance value that was sampled at the corresponding Q photosite. According to various embodiments of the present disclosure, after the chrominance color values are demosaiced at each subpixel, the luminance value of the reconstructed color values (e.g., based on the demosaicing of the chrominance color values at each subpixel) is replaced with the a reconstructed luminance value that was reconstructed (e.g., demosaiced) using information sampled at corresponding Q photosites.
According to various embodiments of the present disclosure, after the chrominance color values are demosaiced at each subpixel, the luminance value of the reconstructed color values (e.g., based on the demosaicing of the chrominance color values at each subpixel) at the subpixels corresponding to Q (e.g., white) photosites is replaced with the sampled luminance value that was sampled at the corresponding Q photosite.
According to various embodiments of the present disclosure, after the chrominance color values are demosaiced at each subpixel, the RGB tri-stimulus values (e.g., the chrominance color values) are converted to a luminance-chrominance format. After the RGB tri-stimulus values are converted to a luminance-chrominance format, the luminance values at the respective subpixels are replaced with the sampled luminance values at the corresponding subpixels. For example, the luminance values at the subpixels corresponding to the Q (e.g., white) photosites is replaced with the sampled luminance value that was sampled at the corresponding Q photosite.
According to various embodiments of the present disclosure, after the reconstructed luminance values are replaced with the sampled luminance values in a luminance-chrominance format (e.g., luminance-chrominance color space), the reconstructed color information is converted to an RGB color space.
According to various embodiments of the present disclosure, after the reconstructed luminance values are replaced with the sampled luminance values in a luminance-chrominance format (e.g., luminance-chrominance color space), the chrominance color values may be normalized according to the sampled luminance values. For example, after the reconstructed luminance values are replaced with the sampled luminance values at the subpixels corresponding to the Q (e.g., white) photosites in a luminance-chrominance format (e.g., luminance-chrominance color space), the chrominance color values may be normalized (e.g., at such subpixels) according to the sampled luminance values.
According to various embodiments of the present disclosure, at the subpixels corresponding to the Q (e.g., white) photosites, the chrominance color values are reconstructed (e.g., demosaiced) and the sampled luminance values are used.
As illustrated in FIGS. 1A and 1B, density of the Q photosites is relatively greater than each of the density of the red color photosites, the blue photosites, and the blue photosites. As a result, the use of sampled luminance values (e.g., sampled at the Q photosites) enhances the demosaicing of the color information across the subpixels of an image. According to the demosaicing methods of the related art, the color information sampled at the Q photosites is effectively disregarded and the chrominance color values (e.g., the red color values, the green color values, and the blue color values) are reconstructed (e.g. interpolated) using sampled chrominance color values sampled at neighboring (or surrounding) subpixels.
FIG. 2 illustrates a flowchart of a method of reconstructing a color representation of an image according to an embodiment of the present disclosure.
Referring to FIG. 2, at operation 205, an electronic device captures color information (e.g., for an image). The electronic device may include an image detector that has a plurality of photo detectors may be used to capture the image. Each of the photo detectors may sample a single color (e.g., a subpixel corresponding to a white photosite is assumed to sample a white color which may be a combination of all red, green, and blue). According to various embodiments of the present disclosure, the color information from the plurality of photo detectors may be stored. According to various embodiments of the present disclosure, the color information constituting an intended image that may require reconstruction of the color representation thereof may be received/stored from a counterpart electronic device.
At operation 210, the electronic device may reconstruct the color values at the subpixels. The electronic device may calculate the red color values, the green color values, the blue color values at each of the subpixels. The electronic device may calculate the red color values, the green color values, the blue color values, and the Q color values at each of the subpixels. The electronic device may calculate the corresponding color values at each subpixel using interpolation or another numerical method (e.g., a local average of a corresponding color value).
At operation 215, the electronic device may convert the red color values, the green color values, and the blue color values to a luminance-chrominance format. According to various embodiments of the present disclosure, the electronic device may convert the RGB color space to a luminance-chrominance space.
At operation 220, the electronic device replaces the reconstructed luminance values with sampled luminance values at corresponding subpixels. According to various embodiments of the present disclosure, the luminance value of the reconstructed color values (e.g., based on the demosaicing of the chrominance color values at each subpixel) at the subpixels corresponding to Q (e.g., white) photosites is replaced with the sampled luminance value that was sampled at the corresponding Q photosite.
At operation 225, the electronic device may convert the color values from the luminance-chrominance format to an RGB format. The electronic device may convert the demosaiced color information (e.g., including chrominance color values, and luminance values) from a luminance-chrominance space to an RGB color space.
At operation 230, the electronic device may store the color information (e.g., the color values) for the respective subpixels. The electronic device may store the color information including the chrominance color values and the luminance values for the respective subpixels. The electronic device may store the image.
At operation 235, the electronic device may render the image. The electronic device may render the image in a preview frame. The electronic device may perform further image processing before rendering the image.
FIG. 3 illustrates a flowchart of a method of reconstructing a color representation of an image according to an embodiment of the present disclosure.
Referring to FIG. 3, an example of reconstructing the color representation of the image by using an XYZ color space as an intermediary within which the reconstructed luminance values at subpixels corresponding to Q photosites may be replaced with the sampled luminance values at such subpixels is provided. The normal formula for luminance uses weighted values of red, green, and blue to convert the color information from an RGB space to a luminance space. According to various embodiments of the present disclosure, the weights used to convert the color information from an RGB space to a luminance space are based on human perception (e.g., using a “photopic sensitivity function”). However, the Q values from an image detector (e.g., a camera sensor) are not weighted. The sampled Q values are proportional to photo counts and will count (weigh) red, green, or blue equally. According to various embodiments of the present disclosure, a method for taking such weighting into account, a new conversion matrix may be generated that converts RGB color values into CIE xyY chromaticity values (e.g., to convert the color information from an RGB color space to a CIE xyY color space). According to various embodiments of the present disclosure, rather than using the normal luminance weights, equally weighted luminance values may be used for conversion between color spaces.
At operation 305, an electronic device captures color information (e.g., for an image). The electronic device may include an image detector that has a plurality of photo detectors may be used to capture the image. Each of the photo detectors may sample a single color (e.g., a subpixel corresponding to a white photosite is assumed to sample a white color which may be a combination of all red, green, and blue). According to various embodiments of the present disclosure, the color information from the plurality of photo detectors may be stored. According to various embodiments of the present disclosure, the color information constituting an intended image that may require reconstruction of the color representation thereof may be received/stored from a counterpart electronic device.
At operation 310, the electronic device may reconstruct the color values at the subpixels. The electronic device may calculate the red color values, the green color values, the blue color values at each of the subpixels. The electronic device may calculate the red color values, the green color values, the blue color values, and the Q color values at each of the subpixels. The electronic device may calculate the corresponding color values at each subpixel using interpolation or another numerical method (e.g., a local average of a corresponding color value).
At operation 315, the electronic device may convert the red color values, the green color values, and the blue color values to a luminance-chrominance format. According to various embodiments of the present disclosure, the electronic device may convert the RGB color space to an XYZ space.
According to various embodiments of the present disclosure, the raw RGB values from the image detector (e.g., the plurality of photo detectors corresponding to the photosites) may be converted into an xyQ chromaticity space. According to various embodiments of the present disclosure, the reconstructed chromatic color values (e.g., the demosaiced color values using RGB values from the image detector) may be converted into an xyQ chromaticity space.
The chromatic color values may be normalized to an RGB color space. The primary color values may be weighted according to Equation (1) below.
$\begin{matrix} P = (\begin{matrix} 0.6400 & 0.3000 & 01500 \\ 0.3300 & 0.6000 & 0.0600 \\ 0.0300 & 0.1000 & 0.7900 \end{matrix}) & (1) \end{matrix}$
Referring to Equation (1), P represents a matrix corresponding to a color space defined by REC 709, sRGB, NTSC primary color values. The sampled color values or the reconstructed color values for the plurality of subpixels are filtered by the matrix P to normalize the color space.
The three primary color values (e.g., red, green, and blue) may be weighted according to Equation (2).
$\begin{matrix} Y = (\begin{matrix} 1 / 3 \\ 1 / 3 \\ 1 / 3 \end{matrix}) & (2) \end{matrix}$
Referring to Equation (2), the three primary color values (e.g., red, green, and blue) may be weighted equally according to the primary color weighting matrix for Y.
The color values may be converted to an XQZ color space using Equation (3) below. For example, the sampled color values or the reconstructed color values may be converted to an XQZ using Equation (3) according to the respective primary color weightings of P and Y using Equations (1) and (2).
$\begin{matrix} M 2 X_{xyz, rgb} = P_{xyz, rgb} \cdot \sum_{xyz} (\frac{P_{xyz, rgb} \cdot Y_{rgb}}{P_{y, rgb}}) & (3) \end{matrix}$
Referring to Equation (3), M2X_xyz,rgbrepresents an equation to generate a matrix to convert the color values from an RGB color space to an XQZ space. According to various embodiments of the present disclosure, an M2X_xyz,rgbconversion matrix to convert the color values from an RGB color space to an XQZ color space using Equation (3) according to the respective primary color weightings of P and Y using Equations (1) and (2), as provided in Equation (4) below.
$\begin{matrix} M 2 X_{xyz, rgb} = (\begin{matrix} 0.64665 & 0.166667 & 0.833333 \\ 0.333333 & 0.333333 & 0.333333 \\ 0.030303 & 0.055556 & 4.388889 \end{matrix}) & (4) \end{matrix}$
Referring to Equation (4), the M2X_xyz,rgbconversion matrix to convert the color values from an RGB color space to an XQZ color space using Equation (3) according to the respective primary color weightings of P and Y using Equations (1) and (2). As provided by the conversion matrix of Equation (4), the elements of the middle row of the conversion matrix are equal to ⅓ reflecting the weighting of the primary color values defined in Equation (2).
Although the conversion matrix of Equation (4) will convert the RGB color space to an XQZ color space, the XQZ color space is not defined by chromaticity values. In other words, the conversion matrix of Equation (4) will convert the RGB values to XQZ values. According to various embodiments of the present disclosure, the XQZ values may be further converted to chromaticity values. For example, the XQZ values may be converted to xyQ chromaticity values.
According to various embodiments of the present disclosure, the formula for CIE xyY chromaticity conversion may be used to convert the XQZ values to xyQ chromaticity values. For example, the XQZ values may be converted to xyQ chromaticity values using the conversion defined by Equation (5).
$\begin{matrix} {\begin{matrix} x = X / X + Q + Z \\ y = Q / X + Q + Z \\ Q = Q \end{matrix} & (5) \end{matrix}$
Referring to Equations (1)-(5), the raw RGB values from the image detector (e.g., camera sensor) may be converted from an RGB space to an xyQ chromaticity space. According to various embodiments of the present disclosure, the reconstructed RGB values (e.g., demosaiced RGB values) may be converted from an RGB color space to the xyQ space using Equations (1)-(5).
At operation 320, the electronic device may replace the calculated Q value for a particular subpixel with an interpolated Q value for the particular subpixel. According to various embodiments of the present disclosure, the electronic device replaces the reconstructed luminance values with sampled luminance values at corresponding subpixels. According to various embodiments of the present disclosure, the luminance value of the reconstructed color values (e.g., based on the demosaicing of the chrominance color values at each subpixel) at the subpixels corresponding to Q (e.g., white) photosites is replaced with the sampled luminance value that was sampled at the corresponding Q photosite.
At operation 325, the electronic device may convert the color values from the luminance-chrominance format to an RGB format. The electronic device may convert the demosaiced color information (e.g., including chrominance color values, and luminance values) from the XYZ space to an RGB color space.
According to various embodiments of the present disclosure, the calculated Q value for a particular subpixel may be replaced with an interpolated Q value for the particular subpixel. Thereafter, the color information is converted from the luminance-chrominance format to an RGB format.
According to various embodiments of the present disclosure, the standard formula for CIE xyY chromaticity conversion to an XYZ space may be used to convert the xyQ values into XQZ values. For example, the xyQ values may be converted to XQZ chromaticity values using the conversion defined by Equation (6).
$\begin{matrix} {\begin{matrix} X = x \cdot Q / y \\ Q = Q \\ Z = (1 - x - y) \cdot Q / y \end{matrix} & (6) \end{matrix}$
Thereafter, after the color values have been converted to XQZ chromaticity values in the XQZ space, the color values may be further converted to an RGB space. The XQZ chromaticity values in the XQZ space may be converted to RGB values for the RGB space according to Equation (7).
$\begin{matrix} (\begin{matrix} R \\ G \\ B \end{matrix}) = M 2 X^{- 1} \cdot (\begin{matrix} X \\ Q \\ Z \end{matrix}) & (7) \end{matrix}$
Referring to Equation (7), M2X⁻¹corresponds to an inverse matrix of the M2X matrix defined in Equation (4).
At operation 330, the electronic device may store the color information (e.g., the color values) for the respective subpixels. The electronic device may store the color information including the chrominance color values and the luminance values for the respective subpixels. The electronic device may store the image.
At operation 335, the electronic device may render the image. The electronic device may render the image in a preview frame. The electronic device may perform further image processing before rendering the image.
FIG. 4 illustrates pseudo code for a method of reconstructing a color representation of an image according to an embodiment of the present disclosure.
Referring to FIG. 4, the pseudo code provided may reconstruct the color representation of an image according to a method such as the method described in relation to FIG. 3. For example, the pseudo code may describe a method for reconstructing the color representation of an image using Equations (1)-(7).
FIG. 5 illustrates a flowchart of a method of reconstructing a color representation of an image according to an embodiment of the present disclosure.
Referring to FIG. 5, a method for converting between a luminance-chrominance format and an RGB format that is computationally simplistic may be beneficial. For example, relatively complex computations require significant memory and processing resources. Indeed, the above conversion from an RGB space to an XQZ space, to an xqQ space, to an XQZ space, and back to an RGB space as described above in relation to FIG. 3 requires a relatively large number of floating point multiples. According to various embodiments of the present disclosure, a method for reconstructing a color representation of an image that uses fewer gates may be preferred.
According to various embodiments of the present disclosure, an RGB space may be converted into a luminance-chrominance space using only integer addition operations, integer subtraction operations, and shift operations. Such a conversion is a memory efficient method that may be used to demosaicing color information by combining chrominance with luminance. According to various embodiments of the present disclosure, a YByRy space may be used as an intermediary for correcting (e.g., replacing) the sampled (or interpolated) Q values during the demosaicing of the sampled color information from an RGBQ image detector. According to various embodiments of the present disclosure, the RGV color space may be converted to the YByRy space using a combination of only integer addition operations, integer subtraction operations, and shift operations.
According to various embodiments of the present disclosure, a Y value in the YByRy space may be defined by Equation (8) below. The YByRy color space was based on a simplified approximation to the photonic sensitivity function for luminance.
$\begin{matrix} Y = \frac{2 \cdot R + 5 \cdot G + B}{8} & (8) \end{matrix}$
According to various embodiments of the present disclosure, the conversion to a YByRy color space may be adapted (e.g., modified) to a color space using Q color values. Such a modified color space may be referred to as a QBqRq color space. According to various embodiments of the present disclosure, the Q value of the QBqRq color space may be defined by Equation (9) below.
$\begin{matrix} Q = \frac{R + G + B}{3} & (9) \end{matrix}$
At operation 505, an electronic device captures color information (e.g., for an image). The electronic device may include an image detector that has a plurality of photo detectors may be used to capture the image. Each of the photo detectors may sample a single color (e.g., a subpixel corresponding to a white photosite is assumed to sample a white color which may be a combination of all red, green, and blue). According to various embodiments of the present disclosure, the color information from the plurality of photo detectors may be stored. According to various embodiments of the present disclosure, the color information constituting an intended image that may require reconstruction of the color representation thereof may be received/stored from a counterpart electronic device.
At operation 510, the electronic device may reconstruct the color values at the subpixels. The electronic device may calculate the red color values, the green color values, the blue color values at each of the subpixels. The electronic device may calculate the red color values, the green color values, the blue color values, and the Q color values at each of the subpixels. The electronic device may calculate the corresponding color values at each subpixel using interpolation or another numerical method (e.g., a local average of a corresponding color value).
At operation 515, the electronic device may convert the red color values, the green color values, and the blue color values to a luminance-chrominance format. According to various embodiments of the present disclosure, the electronic device may convert the RGB color space to a QBqRq space.
The RGB color space may be converted to the QBqRq color space using Equation (9) and Equation (10) below.
$\begin{matrix} {\begin{matrix} B_{q} = B - Q \\ R_{q} = R - Q \end{matrix} & (10) \end{matrix}$
Referring to Equations (9) and (10), the computations for Bq and Rq may be expanded as provided in Equation (11) below.
$\begin{matrix} {\begin{matrix} B_{q} = \frac{2 \cdot B}{3} - \frac{G}{3} - \frac{R}{3} \\ R_{q} = \frac{2 \cdot R}{3} - \frac{G}{3} - \frac{B}{3} \end{matrix} & (11) \end{matrix}$
Referring to Equations (9) and (11), the computation to convert color values from the RGB color space to the QBqRq color space are simple and computationally efficient.
The Equations (9) and (11) for conversion between an RGB color space and a QBqRq color space may be expressed in matrix form as provided by Equation (12).
$\begin{matrix} (\begin{matrix} R \\ G \\ B \end{matrix}) = {(\begin{matrix} \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ - \frac{1}{3} & - \frac{1}{3} & \frac{2}{3} \\ \frac{2}{3} & - \frac{1}{3} & - \frac{1}{3} \end{matrix})}^{- 1} \cdot (\begin{matrix} Q \\ B_{q} \\ R_{q} \end{matrix}) & (12) \end{matrix}$
Referring to Equation (12), the ‘⅓’ may be factored out of the conversion matrix. A resulting matrix enables the conversion of the RGB color space to the QBqRq color space to be done with only a combination of integer addition operations, integer subtraction operations, shift operations, and one divide-by-three operation.
According to various embodiments of the present disclosure, the divide-by-three operation may be done as an integer multiplication operation by a fixed point binary inverse value and a shift right.
At operation 520, the electronic device may replace the calculated Q value for a particular subpixel with an interpolated Q value for the particular subpixel. According to various embodiments of the present disclosure, the electronic device replaces the reconstructed luminance values with sampled luminance values at corresponding subpixels. According to various embodiments of the present disclosure, the luminance value of the reconstructed color values (e.g., based on the demosaicing of the chrominance color values at each subpixel) at the subpixels corresponding to Q (e.g., white) photosites is replaced with the sampled luminance value that was sampled at the corresponding Q photosite.
At operation 525, the electronic device may convert the color values from the luminance-chrominance format to an RGB format. The electronic device may convert the demosaiced color information (e.g., including chrominance color values, and luminance values) from the QBqRq color space to the RGB color space.
According to various embodiments of the present disclosure, the color values may be converted from the QBqRq color space to the RGB color space using Equation (12) below.
$\begin{matrix} (\begin{matrix} R \\ G \\ B \end{matrix}) = {(\begin{matrix} 1 & 0 & 1 \\ 1 & - 1 & - 1 \\ 1 & 1 & 0 \end{matrix})}^{- 1} \cdot (\begin{matrix} Q \\ B_{q} \\ R_{q} \end{matrix}) & (13) \end{matrix}$
Referring to Equation (12), a benefit associated with expressing the conversion of the color values from the RGB color space to the QBqRq color space as provided by Equation (11) is that the conversion matrix may be inverted to create the formula (e.g., the conversion matrix) for converting the color values from the QBqRq color space to the RGB color space.
At operation 530, the electronic device may store the color information (e.g., the color values) for the respective subpixels. The electronic device may store the color information including the chrominance color values and the luminance values for the respective subpixels. The electronic device may store the image.
At operation 535, the electronic device may render the image. The electronic device may render the image in a preview frame. The electronic device may perform further image processing before rendering the image.
FIG. 6 illustrates pseudo code for a method of reconstructing a color representation of an image according to an embodiment of the present disclosure.
Referring to FIG. 6, the pseudo code provided may reconstruct the color representation of an image according to a method such as the method described in relation to FIG. 5. For example, the pseudo code may describe a method for reconstructing the color representation of an image using Equations (8)-(13).
FIG. 7 illustrates a flowchart of a method of reconstructing a color representation of an image according to an embodiment of the present disclosure.
Referring to FIG. 7, color information across a plurality of subpixels may be reconstructed (e.g., demosaiced) using the red color values, the green color values, and the blue color values. Thereafter, the reconstructed color values (e.g., the calculated red color values, the calculated green color values, and the calculated blue color values) may be adjusted to take into account a Q value.
According to various embodiments of the present disclosure, the reconstructed color values (e.g., the calculated red color values, the calculated green color values, and the calculated blue color values) may be adjusted to take into account the Q value without converting the color values from the RGB color space to a luminance-chrominance color space.
According to various embodiments of the present disclosure, a Q value may be approximated by summing the individual sampled RBG values (e.g., the sampled red color value, the sampled blue color value, and the sampled green color value). According to various embodiments of the present disclosure, a Q value may be approximated by summing the individual demosaiced RBG values (e.g., the sampled red color value, the sampled blue color value, and the sampled green color value).
According to various embodiments of the present disclosure, if the luminance (e.g., the Q color value) is defined as the sum of the individual RBG values, then a method for demosaicing color information including consideration of the Q values (e.g., sampled Q values, or demosaiced Q values) may substitute the sampled (or interpolated) Q value as the luminance value while in the RGB color space. According to various embodiments of the present disclosure, the sampled (or interpolated) Q value is substituted while the chrominance is maintained. According to various embodiments of the present disclosure, the sampled (or interpolated) Q value is substituted while the relative proportions of the individual RGB values are maintained.
According to various embodiments of the present disclosure, the RGB color vector is scaled or normalized such that the sum of the individual demosaiced RBG values is equal to the sampled (or interpolated) Q value at a particular subpixel.
At operation 705, an electronic device captures color information (e.g., for an image). The electronic device may include an image detector that has a plurality of photo detectors may be used to capture the image. Each of the photo detectors may sample a single color (e.g., a subpixel corresponding to a white photosite is assumed to sample a white color which may be a combination of all red, green, and blue). According to various embodiments of the present disclosure, the color information from the plurality of photo detectors may be stored. According to various embodiments of the present disclosure, the color information constituting an intended image that may require reconstruction of the color representation thereof may be received/stored from a counterpart electronic device.
At operation 710, the electronic device may reconstruct the color values at the subpixels. The electronic device may calculate the red color values, the green color values, the blue color values at each of the subpixels. The electronic device may calculate the red color values, the green color values, the blue color values, and the Q color values at each of the subpixels. The electronic device may calculate the corresponding color values at each subpixel using interpolation or another numerical method (e.g., a local average of a corresponding color value).
At operation 715, the electronic device may renormalize the reconstructed color value at a particular subpixel. The electronic device may renormalize the reconstructed color value at the particular subpixel using a sampled Q value if the particular subpixel corresponds to a Q photosite. The electronic device may renormalize the reconstructed color value at the particular subpixel using an interpolated Q value if the particular subpixel corresponds to a red color photosite, a blue color photosite, or a green color photosite.
According to various embodiments of the present disclosure, the electronic device may renormalize the reconstructed color values at every subpixel.
According to various embodiments of the present disclosure, the electronic device may renormalize the reconstructed color values using Equation (14) below.
$\begin{matrix} {\begin{matrix} R^{'} = R * \frac{Q}{R + G + B} \\ G^{'} = G * \frac{Q}{R + G + B} \\ B^{'} = B * \frac{Q}{R + G + B} \end{matrix} & (14) \end{matrix}$
Referring to Equation 14, R′ corresponds to a renormalized red color value for a corresponding subpixel, G′ corresponds to a renormalized green color value for a corresponding subpixel, and B′ corresponds to a renormalized blue color value for a corresponding subpixel. Q corresponds to one of a sampled Q value and an interpolated Q value for a corresponding subpixel. For example, Q refers to the sampled Q value if the particular subpixel corresponds to a Q photosite. As another example, Q refers to an interpolated Q value if the particular subpixel corresponds to a red color photosite, a blue color photosite, or a green color photosite. R corresponds to the reconstructed (e.g., demosaiced) red color value at a corresponding subpixel, B corresponds to the reconstructed (e.g., demosaiced) blue color value at a corresponding subpixel, and G corresponds to the reconstructed (e.g., demosaiced) green color value at a corresponding subpixel.
According to various embodiments of the present disclosure, the electronic device may renormalize the reconstructed color values using Equation (15) below.
$\begin{matrix} {\begin{matrix} Δ L = Q - (R + B + G) \\ R^{'} = R + \frac{Δ L}{3} \\ G^{'} = G + \frac{Δ L}{3} \\ B^{'} = B + \frac{Δ L}{3} \end{matrix} & (15) \end{matrix}$
Referring to Equation (15), R′ corresponds to a renormalized red color value for a corresponding subpixel, G′ corresponds to a renormalized green color value for a corresponding subpixel, and B′ corresponds to a renormalized blue color value for a corresponding subpixel. Q corresponds to one of a sampled Q value and an interpolated Q value for a corresponding subpixel. For example, Q refers to the sampled Q value if the particular subpixel corresponds to a Q photosite. As another example, Q refers to an interpolated Q value if the particular subpixel corresponds to a red color photosite, a blue color photosite, or a green color photosite. R corresponds to the reconstructed (e.g., demosaiced) red color value at a corresponding subpixel, B corresponds to the reconstructed (e.g., demosaiced) blue color value at a corresponding subpixel, and G corresponds to the reconstructed (e.g., demosaiced) green color value at a corresponding subpixel.
Referring to Equation (15), the resulting luminance as defined by the sum of the individual RBG values (e.g., R′+B′+G′) is indeed equal to Q. However, the strict chrominance as defined by the relative ratios of R, G, and B is not maintained because there is a saturation/desaturation effect as the requisite white vector (e.g., (ΔL, ΔL, ΔL)) is added or subtracted depending on a sign thereof (e.g., depending on whether the vector is positive or negative). For small modulations of luminance, the saturation deviations are small. However, larger modulations of luminance may cause observable saturation/desaturation artifacts. In other words, although the use of Equation (15) is computationally simpler and thus more cost effective (e.g., less resource intensive), use of Equation (15) to adjust for the measured Q value may result in a color shift.
According to various embodiments of the present disclosure, a method of renormalizing the reconstructed color values may be used for color images (e.g., images) for which the modulations of luminance are relatively small.
At operation 720, the electronic device may store the color information (e.g., the color values) for the respective subpixels. The electronic device may store the color information including the chrominance color values and the luminance values for the respective subpixels. The electronic device may store the image.
At operation 725, the electronic device may render the image. The electronic device may render the image in a preview frame. The electronic device may perform further image processing before rendering the image.
FIG. 8 illustrates pseudo code for a method of reconstructing a color representation of an image according to an embodiment of the present disclosure.
Referring to FIG. 8, the pseudo code provided may reconstruct the color representation of an image according to a method such as the method described in relation to FIG. 7. For example, the pseudo code may describe a method for reconstructing the color representation of an image using Equation (14).
FIG. 9 illustrates pseudo code for a method of reconstructing a color representation of an image according to an embodiment of the present disclosure.
Referring to FIG. 9, the pseudo code provided may reconstruct the color representation of an image according to a method such as the method described in relation to FIG. 7. For example, the pseudo code may describe a method for reconstructing the color representation of an image using Equation (15).
FIG. 10 illustrates a block diagram schematically illustrating a configuration of an electronic device according to an embodiment of the present disclosure.
Referring to FIG. 10, an electronic device 1000 may include a control unit 1010, a storage unit 1020, a camera unit 1030, an image processing unit 1040, a display unit 1050, an input unit 1060, and a communication unit 1070.
According to various embodiments of the present disclosure, the electronic device 1000 comprises at least one control unit 1010. The at least one control unit 1010 may be configured to operatively control the electronic device 1000. For example, the at least one control unit 1010 may control operation of the various components or units included in the electronic device 1000. The at least one control unit 1010 may transmit a signal to the various components included in the electronic device 1000 and control a signal flow between internal blocks of the electronic device 1000. The at least one control unit 1010 may be or otherwise include at least one processor. The at least one control unit 1010 may include an Application Processor (AP), and/or the like.
The storage unit 1020 may be configured to store user data, and/or the like, as well a program which performs operating functions according to various embodiments of the present disclosure. The storage unit 1020 may include a non-transitory computer-readable storage medium. As an example, the storage unit 620 may store a program for controlling general operation of the electronic device 1000, an Operating System (OS) which boots the electronic device, and application program for performing other optional functions such as a camera function, a sound replay function, an image or video replay function, a signal strength measurement function, a route generation function, image processing, and/or the like. Further, the storage unit 1020 may store user data generated according to a user of the electronic device 1000, such as, for example, a text message, a game file, a music file, a movie file, and/or the like. According to various embodiments of the present disclosure, the storage unit 1020 may store an application or a plurality of applications that individually or in combination operate a camera unit (not shown) to capture (e.g., contemporaneously) one or more images, and/or the like. According to various embodiments of the present disclosure, the storage unit 1020 may store an application or a plurality of applications that individually or in combination operate the image processing unit 1040 or the control unit 1010 to determine (e.g., reconstruct) color values at green subpixels, to determine (e.g., reconstruct) color values at red subpixels, to determine (e.g., reconstruct) color values at blue subpixels, to determine (e.g., reconstruct) color values at white (Q) subpixels, to convert the color values (e.g., the reconstructed) color values to store color values for one or more subpixels, to render an image (e.g., using reconstructed color values), to convert the color values (e.g., RGB tri-stimulus values) to a luminance-chrominance format, to convert color values from a luminance-chrominance format to an RGB format, to replace a calculated Q value of the reconstructed color values with a sampled Q value, to replace a calculated Q value of the reconstructed color values with an interpolated Q value, and/or the like. The storage unit 1020 may store an application or a plurality of applications that individually or in combination operate the control unit 1010 and the communication unit 1070 to communicate with a counterpart electronic device to receive color information (e.g., color values) for an image from the counterpart electronic device, and/or the like. The storage unit 1020 may store an application or a plurality of applications that individually or in combination operate the display unit 1050 to display a graphical user interface, an image, a video, and/or the like. The storage unit 1020 may store an application or a plurality of applications that individually or in combination operate the display unit 1050 to display a preview image using reconstructed color information using reconstruction (e.g., demosaicing) methods according to various embodiments of the present disclosure. The preview image may be displayed in a viewfinder while the camera unit 1030 is operated. The preview image may display a view of that may be captured upon command to capture an image.
The camera unit 1030 may capture an image. The camera unit 1030 may include an image detector (not shown) that includes a plurality of photo detectors. Each of the plurality of photo detectors may correspond to a photosite. The camera unit may capture a preview image as the camera unit 1030 is operated. The preview image may be displayed on the display unit 1050 while the camera unit 1030 is operated (e.g., before the camera unit 1030 is instructed to capture the image).
The image processing unit 1040 may be configured to process image data, images, and/or the like. The image processing unit 1040 may include a Sub Pixel Rendering (SPR) unit (not shown), a demosaicing unit (not shown), and/or the like. In the alternative or in addition, the image processing unit 1040 may be configured to perform demosaicing of image data and/or images, SPR, and/or the like. The image processing unit 1040 be configured to determine (e.g., reconstruct) color values at green subpixels, to determine (e.g., reconstruct) color values at red subpixels, to determine (e.g., reconstruct) color values at blue subpixels, to determine (e.g., reconstruct) color values at white (Q) subpixels, to convert the color values (e.g., the reconstructed) color values to store color values for one or more subpixels, to render an image (e.g., using reconstructed color values), to convert the color values (e.g., RGB tri-stimulus values) to a luminance-chrominance format, to convert color values from a luminance-chrominance format to an RGB format, to replace a calculated Q value of the reconstructed color values with a sampled Q value, to replace a calculated Q value of the reconstructed color values with an interpolated Q value, and/or the like.
The display unit 1050 displays information inputted by user or information to be provided to user as well as various menus of the electronic device. For example, the display unit 1050 may provide various screens according to a user of the electronic device, such as an idle screen, a message writing screen, a calling screen, a route planning screen, and the like. According to various embodiments of the present disclosure, the display unit 1050 may display an interface which the user may manipulate or otherwise enter inputs via a touch screen to enter selection of the function relating to the signal strength of the electronic device. The display unit 1050 can be formed as a Liquid Crystal Display (LCD), an Organic Light Emitting Diode (OLED), an Active Matrix Organic Light Emitting Diode (AMOLED), and the like. However, various embodiments of the present disclosure are not limited to these examples. Further, the display unit 1050 can perform the function of the input unit 1060 if the display unit 1050 is formed as a touch screen.
The input unit 1060 may include input keys and function keys for receiving user input. For example, the input unit 1060 may include input keys and function keys for receiving an input of numbers or various sets of letter information, setting various functions, and controlling functions of the electronic device. For example, the input unit 1060 may include a calling key for requesting a voice call, a video call request key for requesting a video call, a termination key for requesting termination of a voice call or a video call, a volume key for adjusting output volume of an audio signal, a direction key, and the like. In particular, according to various embodiments of the present disclosure, the input unit 1060 may transmit to the at least one control unit 1010 signals related to the operation of a camera unit (not shown), to selection of an image, to selection of a viewpoint, and/or the like. Such an input unit 1060 may be formed by one or a combination of input means such as a touch pad, a touchscreen, a button-type key pad, a joystick, a wheel key, and the like.
The communication unit 1070 may be configured for communicating with other electronic devices and/or networks. According to various embodiments of the present disclosure, the communication unit 1070 may be configured to communicate using various communication protocols and various communication transceivers. For example, the communication unit 1070 may be configured to communicate via Bluetooth technology, NFC technology, WiFi technology, 2G technology, 3G technology, LTE technology, or another wireless technology, and/or the like.
According to various embodiments of the present disclosure, demosaiced color values (e.g., demosaiced red color values, demosaiced green color values, and demosaiced blue color values) may be adjusted according to sampled luminance values (e.g., Q values, or otherwise values detected at photo detectors corresponding to Q photosites). According to various embodiments of the present disclosure, demosaiced color values (e.g., demosaiced red color values, demosaiced green color values, and demosaiced blue color values) may be adjusted according to interpolated luminance values (e.g., Q values, or otherwise values detected at photo detectors corresponding to Q photosites).
According to various embodiments of the present disclosure, demosaiced color values (e.g., demosaiced red color values, demosaiced green color values, and demosaiced blue color values) may be normalized according to sampled luminance values (e.g., Q values, or otherwise values detected at photo detectors corresponding to Q photosites). According to various embodiments of the present disclosure, demosaiced color values (e.g., demosaiced red color values, demosaiced green color values, and demosaiced blue color values) may be normalized according to interpolated luminance values (e.g., Q values, or otherwise values detected at photo detectors corresponding to Q photosites). According to various embodiments of the present disclosure, demosaiced color values (e.g., demosaiced red color values, demosaiced green color values, and demosaiced blue color values) may be normalized using a measured (e.g., sampled or interpolated) luminance value (e.g., Q value) while in the RGB color space.
According to various embodiments of the present disclosure, luminance values corresponding to the demosaiced color values (e.g., demosaiced red color values, demosaiced green color values, and demosaiced blue color values) may be replaced with sampled luminance values (e.g., Q values, or otherwise values detected at photo detectors corresponding to Q photosites). According to various embodiments of the present disclosure, luminance values corresponding to the demosaiced color values (e.g., demosaiced red color values, demosaiced green color values, and demosaiced blue color values) may be replaced with interpolated luminance values (e.g., Q values, or otherwise values detected at photo detectors corresponding to Q photosites). According to various embodiments of the present disclosure, luminance values corresponding to the demosaiced color values (e.g., demosaiced red color values, demosaiced green color values, and demosaiced blue color values) may be replaced a measured (e.g., sampled or interpolated) luminance value (e.g., Q value) while in a luminance-chrominance color space.
According to various embodiments of the present disclosure, demosaiced color values (e.g., demosaiced red color values, demosaiced green color values, and demosaiced blue color values) may be adjusted according to sampled luminance values (e.g., Q values, or otherwise values detected at photo detectors corresponding to Q photosites) so as to derive a sharper RGB value.
It will be appreciated that various embodiments of the present disclosure according to the claims and description in the specification can be realized in the form of hardware, software or a combination of hardware and software.
Any such software may be stored in a non-transitory computer readable storage medium. The non-transitory computer readable storage medium stores one or more programs (software modules), the one or more programs comprising instructions, which when executed by one or more processors in an electronic device, cause the electronic device to perform a method of the present disclosure.
Any such software may be stored in the form of volatile or non-volatile storage such as, for example, a storage device like a Read Only Memory (ROM), whether erasable or rewritable or not, or in the form of memory such as, for example, Random Access Memory (RAM), memory chips, device or integrated circuits or on an optically or magnetically readable medium such as, for example, a Compact Disk (CD), Digital Versatile Disc (DVD), magnetic disk or magnetic tape or the like. It will be appreciated that the storage devices and storage media are various embodiments of non-transitory machine-readable storage that are suitable for storing a program or programs comprising instructions that, when executed, implement various embodiments of the present disclosure. Accordingly, various embodiments provide a program comprising code for implementing apparatus or a method as claimed in any one of the claims of this specification and a non-transitory machine-readable storage storing such a program.
While the disclosure has been shown and described with reference to various embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the disclosure as defined by the appended claims and their equivalents.

Claims

What is claimed is:

1. A method for demosaicing sampled color values, the method comprising:

capturing color information using an RGBW image detector;

demosaicing red color values, green color values, and blue color values at subpixels respectively corresponding to red photosites of the image detector, green photosites of the image detector, blue photosites of the image detector, and white photosites of the image detector; and

changing one or more of the demosaiced red color value, the demosaiced blue color value, and the demosaiced green color value at a particular subpixel based at least in part on color information sampled at one or more white subpixels corresponding to the white photosites.

2. The method of claim 1, further comprising:

demosaicing the color information sampled at one or more white subpixels corresponding to the white photosites across red subpixels, green subpixels, blue subpixels, and white subpixels.

3. The method of claim 2, wherein the changing the one or more of the demosaiced red color value, the demosaiced blue color value, and the demosaiced green color value at a particular subpixel based at least in part on color information sampled at one or more white subpixels corresponding to the white photosites comprises:

changing one or more of the demosaiced red color value, the demosaiced blue color value, and the demosaiced green color value at a particular subpixel based at least in part on the demosaiced color information at the one or more white subpixels corresponding to the white photosites across red subpixels, green subpixels, blue subpixels, and white subpixels.

4. The method of claim 2, wherein the color information sampled at one or more white subpixels corresponding to the white photosites includes a luminance value.

5. The method of claim 1, further comprising:

converting a color space of the one or more of the demosaiced red color values, the demosaiced blue color values, and the demosaiced green color values for one or more subpixels from a RGB color space to a luminance-chrominance color space.

6. The method of claim 5, wherein the changing the one or more of the demosaiced red color value, the demosaiced blue color value, and the demosaiced green color value at a particular subpixel based at least in part on color information sampled at one or more white subpixels corresponding to the white photosites comprises:

replacing a calculated luminance value relating to the one or more demosaiced red color values, the demosaiced blue color values, and the demosaiced green color values for one or more subpixels with a luminance value included in color information sampled at one or more white subpixels corresponding to the white photosites.

7. The method of claim 6, wherein the replacing the calculated luminance value relating to the one or more demosaiced red color values, the demosaiced blue color values, and the demosaiced green color values for one or more subpixels with the luminance value included in color information sampled at one or more white subpixels corresponding to the white photosites comprises:

replacing the calculated luminance value for a subpixel corresponding to a particular white photosite with a sampled luminance value captured at the particular white photosite.

8. The method of claim 6, wherein the replacing the calculated luminance value relating to the one or more demosaiced red color values, the demosaiced blue color values, and the demosaiced green color values for one or more subpixels with the luminance value included in color information sampled at one or more white subpixels corresponding to the white photosites comprises:

replacing the calculated luminance value for a subpixel corresponding to a particular non-white photosite with an interpolated luminance value calculated based using one or more sampled luminance values respectively captured at one or more white photosites neighboring the particular non-white photosite.

9. The method of claim 6, further comprising:

converting a color space of the one or more color values comprising a replaced luminance value from the luminance-chrominance color space to the RGB color space.

10. The method of claim 5, wherein the luminance-chrominance color space corresponds to an XQZ color space.

11. The method of claim 5, wherein the luminance-chrominance color space corresponds to an QBqRq color space.

12. The method of claim 1, wherein the changing one or more of the demosaiced red color value, the demosaiced blue color value, and the demosaiced green color value at a particular subpixel based at least in part on color information sampled at one or more white subpixels corresponding to the white photosites occurs in a current color space of the one or more demosaiced color values.

13. The method of claim 1, wherein the changing one or more of the demosaiced red color value, the demosaiced blue color value, and the demosaiced green color value at a particular subpixel based at least in part on color information sampled at one or more white subpixels corresponding to the white photosites comprises:

normalizing one or more of the demosaiced red color value, the demosaiced blue color value, and the demosaiced green color value at the particular subpixel using a luminance value included in color information sampled at one or more white subpixels corresponding to the white photosites.

14. The method of claim 13, wherein the normalizing of the one or more of the demosaiced red color value, the demosaiced blue color value, and the demosaiced green color value at the particular subpixel using a luminance value included in color information sampled at one or more white subpixels corresponding to the white photosites comprises:

normalizing the one or more demosaiced color values at a subpixel corresponding to a particular white photosite using a sampled luminance value captured at the particular white photosite.

15. The method of claim 13, wherein the normalizing of the one or more of the demosaiced red color value, the demosaiced blue color value, and the demosaiced green color value at the particular subpixel using a luminance value included in color information sampled at one or more white subpixels corresponding to the white photosites comprises:

normalizing the one or more demosaiced color values at a subpixel corresponding to a particular non-white photosite with an interpolated luminance value calculated using one or more sampled luminance values respectively captured at one or more white photosites neighboring the particular non-white photosite.

16. A non-transitory computer-readable storage medium storing instructions that, when executed, cause at least one processor to perform the method of claim 1.

17. An apparatus for demosaicing sampled color values, the apparatus comprising:

an RGBW image detector configured to capture color information;

a storage unit configured to store color information; and

at least one processor configured to capture color information using the RGBW image detector; to demosaic red color values, green color values, and blue color values at subpixels respectively corresponding to red photosites of the image detector, green photosites of the image detector, blue photosites of the image detector, and white photosites of the image detector, and to change one or more of the demosaiced red color value, the demosaiced blue color value, and the demosaiced green color value at a particular subpixel based at least in part on color information sampled at one or more white subpixels corresponding to the white photosites.

18. The apparatus of claim 17, wherein the at least one processor is further configured to demosaic the color information sampled at one or more white subpixels corresponding to the white photosites across red subpixels, green subpixels, blue subpixels, and white subpixels.

19. The apparatus of claim 18, wherein the at least one processor is further configured to change one or more of the demosaiced red color value, the demosaiced blue color value, and the demosaiced green color value at a particular subpixel based at least in part on the demosaiced color information at the one or more white subpixels corresponding to the white photosites across red subpixels, green subpixels, blue subpixels, and white subpixels.

20. The apparatus of claim 18, wherein the color information sampled at one or more white subpixels corresponding to the white photosites includes a luminance value.

21. The apparatus of claim 17, wherein the at least one processor is further configured to convert a color space of the one or more of the demosaiced red color values, the demosaiced blue color values, and the demosaiced green color values for one or more subpixels from a RGB color space to a luminance-chrominance color space.

22. The apparatus of claim 21, wherein the at least one processor is further configured to replace a calculated luminance value relating to the one or more demosaiced red color values, the demosaiced blue color values, and the demosaiced green color values for one or more subpixels with a luminance value included in color information sampled at one or more white subpixels corresponding to the white photosites.

23. The apparatus of claim 22, wherein the at least one processor is further configured to replace the calculated luminance value for a subpixel corresponding to a particular white photosite with a sampled luminance value captured at the particular white photosite.

24. The apparatus of claim 22, wherein the at least one processor is further configured to replace the calculated luminance value for a subpixel corresponding to a particular non-white photosite with an interpolated luminance value calculated based using one or more sampled luminance values respectively captured at one or more white photosites neighboring the particular non-white photosite.

25. The apparatus of claim 22, wherein the at least one processor is further configured to convert a color space of the one or more color values comprising a replaced luminance value from the luminance-chrominance color space to the RGB color space.

26. The apparatus of claim 21, wherein the luminance-chrominance color space corresponds to an XQZ color space.

27. The apparatus of claim 21, wherein the luminance-chrominance color space corresponds to an QBqRq color space.

28. The apparatus of claim 17, wherein the at least one processor changes the one or more of the demosaiced red color value, the demosaiced blue color value, and the demosaiced green color value at a particular subpixel based at least in part on color information sampled at one or more white subpixels corresponding to the white photosites in a current color space of the one or more demosaiced color values.

29. The apparatus of claim 17, wherein the at least one processor is further configured to normalize one or more of the demosaiced red color value, the demosaiced blue color value, and the demosaiced green color value at the particular subpixel using a luminance value included in color information sampled at one or more white subpixels corresponding to the white photosites.

30. The apparatus of claim 29, wherein the at least one processor is further configured to normalize the one or more demosaiced color values at a subpixel corresponding to a particular white photosite using a sampled luminance value captured at the particular white photosite.

31. The apparatus of claim 29, wherein the at least one processor is further configured to normalize the one or more demosaiced color values at a subpixel corresponding to a particular non-white photosite with an interpolated luminance value calculated using one or more sampled luminance values respectively captured at one or more white photosites neighboring the particular non-white photosite.

32. A method of demosaicing color information, the method comprising:

capturing color information using an RGBW image detector;

demosaicing the captured color information using chrominance values included in the captured color information; and

modifying the demosaiced color information using luminance values included in the captured color information.