TWI536765B - Imaging systems with clear filter pixels - Google Patents

Description

Imaging system with transparent filter pixels

The present application claims the benefit of U.S. Patent Application Serial No. 13/736,768, filed on Jan. 8, 2013, and U.S. Provisional Application Serial No. 61/612,819, filed on Mar. This is hereby incorporated by reference.

The present invention is directed to imaging devices, and more particularly to imaging devices having transparent image pixels.

Image sensors are commonly used in electronic devices such as cellular phones, cameras, and computers to capture images. In a typical arrangement, the electronic device has an array of image pixels arranged in a column of pixels and rows of pixels. A circuit is typically coupled to each pixel row for reading image signals from the image pixels.

Conventional imaging systems employ a single image sensor in which the visible spectrum is sampled by red, green, and blue (RGB) image pixels arranged in a Bayer mosaic pattern. The Bayer mosaic pattern consists of repeating units of two by two image pixels, two of which are diagonally opposite each other, and the other corners are red and blue. However, the Bayer pattern cannot easily achieve further miniaturization of the image sensor via a smaller image pixel size because of the limitations in the signal-to-noise ratio (SNR) in the image signal captured from the image pixels.

One way to improve SNR is to increase the available image signal by increasing the exposure at low light levels, where SNR limits image quality. One conventional method uses a subtraction filter in which, for example, red, green, and blue image pixels are replaced by cyan, magenta, and yellow image pixels. for. However, such signals typically must be converted to RGB or some equivalent output image signal color to be capable of driving most conventional image displays. This transformation typically involves the use of a color correction matrix (CCM) to modify the captured image signal, which amplifies the noise so that the effects of increased exposure are compromised.

Accordingly, there will be a need for an imaging device that provides improved means of capturing and processing image signals.

10‧‧‧Electronic devices

12‧‧‧ camera module

14‧‧‧ lens

16‧‧‧Image Sensor

18‧‧‧Processing Circuit

40‧‧‧Line

122‧‧‧Control and processing circuits

124‧‧‧ column decoder circuit

126‧‧‧ row decoder circuit

128‧‧‧Control path

190‧‧•Image Sensor Pixels/Image Pixels

192‧‧‧unit unit

194‧‧‧Unit unit

196‧‧‧unit unit

200‧‧‧pixel array

300‧‧‧ processor system

391‧‧‧Input/Output (I/O) devices

392‧‧‧ Random access memory

393‧‧‧ Busbar

394‧‧‧Removable memory

395‧‧‧Central Processing Unit

396‧‧‧ lens

397‧‧‧Shutter release button

2000‧‧‧ imaging device

1 is a diagram of an illustrative electronic circuit having an imaging system in accordance with an embodiment of the present invention.

2 is a diagram of an illustrative pixel array and associated control circuitry for reading pixel data from image pixels along a row line in an image sensor, in accordance with an embodiment of the present invention.

3 through 5 are diagrams of illustrative pixel unit cells having transparent filter pixels in accordance with an embodiment of the present invention.

6 is a flow diagram of illustrative steps that may be performed by processing circuitry in an imaging system to process image signals received from a filtered pixel array, in accordance with an embodiment of the present invention.

7 is a flow diagram of illustrative steps that may be performed by processing circuitry in an imaging system to demosaize and filter image signals received from a filtered pixel array, in accordance with an embodiment of the present invention.

8 is a flow diagram of illustrative steps that may be performed by processing circuitry in an imaging system to apply a point filter to an image signal received from a filtered pixel array, in accordance with an embodiment of the present invention.

9 is a block diagram of a processor system employing the embodiment of FIG. 1 in accordance with an embodiment of the present invention.

Electronic devices such as digital cameras, computers, cellular phones, and other electronic devices include image sensors that collect incident light to capture images. The image sensor can include an image An array of pixels. The pixels in the image sensor may include photosensitive elements such as photodiodes that convert incident light into image signals. The image sensor can have any number of pixels (eg, hundreds or thousands or more). A typical image sensor can have, for example, hundreds or thousands or millions of pixels (eg, megapixels). The image sensor can include control circuitry, such as circuitry for operating image pixels, and readout circuitry for reading image signals corresponding to the charge generated by the photosensitive elements. The readout circuitry can include selectable readout circuitry coupled to each row of pixels that can be enabled or disabled to reduce power consumption in the device and to improve pixel readout operations.

1 is a diagram of an illustrative electronic device that captures an image using an image sensor. The electronic device 10 of FIG. 1 can be a portable electronic device such as a camera, a cellular phone, a video camera, or other imaging device that captures digital image data. The camera module 12 can be used to convert incident light into digital image data. Camera module 12 can include one or more lenses 14 and one or more corresponding image sensors 16. Light from the scene can be focused by lens 14 onto image sensor 16 during the image capture operation. Image sensor 16 may include circuitry for converting analog pixel data into corresponding digital image data to be provided to processing circuitry 18. Camera module 12 may be provided with an array of lenses 14 and an array of corresponding image sensors 16 as desired.

The processing circuit 18 can include one or more integrated circuits (eg, image processing circuits, microprocessors, storage devices such as random access memory and non-volatile memory, etc.), and can be used with camera modules 12 separate components and/or components forming part of camera module 12 (eg, forming part of an integrated circuit associated with image sensor 16 within integrated circuit or module 12 including image sensor 16) The circuit) is implemented. Image data that has been captured by camera module 12 can be processed and stored using processing circuitry 18. The processed image data may optionally be provided to an external device (eg, a computer or other device) using a wired and/or wireless communication path coupled to processing circuitry 18.

As shown in FIG. 2, image sensor 16 can include a pixel array 200 including image sensor pixels 190 (sometimes referred to as image pixels 190) and control and processing circuitry 122. Array 200 can Image sensor pixels 190 containing, for example, hundreds or thousands of columns and rows. Control circuit 122 can be coupled to column decoder circuit 124 and row decoder circuit 126. Column decoder circuit 124 can receive the column address from control circuit 122 and supply corresponding column control signals, such as reset, column select, transfer, and read control signals, to pixel 190 via control path 128. One or more conductive lines, such as row line 40, may be coupled to each of pixels 190 in array 200. The row line 40 can be used to read an image signal from the pixel 190 and to supply a bias signal (eg, a bias current or a bias voltage) to the pixel 190. During the pixel read operation, the pixel columns in array 200 can be selected using column decoder circuit 124, and the image material associated with image pixels 190 in the pixel column can be read along row line 40.

Row decoder circuit 126 may include a sample and hold circuit, an amplifier circuit, an analog/digital conversion circuit, a bias circuit, a line memory, a latch circuit for selectively enabling or disabling a row circuit, or coupled to One or more of the pixels in array 200 are used to operate pixel 190 and to be used to read other signals from pixel 190. Row decoder circuit 126 can be used to selectively provide power to the row circuitry on a selected subset of row lines 40. Readout circuitry, such as signal processing circuitry (eg, sample and hold circuitry, and analog/digital conversion circuitry) associated with row decoder circuitry 126, can be used to supply digital image data on path 210 for pixels in selected pixel rows To processor 18 (Fig. 1).

Image sensor pixels, such as image pixels 190, are conventionally provided with a color filter array that allows a single image sensor to use red, green, and blue image sensor pixels arranged in a Bayer mosaic pattern to pair red, green, and Blue (RGB) light is sampled. The Bayer mosaic pattern consists of repeating unit cells of two by two image pixels, wherein the two green image pixels are diagonally opposite each other and adjacent to the red image pixels diagonally opposite to the blue image pixels. However, the limitations of the signal to noise ratio (SNR) associated with the Bayer mosaic pattern make it difficult to reduce the size of image sensors such as image sensor 16. Therefore, an image sensor capable of providing an improved means of capturing images may be required.

In a suitable example of this article, sometimes discussed as an example, in the Bayer pattern The green pixels are replaced by transparent image pixels, as shown in Figure 3. As shown in FIG. 3, the unit cell 192 of the image pixel 190 can be formed by two transparent image pixels (sometimes referred to herein as white (W) image pixels) that are diagonally opposite each other and adjacent to the same blue. (B) Red (R) image pixels diagonally opposite to the image pixel. The white image pixels 190 in unit cell 192 can be formed using a visually transparent color filter that emits light in the visible spectrum (eg, white pixels 190 can capture white light). The transparent image pixels 190 can have a natural sensitivity defined by the material forming the transparent color filter and/or the material (eg, germanium) that forms the image sensor pixels. The sensitivity of the transparent image pixels 190 can optionally be adjusted to achieve better color reproduction and/or noise characteristics by using a light absorber such as a pigment. Unit cell 192 can be repeated on image pixel array 200 to form a mosaic of red, white, and blue image pixels 190. In this way, the red image pixel can generate a red image signal in response to the red light, the blue image pixel can generate a blue image signal in response to the blue light, and the white image pixel can generate a white image signal in response to the white light. The white image signal may also be generated by white image pixels in response to any suitable combination of red, blue and/or green light.

Unit unit 192 of Figure 3 is merely illustrative. Image pixels of any color adjacent to diagonally opposite white image pixels in unit cell 192 may be formed as desired. For example, the unit cell 194 can be defined by two white image pixels 190, which are formed diagonally opposite each other and adjacent to the red image pixels diagonally opposite to the green (G) image pixels, as shown in FIG. 4 . Shown. In yet another suitable arrangement, the unit cell 196 can be defined by two white image pixels 190 that are formed diagonally opposite each other and adjacent to the blue image pixels diagonally opposite the green image pixels, as shown in FIG. Shown.

White image pixels W can help to add images by concentrating additional light compared to image pixels (such as green image pixels) having narrower color filters (eg, filters that emit light on a subset of the visible spectrum) The signal-to-noise ratio (SNR) of the image signal captured by pixel 190. The white image pixel W can especially improve the SNR under low light conditions, and the SNR may sometimes limit the image quality of the image under low light conditions. Self-white pixel (for example, as shown in Figures 3 to 5) The image signals collected by the image pixel array 200 can be converted into red, green, and blue image signals to be compatible with the circuits and software used to drive most image displays (eg, display screens, monitors, etc.). This conversion typically involves modifying the captured image signal using a color correction matrix (CCM). If not noticed, the color correction operation may undesirably amplify the noise.

In a suitable arrangement, the noise generated by the CCM can be reduced by applying strong de-noise (e.g., chrominance de-noise) prior to applying the CCM to the aggregated image signal. The chrominance denoising can be performed by processing circuitry 18 (Fig. 1) by applying a chrominance filter to the image signals that are gathered by image pixels 190. A chroma filter can be used to increase the noise correlation between image signals (eg, red, white, and blue image signals) from image pixels of different colors. Increasing the noise correlation between image signals from image pixels of different colors can reduce the noise amplification by the CCM, resulting in improved final image quality. In another arrangement, the noise amplified by the CCM can be compensated by applying a so-called "dot filter" to the captured image signal. The point filter can use high fidelity white image signals to enhance the quality of red, green and blue image signals generated using CCM. Image sensor 16 may perform both chroma denoising and dot filters to reduce the noise amplification by the CCM to produce improved brightness performance in the final image, as desired.

6 shows a flow diagram of illustrative steps that may be performed by processing circuitry, such as processing circuitry 18 of FIG. 1, to process image signals that are aggregated by a filtered pixel array, such as pixel array 200 (eg, a pixel array without green image pixels). The steps of FIG. 6 can be performed, for example, by processing circuitry 18 to reduce noise in image signals captured using unit cells having white image pixels such as those shown in FIGS. 3 through 5.

At step 100, image sensor 16 may capture image signals from the scene. The image signal captured by image sensor 16 may include a white image signal that is generated in response to light that is collected by the white pixels. The image signal may also include one or more of a red image signal, a blue image signal, or a green image signal, depending on the configuration, depending on the configuration of the image pixels used (eg, if the unit cell 192 of FIG. 3 is used, The image signal may include red, white and blue image signals; With unit unit 194 of Figure 4, the image signal can include red, white, and green image signals, etc.). In the example of Figure 6, red (R'), white (W'), and blue (B') image signals are captured. The red image signal may have a first spectral response value (integrated signal power level as a function of the frequency of the light received by the red image sensor pixel), the blue image signal may have a second spectral response value, and the white image signal may have For example, a third spectral response value greater than seventy-five percent of the sum of the first spectral response value and the second spectral response value (eg, in the case of a standard CIE illuminant E, the white image signal has a visible spectrum Wide sensitivity of equal energy radiators). The image signal may have image values corresponding to the light captured by each image pixel 190 (eg, the red image signal may include a red image value, the blue image signal may include a blue image value, etc.). The captured image signal can be communicated to processing circuitry 18 for image processing.

At step 102, a white balance operation can be performed on the captured image signal. In the example of FIG. 6, a white balance red image signal (R), a white balance white image signal (W), and a white balance blue image signal (B) can be generated.

At step 104, processing circuitry 18 may demosaic the white balance image signal and apply a chroma filter to extract red, white, and blue image data from the white balance image signal. A chroma filter can be applied to perform chrominance denoising on the white balance image signal. Processing circuitry 18 may, for example, demosaic the image signal and apply the chroma filter simultaneously, sequentially, or in an interlaced manner. The process of applying chroma filters and demosaicing to image signals can be referred to herein as "chroma demosaicing." The chroma filter increases the noise correlation between the image signals of each color (for example, the noise fluctuations in the red, white, and blue channels can be increased or decreased together in a correlated manner). For example, processing circuitry 18 can increase the associated noise between the red, white, and green image signals to 70% or more of all noise associated with the red, white, and green image signals.

By increasing the noise correlation, the processing circuitry 18 can reduce the amount of noise amplification that occurs when the CCM is applied to the image signal. Chroma de-mosaic of the image signal allows for the determination of missing color image signals from available color image signals (eg, images not produced by image pixels) Color image signal). In this example, the green image signal may be lost from the aggregated image signal because the green color filter is not used in unit cell 192 (Fig. 3). The green image signal can be determined using white, red, and blue image signals (eg, by performing a subtraction operation). In general, the available color image signals can be used to determine either of the primary color additions (eg, red, green, and blue). It may be desirable to generate red, green, and blue image signals regardless of the color filters used on image pixel array 200, as display systems typically use red, green, and blue pixels to display images.

At step 106, processing circuitry 18 may apply a color correction matrix (CCM) to the red image data, the white image data, and the blue image data. The CCM can, for example, extract green image data from blush image data to produce red, green, and blue image data. For example, CCM can convert image data into standard red, standard green, and standard blue image data (sometimes collectively referred to as linear sRGB image data or simply sRGB image data). In another suitable arrangement, the CCM can extract green image data from red and/or blue image data. A gamma correction process can be performed on the linear sRGB image data as needed. After gamma correction, sRGB image data can be used for display using an image display device. In some cases, additional noise reduction may be required (eg, by applying a point filter to the sRGB image data) to further alleviate noise amplification due to the application of CCM to red, white, and blue image data. The processing circuitry 18 can store white image data for further processing of the sRGB image data during optional step 108.

At optional step 108, processing circuitry 18 may apply a point filter to the image data (e.g., to the sRGB image data generated after applying the CCM to the red, white, and blue image material). The point filter operates on the sRGB image data to produce corrected sRGB data. A point filter can be used to further reduce noise amplification due to the application of CCM to red, white and blue image data. When displayed using a display system, the corrected sRGB data thereby provides better image quality (e.g., better brightness performance) than the sRGB data prior to applying the dot filter.

7 shows that it can be performed by processing circuitry 18 to receive images from image pixel array 200. Flowchart of illustrative steps like signal demosaicing and filtering. The steps of FIG. 7 may be performed, for example, by processing circuitry 18 to perform chroma demosaicing of the red, white, and blue image signals collected by image pixels 190 to produce sufficient noise correlation in the red, white, and blue image material. The steps of FIG. 7 can be performed, for example, as part of step 104 of FIG.

At step 110, processing circuitry 18 may demosaic the white image signal to produce white image data (eg, white image values for each image pixel). In another suitable arrangement, white image values may be generated for a combination of available image pixels 190. The white image values can be used to calculate different values of the red and blue image signals to increase the noise correlation between the red, white, and blue image signals.

At step 112, processing circuitry 18 may generate a red delta value by subtracting the white image value from the red image value for each pixel. Processing circuitry 18 may generate a blue delta value by subtracting the white image value from the blue image value. The red difference value can be calculated, for example, for each red image pixel, and the blue difference value can be calculated for each blue image pixel of the image pixel array 200.

At step 114, processing circuit 18 may filter the red and blue values using a chrominance filter. The chrominance filter can be applied to the red and blue differences, for example, by performing a weighted average of the differences calculated on the kernel of image pixels 190 (e.g., by performing a weighted average of the groups of differences calculated in step 112). . The core of the image pixels may be defined as a subset of the image pixels in the image pixel array 200 on which the chroma filtering is being performed (eg, the core may include some or all of the image pixels in the image pixel array 200). For example, when a 5 pixel by 5 pixel core is used, a weighted average of the differences is calculated for a 5 pixel by 5 pixel subset of image pixels 190 in image pixel array 200 when performing chroma filtering (eg, using image pixels) The difference between the 25 surrounding image pixels in array 200 calculates a weighted average of the differences for a given image pixel 190). In general, a core having any desired size can be used.

At step 116, the white image values may be subjected to a chroma filtered red difference value and a chroma filtered blue difference value to respectively generate a chroma filtered red image value and a chroma filtered blue image value.

At step 118, the processing circuit 18 may demosamate the chroma filtered red image values and the chroma filtered blue image values to generate red image data and blue image data with increased correlated noise (eg, performed) Chroma to mosaic red and blue image data). The de-mosaic white image data and the chroma-de-mosaic red and blue image data can then be manipulated using CCM to produce standard red, standard green, and standard blue (sRGB) image data as described above in connection with step 106 of FIG.

Figure 7 is merely illustrative. Processing circuitry 18 may demosaze chroma-filtered red and blue image values prior to generating red and blue differences, as desired (eg, processing circuitry 18 may perform step 118 prior to step 112).

If the chrominance filtering of the difference is performed on a sufficiently large core of the image pixels 190, the minimum noise from the red and blue image signals may remain in the red and blue after the chrominance filtering (eg, after performing step 114). In the value. For example, if the core has a size of 15 pixels by 15 pixels or more, chroma filtering can reduce the noise in the difference between the red and blue chroma filtered to a negligible level. The core of the image pixel 190 may include image pixels located in the plurality of image pixel arrays 200, image pixels located in the plurality of image sensors 16, and/or multiple time frames (eg, to allow time), as desired. Go to the noise image pixel. When the white image value is added to the chroma filtered difference, the noise in the white image value can dominate the noise in the difference. In this manner, the noise in the red and blue image data generated at step 116 can be substantially equal to the noise in the white image data. The noise in the red and blue image data can be highly correlated, resulting in a reduced noise amplification caused by CCM. When a Bayer pattern is used for the image pixel array 200, this process can produce less noise amplification caused by the CCM.

At step 106 (FIG. 6), the CCM can operate on red, white, and blue image data to produce linear sRGB data. For example, CCM can extract information from white image data to produce standard green data. White image data (e.g., the demosaiced white image data produced at step 104) may be retained after manipulation of the image material using CCM. The sRGB image data can be represented in a three-dimensional space such as a luminance-chroma-hue (LCH) space. In the LCH space, brightness The channel (L) can be correlated with the brightness of the image captured by the image sensor 16, and the chrominance channel (C) can be related to the color saturation of the image, and the tone channel can be associated with a particular color of the image (eg, red, purple, Yellow, green, etc.) related. The perception of noise and sharpness in the displayed image can be affected by noise and signal changes in the luminance channel. The SNR in the image data can be replaced by the luminance value in the luminance channel by converting the sRGB data into LHC data and using the white image value (which is well correlated with the overall image brightness due to the wide spectrum of the white image signal), and The LHC data is transformed back to the sRGB data for improvement. In this way, the noise amplification caused by the CCM can be suppressed in the luminance channel, where the noise is particularly noticeable to the observer when viewing the displayed image.

As described above in connection with optional step 108 of FIG. 6, a dot filter can be applied to the linear sRGB data to produce corrected sRGB data using white image data. The dot filter can operate on a single image pixel 190 without information from neighboring image pixels 190, while chroma demosaicing can be required from multiple image pixels when applied to image signals at a single image pixel 190 (eg, Image signal (eg, difference) of the image pixel's core. For example, a dot filter can operate on a standard red value, a standard green value, and a standard blue value for each image pixel. To perform a point filter operation on the sRGB data, the processing circuit 18 can calculate the original (original) luminance signal using red image data, white image data, and blue image data (eg, image data prior to application of the CCM). The original luminance signal can be a linear combination of white image data, red image data, and blue image data (eg, weighted sum). White image data can be weighted more heavily in linear combinations than red and blue image data, as needed. Processing circuitry 18 may calculate the implied luminance signal as a linear combination of standard red, standard green, and standard blue image data (e.g., after applying CCM to the image material). The weights used in calculating the linear combination of the suggested luminance signals can be substantially similar to the weights used to calculate the original luminance signals, as desired. These weights can be adjusted to modify the "strength" of the point filter (eg, the degree of point filter transformation or correction of sRGB data).

The processing circuit 18 can, in the simplest case, divide the original luminance signal by the implied The luminance signal is used to generate a scaling value (eg, a scaling factor to be applied to the color corrected image value). The scaling factor can include the numerator and the denominator as needed. The numerator and/or denominator of the scaled value may comprise a weighted sum of the original luminance signal and the implied luminance signal. The scaling value can include an adjustable weighting parameter that can be varied to adjust the intensity of the point filter (eg, the weighting parameters can be varied to adjust the intensity of the point filter from zero to full intensity). To apply the point filter to the sRGB data (eg, to standard red, green, and blue image data), processing circuitry 18 may multiply the sRGB data by a scaling value to produce corrected sRGB data. For example, processing circuitry 18 may multiply standard red image data by a scaling value, multiply standard green image data by a scaling value, and the like. The corrected sRGB data may have approximately preserved hue and chroma channels from before applying the point filter (eg, after converting the corrected sRGB data to the LCH space), as desired. The corrected sRGB data may have improved noise and/or sharpness due to the fidelity of the inheritance of the white image signal.

In the simplest case, the original luminance signal can be approximated by white image data. 8 shows a flow diagram (as an example) that may be performed by processing circuitry 18 to apply a point filter (in the simplest case) to sRGB data after applying the CCM to the red, white, and blue image material. Processing circuitry 18 may apply a point filter to the sRGB data, for example, for each image pixel 190 in image pixel array 200. The steps of FIG. 8 may be performed, for example, as part of step 108 of FIG.

At step 130, processing circuitry 18 may generate a suggested luminance value (eg, a luminance value in LCH space) for a given image pixel 190 by combining red, green, and blue image data (eg, after applying CCM). The suggested brightness value can be calculated, for example, as a linear combination of red, green, and blue image data.

At step 132, processing circuit 18 may generate a scaling value by dividing the white image value by the implied luminance value. The scaling factor can be generated by dividing the white image value by the weighted sum of the implied luminance value and the white image value, as desired. The scaling factor can include an adjustable weighting parameter that can be varied to adjust the strength of the point filter (eg, the weighting parameters can be varied continuously to Adjust the intensity of the point filter from zero to full intensity). The scaling value can be, for example, an operator that operates on sRGB data.

At step 134, processing circuitry 18 may multiply the sRGB data by a scaling value to produce corrected sRGB data (eg, corrected standard red, green, and blue image data). For example, processing circuitry 18 may multiply standard red image data by a scaling value, multiply standard green image data by a scaling value, and the like. The corrected sRGB data can be provided to the image display as needed. The corrected sRGB data may have improved noise and/or sharpness compared to the sRGB data prior to applying the point filter.

The examples of Figures 6 through 8 are merely illustrative. Any desired color filter can be used in conjunction with the white filter shown in Figures 3 through 5 to obtain a color image signal. Any combination of desired color filters (eg, red filter, green filter, cyan filter, infrared filter, ultraviolet filter, blue filter, yellow filter, magenta filter, purple filter, etc.) can be used. Any combination). Any other suitable three-dimensional space can be used to perform the point filter operation, as needed.

Any number of image pixel arrays 200 formed on any number of image sensors 16 can be used to capture images, as desired. Each image pixel array used can be used, for example, for image signals of different colors. For example, the first image pixel array may have a transparent filter for generating a white image signal, the second image pixel array may have a red filter for generating a red image signal, and the third image pixel array may have a A blue filter that produces a blue image signal. Image signals from each of these arrays may be chroma-de-mosaic and/or operated using dot filters. Each image pixel array can be formed on a different image sensor such as image sensor 16 in device 10 (eg, multiple image sensors 16 can be formed in device 10). This embodiment may, for example, allow for shorter camera focal lengths and thinner camera modules.

Figure 9 shows, in simplified form, a typical processor system 300, such as a digital camera, that includes an imaging device 2000 (e.g., using a transparent color filter such as Figures 1 through 8 and for The imaging device 2000 of the imaging sensor 16 of the technique described above. Processor system 300 is an illustrative system having digital circuitry that can include imaging device 2000. Without limitation, the system may include computer systems, static or video camera systems, scanners, machine vision, vehicle navigation, video telephony, surveillance systems, autofocus systems, star tracker systems, motion detection systems, imaging Stabilizing systems, and other systems that use imaging devices.

The processor system 300 generally includes a lens 396 for focusing an image onto the pixel array 200 of the device 2000 when the shutter release button 397 is pressed; a central processing unit (CPU) 395, such as a microprocessor, which controls the camera and a Or multiple video stream functions, via bus 393, with one or more inputs/outputs (I/O (device 391 communication. Imaging device 2000 also communicates with CPU 395 via bus 393. System 300 also includes random access memory) Body (RAM) 392, and may include removable memory 394, such as flash memory, which also communicates with CPU 395 via bus 393. Imaging device 2000 may or may not have a single integrated circuit The memory on the wafer is combined with the CPU. Although the bus 393 is illustrated as a single bus, it can be used to interconnect one or more bus bars or bridges or other communication paths for system components.

Various embodiments have been described which illustrate image sensing with a transparent image pixel filter and image processing techniques (eg, chroma demosaicing, application point filters, etc.) for reducing noise in image signals generated by image signals. Device.

An image sensor may have an array of image sensor pixels, the image sensor pixels including red image pixels that generate red image signals in response to red light, and blue image pixels that generate blue image signals in response to blue light. And transparent image sensor pixels that produce white image signals in response to at least red, green, and blue light (eg, white light). The image pixels may be arranged in an array of pixel unit cells, each pixel unit unit comprising a plurality of image pixels of different colors. The image sensor can be coupled to a processing circuit that performs filtering operations on the red, blue, and white image signals to increase noise associated with the red, blue, and white image signals Related. The processing circuit can perform a filtering operation for a given image pixel, for example, by generating a weighted sum of image signals produced by at least 25 image pixels in the image pixel array. The weighted sum may include an adjustable weight (eg, based on the weight of the observed image feature adjustment). A weighted sum can be generated for image signals captured during multiple time frames or captured from multiple image sensors. By generating a weighted sum of multiple time frames, the processing circuitry can reduce the size of the core of the image pixels while successfully reducing image signal noise.

This example is merely illustrative. In general, an array of image sensor pixels can include image sensor pixels of any desired color (eg, image sensor pixels that are responsive to light of any color). For example, the array of image sensor pixels may include a first group of image sensor pixels that respond to light of the first color, and a second group of image sensor pixels that respond to light of the second color. And a third group of image sensor pixels responsive to light of a third color (eg, red, blue, and white). The first image signal can have a first spectral response level (eg, an integrated signal power level as a function of the frequency of the light received by the first group of image sensor pixels), and the second image signal can have a second spectrum The response level (eg, the integrated signal power level as a function of the frequency of the light received by the second group of image sensor pixels), and the third image signal may have a third spectral response level (eg, as a sense of image) The integrated signal power level as a function of the frequency of the light received by the third group of detector pixels). The third image signal may have a spectral response level greater than the first spectral response level and the second spectral response level (eg, the third spectral response level may be greater than 75 percent of the sum of the first spectral response level and the second spectral response level) ). In other words, the third image signal can be captured in response to an optical frequency that is wider than the first image signal and the second image signal.

The processing circuit may use the first image signal, the second image signal, and the third image signal to generate an estimated luminance value (eg, a luminance value in the LCH space). The processing circuit can convert the first image signal, the second image signal, and the third image signal into the derived three-color space (eg, linear sRGB space, CIE space, XYZ space, Bayer space, etc.) The transformed first image signal, the second image signal, and the third image signal are generated. The processing circuit can generate the transformed first image signal, the second image signal, and the third image signal, for example, by performing a linear combination on the first image signal, the second image signal, and the third image signal. The processing circuit can generate the derived luminance value (eg, the luminance value in the LCH space) by combining the transformed first image signal, the transformed second image signal, and the transformed third image signal. The processing circuit compares the derived luminance value with the estimated luminance value, and modifies the transformed first image signal, the transformed second image signal, and the transformed third image signal such that the derived luminance value is close to the estimated The brightness value (eg, such that the derived brightness value fully matches the estimated brightness value).

The processing circuit may use an image sensor having a processing circuit to process image data including a first image signal of a first color, a second image signal of a second color different from the first color, and a white image signal. The processing circuit can use the white image signal to produce a third image signal that is different from the first color and the third color of the second color. The processing circuit may combine the first image signal, the second image signal, and the third image signal to form the derived brightness value, and calculate the estimated brightness value from the first color image signal, the second color image signal, and the white image signal. . The processing circuit can form the derived luminance value by combining the white image signal with the first image signal, the second image signal, and the third image signal.

The processing circuit can modify the first image signal, the second image signal, and the third image signal using the derived luminance value and the estimated luminance value. For example, the processing circuit may calculate the calibration value based on the derived luminance value and the estimated luminance value, and may multiply the first image signal, the second image signal, and the third image signal by the generated calibration value. . The processing circuit may combine the first image signal, the second image signal, and the third image signal to calculate a linear combination of the first image signal, the second image signal, and the third image signal by using a weighting factor to form the derived luminance value.

The processing circuitry may perform infinite impulse response (IIR) filtering on the captured image signal as needed. The processing circuit can be adjusted by the characteristics of the image signal captured based on the image pixel 190 A filter applied to the captured image signal (e.g., a filter as described in connection with Figures 6-8) performs IIR filtering. Performing IIR filtering increases the efficiency with which the processing circuitry processes the captured image signals.

The processing circuit can perform white balance operations on the red, blue, and white image signals as needed. The processing circuitry can apply a color correction matrix to the white image signal to extract image signals of different colors, such as green image signals, from each of the white image signals. The processing circuit can combine the red image signal, the blue image signal, the green image signal, and the white image signal to form a luminance value (eg, by calculating a linear combination or weighted sum of red, blue, green, and white image signals). The processing circuit divides the white image signal by the luminance value to produce a scaling value. The processing circuitry can modify the red, green, and blue image signals by multiplying the red, green, and blue image signals by a scaling value. The scaling value acts as a point filter when operating on red, green, and blue image signals. Image pixels of any color can be combined with white image signals as needed. The processing circuitry can perform such operations on image signals from a plurality of image pixel arrays, image pixel arrays on a plurality of image sensors, and/or image signals captured during a plurality of time frames, as desired.

Transparent image pixels and associated filtering techniques can be implemented in a system that also includes a central processing unit, a memory, an input-output circuit, and an imaging device, the imaging device further including a pixel array for focusing light to A lens on a pixel array, and a data conversion circuit.

According to an embodiment, an imaging system is provided, comprising: an image sensor having an image sensor pixel array, wherein the image sensor pixel array comprises a red image configured to generate a red image signal in response to red light a sensor pixel, a blue image sensor pixel configured to generate a blue image signal in response to blue light, and a transparent image sensor pixel configured to generate a white image signal in response to at least red, green, and blue light And a processing circuit configured to perform a filtering operation on the red image signal, the blue image signal, and the white image signal, which is associated with noise associated with the red image signal, the blue image signal, and the white image signal.

According to another embodiment, the processing circuit is configured to be associated with a red image signal, a blue image The noise associated with the signal and the white image signal is increased to greater than 70% of all noise associated with the red, blue, and white image signals.

In accordance with another embodiment, the processing circuit is configured to perform a filtering operation by generating a weighted sum of image signals produced by at least 25 image sensor pixels for each image sensor pixel of a given color.

According to another embodiment, the processing circuit is configured to perform a white balance operation on the red image signal, the blue image signal, and the white image signal.

In accordance with another embodiment, the processing circuit is configured to apply a color correction matrix to the white image signal, wherein the color correction matrix extracts the green image signal from each of the white image signals.

In accordance with another embodiment, the image sensor further includes an additional image sensor pixel array.

In accordance with another embodiment, the imaging system further includes an additional image sensor having at least one image sensor pixel array.

According to an embodiment, there is provided a method for processing image data using an image sensor having a processing circuit, the image data comprising a first image signal of a first color, a second image signal of a second color different from the first color, And the white image signal, the method comprising: using the processing circuit, using the white image signal to generate a third image signal different from the first color and the third color of the second color; and combining the first image signal and the second image by using the processing circuit The signal and the third image signal are used to form the derived brightness value; the processing circuit is used to calculate the estimated brightness value based on the first image signal, the second image signal, and the white image signal; and the processing circuit is used to use the derived brightness value and the The estimated brightness value modifies the first image signal, the second image signal, and the third image signal.

According to another embodiment, the method further includes calculating, by the processing circuit, a scaling value based on the derived luminance value and the estimated luminance value, wherein modifying the first image signal, the second image signal, and the third image signal includes The image signal, the second image signal, and the third image signal are multiplied by the generated calibration value.

According to another embodiment, combining the first image signal, the second image signal, and the third image signal to form the derived luminance value includes calculating a linear combination of the first image signal, the second image signal, and the third image signal using a weighting factor.

According to another embodiment, combining the first image signal, the second image signal, and the third image signal to form the derived luminance value further comprises combining the white image signal with the first image signal, the second image signal, and the third image signal The derived brightness value is formed.

According to another embodiment, generating the third image signal of the third color using the white image signal comprises extracting the third image signal from the white image signal using the color correction matrix.

According to another embodiment, the first image signal includes a red image signal, the second image signal includes a blue image signal, and the third image signal includes a green image signal.

According to another embodiment, the first image signal includes a red image signal, the second image signal includes a green image signal, and the third image signal includes a blue image signal.

According to another embodiment, the first image signal comprises a blue image signal, the second image signal comprises a green image signal, and the third image signal comprises a red image signal.

According to an embodiment, a system is provided, comprising: a central processing unit, a memory, an input-output circuit, and an imaging device, wherein the imaging device comprises: a pixel array; a lens that focuses the image on the pixel array; and an image pixel array An image sensor, wherein the image pixel array includes red image pixels configured to generate red image signals in response to red light, blue image pixels configured to generate blue image signals in response to blue light, and configured to a transparent image pixel responsive to at least red, green, and blue light to produce a white image signal; and processing circuitry configured to produce a linear combination of the white image signal and at least the red image signal, the blue image signal, and the green image signal The scaling value of the ratio, and the red image signal, the blue image signal, and the green image signal are modified using the generated calibration value.

In accordance with another embodiment, the image pixel array includes a plurality of pixel unit cells, each pixel unit including one of a red image pixel, one of a blue image pixel, and a transparent image pixel.

In accordance with another embodiment, the processing circuit is further configured to perform filtering operations on the red image signal, the blue image signal, and the white image signal to increase noise correlation associated with the red image signal, the blue image signal, and the white image signal.

In accordance with another embodiment, the processing circuit is further configured to generate a difference between the white image signal and the red image signal and between the white image signal and the blue image signal.

In accordance with another embodiment, the processing circuit is further configured to perform a filtering operation on the red image signal, the blue image signal, and the white image signal using the generated difference.

According to an embodiment, a method for processing an image signal using an image sensor is provided, wherein the image sensor includes an image sensor pixel array and a processing circuit, and wherein the image sensor pixel includes a red image sensor pixel, The blue image sensor pixel and the transparent image sensor pixel, and the method comprises: using the red image sensor pixel to generate a red image signal in response to the red light; using the blue image sensor pixel to generate the light in response to the blue light a blue image signal; a transparent image sensor pixel is used to generate a white image signal in response to at least red, green, and blue light; and a processing circuit is used to perform a filtering operation on the red image signal, the blue image signal, and the white image signal. Add noise related to red image signals, blue image signals, and white image signals.

According to another embodiment, performing the filtering operation includes increasing the noise correlation associated with the red image signal, the blue image signal, and the white image signal to be greater than all of the noise associated with the red image signal, the blue image signal, and the white image signal. 70 percent.

In accordance with another embodiment, performing the filtering operation includes generating a weighted sum of image signals produced by at least 25 image sensor pixels for each image sensor pixel of a given color.

According to an embodiment, a method for processing an image signal using an image sensor is provided, wherein the image sensor includes an image sensor pixel array and a processing circuit, and wherein the image sensor pixel array includes a response to the first color a first group of light image sensor pixels, a second group of image sensor pixels responsive to a second color of light, and a response a third group of image sensor pixels of the third color light, and the method includes: generating a first image signal in response to the first color of the light using the first group of image sensor pixels; utilizing a second group of image sensor pixels, generating a second image signal in response to the second color of light; generating a third group of image sensor pixels in response to at least the first and second colors of light And performing a filtering operation on the first image signal, the second image signal, and the third image signal by using the processing circuit, where the noise associated with the first image signal, the second image signal, and the third image signal is added Information related.

In accordance with another embodiment, the first image signal has a first spectral response level, the second image signal has a second spectral response level, and generating a third image signal in response to the at least first and second colors of light comprises generating greater than A third image signal of a third spectral response level of 75 percent of the sum of the signal response level and the second signal response level.

In accordance with another embodiment, generating the first image signal includes generating a red image signal in response to the red light, generating the second image signal comprising generating a blue image signal in response to the blue light, and generating the third image signal comprising responding to at least red light and blue light generation White image signal.

In accordance with another embodiment, the red image signal has a first spectral response level, the blue image signal has a second spectral response level, and generating the white image signal includes generating a hundred having a greater than a sum of the first spectral response level and the second spectral response level The third spectrum of 75 is the white image signal of the level response.

According to another embodiment, performing a filtering operation on the first image signal, the second image signal, and the white image signal includes performing an infinite impulse response filter on the first image signal, the second image signal, and the third image signal.

According to an embodiment, a method for processing an image signal using an image sensor is provided, wherein the image sensor includes a processing circuit, and a first image sensor pixel that generates a first image signal in response to light of a first color a second group of image sensor pixels that generate a second image signal in response to the light of the second color, and an image sensor pixel that generates a third image signal in response to the light of the third color Three groups, where the first image The signal has a first spectral response, wherein the second image signal has a second spectral response, and wherein the third image signal has a third spectral response greater than the first spectral response and the second spectral response, and the method includes: utilizing a processing circuit, Generating the estimated brightness value by using the first image signal, the second image signal, and the third image signal; and converting the first image signal, the second image signal, and the third image signal to the derived three color space by using the processing circuit Generating a transformed first image signal, a transformed second image signal, and a transformed third image signal; and combining the transformed first image signal, second image signal, and third image by using a processing circuit The signal generates the derived luminance value; and the processed first image signal, the second image signal, and the third image signal are modified by the processing circuit such that the derived luminance value is close to the estimated luminance value.

According to another embodiment, converting the first image signal, the second image signal, and the third image signal into the derived three color space includes converting the first image signal, the second image signal, and the third image signal to a standard red color In the green-blue space.

According to another embodiment, generating the transformed first image signal, the second image signal, and the third image signal includes generating a linear combination of the first image signal, the second image signal, and the third image signal.

In accordance with another embodiment, the first image signal includes a red image signal captured by the first group of image sensor pixels in response to the red light, and the second image signal includes a second group response by the image sensor pixels The blue image signal captured by the blue light, the third image sensor signal includes a white image signal captured by the third group of image sensor pixels in response to at least blue light and red light, and generates a first image signal, The linear combination of the second image signal and the third image signal includes the use of red, blue, and white image signals to produce a linear combination.

In accordance with an embodiment, an imaging system is provided that includes an image sensor having an image sensor pixel array, wherein the image sensor pixel array includes a first image that is configured to generate light in response to a first color a first group of image sensor pixels of the signal, image sensor pixels configured to generate a second image signal in response to light of the second color a second group, and a third group of image sensor pixels configured to generate a third image signal in response to light of at least the first color and the second color; and processing circuitry configured to Performing a filtering operation on the first image signal, the second image signal, and the third image signal, wherein the noise associated with the first image signal, the second image signal, and the third image signal is increased.

In accordance with another embodiment, the first image signal has a first spectral response level, the second image signal has a second spectral response level, and the third image signal has a greater than a sum of the first spectral response level and the second spectral response level. The third spectral response level of 75 is.

In accordance with another embodiment, a first group of image sensor pixels includes red image sensor pixels configured to generate a red image signal in response to red light, and a second group of image sensor pixels includes a A blue image sensor pixel configured to generate a blue image signal in response to blue light, and the third group of image sensor pixels includes a transparent image configured to generate a white image signal in response to at least red and blue light Sensor pixel.

In accordance with another embodiment, a first group of image sensor pixels includes red image sensor pixels configured to generate a red image signal in response to red light, and a second group of image sensor pixels includes a A green image sensor pixel configured to generate a green image signal in response to green light, and the third group of image sensor pixels includes a white image signal configured to respond to at least red and green light Transparent image sensor pixels.

The foregoing merely illustrates the principles of the invention that may be practiced in other embodiments.

190‧‧•Image Sensor Pixels/Image Pixels

192‧‧‧unit unit

200‧‧‧pixel array

Claims (35)

An imaging system comprising: an image sensor having an image sensor pixel array, wherein the image sensor pixel array comprises a red image sensor configured to generate a red image signal in response to red light a pixel, a blue image sensor pixel configured to generate a blue image signal in response to blue light, and a transparent image sensor pixel configured to generate a white image signal in response to at least red, green, and blue light; And processing circuitry configured to perform filtering operations on the red image signals, the blue image signals, and the white image signals to increase the red image signals, the blue image signals, and the white image signals Associated with the noise, wherein the processing circuit is configured to perform the filtering operation on the red image signals, the blue image signals, and the white image signals to increase the red image signals, the blue The image signal is associated with the noise associated with the white image signal such that the red image signal, the blue image signal, and the noise fluctuations in the white image signals can be correlated Increased or decreased. The imaging system of claim 1, wherein the processing circuit is configured to increase the correlation of the noise associated with the red image signals, the blue image signals, and the white image signals to be greater than the red 70% of the image signal, the blue image signal, and all of the noise associated with the white image signal. The imaging system of claim 1, wherein the processing circuit is configured to generate a weighted sum of one of image signals produced by at least 25 image sensor pixels for each image sensor pixel of a given color. Perform these filtering operations. The imaging system of claim 1, wherein the processing circuit is configured to perform a white balance operation on the red image signals, the blue image signals, and the white image signals. An imaging system according to claim 4, wherein the processing circuit is configured to correct a color A matrix is applied to the white image signal, wherein the color correction matrix extracts a green image signal from each white image signal. The imaging system of claim 1, wherein the image sensor further comprises: an additional image sensor pixel array. The imaging system of claim 1, wherein the imaging system further comprises: an additional image sensor having one of the at least one image sensor pixel array. A method for processing image data using an image sensor having a processing circuit, the image data comprising a first image signal of a first color, a second image signal different from a second color of the first color, and white Image signal, the method comprising: using the processing circuit, using the white image signal to generate a third image signal different from the first color and a third color of the second color; using the processing circuit, combining the processing signals The first image signal, the second image signals, and the third image signals are used to form a derived brightness value; and the processing circuit is used to generate the first image signal, the second image signal, and the white image The signal is used to calculate an estimated brightness value; and the processing circuit is configured to modify the first image signal, the second image signal, and the third image using the derived brightness value and the estimated brightness value signal. The method of claim 8, further comprising: calculating, by the processing circuit, a certain value based on the derived luminance value and the estimated luminance value, wherein the first image signal, the second image signal, and The third image signals include the first image signals, the second image signals, and the third image signals multiplied by the generated calibration values. The method of claim 9, wherein combining the first image signals, the second image signals, and the third image signals to form the derived luminance value comprises using weighting The factor is used to calculate a linear combination of the first image signal, the second image signal, and one of the third image signals. The method of claim 10, wherein combining the first image signals, the second image signals, and the third image signals to form the derived luminance value further comprises the white image signals and the first images The signals, the second image signals, and the third image signals are combined to form the derived luminance values. The method of claim 8, wherein the using the white image signal to generate the third image signal of the third color comprises extracting the third image signals from the white image signal using a color correction matrix. The method of claim 8, wherein the first image signals comprise red image signals, the second image signals comprise blue image signals, and the third image signals comprise green image signals. The method of claim 8, wherein the first image signals comprise red image signals, the second image signals comprise green image signals, and the third image signals comprise blue image signals. The method of claim 8, wherein the first image signals comprise blue image signals, the second image signals comprise green image signals, and the third image signals comprise red image signals. An electronic system for performing image sensing, comprising: a central processing unit; a memory; an input-output circuit; and an imaging device, wherein the imaging device comprises: a pixel array; focusing an image on the pixel array a lens; an image sensor having an image pixel array, wherein the image pixel array The column includes red image pixels configured to generate red image signals in response to red light, blue image pixels configured to generate blue image signals in response to blue light, and configured to respond to at least red, green light And a transparent image pixel that produces a white image signal with a blue light; and a processing circuit configured to generate a green image signal using the white image signal, the red image signal, the blue image signal, and the green image signal Forming an derived luminance value, calculating an estimated luminance value based on the red image signals, the blue image signals, and the green image signals, and modifying the brightness values using the derived luminance values and the estimated luminance values Red image signals, the blue image signals, and the green image signals. The system of claim 16, wherein the image pixel array comprises a plurality of pixel unit cells, each pixel unit unit comprising one of the red image pixels, one of the blue image pixels, and the transparent image Two of the pixels. The system of claim 16, wherein the processing circuit is further configured to perform a filtering operation on the red image signals, the blue image signals, and the white image signals to increase the red image signals, the blue The image signal is associated with the noise associated with the white image signal. The system of claim 18, wherein the processing circuit is further configured to generate a difference between the white image signal and the red image signal and between the white image signal and the blue image signal. The system of claim 19, wherein the processing circuit is further configured to perform the filtering operations on the red image signals, the blue image signals, and the white image signals using the differences generated. A method for processing an image signal by using an image sensor, wherein the image sensor includes an image sensor pixel array and a processing circuit, and wherein the image sensor pixels include a red image sensor pixel and a blue image Sensor pixel and transparent shadow For example, the method includes: using the red image sensor pixels to generate a red image signal in response to the red light; using the blue image sensor pixels to generate a blue image signal in response to the blue light; The transparent image sensor pixels generate white image signals in response to at least red, green, and blue light; and the processing circuit performs the red image signals, the blue image signals, and the white image signals a filtering operation to increase the correlation of the red image signals, the blue image signals, and the white image signals associated with the white image signals, wherein the processing circuit is configured to image the red image signals, the blue image signals, and The white image signal performs the filtering operation to increase associated with the red image signals, the blue image signals, and the white image signals, such that the red image signals, the blue image signals, and the like The noise fluctuations in the image signal can be increased or decreased together in a related manner. The method of claim 21, wherein performing the filtering operation comprises: increasing the correlation of the noise associated with the red image signal, the blue image signal, and the white image signals to be greater than the red image 70% of the signal, the blue image signal, and all of the noise associated with the white image signal. The method of claim 21, wherein performing the filtering comprises: generating a weighted sum of one of the image signals produced by the at least 25 image sensor pixels for each image sensor pixel of a given color. A method for processing an image signal using an image sensor, the image sensor having an image sensor pixel array and a processing circuit, wherein the image sensor pixel array includes image sensing in response to a first color of light a first group of one of the pixels, a second group of image sensor pixels responsive to a second color of light, And responding to a third group of image sensor pixels of a third color of light, the method comprising: utilizing the first group of image sensor pixels to generate a first response to the light of the first color An image signal; the second group of image sensor pixels is used to generate a second image signal in response to the second color of light; and the third group of image sensor pixels is used to respond to at least the first a third image signal is generated by the light of the second color and the light of the second color; and the filtering operation is performed on the first image signal, the second image signal, and the third image signal by using the processing circuit, Correlating the first image signal, the second image signal, and the noise associated with the third image signal, wherein generating the first image signal includes generating a red image signal in response to the red light, generating the second The image signal includes generating a blue image signal in response to the blue light, and generating the third image signal includes generating the third image signal in response to at least red light and blue light, the red image signal having a first spectral response level The blue image signals have a second spectral response level, and generating the third image signals includes: generating one of 75 percent greater than the sum of the first signal response level and the second signal response level The third image signal of the three-spectrum response level. The method of claim 24, wherein the first image signals have a first spectral response level, wherein the second image signals have a second spectral response level, and wherein the at least the first color and the second The generating of the third image signal by the light of the color comprises: generating the third image signal having a third spectral response level greater than one of the first signal response level and the second signal response level and 75% of the second signal response level. The method of claim 24, wherein generating the third image signal comprises responding to Having less red and blue light to produce a white image signal and wherein generating the white image signal comprises: generating the third spectral response level having a greater than 75% of the sum of the first spectral response level and the second spectral response level Wait for white image signals. The method of claim 24, wherein performing filtering operations on the first image signals, the second image signals, and the white image signals comprises: the first image signals, the second image signals, and the like The third image signal performs an infinite impulse response filtering. A method for processing an image signal using an image sensor, wherein the image sensor includes a processing circuit, a first group of image sensor pixels that generate a first image signal in response to light of a first color, a second group of image sensor pixels that generate a second image signal in response to a second color of light, and one of image sensor pixels that generate a third image signal in response to a third color of light a third group, wherein the first image signals have a first spectral response, wherein the second image signals have a second spectral response, and wherein the third image signals have greater than the first spectral response and the The second spectral response is a third spectral response, the method comprising: utilizing the processing circuit to generate an estimated luminance value using the first image signal, the second image signal, and the third image signals; The processing circuit generates the transformed first image signal by converting the first image signal, the second image signals, and the third image signals into an derived three color space, Transforming the second image signal and the transformed third image signal; using the processing circuit, by combining the transformed first image signal, the transformed second image signal, and the transformed third image The image signal produces an derived luminance value; and Using the processing circuit, the transformed first image signal, the transformed second image signal, and the transformed third image signal are modified such that the derived luminance value is close to the estimated luminance value. The method of claim 28, wherein converting the first image signal, the second image signal, and the third image signals into the derived three color space comprises: the first image signals, the first image signals, and the like The second image signal and the third image signals are transformed into a standard red-green-blue space. The method of claim 28, wherein generating the transformed first image signals, the transformed second image signals, and the transformed third image signals comprises: generating the first image signals, and the like The second image signal and one of the third image signals are linearly combined. The method of claim 30, wherein the first image signals comprise red image signals captured by the first group of image sensor pixels in response to red light, wherein the second image signals comprise image sensing The second group of pixels of the pixels are captured in response to the blue light, wherein the third image sensor signals are captured by the third group of image sensor pixels in response to at least blue and red light a white image signal, wherein the linear combination of the first image signal, the second image signal, and the third image signal comprises: using the red image signal, the blue image signal, and the white The image signal produces this linear combination. An imaging system comprising: an image sensor having an image sensor pixel array, wherein the image sensor pixel array includes a first image signal configured to generate light in response to a first color a first group of image sensor pixels, a second group of image sensor pixels configured to generate a second image signal in response to a second color of light, and configured to respond in response to At least the first color and the second color of light a third group of image sensor pixels that generate a third image signal; and processing circuitry configured to perform filtering on the first image signals, the second image signals, and the third image signals The operation is related to the noise associated with the first image signal, the second image signals, and the third image signals, wherein the processing circuit is further configured to apply a color correction matrix to the And a third image signal, wherein the color correction matrix extracts a green image signal from the third image signals. The imaging system of claim 32, wherein the first image signals have a first spectral response level, wherein the second image signals have a second spectral response level, and wherein the third image signals have greater than the first A spectral response level is one of the second spectral response levels and one of the 75% of the third spectral response levels. The imaging system of claim 32, wherein the first group of image sensor pixels comprises red image sensor pixels configured to generate a red image signal in response to red light, wherein the image sensor pixels are The second group includes blue image sensor pixels configured to generate a blue image signal in response to blue light, and wherein the third group of image sensor pixels includes configured to respond to at least red and blue light And a transparent image sensor pixel that produces a white image signal. The imaging system of claim 32, wherein the first group of image sensor pixels comprises red image sensor pixels configured to generate a red image signal in response to red light, wherein the image sensor pixels are The second group includes green image sensor pixels configured to generate a green image signal in response to green light, and wherein the third group of image sensor pixels includes a configuration configured to respond to at least red light and A transparent image sensor pixel that produces a white image signal in green light.