TWI389052B - System and method of generating color correction matrix for an image sensor - Google Patents

Description

System and method for generating color correction array of image sensor

The present invention relates to image sensors, and more particularly to a system and method for generating a color correction matrix for an image sensor.

In order to achieve high true color, the image sensor must behave like a human eye, such as the Commission Internationale de Commission. Or the color matching function (CMF) specified by International Commission on Illumination (CIE) is used as a standard color response. A color matching function (CMF) can quantize each wavelength color of a human visible frequency band (eg, 380-750 nanometers (nm)) by a set of color components (eg, R, G, B (red, green, blue)). Each set of color components defines a color space that can cover portions of the human eye that are capable of perceiving color (eg, sRGB or NTSC), or all of the perceived color (eg, XYZ).

Colors can be represented by a point in the color space, and each color can be mapped to a different color space. In fact, the image sensor's color matching function (CMF) usually deviates from the human eye response. Therefore, a fitting algorithm must be used to obtain a color correction matrix (CCM) to achieve the minimum color perception error.

Another important technique for correct color perception is the white balance, which equalizes the R, G, B (red, green, blue) color channels to restore the ideal white color of non-ideal image sensors. response.

Conventional methods for color correction array (CCM) and white balance are generally based on system level measurements using a color chart, such as the Macbeth ColoChecker 24-color color correction card. White or gray color patches are used to achieve the white balance of the image sensor, and the image sensor's R, G, B (red, green, blue) output is compared to the human eye response. Next, a fitting technique is used to obtain a color correction array (CCM).

One of the disadvantages of conventional system-level measurement systems is their complex calibration and maintenance procedures, and considerable effort must be taken to prevent errors from physical measurements.

Since conventional measurement systems and methods cannot provide color correction array (CCM) and white balance efficiently and accurately, there is a need to propose a novel mechanism for simplifying and improving color correction array (CCM) and white balance programs, and enhancing them. Accuracy.

In view of the foregoing, it is an object of the present invention to provide a system and method for generating a color correction array (CCM) based on a quantum efficiency spectrum of an image sensor. Thereby, the accuracy of the color correction array (CCM) can be improved and the error of the physical measurement can be avoided.

In accordance with an embodiment of the invention, the quantum efficiency (QE) spectrum of an image sensor illuminated by a physical light source is first measured. Then, according to the quantum efficiency (QE) spectrum and the preset reference data, the color value of the image sensor and the color value of the preset color space are determined. In an embodiment, the preset reference material includes a preset color matching function (CMF), a power spectral density (PSD) of the luminance, and a reflectance spectrum of the preset color correction patch. Finally, a fitting calculation is performed on the color value of the image sensor and the color value of the preset color space to generate a color correction array (CCM) of the image sensor. In an embodiment, the color correction array (CCM) is further converted to other color spaces.

The block diagram of the first diagram shows a system and method for generating a color correction array (CCM) of an image sensor (eg, a complementary metal oxide semiconductor (CMOS) image sensor) to reduce color perception. error.

In the present embodiment, the image sensor 10 is a non-ideal device and is susceptible to perceptual errors. The image sensor 10 is illuminated by a solid light source 11. The light source 11 of the present embodiment is a wavelength-adjustable light source that can scan a wavelength of a human visible frequency band (e.g., 380-750 nanometers (nm)). A quantum efficiency (QE) measurement device 12 is used to generate a quantum efficiency spectrum of the image sensor 10. The quantum efficiency (QE) measuring device 12 can include a power meter or a color meter for measuring the number of photons entering the pixels of the image sensor 10 and collecting signal electrons from the same pixel. number. In the present embodiment, the ratio of the latter (i.e., the number of electrons) to the former (i.e., the number of photons) is defined as the quantum efficiency (QE) of the pixel. The above measurement is repeated for each wavelength, and thus a quantum efficiency spectrum can be obtained, for example, a spectrogram depicting the quantum efficiency (QE) of the R, G, B (red, green, blue) pixels for each wavelength.

With continued reference to the first figure, computing unit 13 receives the quantum efficiency spectrum (from quantum efficiency measuring device 12) and some preset references for calculating the color value and ideal of image sensor 10 (standard) ) color value. These preset references are usually available from standard organizations or collected from known sources. In a preferred embodiment, these preset references include, but are not limited to, the International Commission on Illumination (CIE) standard XYZ color matching function (CMF), luminance power spectral density (PSD), and standards. The reflectance of the color correction film. Based on the quantum efficiency spectrum and these predetermined references, the computing unit 13 produces color values (e.g., R, G, B) and ideal color values for the image sensor 10.

The ideal color value and (non-ideal) image sensor 10 color values are then fed to a fitting unit 14 that includes a fitting algorithm to generate a color correction array (CCM) of the image sensor 10, whereby Reduce color perception errors. A fitting algorithm for generating a color correction array (CCM) can be implemented using conventional techniques, such as, for example, "Method and apparatus for determining a color correction matrix by minimizing a color difference maximum or average value", for example, U.S. Patent No. 7,265,781. ). The resulting color correction array (CCM) can be further converted to other color spaces by conversion unit 15 if desired. For example, a color correction array (CCM) can be converted to the sRGB color space, which is commonly used in the consumer electronics industry.

In an embodiment, the computing unit 13, the fitting unit 14, and the converting unit 15 can be implemented using a general purpose computer that calculates the output by programming. In another embodiment, the units 13, 14, 15 can also be implemented using separate circuits or in conjunction with a firmware program.

According to one of the features of embodiments of the present invention, the vast majority of computing unit 13 does not include physical devices used in conventional systems, methods, such as physical color correction cards. Thereby, the embodiment of the invention can not only reduce the overall cost, but also improve the accuracy of the color correction array (CCM) without being affected by physical measurement errors (such as old or uncorrected physical devices). In other words, in the embodiment of the present invention, the accuracy of the color correction array (CCM) is determined by the quantum efficiency measuring device 12 of the device level. Furthermore, the acquisition of preset references is very simple and economical. Conversely, traditional systems use system level measurements to produce a color correction array (CCM) that requires complex corrections and maintenance that would otherwise cause errors in the physical measurements.

The flow chart of the second figure shows the detailed operation steps and data flow of the first figure. In step 1, the quantum efficiency (QE) spectrum of the R, G, and B pixels of the image sensor 10 (first image) is measured. The quantum efficiency spectrum QE(λ) can be expressed as follows:

Where λ represents the wavelength, e# represents the number of electrons, and P# represents the number of photons.

Since the quantum efficiency (QE) is normalized by the number of photons, and the color matching function (CMF) of the image sensor 10 is normalized by the optical power, conversion between the two is required. Wherein, in step 2a, the quantum efficiency (QE) spectrum is converted to a power spectral response PSR(λ) (ie, equivalent to the CMF of the image sensor) as follows:

Where h represents the Planck constant and c represents the speed of light in vacuum.

Next, in step 2b, a standard (preset or known) color matching function (CMF) is acquired. In this embodiment, a standard XYZ color matching function (CMF) is obtained from the International Commission on Illumination (CIE), which describes the R, G, B power spectral response (PSR) of a typical human eye.

In step 3, a power spectral density (PSD) of lightness (eg, white light brightness) is acquired. For example, a white light source of 3200K tungsten or D65. In this embodiment, the power spectral density (PSD) of the lightness is defined by a preset data file. In this way, there are no longer problems such as source attenuation and measurement errors that occur in conventional systems/methods. Furthermore, the type of light source in this embodiment is also not limited. This embodiment can obtain standard or conventional light sources from any reliable database or pre-executed measurements.

According to the (step 3) power spectral density (PSD) of the lightness and (step 2a) the R, G, B power spectral response (PSR) of the image sensor 10, the image sensor 10 can be determined in step 4a. White balance factor. In this embodiment, the R, G, and B values generated by the image sensor 10 for the white light luminance reaction can be obtained by overlapping integration of the PSD and the PSR. The calculation interval for this integral is in the range of human visible frequency bands (eg 380-750 nanometers (nm)):

R / G / B =∫ PSD (λ)‧ PSR (λ) d λ

The normalization is performed by the maximum color value of each color channel, and the obtained normal gain value can be used as the white balance coefficient of the image sensor 10. For example, suppose G is the maximum value, and R and B are 0.7 and 0.8, respectively. Therefore, the white balance coefficient of the R channel is 1/0.7, and the white balance coefficient of the B channel is 1/0.8.

On the other hand, in step 4b, the white balance of the standard XYZ color matching function (CMF) is determined according to (step 3) the power spectral density (PSD) of the lightness and (step 2b) the standard XYZ color matching function (CMF). The coefficient, the details of which are similar to step 4a.

Next, in step 5a, the (step 4a) white balance coefficient is applied (or multiplied) (step 2a) the image sensor power spectral response (PSR) (or CMF), thereby obtaining the white balance of the image sensor 10. White balanced color matching function (CMF). The white balance CMF obtained in this embodiment (or conventional) is still unable to match the response of the human eye, and therefore further subsequent processing is required.

On the other hand, in step 5b, the (step 4b) white balance coefficient is applied (step 2b) to the standard XYZ color matching function (CMF), thus obtaining a white balanced color matching function (CMF) of XYZ.

In step 6a, according to (step 3) the power spectral density (PSD) of the light intensity and the standard (preset or known) color correction reflectance spectrum obtained in step 6b, the reflected light of the color correction sheet is determined. Power spectral density (PSD). In this embodiment, the reflected light power spectral density (PSD) can be obtained by overlapping integration of both the above-described lightness PSD and the standard color correction plate reflectance spectrum. The calculation interval for this integral is in the range of human visible frequency bands (eg, 380-750 nanometers (nm)). The color correction film of this embodiment is defined by the reflectance spectrum of the preset data file. As a result, there is no longer any problem with the color correction of the color correction and the accuracy of the color meter appearing in the conventional system/method. Furthermore, the number of color correction sheets in this embodiment is also not limited. Standard or (non-standard) customary physical or non-physical color cards can be obtained by simply adding the relevant reflectance spectrum.

Next, in step 7a, according to (step 6a) the reflected light power spectral density (PSD) of the color correction sheet and (step 5a) the white balance color matching function (CMF) of the image sensor 10 are used to determine (non-ideal) The color value (eg, R, G, B) of the image sensor 10. In this embodiment, the color value of the image sensor 10 can be obtained by overlapping integration of the reflected light power spectral density (PSD) of the color correction sheet and the white balance color matching function (CMF) of the image sensor 10. . The calculation interval for this integral is in the range of human visible frequency bands (eg, 380-750 nanometers (nm)).

On the other hand, in step 7b, according to (step 6a) the reflected light power spectral density (PSD) of the color correction sheet and (step 5b) XYZ white balance color matching function (CMF), used to determine the color of the standard color space. Value (eg R, G, B). In this embodiment, the color value of the standard color space can be obtained by the overlap integration of the power spectral density (PSD) of the color correction chip and the standard white balance color matching function (CMF). The calculation interval for this integral is in the range of human visible frequency bands (eg, 380-750 nanometers (nm)).

Next, (step 7b) the ideal color value and (step 7a) the color value of the image sensor 10 are fitted to calculate a color correction array (CCM) of the image sensor 10 to reduce the color perception error. The fitted calculation criteria are based on human perception, reducing the overall color perception difference of all school color cards. Moreover, in some applications, some school color card customizations can be given different weights. When performing a fitting calculus to produce a color correction array (CCM), conventional techniques can be used, the technical details of which are omitted herein.

The color correction array (CCM) generated in step 8 can be further converted to other color spaces if needed. For example, in step 9a, the Color Correction Array (CCM) is converted to the International Lighting Commission (CIE) XYZ color space, which can cover more human eye perceptible colors. The color correction array (CCM) can be further converted to the sRGB color space by step 9b, which is commonly used in the consumer electronics industry. Conversion from XYZ to sRGB can use a standard 3x3 linear color conversion array (CTM).

The above description is only the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention; all other equivalent changes or modifications which are not departing from the spirit of the invention should be included in the following Within the scope of the patent application.

1-9b. . . step

10. . . Image sensor

11. . . light source

12. . . Quantum efficiency (QE) measuring device

13. . . Computing unit

14. . . Fitting unit

15. . . Conversion unit

The block diagram of the first diagram shows a system and method for generating a color correction array (CCM) of an image sensor in accordance with an embodiment of the present invention.

The flow chart of the second figure shows the detailed operation steps and data flow of the first figure.

1-9b. . . step

Claims (16)

A system for generating a color correction array (CCM) of an image sensor, comprising: a quantum efficiency (QE) measuring device for generating a quantum efficiency (QE) spectrum of an image sensor illuminated by a physical light source; a unit for generating a color value and an ideal color value of the image sensor according to the quantum efficiency (QE) spectrum and a preset reference material; and a fitting unit for generating the image sensor a color correction array (CCM); wherein the preset reference material includes a preset color matching function (CMF), a power spectral density (PSD) of luminance, and a reflectance spectrum of a preset color correction patch, and the calculation unit Performing the following steps: determining a color matching function (CMF) of the image sensor according to the quantum efficiency spectrum; acquiring the preset color matching function (CMF); acquiring the luminance power spectral density (PSD); and according to the luminance power spectrum Density (PSD) and the color matching function (CMF) of the image sensor determine the color balance function (CMF) after white balance; according to the brightness power spectral density (PSD) and the preset color matching function (CMF), After white balance The preset color matching function (CMF); Determining a reflected light power spectral density (PSD) of the color correction sheet according to the brightness power spectral density (PSD) and a reflectance spectrum of the preset color correction sheet; and a spectral density of the reflected light power according to the color correction sheet ( PSD) and the white balance color matching function (CMF), determining the color value of the image sensor; and the reflected light power spectral density (PSD) according to the color correction slice and the preset color matching function after the white balance (CMF), determines the color value of a preset color space. A system for generating a color correction array of an image sensor as described in claim 1, wherein the physical light source is a wavelength-adjustable light source that scans a human visible frequency band. The system for generating a color correction array of an image sensor according to claim 2, wherein the quantum efficiency (QE) measuring device comprises a power meter for measuring the entering the image sensing. The number of photons of the pixel of the device, and the number of signal electrons from the pixel is collected, wherein the ratio of the number of electrons to the number of photons is defined as quantum efficiency (QE). The system for generating a color correction array of an image sensor according to claim 3 of the patent application further repeats the quantum efficiency (QE) measurement for each color channel and each wavelength, thereby accumulating the quantum efficiency spectrum. . A system for generating a color correction array of an image sensor as described in claim 1, wherein the predetermined color matching function (CMF) is a standard XYZ color matching function (CMF) of the International Commission on Illumination (CIE). A system for generating a color correction array of an image sensor according to claim 1, wherein the fitting unit comprises a fitting algorithm to generate the color correction array (CCM) for reducing color perception error. . The system for generating a color correction array of an image sensor as described in claim 1 further includes a conversion unit for converting the color correction array (CCM) to other color spaces. A system for generating a color correction array of an image sensor as described in claim 7 wherein the color correction array (CCM) is converted to an sRGB color space. A method of generating a color correction array (CCM) of an image sensor, comprising: Measuring a quantum efficiency (QE) spectrum of an image sensor illuminated by a physical light source; determining a color value of the image sensor and a preset color space according to the quantum efficiency (QE) spectrum and preset reference data a color value; and a fitting calculation of the color value of the image sensor and the color value of the preset color space to generate a color correction array (CCM) of the image sensor, wherein the preset is The reference material includes a preset color matching function (CMF), a power spectral density (PSD) of the luminance, and a reflectance spectrum of the preset color correction patch, and the determining step of the color value includes the following steps: according to the quantum efficiency spectrum Determining a color matching function (CMF) of the image sensor; acquiring the preset color matching function (CMF); acquiring the luminance power spectral density (PSD); and determining the luminance power spectral density (PSD) and the image sensing Color matching function (CMF) determines the white balance color matching function (CMF); determines the preset color matching after white balance according to the brightness power spectral density (PSD) and the preset color matching function (CMF) Function (CMF) The brightness of the power spectral density (PSD) and the predetermined degree of reflection of the color correction sheet (Reflectance) spectrum of the reflected light determined power spectral density of the color correction sheet (PSD); Determining a color value of the image sensor according to a reflected light power spectral density (PSD) of the school color patch and a color matching function (CMF) after the white balance; and a spectral density of the reflected light power according to the color correction sheet (PSD) And the preset color matching function (CMF) after the white balance determines the color value of a preset color space. A method of producing a color correction array of an image sensor as described in claim 9, wherein the physical light source is a wavelength-adjustable light source that scans a human visible frequency band. A method for generating a color correction array of an image sensor according to claim 10, wherein a power meter is used to measure the number of photons entering a pixel of the image sensor, and collects the number of photons from the image sensor. The number of signal electrons of a pixel, wherein the ratio of the number of electrons to the number of photons is defined as quantum efficiency (QE). The method for generating a color correction array of an image sensor according to claim 11 further repeats the quantum efficiency (QE) measurement for each color channel and each wavelength, thereby accumulating the quantum efficiency spectrum. . A method of generating a color correction array of an image sensor as described in claim 9, wherein the preset color matching function (CMF) is a standard XYZ color matching function (CMF) of the International Commission on Illumination (CIE). A method of generating a color correction array of an image sensor as described in claim 9 wherein said brightness power spectral density (PSD) is defined by a white light source. The method for generating a color correction array of an image sensor according to claim 9 further includes a converting step for converting the color correction array (CCM) to another color space. A method of producing a color correction array of an image sensor as described in claim 15 wherein the color correction array (CCM) is converted to an sRGB color space.