TW201602997A - Color processing system and apparatus - Google Patents

Abstract

A color processing system includes a color interpolation unit coupled to receive color and white (W) signals and accordingly generate interpolated white signals and difference signals; a color correction unit configured to correct the difference signals, thereby resulting in corrected color signals; and a sensitivity control unit configured to generate adjusted color signals according to the corrected color signals, the interpolated white signals, and surrounding illumination.

Description

Color processing system and device

The present invention relates to a color processing, and more particularly to a color processing system and apparatus adapted to brightness.

Image sensors, such as complementary metal oxide semiconductor (CMOS) image sensors, can be used to convert optical images into electrical signals. Image sensors are commonly used in a variety of applications, such as mobile phones and cameras. The image sensor not only converts visible light, but also converts infrared light, thus causing discoloring. Therefore, an infrared band-stop filter is usually added before the image sensor to block the infrared light.

Sensitivity is one of the important parameters regarding the performance of image sensors. The sensitivity of a complementary metal oxide semiconductor image sensor is typically limited by the area of the photosensor that makes up the image sensor. As the density of the image sensor increases, its sensitivity decreases.

In the low brightness state, especially in surveillance systems, the aforementioned excess infrared light becomes a necessity, which can be used to increase visibility. U.S. Patent No. 7,239,344, entitled "Camera and device for switching optical filters", which uses a motor to switch two optical filters depending on the brightness. In the day mode, a general IR-cut filter is used, which blocks infrared light to avoid discoloration; in night mode, another optical filter is used to pass infrared light. This mechanism consumes considerable power and reduces the lifetime of the imaging device, such as a camera.

U.S. Patent No. 8,408,821, entitled "Visible and infrared dual mode imaging system", which uses a dual-band filter that passes through the visible and infrared bands. The center is located at 950 nm). In night mode, an infrared light source is required to emit infrared light, which can be captured by the infrared band of the dual band filter. In the day mode, since the infrared light band passes through the infrared light, it causes discoloration.

U.S. Patent No. 7,864,233, entitled "Image photographing device and method", discloses a color filter array comprising a filter that can pass visible light and infrared light, and another filter Through visible light. The difference between the sensing signals of the two filters can be used to calculate the infrared light band to avoid discoloration. This mechanism requires more processing to filter out the infrared light signal, thus reducing the sampling rate.

U.S. Patent No. 8,619,143, entitled "Image sensor including color and infrared pixels", which uses a notch infrared light cut filter to block visible light bands. A transition band between the band and the infrared light band. This mechanism has low sensitivity in low brightness conditions.

U.S. Patent No. 8,508,633, entitled "Image device with color and brightness signal processing", which uses a complementary color of a pipeline (e.g., WR) as a color filter array (CFA) ). Since the spectral response of the color filter varies depending on the material used, it is not easy to obtain or optimize the response. In addition, in this mechanism, white pixel values directly act as luminance signals, thus causing color reproduction errors, especially color saturation.

Therefore, there is a need to propose a novel color processing system to overcome the lack of traditional color processing systems.

In view of the above, one of the objects of embodiments of the present invention is to provide a color processing system and apparatus adapted to brightness so that discoloration can be avoided in a general brightness state, and visibility can be greatly enhanced in a low brightness state.

According to an embodiment of the invention, the color processing system includes a color interpolation unit, a color correction unit, and a sensitivity control unit. The color interpolation unit receives the color light and the white light signal to generate an interpolated white light signal and a difference signal. The color correction unit corrects the difference signal to generate a corrected color signal. The sensitivity control unit generates an adjusted color signal based on the corrected color signal, the interpolated white light signal, and the ambient brightness.

The first figure shows a schematic diagram of a color processing device 100 in accordance with an embodiment of the present invention. As shown, the incident light 11 first passes through a lens 12 that converges or focuses the incident light 11 into transmitted light 13. The transmitted light 13 is then filtered by an infrared cut-off (IR-cut or IR) filter 14, which passes through visible wavelengths (eg, 380-700 nm) and infrared wavelengths (eg, near-infrared light) 700~1000 nm) and block light from other wavelengths. The infrared cut-off filter 15 from the infrared cut filter 14 is then passed through a color and white filter array (abbreviated as a filter array) 16, which contains color filters (eg, red (R) filters, green (G) filter and blue (B) filter) for filtering the corresponding wavelength range of the infrared light cut filter 15 and including a white (W) filter for allowing the infrared light to cut off the filter 15 by. The red (R) filter, the green (G) filter, the blue (B) filter, and the white (W) filter of the filter array 16 may be arranged as shown in the first figure, but are not limited herein. Although the color filter of the present embodiment is exemplified by a red, green, and blue (RGB) color model, other color models such as a cyan (C) magenta (M) yellow (Y) color model may be used.

The second A diagram illustrates the wavelength spectrum of red/green/blue light and white light 17 from the filter array 16. The second B diagram illustrates another wavelength spectrum of red/green/blue light and white light 17, which uses an infrared light cut filter 14 for passing visible wavelengths (eg, 380-700 nm) and groove-shaped infrared wavelengths. Its center is located at 950 nm. The infrared light cut filter 14 of the second B diagram is therefore also referred to as a groove-shaped infrared light filter.

The colored light and white light 17 from the filter array 16 are then converted to a color light signal and a white light signal 19 by a light sensor 181 of an image sensor 18 (e.g., a complementary metal oxide semiconductor image sensor). Finally, the color light signal and white light signal 19 are processed via color processing system 200, which may be executed by a processor, such as an image processor.

The third figure shows a detailed block diagram of the color processing system 200 (second image) of an embodiment of the present invention. In the present embodiment, the color interpolation unit 21 (from the image sensor 18) receives the color light signal and the white light signal 19, thereby generating the interpolated white light signal 22 and the difference signal 23. In the present specification, "difference signal" means the difference between a white light signal and a color light signal. For example, W-R represents the difference signal 23 of the difference between the white light signal and the red light signal. The fourth A diagram shows a detailed block diagram of the color interpolating unit 21 of the third figure. Wherein, the white light signal 191 (corresponding to the light sensor 181 of the white filter) is processed by the white light interpolation sub-unit 211 for corresponding to the non-white filter (for example, red, green or blue filter). An interpolated white light signal 22 is generated. Taking the filter array 16 shown in FIG. 4 as an example, the white light interpolation sub-unit 211 can perform the following processing: W@R5=(W1+W9)/2 W@B2=(W1+W3)/2 W@ G6=(W1+W3+W9+W11)/4

The color light signal 192 (corresponding to the light sensor 181 of the color filter (e.g., red, green, or blue filter) produces the processing of the sub-unit 212 via the difference signal to generate the difference signal 23. Taking the filter array 16 shown in FIG. 4B as an example, the difference signal generation sub-unit 212 can perform the following processing: WR@R5=W@R5-R5 WR@R7=W@R7-R7 WR@G6=(WR @R5+WR@R7)/2 WR@R13=W@R13-R13 WR@R15=W@R15-R15 WR@W9=(WR@R5+WR@R13)/2 WR@B10=(WR@R5 +WR@R7+WR@R13+WR@R15)/4

In this embodiment, the difference signal 23 (eg, WR@G6) between the white light signal and the red light signal located at the non-red pixel (eg, G6) may be within the adjacent difference signal 23 (eg, WR@R5 and WR@R7). Insert it. The above is exemplified by the difference signal 23 between the white light signal and the red light signal, and the difference signal generating sub-unit 212 can follow a similar operation for the processing of the non-red light signal.

Observing the wavelength spectrum of the second A picture, it can be known that the difference signal WR is substantially equal to cyan (C) and does not contain the infrared light signal, the difference signal WG is approximately equal to magenta (M) and does not contain the infrared light signal, and the difference signal WB is approximately equal to yellow. (Y) and does not contain infrared light signals. In other words, since the color light/white light signal 19 has a similar infrared light band, the difference signal 23 obtained from the color interpolating unit 21 has extremely small infrared light components and can be ignored.

The fourth C diagram shows another detailed block diagram of the color interpolating unit 21 of the third figure. Wherein, the white light signal 191 (corresponding to the light sensor 181 of the white filter) is processed by the white light interpolation sub-unit 211 for corresponding to the non-white filter (for example, red, green or blue filter). An interpolated white light signal 22 is generated. The operation of the white light interpolation sub-unit 211 has been described in the fourth A diagram, and the details thereof are therefore omitted.

As shown in FIG. 4C, color light signal 192 interpolates the processing of sub-unit 213 via colored light (e.g., red, green, or blue light) to produce interpolated color optical signal 214. For example, the color light interpolation sub-unit 213 averages adjacent red light signals (for example, red light signals at R5 and R7) at non-red pixels (for example, G6) to obtain an interpolated red light signal. The subtractor 215 then subtracts the interpolated white light signal 22 from the interpolated color optical signal 214, thereby producing a difference signal 23.

See FIG. Third, the color correction unit 24 corrects the difference signal 23, 25 to produce a corrected color signal, red for example, the correction signal R _o, G _o corrected green signal and a blue correction signal B _o. The color correction performed by color correction unit 24 can be expressed as:

The corrected color signals R _o , G _o and B _o can also be expressed as: Where R _V , G _V and B _V represent the visible light components of the red, green and blue signals, respectively.

For the above two expressions, it is assumed that the visible light component of the white light signal is equal to the sum of the visible light components of all the color signals (ie, W _V = R _V + G _V + B _V ), and the colored light/white light signals have the same infrared light. The composition (ie, W _IR =R _IR =G _IR =B _IR ), if the elements C ₁₁ to C ₃₃ of the matrix C are known or available, the elements D ₁₁ to D _{33 of} the matrix D can be obtained by the following formula :

In an embodiment, the matrix D can be decomposed into three matrices as follows: Where matrix A represents a similar CMY to RGB conversion for converting the difference signal to RGB space, and matrix A can be selected to match the spectral response of conventional RGB; and the matrix (R _g , G _g , B _g ) can be different White balance gain of brightness. In an example using an RGBW filter array 16, the matrix A can be:

Referring again to the third diagram, the sensitivity control unit 26 can adapt to the brightness to produce the adjusted color signal 27 based on the corrected color signal 25 and the interpolated white light signal 22. The fifth A diagram shows a detailed block diagram of the sensitivity control unit 26 of the third figure. Among them, the brightness estimator 261 estimates the brightness around the color processing device 100. The white light gain generator 263 generates a gain value 264 based on the estimated brightness 262 of the brightness estimator 261. In general, the lower the surrounding brightness, the larger the gain value 264, and vice versa.

FIG. 5B illustrates a gain curve that displays respective gain values 264 for different luminance measurements, wherein the luminance measurements multiply the integration time (of image sensor 18) by the gain of image sensor 18. Divide by the original (non-interpolated) white light signal average. As shown in FIG. 5B, when the luminance measurement value is lower than the first critical value I ₁ , the gain value 264 is zero; when the luminance measurement value is higher than the second critical value I ₂ (which is greater than the first critical value I ₁ ), the gain value 264 is 壹. 264 gain value zero value I ₁ is gradually increased to a second threshold value I ₂ from the first One critical.

Continuing to refer to FIG. 5A, the interpolated white light signal 22 (by multiplier 265) is multiplied by the gain value 264 and added (by adder 266) to the corrected color signal 25, thereby producing an adjusted color signal 27, such as adjusting red. The signal R _adj , the green signal G _{adj is} adjusted, and the blue signal B _{adj is} adjusted. The sensitivity control unit 26 can perform the following processing: Wherein W _V signal representative of a visible light component of white light, and white light W _IR signal representative of the infrared light component.

According to the sensitivity control unit 26 of the present embodiment, the infrared light component of the white light signal can be adaptively introduced into the adjustment color signal 27 in a low brightness state, thereby improving the overall sensitivity. However, in the general brightness state, the color signal 27 is adjusted. It does not have or contains very little infrared light, so discoloration can be avoided.

The adjusted color signal 27 obtained from the sensitivity control unit 26 can then be further processed, such as gamma correction, which can be implemented using conventional techniques.

The above description is only the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention; all other equivalent changes or modifications which are not departing from the spirit of the invention should be included in the following Within the scope of the patent application.

100‧‧‧Color processing device
11‧‧‧Incoming light
12‧‧‧ lens
13‧‧‧transmitted light
14‧‧‧Infrared cut-off filter
15‧‧‧Infrared cut-off filter
16‧‧‧Filter array
17‧‧‧Color light and white light
18‧‧‧Image Sensor
181‧‧‧Light sensor
19‧‧‧Color optical signal and white light signal
191‧‧‧White light signal
192‧‧‧Color optical signal
200‧‧‧Color Processing System
21‧‧‧Color Interpolation Unit
211‧‧‧White light interpolation subunit
212‧‧‧Differential signal generation subunit
213‧‧‧Color light interpolation subunit
214‧‧‧Interpolated color light signals
215‧‧‧Subtractor
22‧‧‧Insert white light signal
23‧‧‧ bad signal
24‧‧‧Color Correction Unit
25‧‧‧Correct color signal
26‧‧‧Sensitivity Control Unit
261‧‧‧Brightness estimator
262‧‧‧ Estimated brightness
263‧‧‧White Light Gain Generator
264‧‧‧gain value
265‧‧‧Multiplier
266‧‧‧Adder
27‧‧‧Adjust color signal
W‧‧‧Insert white light signal
R‧‧‧Red
G‧‧‧Green
B‧‧‧Blue
WR‧‧‧Signal difference between white light and red light
WG‧‧‧Signal difference between white light and green light
WB‧‧‧Signal difference between white light and blue light
R _o ‧‧‧correct red signal
G _o ‧‧‧Correct green signal
B _o ‧‧‧correct blue signal
R _adj ‧‧‧Adjust red signal
G _adj ‧‧‧Adjust green signal
Adjusting the blue signal B _adj ‧‧‧
I ₁ ‧‧‧first threshold
I ₂ ‧‧‧second threshold

The first figure shows a schematic diagram of a color processing apparatus according to an embodiment of the present invention. Figure 2A illustrates the wavelength spectrum of red/green/blue light and white light from the filter array (first image). The second B diagram illustrates the wavelength spectrum of another red/green/blue light and white light from the groove-shaped infrared light filter. The third figure shows a detailed block diagram of the color processing system (second drawing) of the embodiment of the present invention. Figure 4A shows a detailed block diagram of the color interpolation unit of the third figure. Figure 4B illustrates an array of filters of the first figure. The fourth C diagram shows another detailed block diagram of the color interpolation unit of the third figure. Figure 5A shows a detailed block diagram of the sensitivity control unit of the third figure. Figure 5B illustrates a gain curve that shows the respective gain values for different luminance measurements.

200‧‧‧Color Processing System

19‧‧‧Color optical signal and white light signal

21‧‧‧Color Interpolation Unit

22‧‧‧Insert white light signal

23‧‧‧ bad signal

24‧‧‧Color Correction Unit

25‧‧‧Correct color signal

26‧‧‧Sensitivity Control Unit

27‧‧‧Adjust color signal

W‧‧‧Insert white light signal

W-R‧‧‧Signal difference between white light and red light

W-G‧‧‧Signal difference between white light and green light

W-B‧‧‧Signal difference between white light and blue light

R _o ‧‧‧correct red signal

G _o ‧‧‧Correct green signal

B _o ‧‧‧correct blue signal

R _adj ‧‧‧Adjust red signal

G _adj ‧‧‧Adjust green signal

B _adj ‧‧‧Adjust blue signal

Claims (19)

A color processing system comprising: a color interpolation unit that receives color light and white light signals to generate an interpolated white light signal and a difference signal; a color correction unit that corrects the difference signal to generate a corrected color signal; and a sensitivity The control unit generates an adjusted color signal based on the corrected color signal, the interpolated white light signal, and ambient brightness. The color processing system of claim 1, wherein the color light signal comprises a red light signal, a green light signal, and a blue light signal; and the difference signal comprises a difference signal between white light and red light, and a difference between white light and green light. Signal, white light and blue light difference signal. The color processing system of claim 1, wherein the color interpolating unit comprises: a white light interpolating sub-unit, generating the interpolating white light signal according to the white light signal; and generating a sub-unit according to the difference signal, according to The white light signal and the color light signal are interpolated to generate the difference signal. The color processing system of claim 3, wherein the difference signal generating sub-cell interpolates the adjacent difference signal to generate the difference signal. The color processing system of claim 1, wherein the color interpolation unit comprises: a white light interpolating sub-unit, the interpolating white light signal is generated according to the white light signal; and a color light interpolating sub-unit, according to The color optical signal produces an interpolated color optical signal; and a subtractor that subtracts the interpolated color optical signal to produce the difference signal. The color processing system of claim 1, wherein the sensitivity control unit comprises: a brightness estimator for estimating ambient brightness; and a white light gain generator for generating a gain value according to an output of the brightness estimator, The lower the surrounding brightness, the greater the gain value, and vice versa; a multiplier that multiplies the interpolated white light signal by the gain value; and an adder that adds the output of the multiplier to the corrected color signal, To generate the adjusted color signal. The color processing system according to claim 6, wherein the output of the brightness estimator is a brightness measurement value, which is obtained by multiplying the integration time of the image sensor by the gain of the image sensor, and dividing by The average of the white light signals. According to the color processing system of claim 7, when the brightness measurement value is lower than the first threshold value, the gain value is zero; when the brightness measurement value is higher than the second threshold value, the gain value is壹, wherein the second threshold is greater than the first threshold; and the gain value is gradually increased from a first threshold to zero to a threshold of a second threshold. A color processing device comprising: an infrared light cut filter for filtering a transmitted light, the infrared light cut filter passing through visible light wavelength and infrared light wavelength, and blocking light of other wavelengths, thereby generating an infrared light cutoff Filtering light; a color and white filter array comprising a color filter and a white filter for filtering the infrared light cut filter to generate colored light and white light; an image sensor comprising light perception The detector is configured to convert the color light and the white light into a color light signal and a white light signal respectively; a color interpolation unit receives the color light and the white light signal to generate an interpolated white light signal and a difference signal; and a color correction unit And correcting the difference signal to generate a corrected color signal; and a sensitivity control unit for generating an adjusted color signal based on the corrected color signal, the interpolated white light signal, and ambient brightness. The color processing device of claim 9, wherein the color spectrum of the colored light and the white light is between 380 nm and 1000 nm. The color processing device according to claim 9, wherein the wavelength spectrum of the colored light and the white light comprises a visible light band of 380 to 700 nm and a groove-shaped infrared light band separated from the visible light band. The color processing device according to claim 9, further comprising a lens through which incident light passes to form the transmitted light. The color processing device of claim 9, wherein the color light signal comprises a red light signal, a green light signal, and a blue light signal; and the difference signal comprises a difference signal between white light and red light, and a difference between white light and green light. Signal, white light and blue light difference signal. The color processing device of claim 9, wherein the color interpolating unit comprises: a white light interpolating sub-unit, generating the interpolated white light signal according to the white light signal; and generating a sub-unit according to the difference signal, according to The white light signal and the color light signal are interpolated to generate the difference signal. The color processing device of claim 14, wherein the difference signal generating sub-cell interpolates the adjacent difference signal to generate the difference signal. The color processing device of claim 9, wherein the color interpolation unit comprises: a white light interpolating sub-unit, the interpolating white light signal is generated according to the white light signal; and a color light interpolating sub-unit, according to The color optical signal produces an interpolated color optical signal; and a subtractor that subtracts the interpolated color optical signal to produce the difference signal. The color processing device according to claim 9, wherein the sensitivity control unit comprises: a brightness estimator for estimating ambient brightness; and a white light gain generator for generating a gain value according to an output of the brightness estimator, The lower the surrounding brightness, the greater the gain value, and vice versa; a multiplier that multiplies the interpolated white light signal by the gain value; and an adder that adds the output of the multiplier to the corrected color signal, To generate the adjusted color signal. The color processing device according to claim 17, wherein the output of the brightness estimator is a brightness measurement value obtained by multiplying an integration time of the image sensor by a gain of the image sensor, and dividing by The average of the white light signals. According to the color processing device of claim 18, when the brightness measurement value is lower than the first threshold value, the gain value is zero; when the brightness measurement value is higher than the second threshold value, the gain value is壹, wherein the second threshold is greater than the first threshold; and the gain value is gradually increased from a first threshold to zero to a threshold of a second threshold.