CN103220476B - Image processing apparatus and method - Google Patents

Abstract

An image processing device and method, the image processing device includes a sampling unit, an amplifier, an analog-digital converter and a white balance unit. The sampling unit is used for sampling a pixel to obtain a pixel voltage. The amplifier is used for amplifying the pixel voltage according to an analog gain. The analog-to-digital converter is used for converting the amplified pixel voltage into digital pixel data, and the digital pixel data comprises a plurality of first sub-pixel data corresponding to different colors. The white balance unit is used for respectively carrying out white balance compensation on the first sub-pixel data according to the analog gain and a plurality of white balance gains corresponding to different sub-pixels to obtain a plurality of second sub-pixel data.

Description

Image processing device and method

technical field

The invention relates to an image processing device and method.

Background technique

Existing photosensitive elements include two types: charge coupled device (CCD) and complementary metal oxide semiconductor (complementary metal oxide semiconductor, CMOS). Since the saturation potential (maximum well capacity) of the photosensitive element is usually designed within the working range of the analog-digital converter, when the analog-digital converter adopts the minimum analog gain, the saturation region will be clamped within the working range. Digital image processing of pixel data often results in high luminance color cast or high luminance noise. In general applications, a larger analog gain value is given to clamp the saturation region above the working range of the analog-to-digital converter, so that image artifacts can be avoided, but a larger gain value will sacrifice dynamic range and increase overall noise .

Contents of the invention

The disclosure relates to an image processing device and method, which can improve the dynamic range of an image.

According to a first aspect of the present disclosure, an image processing device is provided, including a sampling unit, an amplifier, an analog-to-digital converter, and a white balance unit. The sampling unit is used for sampling a pixel to obtain a pixel voltage. The amplifier is used for amplifying the pixel voltage according to an analog gain. The analog-to-digital converter is used to convert the amplified pixel voltage into digital pixel data, and the digital pixel data includes a plurality of first sub-pixel data corresponding to different colors. The white balance unit is used for respectively performing white balance compensation on the first sub-pixel data according to the analog gain and a plurality of white balance gains corresponding to different sub-pixels to obtain a plurality of second sub-pixel data.

According to a second aspect of the present disclosure, an image processing method is provided, including the following steps. A pixel voltage is obtained by sampling a pixel. The pixel voltage is amplified according to an analog gain. The amplified pixel voltage is converted into digital pixel data, and the digital pixel data includes a plurality of first sub-pixel data corresponding to different colors. According to the analog gain and a plurality of white balance gains corresponding to different sub-pixels, white balance compensation is performed on the first sub-pixel data respectively to obtain a plurality of second sub-pixel data.

In order to have a better understanding of the above-mentioned and other aspects of the present disclosure, an embodiment is given below, together with the accompanying drawings, for detailed description as follows:

Description of drawings

FIG. 1 is a block diagram of an image processing device according to an embodiment.

FIG. 2 illustrates a flowchart of exposure expansion of a white balance unit according to an embodiment.

FIG. 3 is a schematic diagram of second sub-pixel data versus exposure time/sensitivity according to an embodiment.

[Description of main component symbols]

100: image processing device

105: Automatic Exposure Unit

110: sampling unit

120: Amplifier

130: Analog-to-Digital Converter

140: Black calibration unit

150: Automatic white balance unit

150: White balance unit

170: Color interpolation unit

detailed description

The image processing device and method proposed in the present disclosure can improve the dynamic range of the image and take into account applications with less noise.

The disclosure proposes an image processing device, including a sampling unit, an amplifier, an analog-to-digital converter, and a white balance unit. The sampling unit samples a pixel to obtain a pixel voltage. The amplifier amplifies the pixel voltage according to an analog gain. The analog-to-digital converter converts the amplified pixel voltage into digital pixel data, and the digital pixel data includes a plurality of first sub-pixel data corresponding to different colors. The white balance unit respectively performs white balance compensation on the first sub-pixel data according to the analog gain and a plurality of white balance gains corresponding to different sub-pixels to obtain a plurality of second sub-pixel data.

In addition, in the process of generating the second sub-pixel data, the white balance unit can generate a plurality of third sub-pixel data according to the first sub-pixel data and the white balance gains, and then according to the analog gain and a pixel The maximum well capacity (fullwell level) determines whether to clamp (clamp, also referred to as “clamping”) the third sub-pixel data to generate the second sub-pixel data.

Please refer to FIG. 1 , which shows a block diagram of an image processing device according to an embodiment. The image processing device 100 includes an automatic exposure (autoexposure, AE) unit 105, a sampling unit 110, an amplifier 120, an analog-to-digital converter 130, a black calibration (DigitalBlackLevelCalibration, DBLC) unit 140, an automatic white balance (autowhitebalance, AWB) unit 150 , a white balance unit 160 and a color interpolation (CFA interpolation) unit 170 . Wherein, the sampling unit 110, the amplifier 120 and the analog-to-digital converter 130 are, for example, integrated in an image sensor (image sensor), and the image sensor may also include or be externally coupled to a photosensitive element, such as a charge-coupled device (CCD, Charge-coupled Device) or complementary metal oxide semiconductor (complementary metal oxide semiconductor, CMOS) to provide the image input signal required by the sampling unit 110 . In addition, the black calibration unit 140 , the white balance unit 160 and the color interpolation unit 170 are, for example, integrated in an image processor, but not limited thereto. In addition, the image processor may further include a color correction unit, a gamma correction (GammaCorrection) unit, and an image enhancement (Image Enhancement, also referred to as “enhancement”) unit.

The automatic exposure unit 105 controls the sampling unit 110 to sample a pixel to obtain a pixel voltage. The automatic exposure unit 105 also provides an analog gain Analog_Gain to the amplifier 120 so that the amplifier 120 amplifies the pixel voltage according to the analog gain Analog_Gain. The analog-to-digital converter 130 converts the amplified pixel voltage into digital pixel data, which includes a plurality of first sub-pixel data corresponding to different colors, such as red sub-pixel data R, green sub-pixel data Gr/Gb and Blue sub-pixel data B, etc.

The black calibration unit 140 is coupled between the digital-to-analog converter 130 and the white balance unit 160 , and performs black calibration on the digital pixel data.

The automatic exposure unit 105 provides the analog gain Analog_Gain to the white balance unit 160; meanwhile, the automatic white balance unit 150 also provides multiple white balance gains WB_Gains corresponding to different sub-pixels to the white balance unit 160. The multiple white balance gains WB_Gains include, for example The white balance gain WB_R_Gain corresponding to the red sub-pixel data R, the white balance gain WB_Gr_Gain/WB_Gb_Gain corresponding to the green sub-pixel data Gr/Gb, the white balance gain WB_B_Gain corresponding to the blue sub-pixel data B, and the like. According to the analog gain Analog_Gain provided by the automatic exposure unit 105 and the multiple white balance gains WB_Gains provided by the automatic white balance unit 150, the white balance unit 160 respectively performs white balance compensation on the first sub-pixel data to obtain a plurality of second sub-pixel data. pixel data, and provide a raw image (RAWimage) data including the second sub-pixel data to the white balance unit 160 . The color interpolation unit 170 interpolates the original image data output by the white balance unit 160 to obtain a three primary color image (RGB image) data.

Please refer to FIG. 2 , which shows a flow chart of the exposure expansion of the white balance unit according to an embodiment, which can be applied to the white balance compensation operation performed by the white balance unit 160 shown in FIG. 1 . In step S200 , the white balance unit 160 obtains a plurality of third sub-pixel data according to the first sub-pixel data and the corresponding white balance gains WB_Gains. Preferably, the white balance unit 160 respectively calculates the product of the white balance gain WB_Gains and the first sub-pixel data as the third sub-pixel data. For example, the product of the red sub-pixel data R and the white balance gain WB_R_Gain is the third sub-pixel data R', the product of the green sub-pixel data Gr/Gb and the white balance gain WB_Gr_Gain/WB_Gb_Gain is the third sub-pixel data Gr'/Gb' , and the product of the blue sub-pixel data B and the white balance gain WB_B_Gain is the third sub-pixel data B′.

In step S210, the white balance unit 160 determines whether a maximum well level of the pixel is within a range corresponding to the analog gain Analog_Gain, and the range is defined by the analog gain Analog_Gain and the number of bits of the analog-to-digital converter 130, For example, this range is the range between the digital pixel data when the analog gain Analog_Gain is 1 and the maximum digital pixel data 2n that can be converted by the n-bit ADC 130 . When the maximum well capacity is not within the range, the process ends. In other words, the white balance unit 160 does not clamp the third sub-pixel data, and outputs the third sub-pixel data as the second sub-pixel data.

However, when the maximum well capacity is within the range, steps S220 and S230 are performed sequentially. In step 220 , the white balance unit 160 can calculate a current maximum well capacity according to the maximum well capacity and the analog gain Analog_Gain. Preferably, the white balance unit 160 can calculate the product of the analog gain Analog_Gain and the maximum well capacity as the current maximum well capacity. Furthermore, the white balance unit 160 will limit the current maximum well capacity to no more than 2 ^n-1 , where n is the number of bits of a pixel. In other words, if the product of the analog gain Analog_Gain and the maximum well capacity is greater than 2 ^n-1 , the current maximum well capacity can be set to be substantially equal to 2 ^n-1 .

Next, in step S230 , the white balance unit 160 compares the third sub-pixel data with the current maximum well capacity to determine whether the third sub-pixel data are greater than the current maximum well capacity. When the judgment result is no, the process ends, that is, the white balance unit 160 does not clamp the third sub-pixel data not larger than the current maximum well capacity, and outputs the third sub-pixel data as the corresponding second sub-pixel data. On the contrary, when the judgment result is yes, in step S240 , the white balance unit 160 clamps the third sub-pixel data larger than the current maximum well capacity, for example, may output the current maximum well capacity as the corresponding second sub-pixel data.

More specifically, when the third sub-pixel data R' is greater than the current maximum well capacity, the corresponding second sub-pixel data can be set equal to the current maximum well capacity; otherwise, when the third sub-pixel data R' is not greater than the current maximum well capacity Well capacity, the corresponding second sub-pixel data can be set equal to the third sub-pixel data R′. Similarly, when the three sub-pixel data Gr'/Gb' is greater than the current maximum well capacity, the corresponding second sub-pixel data can be set equal to the current maximum well capacity; otherwise, when the third sub-pixel data Gr'/Gb' is not If it is larger than the current maximum well capacity, the corresponding second sub-pixel data can be set to be equal to the third sub-pixel data Gr'/Gb'. Similarly, when the third sub-pixel data B' is greater than the current maximum well capacity, the corresponding second sub-pixel data can be set equal to the current maximum well capacity; otherwise, when the third sub-pixel data B' is not greater than the current maximum well capacity , then the corresponding second sub-pixel data can be set equal to the third sub-pixel data B′.

To sum up, in this embodiment, the second sub-pixel data will be clamped at the current maximum well capacity. Please refer to FIG. 3 , which shows a schematic diagram of second sub-pixel data versus exposure time/sensitivity according to an embodiment. It can be seen from FIG. 3 that when the image processing device increases the sensitivity (that is, increases the analog gain Analog_Gain), the second sub-pixel data will be clamped at the current maximum well capacity, so it will not be affected by the larger analog gain Analog_Gain. Therefore, high-brightness color shift or high-brightness noise is generated.

The present disclosure also proposes an image processing method, including the following steps. A pixel voltage is obtained by sampling a pixel. The pixel voltage is amplified according to an analog gain. The amplified pixel voltage is converted into digital pixel data, and the digital pixel data includes a plurality of first sub-pixel data corresponding to different colors. According to the analog gain and a plurality of white balance gains corresponding to different sub-pixels, white balance compensation is performed on the first sub-pixel data respectively to obtain a plurality of second sub-pixel data. The operating principle of the image processing method has been described in detail in the content of the image processing device 100 and its related operations, so it will not be repeated here.

The image processing device and method disclosed in the above-mentioned embodiments of the present disclosure perform white balance compensation on the pixel data according to the analog gain and the white balance gain, and the comparison between the current maximum well capacity and the pixel data can also be used to effectively clamp the pixel data . Therefore, the above-mentioned embodiments can improve the phenomenon of luminance color shift and high luminance noise, so the dynamic range of the image can be increased, and the application with less noise can be considered to improve the overall image quality.

In summary, although the present invention has been disclosed as above with a number of embodiments, they are not intended to limit the present invention. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention should be defined by the appended claims.

Claims (20)

1. An image processing device, comprising: a sampling unit for sampling a pixel to obtain a pixel voltage; an amplifier for amplifying the pixel voltage according to an analog gain; an analog-to-digital converter for converting the amplified pixel voltage into digital pixel data, the digital pixel data including a plurality of first sub-pixel data corresponding to different colors; and a white balance unit, for performing white balance compensation on the first sub-pixel data respectively according to the analog gain and a plurality of white balance gains corresponding to different sub-pixels to obtain a plurality of second sub-pixel data, Wherein the white balance unit generates a plurality of third sub-pixel data according to the first sub-pixel data and the white balance gains, and judges whether to clamp ( clamp) the third sub-pixel data to generate the second sub-pixel data. 2. The image processing device as claimed in claim 1, wherein the white balance unit respectively calculates the products of the white balance gains and the first sub-pixel data as the third sub-pixel data. 3. The image processing device according to claim 1, wherein the white balance unit judges whether the maximum well capacity is within a range corresponding to the analog gain, and when the maximum well capacity is within the range, according to the maximum well capacity and The analog gain calculates a current maximum well capacity, and compares the third sub-pixel data with the current maximum well capacity to determine whether to clamp the third sub-pixel data to generate the second sub-pixel data, wherein the Range is defined by the analog gain and the number of bits of the ADC. 4. The image processing device as claimed in claim 3, wherein when the maximum well capacity is not within the range, the white balance unit does not clamp the third sub-pixel data. 5. The image processing device as claimed in claim 3, wherein the white balance unit calculates the product of the analog gain and the maximum well capacity as the current maximum well capacity. 6. The image processing device as claimed in claim 5, wherein the white balance unit further limits the current maximum well capacity to not more than 2 ⁽ⁿ⁻¹⁾ , where n is the number of bits of the pixel. 7. The image processing device according to claim 3, wherein the white balance unit respectively judges whether the third sub-pixel data is larger than the current maximum well capacity, and when the judgment result is yes, the white balance unit clamps the third Sub-pixel data, when the determination result is negative, the white balance unit does not clamp the third sub-pixel data. 8. The image processing device as claimed in claim 3, wherein when the white balance unit clamps the third sub-pixel data, the white balance unit outputs the current maximum well capacity as the second sub-pixel data, and when the white balance unit The balance unit does not clamp the third sub-pixel data, and the white balance unit outputs the third sub-pixel data as the second sub-pixel data. 9. The image processing apparatus according to claim 1, further comprising: An automatic exposure unit is used to control the sampling unit and provide the analog gain to the amplifier and the white balance unit. 10. The image processing apparatus according to claim 1, further comprising: An automatic white balance unit is used to provide the white balance gains of the white balance unit. 11. The image processing apparatus according to claim 1, further comprising: A black calibration unit is coupled between the digital-to-analog converter and the white balance unit for performing black calibration on the digital pixel data. 12. The image processing apparatus according to claim 1, further comprising: A color interpolation unit is used for interpolating an original image data output by the white balance unit to obtain image data of three primary colors, and the original image data includes the second sub-pixel data. 13. An image processing method, comprising: (i) sampling a pixel to obtain a pixel voltage; (ii) amplifying the pixel voltage according to an analog gain; (iii) converting the amplified pixel voltage into digital pixel data, the digital pixel data including a plurality of first sub-pixel data corresponding to different colors; and (iv) performing white balance compensation on the first sub-pixel data according to the analog gain and a plurality of white balance gains corresponding to different sub-pixels to obtain a plurality of second sub-pixel data, Wherein step (iv) comprises: (a) generating a plurality of third sub-pixel data according to the first sub-pixel data and the white balance gains; and (b) judging whether to clamp the third sub-pixel data according to the analog gain and a maximum fullwell level of the pixel, so as to generate the second sub-pixel data. 14. The image processing method as claimed in claim 13, wherein step (a) comprises calculating the products of the white balance gains and the first sub-pixel data respectively as the third sub-pixel data. 15. The image processing method as claimed in claim 13, wherein step (b) comprises: (b1) judging whether the maximum well capacity is within a range corresponding to the analog gain; (b2) When the maximum well capacity is within the range, calculate a current maximum well capacity according to the maximum well capacity and the simulation gain; and (b3) Comparing the third sub-pixel data with the current maximum well capacity to determine whether to clamp the third sub-pixel data to generate the second sub-pixel data. 16. The image processing method as claimed in claim 15, wherein step (b3) comprises: When the maximum well capacity is not within the range, the third sub-pixel data are not clamped. 17. The image processing method as claimed in claim 15, wherein step (b2) comprises: Calculate the product of the analog gain and the maximum well capacity as the current maximum well capacity. 18. The image processing method as claimed in claim 17, further comprising: Limit the current maximum well capacity to no more than 2 ^(n-1) , where n is the number of bits of the pixel. 19. The image processing method as claimed in claim 15, wherein step (b3) comprises: Respectively determine whether the third sub-pixel data is greater than the current maximum well capacity; and When the judgment result is yes, the third sub-pixel data are clamped, and when the judgment result is no, the third sub-pixel data are not clamped. 20. The image processing method according to claim 15, wherein when clamping the third sub-pixel data, the current maximum well capacity is output as the second sub-pixel data; and when not clamping the third sub-pixel data , output the third sub-pixel data as the second sub-pixel data.