CN117082351A - Image brightness processing method and image sensor - Google Patents
Image brightness processing method and image sensor Download PDFInfo
- Publication number
- CN117082351A CN117082351A CN202210944199.8A CN202210944199A CN117082351A CN 117082351 A CN117082351 A CN 117082351A CN 202210944199 A CN202210944199 A CN 202210944199A CN 117082351 A CN117082351 A CN 117082351A
- Authority
- CN
- China
- Prior art keywords
- white
- pixel points
- low
- image
- white pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 14
- 238000005286 illumination Methods 0.000 claims abstract description 44
- 238000005070 sampling Methods 0.000 claims abstract description 14
- 125000004122 cyclic group Chemical group 0.000 claims abstract description 9
- 238000012545 processing Methods 0.000 claims abstract description 8
- 239000011159 matrix material Substances 0.000 claims abstract description 5
- 230000009191 jumping Effects 0.000 claims abstract description 4
- 238000000034 method Methods 0.000 claims description 14
- 230000005484 gravity Effects 0.000 claims description 12
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 7
- 238000013459 approach Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 239000006185 dispersion Substances 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 101100248200 Arabidopsis thaliana RGGB gene Proteins 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses an image brightness processing method and an image sensor, wherein the processing method comprises the following steps: s1, determining a low-illumination sampling area, and configuring white pixel points according to a minimum circulation unit; s2, sampling the electric signals of the white pixel points; s3, calculating an average value of pixel values of white pixel points in each minimum cyclic unit; s4, restoring to obtain a full-color pixel matrix; s5, judging the low-illumination environment, and jumping to the step S6 when the low-illumination environment is judged; when the environment is judged not to be the low-illumination environment, directly outputting image data; s6, carrying out brightness compensation on the image by using the average value of the pixel values of the white pixel points obtained in the step S3. The image sensor can automatically judge the low-illumination environment and perform image brightness processing based on the judgment result.
Description
Technical Field
The invention relates to the technical field of image sensors, in particular to an image brightness processing method and an image sensor.
Background
Night shooting is a common use scenario of current image sensors, and low illumination can greatly reduce the visual quality of the image. In order to obtain an image with required brightness in a dark scene, common processing methods include: simultaneously, two image sensor chips are carried, one is used for collecting image color information, the other is used for collecting image brightness information, the two information are processed through an algorithm, and finally, low-illumination image enhancement is realized, so that the cost is relatively high. In order to reduce the cost, a method of configuring an optical filter on an image sensor is also proposed, and a common configuration method is RCCB, RCCC, GRBW. The method replaces a part of color filter (color filter) configuration in the RGB pixels of the original Bayer/Fourier cell with white color (clear), and adjusts image information on the color filter by adopting brightness values of adjacent white pixels so as to realize brightness enhancement. This is done at the expense of color filtering, with the result that the color of the output image is lost, ultimately affecting the quality of the image.
It is therefore desirable to provide an image sensor that can automatically detect the intensity of illumination and automatically adjust the image output value.
Disclosure of Invention
The invention aims to solve the technical problem that the traditional image sensor influences the image quality due to insufficient brightness in a low-illumination environment. The invention aims to provide a method and an image sensor capable of automatically detecting illumination intensity, judging a low-illumination environment and automatically performing image brightness processing.
In order to solve the above problems, the present invention provides an image brightness processing method, comprising the steps of:
s1, determining a low-illumination sampling area, and configuring white pixel points in the area according to a minimum circulation unit;
s2, sampling the electric signals of the white pixel points, and converting the electric signals into digital signals after analog-to-digital conversion;
s3, calculating an average value of pixel values of white pixel points in each minimum cyclic unit;
s4, replacing the position of the configured white pixel point with the original pixel point, and restoring a full-color pixel matrix;
s5, judging the low-illumination environment, and jumping to the step S6 when the low-illumination environment is judged; when the environment is judged not to be the low-illumination environment, directly outputting image data;
s6, carrying out brightness compensation on the image by using the average value of the pixel values of the white pixel points obtained in the step S3.
Further, in step S1, the low-illuminance sampling area is an area between the effective pixel area and the photosensitive pixel area.
Further, in step S1, the configuring the white pixel point needs to satisfy the following conditions:
uniformly configuring white pixel points outwards from the central area, wherein the white pixel points replace green pixel points (comprising Gr and Gb), and the number of the pixel points replaced by Gr and Gb is ensured to be the same; the number of the white pixel points contained in every N rows and every N columns is the same;
the principle of uniformly dispersing the central area is satisfied, the central area is positioned in the range where the white pixel points of the replacement Gr are connected, and the central area is positioned in the range where the white pixel points of the replacement Gb are connected;
meanwhile, the gravity center distribution principle is met, the number of white points is consistent in four areas divided by diagonal lines by taking the diagonal lines of the WMCU as a reference, each white point has symmetrical points by taking the diagonal lines as symmetry axes, and the white points can be overlapped with the gravity center of the WMCU after being connected.
Further, in step S5, the method for determining a low-illuminance environment includes:
setting a brightness threshold V th Average value AVE_W of white pixel points in each WMCU unit and brightness threshold V th Comparing, the statistical average value AVE_W is smaller than the threshold value V th Is a ratio N of the number of WMCUs to the total number of WMCUs. When the proportion N is more than or equal to 50%, the environment with low illumination is judged.
Further, the luminance threshold vth=minimum luminance value+l× (maximum luminance value-minimum luminance value), wherein 15% l.ltoreq.50%.
Further, in step S6, the brightness compensation method includes:
luminance compensation value of any WMCU unit = sum of pixel averages of white pixels of other 8 WMCUs adjacent x N/8; wherein N is that average value AVE_W is smaller than threshold value V th The number of WMCUs is a proportion of the total number of WMCUs.
In a second aspect, the present invention also provides an image sensor capable of judging a low-illuminance environment and automatically performing brightness processing, including:
a white pixel point configuration unit for configuring white point pixels for brightness acquisition in the LISA region;
an analog-to-digital conversion unit for converting the electric signal of the pixel into a digital signal;
the average value calculation unit of the WMCU unit is used for calculating the average value of the pixel values of the white pixel points in the WMCU;
the low-illumination environment judging unit is used for judging the current shooting environment;
and the image brightness compensation unit is used for compensating the image brightness by using the average value of the white pixel points in a low-illumination environment.
Compared with the prior art, the image brightness compensation method can automatically realize image brightness compensation in a low-illumination environment, and the effective configuration of the white pixel points realizes the low occupation ratio of the white pixel points, so that the image resolution is ensured. In addition, the image sensor can reduce cost on the premise of ensuring image quality.
Drawings
In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the drawings that are needed in the examples will be briefly described below, it being understood that the following drawings only illustrate some examples of the present invention and therefore should not be considered as limiting the scope, and that other related drawings may be obtained from these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of an image brightness processing flow according to the present invention;
fig. 2 is a schematic diagram of the positions of the low-illumination sampling area LISA and the minimum cyclic unit WMCU;
FIG. 3 is a schematic diagram of an 8×8 Byer array;
fig. 4 is a schematic diagram showing the arrangement of white pixels in an 8×8 bayer array;
FIG. 5 is a schematic illustration of the center of all white pixels in an 8×8 Byer array coinciding with the center of gravity of the array;
FIG. 6 is a schematic diagram of a 16×16 Byer array;
fig. 7 is a schematic diagram showing the arrangement of white pixels in a 16×16 bayer array;
FIG. 8 is a schematic illustration of the center of all white pixels in a 16×16 Byer array coinciding with the center of gravity of the array;
fig. 9 is a schematic diagram illustrating calculation of brightness compensation values of WMCU units.
Detailed Description
For the purpose of making apparent the objects, technical solutions and advantages of the present invention, the present invention will be further described in detail with reference to the following examples and the accompanying drawings, wherein the exemplary embodiments of the present invention and the descriptions thereof are for illustrating the present invention only and are not to be construed as limiting the present invention.
In order to obtain an image of a desired brightness in a dark scene, it is common practice to: (1) Two image sensor chips are carried, namely an RGB structure and a Mono (Monochrome) pure white structure, and the two sensors are of the same pixel size. In a dim light scene, a chip with an RGB structure is responsible for collecting picture color information, and another chip with a Mono pure white structure is responsible for collecting picture brightness information, and the two are combined through an algorithm in the later stage to realize low-illumination image enhancement. This method requires two sensor chips, two sets of lenses, and is costly. (2) Four filters G/R/B/C (W) are arranged on the image sensor, and are commonly arranged in RCCB/RCCC/GRBW and the like. This configuration realizes low-illuminance luminance enhancement by replacing a part of color filter (color filter) configuration in RGB pixels of the original Bayer/fourier cell with white (clear), and adjusting the image information on the color filters with luminance values of adjacent white pixels, respectively. However, this configuration sacrifices a significant portion of the color filtering configuration. RCCC white duty cycle 75%, RCCB and GRBW white duty cycle 50% (RCCC and RCCB lack G/B color configuration, typically not used for cell phones). The second approach is less costly than the first approach, but due to the excessively high white point occupancy, half of the colors of the final output image are lost (half and more of the color configurations are replaced with white points).
Because the traditional image sensor has defects in improving the output brightness of the image under the low-illumination condition, the invention aims at the defects to provide the image sensor capable of realizing automatic illumination detection and brightness processing in a dark scene. The invention adopts a mode of inserting the white pixel point into the color filter array of the image sensor according to the minimum circulation unit, and adjusts the brightness of the current image by using the lighting intensity of the white pixel point, thereby realizing the adjustment of the brightness of the image under the condition of low illumination and no increase of exposure time. Meanwhile, after the white pixel data are sampled, the full-color pixel matrix is restored to ensure the integrity of the image data.
Example 1
As shown in fig. 1, the present embodiment provides a method for processing image brightness, which includes the following steps:
s1, determining a low-illumination sampling area (LISA area for short), and configuring white pixel points in the area according to a minimum circulation unit;
s2, sampling the electric signals of the white pixel points, and converting the electric signals into digital signals after analog-to-digital conversion;
s3, calculating an average value of pixel values of white pixel points in each minimum cyclic unit;
s4, replacing the position of the configured white pixel point with the original pixel point, and restoring a full-color pixel matrix;
s5, judging the low-illumination environment, and jumping to the step S6 when the low-illumination environment is judged; when the environment is judged not to be the low-illumination environment, directly outputting image data;
s6, carrying out brightness compensation on the image by using the average value of the pixel values of the white pixel points obtained in the step S3.
By configuring the white pixel points in the low-illumination sampling area, when the shooting environment is low-illumination, the actual lighting intensity of the white pixel points is adopted to realize image brightness compensation. Meanwhile, the configured white pixel points are replaced by original pixel points, so that the integrity of image information is ensured.
In order to ensure a higher resolution of an image, it is required that the ratio of the configured white pixels is as low as possible, but the lighting intensity under the low illumination condition is satisfied at the same time. In order to achieve the object, the present invention provides a possible white pixel configuration method, and specific embodiments are as follows:
as shown in fig. 2, in step S1, the range of the low-illumination sampling area (Low Illumination Sampling Area, abbreviated as LISA area) may be a portion between the effective pixel area and the photosensitive pixel area, the minimum range is the effective pixel area, and the maximum range is the photosensitive pixel area. The effective pixel interval has a meaning known in the art, and refers to an interval in which a pixel value is actually output, and the pixel value of the interval is to be completely sampled.
The same number of white pixels are inserted in the LISA region with WMCU (White Minimum Cyclic Unit) as the minimum cyclic unit, so that the white pixels are uniformly dispersed in the LISA region. In WMCU, the inserted white pixel needs to satisfy the following conditions:
uniformly configuring white pixel points outwards from a central area (minimum RGGB unit in the center of the WMCU), wherein the white pixel points replace green pixel points (comprising Gr and Gb), and the number of the pixel points replaced by the Gr and the Gb is ensured to be the same; the number of the white pixel points contained in every N rows and every N columns is the same;
the principle of uniformly dispersing the central area is satisfied, the central area is positioned in the range where the white pixel points of the replacement Gr are connected, and the central area is positioned in the range where the white pixel points of the replacement Gb are connected;
meanwhile, the gravity center distribution principle is met, the number of white points is consistent in four areas divided by diagonal lines by taking the diagonal lines of the WMCU as a reference, each white point has symmetrical points by taking the diagonal lines as symmetry axes, and the white points can be overlapped with the gravity center of the WMCU after being connected.
To more specifically explain the arrangement of the white pixels, a minimum cyclic unit WMCU of a Bayer array of 8×8 and 16×16 will be described as follows:
as shown in fig. 3 to 5, for a Bayer array with WMCU of 8×8, white pixels are disposed outwards from a central area, and replace the same number of Gr and Gb pixels, and the same number of white pixels (2) are disposed every 2 rows and every 2 columns, so as to ensure that the white pixels are uniformly dispersed in the LISA region. Simultaneously, the principle of uniform dispersion of the central area and the principle of gravity center distribution are satisfied, namely, the central area (RGrGbB) is positioned in the range connected with the white pixel points of the replacement Gr, and the central area is positioned in the range connected with the white pixel points of the replacement Gb; in addition, with the diagonal line of the WMCU as a reference, in four areas divided by the diagonal line, the number of white points is consistent (2 white points are all) and each white point has symmetrical points with the diagonal line as a symmetrical axis, and the white points can be overlapped with the gravity center of the WMCU after being connected.
As shown in fig. 6 to 8, for a Bayer array of WMCU 16×16, white pixels are arranged outward from the center area, with the white pixels replacing 4 Gr and 4 Gb, and 1 white pixel is arranged every 2 columns and every 2 rows. Simultaneously, the principle of uniform dispersion of the central area and the principle of gravity center distribution are satisfied, namely, the central area (RGrGbB) is positioned in the range connected with the white pixel points of the replacement Gr, and the central area is positioned in the range connected with the white pixel points of the replacement Gb; in addition, with the diagonal line of the WMCU as a reference, in four areas divided by the diagonal line, the number of white points is consistent (2 white points are all) and each white point has symmetrical points with the diagonal line as a symmetrical axis, and the white points can be overlapped with the gravity center of the WMCU after being connected.
The configuration mode of the white pixel points ensures that the white pixel points are uniformly distributed in the LISA region, and the duty ratio of the white pixel points is reduced as much as possible on the premise of meeting the brightness requirement, thereby ensuring the resolution of the image.
In the step S3, the average value of the pixel values of the white pixels in the WMCU unit is calculated by adding the pixel values of all the white pixels in the unit and dividing by the total number of the white pixels.
In the step S4, the configured white pixel point is restored to the original green pixel point, and the pixel point may be replaced by an interpolation method or other data processing methods as known in the art, which will not be described herein.
One possible implementation of the low-light environment determination in step S5 is as follows:
setting a brightness threshold V th Average value AVE_W of white pixel points in each WMCU unit and brightness threshold V th Comparing, the statistical average value AVE_W is smaller than the threshold value V th Is a ratio N of the number of WMCUs to the total number of WMCUs. When the proportion N is more than or equal to 50%, the environment with low illumination is judged.
The luminance threshold V th Can be set as desired by those skilled in the art. Alternatively, the luminance threshold vth=minimum luminance value+l× (maximum luminance value-minimum luminance value), where L is any value of 15% -50%, the maximum luminance value and the minimum luminance value are obtained by the chip test.
As a possible implementation manner, the method of step S6 image brightness compensation is as follows:
calculating the pixel average value of the white pixel points of 8 other WMCUs adjacent to the WMCU, wherein the brightness compensation value of any WMCU unit is equal to the sum of the pixel average values of the white pixel points of the other 8 adjacent WMCUs multiplied by N/8; wherein N is that average value AVE_W is smaller than threshold value V th The number of WMCUs is a proportion of the total number of WMCUs.
As shown in fig. 9, the luminance compensation value of wmcu_0= (ave_w1+ave_w2+ave_w3+ave_w4+ave_w5+ave_w6+ave_w7+ave_w8) x w_gain/8.
Example 2
The present embodiment provides an image sensor capable of automatic illumination detection and luminance processing, which is applicable to embodiment 1, and includes:
a white pixel point configuration unit for configuring white point pixels for brightness acquisition in the LISA region;
an analog-to-digital conversion unit for converting the electric signal of the pixel into a digital signal;
the average value calculation unit of the WMCU unit is used for calculating the average value of the pixel values of the white pixel points in the WMCU;
the low-illumination environment judging unit is used for judging the current shooting environment;
and the image brightness compensation unit is used for compensating the image brightness by using the average value of the white pixel points in a low-illumination environment.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, etc. that fall within the spirit and principles of the invention are to be construed as being included therein.
Claims (7)
1. An image brightness processing method, comprising:
s1, determining a low-illumination sampling area, and configuring white pixel points in the area according to a minimum cyclic unit WMCU;
s2, sampling the electric signals of the white pixel points, and converting the electric signals into digital signals after analog-to-digital conversion;
s3, calculating an average value of pixel values of white pixel points in each minimum cyclic unit WMCU;
s4, replacing the position of the configured white pixel point with the original pixel point, and restoring a full-color pixel matrix;
s5, judging the low-illumination environment, and jumping to the step S6 when the low-illumination environment is judged; when the environment is judged not to be the low-illumination environment, directly outputting image data;
s6, carrying out brightness compensation on the image by using the average value of the pixel values of the white pixel points obtained in the step S3.
2. The image brightness processing method according to claim 1, wherein the low-illuminance sampling area in step S1 is an area between an effective pixel section and a photosensitive pixel section.
3. The image brightness processing method according to claim 1, wherein the configuring of the white pixel in step S1 satisfies the following condition:
uniformly configuring white pixel points outwards from the central area, wherein the white pixel points replace green pixel points, and the number of the pixel points replaced by Gr and Gb is ensured to be the same; the number of the white pixel points contained in every N rows and every N columns is the same;
the principle of uniformly dispersing the central area is satisfied, the central area is positioned in the range where the white pixel points of the replacement Gr are connected, and the central area is positioned in the range where the white pixel points of the replacement Gb are connected;
the gravity center distribution principle is met, the number of white points is consistent in four areas divided by diagonal lines by taking the diagonal lines of the WMCU as a reference, each white point has symmetrical points by taking the diagonal lines as symmetrical axes, and the white points can be overlapped with the gravity centers of the WMCU after being connected.
4. The image brightness processing method according to claim 1, wherein in step S5, the method for determining a low-illuminance environment includes:
setting a brightness threshold V th Average value AVE_W of white pixel points in each WMCU unit and brightness threshold V th Comparing, the statistical average value AVE_W is smaller than the threshold value V th And if the number of the WMCUs is more than or equal to 50%, judging that the WMCUs are in a low-illumination environment.
5. The image brightness processing method according to claim 4, wherein the brightness threshold vth=minimum brightness value+l× (maximum brightness value-minimum brightness value), wherein 15% +..
6. The image brightness processing method according to claim 1, wherein in step S6, the brightness compensation method includes:
luminance compensation value of any WMCU unit = sum of pixel averages of white pixels of other 8 WMCUs adjacent x N/8; wherein N is that average value AVE_W is smaller than threshold value V th The number of WMCUs is a proportion of the total number of WMCUs.
7. An image sensor capable of performing the image brightness processing of any one of claims 1-6, the image sensor comprising:
a white pixel point configuration unit for configuring white point pixels for brightness acquisition in the LISA region;
an analog-to-digital conversion unit for converting the electric signal of the pixel into a digital signal;
the average value calculation unit of the WMCU unit is used for calculating the average value of the pixel values of the white pixel points in the WMCU;
the low-illumination environment judging unit is used for judging the current shooting environment;
and the image brightness compensation unit is used for compensating the image brightness by using the average value of the white pixel points in a low-illumination environment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210944199.8A CN117082351A (en) | 2022-08-05 | 2022-08-05 | Image brightness processing method and image sensor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210944199.8A CN117082351A (en) | 2022-08-05 | 2022-08-05 | Image brightness processing method and image sensor |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117082351A true CN117082351A (en) | 2023-11-17 |
Family
ID=88710262
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210944199.8A Pending CN117082351A (en) | 2022-08-05 | 2022-08-05 | Image brightness processing method and image sensor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117082351A (en) |
-
2022
- 2022-08-05 CN CN202210944199.8A patent/CN117082351A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7471820B2 (en) | Correction method for defects in imagers | |
| US20090002530A1 (en) | Apparatus and method for processing images | |
| US7804525B2 (en) | Method, apparatus, and system for selecting pixels for automatic white balance processing | |
| EP0997845B1 (en) | A test efficient method of classifying image quality of an optical sensor using three categories of pixels | |
| US20080239121A1 (en) | Imaging apparatus including noise suppression circuit | |
| US7034874B1 (en) | Automatic bad pixel correction in image sensors | |
| US8482655B2 (en) | Method of controlling imaging apparatus | |
| EP1557790A2 (en) | Noise reduction in low-illuminance image | |
| CN100426870C (en) | Video signal processing apparatus and video signal processing method | |
| CN101510955B (en) | Image pickup device | |
| WO2018146945A1 (en) | Moving body monitoring device and moving body monitoring system | |
| US7450167B2 (en) | Method for improving image quality | |
| CN117082351A (en) | Image brightness processing method and image sensor | |
| US20070285529A1 (en) | Image input device, imaging module and solid-state imaging apparatus | |
| US20130266220A1 (en) | Color signal processing circuit, color signal processing method, color reproduction evaluating method, imaging apparatus, electronic apparatus and testing device | |
| WO2022032666A1 (en) | Image processing method and related apparatus | |
| US7940314B2 (en) | Image-signal processing device | |
| US20070046787A1 (en) | Chrominance filter for white balance statistics | |
| JP2008153848A (en) | Image processing apparatus | |
| US20240147060A1 (en) | Day and night filter switcher, switching method therefor, and switching device therefor | |
| JPS601981A (en) | Solid state image pickup device | |
| KR20050096018A (en) | Camera phone had a fuction of compensation of the image distortion | |
| KR100639155B1 (en) | Extended dynamic range imaging system and method | |
| JPH08125926A (en) | Exposure controller for digital still video camera | |
| JPH06113193A (en) | Image pickup device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |