CN114143481A - Method for optimizing performance of image sensor based on self-adaptive programmable analog-to-digital conversion - Google Patents
Method for optimizing performance of image sensor based on self-adaptive programmable analog-to-digital conversion Download PDFInfo
- Publication number
- CN114143481A CN114143481A CN202010919925.1A CN202010919925A CN114143481A CN 114143481 A CN114143481 A CN 114143481A CN 202010919925 A CN202010919925 A CN 202010919925A CN 114143481 A CN114143481 A CN 114143481A
- Authority
- CN
- China
- Prior art keywords
- digital conversion
- analog
- image sensor
- mode
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/268—Signal distribution or switching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/50—Control of the SSIS exposure
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
Abstract
The invention provides a method for optimizing the performance of an image sensor based on self-adaptive programmable analog-to-digital conversion, which is characterized in that one or more luminance intervals interested by a user are obtained from original image information according to environment information or a self-defined mode, the analog-to-digital conversion precision of the interested luminance intervals is improved, and the analog-to-digital conversion precision is reduced for other luminance intervals so as to improve the analog-to-digital conversion speed, so that the conversion precision and the conversion speed in the whole analog-to-digital conversion process are considered, the analog-to-digital conversion relation is flexible and changeable, the high dynamic environment in the natural world can be better restored, the analog-to-digital conversion is closer to the response mechanism of human eyes, or based on the requirements of certain machine vision, the preference requirements of different scenes and the user are met, and the overall performance of the image sensor is improved.
Description
Technical Field
The invention relates to a method for optimizing the performance of an image sensor based on self-adaptive programmable analog-to-digital conversion.
Background
The light intensity variation range of the nature is large, and the whole and integral phase difference of the light brightness from the dazzling sunlight to the weak starlight is 108In fact, the difference between the reflector and the illuminant is added, the real dynamic range far exceeds the value, and the dynamic range of most scenes in real life is over 100 dB.
The human eye is more adaptive to these highly dynamic scenes because the human eye does not have a linear but logarithmic relationship to the luminance response (the relationship between the subjective luminance S and the ambient luminance B is as follows: S = KlgB + K)0K and K0Are all constants). A large number of visual cells are concentrated on the retina of the human eye, and are mainly classified into two types: coneThe cone-shaped cells are mainly distributed in the center of the retina, about 600W-700W are distributed in each eye, each cone-shaped cell is connected to a nerve ending, and the cone-shaped cells have higher resolution, can distinguish the intensity of light and are sensitive to color, so the cone-shaped cells are also called photopic vision; the rod-shaped cells are distributed on the whole retina surface, about 7500W-15000W are provided, and a plurality of rod-shaped cells are connected to the same nerve ending, so that the resolution is lower, the rod-shaped cells are sensitive to low illumination, and the dark vision with high sensitivity and no visual sensation function is formed. It is through this logarithmic response to light that the human eye can receive widths of up to 108The dynamic range of the brightness range can be as high as 100dB to 120 dB.
However, in the mainstream CMOS image sensor, the analog-to-digital conversion mode includes two types, i.e., linear mode and piecewise linear mode: the linear conversion shown in FIG. 1 is performed in the whole brightness interval with DN values of 0-200, and no matter the bright place and the dark place use the same conversion precision, if the conversion precision is low, although the conversion speed is high, the detail resolution is not enough, if the conversion precision is high, although the detail effect is good, the conversion speed influence is large, so that the high-dynamic world can only be subjected to a trade-off, and either the conversion precision or the conversion speed is lost; although the piecewise linear conversion shown in fig. 2 can have higher conversion accuracy in a low-brightness scene (for example, a low-brightness interval of 0 to 90) while increasing the conversion speed, and ensure the resolution of dim details, the conversion accuracy in a high-brightness scene (for example, a high-brightness interval of 180 to 200) is sacrificed, so that different scenes cannot be considered, and different user preference requirements are different, a user may be interested in a high-brightness interval of 90 to 180 or 180 to 200 instead of a low-brightness interval of 0 to 90, and even the brightness interval of interest of the user may change along with environmental information or a user-defined mode selected by the user, whereas the conventional piecewise linear conversion with a fixed shape cannot meet the specific or changed user preference. That is, the conventional image sensor cannot satisfy both the conversion accuracy and the conversion speed, and cannot simultaneously satisfy the favorite requirements of different scenes and users.
Disclosure of Invention
The invention aims to provide a method for optimizing the performance of an image sensor based on self-adaptive programmable analog-to-digital conversion, which considers conversion precision and conversion speed, meets the favorite requirements of different scenes and users and improves the overall performance of the image sensor.
Based on the above considerations, the present invention provides a method for optimizing the performance of an image sensor based on adaptive programmable analog-to-digital conversion, comprising: acquiring environmental information or a user-defined mode; and acquiring one or more brightness intervals interested by the user from the original image information according to the environment information or the user-defined mode, and improving the analog-to-digital conversion precision of the interested brightness intervals so as to improve the image quality.
Preferably, analog-to-digital conversion curves with different analog-to-digital conversion precisions are generated according to the environment information or the user-defined mode, and the analog-to-digital conversion curves with the different analog-to-digital conversion precisions are subjected to normalization processing and then transmitted to an external platform.
Preferably, the analog-to-digital conversion curves with different analog-to-digital conversion precisions include a nonlinear part, and the nonlinear part is subjected to linear calibration and then normalization processing.
Preferably, the analog-to-digital conversion curves with different analog-to-digital conversion precisions are continuously derivable curves.
Preferably, a corresponding analog-to-digital conversion curve is generated according to the environment information or the custom mode, the analog-to-digital conversion curve comprises a linear part and/or a nonlinear part, and data of the nonlinear part of the analog-to-digital conversion curve after linear calibration processing is transmitted to an external platform.
Preferably, the environment information includes any one or a combination of histogram information, environment brightness, image brightness, f-number, exposure time, and sensitivity.
Preferably, the user-defined mode comprises a license plate mode, a portrait mode, a night mode and a sky mode.
The method for optimizing the performance of the image sensor based on the self-adaptive programmable analog-to-digital conversion acquires one or more luminance intervals interested by a user from original image information according to environment information or a user-defined mode, improves the analog-to-digital conversion precision of the interested luminance intervals, reduces the analog-to-digital conversion precision for other luminance intervals to improve the analog-to-digital conversion speed, and accordingly considers the conversion precision and the conversion speed of the whole analog-to-digital conversion process, enables the analog-to-digital conversion relation to be flexible and changeable, can better restore the high dynamic environment in the natural world, is closer to the response mechanism of human eyes, or meets the favorite requirements of different scenes and users based on the requirements of certain machine vision, and improves the overall performance of the image sensor.
Drawings
Other features, objects and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments thereof, which proceeds with reference to the accompanying drawings.
FIG. 1 is a graph of a linear analog-to-digital conversion of a prior art image sensor;
FIG. 2 is a graph of a piecewise linear analog-to-digital conversion of a prior art image sensor;
FIG. 3 is a flow chart of a method for optimizing image sensor performance based on adaptive programmable analog to digital conversion in accordance with the present invention;
FIG. 4 is a graph of an analog-to-digital conversion of an image sensor according to one embodiment of the invention;
fig. 5 is a graph of analog-to-digital conversion of an image sensor according to another embodiment of the present invention.
In the drawings, like or similar reference numbers indicate like or similar devices (modules) or steps throughout the different views.
Detailed Description
In order to solve the problems in the prior art, the invention provides a method for optimizing the performance of an image sensor based on adaptive programmable analog-to-digital conversion, which is characterized in that one or more luminance intervals interested by a user are obtained from original image information according to environment information or a user-defined mode, the analog-to-digital conversion precision of the interested luminance intervals is improved, and the analog-to-digital conversion precision is reduced for other luminance intervals so as to improve the analog-to-digital conversion speed, so that the conversion precision and the conversion speed in the whole analog-to-digital conversion process are considered, the analog-to-digital conversion relation is flexible and changeable, the high-dynamic environment in the natural world can be better restored, the analog-to-digital conversion relation is closer to the response mechanism of human eyes, or based on the requirements of certain machine vision, the preference requirements of different scenes and the user are met, and the overall performance of the image sensor is improved.
In the following detailed description of the preferred embodiments, reference is made to the accompanying drawings which form a part hereof. The accompanying drawings illustrate, by way of example, specific embodiments in which the invention may be practiced. The illustrated embodiments are not intended to be exhaustive of all embodiments according to the invention. It is to be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present invention. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.
The present invention will be described in detail with reference to specific examples.
FIG. 3 shows the main flow of the method for optimizing the performance of an image sensor based on adaptive programmable analog-to-digital conversion according to the present invention: firstly, acquiring environment information or a self-defined mode, wherein the environment information can comprise any one or combination of histogram information, environment brightness, image brightness, diaphragm number, exposure time and light sensitivity, and the self-defined mode can comprise a license plate mode, a portrait mode, a night mode, a sky mode and the like; then, acquiring one or more brightness intervals which are interesting to the user from the original image information according to the environment information or the user-defined mode; then, increasing the analog-to-digital conversion accuracy of the interested brightness interval to improve the image quality, where it is to be noted that, by setting that the interested brightness interval has an analog-to-digital conversion accuracy higher than that of other brightness intervals, image details of a key area concerned by a user are better shown, and other brightness intervals have an analog-to-digital conversion speed higher than that of the interested brightness interval, so as to meet a requirement for a shooting speed, specifically, an analog-to-digital conversion curve with different analog-to-digital conversion accuracies is generated according to the environment information or a custom mode; and finally, normalizing the analog-to-digital conversion curves with different analog-to-digital conversion accuracies and transmitting the normalized analog-to-digital conversion curves to an external platform, wherein if the analog-to-digital conversion curves with different analog-to-digital conversion accuracies comprise a nonlinear part, the nonlinear part needs to be subjected to linear calibration and then normalized.
According to the invention, the analog-to-digital conversion precision is improved for the interested brightness interval, and the analog-to-digital conversion precision is reduced for other brightness intervals so as to improve the analog-to-digital conversion speed, so that the conversion precision and the conversion speed in the whole analog-to-digital conversion process are taken into account, the analog-to-digital conversion relation is flexible and changeable, the high dynamic environment in the nature can be better restored, the response mechanism of human eyes is more approximate, or based on the requirements of some machine vision, the favorite requirements of different scenes and users are met, and the overall performance of the image sensor is improved.
Example one
Fig. 4 illustrates an analog-to-digital conversion graph of an image sensor according to an embodiment of the present invention.
In the application of the image sensor of this embodiment, firstly, the environment information or the custom mode is obtained through the image sensor or the external platform, then one or more luminance intervals in which the user is interested are obtained from the original image information according to the environment information or the custom mode, for example, according to any one or a combination of histogram information, environment luminance, image luminance, f-number, exposure time and sensitivity, or according to the custom mode, it is determined that the region in which the user is interested is concentrated on the license plate and the face, the Digital Number (DN) of the luminance interval corresponding to the license plate and the face is statistically obtained through the platform algorithm and is approximately between 40 to 80 and 100 to 140 of 8bit data, then the analog-to-digital conversion module is controlled by the digital signal to generate the segmented ramp signal, so that the analog-to-digital conversion precision of the luminance intervals corresponding to the license plate and the face region in which the user is interested is higher than other luminance intervals, the obtained analog-to-digital conversion curves with different analog-to-digital conversion accuracies are shown in fig. 4, wherein the slope of the curve in the brightness interval of 40-80 and 100-140 is smaller than the slope of other brightness intervals, that is, the brightness interval of 40-80 and 100-140 has the analog-to-digital conversion accuracy higher than that of other brightness intervals. Therefore, the image quality can be effectively improved by improving the analog-to-digital conversion precision of the brightness interval in which the user is interested, the image details of the key area concerned by the user can be better shown, and the analog-to-digital conversion precision can be reduced for other brightness intervals to improve the analog-to-digital conversion speed so as to meet the requirement on the shooting speed. And finally, carrying out normalization processing on the analog-to-digital conversion curves with different analog-to-digital conversion precisions in the graph 4 and transmitting the normalized analog-to-digital conversion curves to an external platform, wherein the normalization processing is to convert linear parts with different slopes of the curve shown in the graph 4 into a straight line with the same slope.
Example two
Fig. 5 illustrates an analog-to-digital conversion graph of an image sensor according to another embodiment of the present invention.
In the image sensor application of the embodiment, firstly, the environment information or the custom mode is obtained through the image sensor or the external platform, and then one or more luminance intervals in which a user is interested are obtained from the original image information according to the environment information or the custom mode, for example, according to any one or a combination of histogram information, environment luminance, image luminance, f-number, exposure time and sensitivity, or according to the custom mode, the outdoor scene shooting scene with high dynamic is determined, in this case, the region in which the user is interested is usually a scene part with lower luminance, and the DN value of the corresponding luminance interval is about between 20 and 100, for this requirement, the analog-to-digital conversion module can be controlled by the digital signal to generate a nonlinear ramp signal, so as to improve the analog-to-digital conversion accuracy of the DN value between 20 and 100, and better show the image details of the scene region in which the user is interested, in the highlight section with the DN value of more than 100, the analog-to-digital conversion precision is reduced to increase the analog-to-digital conversion speed, so as to achieve the purpose of increasing the frame rate of the image sensor and increasing the shooting speed under the same data bit width, and an analog-to-digital conversion curve with different analog-to-digital conversion precisions is obtained as shown in fig. 5, wherein the slope of the curve in the 20-100 brightness section is smaller than that in the highlight section with more than 100, that is, the 20-100 brightness section has the analog-to-digital conversion precision in the highlight section with more than 100. Preferably, the analog-to-digital conversion curves with different analog-to-digital conversion accuracies shown in fig. 3 are continuously derivable curves, and specifically, since the analog-to-digital conversion curve shown in fig. 5 includes a linear portion in a 20-100 luminance interval and a non-linear portion in a 100 or more highlight interval (the non-linear portion means that the quantization step size in the 100 or more highlight interval is gradually changed), the quantization step size at the critical point of the 20-100 luminance interval and the 100 or more highlight interval is also gradually changed rather than suddenly changed, that is, the transition between the luminance intervals with different analog-to-digital conversion accuracies is smooth rather than sharp, so as to further improve the imaging effect. In addition, for the non-linear part of the highlight interval above 100, linear calibration is required first, then normalization processing is required, and then the non-linear part is transmitted to an external platform.
In summary, according to the method for optimizing the performance of the image sensor based on the adaptive programmable analog-to-digital conversion of the present invention, one or more luminance intervals in which a user is interested are obtained from original image information according to environment information or a user-defined mode, so as to improve the analog-to-digital conversion precision of the luminance intervals in which the user is interested, and the analog-to-digital conversion precision is reduced for other luminance intervals to improve the analog-to-digital conversion speed, thereby taking into account the conversion precision and the conversion speed of the whole analog-to-digital conversion process, so that the analog-to-digital conversion relationship is flexible and changeable, a high dynamic environment in the natural world can be better restored, the method is closer to a response mechanism of human eyes, or based on some machine vision requirements, the favorite requirements of different scenes and the user are met, and the overall performance of the image sensor is improved. The method for optimizing the performance of the image sensor based on the self-adaptive programmable analog-to-digital conversion has wide application range and strong adaptability, and can be applied to both high-end and low-end image sensors.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive. Furthermore, it will be obvious that the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. Several elements recited in the apparatus claims may also be implemented by one element. The terms first, second, etc. are used to denote names, but not any particular order.
Claims (6)
1. A method for optimizing image sensor performance based on adaptive programmable analog to digital conversion, comprising:
acquiring environmental information or a user-defined mode;
and acquiring one or more brightness intervals interested by the user from the original image information according to the environment information or the user-defined mode, and improving the analog-to-digital conversion precision of the interested brightness intervals so as to improve the image quality.
2. The method of claim 1, wherein analog-to-digital conversion curves with different analog-to-digital conversion precisions are generated according to the environmental information or the custom mode, and the analog-to-digital conversion curves with different analog-to-digital conversion precisions are normalized and transmitted to an external platform.
3. The method of claim 2, wherein the analog-to-digital conversion curves with different analog-to-digital conversion accuracies include a non-linear portion, and the non-linear portion is normalized after being linearly calibrated.
4. The method for optimizing image sensor performance based on adaptively programmable analog-to-digital conversion of claim 3, wherein the analog-to-digital conversion curves with different analog-to-digital conversion accuracies are continuously derivable curves.
5. The method of claim 1, wherein the environment information comprises any one or a combination of histogram information, environment brightness, image brightness, f-number, exposure time, sensitivity.
6. The method of optimizing image sensor performance based on adaptive programmable analog-to-digital conversion of claim 1, wherein the custom mode comprises a license plate mode, a portrait mode, a night mode, a sky mode.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010919925.1A CN114143481A (en) | 2020-09-04 | 2020-09-04 | Method for optimizing performance of image sensor based on self-adaptive programmable analog-to-digital conversion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010919925.1A CN114143481A (en) | 2020-09-04 | 2020-09-04 | Method for optimizing performance of image sensor based on self-adaptive programmable analog-to-digital conversion |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114143481A true CN114143481A (en) | 2022-03-04 |
Family
ID=80438259
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010919925.1A Pending CN114143481A (en) | 2020-09-04 | 2020-09-04 | Method for optimizing performance of image sensor based on self-adaptive programmable analog-to-digital conversion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114143481A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6734904B1 (en) * | 1998-07-23 | 2004-05-11 | Iteris, Inc. | Imaging system and method with dynamic brightness control |
| CN1835404A (en) * | 2005-03-17 | 2006-09-20 | 北京思比科微电子技术有限公司 | Segmental linear module converter |
| CN101800858A (en) * | 2009-02-06 | 2010-08-11 | 佳能株式会社 | Picture pick-up device and control method thereof |
| CN104767909A (en) * | 2015-04-15 | 2015-07-08 | 中国电子科技集团公司第四十四研究所 | Histogram equalization analog-digital conversion circuit of image sensor and method |
| US20170094113A1 (en) * | 2015-09-24 | 2017-03-30 | Canon Kabushiki Kaisha | Signal processing apparatus, image processing apparatus, and signal processing method |
| CN107154803A (en) * | 2016-03-02 | 2017-09-12 | 佳能株式会社 | Signal handling equipment, signal processing method and picture pick-up device |
| CN107181922A (en) * | 2016-03-10 | 2017-09-19 | 佳能株式会社 | Picture pick-up device and its control method |
| CN107707845A (en) * | 2017-10-10 | 2018-02-16 | 德淮半导体有限公司 | For the method for imaging sensor, imaging sensor and imaging device |
-
2020
- 2020-09-04 CN CN202010919925.1A patent/CN114143481A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6734904B1 (en) * | 1998-07-23 | 2004-05-11 | Iteris, Inc. | Imaging system and method with dynamic brightness control |
| CN1835404A (en) * | 2005-03-17 | 2006-09-20 | 北京思比科微电子技术有限公司 | Segmental linear module converter |
| CN101800858A (en) * | 2009-02-06 | 2010-08-11 | 佳能株式会社 | Picture pick-up device and control method thereof |
| CN104767909A (en) * | 2015-04-15 | 2015-07-08 | 中国电子科技集团公司第四十四研究所 | Histogram equalization analog-digital conversion circuit of image sensor and method |
| US20170094113A1 (en) * | 2015-09-24 | 2017-03-30 | Canon Kabushiki Kaisha | Signal processing apparatus, image processing apparatus, and signal processing method |
| CN107154803A (en) * | 2016-03-02 | 2017-09-12 | 佳能株式会社 | Signal handling equipment, signal processing method and picture pick-up device |
| CN107181922A (en) * | 2016-03-10 | 2017-09-19 | 佳能株式会社 | Picture pick-up device and its control method |
| CN107707845A (en) * | 2017-10-10 | 2018-02-16 | 德淮半导体有限公司 | For the method for imaging sensor, imaging sensor and imaging device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7415203B2 (en) | Method and apparatus for controlling automatic exposure in a camera | |
| CN110619593B (en) | Double-exposure video imaging system based on dynamic scene | |
| CN110536051B (en) | Image pickup apparatus, control method thereof, and storage medium | |
| KR101812807B1 (en) | A method of adaptive auto exposure contol based upon adaptive region's weight | |
| US7742637B2 (en) | Apparatus, method, and program for taking an image, and apparatus, method, and program for processing an image | |
| KR20010033768A (en) | Vehicle vision system | |
| CN111064904A (en) | Dark light image enhancement method | |
| CA2177517C (en) | Ambient light automatic gain control for electronic imaging cameras and the like | |
| CN101115149B (en) | Image processing method | |
| JP5005731B2 (en) | Camera apparatus and exposure control method | |
| KR102377318B1 (en) | High Dynamic Range technique for light exposure compensation and image quality improvement | |
| US20190108626A1 (en) | Image processing apparatus, image processing method, and storage medium | |
| CN104869319B (en) | Image treatment method and image processor | |
| CN110166711A (en) | Image processing method, device, electronic equipment and storage medium | |
| CN114666511B (en) | Method and device for automatically obtaining optimal exposure value on tunable spectrum camera | |
| CN116416122A (en) | Image processing method and related device | |
| US20040179134A1 (en) | Portable terminal device and method and program for varying light illuminance used therein | |
| CN113822826A (en) | Low-illumination image brightness enhancement method | |
| CN111724316B (en) | Method and apparatus for processing high dynamic range image | |
| JP2009124264A (en) | Image processing apparatus and image processing method | |
| CN114143481A (en) | Method for optimizing performance of image sensor based on self-adaptive programmable analog-to-digital conversion | |
| FR2838016A1 (en) | PROCESS FOR REAL-TIME PROCESSING OF AN IMAGE REPRESENTATIVE SIGNAL | |
| EP0743787A3 (en) | An image pickup apparatus for obtaining a color image over a wider luminance range | |
| CN114467110A (en) | Image dynamic range processing method and device | |
| CN102804227B (en) | Use the lenses attenuate correct operation of the value corrected based on monochrome information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |