CN113709393A - Calibration curve acquisition method and calibration method for optical dark area of image sensor - Google Patents
Calibration curve acquisition method and calibration method for optical dark area of image sensor Download PDFInfo
- Publication number
- CN113709393A CN113709393A CN202110959259.9A CN202110959259A CN113709393A CN 113709393 A CN113709393 A CN 113709393A CN 202110959259 A CN202110959259 A CN 202110959259A CN 113709393 A CN113709393 A CN 113709393A
- Authority
- CN
- China
- Prior art keywords
- image sensor
- temperature
- optical dark
- dark area
- brightness
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N25/63—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to dark current
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/70—SSIS architectures; Circuits associated therewith
- H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
- H04N25/77—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components
Abstract
The invention discloses a calibration curve acquisition method and a calibration method for an optical dark area of an image sensor. The method for acquiring the calibration curve of the optical dark area of the image sensor comprises the following steps: selecting a partial image sensor from all image sensors to be calibrated as a sample image sensor; counting the brightness values of optical dark areas of the sample image sensors in different temperature intervals, and selecting a high-quality image sensor from the sample image sensors; performing curve fitting on the optical dark area brightness values of the high-quality image sensor in all temperature intervals to obtain a temperature brightness fitting curve of the high-quality image sensor; measuring the optical dark area brightness values of all image sensors to be calibrated at key temperature points; and adjusting the temperature brightness fitting curve according to the brightness value of the optical dark area of the key temperature point to obtain a calibration curve of the optical dark area of the image sensor. The embodiment of the invention can realize the dynamic calibration of the optical dark area of the image sensor according to the operating temperature and reduce the acquisition cost of the calibration curve.
Description
Technical Field
The embodiment of the invention relates to the technical field of image processing, in particular to a calibration curve acquisition method and a calibration method for an optical dark area of an image sensor.
Background
In a camera or the like, an image sensor is generally provided to convert light into an electrical signal for imaging. The image sensor includes pixels (pixels) arranged in an array, and due to the influence of factors such as the pixel characteristics of the image sensor itself and circuit offset, the image sensor may also present a certain brightness value when no external light is used as an input signal, that is, there is an Optical dark (OB), which may cause image quality degradation and color balance error in a dark environment from the viewpoint of image representation.
Currently, the prior art generally adopts a technology of Optical dark field calibration (OBC), also called Black Level Compensation (BLC), to solve the image quality problem caused by OB. Among them, the Image Signal Processor (ISP) needs to perform post-Image processing according to the OB value (i.e., the optical dark area brightness value) to optimize the Image quality. However, with the change of the operating temperature, the Dark current (Dark Noise) inside the image sensor changes, which causes the shift of the OB value, further affects the image processing effect, and reduces the image quality. In the prior art, the influence of temperature on the calibration of an optical dark area is not considered, and the image quality is difficult to ensure; even if the temperature influence is considered, the method for acquiring the calibration curve of each image sensor relative to the temperature in the prior art has the problem of high industrial application cost aiming at the application requirements of a large number of image sensors.
Disclosure of Invention
The embodiment of the invention provides a calibration curve acquisition method and a calibration method for an optical dark area of an image sensor, which are used for realizing dynamic calibration of the optical dark area of the image sensor according to the operating temperature and reducing the acquisition cost of the calibration curve.
In a first aspect, an embodiment of the present invention provides a method for acquiring a calibration curve of an optical dark area of an image sensor, including:
selecting a partial image sensor from all image sensors to be calibrated as a sample image sensor;
counting the optical dark area brightness values of the sample image sensor in different temperature intervals, and selecting a high-quality image sensor from the sample image sensor;
performing curve fitting on the optical dark space brightness values of the high-quality image sensor in all the temperature intervals to obtain a temperature brightness fitting curve of the high-quality image sensor;
measuring the optical dark area brightness values of all the image sensors to be calibrated at key temperature points; and adjusting the temperature brightness fitting curve according to the optical dark space brightness value of the key temperature point to obtain a calibration curve of the optical dark space of the image sensor.
Optionally, counting optical dark-area brightness values of the sample image sensor in different temperature intervals, including:
the sample image sensor continuously operates in a completely black environment; the sample image sensor acquires an original image data graph at intervals of preset time;
according to the reading of a temperature register arranged in the image sensor, matching a temperature interval corresponding to the temperature when the original image data graph is obtained;
and selecting an interested area from the original image data map, and calculating an average optical dark area brightness value of the interested area as an optical dark area brightness value corresponding to the temperature interval where the original image data map is obtained.
Optionally, if the temperatures when the original image data maps are acquired for multiple times are in the same temperature interval, taking the average value of the average optical dark brightness values of the region of interest calculated for multiple times as the optical dark brightness value corresponding to the temperature interval.
Optionally, selecting a good quality image sensor from the sample image sensors includes:
acquiring normal distribution of optical dark space brightness values of all the sample image sensors in any temperature interval;
and selecting the sample image sensor positioned in a set area in the normal distribution as the high-quality image sensor.
Optionally, after selecting the good image sensor, the method further includes: selecting an intersection of the high-quality image sensors selected from all the temperature intervals to obtain a global high-quality image sensor;
correspondingly, performing curve fitting on the optical dark area brightness values of the high-quality image sensor in all the temperature intervals to obtain a temperature brightness fitting curve of the high-quality image sensor, including:
and performing curve fitting on the optical dark space brightness values of any global high-quality image sensor in all the temperature intervals to obtain a temperature brightness fitting curve of the high-quality image sensor.
Optionally, the critical temperature points include: a minimum operating temperature of the image sensor, a maximum operating temperature of the image sensor, and a room temperature.
Optionally, before selecting a part of image sensors from all image sensors to be calibrated as sample image sensors, the method further includes:
checking whether the factory-measured optical dark area brightness value of the image sensor is qualified or not; if the image sensor is qualified, the step of selecting the sample image sensor is executed; and if the image sensor is unqualified, rejecting the unqualified image sensor.
In a second aspect, an embodiment of the present invention provides a method for calibrating an optical dark area of an image sensor, including:
acquiring the current temperature of the image sensor;
obtaining a current optical dark area brightness value of the image sensor according to the current temperature and the calibration curve; the calibration curve is obtained according to the calibration curve obtaining method for the optical dark area of the image sensor provided by any embodiment of the invention;
obtaining an optical dark space brightness calibration value corresponding to the current temperature according to the current optical dark space brightness value and a factory-measured optical dark space brightness value of the image sensor;
and acquiring original image data of the image sensor, and calibrating the original image data by adopting the optical dark area brightness calibration value.
Optionally, acquiring the current temperature of the image sensor includes:
and acquiring the current temperature from a temperature register built in the image sensor according to the set sampling frame rate.
Optionally, the optical dark space brightness calibration value corresponding to the current temperature is calculated by the following formula:
OBdst＝OBbase*α+OBtemp*β+shift
wherein, OBdstFor the optical dark space brightness calibration value, OBbaseIs the brightness value of the factory-measured optical dark area, alpha is the weight of the brightness value of the factory-measured optical dark area, OBtempAnd the current optical dark area brightness value is taken as the brightness value of the current optical dark area, beta is the weight of the brightness value of the current optical dark area, and shift is a retention parameter.
According to the method for acquiring the calibration curve of the optical dark area of the image sensor, the brightness value of the optical dark area in the whole temperature interval is counted by selecting a part of image sensors to be calibrated as sample image sensors; and selecting a high-quality image sensor from the sample image sensors, and performing curve fitting to obtain a temperature and brightness fitting curve, so that the measurement and calculation processes can be effectively simplified. Meanwhile, the temperature and brightness fitting curve is not used as a final calibration curve in the embodiment, the temperature and brightness fitting curve is adjusted for each image sensor to be calibrated, so that the finally obtained calibration curve accords with the self working characteristics of each image sensor to be calibrated, and the uniqueness and the accuracy of the calibration curve are ensured. And when the temperature brightness fitting curve is subjected to targeted adjustment, only the optical dark brightness value of the key temperature point needs to be acquired, so that the measurement and calculation processes are further simplified. Therefore, compared with the prior art, the embodiment of the invention can ensure the accuracy of the calibration curve, so that the image sensor can realize the dynamic calibration of the optical dark area according to the actual operation temperature and the calibration curve, and the acquisition cost of the calibration curve is reduced.
Drawings
Fig. 1 is a schematic flowchart of a calibration curve obtaining method for an optical dark area of an image sensor according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a statistical method for optical dark luminance values of a sample image sensor according to an embodiment of the present invention;
fig. 3 is a schematic flowchart of a method for selecting a good quality image sensor according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating normal distribution of brightness values of optical dark areas of a sample image sensor in different temperature ranges according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a process for fitting a temperature-luminance fitting curve according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a prior art image sensor optical dark area calibration process;
FIG. 7 is a method for calibrating an optical dark area of an image sensor according to an embodiment of the present invention;
fig. 8 is a diagram illustrating another method for calibrating an optical dark area of an image sensor according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
As described in the background, temperature variations can cause shifts in the optical dark area brightness values (OB values) of the image sensor, and the factors that cause shifts in OB values and the resulting effects are described below.
Record the variation of OB value, or shift value
The influence factor can be referred to the following formula:
wherein resolution represents resolution, FPS represents frame rate, capabilities represents functions of an Image Signal Processor (ISP), time represents usage time, δ represents other factors, temperature represents temperature, and IQ represents image quality. As can be seen from equation 1: variation of OB value
Proportional to the resolution, the frame rate FPS, the capabilities of the image signal processor ISP, the time of use, other factors δ, and the temperature. In particular, resolution geometry has increased, for example, one hundred million pixel CMOS image sensors have been introduced on the cell phone market; the frame rate FPS is improved, for example, the frame rate of an image sensor in the mobile phone market is increased to 240 FPS; the capabilities of the image signal processor are increased; the use time is increased, and application scenes such as Vlog, code scanning, face recognition and the like enable the use frequency and the duration of the camera to be longer and longer; as well as other factors, delta, increase results in increased power consumption and increased temperature of the image sensor. The rapid rise in the temperature parameter of the image sensor may cause a drift in the OB value. Variation of image quality IQ and OB values
In inverse proportion; variation of OB value
The larger the image quality IQ is, the worse. Therefore, the effect of temperature variation on the optical dark area calibration needs to be considered.
In the prior art, manufacturers such as mobile phones and cameras generally purchase a large number (e.g., millions) of image sensors at a time for production. Manufacturers usually measure OB values at different temperatures for each image sensor, so as to obtain a corresponding relationship between the temperature and the OB value of each image sensor. But such an operation would be time and cost intensive.
Based on the above research, the embodiment of the invention provides a calibration curve obtaining method for an optical dark area of an image sensor, which can be applied to the obtaining requirement of manufacturers on the mapping relation between the brightness value of the optical dark area and the temperature of a large number of image sensors and can be realized through programming; the image sensor may be a CMOS or the like type image sensor. Fig. 1 is a schematic flowchart of a calibration curve obtaining method for an optical dark area of an image sensor according to an embodiment of the present invention. Referring to fig. 1, the method for acquiring the calibration curve of the optical dark area of the image sensor comprises the following steps:
and S110, selecting partial image sensors from all the image sensors to be calibrated as sample image sensors.
The OB value of the all-temperature interval is measured only by sampling the image sensor to be calibrated, so that the overall acquisition time and cost of all calibration curves of the image sensor to be calibrated can be effectively reduced. Illustratively, one or several image sensors may be randomly drawn from each bin of image sensors as a sample.
And S120, counting the optical dark area brightness values of the sample image sensors in different temperature intervals, and selecting a high-quality image sensor from the sample image sensors.
The division of the temperature interval may be based on the operating temperature range of the image sensor. For example, the image Sensor is provided with a Temperature Register (Sensor Temperature Register) capable of sensing the Temperature inside the Sensor, and the working Temperature range of the image Sensor can be the range given by the data table. For example, the working temperature range may be divided into n temperature intervals according to the precision required by a specific application scenario; the temperature ranges encompassed by the respective temperature intervals may be equal or unequal.
Illustratively, the number of the good image sensors can be one or more, and the good image sensors can be selected according to the optical dark area brightness values of the sample image sensor in each temperature interval; for example, a plurality of sample image sensors with better consistency are selected as good-quality image sensors.
S130, performing curve fitting on the optical dark area brightness values of the high-quality image sensor in all temperature intervals to obtain a temperature brightness fitting curve of the high-quality image sensor.
The curve fitting can be performed by polynomial fitting, exponential fitting, gaussian fitting, interpolation or the like according to the distribution of the OB values of the high-quality image sensor. Illustratively, the curve fitting process may be only for any one good image sensor to reduce computation time. Or, the curve fitting process can be directed at all high-quality image sensors, so that the fitting result is more accurate, and the influence of accidental factors is eliminated; specifically, all good quality image sensors may be fitted uniformly with OB values in different temperature ranges; or after respective fitting curves of all the high-quality image sensors are formed, fitting the plurality of fitting curves again to obtain a final temperature brightness fitting curve; and then or firstly obtaining the mean value of the OB values of the high-quality image sensor in each temperature interval, and then fitting the mean values in different temperature intervals.
S140, measuring the optical dark area brightness values of all image sensors to be calibrated at key temperature points; and adjusting the temperature brightness fitting curve according to the brightness value of the optical dark area of the key temperature point to obtain a calibration curve of the optical dark area of the image sensor.
The step is carried out for all the image sensors to be calibrated, and whether the image sensors to be calibrated are sample image sensors or high-quality image sensors is not distinguished. Therefore, the selection step can be omitted, and the calibration curve acquisition process is simplified. If an image sensor to be calibrated is just a good image sensor for curve fitting in S130, the temperature brightness fitting curve is substantially the calibration curve. The number of the key temperature points and the temperature value can be selected according to requirements.
According to the method for acquiring the calibration curve of the optical dark area of the image sensor, the brightness value of the optical dark area in the whole temperature interval is counted by selecting a part of image sensors to be calibrated as sample image sensors; and selecting a high-quality image sensor from the sample image sensors, and performing curve fitting to obtain a temperature and brightness fitting curve, so that the measurement and calculation processes can be effectively simplified. Meanwhile, the temperature and brightness fitting curve is not used as a final calibration curve in the embodiment, the temperature and brightness fitting curve is adjusted for each image sensor to be calibrated, so that the finally obtained calibration curve accords with the self working characteristics of each image sensor to be calibrated, and the uniqueness and the accuracy of the calibration curve are ensured. And when the temperature brightness fitting curve is subjected to targeted adjustment, only the optical dark brightness value of the key temperature point needs to be acquired, so that the measurement and calculation processes are further simplified. Therefore, the embodiment of the invention can ensure the accuracy of the calibration curve, so that the image sensor can realize the dynamic calibration of the optical dark area according to the actual operation temperature and the calibration curve, and the acquisition cost of the calibration curve is reduced, thereby being convenient for popularization and application.
Fig. 2 is a flowchart illustrating a statistical method for optical dark luminance values of a sample image sensor according to an embodiment of the present invention. Referring to fig. 2, the present embodiment refines the step of counting the optical dark luminance values of the sample image sensor in different temperature intervals, and specifically includes the following steps:
s210, continuously operating the sample image sensor in a completely black environment; and the sample image sensor acquires an original image data map once every preset time.
The sample image sensor can operate in a specific mode (sensor register setting mode) in a completely black environment, so that interference of incident light rays is avoided, and an OB value obtained according to an original image data map is more accurate. As the image sensor continues to operate, the temperature will rise as the operating time rises; based on the OB value, the OB value corresponding to each temperature interval can be obtained by arranging the original image data acquired at preset time intervals.
And S220, matching a temperature interval corresponding to the temperature when the original image data graph is obtained according to the reading of a temperature register arranged in the image sensor.
The reading of the built-in temperature register can be read by using an IIC protocol, and the read data is converted into the degree centigrade information according to a conversion formula corresponding to the image sensor.
S230, selecting an interested area from the original image data map, and calculating the average optical dark area brightness value of the interested area as the optical dark area brightness value corresponding to the temperature interval where the original image data map is obtained.
The region of interest (ROI) can be flexibly selected according to requirements, for example, the center, the edge, or the four corners of the image sensor are selected. The pixels in the image sensor may be at BayAn er format arrangement, wherein the image can be coded in a YUV format; the average optical dark brightness value (marked as OB) can be calculated according to the Y value (namely, the brightness value) of each pixel point in the region of interestROI). The calculation formula is as follows:
wherein, avgYROIRepresenting the average luminance value of a region of interest in a Pixel array of an image sensor, frameLength representing the number of pixels of the region of interest in the vertical direction, lineLength representing the number of pixels of the region of interest in the horizontal direction, Pixeli,jThe intensity value of the pixel corresponding to the (i, j) coordinate point in the array of pixels representing the region of interest.
The embodiment realizes the statistics of the optical dark brightness values of the sample image sensor in different temperature intervals through S210-S230. It should be noted that the regions of interest of all the sample image sensors need to be selected according to a uniform standard, so as to facilitate the subsequent steps.
In addition to the above embodiments, alternatively, if the temperatures when the original image data maps are acquired a plurality of times are in the same temperature interval, the average value of the average optical dark brightness values of the region of interest calculated a plurality of times is used as the optical dark brightness value corresponding to the temperature interval.
Fig. 3 is a flowchart illustrating a method for selecting a good image sensor according to an embodiment of the present invention. Referring to fig. 3, the present embodiment refines the step of selecting a good-quality image sensor from the sample image sensors, and specifically includes the following steps:
s310, acquiring normal distribution of optical dark space brightness values of all sample image sensors in any temperature interval.
And S320, selecting the sample image sensor positioned in the set area in normal distribution as a high-quality image sensor.
Wherein, the normal distribution of OB values of the sample image sensor in different temperature intervals is shown in fig. 4, and fig. 4 exemplarily shows the normal distribution of optical dark area brightness values of all sample image sensors in three temperature intervals. For example, the operating temperature range recorded in the data table of the image sensor is-30-70 ℃, the operating temperature range can be divided into three temperature intervals of [ -30,0), [0,30 ], and [30,70], wherein the interval 1 represents the interval of [ -30,0), the interval 2 represents the interval of [0,30), and the interval 3 represents the interval of [30,70 ]; alternatively, the operating temperature range may be equally divided into 10 temperature intervals, interval 1 representing the [ -30, -20) interval, interval 2 representing the [20,30) interval, and interval 3 representing the [60,70] interval.
Referring to fig. 4, the abscissa represents the optical dark luminance value and the ordinate represents the probability. The unit of the brightness value of the optical dark area is code, and the meaning of the brightness value is the brightness represented by one coding unit after the lowest to the highest brightness of the pixel is equally divided according to a rule; for example, for 10-bit encoding, the luminance range of each pixel is divided equally from 0 to 1023. In fig. 4, the most sparsely dotted shaded portion and the broken line indicate normal distributions of the luminance values of the optical dark regions in the section 1, where μ is 62 and the standard deviation σ is 1, which are mathematically expected; the sample image sensor located in area a1 may be selected as the good quality image sensor for temperature interval 1. The sparser dotted shaded portion and the solid line represent a normal distribution of the luminance values of the optical dark regions in the interval 2, and the mathematical expectation μ is 64 and the standard deviation σ is 1; the sample image sensor located in area a2 may be selected as the good quality image sensor for temperature interval 2. The most densely dotted shaded portion and the dashed line represent a normal distribution of the luminance values of the optical dark regions in the section 3, and the mathematical expectation μ is 66, and the standard deviation σ is 1; the sample image sensor located in area a3 may be selected as the good quality image sensor for temperature interval 3.
For example, a central region near a mathematical expectation in a normal distribution may be used as a set region; the high-quality image sensors in each temperature interval can be multiple, and the high-quality image sensors in different temperature intervals can be different.
On the basis of the foregoing embodiments, optionally, the step of selecting a good-quality image sensor further includes:
s330, taking intersection of the high-quality image sensors selected from all the temperature intervals to obtain a global high-quality image sensor.
The high-quality image sensors selected in each temperature interval are not completely the same, and if only the measurement data of one high-quality image sensor is selected randomly for curve fitting, the random risk is high; if curve fitting is performed on the measured data of all the high-quality image sensors, the calculation process is complex. In this embodiment, by taking an intersection of the high-quality image sensors in all temperature intervals, a plurality of global high-quality image sensors that perform well and have good consistency in each temperature interval can be obtained. Then, the measured data of any global high-quality image sensor is selected for fitting until a better temperature brightness fitting curve can be obtained, and the process of curve fitting can be simplified.
Correspondingly, the curve fitting is carried out on the optical dark area brightness values of the high-quality image sensor in all temperature intervals to obtain a temperature brightness fitting curve of the high-quality image sensor, and the curve fitting method comprises the following steps: and performing curve fitting on the optical dark space brightness values of any global high-quality image sensor in all temperature intervals to obtain a temperature brightness fitting curve of the high-quality image sensor.
The OB value versus temperature for a set of global quality image sensors and other sample image sensors is given as an example in table 1.
TABLE 1
In table 1, the working temperature ranges of the batch of image sensors are-30-70 ℃, each temperature interval is divided every ten degrees centigrade, and the temperature interval of 70 ℃ is an eleventh temperature interval; the lowest temperature value in each temperature interval is used as the OB representative value in the interval. Golden1, Golden2, and Golden3 represent three global quality image sensors; unit1 and Unit2 represent two sample image sensors other than a global good image sensor; avg _ temp represents the optical dark area brightness mean value of different image sensors in each temperature interval; avg _ OB represents the average value of the optical dark area brightness of a certain image sensor at all temperature intervals. Counting the measurement data of all sample image sensors as above, and obtaining the overall change trend of the brightness values of the optical dark areas of the image sensors and the approximate distribution of OB values in different temperature intervals by comparing the Avg _ temp; the quality of the single image sensor itself can also be judged by comparing Avg _ OB.
Fig. 5 is a schematic diagram of a fitting process of a temperature brightness fitting curve according to an embodiment of the present invention. Referring to fig. 5, the abscissa represents temperature and the ordinate represents optical dark area brightness value. Illustratively, with Golden1 in table 1 as a fitting object, a binomial fit can be employed to the OB value-temperature characteristic of the image sensor.
After the temperature brightness fitting curve is obtained, the temperature brightness fitting curve can be adjusted according to the OB value of the key temperature point of the image sensor to be calibrated, and the calibration curve of the optical dark area of the image sensor to be calibrated is obtained. For example, key temperature points include: the temperature measurement device comprises three temperature values of the lowest working temperature of the image sensor, the highest working temperature of the image sensor and the room temperature. The room temperature is 25 ℃ or 27 ℃ as the daily use environment temperature of the image sensor.
Illustratively, still taking the temperature intensity fitting curve obtained in FIG. 5 as an example, suppose that the OB value of an image sensor to be calibrated is 61code at-30 ℃, 65code at 27 ℃ and 67code at 70 ℃. Then, the left end of the temperature brightness fitting curve is adjusted down adaptively, and the middle part of the temperature brightness fitting curve is adjusted up adaptively, so that the calibration curve of the image sensor to be calibrated can be obtained.
On the basis of the foregoing embodiments, before selecting a partial image sensor from all image sensors to be calibrated as a sample image sensor, the method further includes:
checking whether the factory-measured optical dark area brightness value of the image sensor is qualified or not; if the image sensor is qualified, executing the step of selecting the sample image sensor; and if the image sensor is unqualified, rejecting the unqualified image sensor.
The factory-measured optical dark area brightness value is a factory-measured reference value of the optical dark area brightness recorded in the specification of the image sensor when the optical dark area brightness value leaves a factory; that is, the manufacturer (module factory) of the image sensor masks the pixel matrix area at the edge portion at the time of designing, and measures the OB value of the masked black portion obtained in the laboratory environment at the factory.
The step of verifying whether the factory-measured optical dark area brightness value of the image sensor is qualified specifically comprises the following steps:
completely shading the image sensor, and shooting an original image data graph (raw graph); selecting an area of interest, and calculating an initial OB value according to a formula 2; comparing the initial OB value with the brightness value of the factory-measured optical dark area, and if the difference value of the initial OB value and the brightness value of the factory-measured optical dark area is within an error range, determining that the image sensor is qualified; if the difference value of the two is out of the error range, the image sensor is unqualified and can be returned to the factory for processing.
The calibration step can be carried out at room temperature, and the factory-measured optical dark area brightness value is actually obtained at room temperature, so that the calibration result can be more accurate. In this embodiment, the image sensor that is qualified for verification may be referred to as a sensor to be calibrated.
Fig. 6 is a schematic diagram of a prior art image sensor optical dark area calibration process. Referring to fig. 6, in the prior art, no matter how long the image sensor continues to operate, the image signal processor ISP always uses the same optical dark area luminance Value OBC Value as a calibration basis, and subtracts the OBC Value from the original image data map (raw map) to obtain an intermediate process data map (process raw map) for subsequent processing. That is to say, the influence of temperature on OBC Value is not considered in the prior art, the influence of temperature rise on image quality during continuous operation cannot be solved, and the method is not suitable for image sensor products with large deviation between application environment and room temperature.
Based on the above research, an embodiment of the present invention further provides a calibration method for an Optical dark area of an image sensor, which is based on a calibration curve obtained by the calibration curve obtaining method for an Optical dark area of an image sensor according to any embodiment of the present invention, and may dynamically adjust a calibration value of the Optical dark area by monitoring a value of a temperature register built in an image sensor chip, which may be referred to as a Dynamic Optical Black Compensation (DOBC). The calibration method is particularly suitable for image sensor products with long continuous operation time or large deviation of application environment and room temperature. Fig. 7 is a method for calibrating an optical dark area of an image sensor according to an embodiment of the present invention. Referring to fig. 7, the method for calibrating the optical dark area of the image sensor includes:
and S410, acquiring the current temperature of the image sensor.
Wherein, the current temperature can be obtained by reading conversion of a temperature register built in the image sensor.
And S420, obtaining the current optical dark brightness value of the image sensor according to the current temperature and the calibration curve.
And S430, obtaining an optical dark space brightness calibration value corresponding to the current temperature according to the current optical dark space brightness value and the factory-measured optical dark space brightness value of the image sensor.
The optical dark area brightness calibration value can be a weighted sum of the current optical dark area brightness value and the factory-measured optical dark area brightness value, and the weights of the current optical dark area brightness value and the factory-measured optical dark area brightness value can be selected according to actual requirements.
And S440, acquiring original image data of the image sensor, and calibrating the original image data by adopting an optical dark area brightness calibration value.
The original image data can be obtained from an original image data map of the image sensor, the intermediate data can be obtained by subtracting the optical dark area brightness calibration value from the original image data, and the final image data can be obtained by encoding the intermediate data according to a predetermined format and other operations. The final image data may be embodied in RGB or YUV format.
According to the calibration method for the optical dark area of the image sensor, provided by the embodiment of the invention, the current temperature is obtained in real time, and the corresponding optical dark area brightness calibration value is calculated, so that the negative influence on the image effect due to the inaccuracy of the OB value obtained by the image signal processor can be reduced. In addition, the internal operating temperature is obtained by directly adopting a temperature register in the image sensor, so that the hardware design can be simplified, and the circuit area and the cost can be saved; and compare in traditional external temperature sensor and can only measure image sensor surface temperature, the measurement of this embodiment is more accurate. And in the calculation process of the optical dark space brightness calibration value, the current optical dark space brightness value and the factory-measured optical dark space brightness value are considered at the same time, so that negative effects caused by simplified calculation of a calibration curve can be effectively avoided, the weights of the two factors can be flexibly adjusted, and the calibration flexibility is improved.
Fig. 8 is a diagram illustrating another method for calibrating an optical dark area of an image sensor according to an embodiment of the present invention. The acquisition of the calibration curve and the implementation of the calibration process are integrated in fig. 8, which summarizes the overall flow of the implementation of the calibration method.
Referring to fig. 8, the calibration method includes:
and S510, carrying out temperature division according to the working temperature range of the image sensor to form a plurality of temperature intervals.
S520, checking whether the factory-measured optical dark area brightness value of the image sensor is qualified or not; if yes, go to S530; if not, go to S540.
S530, selecting a sample image sensor, and counting the optical dark area brightness values of the sample image sensor in different temperature intervals.
And S540, rejecting unqualified image sensors.
S550, selecting a high-quality image sensor from the sample image sensors, and counting temperature and brightness corresponding data of the high-quality image sensor.
And S560, measuring the optical dark brightness values of all the image sensors to be calibrated at the key temperature points.
S570, writing the optical dark brightness values of the image sensor to be calibrated at all key temperature points and the optical dark brightness values of any high-quality image sensor in all temperature intervals into a storage space of the image sensor to be calibrated.
Wherein, the memory of each image sensor to be calibrated may include an OTP partition address as a storage space of OB values. The storage space can be divided into two subspaces, and one subspace is used for storing the optical dark area brightness values of the image sensor to be calibrated at all key temperature points; the other subspace is used for storing the optical dark brightness values of any one good image sensor in all temperature intervals.
And S580, calculating a temperature brightness fitting curve of the high-quality image sensor, and adjusting the temperature brightness fitting curve according to the optical dark space brightness value of the image sensor to be calibrated at the key temperature point to obtain a calibration curve.
The image signal processor ISP may read OB values of the high-quality image sensor and the image sensor to be calibrated from the storage space through the IIC protocol, and use the OB values to calculate the calibration curve.
And S590, acquiring the current temperature from the built-in temperature register of the image sensor according to the set sampling frame rate.
The OB value corresponding to the current temperature is dynamically calculated every m frames (frames), so that the accuracy of calibration can be ensured. Since sampling is performed at a set frame rate, a situation may occur in which calibration is performed in the same temperature interval for a plurality of consecutive times.
And S5A0, weighting and calculating the optical dark space brightness calibration value corresponding to the current temperature.
The optical dark space brightness calibration value corresponding to the current temperature can be calculated by the following formula:
OBdst＝OBbase*α+OBtemp*β+shift (3)
wherein, OBdstFor the optical dark-field luminance calibration value, OBbaseThe brightness value of the optical dark area measured by the factory is taken as the weight, OB, of the brightness value of the optical dark area measured by the factorytempThe current optical dark area brightness value, the weight of the current optical dark area brightness value and the shift are retention parameters, and are related to a use scene.
Exemplarily, when α + β is 1, shift is 0; when α + β ≠ 1, shift can be set according to the usage scenario.
And S5B0, acquiring raw image data of the image sensor, and calibrating the raw image data by adopting an optical dark area brightness calibration value.
Alternatively, the above S510 to S570 may be performed automatically by programming by the purchasing manufacturer of the image sensor as a preparation stage of calibration, or a calibration stage of the calibration curve; the above-mentioned S580-S5B0 as an operation phase of the calibration, or an application phase of the calibration curve, may be performed by the image signal processor ISP within the image sensor.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A calibration curve acquisition method for an optical dark area of an image sensor is characterized by comprising the following steps:
selecting a partial image sensor from all image sensors to be calibrated as a sample image sensor;
counting the optical dark area brightness values of the sample image sensor in different temperature intervals, and selecting a high-quality image sensor from the sample image sensor;
performing curve fitting on the optical dark space brightness values of the high-quality image sensor in all the temperature intervals to obtain a temperature brightness fitting curve of the high-quality image sensor;
measuring the optical dark area brightness values of all the image sensors to be calibrated at key temperature points; and adjusting the temperature brightness fitting curve according to the optical dark space brightness value of the key temperature point to obtain a calibration curve of the optical dark space of the image sensor.
2. The method for obtaining the calibration curve of the optical dark area of the image sensor according to claim 1, wherein the counting the brightness values of the optical dark area of the sample image sensor in different temperature intervals comprises:
the sample image sensor continuously operates in a completely black environment; the sample image sensor acquires an original image data graph at intervals of preset time;
according to the reading of a temperature register arranged in the image sensor, matching a temperature interval corresponding to the temperature when the original image data graph is obtained;
and selecting an interested area from the original image data map, and calculating an average optical dark area brightness value of the interested area as an optical dark area brightness value corresponding to the temperature interval where the original image data map is obtained.
3. The method according to claim 2, wherein if the temperatures at which the original image data maps are obtained for a plurality of times are in the same temperature range, the average value of the average optical dark brightness values of the region of interest calculated for the plurality of times is used as the optical dark brightness value corresponding to the temperature range.
4. The method for obtaining the calibration curve of the optical dark area of the image sensor according to claim 1, wherein selecting a good image sensor from the sample image sensors comprises:
acquiring normal distribution of optical dark space brightness values of all the sample image sensors in any temperature interval;
and selecting the sample image sensor positioned in a set area in the normal distribution as the high-quality image sensor.
5. The method for obtaining the calibration curve of the optical dark area of the image sensor as claimed in claim 4, further comprising, after selecting the good image sensor: selecting an intersection of the high-quality image sensors selected from all the temperature intervals to obtain a global high-quality image sensor;
correspondingly, performing curve fitting on the optical dark area brightness values of the high-quality image sensor in all the temperature intervals to obtain a temperature brightness fitting curve of the high-quality image sensor, including:
and performing curve fitting on the optical dark space brightness values of any global high-quality image sensor in all the temperature intervals to obtain a temperature brightness fitting curve of the high-quality image sensor.
6. The method for obtaining the calibration curve of the optical dark area of the image sensor according to claim 1, wherein the key temperature points comprise: a minimum operating temperature of the image sensor, a maximum operating temperature of the image sensor, and a room temperature.
7. The method for obtaining the calibration curve of the optical dark area of the image sensor according to claim 1, before selecting a part of image sensors from all image sensors to be calibrated as the sample image sensors, further comprising:
checking whether the factory-measured optical dark area brightness value of the image sensor is qualified or not; if the image sensor is qualified, the step of selecting the sample image sensor is executed; and if the image sensor is unqualified, rejecting the unqualified image sensor.
8. A calibration method for an optical dark area of an image sensor, comprising:
acquiring the current temperature of the image sensor;
obtaining a current optical dark area brightness value of the image sensor according to the current temperature and the calibration curve; wherein, the calibration curve is obtained according to the calibration curve acquisition method of the image sensor optical dark area of any one of claims 1 to 7;
obtaining an optical dark space brightness calibration value corresponding to the current temperature according to the current optical dark space brightness value and a factory-measured optical dark space brightness value of the image sensor;
and acquiring original image data of the image sensor, and calibrating the original image data by adopting the optical dark area brightness calibration value.
9. The method for calibrating the optical dark area of the image sensor according to claim 8, wherein obtaining the current temperature of the image sensor comprises:
and acquiring the current temperature from a temperature register built in the image sensor according to the set sampling frame rate.
10. The method for calibrating the optical dark area of the image sensor according to claim 8, wherein the optical dark area brightness calibration value corresponding to the current temperature is calculated according to the following formula:
OBdst＝OBbase*α+OBtemp*β+shift
wherein, OBdstFor the optical dark space brightness calibration value, OBbaseIs the brightness value of the factory-measured optical dark area, alpha is the weight of the brightness value of the factory-measured optical dark area, OBtempAnd the current optical dark area brightness value is taken as the brightness value of the current optical dark area, beta is the weight of the brightness value of the current optical dark area, and shift is a retention parameter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110959259.9A CN113709393B (en) | 2021-08-20 | 2021-08-20 | Calibration curve acquisition method and calibration method for optical dark area of image sensor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110959259.9A CN113709393B (en) | 2021-08-20 | 2021-08-20 | Calibration curve acquisition method and calibration method for optical dark area of image sensor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113709393A true CN113709393A (en) | 2021-11-26 |
| CN113709393B CN113709393B (en) | 2023-06-27 |
Family
ID=78653564
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110959259.9A Active CN113709393B (en) | 2021-08-20 | 2021-08-20 | Calibration curve acquisition method and calibration method for optical dark area of image sensor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113709393B (en) |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040047515A1 (en) * | 2002-09-10 | 2004-03-11 | Umax Data Systems Inc. | Method for adjusting image data with shading curve |
| KR100780464B1 (en) * | 2006-12-22 | 2007-11-28 | 한국항공우주연구원 | Adaptive star centroiding method in star sensor based |
| JP2008034942A (en) * | 2006-07-26 | 2008-02-14 | Canon Inc | Image processor |
| JP2013115702A (en) * | 2011-11-30 | 2013-06-10 | Canon Inc | Imaging apparatus |
| JP2013150144A (en) * | 2012-01-19 | 2013-08-01 | Hitachi Kokusai Electric Inc | Imaging method and imaging apparatus |
| US20160071487A1 (en) * | 2014-09-05 | 2016-03-10 | Hisense Electric Co., Ltd. | Brightness compensating method and self-illuminating display device |
| US20160104284A1 (en) * | 2014-10-10 | 2016-04-14 | Facebook, Inc. | Post-manufacture camera calibration |
| US20160247253A1 (en) * | 2015-02-24 | 2016-08-25 | Samsung Electronics Co., Ltd. | Method for image processing and electronic device supporting thereof |
| US20170295330A1 (en) * | 2016-04-08 | 2017-10-12 | Dongbu HiTek, Co. Ltd. | Image sensor and method of sensing image |
| CN107784661A (en) * | 2017-09-08 | 2018-03-09 | 上海电力学院 | Substation equipment infrared image classifying identification method based on region-growing method |
| CN207556551U (en) * | 2017-05-31 | 2018-06-29 | 江苏紫米软件技术有限公司 | A kind of light sensitive device dark current correction of temperature drift system |
| JP2018141912A (en) * | 2017-02-28 | 2018-09-13 | キヤノン株式会社 | Display device, method for controlling display device, and illumination device |
| CN110418060A (en) * | 2019-08-05 | 2019-11-05 | 苏州中科全象智能科技有限公司 | A kind of method for correcting image and system of high speed camera |
| US20200210764A1 (en) * | 2018-12-28 | 2020-07-02 | Adhark, Inc. | Systems, methods, and storage media for training a machine learning model |
| CN112104860A (en) * | 2020-08-18 | 2020-12-18 | 欧菲微电子技术有限公司 | Calibration method, calibration device, computer device and readable storage medium |
| JP2021060847A (en) * | 2019-10-08 | 2021-04-15 | 株式会社ザクティ | Noise removal system |
| WO2021108732A1 (en) * | 2019-11-25 | 2021-06-03 | Essenlix Corporation | Efficient training and accuracy improvement of imaging based assay |
-
2021
- 2021-08-20 CN CN202110959259.9A patent/CN113709393B/en active Active
Patent Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040047515A1 (en) * | 2002-09-10 | 2004-03-11 | Umax Data Systems Inc. | Method for adjusting image data with shading curve |
| JP2008034942A (en) * | 2006-07-26 | 2008-02-14 | Canon Inc | Image processor |
| KR100780464B1 (en) * | 2006-12-22 | 2007-11-28 | 한국항공우주연구원 | Adaptive star centroiding method in star sensor based |
| JP2013115702A (en) * | 2011-11-30 | 2013-06-10 | Canon Inc | Imaging apparatus |
| JP2013150144A (en) * | 2012-01-19 | 2013-08-01 | Hitachi Kokusai Electric Inc | Imaging method and imaging apparatus |
| US20160071487A1 (en) * | 2014-09-05 | 2016-03-10 | Hisense Electric Co., Ltd. | Brightness compensating method and self-illuminating display device |
| US20160104284A1 (en) * | 2014-10-10 | 2016-04-14 | Facebook, Inc. | Post-manufacture camera calibration |
| US20160247253A1 (en) * | 2015-02-24 | 2016-08-25 | Samsung Electronics Co., Ltd. | Method for image processing and electronic device supporting thereof |
| US20170295330A1 (en) * | 2016-04-08 | 2017-10-12 | Dongbu HiTek, Co. Ltd. | Image sensor and method of sensing image |
| JP2018141912A (en) * | 2017-02-28 | 2018-09-13 | キヤノン株式会社 | Display device, method for controlling display device, and illumination device |
| CN207556551U (en) * | 2017-05-31 | 2018-06-29 | 江苏紫米软件技术有限公司 | A kind of light sensitive device dark current correction of temperature drift system |
| CN107784661A (en) * | 2017-09-08 | 2018-03-09 | 上海电力学院 | Substation equipment infrared image classifying identification method based on region-growing method |
| US20200210764A1 (en) * | 2018-12-28 | 2020-07-02 | Adhark, Inc. | Systems, methods, and storage media for training a machine learning model |
| CN110418060A (en) * | 2019-08-05 | 2019-11-05 | 苏州中科全象智能科技有限公司 | A kind of method for correcting image and system of high speed camera |
| JP2021060847A (en) * | 2019-10-08 | 2021-04-15 | 株式会社ザクティ | Noise removal system |
| WO2021108732A1 (en) * | 2019-11-25 | 2021-06-03 | Essenlix Corporation | Efficient training and accuracy improvement of imaging based assay |
| CN112104860A (en) * | 2020-08-18 | 2020-12-18 | 欧菲微电子技术有限公司 | Calibration method, calibration device, computer device and readable storage medium |
Non-Patent Citations (3)
| Title |
|---|
| YANTING CHENG ET AL.: ""Compensation of optical path difference for colorimetric temperature imaging "", 《2011 IEEE INTERNATIONAL CONFERENCE ON IMAGING SYSTEMS AND TECHNIQUES》 * |
| 姚萍萍等: ""科学级CCD制冷系统设计及其温度特性分析"", 《 光学学报 》, no. 17 * |
| 张斐然等: ""高分七号"卫星双线阵相机暗电流特性分析及校正"", 《航天返回与遥感》, vol. 41, no. 02 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113709393B (en) | 2023-06-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Matsushita et al. | Radiometric calibration from noise distributions | |
| US10630906B2 (en) | Imaging control method, electronic device and computer readable storage medium | |
| US9883178B2 (en) | Method for measuring performance parameters and detecting bad pixels of an infrared focal plane array module | |
| CN110858872B (en) | Optical axis offset compensation method and device | |
| TWI395958B (en) | Defective pixel detection and correction devices, systems, and methods for detecting and correcting defective pixel | |
| US20140313350A1 (en) | Imaging systems with reference pixels for image flare mitigation | |
| CN109685853B (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| EP1525744A1 (en) | Device and method of detection of erroneous image sample data of defective image samples | |
| US8693803B2 (en) | High linear dynamic range imaging | |
| US8481918B2 (en) | System and method for improving the quality of thermal images | |
| CN113747066B (en) | Image correction method, image correction device, electronic equipment and computer readable storage medium | |
| Nixon et al. | The importance of a device specific calibration for smartphone colorimetry | |
| CN113873222B (en) | Linearity correction method and device for industrial camera | |
| CN108871590B (en) | Method and device for correcting non-uniform response rate of uncooled infrared focal plane detector | |
| CN111179184A (en) | Fish-eye image effective region extraction method based on random sampling consistency | |
| CN114331907A (en) | Color shading correction method and device | |
| CN107454388B (en) | Image processing method and apparatus using the same | |
| CN113709393B (en) | Calibration curve acquisition method and calibration method for optical dark area of image sensor | |
| CN117612470A (en) | Color lookup table generating method and color correcting method | |
| US11303876B2 (en) | Temperature compensation for image acquisition and processing apparatus and methods | |
| CN109084899B (en) | Infrared focal plane detector non-uniform body output correction method and device | |
| CN114518175B (en) | Temperature correction method and related device for infrared thermal imaging image | |
| US11546565B2 (en) | Image sensing device and operating method thereof | |
| CN111345032B (en) | Image sensor, light intensity sensing system and method | |
| CN117714905B (en) | Method for correcting radiation response characteristic of CMOS image sensor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |