CN118018870A - Image sensor noise calibration method, control equipment and system - Google Patents
Image sensor noise calibration method, control equipment and system Download PDFInfo
- Publication number
- CN118018870A CN118018870A CN202410221223.4A CN202410221223A CN118018870A CN 118018870 A CN118018870 A CN 118018870A CN 202410221223 A CN202410221223 A CN 202410221223A CN 118018870 A CN118018870 A CN 118018870A
- Authority
- CN
- China
- Prior art keywords
- noise
- image sensor
- image
- tested
- calibration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000012360 testing method Methods 0.000 claims abstract description 89
- 238000004590 computer program Methods 0.000 claims description 13
- 238000003860 storage Methods 0.000 claims description 7
- 230000000295 complement effect Effects 0.000 claims description 6
- 229910044991 metal oxide Inorganic materials 0.000 claims description 6
- 150000004706 metal oxides Chemical class 0.000 claims description 6
- 239000004065 semiconductor Substances 0.000 claims description 6
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 claims description 5
- 239000004973 liquid crystal related substance Substances 0.000 claims description 5
- 229910052710 silicon Inorganic materials 0.000 claims description 5
- 239000010703 silicon Substances 0.000 claims description 5
- 238000012545 processing Methods 0.000 description 19
- 238000010586 diagram Methods 0.000 description 18
- 230000000875 corresponding effect Effects 0.000 description 15
- 230000008569 process Effects 0.000 description 13
- 238000009826 distribution Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000003384 imaging method Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000033228 biological regulation Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000012790 confirmation Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000002360 preparation method Methods 0.000 description 2
- 235000012434 pretzels Nutrition 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000003705 background correction Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000000306 component Substances 0.000 description 1
- 239000008358 core component Substances 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000011017 operating method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000007430 reference method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The embodiment of the invention provides an image sensor noise calibration method, control equipment and a system. The method comprises the following steps: receiving a test instruction for carrying out noise calibration on an image sensor to be tested; based on the test instruction, determining sensor parameters of the image sensor to be tested, and sending the sensor parameters to the image sensor to be tested; based on the test instruction, determining a noise image card, and sending the noise image card to the projection equipment so that the projection equipment can modulate and reflect the light source according to the noise image card and project the generated projection pattern to the image sensor to be tested; receiving a noise image sent by an image sensor to be detected after capturing a projection pattern according to sensor parameters; and according to the received noise image and the test instruction, performing noise calibration on the image sensor to be tested. The embodiment of the invention realizes the minimization of the dependence on the calibration environment, simplifies the calibration operation flow and ensures the high precision of the calibration result.
Description
Technical Field
The present invention relates to the field of imaging technologies, and in particular, to an image sensor noise calibration method, a control device, and an image sensor noise calibration system.
Background
Images, which are an optical information carrier that is captured and digitized by a specific sensor device, are responsible for a rich visual data transmission and play a vital role in many fields. Image sensors are the core component in modern image acquisition systems, responsible for converting optical signals into electrical signals for subsequent digitizing and image processing. However, in this process, noise is inevitably mixed into the signal, affecting the image quality and reducing its usability. Image sensor noise can be attributed to a number of sources and mechanisms including thermal noise, shot noise, dark current noise, readout noise, quantum efficiency noise, and the like. These noise types collectively contribute to the overall noise level and are present in the acquired image in respective unique characteristics and distribution patterns.
The noise reduces the signal-to-noise ratio (SNR) of the image signal, and can also cause loss of detail information in the image, so that the accuracy of subsequent image processing and analysis is affected, and especially in the fields with extremely high requirements on image quality such as medical imaging, astronomical observation, satellite remote sensing and the like, uncorrected sensor noise is more likely to cause serious errors of data interpretation.
Because of the prevalence of image sensor noise and its significant impact on image quality, noise calibration is an indispensable step to improve image quality, ensuring data accuracy. Noise components can be accurately measured and compensated through noise calibration, so that the definition and resolution capability of an image are improved.
The conventional image sensor noise calibration method comprises black frame correction, flat field correction, noise model calibration based on statistics and the like, however, the conventional calibration method has respective limitations depending on acquisition work of a target image card in an actual scene, such as incapability of correcting time-varying noise, extremely severe requirements on environmental illumination conditions, high uniformity of a control light source and the like, which results in complicated operation procedures, high calculation complexity and the like, and cannot meet the requirements of actual imaging application.
Disclosure of Invention
Aiming at the defects in the prior art, the embodiment of the invention provides an image sensor noise calibration method, control equipment and an image sensor noise calibration system.
In a first aspect, an embodiment of the present invention provides a method for calibrating noise of an image sensor, including:
receiving a test instruction for carrying out noise calibration on an image sensor to be tested;
based on the test instruction, determining sensor parameters of the image sensor to be tested, and sending the sensor parameters to the image sensor to be tested;
based on the test instruction, determining a noise image card, and sending the noise image card to a projection device so that the projection device projects a generated projection pattern to the image sensor to be tested according to the noise image card modulation and reflection light source;
Receiving a noise image sent by the image sensor to be detected after capturing the projection pattern according to the sensor parameters;
and according to the received noise image and the test instruction, performing noise calibration on the image sensor to be tested.
In the above method, optionally, the determining a noise graphic card based on the test instruction, and sending the noise graphic card to a projection device, so that the projection device modulates and reflects a light source according to the noise graphic card, and projects a generated projection pattern to the image sensor to be measured, including:
And determining a plurality of noise image cards and loading sequences based on the test instruction, and sending the noise image cards and the loading sequences to a projection device so that the projection device modulates and reflects light sources for the corresponding noise image cards according to the loading sequences, and sequentially projecting generated projection patterns to the image sensor to be tested.
The method optionally further includes, after receiving a test instruction for noise calibration of the image sensor to be tested:
Initializing a noise model and noise parameters according to the test instruction;
Correspondingly, the performing noise calibration on the image sensor to be tested according to the received noise image and the test instruction includes:
extracting noise characteristics of the image sensor to be detected according to the received multiple noise images and the noise parameters;
And optimizing the noise model according to the noise characteristics.
In a second aspect, an embodiment of the present invention provides a control apparatus, including:
the test instruction receiving module is used for receiving a test instruction for carrying out noise calibration on the image sensor to be tested;
the sensor parameter sending module is used for determining sensor parameters of the image sensor to be tested based on the test instruction and sending the sensor parameters to the image sensor to be tested;
The noise image card sending module is used for determining a noise image card based on the test instruction, and loading the noise image card to the projection equipment so that the projection equipment can project the generated projection pattern to the image sensor to be tested according to the noise image card modulation and reflection light source;
The noise image receiving module is used for receiving a noise image sent by the image sensor to be detected after capturing the projection pattern according to the sensor parameters;
and the noise calibration module is used for carrying out noise calibration on the image sensor to be tested according to the received noise image and the test instruction.
As mentioned above, optionally, the noise map card sending module is specifically configured to:
And determining a plurality of noise image cards and loading sequences based on the test instruction, and sending the noise image cards and the loading sequences to a projection device so that the projection device modulates and reflects light sources for the corresponding noise image cards according to the loading sequences, and sequentially projecting generated projection patterns to the image sensor to be tested.
The control apparatus as described above, optionally, further includes:
The initialization module is used for initializing a noise model and noise parameters according to the test instruction;
Correspondingly, the noise calibration module is specifically configured to:
extracting noise characteristics of the image sensor to be detected according to the received multiple noise images and the noise parameters;
And optimizing the noise model according to the noise characteristics.
In a third aspect, an embodiment of the present invention provides an image sensor noise calibration system, including:
the device comprises an image sensor to be tested, a light source, a projection device and a control device;
The control equipment is used for receiving a test instruction for carrying out noise calibration on the image sensor to be tested, determining sensor parameters of the image sensor to be tested based on the test instruction, and sending the sensor parameters to the image sensor to be tested;
the control device is further used for determining a noise figure card based on the test instruction and sending the noise figure card to the projection device;
The projection equipment is used for receiving the noise image card sent by the control equipment, modulating and reflecting the light source according to the noise image card, and projecting the generated projection pattern to the image sensor to be detected;
The image sensor to be detected is used for capturing the projection pattern according to the sensor parameters, generating a noise image and sending the noise image to the control equipment;
the control equipment is also used for carrying out noise calibration on the image sensor to be tested according to the received noise image and the test instruction.
As in the above system, optionally, the projection device comprises:
A projection device, a first lens group, and a second lens group;
The first lens group is used for diffusing light generated by the light source to the surface of the projection device;
the projection device is used for modulating and reflecting the light rays according to the noise graphic card to generate a projection pattern;
the second lens group is used for projecting the projection pattern to the image sensor to be detected.
As in the above system, optionally, the projection device includes: digital micromirror devices or liquid crystal on silicon modulators.
As in the above system, optionally, the image sensor to be measured includes: charge coupled devices or complementary metal oxide semiconductors.
In a fourth aspect, an embodiment of the present invention provides an electronic device, including:
the device comprises a memory and a processor, wherein the processor and the memory are communicated with each other through a bus; the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the image sensor noise calibration method according to any of the first aspects above.
In a fifth aspect, an embodiment of the present invention provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the image sensor noise calibration method according to any one of the first aspects above.
According to the image sensor noise calibration method provided by the embodiment of the invention, based on the novel digital projection architecture, the entity calibration image card is converted into digital projection information, and then the special projection equipment is used for loading and processing, so that the noise calibration of the image sensor is realized, the harsh requirements of the noise calibration on the environment are reduced, and the operation complexity is reduced. In addition, according to different noise calibration requirements, the projection equipment can flexibly switch various patterns to provide various calibration schemes, so that the minimization of the dependence on the calibration environment is realized, and the calibration operation flow is simplified. The accurate control capability of the projection equipment further ensures the high precision of the calibration result, so that the repeatability and the accuracy of the calibration are obviously improved.
Drawings
FIG. 1 is a flow chart of steps of an embodiment of a method for calibrating noise of an image sensor according to the present invention;
FIG. 2 is a schematic diagram of a noise chart in an embodiment of a method for calibrating noise of an image sensor according to the present invention;
FIGS. 3 (a) and 3 (b) are diagrams illustrating the noise calibration results in an embodiment of an image sensor noise calibration method according to the present invention;
FIG. 4 is a schematic diagram of a system architecture in an embodiment of a method for calibrating noise of an image sensor according to the present invention;
FIG. 5 is a block diagram of one embodiment of a control device of the present invention;
FIG. 6 is a block diagram of an embodiment of an image sensor noise calibration system of the present invention;
fig. 7 is a block diagram of an embodiment of an electronic device of the present invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
Referring to fig. 1, a flowchart illustrating steps of an embodiment of a noise calibration method for an image sensor according to the present invention may specifically include the following steps:
Step S110, receiving a test instruction for carrying out noise calibration on an image sensor to be tested;
In particular, image sensor noise calibration is an active research field, and continuous innovation and development of calibration technology are of great importance for advancing image processing science and application thereof. Through accurate noise calibration, the image quality is obviously improved, and more accurate information interpretation and better decision support are brought to related industries. Conventional image sensor noise calibration generally relies on the acquisition of a target calibration graphics card in an actual scene, making the requirements on ambient lighting conditions extremely stringent and resulting in cumbersome operating procedures. In order to solve the problem, the embodiment of the invention adopts a digital projection architecture, and comprises a control device, a light source, a projection device and an image sensor to be tested which is required to be tested and is connected in a replaceable manner, wherein the control device is used for converting an entity calibration graphic card into digital pattern information, and then the special projection device is used for loading and processing.
Firstly, when noise calibration is required to be carried out on a certain image sensor, the image sensor is marked as an image sensor to be measured, a test instruction is determined based on the noise calibration target, a control device receives the test instruction, and the control device can be an electronic device such as a computer which comprises a control unit and can carry out data transmission processing. For example, the control device may be provided with an interface for user selection input, the user selects or inputs the image sensor to be tested in the interface, selects the noise type and the noise parameter, and then submits the test, and the control device may acquire the test instruction.
Step S120, based on the test instruction, determining sensor parameters of the image sensor to be tested, and sending the sensor parameters to the image sensor to be tested;
Specifically, after the control device obtains the test instruction, the sensor parameters of the image sensor to be tested are determined according to the test instruction, as shown in table 1, the gain setting, the black level setting, the exposure time, the continuous shooting number and the like of the image sensor to be tested are sent to the image sensor to be tested by the control device, after the sensor parameters sent by the control device are received by the image sensor to be tested, the image sensor is set according to the sensor parameters, and is adjusted to a mode corresponding to the test instruction, so that the image is ready to be collected.
Table 1 sensor parameter table
It should be noted that, as the target to be calibrated, the image sensor in the embodiment of the present invention supports a mainstream array sensor type such as a Charge-Coupled Device (CCD) or a complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor, CMOS) camera sensor.
Step S130, a noise image card is determined based on the test instruction, and the noise image card is sent to a projection device, so that the projection device projects a generated projection pattern to the image sensor to be tested according to the noise image card modulation and reflection light source;
Specifically, the control device determines the noise calibration purpose and parameters according to the test instruction, and determines the corresponding noise chart card, and fig. 2 is a schematic diagram of the noise chart card in the embodiment of the noise calibration method of the image sensor, as shown in fig. 2, where the noise chart card is used to provide blocks with different brightness values under the irradiation of a uniform light source so as to facilitate the detection of the image sensor. For example, for dark noise calibration of the image sensor, the control device may determine that the noise map card is a full black map card, and for additive noise calibration of the image sensor, the control device may determine that the noise map card is a gray scale map card. In practical application, the control device can select a standard noise chart card corresponding to noise, and can also design a digital projection pattern by itself as the noise chart card. The control device can flexibly design the seed patterns according to the test instructions so as to provide various calibration schemes.
After the control device determines the noise figure card, the noise figure card is sent to the projection device, the projection device loads the noise figure card, and the light source is modulated and reflected through the internal modulation array to generate a projection pattern corresponding to the noise figure card. The projection equipment has accurate control capability, can realize the regulation and control of light rays by taking pixels as units, and ensures the high precision of noise calibration results by the accurate control capability of the projection equipment, so that the repeatability and the accuracy of the calibration are obviously improved.
In a specific implementation, the projection device may include a projection device and two lens groups, denoted as a first lens group and a second lens group, where the projection device is not limited in size, and may have a time-varying function to flexibly load different graphics cards, and may be a digital micromirror device (Digital Micromirror Devices, DMD) or a liquid crystal on silicon modulator (Liquid Crystal On Silicon, LCoS), or the like. The first lens group diffuses light of the light source to the surface of the projection device, the projection device modulates and reflects the light according to the noise image card to generate a projection pattern, and the projection pattern is projected to the image sensor to be detected through the second lens group.
The embodiment of the invention can realize the pixel brightness regulation and control of single pixels (micron level) through the projection device, and the prior art scheme can not realize such accurate regulation and control.
In practical applications, the control device may further set an on command, and after the projection device receives the on command sent by the control device, the light is modulated and reflected.
In practical applications, the image sensor noise calibration system may further include a controllable light source, and the control device may set parameters of the light source according to the test instruction, for example, whether to turn on the light source, and the control device sends the parameters of the light source to work with the corresponding parameters. In order to reduce the interference of ambient light, in practical application, a calibration environment is built in an optical darkroom, so that the interference of ambient light is avoided, the stability of conditions such as temperature and humidity is ensured, and the influence of environmental change on a measurement result is prevented. It should be noted that, in the conventional scheme, the light source needs to be adjusted to be uniformly distributed, so that the requirement on the light source is high, and the embodiment of the invention has no strict requirement on the uniformity of the light source.
Step S140, receiving a noise image sent by the image sensor to be detected after capturing the projection pattern according to the sensor parameter;
Specifically, the image sensor converts the incident light from the object into a digital signal, and finally converts the digital signal into a final RGB format image through the image processing system, and stores the final RGB format image as a common JPG file. Image sensor noise is introduced during the conversion of the optical signal into an electrical signal. The image sensor captures the projection pattern in a sensor parameter mode, and finally forms a noise image corresponding to the projection pattern through conversion, and sends the noise image to the control device.
And step S150, performing noise calibration on the image sensor to be tested according to the received noise image and the test instruction.
Specifically, after the control equipment receives the noise image, the noise calibration is carried out on the image sensor to be tested according to the test instruction. For example, the control device determines the parameter value in the noise model through analysis to obtain the noise model of the image sensor to be detected, so that the image processing system can reduce noise according to the interior of the noise model to obtain a high-quality output image.
Through the process, the noise characteristic of the image sensor can be accurately marked, the processing algorithm of the image sensor is calibrated, and the final imaging quality is improved.
According to the image sensor noise calibration method provided by the embodiment of the invention, based on the novel digital projection architecture, the entity calibration image card is converted into digital projection information, and then the special projection equipment is used for loading and processing, so that the noise calibration of the image sensor is realized, the harsh requirements of the noise calibration on the environment are reduced, and the operation complexity is reduced. In addition, according to different noise calibration requirements, the projection equipment can flexibly switch various patterns to provide various calibration schemes, so that the minimization of the dependence on the calibration environment is realized, and the calibration operation flow is simplified. The accurate control capability of the projection equipment further ensures the high precision of the calibration result, so that the repeatability and the accuracy of the calibration are obviously improved.
On the basis of the foregoing embodiment, further, the determining, based on the test instruction, a noise chart card, and sending the noise chart card to a projection device, so that the projection device modulates and reflects a light source according to the noise chart card, and projects a generated projection pattern to the image sensor to be measured, including:
And determining a plurality of noise image cards and loading sequences based on the test instruction, and sending the noise image cards and the loading sequences to a projection device so that the projection device modulates and reflects light sources for the corresponding noise image cards according to the loading sequences, and sequentially projecting generated projection patterns to the image sensor to be tested.
Specifically, the noise calibration target is to acquire the noise distribution characteristics of the image sensor in the detected image under different parameter conditions. The calibration process is mainly divided into three parts: noise modeling, image acquisition, parameter estimation. The noise modeling is used for confirming the type of noise and the noise parameters to be estimated and determining the corresponding sensor parameters; the image acquisition is to acquire different noisy image data according to different sensor parameters, the parameter estimation is to analyze the image data to perform parameter estimation, and the noise distribution characteristic of the sensor is output.
According to different noise characteristic requirements, the control device can flexibly set and switch various digital noise image cards, set loading sequences, send the noise image cards and the loading sequences to the projection device, load the corresponding noise image cards according to the loading sequences, modulate and reflect the light source through the internal control element to generate a plurality of projection patterns, and sequentially project the projection patterns to the image sensor to be tested. In addition, the efficiency of switching the graphics cards is different, in order to switch other graphics cards or adjust the overall relative brightness of the graphics cards, the traditional scheme needs to change different entity graphics cards, the efficiency of the process of manufacturing the graphics cards is low, the cost is high, and the embodiment of the invention only needs to program to realize new graphics cards, and the speed is high, the efficiency is high, and the cost is low.
The image sensor to be measured captures the reflected light beam with preset exposure time and gain to form a noise image, and sends the noise image to the control equipment, and the capturing process is repeated according to the parameters of the table 1 so as to obtain a series of noise images under different noise conditions.
According to the image sensor noise calibration method provided by the embodiment of the invention, the control equipment can flexibly set and switch various digital noise graphics cards and set the loading sequence, so that a series of quick and adjustable calibration schemes are provided, the minimization of the dependence on the calibration environment is realized, and the calibration operation flow is simplified.
On the basis of the above embodiments, further, after receiving the test instruction for performing noise calibration on the image sensor to be tested, the method further includes:
Initializing a noise model and noise parameters according to the test instruction;
Correspondingly, the performing noise calibration on the image sensor to be tested according to the received noise image and the test instruction includes:
extracting noise characteristics of the image sensor to be detected according to the received multiple noise images and the noise parameters;
And optimizing the noise model according to the noise characteristics.
Specifically, after receiving the test instruction, the control device determines the type of the tested noise, such as gaussian noise, pretzel noise, and the like, and designs a corresponding noise model and noise parameters.
After receiving the noise image sent by the image sensor to be detected, the control device extracts noise characteristics of the image sensor to be detected, such as noise mean value, noise variance and the like, according to the noise parameters. And finally, reestablishing or optimizing a noise model according to the statistical result to describe the noise characteristic of the image sensor to be tested. Referring to fig. 3 (a) and 3 (b), a schematic diagram of a noise calibration result in an embodiment of a noise calibration method for an image sensor according to the present invention is shown, the horizontal axis is Gain, the vertical axis is noise variance, real (real) noise and noise according to final optimized noise model fitting (fixing) are calibrated respectively, wherein fig. 3 (a) is independent noise, fig. 3 (b) is correlated noise, and as can be seen from fig. 3 (a) and 3 (b), the noise variance increases as the Gain increases, and the noise model calibrated by the embodiment of the present invention conforms to actual noise distribution.
Taking a control device as a computer, the projection device comprises a digital micromirror device DMD, a lens group 1 and a lens group 2, and the image sensor is a complementary metal oxide semiconductor CMOS for example, referring to fig. 4, a schematic diagram of a system architecture in an embodiment of a noise calibration method of the image sensor of the present invention is shown. The light source irradiates the surface of the DMD, the light is modulated and reflected by the DMD, the target image card is projected on the surface of the detection CMOS, and the CMOS digitizes the light signal to complete the acquisition of a calibration image. The light source provides illumination conditions for the DMD, the lens group 1 diffuses light to the surface of the DMD, the DMD loads a target noise image card to output a corresponding projection pattern, the lens group 2 projects the projection pattern to the CMOS, and the CMOS completes collection and output of the noise-containing image.
Specifically, the sensor noise calibration process includes: calibration preparation, noise chart loading, data acquisition, data analysis, calibration result application, optimization and verification.
(1) Calibration preparation
And (3) equipment debugging: ensure pixel matching between an image sensor (e.g., CMOS or CCD camera) and a Digital Micromirror Device (DMD) and the entire system operates stably.
And (3) setting up an environment: setting up a calibration environment in the optical darkroom, so as to avoid the interference of ambient light; and the stability of conditions such as temperature and humidity is ensured, and the influence of environmental change on the measurement result is prevented.
Noise confirmation: and designing a corresponding noise model and noise parameters according to the noise types (such as Gaussian noise, pretzel noise and the like) expected to be calibrated.
Parameter confirmation: and confirming relevant camera parameters according to the noise model, wherein the parameters comprise camera gain setting, black level setting, exposure time, continuous shooting quantity and the like. Furthermore, the light source parameters also need to be determined according to the noise type.
(2) Noise graphics card loading
And (3) designing a picture card: standard graphics cards or self-designed digital projection graphics cards are used.
Loading a graphics card: the graphics card is loaded into the micromirror array of the DMD by control software.
(3) Data acquisition
Parameter setting: the DMD is started to reflect the light beam to the image sensor according to the preset image card. The light source parameters (on/off), the camera parameters (gain, exposure time, black level, number of continuous beats) are set according to the noise type.
Image capturing: the image sensor captures a noise pattern formed by the reflected light beam with a preset exposure time and gain.
Repeated measurement: the above-described capturing process is repeated to obtain a series of noisy images under different noise conditions.
(4) Data analysis
Image processing: noise features (such as mean, variance, etc.) are extracted by processing the noise image according to the noise model.
Statistical analysis: statistical analysis is performed on the extracted noise data to evaluate noise characteristics, such as whether the distribution meets an expected noise model.
And (3) determining a calibration result: and establishing or optimizing a noise model for describing the noise characteristic of the image sensor according to the statistical result.
(5) Calibration result application
Parameter extraction: key noise parameters, such as mean, standard deviation, spectral distribution, etc., of the noise are extracted from the noise model.
And (3) verifying results: and applying the extracted noise parameters to a signal processing chain of the image sensor to perform verification work of noise suppression or image restoration.
(6) Optimization and verification
And (3) system feedback: and combining the results after image restoration or noise suppression, and carrying out feedback adjustment on the noise calibration process.
Cross-validation: other reference methods can be adopted to verify the calibration result, so that the calibration accuracy and reliability are ensured.
Through the process, the noise characteristic of the image sensor can be accurately marked, the processing algorithm of the image sensor is calibrated, and the final imaging quality is improved. The constantly repeated optimization and check ring can ensure the calibration effect and the robustness of the system.
According to the image sensor noise calibration method provided by the embodiment of the invention, based on the novel digital projection architecture, the entity calibration image card is converted into digital projection information, and then the special projection equipment is used for loading and processing, so that the noise calibration of the image sensor is realized, the harsh requirements of the noise calibration on the environment are reduced, and the operation complexity is reduced. In addition, according to different noise calibration requirements, the projection equipment can flexibly switch various patterns to provide various calibration schemes, so that the minimization of the dependence on the calibration environment is realized, and the calibration operation flow is simplified. The accurate control capability of the projection equipment further ensures the high precision of the calibration result, so that the repeatability and the accuracy of the calibration are obviously improved.
It should be noted that, for simplicity of description, the method embodiments are shown as a series of acts, but it should be understood by those skilled in the art that the embodiments are not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred embodiments, and that the acts are not necessarily required by the embodiments of the invention.
Referring to fig. 5, a block diagram of an embodiment of a control device of the present invention is shown, and may specifically include the following modules:
The test instruction receiving module 510 is configured to receive a test instruction for performing noise calibration on the image sensor to be tested;
A sensor parameter sending module 520, configured to determine a sensor parameter of the image sensor to be tested based on the test instruction, and send the sensor parameter to the image sensor to be tested;
The noise image card sending module 530 is configured to determine a noise image card based on the test instruction, and load the noise image card to a projection device, so that the projection device projects a generated projection pattern to the image sensor to be tested according to the noise image card modulation and reflection light source;
a noise image receiving module 540, configured to receive a noise image sent by the image sensor to be detected after capturing the projection pattern according to the sensor parameter;
And the noise calibration module 550 is configured to perform noise calibration on the image sensor to be tested according to the received noise image and the test instruction.
As mentioned above, optionally, the noise card sending module 530 is specifically configured to:
And determining a plurality of noise image cards and loading sequences based on the test instruction, and sending the noise image cards and the loading sequences to a projection device so that the projection device modulates and reflects light sources for the corresponding noise image cards according to the loading sequences, and sequentially projecting generated projection patterns to the image sensor to be tested.
The control apparatus as described above, optionally, further includes:
The initialization module is used for initializing a noise model and noise parameters according to the test instruction;
accordingly, the noise calibration module 550 is specifically configured to:
extracting noise characteristics of the image sensor to be detected according to the received multiple noise images and the noise parameters;
And optimizing the noise model according to the noise characteristics.
For the control device embodiment, since the control device embodiment is substantially similar to the method embodiment, the description is relatively simple, and the relevant points only need to be referred to the part of the description of the method embodiment, which is not repeated here.
Referring to fig. 6, a block diagram of an embodiment of an image sensor noise calibration system according to the present invention is shown, which may specifically include the following modules:
An image sensor 610 to be measured, a light source 620, a projection device 630 and a control device 640;
the control device 640 is configured to receive a test instruction for performing noise calibration on the image sensor 610 to be tested, determine a sensor parameter of the image sensor 610 to be tested based on the test instruction, and send the sensor parameter to the image sensor 610 to be tested;
the control device 640 is further configured to determine a noise chart based on the test instruction, and send the noise chart to the projection device 630;
The projection device 630 is configured to receive the noise chart sent by the control device, modulate and reflect the light source 620 according to the noise chart, and project the generated projection pattern to the image sensor 610 to be measured;
The image sensor to be measured 610 is configured to capture the projection pattern according to the sensor parameter, generate a noise image, and send the noise image to the control device 640;
The control device 640 is further configured to perform noise calibration on the image sensor 610 to be tested according to the received noise image and the test instruction.
As with the system described above, the projection device 630 optionally includes:
A projection device, a first lens group, and a second lens group;
The first lens group is used for diffusing the light generated by the light source 620 to the surface of the projection device;
the projection device is used for modulating and reflecting the light rays according to the noise graphic card to generate a projection pattern;
The second lens group is used for projecting the projection pattern to the image sensor 610 to be measured.
As in the above system, optionally, the projection device includes: digital micromirror devices or liquid crystal on silicon modulators.
As in the system described above, optionally, the image sensor under test 610 includes: charge coupled devices or complementary metal oxide semiconductors.
For the system embodiment, since the system embodiment is substantially similar to the method embodiment, the description is relatively simple, and the relevant points only need to be referred to the part of the description of the method embodiment, which is not repeated herein.
Referring to fig. 7, there is shown a block diagram of an embodiment of an electronic device of the present invention, the device comprising: a processor (processor) 710, a memory (memory) 720, and a bus 730;
Wherein processor 710 and memory 720 communicate with each other via bus 730;
Processor 710 is configured to invoke program instructions in memory 720 to perform the methods provided by the method embodiments described above, including, for example: receiving a test instruction for carrying out noise calibration on an image sensor to be tested; based on the test instruction, determining sensor parameters of the image sensor to be tested, and sending the sensor parameters to the image sensor to be tested; based on the test instruction, determining a noise image card, and sending the noise image card to a projection device so that the projection device projects a generated projection pattern to the image sensor to be tested according to the noise image card modulation and reflection light source; receiving a noise image sent by the image sensor to be detected after capturing the projection pattern according to the sensor parameters; and according to the received noise image and the test instruction, performing noise calibration on the image sensor to be tested.
Embodiments of the present invention disclose a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the methods provided by the method embodiments described above, for example comprising: receiving a test instruction for carrying out noise calibration on an image sensor to be tested; based on the test instruction, determining sensor parameters of the image sensor to be tested, and sending the sensor parameters to the image sensor to be tested; based on the test instruction, determining a noise image card, and sending the noise image card to a projection device so that the projection device projects a generated projection pattern to the image sensor to be tested according to the noise image card modulation and reflection light source; receiving a noise image sent by the image sensor to be detected after capturing the projection pattern according to the sensor parameters; and according to the received noise image and the test instruction, performing noise calibration on the image sensor to be tested.
Embodiments of the present invention provide a non-transitory computer readable storage medium storing computer instructions that cause a computer to perform the methods provided by the above-described method embodiments, for example, including: receiving a test instruction for carrying out noise calibration on an image sensor to be tested; based on the test instruction, determining sensor parameters of the image sensor to be tested, and sending the sensor parameters to the image sensor to be tested; based on the test instruction, determining a noise image card, and sending the noise image card to a projection device so that the projection device projects a generated projection pattern to the image sensor to be tested according to the noise image card modulation and reflection light source; receiving a noise image sent by the image sensor to be detected after capturing the projection pattern according to the sensor parameters; and according to the received noise image and the test instruction, performing noise calibration on the image sensor to be tested.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.
It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the invention may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or terminal device that comprises the element.
The above description of the image sensor noise calibration method, the control device and the image sensor noise calibration system provided by the invention applies specific examples to illustrate the principles and the implementation of the invention, and the above examples are only used for helping to understand the method and the core idea of the invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.
Claims (10)
1. An image sensor noise calibration method is characterized by comprising the following steps:
receiving a test instruction for carrying out noise calibration on an image sensor to be tested;
based on the test instruction, determining sensor parameters of the image sensor to be tested, and sending the sensor parameters to the image sensor to be tested;
based on the test instruction, determining a noise image card, and sending the noise image card to a projection device so that the projection device projects a generated projection pattern to the image sensor to be tested according to the noise image card modulation and reflection light source;
Receiving a noise image sent by the image sensor to be detected after capturing the projection pattern according to the sensor parameters;
and according to the received noise image and the test instruction, performing noise calibration on the image sensor to be tested.
2. The method of claim 1, wherein the determining a noise figure card based on the test instructions, transmitting the noise figure card to a projection device so that the projection device projects the generated projection pattern to the image sensor under test according to the noise figure card modulation and reflection light source, comprises:
And determining a plurality of noise image cards and loading sequences based on the test instruction, and sending the noise image cards and the loading sequences to a projection device so that the projection device modulates and reflects light sources for the corresponding noise image cards according to the loading sequences, and sequentially projecting generated projection patterns to the image sensor to be tested.
3. The method according to claim 2, further comprising, after receiving the test instruction for noise calibration of the image sensor to be tested:
Initializing a noise model and noise parameters according to the test instruction;
Correspondingly, the performing noise calibration on the image sensor to be tested according to the received noise image and the test instruction includes:
extracting noise characteristics of the image sensor to be detected according to the received multiple noise images and the noise parameters;
And optimizing the noise model according to the noise characteristics.
4. A control apparatus, characterized by comprising:
the test instruction receiving module is used for receiving a test instruction for carrying out noise calibration on the image sensor to be tested;
the sensor parameter sending module is used for determining sensor parameters of the image sensor to be tested based on the test instruction and sending the sensor parameters to the image sensor to be tested;
The noise image card sending module is used for determining a noise image card based on the test instruction, and loading the noise image card to the projection equipment so that the projection equipment can project the generated projection pattern to the image sensor to be tested according to the noise image card modulation and reflection light source;
The noise image receiving module is used for receiving a noise image sent by the image sensor to be detected after capturing the projection pattern according to the sensor parameters;
and the noise calibration module is used for carrying out noise calibration on the image sensor to be tested according to the received noise image and the test instruction.
5. An image sensor noise calibration system, comprising:
the device comprises an image sensor to be tested, a light source, a projection device and a control device;
The control equipment is used for receiving a test instruction for carrying out noise calibration on the image sensor to be tested, determining sensor parameters of the image sensor to be tested based on the test instruction, and sending the sensor parameters to the image sensor to be tested;
the control device is further used for determining a noise figure card based on the test instruction and sending the noise figure card to the projection device;
The projection equipment is used for receiving the noise image card sent by the control equipment, modulating and reflecting the light source according to the noise image card, and projecting the generated projection pattern to the image sensor to be detected;
The image sensor to be detected is used for capturing the projection pattern according to the sensor parameters, generating a noise image and sending the noise image to the control equipment;
the control equipment is also used for carrying out noise calibration on the image sensor to be tested according to the received noise image and the test instruction.
6. The system of claim 5, wherein the projection device comprises:
A projection device, a first lens group, and a second lens group;
The first lens group is used for diffusing light generated by the light source to the surface of the projection device;
the projection device is used for modulating and reflecting the light rays according to the noise graphic card to generate a projection pattern;
the second lens group is used for projecting the projection pattern to the image sensor to be detected.
7. The system of claim 6, wherein the projection device comprises: digital micromirror devices or liquid crystal on silicon modulators.
8. The system of claim 6, wherein the image sensor under test comprises: charge coupled devices or complementary metal oxide semiconductors.
9. An electronic device, comprising:
The device comprises a memory and a processor, wherein the processor and the memory are communicated with each other through a bus; the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of any of claims 1-3.
10. A computer readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the method according to any of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410221223.4A CN118018870A (en) | 2024-02-28 | 2024-02-28 | Image sensor noise calibration method, control equipment and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410221223.4A CN118018870A (en) | 2024-02-28 | 2024-02-28 | Image sensor noise calibration method, control equipment and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118018870A true CN118018870A (en) | 2024-05-10 |
Family
ID=90949901
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410221223.4A Pending CN118018870A (en) | 2024-02-28 | 2024-02-28 | Image sensor noise calibration method, control equipment and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118018870A (en) |
-
2024
- 2024-02-28 CN CN202410221223.4A patent/CN118018870A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA3016429C (en) | Combined hdr/ldr video streaming | |
| US9451187B2 (en) | Lens shading calibration for cameras | |
| CN109076169B (en) | Method, apparatus and computer readable medium for capturing color image data | |
| TWI484283B (en) | Image measurement method, image measurement apparatus and image inspection apparatus | |
| TW563389B (en) | Compensation system for use in a printed circuit board inspection system | |
| CN109743505B (en) | Video shooting method and device based on laser ranging and electronic equipment | |
| US9894339B2 (en) | Image processing apparatus, image processing method and program | |
| US8334908B2 (en) | Method and apparatus for high dynamic range image measurement | |
| US20190188841A1 (en) | Image processing system, computer readable recording medium, and image processing method | |
| KR101158260B1 (en) | sham image projection device based on for linear sensor and sham image projection method | |
| Masaoka et al. | Realtime measurement of ultrahigh-definition camera modulation transfer function | |
| CN118018870A (en) | Image sensor noise calibration method, control equipment and system | |
| KR101653649B1 (en) | 3D shape measuring method using pattern-light with uniformity compensation | |
| US10225537B2 (en) | Image processing apparatus, image capturing apparatus, image processing method, and storage medium | |
| CN109813533B (en) | Method and device for testing DOE diffraction efficiency and uniformity in batch | |
| CN110300291B (en) | Apparatus and method for determining color value, digital camera, application, and computer device | |
| CN106993184B (en) | Test method for CMOS sensor | |
| CN118018713A (en) | Dynamic range calibration method, control equipment and system of image sensor | |
| CN106973287B (en) | Test system for CMOS sensor | |
| US11037316B2 (en) | Parallax calculation apparatus, parallax calculation method, and control program of parallax calculation apparatus | |
| JP2004048610A (en) | Imaging apparatus | |
| CN109213037B (en) | Sampling time self-adaptive single photon compression imaging control method and control device | |
| CN116442523B (en) | Light uniformity adjusting method, device, equipment and storage medium | |
| US20200212091A1 (en) | Pixel crosstalk correction | |
| Kao et al. | Automatic sensor and mechanical shutter calibration for digital still cameras |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |