WO2019100270A1 - 图像噪声标定方法及装置、图像降噪方法及装置、图像处理装置 - Google Patents

图像噪声标定方法及装置、图像降噪方法及装置、图像处理装置 Download PDF

Info

Publication number
WO2019100270A1
WO2019100270A1 PCT/CN2017/112440 CN2017112440W WO2019100270A1 WO 2019100270 A1 WO2019100270 A1 WO 2019100270A1 CN 2017112440 W CN2017112440 W CN 2017112440W WO 2019100270 A1 WO2019100270 A1 WO 2019100270A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
data
noise reduction
fpn
calibration data
Prior art date
Application number
PCT/CN2017/112440
Other languages
English (en)
French (fr)
Inventor
胡涛
刘怀宇
曹子晟
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2017/112440 priority Critical patent/WO2019100270A1/zh
Priority to CN201780087782.9A priority patent/CN110574363A/zh
Publication of WO2019100270A1 publication Critical patent/WO2019100270A1/zh
Priority to US16/725,711 priority patent/US20200137334A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/67—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00—Diagnosis, testing or measuring for television systems or their details
    • H04N17/002—Diagnosis, testing or measuring for television systems or their details for television cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/67—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response
    • H04N25/671—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response for non-uniformity detection or correction
    • H04N25/672—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response for non-uniformity detection or correction between adjacent sensors or output registers for reading a single image

Definitions

  • the invention relates to an image noise reduction technology, in particular to an image noise calibration method and device applied to an image processing device, an image noise reduction method and device, and an image processing device.
  • CMOS Complementary Metal-Oxide-Semiconductor Transistor
  • FPN Fixed Pattern Noise
  • ADC amplifier For megapixel image sensors, a large number of ADC amplifiers are required. Due to individual differences in photodiodes per pixel, the concentration of the dopants, and the variation of the FET, the spatial difference in the output signal is caused to the pixel, and this difference usually does not change with time, thereby causing corresponding fixed-mode noise.
  • on-chip noise reduction There are two main types of noise suppression algorithms for fixed-mode noise FPN, namely on-chip noise reduction and off-chip noise reduction.
  • the principle of on-chip noise reduction is that, first, after an integration time, the pixel outputs a signal containing the photo-generated signal and the amplifier offset, which is stored in the on-chip memory cell. Next, after the pixel is reset, a signal containing only the amplifier offset is output, which is stored in another on-chip memory cell. By making a difference between the two outputs, the offset of the amplifier can be eliminated, thereby achieving the purpose of eliminating the FPN.
  • On-chip noise reduction requires an image sensor with on-chip special hardware circuitry and several memory locations for signal storage and comparison.
  • Off-chip noise reduction requires the back-end Image Signal Processor (ISP) to have FPN noise reduction and an additional frame buffer, which is similar to the on-chip solution. Due to the existence of the frame buffer, in addition to requiring a large storage overhead, a large delay and more sensor mode switching are brought about, thereby affecting the real-time performance and stability of the system. In addition, the FPN is reacquired each time the phone is turned on or each time the mode is changed, which may cause a delay.
  • ISP Image Signal Processor
  • An image noise calibration method includes:
  • the image sensor comprising an image sensitive element array, the raw image data being output by the image sensitive cell array in an optical black state;
  • the FPN calibration data is used for noise reduction of the image sensitive unit array, and the number of the FPN calibration data Less than the number of image sensitive units in the image sensor.
  • An image noise reduction method includes:
  • the original image data is compensated based on the compensation data.
  • An image noise calibration apparatus includes a processor that executes a computer readable instruction set implementation:
  • the image sensor comprising an image sensitive element array, the raw image data being output by the image sensitive cell array in an optical black state;
  • the FPN calibration data is used for noise reduction of the image sensitive unit array, and the number of the FPN calibration data Less than the number of image sensitive units in the image sensor.
  • An image noise reduction device includes a noise reduction module, and the noise reduction module is configured to perform noise reduction processing on original image data output by an image sensor, where the noise reduction processing includes:
  • the original image data is compensated based on the compensation data.
  • An image processing apparatus comprising:
  • An image sensor for outputting raw image data
  • An image processor communicatively coupled to the image sensor for processing image data
  • the noise reduction circuit pre-stores fixed pattern noise FPN calibration data of the image sensor, the noise reduction circuit is respectively connected with the image sensor and the image sensor; and the noise reduction circuit is used for Performing a noise reduction process on the original image data output by the image sensor, the noise reduction process includes: acquiring original image data output by the image sensor; calculating compensation data of the original image data according to the FPN calibration data, and The original image data is compensated based on the compensation data.
  • the image noise calibration method and device, the image noise reduction method and device, and the image processing device the data amount of the calibration data is smaller than the number of pixels of the original image data, which can save storage space and simplify the calculation process.
  • the image noise reduction method because the FPN calibration data is pre-stored, does not need to reacquire the FPN every time the power is turned on or each time the mode is changed, so the delay caused thereby can be avoided.
  • FIG. 1 is a block diagram showing the structure of an image processing apparatus according to an embodiment of the present invention.
  • FIG. 2 is a flow chart showing the characteristic calibration of an image noise reduction method according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of characteristic calibration according to an embodiment of the present invention.
  • FIG. 4 is a flow chart of adaptive noise reduction of an image noise reduction method according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of image compensation according to an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of comparison of noise reduction effects according to an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of an image noise calibration apparatus according to an embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of an image noise reduction device according to an embodiment of the present invention.
  • Figure 9 is a block diagram showing the structure of an image processing apparatus according to an embodiment of the present invention.
  • a component when referred to as being “fixed” to another component, it can be directly on the other component or the component can be present.
  • a component When a component is considered to "connect” another component, it can be directly connected to another component or possibly a central component.
  • a component When a component is considered to be “set to” another component, it can be placed directly on another component or possibly with a centered component.
  • the terms “vertical,” “horizontal,” “left,” “right,” and the like, as used herein, are for illustrative purposes only.
  • the present disclosure provides an image processing apparatus, which may be an image processing module applied to various electronic devices, such as a camera integrated in a terminal electronic device such as a mobile phone or a tablet, a heat dissipation structure, or may be independent.
  • a camera integrated in a terminal electronic device such as a mobile phone or a tablet
  • a heat dissipation structure or may be independent.
  • Shooting device such as a camera.
  • the camera can be used in a mobile platform including, but not limited to, an aircraft, a spacecraft, and the like.
  • the image processing device includes an image sensor for sensing an optical signal to obtain raw image data.
  • the image sensor includes an image sensor array, and the original image data may be initial data obtained by analog-to-digital conversion of a voltage or current signal output by the image sensor cell array.
  • the present disclosure provides an image noise reduction method.
  • the image noise reduction method performs real-time noise reduction on the original image data output by the image sensor based on pre-stored FPN calibration data.
  • the FPN calibration data may be pre-stored in a storage unit of the noise reduction module of the image processing device or in a storage unit of the third-party processing device.
  • the noise reduction module is the image sensor, or an image processor, or a noise reduction circuit connected to the image sensor.
  • the compensation of the original image data may be implemented in a noise reduction module of the image processing device, the noise reduction module being the image sensor, or an image processor, or The noise reduction circuit connected to the image sensor.
  • the image noise reduction method includes: acquiring original image data output by the image sensor; calculating compensation data of the original image data according to pre-stored fixed pattern noise FPN calibration data of the image sensor; The original image data is compensated.
  • the image noise reduction method further comprises: first generating a compensation level K according to the exposure information of the image sensor. Since the FPN calibration data is calculated under a specific calibration environment (for example, specific exposure parameters, etc.), and the image sensor is automatically adjusted when the image data is sensed under normal conditions, it may be different from the calibration. The value in the environment, therefore, needs to be adjusted by the compensation level K. After the compensation level K value is determined, the original image data outputted by the image sensor is compensated according to the FPN calibration data and the compensation level K value, thereby achieving the purpose of denoising.
  • the specific K value algorithm and the algorithm for compensating the original image data according to the FPN calibration data and the K value are further detailed in the following embodiments.
  • the present disclosure also provides an image noise calibration method, wherein the image noise calibration method includes: acquiring original image data output by an image sensor, the image sensor includes an image sensor array, and the original image data is Outputting the image sensitive cell array in an optical black state; determining fixed mode noise FPN calibration data of the image sensor based on original image data output by the image sensitive cell array in an optical black state, the FPN calibration data being used
  • the noise reduction of the image sensitive unit array, and the number of the FPN calibration data is smaller than the number of image sensitive units in the image sensor.
  • the image noise calibration method further includes: acquiring a dark current correction value (Optical Black, OB) of the image sensitive cell array; and original image data and the image output based on the image sensitive cell array
  • the dark current correction value of the sensitive cell array determines the FPN calibration data of the image sensor, and the FPN calibration data of the image sensor is a data value from which the dark current correction value is removed.
  • the dark current correction value OB is data output by the sensor due to the presence of dark current under optical black conditions; this data is related to the image sensor itself and is a constant value, which is usually measured and provided by the manufacturer when the image sensor is shipped.
  • the OB value may also be determined from the sensed data output by the image sensor, for example, the vertical OB and/or the horizontal OB are calculated from the raw image data.
  • the pixel value of the original image data sensed by the image sensor in the OB state is the same as the OB value, and various sensors such as the impurity concentration due to the individual difference of the sensing unit corresponding to each pixel.
  • the intrinsic factor causes the pixel value to deviate from the OB value, and thus the FPN calibration data is calculated based on the deviation of the pixel value from the OB value.
  • the generation of the FPN calibration data may be generated before leaving the factory, or may be generated when the user first uses the image processing device, or may be generated later according to actual needs of the user (eg, resetting the image processing device). Time).
  • FIG. 1 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention.
  • the image processing device 1 includes an image sensor 10, a noise reduction circuit 12, and an image processor 14.
  • the image sensor 10 is configured to sense an optical signal to obtain original image data.
  • the image sensor 10 includes an image sensor array, and the raw image data may be RGB mode digital initial data obtained by analog-to-digital conversion of a voltage or current signal output by the image sensitive cell array.
  • the raw image data is a pixel value arranged in rows and columns (as shown in FIG. 3).
  • the image sensor 10 may be a CCD (Charge Coupled Device) and a CMOS (Complementary Metal-Oxide Semiconductor) or other similar device capable of converting an optical image into an electronic signal.
  • CCD Charge Coupled Device
  • CMOS Complementary Metal-Oxide Semiconductor
  • the noise reduction circuit 12 can be a complex programmable logic device (Complex Programmable) Logic Device, CPLD), Field Programmable Gate Array (FPGA) or other similar programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • the noise reduction circuit 12 is used to store FPN calibration data.
  • the noise reduction circuit 12 calculates the compensation data according to the FPN calibration data when performing the noise reduction function, and compensates at least part of the pixels in the original image data sensed by the image sensor 10 according to the compensation data.
  • the FPN calibration data is determined according to a compensation level, and in other embodiments, the FPN calibration data may also be determined without a compensation level, for example, the image sensor is under different conditions (different exposure gains or When the change is not large at different temperatures, the compensation level may not be calculated, but the compensation data may be directly determined by the FPN calibration data.
  • the image processor 14 is configured to acquire statistical information of the image sensor 10 and determine whether the noise reduction function of the noise reduction circuit 12 is enabled according to statistical information. In some embodiments, the image processor 14 is further configured to generate a compensation level according to the statistical information of the image sensor 10. In some embodiments, the statistical information of the image sensor 10 includes, but is not limited to, Exposure Gain (EG), exposure time (etc.), and the exposure gain EG includes analog gain (AG), digital gain. (Digital Gain, DG). In some embodiments, the system exposure gain value of the image processing device 1 can be set to the product of the analog gain AG and the digital gain DG.
  • the image processor 14 may be a central processing unit (CPU), or may be another general-purpose processor, a digital signal processor (DSP), or an application specific integrated circuit (ASIC). , Field-Programmable Gate Array (FPGA), etc.
  • the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, and the processor 71 is a control center of the image processing apparatus 1, and connects the entire image processing apparatus 1 by using various interfaces and lines. Various parts of it.
  • the image sensor 10 is connected to the noise reduction circuit 12 and the image processor 14, respectively.
  • the image sensor 10 can be coupled to the noise reduction circuit 12 and the image processor 14 in a variety of serial or parallel manners.
  • I2C Inter-Integrated Circuit
  • GPIO General Purpose I/O Ports
  • USB Universal Serial Bus
  • Controller area network Controller area network
  • the image sensor 10 communicates with the noise reduction circuit 12 and the image processor 14 through an I2C bus. Pick up.
  • the image processor 14 and the noise reduction circuit 12 can also be communicatively coupled through various serial or parallel communication interfaces. For example, Mobile Industry Processor Interface (MIPI), Low-Voltage Differential Signaling (LVDS), High Definition Multimedia Interface (HDMI), and integrated circuits (Inter -Integrated Circuit, I2C) Bus, General Purpose I/O Ports (GPIO), etc.
  • the image processor 14 reads data from the noise reduction circuit 12 and the image sensor 10 and transmits control commands through the same or different communication interfaces, for example, the image processor 14 Data can be read from the noise reduction circuit 12 via MIPI, and control commands can be sent to the noise reduction circuit 12 via I2C or GPIO.
  • the image processor 14 can be an Image Signal Processor (ISP).
  • ISP Image Signal Processor
  • FIG. 2 is a flowchart of a method for image noise calibration according to an embodiment of the present invention.
  • Step 201 Acquire original image data output by the image sensor.
  • the image sensor 10 includes an array of image sensitive elements, the raw image data being outputted by the image sensitive cell array in an optical black state.
  • the calibration environment of the image sensor is first set.
  • the calibration environment setting includes:
  • the image sensor 10 is in an optical black (OB) state
  • the noise reduction circuit 12 operates in a non-FPN denoising state; (At this time, the FPN calibration data is empty, and the FPN check data is 0).
  • the image sensor 10 still has a dark current under the optical black condition, and outputs data which is related to the property of the image sensor 10 and is a certain value.
  • the operating parameters of the image sensor are also set.
  • the inventors have found that the FPN of the image sensor changes with the change of the operating parameters of the image sensor, then the operating parameters of the image sensor are set such that the FPN of the image sensor is more obvious, for example, the operating parameter settings include : setting the Auto Exposure (AE) mode to the Manual mode; dividing the analog gain AG and the digital gain DG of the image sensor 10 Do not set to the default value.
  • the analog gain AG value is set to 4x
  • the value of the digital gain DG is set to 1x.
  • the preset value can be appropriately set according to the actual needs and the accumulated experience value, and is not limited to the value defined in the embodiment.
  • the inventors found that the FPN of the image sensor does not change significantly with the operating parameters of the image sensor, so the operating parameters of the image sensor can be set more relaxed.
  • step 201 it is further included determining that there is no valid FPN calibration data.
  • the noise reduction circuit 12 or the image processor 14 first reads the data in the storage unit of the noise reduction circuit 12 to determine whether there is valid FPN calibration data. If there is no valid FPN calibration data, Then step 201 is performed. Determining whether there is valid FPN calibration data, including determining whether the FPN calibration data and the FPN calibration data are correctly verified in the noise reduction circuit, if the FPN calibration data exists in the noise reduction circuit and the FPN calibration The data is verified correctly and it is determined that there is valid FPN calibration data in the memory unit. The absence of the FPN calibration data includes the FPN calibration data being 0 or null or a default value.
  • the image sensor 10 is controlled to acquire at least one frame of original image data, and the original image data may be initial data obtained by analog-to-digital conversion of a voltage or current signal collected by the image sensor 10.
  • each frame of raw image data is a pixel value arranged in rows and columns (as shown in FIG. 3).
  • the original image data shown in FIG. 3 is raw image data of the RGB Bayer domain. It can be understood that, in some embodiments, the original image data may also be data of other modes.
  • Step 202 Determine fixed mode noise FPN calibration data of the image sensor based on the original image data.
  • the generation of the FPN calibration data may be performed on an external processing device (eg, a PC or other type of computing device, etc.) having data processing capabilities.
  • the raw image data is output to the processing device, and FPN calibration data is generated by an FPN calibration instruction set running on the processing device.
  • the quantity of the original image data is at least one frame, and determining the FPN calibration data of the image sensor based on the original image data output by the image sensitive unit array includes:
  • the FPN calibration data of the image sensor is determined based on an average of at least one frame of raw image data output by the image sensitive cell array.
  • determining the FPN calibration data of the image sensor comprises determining FPN calibration data of each image sensitive unit based on original image data output by the image sensitive unit array.
  • the FPN calibration data for each image sensitive unit is used as the FPN calibration data for the image sensor.
  • only FPN data greater than a threshold in the FPN data of each of the image sensitive units is used as FPN calibration data for the image sensor.
  • the FPN calibration data of the image sensor whose noise is less than the threshold is 0 or null or a default value.
  • determining the fixed mode noise FPN calibration data of the image sensor based on the original image data comprises:
  • the dark current correction value is also referred to as an optical black (OB) value, which is data output by the image sensor due to the presence of dark current under optical black conditions; this data is related to the image sensor itself and is a constant value, usually The image sensor has been measured and provided by the manufacturer when it is shipped from the factory;
  • OB optical black
  • the FPN calibration data of the image sensor is a dark current correction value removed Data value.
  • the output raw image data contains dark current values and FPN, and the dark current correction value (OB value) of the image sensor is generally left to be corrected by the image processor, that is, the OB value is removed in the image processor.
  • the calibration data may also be selected to include both the OB value and the FPN, and the OB value is not required to be removed by the image processor.
  • the original image data of one frame is taken as an example, wherein the FPN calibration data of each image sensitive unit is used as the FPN calibration data of the image sensor.
  • FIG. 1 The schematic diagram of the generation of the FPN calibration data is shown in FIG.
  • the resulting raw data is arranged as shown in the left frame of FIG.
  • a pixel array of m (each pixel array corresponding to an image sensitive unit), the pixel array being a Bayer pattern.
  • the number of the FPN calibration data may be the same as the number of the image sensitive units, or may be small The number of image sensitive units.
  • the generation rule of the FPN calibration data may be:
  • FGr ij , FGb ij , FR j , FB j are calibration data.
  • the FPN calibration data is generated by the rule that the image sensitive unit array is One of the column of image sensitive units determines FPN calibration data of the column of image sensitive cells based on raw image data output by the column of image sensitive cells, and the FPN calibration data of the column of image sensitive cells is used for the column of image sensitive cells Noise reduction.
  • the FPN calibration data of the line of image sensitive units may be determined based on the original image data output by the line of image sensitive units, the FPN of the line of the sensitive unit The calibration data is used for noise reduction of the line of image sensitive units.
  • the column of image sensitive cells is used to output image data of a class M channel
  • the FPN calibration data of the column of image sensitive cells includes FPN calibration data of the class M channel.
  • the original image data includes a blue channel (B ij ), a red channel (R ij ), and two green channels (Gr ij , Gb ij ).
  • the FPN calibration data includes calibration data (Gr j , Gb j , R j , B j ) of four channels.
  • the number of FPN calibration data of a column of image sensitive cells is less than the number of the column of image sensitive cells. Further, the number of FPN calibration data corresponding to each type of channel is smaller than the number of image sensitive units corresponding to the one type of channel. This can reduce the storage of FPN calibration data, simplify the calibration process and noise reduction process.
  • the column of image sensitive cells is used to output image data of a class M channel, and the number of FPN calibration data of the column of image sensitive cells is M.
  • the generation rule of the FPN calibration data may be:
  • Gr j OB-Avg(Gr 1j :Gr nj )
  • Gb j OB-Avg(Gb 1j :Gb nj )
  • R j OB-Avg(R 1j :R nj )
  • j is the number of columns
  • Avg(a 1 : a s ) is an averaging function defined as
  • the meaning is to take the average value of the Gr pixels in a certain pixel column.
  • the FPN calibration data generated according to the above rules is a 2*m array, and its size is 2*m bytes, as shown in the box in the upper right corner of FIG.
  • the FPN calibration value corresponding to each pixel is the difference between the OB and the average value of the pixel on one column.
  • the FPN calibration data is an array of 2*m. For a column of image sensitive units, the number of image sensitive units is 4n, and there are only four FPN calibration data, that is, one FPN calibration data for each channel.
  • the same column of the image sensitive unit can be divided into blocks, and the FPN calibration data is separately calculated for each block.
  • the image is divided into upper and lower blocks, and for each of the upper and lower blocks, a 2*m array is calculated, and then the pixel compensation of the upper and lower blocks is respectively given to the corresponding calibration data array.
  • the determination rule can be:
  • the array is divided into N pixel array sub-regions (each pixel array sub-region corresponds to one image block), and the starting row number and the last row of each pixel array sub-region are respectively s, t, N is greater than or equal to 2, s is a positive integer greater than or equal to 1; t is a positive integer greater than or equal to 2;
  • the FPN calibration data is N m*2 arrays Gr j , R j , B j , G j , and the determination rule is:
  • Gr j OB-Avg(Gr sj :Gr tj );
  • Gb j OB-Avg(Gb sj :Gb tj );
  • R j OB-Avg(R sj :R tj );
  • B j OB-Avg(B sj :B tj ).
  • the channel ie, Gr, R, B, Gb, etc.
  • the FPN data of the channel is the same.
  • the number of FPN calibration data is one.
  • the FPN calibration data is a row of data F j , and the determination rule is:
  • F j OB - Avg (Gr 1j : Gr nj , Gb 1j : Gb nj , R 1j : R nj , B 1j : B nj ).
  • Step 203 Store the FPN calibration data to a preset memory.
  • the FPN calibration data is burned into a storage unit of the internal noise reduction module of the image processing device, and the noise reduction module is the image sensor 10, or the image processor 14, or A noise reduction circuit 12 coupled to the image sensor. If stored in the storage unit of the noise reduction circuit 12, the FPN calibration data and the verification data may be first transmitted to the image processor 14, and then the image processor 14 may calibrate the FPN calibration data. The test data is burned into the memory unit of the noise reduction circuit 12.
  • the FPN calibration method further includes: generating verification data according to the FPN calibration data, and saving verification data of the FPN calibration data to the preset memory.
  • the FPN check data is generated based on the FPN calibration data, and the check data is a check value calculated for the original data by a specified algorithm to protect the integrity of the data. When the receiver uses the same algorithm to calculate the check value again, if the two check values are the same, the data is complete.
  • the check data may be generated by various suitable check data algorithms, such as Parity Check, BCC block check character, and LRC vertical redundancy check ( Longitudinal Redundancy Check), Cyclic Redundancy Check (CRC), MD5, SHA, MAC and other digest algorithms.
  • the check data is generated by using a CRC algorithm, and the check data of the FPN calibration data is obtained as CRC data.
  • the FPN calibration method further includes: verifying whether the FPN calibration data stored in the preset memory is complete.
  • the verification can be done in the processing device.
  • the processing device reads the FPN calibration data from the storage unit of the noise reduction circuit 12 by the image processor 14, and then calculates the verification value by using the same algorithm as the verification data in the storage unit. If the check value is consistent with the check data of the FPN calibration data stored in the storage unit, indicating the FPN stored in the storage unit of the noise reduction circuit 12 The calibration data is burned completely.
  • the generation of the FPN calibration data can be generated before the image processing apparatus 1 is shipped from the factory and stored in the noise reduction circuit 12, or can be generated when the image processing apparatus 1 is first run.
  • the generation of the FPN calibration data can be performed in any processing device having data processing capabilities.
  • the processing device includes a processor capable of executing a predetermined set of computer readable instructions to implement the image noise calibration method.
  • the processor may generate the FPN calibration data according to the generation rule of the FPN calibration data described above when the processor executes the computer readable instruction set.
  • the image noise reduction method calculates compensation data of the original image data according to the FPN calibration data of the image sensor 10 and the original image data output by the image sensor 10, and according to the compensation data pair.
  • the raw image data is compensated to achieve noise reduction.
  • the FPN calibration data of the image sensor is pre-stored in a storage unit of an internal noise reduction module of the image processing device, the image noise reduction method being applied to the noise reduction module.
  • the noise reduction module is the image sensor 10, or the image processor 14, or the noise reduction circuit 12 connected to the image sensor.
  • the FPN calibration data of the image sensor 10 is stored in a memory unit of the noise reduction circuit 12, and the image noise reduction method is performed by the noise reduction circuit 12.
  • the FPN calibration data of the image sensor 10 is stored in a noise reduction module of the image sensor 10, the image noise reduction method being performed by a noise reduction module of the image sensor.
  • the image sensor 10 is directly connected to the image processor 14, and outputs image data after performing noise reduction to the image processor 14.
  • the FPN calibration data of the image sensor 10 is stored in the image processor, the image noise reduction method being performed by the image processor. At this time, it is not necessary to provide the noise reduction circuit 12, and the image sensor 10 is directly connected to the image processor 14.
  • the image noise reduction method is performed when it is determined that the exposure information corresponding to the original image data satisfies a predetermined condition, and/or when it is determined that the FPN calibration data of the image sensor is successfully verified.
  • the exposure information includes an exposure gain.
  • the exposure gain is determined based on a product of the analog gain and the digital gain.
  • the exposure gain corresponding to the original image data is not less than the exposure gain corresponding to the FPN calibration data, it is determined that the exposure information corresponding to the original image data satisfies a predetermined condition.
  • Performing verification on the FPN calibration data of the image sensor includes: calculating calibration data according to the pre-stored FPN calibration data, and the calculated verification data is consistent with the pre-stored verification data, and determining the FPN calibration data. It is valid, that is, it is determined that the FPN calibration data of the image sensor is successfully verified.
  • FIG. 4 is a flowchart of an image noise reduction method according to an embodiment of the present invention.
  • the FPN calibration data of the image sensor 10 is stored in a storage unit of the noise reduction circuit 12, and the image noise reduction method is performed by the noise reduction circuit 12.
  • the image processor 14 determines that the exposure information corresponding to the original image data satisfies a predetermined condition, and/or determines that the FPN calibration data in the storage unit of the noise reduction circuit 12 is successfully verified, the noise reduction The circuit performs the image noise reduction method based on the enabling of the image processor 14.
  • the image processor 14 reads exposure information of the image sensor 10 from the image sensor 10.
  • the exposure information of the image sensor 10 includes, but is not limited to, an exposure gain, an exposure time, an exposure amount, and the like.
  • the exposure gain includes an analog gain AG, a digital gain DG, and the exposure gain is determined based on the analog gain AG and the digital gain DG, and the calculation method may be a conventional calculation method such as addition, multiplication, or weighted averaging. The specific calculation method can also be calculated based on the statistical data of the experimental data.
  • the exposure gain is determined based on a product of the analog gain AG and the digital gain DG.
  • the exposure gain corresponding to the original image data is not less than the exposure gain corresponding to the FPN calibration data, it is determined that the exposure information corresponding to the original image data satisfies a predetermined condition.
  • the exposure gain is determined to be 4x when determining the FPN calibration data, and if the exposure gain is not less than 4x, the adaptive denoising function of the noise reduction circuit 12 is determined to be enabled.
  • the verification data of the image sensor is also stored in advance in the storage unit of the noise reduction circuit.
  • the image processor 14 reads the FPN calibration data from the storage unit of the noise reduction circuit, and calculates the calibration based on the read FPN calibration data. The data is verified, and the calculated verification data is consistent with the verification data stored in the storage unit of the noise reduction circuit, and then the FPN calibration data is determined to be valid. If the calculated parity data does not match the parity data stored in the memory unit of the noise reduction circuit, the adaptive noise reduction function of the noise reduction circuit 12 is turned off. In some embodiments, after the shutdown, the user may be prompted whether to generate FPN calibration data, and after the user determines that the FPN calibration data needs to be generated, the method described in FIG. 2 is entered. Image noise calibration method.
  • Step 401 Acquire original image data output by the image sensor.
  • the image sensor 10 includes an array of image sensitive elements, the raw image data being output by the image sensitive cell array in a normal operating mode.
  • the image sensor 10 is controlled to acquire at least one frame of original image data, and the original image data may be digital initial data obtained by analog-to-digital conversion of a voltage or current signal collected by the image sensor 10.
  • each frame of raw image data is a pixel value arranged in rows and columns (as shown in Figure 5).
  • the original image data shown in FIG. 5 is raw image data of the RGB Bayer domain. It can be understood that, in some embodiments, the original image data may also be data of other modes, such as Ycbcr mode data.
  • Step 402 Calculate compensation data of the original image data according to the FPN calibration data of the image sensor and the original image data stored in advance.
  • the calculating the compensation data of the original image data according to the FPN calibration data of the image sensor and the original image data stored in advance includes: acquiring a compensation level; calculating based on the FPN calibration data and the compensation level Compensation data of the original image data.
  • the compensation level may also be determined based on the FPN calibration data without calculating a compensation level. For example, for an image sensor that does not change much under different conditions (for example, different exposure gains or different temperatures), at this time, the compensation data may be directly determined by using FPN calibration data without calculating a compensation level, for example, by using original image data and The addition or subtraction of the FPN calibration data is arithmetically performed to calculate the compensation data.
  • the compensation level is determined based on exposure information corresponding to the FPN calibration data of the image sensor and exposure information corresponding to the original image data, and/or temperature information and information corresponding to the FPN calibration data of the image sensor.
  • the temperature information corresponding to the original image data is determined.
  • the compensation level is positively correlated with the exposure gain, and the greater the exposure gain, the greater the compensation level.
  • a curve of the compensation level K value and the exposure gain value (for example, a linear curve or an exponential curve or other function curve) can be simulated according to the statistical data, and then obtained according to the curve and the exposure gain value.
  • Compensation level K value it is also possible to determine the K value corresponding to different exposure gain values based on an interpolation table obtained in advance. Typically the noise introduced by the analog gain will be slightly smaller, so in some embodiments, the compensation level K value is determined based on the digital gain DG, which is positively correlated with the digital gain DG, the larger the digital gain DG, The compensation level K value is larger.
  • the compensation level K value and the number The gain DG is the same, for example, when the digital gain DG is 1x, the compensation level K is 1.
  • the variation curve of the compensation level K value and the digital gain DG can be simulated according to the statistical data, and then the compensation level K value is obtained according to the variation curve and the digital gain value.
  • the auto exposure AE mode of the image sensor 10 is set to an automatic mode, and therefore, during operation of the image sensor 10, the analog gain AG and the digital gain DG of the image sensor 10 are at
  • the image processor 14 determines the compensation level K value of each frame image based on the real-time acquired analog gain AG and digital gain DG, and transmits the compensation level K value of each frame at the frame rate of the image sensor 10.
  • the image processor 14 sends the compensation level K value to the noise reduction circuit 12. In some embodiments, the image processor 14 transmits a compensation level K value per frame to the noise reduction circuit 12 at a frame rate of the image sensor 10.
  • the noise reduction circuit receives a compensation level K value transmitted from the image processor 14; and calculates compensation data of the original image data based on the FPN calibration data and the compensation level.
  • Step 403 the noise reduction circuit 12 receives the compensation level K value sent from the image processor 14; and calculates compensation data of the original image data based on the FPN calibration data and the compensation level.
  • the compensation data is calculated based on a product of the FPN calibration data and the compensation level.
  • the number of the FPN calibration data is smaller than the number of the image sensing units in the image sensor unit array of the image sensor.
  • Compensating data of the original image data output by the one column of the image sensitive unit is calculated based on the FPN calibration data of the one column of the image sensitive unit for one of the image sensitive units in the image sensitive cell array when calculating the compensation data, wherein The number of FPN calibration data of the column of image sensitive cells is smaller than the number of image sensitive cells in the column of image sensitive cells.
  • the column of image sensitive units is configured to output image data of the M-type channel
  • the FPN calibration data of the column of the image sensitive unit includes FPN calibration data of the M-type channel
  • the FPN calibration data corresponding to each type of channel The number is smaller than the number of image sensitive units corresponding to the one type of channel. This can reduce the storage of FPN calibration data, simplify the calibration process and noise reduction process.
  • the column of image sensitive cells is used to output image data of a class M channel, and the number of FPN calibration data of the column of image sensitive cells is M.
  • the original image data includes a blue channel (B ij ), a red channel (R ij ), and two green channels (Gr ij , Gb ij ).
  • the FPN calibration data includes four channels of calibration data (Gr j , Gb j , R j , B j ), the FPN calibration data is an array of 2*m, and for a column of image sensitive units, the number of image sensitive units is 4n, the number of FPN calibration data is four, that is, one FPN calibration data corresponding to each type of channel.
  • the FPN calibration data of the image sensor is FPN calibration data of a portion of the image sensitive unit in the image sensitive unit array, and each FPN calibration data is used to calculate an image sensitive unit output corresponding to the FPN calibration data.
  • the compensation data of the original image data For example, in the FPN calibration process of image noise, for a pixel unit whose FPN calibration data is smaller than a preset threshold, the FPN calibration data is not used as the FPN calibration data of the image sensor 10, and only the pixel unit whose FPN calibration data reaches a preset threshold is used.
  • the FPN calibration data is used as the FPN calibration data of the image sensor 10. At this time, when the compensation data is calculated for the original image data, the compensation data is calculated only for the pixel unit whose FPN calibration data reaches the preset threshold.
  • Step 404 the noise reduction circuit 12 compensates each pixel in the original image data according to the compensation data.
  • the compensated image data is output to the image processor 14 for further image processing.
  • the image processing includes, but is not limited to, data compression and back-end interface interface control, as well as data transmission, control, and the like, including image preview, lens focus control, and user interface.
  • Compensating the original image data according to the compensation data comprises: adding or subtracting the compensation data from the original image data.
  • the left border of FIG. 5 shows a pixel array of original image data (each pixel corresponds to an image sensitive unit of the image sensor), and the box in the upper right corner shows the FPN calibration data, and the box in the lower right corner Is the compensation rule.
  • the compensation data is the product of K and the corresponding calibration data, and the compensated pixel value is the sum of the original pixel value and the compensation data.
  • the compensation rules are as follows:
  • Gr ij ' Gr ij +k*Gr j
  • R ij ' R ij +k*R j
  • i is the number of rows, j is the number of columns, and k is the compensation level;
  • the step of setting the image sensor 10 may be further included, in which the working mode of the image sensor 10 may be manually or automatically set to a normal working mode,
  • the AE mode of the image sensor is an automatic mode in which the analog gain and digital gain are automatically adjusted with the shooting environment.
  • the compensation rule for the pixels in the above-described embodiment is implemented by simple addition and multiplication. It can be understood that other functions based on K and FPN calibration data can also be used to determine the compensated pixel values. It can be appreciated that in other embodiments, other algorithms may be employed to compensate for pixels of the original image data. It is also possible to consider the effects of noise caused by temperature changes of the sensor, wherein the effect of temperature on the noise can be determined by a temperature-noise curve or an interpolation table.
  • FIG. 6 is a comparison chart before and after correction, wherein the uppermost one is uncorrected, and the middle is the effect of the image sensor on the inside of the image noise reduction, and the bottom is the drop described by the disclosure.
  • the effect of noise reduction after noise reduction is obvious, and the noise reduction method described in the present disclosure is more effective.
  • the FPN calibration data is a row of data, and therefore is processed for pixel data of one line and one row at the time of processing, so it is a row level cache.
  • the row-level cache is mainly a mode in which the storage corresponding to the image processing block is read line by line.
  • the traditional noise reduction method requires one frame of all pixel data to be performed, so it is a frame level buffer.
  • the noise reduction method shown in the present disclosure adopts row-level buffering, pixel-level processing delay, and can perform FPN noise reduction processing in real time without using frame-level buffering and frame-level delay for FPN noise reduction, so it is very suitable. Imaging systems with high requirements for real-time and image quality, such as first-person view wireless image transmission devices.
  • the noise reduction method shown in the present disclosure may analyze the relevant statistical value information by collecting the original image data sent by the image sensor 10, and calculate the compensation level K value in real time based on the statistical value information, and real-time feedback. Giving the noise reduction circuit 12 a self-adaptive implementation Noise reduction.
  • the FPN noise reduction method introduced by the noise reduction method of the present disclosure is interposed between the image sensor and the back-end image signal processor, and solves the image sensor and the back-end image processor that do not have the FPN noise reduction. awkward.
  • the noise reduction method shown in the present disclosure works in the original image data (for example, Bayer Raw) domain, does not disturb the existing image signal processing process, and avoids the trouble of complicated post-processing denoising.
  • the FPN calibration data fits perfectly with the image sensor's Bayer pattern and data memory arrangement, eliminating the need for frame buffering and data reordering throughout the process.
  • the noise reduction method shown in the present disclosure performs off-chip noise reduction, avoids the need for on-chip noise reduction for special hardware circuits and memory cells, and uses the noise reduction circuit 12 to implement adaptive noise reduction. More flexible and convenient.
  • the noise reduction method shown in the present disclosure can automatically turn on according to ambient light without utilizing user intervention by using the computing resources and storage space of the noise reduction circuit 12 and the software intelligent configuration. Turn off the FPN noise reduction function, which is more intelligent and has great practical value.
  • FIG. 7 is a schematic structural diagram of an image noise calibration device.
  • the image noise calibration device 7 includes a processor 71, a memory 72, and a communication device 73.
  • the memory 72 can be used to store a computer program and/or module or computer readable instruction set, the processor 71 running or executing a computer program and/or module or computer readable instruction set stored in the memory 72, Calibration of image noise (such as the image noise calibration method shown in Figure 2).
  • the memory 72 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function (such as an audio and video playback function, etc.), and the like; the storage data area may be stored according to the image. The data created by the use of the noise calibration device 7 and the like.
  • the memory 72 may include a high-speed random access memory, and may also include a non-volatile memory such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SMC), and a secure digital (SD). Card, flash card, at least one disk storage device, flash device, or other volatile solid state storage device.
  • a non-volatile memory such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SMC), and a secure digital (SD).
  • SSD secure digital
  • flash card at least one disk storage device, flash device, or other volatile solid state storage device.
  • the processor 71 may be a central processing unit (CPU), or may be another general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), Ready-made programmable gate Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.
  • the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, and the processor 71 is a control center of the image noise calibration device 7, and is connected to the image noise calibration device by using various interfaces and lines. The various parts of 7.
  • the image noise calibration device 7 further includes at least one communication device 73.
  • the communication device 73 can be a wired communication device or a wireless communication device.
  • the wired communication device includes a communication port, such as a universal serial bus (USB), a controller area network (CAN), a serial and/or other standard network connection, and an integrated circuit (Inter- Integrated Circuit, I2C) bus, etc.
  • the wireless communication device can employ any type of wireless communication system, such as Bluetooth, infrared, Wireless Fidelity (WiFi), cellular technology, satellite, and broadcast.
  • the cellular technology may include mobile communication technologies such as second generation (2G), third generation (3G), fourth generation (4G) or fifth generation (5G).
  • the projection control device 3 is configured to communicate with the depth image acquisition system 1 through the communication device 73 to acquire an image acquired by the depth image acquisition system, and analyze and process the image. A 3D point cloud is obtained, and the control object and the projection plane are recognized from the 3D point cloud, and the positional relationship between the projected picture boundary and the control object relative to the projected picture is calculated, and a control signal is generated according to the positional relationship.
  • the projection control device 3 is further configured to communicate with the projection source device 4 through the communication device 73 to transmit the control signal to the projection source device to implement an interactive operation on the projection content.
  • the schematic diagram is only an example of the image noise calibration device 7, and does not constitute a limitation on the image noise calibration device 7, and may include more or less components than illustrated, or Combining certain components, or different components, for example, the image noise calibration device 7 may also include an input/output device, a display device, and the like according to actual needs.
  • the input and output device can include any suitable input device including, but not limited to, a mouse, a keyboard, a touch screen, or a contactless input, such as gesture input, voice input, and the like.
  • the display device may be a liquid crystal display (LCD), a light emitting diode (LED) display, an Organic Light-Emitting Diode (OLED) or other suitable display. .
  • the processor 71 executes the computer program and/or module or computer readable instruction set calculation Machine readable instruction set implementation:
  • the image sensor comprising an image sensitive element array, the raw image data being output by the image sensitive cell array in an optical black state;
  • the FPN calibration data is used for noise reduction of the image sensitive unit array, and the number of the FPN calibration data Less than the number of image sensitive units in the image sensor.
  • Determining the FPN calibration data of the image sensor based on the original image data output by the image sensing unit array including:
  • FPN calibration data of the column of image sensitive cells based on original image data output by the column of image sensitive cells for one of the column of image sensitive cells in the array of image sensitive cells, wherein the FPN calibration data of the column of image sensitive cells is used
  • the noise reduction of the column of image sensitive cells is performed, and the number of FPN calibration data of the column of image sensitive cells is less than the number of the column of image sensitive cells.
  • the column of image sensitive units is configured to output image data of the M-type channel
  • the FPN calibration data of the column of the image sensitive unit includes FPN calibration data of the M-type channel
  • the FPN calibration data corresponding to each type of channel The number is smaller than the number of image sensitive units corresponding to the one type of channel.
  • the raw image data is image data of a Bayer domain.
  • the column of image sensitive cells is used to output image data of a class M channel, and the number of FPN calibration data of the column of image sensitive cells is M.
  • the original image data is image data of an RGB Bayer domain
  • the M-type channel includes a blue channel, a red channel, and two green channels.
  • the number of FPN calibration data for the column of image sensitive cells is one.
  • determining the FPN calibration data of the image sensor based on the original image data output by the image sensitive cell array comprises:
  • the FPN data larger than the threshold among the FPN data of each of the image sensitive units is used as the FPN calibration data of the image sensor.
  • the processor executing the set of computer readable instructions further implements:
  • Determining the image sensor based on raw image data output by the image sensitive cell array FPN calibration data including:
  • the FPN calibration data of the image sensor is a dark current correction value removed Data value.
  • the number of the original image data is at least one frame.
  • Determining the FPN calibration data of the image sensor based on the original image data output by the image sensing unit array including:
  • the FPN calibration data of the image sensor is determined based on an average of at least one frame of raw image data output by the image sensitive cell array.
  • the image noise calibration device 7 also burns the FPN calibration data of the image sensor into a memory unit of the noise reduction module.
  • the noise reduction module is the image sensor, or is an image processor, or is a noise reduction circuit connected to the noise reduction circuit.
  • the image noise calibration apparatus is further configured to generate verification data according to the FPN calibration data; and burn the FPN calibration data and the verification data to a storage unit of the noise reduction module. in.
  • the check data is generated using a cyclic redundancy check algorithm.
  • the image noise calibration device is further configured to verify the FPN calibration data based on the verification data.
  • the image noise calibration apparatus is further configured to: determine that no valid FPN calibration data is stored in the image sensitive cell array.
  • the method for determining that the FPN calibration data is not stored in the image sensitive unit array includes: reading data in the storage unit, and if there is no FPN calibration data or the FPN calibration data fails to be verified, determining No valid FPN calibration data is stored in the image sensitive cell array.
  • FIG. 8 is a schematic structural diagram of an image noise reduction device 8 according to the present invention.
  • the image noise reduction device 8 is communicatively coupled to the image sensor for performing noise reduction processing on the original image data output by the image sensor.
  • the image noise reduction device 8 includes a noise reduction module 81, a storage unit 82, and a communication unit 83.
  • the storage unit 82 is configured to pre-store FPN calibration data of the image sensor.
  • the noise reduction module 81 is configured to perform noise reduction processing on the original image data output by the image sensor based on the FPN calibration data.
  • the communication unit 83 is configured to be in communication with the image sensor.
  • the storage unit 82 and the communication unit 83 are similar to the memory 72 and the communication device 73 of the image noise calibration device 7, and the memory 72 and the communication device 73 of any of the image noise calibration devices 7 are also applicable thereto. Let me repeat.
  • the noise reduction process includes:
  • the original image data is compensated based on the compensation data.
  • the calculating the compensation data of the original image data according to the FPN calibration data of the image sensor and the original image data stored in advance includes:
  • Compensating data of the original image data is calculated based on the FPN calibration data and the compensation level.
  • the noise reduction process is performed when it is determined that exposure information corresponding to the original image data satisfies a predetermined condition, and/or when it is determined that the FPN calibration data verification of the image sensor is successful.
  • the exposure information includes an exposure gain.
  • the compensation level is positively correlated with the exposure gain
  • the greater the exposure gain the greater the compensation level
  • the exposure gain corresponding to the original image data is not less than the exposure gain corresponding to the FPN calibration data, it is determined that the exposure information corresponding to the original image data satisfies a predetermined condition.
  • the exposure gain is determined based on a product of analog gain and digital gain.
  • the FPN calibration data of the image sensor is pre-stored in a memory unit of the noise reduction module.
  • the noise reduction module is the image sensor, or an image processor, or a noise reduction circuit coupled to the image sensor.
  • the noise reduction module is a noise reduction circuit, and the noise reduction circuit is connected to the image processor;
  • the noise reduction circuit is based on the image
  • the processor is enabled to perform the noise reduction process.
  • the noise reduction circuit further stores calibration data of the image sensor in advance
  • the noise reduction process further includes:
  • the noise reduction circuit transmits FPN calibration data and verification data of the image sensor to the image processor, and the image processor performs verification on the FPN calibration data of the image sensor.
  • the noise reduction module is in a noise reduction circuit; the noise reduction process further includes:
  • the noise reduction circuit receives a compensation level transmitted from the image processor
  • the noise reduction circuit calculates compensation data of the original image data based on the image processor calibration data and the compensation level.
  • the compensation level is determined based on exposure information corresponding to the FPN calibration data of the image sensor and exposure information corresponding to the original image data, and/or based on FPN calibration data of the image sensor Corresponding temperature information and temperature information corresponding to the original image data are determined.
  • the compensation data is calculated based on a product of the FPN calibration data and the compensation level.
  • the image sensor includes an array of image sensitive cells, the number of FPN calibration data being less than the number of image sensitive cells in the array of image sensitive cells.
  • the calculating the compensation data of the original image data according to the FPN calibration data of the image sensor stored in advance includes:
  • the raw image data is image data of a Bayer domain.
  • the column of image sensitive units is configured to output image data of the M-type channel
  • the FPN calibration data of the column of the image sensitive unit includes FPN calibration data of the M-type channel
  • the FPN calibration data corresponding to each type of channel The number is smaller than the number of image sensitive units corresponding to the one type of channel.
  • the column of image sensitive units is used to output image data of the M type channel, and the number of FPN calibration data of the column of the image sensitive unit is M;
  • Compensating data of image data of the M-type channel in the column of the image sensitive unit is respectively calculated according to the M FPN calibration data, wherein each FPN calibration data is used to calculate compensation data of all image data in one type of channel.
  • the FPN calibration data of the image sensor is FPN calibration data of a portion of the image sensitive unit in the image sensitive unit array, and each FPN calibration data is used to calculate an image sensitive unit output corresponding to the FPN calibration data.
  • the compensation data of the original image data is FPN calibration data of a portion of the image sensitive unit in the image sensitive unit array, and each FPN calibration data is used to calculate an image sensitive unit output corresponding to the FPN calibration data.
  • the compensating the original image data according to the compensation data comprises adding or subtracting the compensation data from the original image data.
  • FIG. 9 is a schematic structural diagram of an image processing apparatus 9 according to an embodiment of the present invention.
  • the image processing device 9 includes an image sensor 90, an image processor 92, and a noise reduction circuit 94.
  • the image sensor 90 is used to output raw image data.
  • the image processor 92 is communicatively coupled to the image sensor 90 for processing image data.
  • the noise reduction circuit 94 prestores fixed pattern noise FPN calibration data of the image sensor 90, and the noise reduction circuit 94 communicates with the image sensor 90 and the image processor 92, respectively.
  • the noise reduction circuit 94 is configured to perform noise reduction processing on the original image data output by the image sensor.
  • the noise reduction process includes: acquiring original image data output by the image sensor; calculating compensation data of the original image data according to the FPN calibration data, and compensating the original image data according to the compensation data.
  • the image processor 92 is configured to acquire exposure information corresponding to the original image data, and when the exposure information corresponding to the original image data is determined to meet a predetermined condition, enabling the noise reduction circuit to execute Denoising processing;
  • the image processor 92 is configured to acquire the FPN calibration data of the image sensor 90 from the noise reduction circuit 94, and enable the noise reduction circuit 94 to perform when it is determined that the FPN calibration data verification of the image sensor 90 is successful.
  • the noise reduction process is configured to acquire the FPN calibration data of the image sensor 90 from the noise reduction circuit 94, and enable the noise reduction circuit 94 to perform when it is determined that the FPN calibration data verification of the image sensor 90 is successful. The noise reduction process.
  • the exposure information includes an exposure gain.
  • the compensation level is positively correlated with the exposure gain
  • the greater the exposure gain the greater the compensation level
  • the exposure gain corresponding to the original image data is not less than the exposure gain corresponding to the FPN calibration data, it is determined that the exposure information corresponding to the original image data satisfies a predetermined condition.
  • the exposure gain is determined based on a product of analog gain and digital gain.
  • the noise reduction circuit 94 also prestores parity data for the FPN calibration data of the image sensor.
  • the image processor 92 is further configured to acquire, from the noise reduction circuit 94, verification data of the FPN calibration data of the image sensor 90, when the calibration data and the verification data are based on the FPN.
  • the noise reduction circuit 94 is enabled to perform the noise reduction process when it is determined that the FPN calibration data of the image sensor 90 is successfully verified.
  • the noise reduction circuit 94 is further configured to receive a compensation level transmitted from the image processor 92; and calculate the original image data based on the image processor 92 calibration data and the compensation level. Compensation data.
  • the compensation level is determined based on exposure information corresponding to the FPN calibration data of the image sensor 90 and exposure information corresponding to the original image data, and/or based on the FPN of the image sensor 90.
  • the temperature information corresponding to the calibration data and the temperature information corresponding to the original image data are determined.
  • the compensation data is calculated based on a product of the FPN calibration data and the compensation level.
  • the image sensor 90 includes an array of image sensitive cells, the number of FPN calibration data being less than the number of image sensitive cells in the array of image sensitive cells.
  • the FPN calibration number of the image sensor according to pre-storage The compensation data of the original image data is calculated, including:
  • the raw image data is image data of a Bayer domain.
  • the column of image sensitive units is configured to output image data of the M-type channel
  • the FPN calibration data of the column of the image sensitive unit includes FPN calibration data of the M-type channel
  • the FPN calibration data corresponding to each type of channel The number is smaller than the number of image sensitive units corresponding to the one type of channel.
  • the column of image sensitive units is used to output image data of the M type channel, and the number of FPN calibration data of the column of the image sensitive unit is M;
  • Compensating data of image data of the M-type channel in the column of the image sensitive unit is respectively calculated according to the M FPN calibration data, wherein each FPN calibration data is used to calculate compensation data of all image data in one type of channel.
  • the FPN calibration data of the image sensor is FPN calibration data of a portion of the image sensitive unit in the image sensitive unit array, and each FPN calibration data is used to calculate an image sensitive unit output corresponding to the FPN calibration data.
  • the compensation data of the original image data is FPN calibration data of a portion of the image sensitive unit in the image sensitive unit array, and each FPN calibration data is used to calculate an image sensitive unit output corresponding to the FPN calibration data.
  • the compensating the original image data according to the compensation data comprises adding or subtracting the compensation data from the original image data.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
  • Studio Devices (AREA)

Abstract

一种图像噪声标定方法和装置、图像降噪方法及装置、图像处理装置（1，9），所述图像标定方法包括：获取图像传感器（10，90）输出的原始图像数据，所述图像传感器（10，90）包括像敏单元阵列，所述原始图像数据是所述像敏单元阵列在光学黑状态下输出的；基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器（10，90）的固定模式噪声FPN标定数据，所述FPN标定数据用于所述像敏单元阵列的降噪，且所述FPN标定数据的数量小于所述图像传感器（10，90）中像敏单元的数量。所述图像噪声标定方法及装置、图像降噪方法及装置、图像处理装置（1，9）减少了FPN标定数据的数量，所述图像降噪方法及装置、图像处理装置（1，9）预先存储有FPN标定数据，避免每次开机或切换模式时重新确定FPN标定数据。

Description

图像噪声标定方法及装置、图像降噪方法及装置、图像处理装置 技术领域
本发明涉及一种图像降噪技术，尤其涉及一种应用于图像处理装置的图像噪声标定方法及装置、图像降噪方法及装置、图像处理装置。
背景技术
近年来，随着图像传感器，例如互补金属氧化物半导体(Complementary Metal-Oxide-Semiconductor Transistor，CMOS)的普及和广泛应用，图像质量与噪声处理也备受关注。固定模式噪声(Fixed Pattern Noise，FPN)作为一种非随机噪声，常发生于传感器中。由于图像传感器的构成中，每个感光二极管都需要搭配一个ADC放大器。针对百万像素级别的图像传感器，需要大量的ADC放大器。由于每像素的光电二极管个体差异，参杂浓度，以及场效应管的偏差，会对像素造成输出信号的空间差异，而这种差异通常不随时间而变化，由此会带来相应的固定模式噪声。
现有的固定模式噪声FPN的噪声抑制算法主要有两种，分别是片内降噪和片外降噪。片内降噪的原理是，首先，经过一段积分时间，像素输出一个包含光生信号和放大器失调的信号，该信号存储在片内存储单元中。接着，像素复位后输出一个仅包含放大器失调的信号，该信号被存储在另一个片内存储单元中。通过对两次输出做差，则可以消除放大器的失调，从而达到消除FPN的目的。片内降噪需要图像传感器具备片内特殊硬件电路，而且需要若干存储单元作为信号的存储和比较。片外降噪需要后端图像信号处理器(Image Signal Processor,ISP)具备FPN降噪功能，并能提供额外的帧缓存，其原理类似于片内方案。由于帧缓存的存在，除了需要较大的存储开销外，还会带来较大的延时和更多的传感器模式切换，从而影响系统的实时性和稳定性。此外，每次开机或者每次更换模式时重新获取FPN，因此可能会带来延迟。
发明内容
有鉴于此，有必要提供一种能够解决上述问题的图像噪声标定方法及装置、图像降噪方法及装置、图像处理装置。
一种图像噪声标定方法，包括：
获取图像传感器输出的原始图像数据，所述图像传感器包括像敏单元阵列，所述原始图像数据是所述像敏单元阵列在光学黑状态下输出的；
基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器的固定模式噪声FPN标定数据，所述FPN标定数据用于所述像敏单元阵列的降噪，且所述FPN标定数据的数量小于所述图像传感器中像敏单元的数量。
一种图像降噪方法，包括：
获取图像传感器输出的原始图像数据；根据预先存储的所述图像传感器的固定模式噪声FPN标定数据计算所述原始图像数据的补偿数据；
根据所述补偿数据对所述原始图像数据进行补偿。
一种图像噪声标定装置，包括处理器，所述处理器执行计算机可读指令集实现：
获取图像传感器输出的原始图像数据，所述图像传感器包括像敏单元阵列，所述原始图像数据是所述像敏单元阵列在光学黑状态下输出的；
基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器的固定模式噪声FPN标定数据，所述FPN标定数据用于所述像敏单元阵列的降噪，且所述FPN标定数据的数量小于所述图像传感器中像敏单元的数量。
一种图像降噪装置，所述图像降噪装置包括降噪模块，所述降噪模块用于对图像传感器输出的原始图像数据进行降噪处理，所述降噪处理包括：
获取图像传感器输出的原始图像数据；
根据预先存储的所述图像传感器的固定模式噪声FPN标定数据计算所述原始图像数据的补偿数据；
根据所述补偿数据对所述原始图像数据进行补偿。
一种图像处理装置，包括：
图像传感器，用于输出原始图像数据；
图像处理器，所述图像处理器与所述图像传感器通信连接，用于处理图像数据；
降噪电路，所述降噪电路预存储有所述图像传感器的固定模式噪声FPN标定数据，所述降噪电路分别与所述图像传感器、所述图像传感器通信连接；所述降噪电路用于对所述图像传感器输出的原始图像数据执行降噪处理，所述降噪处理包括：获取所述图像传感器输出的原始图像数据；根据所述FPN标定数据计算所述原始图像数据的补偿数据，及根据所述补偿数据对所述原始图像数据进行补偿。
所述图像噪声标定方法及装置、图像降噪方法及装置、图像处理装置，其标定数据的数据量小于原始图像数据的像素数量，可节省存储空间及简化计算流程。所述图像降噪方法，因为预存储有FPN标定数据，不用每次开机或者每次更换模式时重新获取FPN，因此可避免因此带来的延迟。
附图说明
图1是本发明一实施例的图像处理装置的结构示意图。
图2是本发明一实施例的图像降噪方法的特性标定流程图。
图3是本发明一实施例的特性标定示意图。
图4是本发明一实施例的图像降噪方法的自适应降噪流程图。
图5是本发明一实施例的图像补偿示意图。
图6是本发明一实施例的降噪效果比对示意图。
图7是本发明一实施例的图像噪声标定装置的结构示意图。
图8是本发明一实施例的图像降噪装置的结构示意图。
图9是本发明一实施例的图像处理装置的结构示意图。
主要元件符号说明
图像处理装置            1，9
图像传感器              10，90
降噪电路                12，94
图像处理器              14，92
图像噪声标定装置        7
处理器                  71
存储器                  72
通信装置                73
图像降噪装置            8
降噪模块                81
存储单元                82
通信单元                83
图像处理装置            9
如下具体实施方式将结合上述附图进一步说明本发明。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
需要说明的是，当组件被称为“固定于”另一个组件，它可以直接在另一个组件上或者也可以存在居中的组件。当一个组件被认为是“连接”另一个组件，它可以是直接连接到另一个组件或者可能同时存在居中组件。当一个组件被认为是“设置于”另一个组件，它可以是直接设置在另一个组件上或者可能同时存在居中组件。本文所使用的术语“垂直的”、“水平的”、“左”、“右”以及类似的表述只是为了说明的目的。
除非另有定义，本文所使用的所有的技术和科学术语与属于本发明的技术领域的技术人员通常理解的含义相同。本文中在本发明的说明书中所使用的术语只是为了描述具体的实施例的目的，不是旨在于限制本发明。本文所使用的术语“及/或”包括一个或多个相关的所列项目的任意的和所有的组合。
为使本发明的上述目的、特征和优点能够更加明显易懂，下面结合附图对本发明的具体实施方式做详细的说明。其中，本发明实施例结合示意图进行详细描述，在详述本发明实施例时，为便于说明，表示器件结构的剖面图会不依一般比例作局部放大，而且所述示意图只是示例，其在此不应限制本发明保护的范围。
本揭露提供一种图像处理装置，所述图像处理装置可为应用在各种电子装置中的图像处理模块，例如集成在手机、平板等终端电子装置中的拍摄装置、散热结构，也可以是独立的拍摄装置，例如相机等。所述相机可应用在移动平台中，所述移动平台包括，但不限于飞行器、航天器等。
所述图像处理装置包括图像传感器，所述图像传感器用于感测光信号得到原始图像数据。所述图像传感器包括像敏单元阵列，所述原始图像数据可为所述像敏单元阵列输出的电压或电流信号经过模数转换后得到的初始数据。
本揭露提供一种图像降噪方法。所述图像降噪方法基于预存储的FPN标定数据对所述图像传感器输出的原始图像数据进行实时降噪。所述FPN标定数据可以预存储在所述图像处理装置的降噪模块的存储单元中或所述第三方处理装置的存储单元中。所述降噪模块为所述图像传感器，或者为图像处理器，或者为与所述图像传感器连接的降噪电路。
在图像降噪方法中，对所述原始图像数据进行补偿可在所述图像处理装置的降噪模块中实现，所述降噪模块为所述图像传感器，或者为图像处理器，或者为与所述图像传感器连接的降噪电路。
所述图像降噪方法包括：获取图像传感器输出的原始图像数据；根据预先存储的所述图像传感器的固定模式噪声FPN标定数据计算所述原始图像数据的补偿数据；根据所述补偿数据对所述原始图像数据进行补偿。
在一实施例中，所述图像降噪方法还包括：先根据所述图像传感器的曝光信息生成补偿级别K。由于在计算FPN标定数据时是在特定的标定环境下(例如特定的曝光参数等)，而所述图像传感器在正常状态下感测图像数据时其曝光信息是自动调整的，有可能不同于标定环境下的值，因此，需要通过补偿级别K来进行调整。补偿级别K值确定后，对所述图像传感器输出的原始图像数据根据FPN标定数据及补偿级别K值进行补偿，从而达到去噪的目的。具体的K值算法及根据FPN标定数据及K值来补偿原始图像数据的算法在如下实施例中进一步详述。
预先存储的FPN标定数据的获取方法有多种。本揭露还提供一种图像噪声标定方法，其中，所述图像噪声标定方法包括：获取图像传感器输出的原始图像数据，所述图像传感器包括像敏单元阵列，所述原始图像数据是所 述像敏单元阵列在光学黑状态下输出的；基于所述像敏单元阵列在光学黑状态下输出的原始图像数据确定所述图像传感器的固定模式噪声FPN标定数据，所述FPN标定数据用于所述像敏单元阵列的降噪，且所述FPN标定数据的数量小于所述图像传感器中像敏单元的数量。
在一实施例中，所述图像噪声标定方法还包括：获取像敏单元阵列的暗电流矫正值(光学黑Optical Black，OB)；基于所述像敏单元阵列输出的原始图像数据和所述像敏单元阵列的暗电流矫正值确定所述图像传感器的FPN标定数据，所述图像传感器的FPN标定数据为去除了暗电流矫正值的数据值。
暗电流矫正值OB为光学黑条件下传感器因存在暗电流而输出的数据；这个数据与图像传感器本身相关，是一个常数值，通常在所述图像传感器出厂时已由厂商测定并提供。在一些实施例中，也可以根据图像传感器输出的感测数据确定OB值，例如，通过原始图像数据计算得到垂直OB及/或水平OB。通常在理想标准状态下，所述图像传感器在OB状态下感测的原始图像数据的像素值是与OB值相同的，而由于对应每像素的感测单元个体差异，参杂浓度等各种传感器内在因素导致像素值相对OB值存在偏差，因此基于所述像素值相对所述OB值的偏差来计算所述FPN标定数据。具体的FPN标定数据的计算方法请参阅如下具体实施例。在一些实施例中，FPN标定数据的生成可以在出厂前即生成，也可在使用者首次使用所述图像处理装置时生成，还可以在后续根据使用者实际需求生成(例如重置图像处理装置时)。
请参阅图1所示，为本发明一实施例的图像处理装置的结构示意图。所述图像处理装置1包括图像传感器10、降噪电路12及图像处理器14。其中所述图像传感器10用于感测光信号得到原始图像数据。所述图像传感器10包括像敏单元阵列，所述原始图像数据可为所述像敏单元阵列输出的电压或电流信号经过模数转换后得到的RGB模式的数字初始数据(Raw data)。在一些实施例中，所述原始图像数据为行列排布的像素值(如图3所示)。所述图像传感器10可为CCD(Charge Coupled Device，电荷耦合元件)和CMOS(Complementary Metal-Oxide Semiconductor，金属氧化物半导体元件)或其他能够将光学图像转换成电子信号的类似器件。
所述降噪电路12可为复杂可编程逻辑器件(Complex Programmable  Logic Device，CPLD)、现场可编程门阵列(Field Programmable Gate Array，FPGA)或其他类似可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。所述降噪电路12用于存储FPN标定数据。所述降噪电路12在执行降噪功能时，具体根据FPN标定数据计算补偿数据，并根据补偿数据对所述图像传感器10所感测得到的原始图像数据中的至少部分像素进行补偿。在一些实施例中，所述FPN标定数据根据补偿级别来确定，在其他一些实施例中，也可以不用补偿级别来确定所述FPN标定数据，例如，图像传感器在不同条件下(不同曝光增益或不同温度下)变化不大时，可以不需要计算补偿级别，而直接以所述FPN标定数据来确定所述补偿数据。
在一些实施例中，所述图像处理器14用于获取所述图像传感器10的统计信息，并根据统计信息确定是否使能所述降噪电路12的降噪功能。在一些实施例中，所述图像处理器14还用于根据所述图像传感器10的统计信息生成补偿级别。在一些实施例中，所述图像传感器10的统计信息包括但不限于曝光增益(Exposure Gain,EG)、曝光时间(等)，所述曝光增益EG包括模拟增益(Analogue Gain,AG)、数字增益(Digital Gain，DG)。在一些实施例中，所述图像处理装置1的系统曝光增益值可设置为模拟增益AG和数字增益DG的乘积。所述图像处理器14可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等，所述处理器71是所述图像处理装置1的控制中心，利用各种接口和线路连接整个图像处理装置1的各个部分。
所述图像传感器10分别与所述降噪电路12及所述图像处理器14相连接。在一些实施例中，所述图像传感器10可通过各种串行或并行的方式与所述降噪电路12及所述图像处理器14相连接。例如集成电路间(Inter-Integrated Circuit，I2C)总线、通用输入/输出端口(General Purpose I/O Ports，GPIO)、通用串行总线(universal serial bus,USB)、控制器局域网(Controller area network，CAN)、其他串行或并行通信连接接口等。在本实施例中，所述图像传感器10与降噪电路12及所述图像处理器14通过I2C总线通信连 接。
所述图像处理器14与所述降噪电路12也可通过各种串行或并行通信接口通信连接。例如，移动产业处理器接口(Mobile Industry Processor Interface简称MIPI)、低电压差分信号接口(Low-Voltage Differential Signaling，LVDS)、高清晰度多媒体接口(High Definition Multimedia Interface，HDMI)、集成电路间(Inter-Integrated Circuit，I2C)总线、通用输入/输出端口(General Purpose I/O Ports，GPIO)等。在一些实施例中，所述图像处理器14从所述降噪电路12及所述图像传感器10读取数据与发送控制命令可通过相同的或不同的通信接口，例如，所述图像处理器14可通过MIPI从所述降噪电路12读取数据，可通过I2C或GPIO发送控制指令至所述降噪电路12。所述图像处理器14可为图像信号处理器(Image Signal Processor,ISP)。
下面结合图2对获取预存的FPN标定数据的一种实施例进行描述。请参阅图2所示，为本发明一实施例的图像噪声标定方法的流程图。
步骤201，获取图像传感器输出的原始图像数据。
所述图像传感器10包括像敏单元阵列，所述原始图像数据是所述像敏单元阵列在光学黑状态下输出的。
一些实施方式中，首先设置图像传感器的标定环境。可选的，所述标定环境设置包括：
a)图像传感器10、图像处理器14连接正常，供电正常；
b)图像传感器10处于光学黑(Optical Black，OB)状态；
c)图像处理器14正常工作；
d)降噪电路12工作在非FPN去噪状态下；(此时FPN标定数据为空，FPN校验数据为0)。
所述图像传感器10在所述光学黑条件下仍存在暗电流，会输出数据，这个数据与所述图像传感器10的本身属性相关，是一个确定的值。
可选的，还设置图像传感器的工作参数。针对一些图像传感器，发明人发现图像传感器的FPN会随着图像传感器的工作参数的变化而变化，那么将图像传感器的工作参数设置在使得该图像传感器的FPN较明显，例如，该工作参数设置包括：将自动曝光(Auto Exposure，AE)模式设置为手动(Manual)模式；将所述图像传感器10的模拟增益AG和数字增益DG分 别设置为预设值。在本实施例中，所述模拟增益AG值设置为4x，所述数字增益DG的值设置为1x。可以理解的是，这个预设值可根据实际需要及累计经验值进行适当设置，并不限于本实施例所限定的数值。针对一些图像传感器，发明人发现图像传感器的FPN随着图像传感器的工作参数变化不明显，那么图像传感器的工作参数设置条件可以较宽松些。
可以理解的是，在一些实施例中，在执行步骤201之前还包括确定没有存在有效的FPN标定数据。例如，由所述降噪电路12或所述图像处理器14先读取所述降噪电路12的存储单元中的数据，判断是否存在有效的FPN标定数据，若不存在有效的FPN标定数据，则执行步骤201。其中判断是否存在有效的FPN标定数据，包括判断所述降噪电路中是否存在FPN标定数据及所述FPN标定数据是否校验正确，若所述降噪电路中存在FPN标定数据且所述FPN标定数据校验正确，且确定所述存储单元中存在有效的FPN标定数据。其中不存在所述FPN标定数据包括所述FPN标定数据为0或空或者一个默认值。
最后，获取原始图像数据。控制所述图像传感器10获取至少一帧原始图像数据，所述原始图像数据可为所述图像传感器10所采集的电压或电流信号经过模数转换后得到的初始数据(Raw data)。在一些实施例中，每一帧原始图像数据为行列排布的像素值(如图3所示)。图3所示的原始图像数据为RGB Bayer域的原始图像数据，可以理解的是，在一些实施例中，所述原始图像数据也可以是其他模式的数据。
步骤202，基于所述原始图像数据确定所述图像传感器的固定模式噪声FPN标定数据。在一实施例中，所述FPN标定数据的生成可在一具有数据处理能力的外部处理装置(例如PC或其他类型的计算设备等)上执行。输出所述原始图像数据至所述处理装置，通过运行在所述处理装置上的FPN标定指令集生成FPN标定数据。
其中，所述原始图像数据的数量为至少一帧，所述基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器的FPN标定数据，包括：
基于所述像敏单元阵列输出的每帧原始图像数据确定一帧FPN标定数据；将每帧FPN标定数据进行平均得到所述图像传感器的FPN标定数据；其中所述平均包括，但不限于算术平均、几何平均、平方平均、调和平均或 加权平均等各种计算平均值的方法。
或者，基于所述像敏单元阵列输出的至少一帧原始图像数据的均值确定所述图像传感器的FPN标定数据。
其中，确定所述图像传感器的FPN标定数据包括，基于所述像敏单元阵列输出的原始图像数据确定每个像敏单元的FPN标定数据。在一些实施例中，将每个像敏单元的FPN标定数据均作为所述图像传感器的FPN标定数据。在另一些实施例中，仅将所述每个像敏单元的FPN数据中大于阈值的FPN数据作为所述图像传感器的FPN标定数据。而噪声小于阈值的图像传感器的FPN标定数据为0或空或者一个默认值。
其中，基于所述原始图像数据确定所述图像传感器的固定模式噪声FPN标定数据包括：
获取像敏单元阵列的暗电流矫正值。其中所述暗电流矫正值又称为光学黑(Optical Black，OB)值，为光学黑条件下图像传感器因存在暗电流而输出的数据；这个数据与图像传感器本身相关，是一个常数值，通常在所述图像传感器出厂时已由厂商测定并提供；
基于所述像敏单元阵列输出的原始图像数据和所述像敏单元阵列的暗电流矫正值确定所述图像传感器的FPN标定数据，所述图像传感器的FPN标定数据为去除了暗电流矫正值的数据值。
输出的原始图像数据中是包含暗电流值和FPN的，图像传感器的暗电流矫正值(OB值)一般留在后面由图像处理器来矫正，也即在图像处理器中去除这OB值。可选的，在一些实施例中，也可以选择标定数据同时包含OB值和FPN，不需要由图像处理器另去除OB值。
如下，以一帧的原始图像数据为例进行说明，其中每个像敏单元的FPN标定数据均作为所述图像传感器的FPN标定数据。
所述FPN标定数据的生成示意图请参阅图3所示。
假定所述图像处理装置1的分辨率为n*m(n,m为大于1的正整数)，得到的原始数据排列如图3左边框中所示。原始数据中的像素以Grij,Rij,Bij,Gbij(i＝1,2,3,…,n；j＝1,2,3,…,m)的形式行列排布呈2n*m的像素阵列(每一像素阵列对应一像敏单元)，所述像素阵列为拜尔滤色阵列(Bayer Pattern)。
所述FPN标定数据的数量可以与所述像敏单元的数量一致，也可以小 于像敏单元的数量。例如，所述FPN标定数据的数量可以与所述像敏单元的数量一致时，所述FPN标定数据的生成规则可为：
FGrij＝OB-Grij
FGbij＝OB-Gbij
FRj＝OB-Rij
FBj＝OB-Bij
其中，FGrij，FGbij，FRj，FBj为标定数据。
发明人发现，现有的常规图像传感器的FPN表现为竖向条纹，每一列的偏移值都大致相同，因此上述实施例中，FPN标定数据的生成规则是：对于所述像敏单元阵列中的其中一列像敏单元，基于所述一列像敏单元输出的原始图像数据确定所述一列像敏单元的FPN标定数据，所述一列像敏单元的FPN标定数据用于所述一列像敏单元的降噪。可以理解的是，在其他实施例中，若FPN表现为横向条纹，也可以基于一行像敏单元输出的原始图像数据确定所述一行像敏单元的FPN标定数据，所述一行像敏单元的FPN标定数据用于所述一行像敏单元的降噪。
在一些实施例中，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据包括M类通道的FPN标定数据。如上所述的实施例中，所述原始图像数据包括一个蓝色通道(Bij)、一个红色通道(Rij)和两个绿色通道(Grij,Gbij)。所述FPN标定数据包括四个通道的标定数据(Grj，Gbj，Rj，Bj)。
在本揭露中，一列像敏单元的FPN标定数据的数量小于所述一列像敏单元的数量。进一步地，针对每一类通道对应的FPN标定数据的数量小于所述一类通道对应的像敏单元的数量。如此可以减少FPN标定数据的存储量，简化标定流程及降噪流程。在一些实施例中，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据的数量为M。
如图3所示的实施例中，所述FPN标定数据的生成规则可为：
Grj＝OB-Avg(Gr1j:Grnj)
Gbj＝OB-Avg(Gb1j:Gbnj)
Rj＝OB-Avg(R1j:Rnj)
Bj＝OB-Avg(B1j:Bnj)
其中：j为列数；
Avg(a1:as)为取平均函数，定义为
Figure PCTCN2017112440-appb-000001
例如，对Avg(Gr1j:Grnj)，其计算式为
Figure PCTCN2017112440-appb-000002
其含义为取某一像素列中Gr像素的平均值。
依照上述规则生成的FPN标定数据为2*m排布的阵列，其大小为2*m字节，如图3右上角的框中所示。对应于每一像素的FPN标定值均为OB与该像素在一列上的平均值的差值。
所述FPN标定数据为2*m的阵列，针对一列像敏单元，像敏单元的数量为4n，其FPN标定数据只有4个，即对应每个通道一个FPN标定数据。
可以理解的是，如果同一列像敏单元的FPN噪声差异较大，可以将同一列像敏单元进行区块划分，针对每一区块分别计算FPN标定数据。
例如，将图像划分为上下两块，针对上下两块，分别计算得到一个2*m的阵列，然后上下两块的像素补偿分别给予对应的标定数据阵列来进行。确定规则可为：
所述像素阵列为拜尔滤色阵列Grij,Rij,Bij,Gbij，其中i＝1,2,3,…,n；j＝1,2,3,…,m；所述像素阵列分割为N个像素阵列子区域(每个像素阵列子区域对应于一个图像区块)，每一像素阵列子区域的起始行号和最后一行的行号分别为s,t，N大于等于2，s为大于等于1的正整数；t为大于等于2的正整数；
所述FPN标定数据为N个m*2阵列Grj，Rj，Bj，Gj，确定规则为：
Grj＝OB-Avg(Grsj:Grtj)；
Gbj＝OB-Avg(Gbsj:Gbtj)；
Rj＝OB-Avg(Rsj:Rtj)；
Bj＝OB-Avg(Bsj:Btj)。
可以理解的是，在一些实施例中，为了简化计算，也可以忽略通道(channel，即Gr,R,B,Gb等)，计算得到一个1*m的阵列，这种计算方法下，几个通道的FPN数据是相同的。此时，对于一列像敏单元，其FPN标定数据的数量为1。确定规则可为：所述像素阵列为拜尔滤色阵列Grij,Rij,Bij,Gbij，其中i＝1,2,3,…,n；j＝1,2,3,…,m；
所述FPN标定数据为一行数据Fj，确定规则为：
Fj＝OB-Avg(Gr1j:Grnj，Gb1j:Gbnj，R1j:Rnj，B1j:Bnj)。
步骤203，存储所述FPN标定数据至预设存储器。在本实施例中，所述FPN标定数据烧录至所述图像处理装置的内部降噪模块的存储单元中，所述降噪模块为所述图像传感器10，或者为图像处理器14，或者为与所述图像传感器连接的降噪电路12。若存储在所述降噪电路12的存储单元中，可先传送所述FPN标定数据及校验数据至所述图像处理器14，然后所述图像处理器14将所述FPN标定数据及其校验数据烧录至所述降噪电路12的存储单元中。
进一步地，在一些实施例中，所述FPN标定方法还包括：根据FPN标定数据生成校验数据，及保存所述FPN标定数据的校验数据至所述预设存储器。所述FPN校验数据基于所述FPN标定数据生成，所述校验数据是为保护数据的完整性，用一种指定的算法对原始数据计算出的一个校验值。当接收方用同样的算法再算一次校验值，如果两次校验值一样，表示数据完整。所述校验数据可采用各种适宜的校验数据算法生成，例如奇偶校验(Parity Check)、BCC异或校验法(block check character，块校验码)、LRC纵向冗余校验(Longitudinal Redundancy Check)、循环冗余校验(Cyclic Redundancy Check，CRC)、MD5、SHA、MAC等摘要算法。本实施例中，所述校验数据采用CRC算法生成，得到FPN标定数据的校验数据为CRC数据。
进一步地，在一些实施例中，所述FPN标定方法还包括：校验存储在所述预设存储器中的FPN标定数据是否完整。所述校验可在所述处理装置中完成。所述处理装置通过所述图像处理器14从所述降噪电路12的存储单元中读取所述FPN标定数据，然后采用与生成所述存储单元中的校验数据相同的算法计算校验值，若所述校验值与所述存储单元中存储的FPN标定数据的校验数据一致，则表明存储在所述降噪电路12的存储单元中的所述FPN 标定数据烧录完整。
可以理解的是，所述FPN标定数据的生成可在所述图像处理装置1出厂前即生成并存储在所述降噪电路12中，也可在首次运行所述图像处理装置1时生成。所述FPN标定数据的生成可在任意一具备数据处理能力的处理装置中完成。所述处理装置包括处理器，所述处理器能够执行预定计算机可读指令集来实现所述图像噪声标定方法。所述处理器执行所述计算机可读指令集时可根据上所述的FPN标定数据的生成规则生成所述FPN标定数据。
本揭露所述的图像降噪方法根据预先存储的所述图像传感器10的FPN标定数据和所述图像传感器10输出的原始图像数据计算所述原始图像数据的补偿数据，并根据所述补偿数据对所述原始图像数据进行补偿以实现降噪。
在一些实施例中，所述图像传感器的FPN标定数据预先存储在所述图像处理装置的内部降噪模块的存储单元中，所述图像降噪方法应用于所述降噪模块中。所述所述降噪模块为所述图像传感器10，或者为图像处理器14，或者为与所述图像传感器连接的降噪电路12。例如，在一些实施例中，所述图像传感器10的FPN标定数据存储在所述降噪电路12的存储单元中，所述图像降噪方法由所述降噪电路12执行。在一些实施例中，所述图像传感器10的FPN标定数据存储在所述图像传感器10的降噪模块中，所述图像降噪方法由所述图像传感器的降噪模块执行。此时，不需要设置所述降噪电路12，所述图像传感器10直接与所述图像处理器14连接，将执行降噪后的图像数据输出至所述图像处理器14。在一些实施例中，所述图像传感器10的FPN标定数据存储在所述图像处理器中，所述图像降噪方法由所述图像处理器执行。此时，不需要设置所述降噪电路12，所述图像传感器10直接与所述图像处理器14连接。
其中，所述图像降噪方法是在确定所述原始图像数据对应的曝光信息满足预定条件时，和/或，确定所述图像传感器的FPN标定数据校验成功时执行的。
所述曝光信息包括曝光增益。所述曝光增益是基于所述模拟增益和所述数字增益的乘积确定的。当所述原始图像数据对应的曝光增益不小于所述FPN标定数据对应的曝光增益时，确定所述原始图像数据对应的曝光信息满足预定条件。
对所述图像传感器的FPN标定数据进行校验包括，根据预先存储的FPN标定数据计算其校验数据，所计算得到的校验数据与预先存储的校验数据一致，则判断所述FPN标定数据有效，即确定所述图像传感器的FPN标定数据校验成功。
请参阅图4所示，为本发明一实施例的图像降噪方法的流程图。其中，所述图像传感器10的FPN标定数据存储在所述降噪电路12的存储单元中，所述图像降噪方法由所述降噪电路12执行。当所述图像处理器14确定所述原始图像数据对应的曝光信息满足预定条件时，和/或，确定所述降噪电路12的存储单元中的FPN标定数据校验成功时，所述降噪电路基于所述图像处理器14的使能执行所述图像降噪方法。
所述图像处理器14从所述图像传感器10读取所述图像传感器10的曝光信息。在一些实施例中，所述图像传感器10的曝光信息包括，但不限于曝光增益、曝光时间、曝光量等。其中所述曝光增益包括模拟增益AG、数字增益DG，所述曝光增益基于所述模拟增益AG、数字增益DG确定，所述计算方法可为相加、相乘或加权平均等常规计算方法。具体计算方法也可依据实验数据统计计算得到。在一些实施例中，所述曝光增益基于所述模拟增益AG与所述数字增益DG的乘积确定。当所述原始图像数据对应的曝光增益不小于所述FPN标定数据对应的曝光增益时，确定所述原始图像数据对应的曝光信息满足预定条件。例如，在确定FPN标定数据时所述曝光增益确定为4x，若所述曝光增益不小于4x，确定使能所述降噪电路12的自适应去噪功能。在一些实施例中，还可以根据曝光量等参数值来确定是否使能自适应去噪功能。例如，若曝光量低于预设值，使能所述自适应去噪功能。
所述降噪电路的存储单元中还预先存储有所述图像传感器的校验数据。在对所述降噪电路12的存储单元中的FPN标定数据进行校验时，所述图像处理器14从降噪电路的存储单元读取FPN标定数据，根据读取的FPN标定数据计算其校验数据，所计算得到的校验数据与所述降噪电路的存储单元中存储的校验数据一致，则判断所述FPN标定数据有效。若所计算得到的校验数据与所述降噪电路的存储单元中存储的校验数据不一致，则关闭所述降噪电路12的自适应降噪功能。在一些实施例中，关闭后可提示用户是否需要生成FPN标定数据，在用户确定需要生成FPN标定数据后进入图2所述的 图像噪声标定方法。
步骤401，获取图像传感器输出的原始图像数据。
所述图像传感器10包括像敏单元阵列，所述原始图像数据是所述像敏单元阵列在正常工作模式下输出的。控制所述图像传感器10获取至少一帧原始图像数据，所述原始图像数据可为所述图像传感器10所采集的电压或电流信号经过模数转换后得到的数字初始数据(Raw data)。在一些实施例中，每一帧原始图像数据为行列排布的像素值(如图5所示)。图5所示的原始图像数据为RGB Bayer域的原始图像数据，可以理解的是，在一些实施例中，所述原始图像数据也可以是其他模式的数据，例如Ycbcr模式的数据。
步骤402，根据预先存储的所述图像传感器的FPN标定数据和所述原始图像数据计算所述原始图像数据的补偿数据。
其中，所述根据预先存储的所述图像传感器的FPN标定数据和所述原始图像数据计算所述原始图像数据的补偿数据，包括：获取补偿级别；基于所述FPN标定数据和所述补偿级别计算所述原始图像数据的补偿数据。可以理解的是，在一些实施例中，也可以不需要计算补偿级别，而直接基于所述FPN标定数据来确定所述补偿数据。例如，对于图像传感器在不同条件下(例如不同曝光增益或不同温度下)变化不大，此时，可以不用计算补偿级别，直接采用FPN标定数据来确定所述补偿数据，例如通过原始图像数据与所述FPN标定数据的加减等算术来计算所述补偿数据。
所述补偿级别是基于所述图像传感器的FPN标定数据对应的曝光信息和所述原始图像数据对应的曝光信息确定的，和/或，基于所述图像传感器的FPN标定数据对应的温度信息和所述原始图像数据对应的温度信息确定的。
其中所述补偿级别与所述曝光增益正相关，所述曝光增益越大所述补偿级别越大。在实际操作中，可根据统计数据模拟出补偿级别K值与所述曝光增益值的变化曲线(例如线性变化曲线或指数变化曲线或其他函数变化曲线)，然后根据该变化曲线及曝光增益值得到补偿级别K值。此外，还可以根据预先统计获得的插值表来确定不同的曝光增益值对应的K值。通常模拟增益引入的噪声会稍小，因此在一些实施例中，所述补偿级别K值基于所述数字增益DG确定，其与所述数字增益DG正相关，所述数字增益DG越大，所述补偿级别K值越大。在一些实施例中，所述补偿级别K值与所述数字 增益DG相同，例如数字增益DG为1x时，所述补偿级别K值为1。在实际操作中，可根据统计数据模拟出补偿级别K值与所述数字增益DG的变化曲线，然后根据该变化曲线及数字增益值得到补偿级别K值。
在所述自适应去噪状态，所述图像传感器10的自动曝光AE模式设置为自动模式，因此，在所述图像传感器10工作过程中，所述图像传感器10的模拟增益AG和数字增益DG处于变化状态，所述图像处理器14根据实时获取的模拟增益AG和数字增益DG确定每帧图像的补偿级别K值，并以所述图像传感器10的帧率发送每帧的补偿级别K值。
所述图像处理器14将所述补偿级别K值发送至所述降噪电路12。在一些实施例中，所述图像处理器14以所述图像传感器10的帧率发送每帧的补偿级别K值至所述降噪电路12。
所述降噪电路接收来自所述图像处理器14发送的补偿级别K值；并基于所述FPN标定数据和所述补偿级别计算所述原始图像数据的补偿数据。
步骤403，所述降噪电路12接收来自所述图像处理器14发送的补偿级别K值；并基于所述FPN标定数据和所述补偿级别计算所述原始图像数据的补偿数据。在一些实施例中，所述补偿数据是基于所述FPN标定数据与所述补偿级别的乘积计算的。
本揭露的降噪方法中，所述FPN标定数据的数量小于所述图像传感器的像敏单元阵列中像敏单元的数量。
在计算补偿数据时，对于所述像敏单元阵列中的其中一列像敏单元，基于所述一列像敏单元的FPN标定数据计算所述一列像敏单元输出的原始图像数据的补偿数据，其中，所述一列像敏单元的FPN标定数据的数量小于所述一列像敏单元中像敏单元的数量。
在一些实施例中，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据包括M类通道的FPN标定数据，每一类通道对应的FPN标定数据的数量小于所述一类通道对应的像敏单元的数量。如此可以减少FPN标定数据的存储量，简化标定流程及降噪流程。
在一些实施例中，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据的数量为M。所述根据预先存储的所述图像传感器的固定模式噪声FPN标定数据计算所述原始图像数据的补偿数据， 包括：根据所述M个FPN标定数据分别计算所述一列像敏单元中M类通道的图像数据的补偿数据，其中，每一个FPN标定数据用于计算一类通道中所有图像数据的补偿数据。如图5所示，所述原始图像数据包括一个蓝色通道(Bij)、一个红色通道(Rij)和两个绿色通道(Grij,Gbij)。所述FPN标定数据包括四个通道的标定数据(Grj，Gbj，Rj，Bj)，所述FPN标定数据为2*m的阵列，针对一列像敏单元，像敏单元的数量为4n，其FPN标定数据的数量为4个，即对应每类通道一个FPN标定数据。
在一些实施例中，所述图像传感器的FPN标定数据为所述像敏单元阵列中部分像敏单元的FPN标定数据，每一个FPN标定数据用于计算所述FPN标定数据对应的像敏单元输出的原始图像数据的补偿数据。例如，在图像噪声的FPN标定流程中，对于FPN标定数据小于预设阈值的像素单元，其FPN标定数据不作为所述图像传感器10的FPN标定数据，只有FPN标定数据达到预设阈值的像素单元的FPN标定数据才作为所述图像传感器10的FPN标定数据。此时，在对原始图像数据计算补偿数据时，仅针对FPN标定数据达到预设阈值的像素单元进行计算补偿数据。
步骤404，所述降噪电路12根据所述补偿数据对原始图像数据中的每一像素进行补偿。补偿后的图像数据输出至所述图像处理器14进行进一步的图像处理。所述图像处理包括，但不限于数据压缩与后端接口界面控制，以及数据传输、控制等工作，其中还包括影像预览、镜头对焦控制、使用界面等。
其中根据所述补偿数据对所述原始图像数据进行补偿，包括：将所述原始图像数据加上或减去所述补偿数据。
具体的FPN补偿数据的生成规则及像素的补偿规则请参阅图5所示。
图5左边框示出的是原始图像数据的像素阵列(每一像素对应所述图像传感器的一像敏单元)，所述右上角的框中示出的是FPN标定数据，右下角的框中是补偿规则。其中，补偿数据为K与相对应的标定数据的乘积，补偿后的像素值为原始像素值与补偿数据的和。
具体地，补偿规则如下：
Grij’＝Grij+k*Grj
Gbij’＝Gbij+k*Gbj
Rij’＝Rij+k*Rj
Bij’＝Bij+k*Bj
其中：i为行数，j为列数，k为补偿级别；
Grij、Gbij、Rij、Bij为原始感测的像素值；Grij’、Gbij’、Rij’、Bij’为补偿后的像素值。
可以理解的是，在所述步骤401前还可包括设置所述图像传感器10的步骤，在所述设置步骤中，可手动或自动设置所述图像传感器10的工作模式为正常工作模式，所述图像传感器的AE模式为自动模式，在该自动模式下，所述模拟增益和数字增益会随着拍摄环境自动调整。
为了简化所述降噪电路12的计算量，上所述的实施例中对像素的补偿规则采用简单的加法与乘法来实现。可以理解的是，还可以采用基于K与FPN标定数据的其他函数来确定补偿后的像素值。可以理解的是，在其他一些实施例中，还可以采用其他算法来对原始图像数据的像素进行补偿。还可以考虑传感器的温度变化带来的噪声影响，其中温度对噪声的影响可通过温度-噪声的变化曲线或插值表来确定。
请参阅图6所示，为校正前后的对比图，其中最上边的是没有校正过的，中间的是所述图像传感器片内降噪后的效果，最下边的是采用本揭露所述的降噪方法降噪后的效果，很明显，采用本揭露所述的降噪方法效果更好。
由图5及上述补偿规则可以看出，位于同一列的像素，其补偿数据是相同的。上述一些实施例中，所述FPN标定数据是一行数据，因此在处理时是针对一行一行的像素数据来进行处理的，所以是行级缓存。行级缓存主要是对应于图像处理这块的存储是采用的一行一行进行读取的模式。传统降噪方法需要一帧全部的像素数据后进行，所以是帧级缓存。本揭露所示的降噪方法采用行级缓存，像素级别的处理延迟，可以实时进行FPN降噪处理，而无需像其他方案采用帧级缓存，帧级别延迟来进行FPN的降噪，因此非常适合于对实时性和图像质量要求都很高的成像系统，比如第一人称视角无线图传设备。
上述一些实施例中，本揭露所示的降噪方法可以通过收集所述图像传感器10发送的原始图像数据，分析相关统计值信息，并基于所述统计值信息实时计算补偿级别K值，实时反馈给所述降噪电路12，从而实现自适应实 时降噪。
上述一些实施例中，本揭露所示的降噪方法引入的FPN降噪介于图像传感器和后端图像信号处理器之间，解决了不具备FPN降噪的图像传感器和后端图像处理器的尴尬。同时，本揭露所示的降噪方法工作在原始图像数据(例如Bayer Raw)域，不打乱现有的图像信号处理过程，避免了复杂的后处理去噪的麻烦。另外，FPN标定数据能够完美的配合图像传感器的像素排布模式(bayer pattern)和数据内存的排布，从而在整个处理过程中不需要帧缓存和数据的重排(reorder)。
上述一些实施例中，本揭露所示的降噪方法为片外执行降噪，可避免片内降噪对特殊硬件电路及存储单元的需求，且采用降噪电路12来实现自适应降噪，更灵活便利。
上述一些实施例中，本揭露所示的降噪方法通过利用所述降噪电路12的计算资源和存储空间，结合软件智能配置，可以在不需要用户介入的情况下，自动根据环境光线开启或关闭FPN降噪功能，更加智能化，具有很大的实用价值。
请参阅图7所示，为一种图像噪声标定装置的结构示意图。所述图像噪声标定装置7包括包括处理器71、存储器72及通信装置73。
所述存储器72可用于存储计算机程序和/或模块或计算机可读指令集，所述处理器71通过运行或执行存储在所述存储器72内的计算机程序和/或模块或计算机可读指令集，实现图像噪声的标定(例如图2所示的图像噪声标定方法)。所述存储器72可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如音视频播放功能等)等；存储数据区可存储根据图像噪声标定装置7的使用所创建的数据等。此外，存储器72可以包括高速随机存取存储器，还可以包括非易失性存储器，例如硬盘、内存、插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)、至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
所述处理器71可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门 阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等，所述处理器71是所述图像噪声标定装置7的控制中心，利用各种接口和线路连接图像噪声标定装置7的各个部分。
所述图像噪声标定装置7还包括至少一个通信装置73。
所述通信装置73可以是有线通信装置也可以是无线通信装置。其中所述有线通信装置包括通信端口，例如通用串行总线(universal serial bus,USB)、控制器局域网(Controller area network，CAN)、串行及/或其他标准网络连接、集成电路间(Inter-Integrated Circuit，I2C)总线等。所述无线通信装置可采用任意类别的无线通信系统，例如，蓝牙、红外线、无线保真(Wireless Fidelity,WiFi)、蜂窝技术，卫星，及广播。其中所述蜂窝技术可包括第二代(2G)、第三代(3G)、第四代(4G)或第五代(5G)等移动通信技术。
在本发明实施例中，所述投影控制装置3用于通过所述通信装置73与所述深度图像采集系统1通信以获取所述深度图像采集系统采集的图像，并对所述图像进行分析处理得到3D点云，并从3D点云中识别出控制物体及投影平面，计算出投影画面边界及控制物体相对所述投影画面的位置关系，并根据所述位置关系生成控制信号。所述投影控制装置3还用于通过所述通信装置73与所述投影源设备4通信连接以传送所述控制信号至所述投影源设备以实现对投影内容的交互操作。
本领域技术人员可以理解，所述示意图仅仅是所述图像噪声标定装置7的示例，并不构成对所述图像噪声标定装置7的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如所述图像噪声标定装置7还可以根据实际需要包括输入输出设备、显示装置等。所述输入输出设备可包括任意适宜的输入设备，包括但不限于，鼠标、键盘、触摸屏、或非接触式输入，例如，手势输入、声音输入等。所述显示装置可以是触液晶显示屏(Liquid Crystal Display,LCD)、发光二极管(Light Emitting Diode，LED)显示屏、有机电激光显示屏(Organic Light-Emitting Diode，OLED)或其他适宜的显示屏。
所述处理器71执行所述计算机程序和/或模块或计算机可读指令集计算 机可读指令集实现：
获取图像传感器输出的原始图像数据，所述图像传感器包括像敏单元阵列，所述原始图像数据是所述像敏单元阵列在光学黑状态下输出的；
基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器的固定模式噪声FPN标定数据，所述FPN标定数据用于所述像敏单元阵列的降噪，且所述FPN标定数据的数量小于所述图像传感器中像敏单元的数量。
所述基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器的FPN标定数据，包括：
对于所述像敏单元阵列中的其中一列像敏单元，基于所述一列像敏单元输出的原始图像数据确定所述一列像敏单元的FPN标定数据，所述一列像敏单元的FPN标定数据用于所述一列像敏单元的降噪，且所述一列像敏单元的FPN标定数据的数量小于所述一列像敏单元的数量。
在一些实施例中，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据包括M类通道的FPN标定数据，每一类通道对应的FPN标定数据的数量小于所述一类通道对应的像敏单元的数量。
在一些实施例中，所述原始图像数据为Bayer域的图像数据。
在一些实施例中，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据的数量为M。
在一些实施例中，所述原始图像数据为RGB Bayer域的图像数据；
所述M类通道包括一个蓝色通道、一个红色通道和两个绿色通道。
在一些实施例中，所述一列像敏单元的FPN标定数据的数量为1。
在一些实施例中，所述基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器的FPN标定数据，包括：
基于所述像敏单元阵列输出的原始图像数据确定每个像敏单元的FPN数据；
将所述每个像敏单元的FPN数据中大于阈值的FPN数据作为所述图像传感器的FPN标定数据。
在一些实施例中，所述处理器执行计算机可读指令集还实现：
获取像敏单元阵列的暗电流矫正值；
所述基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器 的FPN标定数据，包括：
基于所述像敏单元阵列输出的原始图像数据和所述像敏单元阵列的暗电流矫正值确定所述图像传感器的FPN标定数据，所述图像传感器的FPN标定数据为去除了暗电流矫正值的数据值。
在一些实施例中，所述原始图像数据的数量为至少一帧，
所述基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器的FPN标定数据，包括：
基于所述像敏单元阵列输出的每帧原始图像数据确定一帧FPN标定数据；将每帧FPN标定数据进行平均得到所述图像传感器的FPN标定数据；
或者，
基于所述像敏单元阵列输出的至少一帧原始图像数据的均值确定所述图像传感器的FPN标定数据。
在一些实施例中，所述的图像噪声标定装置7还将所述图像传感器的FPN标定数据烧录至降噪模块的存储单元中。
在一些实施例中，所述降噪模块为所述图像传感器，或者为图像处理器，或者为与所述降噪电路连接的降噪电路。
在一些实施例中，所述图像噪声标定装置还用于根据所述FPN标定数据生成校验数据；及将所述FPN标定数据和所述校验数据烧录至所述降噪模块的存储单元中。
在一些实施例中，所述校验数据采用循环冗余校验算法生成。
在一些实施例中，所述图像噪声标定装置还用于基于所述校验数据对所述FPN标定数据进行校验。
在一些实施例中，所述图像噪声标定装置还用于：确定所述像敏单元阵列中没有存储有效的FPN标定数据。其中“确定所述像敏单元阵列中没有存储有效的FPN标定数据所述方法”包括：读取所述存储单元中的数据，若不存在FPN标定数据或所述FPN标定数据校验失败，确定所述像敏单元阵列中没有存储有效的FPN标定数据。
请参阅图8所示，为本发明一种图像降噪装置8的结构示意图。所述图像降噪装置8与图像传感器通信连接，用于对所述图像传感器输出的原始图像数据进行降噪处理。
所述图像降噪装置8包括降噪模块81、存储单元82及通信单元83。所述存储单元82用于预先存储所述图像传感器的FPN标定数据。所述降噪模块81用于基于所述FPN标定数据对所述图像传感器输出的原始图像数据进行降噪处理。所述通信单元83用于与所述图像传感器通信连接。所述存储单元82与所述通信单元83类似于所述图像噪声标定装置7的存储器72、通信装置73，任何所述图像噪声标定装置7的存储器72、通信装置73也可适用于此，不再赘述。
所述降噪处理包括：
获取图像传感器输出的原始图像数据；
根据预先存储的所述图像传感器的固定模式噪声FPN标定数据计算所述原始图像数据的补偿数据；
根据所述补偿数据对所述原始图像数据进行补偿。
在一些实施例中，所述根据预先存储的所述图像传感器的FPN标定数据和所述原始图像数据计算所述原始图像数据的补偿数据，包括：
获取补偿级别；
基于所述FPN标定数据和所述补偿级别计算所述原始图像数据的补偿数据。
在一些实施例中，所述降噪处理是在确定所述原始图像数据对应的曝光信息满足预定条件时，和/或，确定所述图像传感器的FPN标定数据校验成功时执行的。
在一些实施例中，所述曝光信息包括曝光增益。
在一些实施例中，其中所述补偿级别与所述曝光增益正相关，所述曝光增益越大所述补偿级别越大。
在一些实施例中，当所述原始图像数据对应的曝光增益不小于所述FPN标定数据对应的曝光增益时，确定所述原始图像数据对应的曝光信息满足预定条件。
在一些实施例中，其中所述曝光增益是基于模拟增益和数字增益的乘积确定的。
在一些实施例中，所述图像传感器的FPN标定数据预先存储在降噪模块的存储单元中。
在一些实施例中，所述降噪模块为所述图像传感器，或者为图像处理器，或者为与所述图像传感器连接的降噪电路。
在一些实施例中，所述降噪模块为降噪电路，所述降噪电路与所述图像处理器相连；
当所述图像处理器确定所述原始图像数据对应的曝光信息满足预定条件时，和/或，确定所述像敏单元阵列的FPN标定数据校验成功时，所述降噪电路基于所述图像处理器的使能执行所述降噪处理。
在一些实施例中，所述降噪电路还预先存储有所述图像传感器的校验数据；
所述降噪处理还包括：
所述降噪电路将所述图像传感器的FPN标定数据和校验数据发送至所述图像处理器，用于所述图像处理器对所述图像传感器的FPN标定数据进行校验。
在一些实施例中，所述降噪模块为降噪电路中；所述降噪处理还包括：
所述降噪电路接收来自所述图像处理器发送的补偿级别；
所述降噪电路基于所述图像处理器标定数据和所述补偿级别计算所述原始图像数据的补偿数据。
在一些实施例中，所述补偿级别是基于所述图像传感器的FPN标定数据对应的曝光信息和所述原始图像数据对应的曝光信息确定的，和/或，基于所述图像传感器的FPN标定数据对应的温度信息和所述原始图像数据对应的温度信息确定的。
在一些实施例中，所述补偿数据是基于所述FPN标定数据与所述补偿级别的乘积计算的。
在一些实施例中，所述图像传感器包括像敏单元阵列，所述FPN标定数据的数量小于所述像敏单元阵列中像敏单元的数量。
在一些实施例中，所述根据预先存储的所述图像传感器的FPN标定数据计算所述原始图像数据的补偿数据，包括：
对于所述像敏单元阵列中的其中一列像敏单元，基于所述一列像敏单元的FPN标定数据计算所述一列像敏单元输出的原始图像数据的补偿数据，其中，所述一列像敏单元的FPN标定数据的数量小于所述一列像敏单元中 像敏单元的数量。
在一些实施例中，所述原始图像数据为Bayer域的图像数据。
在一些实施例中，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据包括M类通道的FPN标定数据，每一类通道对应的FPN标定数据的数量小于所述一类通道对应的像敏单元的数量。
在一些实施例中，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据的数量为M；
所述根据预先存储的所述图像传感器的固定模式噪声FPN标定数据计算所述原始图像数据的补偿数据，包括：
根据M个FPN标定数据分别计算所述一列像敏单元中M类通道的图像数据的补偿数据，其中，每一个FPN标定数据用于计算一类通道中所有图像数据的补偿数据。
在一些实施例中，所述图像传感器的FPN标定数据为所述像敏单元阵列中部分像敏单元的FPN标定数据，每一个FPN标定数据用于计算所述FPN标定数据对应的像敏单元输出的原始图像数据的补偿数据。
在一些实施例中，所述根据所述补偿数据对所述原始图像数据进行补偿，包括：将所述原始图像数据加上或减去所述补偿数据。
请参阅图9所示，为本发明一实施例的图像处理装置9的结构示意图。所述图像处理装置9包括图像传感器90、图像处理器92及降噪电路94。其中所述图像传感器90用于输出原始图像数据。所述图像处理器92与所述图像传感器90通信连接，用于处理图像数据。所述降噪电路94，所述降噪电路94预存储有所述图像传感器90的固定模式噪声FPN标定数据，所述降噪电路94分别与所述图像传感器90、所述图像处理器92通信连接；所述降噪电路94用于对所述图像传感器输出的原始图像数据执行降噪处理。
所述降噪处理包括：获取所述图像传感器输出的原始图像数据；根据所述FPN标定数据计算所述原始图像数据的补偿数据，及根据所述补偿数据对所述原始图像数据进行补偿。
在一些实施例中，所述图像处理器92用于获取所述原始图像数据对应的曝光信息，当确定所述原始图像数据对应的曝光信息满足预定条件时，使能所述降噪电路执行所述降噪处理；
和/或，
所述图像处理器92用于从所述降噪电路94获取所述图像传感器90的FPN标定数据，当确定所述图像传感器90的FPN标定数据校验成功时使能所述降噪电路94执行所述降噪处理。
在一些实施例中，所述曝光信息包括曝光增益。
在一些实施例中，其中所述补偿级别与所述曝光增益正相关，所述曝光增益越大所述补偿级别越大。
在一些实施例中，当所述原始图像数据对应的曝光增益不小于所述FPN标定数据对应的曝光增益时，确定所述原始图像数据对应的曝光信息满足预定条件。
在一些实施例中，其中所述曝光增益是基于模拟增益和数字增益的乘积确定的。
在一些实施例中，所述降噪电路94还预存有所述图像传感器的FPN标定数据的校验数据。
在一些实施例中，所述图像处理器92还用于从所述降噪电路94获取所述图像传感器90的FPN标定数据的校验数据，当基于所述FPN标定数据和所述校验数据确定所述图像传感器90的FPN标定数据校验成功时使能所述降噪电路94执行所述降噪处理。
在一些实施例中，所述降噪电路94还用于接收来自所述图像处理器92发送的补偿级别；并基于所述图像处理器92标定数据和所述补偿级别计算所述原始图像数据的补偿数据。
在一些实施例中，所述补偿级别是基于所述图像传感器90的FPN标定数据对应的曝光信息和所述原始图像数据对应的曝光信息确定的，和/或，基于所述图像传感器90的FPN标定数据对应的温度信息和所述原始图像数据对应的温度信息确定的。
在一些实施例中，所述补偿数据是基于所述FPN标定数据与所述补偿级别的乘积计算的。
在一些实施例中，所述图像传感器90包括像敏单元阵列，所述FPN标定数据的数量小于所述像敏单元阵列中像敏单元的数量。
在一些实施例中，所述根据预先存储的所述图像传感器的FPN标定数 据计算所述原始图像数据的补偿数据，包括：
对于所述像敏单元阵列中的其中一列像敏单元，基于所述一列像敏单元的FPN标定数据计算所述一列像敏单元输出的原始图像数据的补偿数据，其中，所述一列像敏单元的FPN标定数据的数量小于所述一列像敏单元中像敏单元的数量。
在一些实施例中，所述原始图像数据为Bayer域的图像数据。
在一些实施例中，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据包括M类通道的FPN标定数据，每一类通道对应的FPN标定数据的数量小于所述一类通道对应的像敏单元的数量。
在一些实施例中，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据的数量为M；
所述根据预先存储的所述图像传感器的固定模式噪声FPN标定数据计算所述原始图像数据的补偿数据，包括：
根据M个FPN标定数据分别计算所述一列像敏单元中M类通道的图像数据的补偿数据，其中，每一个FPN标定数据用于计算一类通道中所有图像数据的补偿数据。
在一些实施例中，所述图像传感器的FPN标定数据为所述像敏单元阵列中部分像敏单元的FPN标定数据，每一个FPN标定数据用于计算所述FPN标定数据对应的像敏单元输出的原始图像数据的补偿数据。
在一些实施例中，所述根据所述补偿数据对所述原始图像数据进行补偿，包括：将所述原始图像数据加上或减去所述补偿数据。
另外，对于本领域的普通技术人员来说，可以根据本发明的技术构思做出其它各种相应的改变与变形，而所有这些改变与变形都应属于本发明权利要求的保护范围。

Claims (94)

  1. 一种图像噪声标定方法，其特征在于，所述方法包括：
    获取图像传感器输出的原始图像数据，所述图像传感器包括像敏单元阵列，所述原始图像数据是所述像敏单元阵列在光学黑状态下输出的；
    基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器的固定模式噪声FPN标定数据，所述FPN标定数据用于所述像敏单元阵列的降噪，且所述FPN标定数据的数量小于所述图像传感器中像敏单元的数量。
  2. 根据权利要求1所述的图像噪声标定方法，其特征在于，
    所述基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器的FPN标定数据，包括：
    对于所述像敏单元阵列中的其中一列像敏单元，基于所述一列像敏单元输出的原始图像数据确定所述一列像敏单元的FPN标定数据，所述一列像敏单元的FPN标定数据用于所述一列像敏单元的降噪，且所述一列像敏单元的FPN标定数据的数量小于所述一列像敏单元的数量。
  3. 如权利要求2所述的图像噪声标定方法，其特征在于，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据包括M类通道的FPN标定数据，每一类通道对应的FPN标定数据的数量小于所述一类通道对应的像敏单元的数量。
  4. 如权利要求1或3所述的图像噪声标定方法，其特征在于，所述原始图像数据为Bayer域的图像数据。
  5. 如权利要求4所述的图像噪声标定方法，其特征在于，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据的数量为M。
  6. 如权利要求5所述的图像噪声标定方法，其特征在于，所述原始图像 数据为RGB Bayer域的图像数据；
    所述M类通道包括一个蓝色通道、一个红色通道和两个绿色通道。
  7. 如权利要求2所述的图像噪声标定方法，其特征在于，所述一列像敏单元的FPN标定数据的数量为1。
  8. 根据权利要求1所述的图像噪声标定方法，其特征在于，所述基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器的FPN标定数据，包括：
    基于所述像敏单元阵列输出的原始图像数据确定每个像敏单元的FPN数据；
    将所述每个像敏单元的FPN数据中大于阈值的FPN数据作为所述图像传感器的FPN标定数据。
  9. 如权利要求1至8任一项所述的图像噪声标定方法，其特征在于，所述方法还包括：
    获取像敏单元阵列的暗电流矫正值；
    所述基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器的FPN标定数据，包括：
    基于所述像敏单元阵列输出的原始图像数据和所述像敏单元阵列的暗电流矫正值确定所述图像传感器的FPN标定数据，所述图像传感器的FPN标定数据为去除了暗电流矫正值的数据值。
  10. 如权利要求1至9任一项所述的图像噪声标定方法，其特征在于，所述原始图像数据的数量为至少一帧，
    所述基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器的FPN标定数据，包括：
    基于所述像敏单元阵列输出的每帧原始图像数据确定一帧FPN标定数据；将每帧FPN标定数据进行平均得到所述图像传感器的FPN标定数据；
    或者，
    基于所述像敏单元阵列输出的至少一帧原始图像数据的均值确定所述图像传感器的FPN标定数据。
  11. 如权利要求1所述的图像噪声标定方法，其特征在于，所述方法还包括:
    将所述图像传感器的FPN标定数据烧录至降噪模块的存储单元中。
  12. 如权利要求11所述的图像噪声标定方法，其特征在于，所述降噪模块为所述图像传感器，或者为图像处理器，或者为与所述图像传感器连接的降噪电路。
  13. 如权利要求11所述的图像噪声标定方法，其特征在于，所述图像降噪方法还包括：
    根据所述FPN标定数据生成校验数据；
    将所述FPN标定数据和所述校验数据烧录至所述降噪模块的存储单元中。
  14. 如权利要求13所述的图像噪声标定方法，其特征在于，所述校验数据采用循环冗余校验算法生成。
  15. 如权利要求13或14所述的图像噪声标定方法，其特征在于，所述图像降噪方法还包括：
    基于所述校验数据对所述FPN标定数据进行校验。
  16. 如权利要求1所述的图像噪声标定方法，其特征在于，在所述图像传感器获取图像之前，所述方法还包括：
    确定所述像敏单元阵列中没有存储有效的FPN标定数据。
  17. 如权利要求16所述的图像噪声标定方法，其特征在于，其中“确定所述像敏单元阵列中没有存储有效的FPN标定数据所述方法”包括：
    读取所述存储单元中的数据，若不存在FPN标定数据或所述FPN标定数据校验失败，确定所述像敏单元阵列中没有存储有效的FPN标定数据。
  18. 一种图像降噪方法，其特征在于，所述图像降噪方法包括：
    获取图像传感器输出的原始图像数据；
    根据预先存储的所述图像传感器的固定模式噪声FPN标定数据计算所述原始图像数据的补偿数据；
    根据所述补偿数据对所述原始图像数据进行补偿。
  19. 如权利要求18所述的图像降噪方法，其特征在于，所述根据预先存储的所述图像传感器的FPN标定数据和所述原始图像数据计算所述原始图像数据的补偿数据，包括：
    获取补偿级别；
    基于所述FPN标定数据和所述补偿级别计算所述原始图像数据的补偿数据。
  20. 如权利要求18所述的图像降噪方法，其特征在于，所述图像降噪方法是在确定所述原始图像数据对应的曝光信息满足预定条件时，和/或，确定所述图像传感器的FPN标定数据校验成功时执行的。
  21. 如权利要求20所述的图像降噪方法，其特征在于，所述曝光信息包括曝光增益。
  22. 如权利要求21所述的图像降噪方法，其特征在于，其中所述补偿级别与所述曝光增益正相关，所述曝光增益越大所述补偿级别越大。
  23. 如权利要求21所述的图像降噪方法，其特征在于，当所述原始图像数据对应的曝光增益不小于所述FPN标定数据对应的曝光增益时，确定所述原始图像数据对应的曝光信息满足预定条件。
  24. 如权利要求21所述的图像降噪方法，其特征在于，其中所述曝光增益是基于模拟增益和数字增益的乘积确定的。
  25. 如权利要求18所述的图像降噪方法，其特征在于，所述图像传感器的FPN标定数据预先存储在降噪模块的存储单元中，所述图像降噪方法应用于所述降噪模块中。
  26. 如权利要求25所述的图像降噪方法，其特征在于，所述降噪模块为所述图像传感器，或者为图像处理器，或者为与所述图像传感器连接的降噪电路。
  27. 如权利要求26所述的图像降噪方法，其特征在于，所述降噪模块为降噪电路，所述降噪电路与所述图像处理器相连；
    当所述图像处理器确定所述原始图像数据对应的曝光信息满足预定条件时，和/或，确定所述像敏单元阵列的FPN标定数据校验成功时，所述降噪电路基于所述图像处理器的使能执行所述图像降噪方法。
  28. 如权利要求27所述的图像降噪方法，其特征在于，所述降噪电路还预先存储有所述图像传感器的校验数据；
    所述方法还包括：
    所述降噪电路将所述图像传感器的FPN标定数据和校验数据发送至所述图像处理器，用于所述图像处理器对所述图像传感器的FPN标定数据进行校验。
  29. 如权利要求18所述的图像降噪方法，其特征在于，所述图像降噪方法应用于所述降噪电路中；所述方法还包括：
    所述降噪电路接收来自所述图像处理器发送的补偿级别；
    所述降噪电路基于所述图像处理器标定数据和所述补偿级别计算所述原始图像数据的补偿数据。
  30. 如权利要求29所述的图像降噪方法，其特征在于，
    所述补偿级别是基于所述图像传感器的FPN标定数据对应的曝光信息和所述原始图像数据对应的曝光信息确定的，和/或，基于所述图像传感器的FPN标定数据对应的温度信息和所述原始图像数据对应的温度信息确定的。
  31. 如权利要求30所述的图像降噪方法，其特征在于，所述补偿数据是基于所述FPN标定数据与所述补偿级别的乘积计算的。
  32. 如权利要求18所述的图像降噪方法，其特征在于，所述图像传感器包括像敏单元阵列，所述FPN标定数据的数量小于所述像敏单元阵列中像敏单元的数量。
  33. 如权利要求32所述的图像降噪方法，其特征在于，
    所述根据预先存储的所述图像传感器的FPN标定数据计算所述原始图像数据的补偿数据，包括：
    对于所述像敏单元阵列中的其中一列像敏单元，基于所述一列像敏单元的FPN标定数据计算所述一列像敏单元输出的原始图像数据的补偿数据，其中，所述一列像敏单元的FPN标定数据的数量小于所述一列像敏单元中像敏单元的数量。
  34. 如权利要求17或33所述的图像降噪方法，其特征在于，所述原始图像数据为Bayer域的图像数据。
  35. 如权利要求17或33所述的图像噪声标定方法，其特征在于，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据包括M类通道的FPN标定数据，每一类通道对应的FPN标定数据的数量小于所述一类通道对应的像敏单元的数量。
  36. 如权利要求35所述的图像噪声标定方法，其特征在于，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数 据的数量为M；
    所述根据预先存储的所述图像传感器的固定模式噪声FPN标定数据计算所述原始图像数据的补偿数据，包括：
    根据M个FPN标定数据分别计算所述一列像敏单元中M类通道的图像数据的补偿数据，其中，每一个FPN标定数据用于计算一类通道中所有图像数据的补偿数据。
  37. 如权利要求32所述的图像降噪方法，其特征在于，
    所述图像传感器的FPN标定数据为所述像敏单元阵列中部分像敏单元的FPN标定数据，每一个FPN标定数据用于计算所述FPN标定数据对应的像敏单元输出的原始图像数据的补偿数据。
  38. 如权利要求18所述的图像降噪方法，其特征在于，所述根据所述补偿数据对所述原始图像数据进行补偿，包括：
    将所述原始图像数据加上或减去所述补偿数据。
  39. 一种图像噪声标定装置，其特征在于，所述图像噪声标定装置包括处理器，所述处理器执行计算机可读指令集实现：
    获取图像传感器输出的原始图像数据，所述图像传感器包括像敏单元阵列，所述原始图像数据是所述像敏单元阵列在光学黑状态下输出的；
    基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器的固定模式噪声FPN标定数据，所述FPN标定数据用于所述像敏单元阵列的降噪，且所述FPN标定数据的数量小于所述图像传感器中像敏单元的数量。
  40. 根据权利要求39所述的图像噪声标定装置，其特征在于，
    所述基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器的FPN标定数据，包括：
    对于所述像敏单元阵列中的其中一列像敏单元，基于所述一列像敏单元输出的原始图像数据确定所述一列像敏单元的FPN标定数据，所述一列像敏单元的FPN标定数据用于所述一列像敏单元的降噪，且所述一列像敏单 元的FPN标定数据的数量小于所述一列像敏单元的数量。
  41. 如权利要求40所述的图像噪声标定装置，其特征在于，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据包括M类通道的FPN标定数据，每一类通道对应的FPN标定数据的数量小于所述一类通道对应的像敏单元的数量。
  42. 如权利要求39或41所述的图像噪声标定装置，其特征在于，所述原始图像数据为Bayer域的图像数据。
  43. 如权利要求4所述的图像噪声标定装置，其特征在于，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据的数量为M。
  44. 如权利要求43所述的图像噪声标定装置，其特征在于，所述原始图像数据为RGB Bayer域的图像数据；
    所述M类通道包括一个蓝色通道、一个红色通道和两个绿色通道。
  45. 如权利要求40所述的图像噪声标定装置，其特征在于，所述一列像敏单元的FPN标定数据的数量为1。
  46. 根据权利要求39所述的图像噪声标定装置，其特征在于，所述基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器的FPN标定数据，包括：
    基于所述像敏单元阵列输出的原始图像数据确定每个像敏单元的FPN数据；
    将所述每个像敏单元的FPN数据中大于阈值的FPN数据作为所述图像传感器的FPN标定数据。
  47. 如权利要求39至46任一项所述的图像噪声标定装置，其特征在于， 所述处理器执行计算机可读指令集还实现：
    获取像敏单元阵列的暗电流矫正值；
    所述基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器的FPN标定数据，包括：
    基于所述像敏单元阵列输出的原始图像数据和所述像敏单元阵列的暗电流矫正值确定所述图像传感器的FPN标定数据，所述图像传感器的FPN标定数据为去除了暗电流矫正值的数据值。
  48. 如权利要求39至47任一项所述的图像噪声标定装置，其特征在于，所述原始图像数据的数量为至少一帧，
    所述基于所述像敏单元阵列输出的原始图像数据确定所述图像传感器的FPN标定数据，包括：
    基于所述像敏单元阵列输出的每帧原始图像数据确定一帧FPN标定数据；将每帧FPN标定数据进行平均得到所述图像传感器的FPN标定数据；
    或者，
    基于所述像敏单元阵列输出的至少一帧原始图像数据的均值确定所述图像传感器的FPN标定数据。
  49. 如权利要求39所述的图像噪声标定装置，其特征在于，所述处理器执行计算机可读指令集还实现：:
    将所述图像传感器的FPN标定数据烧录至降噪模块的存储单元中。
  50. 如权利要求49所述的图像噪声标定装置，其特征在于，所述降噪模块为所述图像传感器，或者为图像处理器，或者为与所述降噪电路连接的降噪电路。
  51. 如权利要求49所述的图像噪声标定装置，其特征在于，所述处理器执行计算机可读指令集还实现：：
    根据所述FPN标定数据生成校验数据；
    将所述FPN标定数据和所述校验数据烧录至所述降噪模块的存储单元 中。
  52. 如权利要求51所述的图像噪声标定装置，其特征在于，所述校验数据采用循环冗余校验算法生成。
  53. 如权利要求51或52所述的图像噪声标定装置，其特征在于，所述处理器执行计算机可读指令集还实现：：
    基于所述校验数据对所述FPN标定数据进行校验。
  54. 如权利要求39所述的图像噪声标定装置，其特征在于，在所述图像传感器获取图像之前，处理器执行计算机可读指令集还实现：：
    确定所述像敏单元阵列中没有存储有效的FPN标定数据。
  55. 如权利要求54所述的图像噪声标定装置，其特征在于，其中“确定所述像敏单元阵列中没有存储有效的FPN标定数据所述方法”包括：
    读取所述存储单元中的数据，若不存在FPN标定数据或所述FPN标定数据校验失败，确定所述像敏单元阵列中没有存储有效的FPN标定数据。
  56. 一种图像降噪装置，其特征在于，所述图像降噪装置包括降噪模块，所述降噪模块用于对图像传感器输出的原始图像数据进行降噪处理，所述降噪处理包括：
    获取图像传感器输出的原始图像数据；
    根据预先存储的所述图像传感器的固定模式噪声FPN标定数据计算所述原始图像数据的补偿数据；
    根据所述补偿数据对所述原始图像数据进行补偿。
  57. 如权利要求56所述的图像降噪装置，其特征在于，所述根据预先存储的所述图像传感器的FPN标定数据和所述原始图像数据计算所述原始图像数据的补偿数据，包括：
    获取补偿级别；
    基于所述FPN标定数据和所述补偿级别计算所述原始图像数据的补偿数据。
  58. 如权利要求56所述的图像降噪装置，其特征在于，所述降噪处理是在确定所述原始图像数据对应的曝光信息满足预定条件时，和/或，确定所述图像传感器的FPN标定数据校验成功时执行的。
  59. 如权利要求58所述的图像降噪装置，其特征在于，所述曝光信息包括曝光增益。
  60. 如权利要求59所述的图像降噪装置，其特征在于，其中所述补偿级别与所述曝光增益正相关，所述曝光增益越大所述补偿级别越大。
  61. 如权利要求59所述的图像降噪装置，其特征在于，当所述原始图像数据对应的曝光增益不小于所述FPN标定数据对应的曝光增益时，确定所述原始图像数据对应的曝光信息满足预定条件。
  62. 如权利要求59所述的图像降噪装置，其特征在于，其中所述曝光增益是基于模拟增益和数字增益的乘积确定的。
  63. 如权利要求56所述的图像降噪装置，其特征在于，所述图像传感器的FPN标定数据预先存储在降噪模块的存储单元中。
  64. 如权利要求63所述的图像降噪装置，其特征在于，所述降噪模块为所述图像传感器，或者为图像处理器，或者为与所述图像传感器连接的降噪电路。
  65. 如权利要求64所述的图像降噪装置，其特征在于，所述降噪模块为降噪电路，所述降噪电路与所述图像处理器相连；
    当所述图像处理器确定所述原始图像数据对应的曝光信息满足预定条 件时，和/或，确定所述像敏单元阵列的FPN标定数据校验成功时，所述降噪电路基于所述图像处理器的使能执行所述降噪处理。
  66. 如权利要求65所述的图像降噪装置，其特征在于，所述降噪电路还预先存储有所述图像传感器的校验数据；
    所述降噪处理还包括：
    所述降噪电路将所述图像传感器的FPN标定数据和校验数据发送至所述图像处理器，用于所述图像处理器对所述图像传感器的FPN标定数据进行校验。
  67. 如权利要求56所述的图像降噪装置，其特征在于，所述降噪模块为降噪电路中；所述降噪处理还包括：
    所述降噪电路接收来自所述图像处理器发送的补偿级别；
    所述降噪电路基于所述图像处理器标定数据和所述补偿级别计算所述原始图像数据的补偿数据。
  68. 如权利要求67所述的图像降噪装置，其特征在于，
    所述补偿级别是基于所述图像传感器的FPN标定数据对应的曝光信息和所述原始图像数据对应的曝光信息确定的，和/或，基于所述图像传感器的FPN标定数据对应的温度信息和所述原始图像数据对应的温度信息确定的。
  69. 如权利要求67所述的图像降噪装置，其特征在于，所述补偿数据是基于所述FPN标定数据与所述补偿级别的乘积计算的。
  70. 如权利要求56所述的图像降噪装置，其特征在于，所述图像传感器包括像敏单元阵列，所述FPN标定数据的数量小于所述像敏单元阵列中像敏单元的数量。
  71. 如权利要求70所述的图像降噪装置，其特征在于，
    所述根据预先存储的所述图像传感器的FPN标定数据计算所述原始图 像数据的补偿数据，包括：
    对于所述像敏单元阵列中的其中一列像敏单元，基于所述一列像敏单元的FPN标定数据计算所述一列像敏单元输出的原始图像数据的补偿数据，其中，所述一列像敏单元的FPN标定数据的数量小于所述一列像敏单元中像敏单元的数量。
  72. 如权利要求56或71所述的图像降噪装置，其特征在于，所述原始图像数据为Bayer域的图像数据。
  73. 如权利要求56或71所述的图像降噪装置，其特征在于，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据包括M类通道的FPN标定数据，每一类通道对应的FPN标定数据的数量小于所述一类通道对应的像敏单元的数量。
  74. 如权利要求73所述的图像降噪装置，其特征在于，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据的数量为M；
    所述根据预先存储的所述图像传感器的固定模式噪声FPN标定数据计算所述原始图像数据的补偿数据，包括：
    根据M个FPN标定数据分别计算所述一列像敏单元中M类通道的图像数据的补偿数据，其中，每一个FPN标定数据用于计算一类通道中所有图像数据的补偿数据。
  75. 如权利要求71所述的图像降噪装置，其特征在于，
    所述图像传感器的FPN标定数据为所述像敏单元阵列中部分像敏单元的FPN标定数据，每一个FPN标定数据用于计算所述FPN标定数据对应的像敏单元输出的原始图像数据的补偿数据。
  76. 如权利要求56所述的图像降噪装置，其特征在于，所述根据所述补偿数据对所述原始图像数据进行补偿，包括：
    将所述原始图像数据加上或减去所述补偿数据。
  77. 一种图像处理装置，其特征在于，所述图像处理装置包括：
    图像传感器，用于输出原始图像数据；
    图像处理器，所述图像处理器与所述图像传感器通信连接，用于处理图像数据；
    降噪电路，所述降噪电路预存储有所述图像传感器的固定模式噪声FPN标定数据，所述降噪电路分别与所述图像传感器、所述图像传感器通信连接；所述降噪电路用于对所述图像传感器输出的原始图像数据执行降噪处理，所述降噪处理包括：获取所述图像传感器输出的原始图像数据；根据所述FPN标定数据计算所述原始图像数据的补偿数据，及根据所述补偿数据对所述原始图像数据进行补偿。
  78. 如权利要求77所述的图像处理装置，其特征在于，所述图像处理器用于获取所述原始图像数据对应的曝光信息，当确定所述原始图像数据对应的曝光信息满足预定条件时，使能所述降噪电路执行所述降噪处理；
    和/或，
    所述图像处理器用于从所述降噪电路获取所述图像传感器的FPN标定数据，当确定所述图像传感器的FPN标定数据校验成功时使能所述降噪电路执行所述降噪处理。
  79. 如权利要求78所述的图像处理装置，其特征在于，所述曝光信息包括曝光增益。
  80. 如权利要求79所述的图像处理装置，其特征在于，其中所述补偿级别与所述曝光增益正相关，所述曝光增益越大所述补偿级别越大。
  81. 如权利要求79所述的图像处理装置，其特征在于，当所述原始图像数据对应的曝光增益不小于所述FPN标定数据对应的曝光增益时，确定所述原始图像数据对应的曝光信息满足预定条件。
  82. 如权利要求79所述的图像处理装置，其特征在于，其中所述曝光增益是基于模拟增益和数字增益的乘积确定的。
  83. 如权利要求78所述的图像处理装置，其特征在于，所述降噪电路还预存有所述图像传感器的FPN标定数据的校验数据。
  84. 如权利要求83所述的图像处理装置，其特征在于，所述图像处理器还用于从所述降噪电路获取所述图像传感器的FPN标定数据的校验数据，当基于所述FPN标定数据和所述校验数据确定所述图像传感器的FPN标定数据校验成功时使能所述降噪电路执行所述降噪处理。
  85. 如权利要求77所述的图像处理装置，其特征在于，所述降噪电路还用于接收来自所述图像处理器发送的补偿级别；并基于所述图像处理器标定数据和所述补偿级别计算所述原始图像数据的补偿数据。
  86. 如权利要求85所述的图像处理装置，其特征在于，
    所述补偿级别是基于所述图像传感器的FPN标定数据对应的曝光信息和所述原始图像数据对应的曝光信息确定的，和/或，基于所述图像传感器的FPN标定数据对应的温度信息和所述原始图像数据对应的温度信息确定的。
  87. 如权利要求86所述的图像处理装置，其特征在于，所述补偿数据是基于所述FPN标定数据与所述补偿级别的乘积计算的。
  88. 如权利要求77所述的图像处理装置，其特征在于，所述图像传感器包括像敏单元阵列，所述FPN标定数据的数量小于所述像敏单元阵列中像敏单元的数量。
  89. 如权利要求88所述的图像处理装置，其特征在于，
    所述根据预先存储的所述图像传感器的FPN标定数据计算所述原始图 像数据的补偿数据，包括：
    对于所述像敏单元阵列中的其中一列像敏单元，基于所述一列像敏单元的FPN标定数据计算所述一列像敏单元输出的原始图像数据的补偿数据，其中，所述一列像敏单元的FPN标定数据的数量小于所述一列像敏单元中像敏单元的数量。
  90. 如权利要求77或89所述的图像处理装置，其特征在于，所述原始图像数据为Bayer域的图像数据。
  91. 如权利要求77或89所述的图像处理装置，其特征在于，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据包括M类通道的FPN标定数据，每一类通道对应的FPN标定数据的数量小于所述一类通道对应的像敏单元的数量。
  92. 如权利要求91所述的图像处理装置，其特征在于，所述一列像敏单元用于输出M类通道的图像数据，所述一列像敏单元的FPN标定数据的数量为M；
    所述根据预先存储的所述图像传感器的固定模式噪声FPN标定数据计算所述原始图像数据的补偿数据，包括：
    根据M个FPN标定数据分别计算所述一列像敏单元中M类通道的图像数据的补偿数据，其中，每一个FPN标定数据用于计算一类通道中所有图像数据的补偿数据。
  93. 如权利要求88所述的图像处理装置，其特征在于，
    所述图像传感器的FPN标定数据为所述像敏单元阵列中部分像敏单元的FPN标定数据，每一个FPN标定数据用于计算所述FPN标定数据对应的像敏单元输出的原始图像数据的补偿数据。
  94. 如权利要求77所述的图像处理装置，其特征在于，所述根据所述补偿数据对所述原始图像数据进行补偿，包括：
    将所述原始图像数据加上或减去所述补偿数据。
PCT/CN2017/112440 2017-11-22 2017-11-22 图像噪声标定方法及装置、图像降噪方法及装置、图像处理装置 WO2019100270A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
PCT/CN2017/112440 WO2019100270A1 (zh) 2017-11-22 2017-11-22 图像噪声标定方法及装置、图像降噪方法及装置、图像处理装置
CN201780087782.9A CN110574363A (zh) 2017-11-22 2017-11-22 图像噪声标定方法及装置、图像降噪方法及装置、图像处理装置
US16/725,711 US20200137334A1 (en) 2017-11-22 2019-12-23 Image noise calibration method and apparatus, image noise reduction method and apparatus, and image processing apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/112440 WO2019100270A1 (zh) 2017-11-22 2017-11-22 图像噪声标定方法及装置、图像降噪方法及装置、图像处理装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/725,711 Continuation US20200137334A1 (en) 2017-11-22 2019-12-23 Image noise calibration method and apparatus, image noise reduction method and apparatus, and image processing apparatus

Publications (1)

Publication Number Publication Date
WO2019100270A1 true WO2019100270A1 (zh) 2019-05-31

Family

ID=66631347

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/112440 WO2019100270A1 (zh) 2017-11-22 2017-11-22 图像噪声标定方法及装置、图像降噪方法及装置、图像处理装置

Country Status (3)

Country Link
US (1) US20200137334A1 (zh)
CN (1) CN110574363A (zh)
WO (1) WO2019100270A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102637732B1 (ko) * 2018-09-21 2024-02-19 삼성전자주식회사 이미지 신호 프로세서, 상기 이미지 신호 프로세서의 동작 방법 및 상기 이미지 신호 프로세서를 포함하는 애플리케이션 프로세서
CN111307182B (zh) * 2020-03-06 2022-08-23 宁波飞芯电子科技有限公司 数据处理方法及阵列型传感器
KR20220095595A (ko) * 2020-12-30 2022-07-07 에스케이하이닉스 주식회사 이미지 센서, 이를 이용한 이미지 센서 테스트 시스템 및 방법
CN113822812B (zh) * 2021-09-15 2024-09-06 维沃移动通信有限公司 图像降噪方法和电子设备
CN115665357B (zh) * 2022-10-24 2023-08-08 昆易电子科技(上海)有限公司 图像数据传输方法、系统、注入方法及电子设备
WO2024199929A1 (en) * 2023-03-27 2024-10-03 Sony Semiconductor Solutions Corporation Sensor device and method for operating a sensor device

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040113052A1 (en) * 2002-12-11 2004-06-17 Dialog Semiconductor Gmbh. Fixed pattern noise compensation with low memory requirements
CN101448078A (zh) * 2008-12-15 2009-06-03 昆山锐芯微电子有限公司 图像传感器和图像信号处理方法
CN101764959A (zh) * 2008-12-25 2010-06-30 昆山锐芯微电子有限公司 图像拾取系统及图像处理方法
CN102131059A (zh) * 2011-04-20 2011-07-20 中国科学院半导体研究所 面向实时视觉芯片的高速行并行图像传感器
CN102663714A (zh) * 2012-03-28 2012-09-12 中国人民解放军国防科学技术大学 基于显著性的红外图像强固定模式噪声抑制方法
CN103338337A (zh) * 2013-05-30 2013-10-02 益海芯电子技术江苏有限公司 光学指示装置中的图像噪声消除方法

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7317481B2 (en) * 2002-02-20 2008-01-08 Canon Kabushiki Kaisha Image data correction processing based on sensitivity
US7362911B1 (en) * 2003-11-13 2008-04-22 Pixim, Inc. Removal of stationary noise pattern from digital images
EP2001220B1 (en) * 2006-03-24 2015-01-14 Nikon Corporation Signal processing method, signal processing system, coefficient generating device, and digital camera
US8466991B2 (en) * 2010-07-19 2013-06-18 Aptina Imaging Corporation Optical black pixel cell readout systems and methods
JP6053447B2 (ja) * 2012-10-23 2016-12-27 オリンパス株式会社 撮像装置
TWI528816B (zh) * 2013-04-18 2016-04-01 聯詠科技股份有限公司 固定樣式雜訊移除方法
KR20160112431A (ko) * 2015-03-19 2016-09-28 에스케이하이닉스 주식회사 이미지 센싱 장치 및 그의 구동 방법
KR102446723B1 (ko) * 2016-01-29 2022-09-27 에스케이하이닉스 주식회사 이미지 센싱 장치 및 그의 구동 방법

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040113052A1 (en) * 2002-12-11 2004-06-17 Dialog Semiconductor Gmbh. Fixed pattern noise compensation with low memory requirements
CN101448078A (zh) * 2008-12-15 2009-06-03 昆山锐芯微电子有限公司 图像传感器和图像信号处理方法
CN101764959A (zh) * 2008-12-25 2010-06-30 昆山锐芯微电子有限公司 图像拾取系统及图像处理方法
CN102131059A (zh) * 2011-04-20 2011-07-20 中国科学院半导体研究所 面向实时视觉芯片的高速行并行图像传感器
CN102663714A (zh) * 2012-03-28 2012-09-12 中国人民解放军国防科学技术大学 基于显著性的红外图像强固定模式噪声抑制方法
CN103338337A (zh) * 2013-05-30 2013-10-02 益海芯电子技术江苏有限公司 光学指示装置中的图像噪声消除方法

Also Published As

Publication number Publication date
CN110574363A (zh) 2019-12-13
US20200137334A1 (en) 2020-04-30

Similar Documents

Publication Publication Date Title
WO2019100270A1 (zh) 图像噪声标定方法及装置、图像降噪方法及装置、图像处理装置
US9819874B2 (en) Camera color temperature compensation system and smart terminal employing same
US9578211B2 (en) Image de-noising methods and apparatuses using the same
TWI545554B (zh) 視訊牆之自動安裝方法及其系統
US10110806B2 (en) Electronic device and method for operating the same
JP6525543B2 (ja) 画像処装置および画像処理方法、並びにプログラム
KR102072731B1 (ko) 촬영 장치, 그 제어 방법, 및 컴퓨터 판독가능 저장매체
US20150365612A1 (en) Image capture apparatus and image compensating method thereof
JP7357620B2 (ja) 画像処理装置、および画像処理システム
US20140306943A1 (en) Electronic device and method for adjusting backlight of electronic device
US20240223905A1 (en) Control device, control method and non-transitory computer readable storage medium
EP2070312A1 (en) A hand jitter reduction system for cameras
JP2015130630A (ja) 撮像装置及びその制御方法、プログラム、記憶媒体
US20140300620A1 (en) Adjustment device and method for adjusting chrominance and luminance of display device
US20170026584A1 (en) Image processing apparatus and method of operating the same
US10764632B2 (en) Video signal processing apparatus, video signal processing method, and program
US20230164451A1 (en) Information processing apparatus, method, medium, and system for color correction
TW201123886A (en) Look up table update method.
US11562712B2 (en) Video reproduction system, video reproduction device, and calibration method for video reproduction system
CN109547762B (zh) 一种白平衡自动切换的方法及装置
US9043514B2 (en) Data transfer device and method
CN102645816B (zh) 曝光参数补正方法及成像装置
KR20210151613A (ko) 이미지 처리 장치 및 이미지 처리 방법
US9185310B2 (en) Solid-state imaging device, illuminance measuring method performed by solid-state imaging device, and camera module
US12047689B2 (en) White balance coefficient correction method and electronic device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17932630

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17932630

Country of ref document: EP

Kind code of ref document: A1