CN105869125A - Infrared image enhancement algorithm based on optical readout infrared chip - Google Patents
Infrared image enhancement algorithm based on optical readout infrared chip Download PDFInfo
- Publication number
- CN105869125A CN105869125A CN201610180188.1A CN201610180188A CN105869125A CN 105869125 A CN105869125 A CN 105869125A CN 201610180188 A CN201610180188 A CN 201610180188A CN 105869125 A CN105869125 A CN 105869125A
- Authority
- CN
- China
- Prior art keywords
- infrared
- image
- thermal
- matrix
- thermal response
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003287 optical effect Effects 0.000 title claims abstract description 18
- 238000004422 calculation algorithm Methods 0.000 title claims abstract description 17
- 239000011159 matrix material Substances 0.000 claims abstract description 47
- 230000004044 response Effects 0.000 claims abstract description 29
- 230000009466 transformation Effects 0.000 claims abstract description 21
- 238000004364 calculation method Methods 0.000 claims abstract description 13
- 230000009467 reduction Effects 0.000 claims abstract description 12
- 238000000034 method Methods 0.000 claims abstract description 11
- 238000003384 imaging method Methods 0.000 claims abstract description 7
- 230000008859 change Effects 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 4
- 238000005516 engineering process Methods 0.000 description 11
- 238000009792 diffusion process Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 7
- 238000001514 detection method Methods 0.000 description 4
- 238000003331 infrared imaging Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000005855 radiation Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000001816 cooling Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000005057 refrigeration Methods 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 230000001131 transforming effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 150000007529 inorganic bases Chemical class 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000011895 specific detection Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
- Photometry And Measurement Of Optical Pulse Characteristics (AREA)
Abstract
The invention discloses an infrared image enhancement algorithm based on an optical readout infrared chip. The algorithm includes: (1) thermal response matrix calibration: imaging a standard blackbody under a set specification, acquiring N different infrared original images; conducting noise reduction on the infrared original image through the noise reduction method to obtain a noise reduction image; based on the acquired N infrared noise reduction images, calculating a thermal response matrix of each sensible heat unit in a substrate-free FPA; (2) inverse transformation matrix calculation: based on the matrix calculation, calculating an inverse transformation matrix of the thermal response matrix of each sensible heat unit; (3) image enhancement real-time calculation. According to the invention, the algorithm effectively enhances clarity of the optical readout infrared image based on the substrate-free FPA, has high calculating efficiency and can realize real-time implementation.
Description
Technical Field
The invention relates to the technical field of infrared image enhancement algorithms, in particular to an infrared image enhancement algorithm based on an optical readout infrared chip.
Background
Infrared imaging technology has been widely used in such fields as military, aerospace, transportation, medical treatment, security, and scientific research. According to the imaging principle, there are roughly classified into an optical quantum type (cooling type) and a thermal type (non-cooling type). Compared with the refrigeration technology, the non-refrigeration technology has the advantages of low cost, small volume, low energy consumption, convenient maintenance and the like, thereby being more widely concerned and applied. The uncooled technology can be further divided into electrical readout and optical readout according to the signal detection principle. The electrical readout technology is to integrate a high-gain and high-precision micro readout circuit in a thermal unit and detect the change of an electrical parameter caused by the thermal temperature rise of the thermal unit through the circuit. The optical readout technology is to use an optical path system to detect the change of other physical parameters (such as displacement, rotation angle, etc.) caused by the thermal temperature rise of the thermal sensing unit.
Related researchers have designed substrate-free Focal Plane Arrays (FPAs) of many different configurations for optical readout technologies. Thermally, the thermal cells of such a baseless FPA absorb infrared radiation and the heat will propagate through the support frame in some fashion to adjacent thermal cells. Therefore, the thermal responses of the respective thermal units are correlated with each other and affect each other. In addition, changes in physical parameters (such as displacement, rotation angle, etc.) caused by thermal response are acquired by the image sensor, which will couple various circuit and light source noises.
Researchers have proposed an infrared image restoration algorithm based on a point spread function, that is:
1) the thermal response of the thermal unit is assumed as a point spread function to obtain a restoration image
2) Processing the noise through wavelet transformation to obtain a noise reduction map
3) And fusing the restoration image and the noise reduction image by an image fusion technology to obtain a final result.
But the defects of the prior art scheme are as follows:
1) because different thermal units are different, particularly the structures cannot be completely symmetrical, the thermal units are not accurate point diffusion functions, and correct restoration images cannot be obtained only based on the point diffusion functions;
2) the wavelet transformation algorithm has large calculation amount and cannot realize real time;
3) the image fusion technology has large calculation amount and is difficult to realize real time.
Disclosure of Invention
Aiming at the defects of the prior art and the problems of the optical readout technology, namely the problems that the thermal responses of the thermal sensing units not only influence each other, but also couple various noises, the invention provides an infrared image enhancement algorithm based on an optical readout infrared chip.
In order to achieve the technical purpose, the invention adopts the following technical scheme: an infrared image enhancement algorithm based on a light readout infrared chip,
1) calibration of the thermal response matrix:
acquiring N different infrared original images by imaging the standard black body under a set specification condition;
denoising the infrared original image by a denoising method to obtain a denoising image;
calculating a thermal response matrix of each thermal sensing unit in the wireless base FPA according to the acquired N infrared noise reduction maps;
2) inverse transformation matrix calculation:
calculating an inverse transformation matrix of each thermal response matrix of each thermal sensing unit according to matrix operation;
3) image enhancement real-time calculation:
and calculating the infrared original image in real time according to the inverse transformation matrix to obtain an enhanced and noise-reduced infrared image.
Further, the noise reduction method is mean filtering or gaussian filtering.
Further, the thermal spread of the thermal unit is not a point spread function.
Further, the function of the thermal response matrix is that the original sharp image f (x, y) becomes a baseless FPA function g (x, y) by image transformation h (f); the inverse matrix process of the thermal response matrix is to change a base-free FPA function g (x, y) into an original clear image f (x, y) through image transformation H (f), and enhance the infrared image of the base-free FPA.
The invention has the technical characteristics and effects that: the definition of an optically read infrared image based on the substrate-free FPA is effectively enhanced; the algorithm has high calculation efficiency and can realize real time; acquiring N different infrared original images by imaging the standard black body under a set specification condition; denoising the infrared original image by a denoising method to obtain a denoising image; and calculating a thermal response matrix of each thermal unit in the inorganic base FPA according to the collected N infrared noise reduction maps.
Drawings
FIG. 1 is a diagram of an imaging process for a baseless FPA of the present invention.
FIG. 2 is a first thermal response diagram of each thermal unit of the present invention.
FIG. 3 is a second thermal response diagram of each thermal unit of the present invention.
FIG. 4 is a first pixel diagram of the present invention.
FIG. 5 is a second diagram of the pixel of the present invention.
FIG. 6 is a third diagram of the pixel of the present invention.
FIG. 7 is a first analysis diagram of the thermal diffusion simulation of the present invention.
FIG. 8 is a second analysis chart of the thermal diffusion simulation of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention provides that as shown in fig. 1, the invention comprises three sub-modules:
firstly, calibrating a thermal response matrix;
n different infrared original images are obtained by imaging the standard black body under the set specification condition.
And denoising the infrared original image by a denoising method (mean filtering, Gaussian filtering and the like) to obtain a denoising image.
Calculating a thermal response matrix of each thermal sensing unit in the wireless base FPA according to the acquired N infrared noise reduction maps; measured thermal response matrix: h (the measured matrix H of each chip is different), and the inverse matrix is obtained: h-1. Then for the infrared image g, the rectified image is: h-1(g)。
Wherein the noise equivalent temperature difference-NETD: i.e. the temperature difference of the heat source when the temperature difference of the infrared heat source and the background is equal to the system output signal on the detector. The smaller the NETD of the infrared detection system, the better its performance. The expression formula is as follows:
Inoisedefined as the noise gray scale of the system, since in practical optical readout systems both signal and noise are measured in CCD gray scale quantization levels. At the same timeDefined as the grey scale phase of the systemThe physical meaning is the gray scale change caused by the unit temperature change of the infrared target. Noise equivalent power-NEP: for a certain radiation power P, the radiation power when it generates a signal in the FPA exactly equal to the signal of the average noise; it is clear that the smaller the NEP, the better the detection performance:
normalized detectivity: (I.e. specific detectivity), in order to eliminate the area of the detectorNoise equivalent bandwidth of a sum circuit amplifierTo pairThe influence, namely:with this formula, specific detectivity can be obtainedIs expressed as:
thus the higher the specific detection rate, the higher the signal-to-noise ratio of the system performance, the better the detection.
Secondly, inverse transformation matrix calculation:
and calculating an inverse transformation matrix of the thermal response matrix of each thermal sensing unit according to the matrix operation.
And finally, image enhancement real-time calculation:
and calculating the infrared original image in real time according to the inverse transformation matrix to obtain an enhanced and noise-reduced infrared image.
From FIG. 4, it is evident that the pixel value is 200X 200. mu.m2
Fig. 5 clearly shows that the pixel value is pixel size: 60 x 60 microns2
FIG. 6 shows that the pixel value is 30X 30 μm2
Due to the diffusion of thermal response, the infrared image obtained based on the baseless FPA is equivalent to the thermal response matrix which is subjected to image transformation H (f) on the basis of the original image. Therefore, as long as the thermal response matrix is obtained, the infrared image of the substrate-free FPA can be enhanced and the definition can be improved through the inverse matrix.
Namely, on the basis of realizing one-to-one image reconstruction, a thermal image g (x, y) of the infrared target in the FPA is obtained. The infrared image obtained by the baseless FPA is adopted to perform thermal crosstalk restoration processing, that is, the inverse transformation of the infrared imaging process of the baseless FPA needs to be realized by a digital signal processing technology, which can be expressed as:
wherein,for transforming imagesAnd (4) inverse transforming.
For this purpose, an original clear thermal image of the infrared target is definedIs composed ofOf the two-dimensional data matrixAnd converting the equivalent thereof into a one-dimensional column vectorDefining an infrared image g (x, y) obtained after thermal crosstalk on a baseless FPA asOf the two-dimensional data matrixIts equivalent is also transformed into a one-dimensional column vector. Since the thermal response of the baseless FPA is equivalent to the linear superposition of the point spread functions of the individual micro-beam elements, the thermal crosstalk is transformedIs closely related to the point spread function.
Accordingly, a two-dimensional diffusion matrix with m × n point diffusion matrix is defined, wherein the point diffusion matrix is diffused to the whole FPA unit at any point (x, y) (x is more than 0 and less than or equal to m, y is more than 0 and less than or equal to n) on the original clear thermal image f (x, y) of the infrared targetIts equivalent is transformed into a one-dimensional column vector
Thermal crosstalk transformation of a baseless FPA of size m × n according to the principle of linear superposition of point spread functions satisfied in baseless FPA infrared imagingCan be expressed as:
the above formula yields: thermal crosstalk conversion for baseless FPAI.e. an image transformation matrixThen the formula is expressed as follows:
and
wherein,is thatThe inverse matrix of (c).
As shown in fig. 2, in addition to the thermal response of the thermal unit itself, it is further diffused to the surrounding thermal units. However:
A. since the structure of the thermal unit is not completely symmetrical, the diffusion is not a standard point spread function;
B. the periphery of the substrate-free FPA is affected by the frame, and the thermal diffusion phenomenon of the thermal unit close to the frame is completely different from that of the thermal unit in the central area;
in summary, the thermal spread of a thermal cell is not a point spread function.
The invention effectively enhances the definition of the optical read-out infrared image based on the baseless FPA, has high algorithm calculation efficiency and can realize real time.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (4)
1. The infrared image enhancement algorithm based on the optical readout infrared chip is characterized in that:
1) calibration of the thermal response matrix:
acquiring N different infrared original images by imaging the standard black body under a set specification condition;
denoising the infrared original image by a denoising method to obtain a denoising image;
calculating a thermal response matrix of each thermal sensing unit in the wireless base FPA according to the acquired N infrared noise reduction maps;
2) inverse transformation matrix calculation:
calculating an inverse transformation matrix of each thermal response matrix of each thermal sensing unit according to matrix operation;
3) image enhancement real-time calculation:
and calculating the infrared original image in real time according to the inverse transformation matrix to obtain an enhanced and noise-reduced infrared image.
2. The infrared image enhancement algorithm based on the optical readout infrared chip of claim 1, characterized in that: the noise reduction method is mean filtering or Gaussian filtering.
3. The infrared image enhancement algorithm based on the optical readout infrared chip of claim 1, characterized in that: the thermal spread of the thermal unit is not a point spread function.
4. The infrared image enhancement algorithm based on the optical readout infrared chip of claim 1, characterized in that: the function of the thermal response matrix is that an original clear image f (x, y) is changed into a base-free FPA function g (x, y) through image transformation H (f); the inverse matrix process of the thermal response matrix is to change a base-free FPA function g (x, y) into an original clear image f (x, y) through image transformation H (f), and enhance the infrared image of the base-free FPA.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610180188.1A CN105869125A (en) | 2016-03-28 | 2016-03-28 | Infrared image enhancement algorithm based on optical readout infrared chip |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610180188.1A CN105869125A (en) | 2016-03-28 | 2016-03-28 | Infrared image enhancement algorithm based on optical readout infrared chip |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105869125A true CN105869125A (en) | 2016-08-17 |
Family
ID=56624813
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610180188.1A Pending CN105869125A (en) | 2016-03-28 | 2016-03-28 | Infrared image enhancement algorithm based on optical readout infrared chip |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105869125A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107862670A (en) * | 2017-11-30 | 2018-03-30 | 电子科技大学 | A kind of image recovery method for infrared imaging electrical crosstalk |
| CN110073185A (en) * | 2016-10-21 | 2019-07-30 | 瑞柏丽恩光子股份有限公司 | Mobile gas and chemicals image camera |
| CN111445411A (en) * | 2020-03-26 | 2020-07-24 | 深圳数联天下智能科技有限公司 | Image denoising method and device, computer equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101059459A (en) * | 2007-06-05 | 2007-10-24 | 北京理工大学 | Microscopic thermal imaging method and device thereof |
| CN102519603A (en) * | 2012-01-10 | 2012-06-27 | 北京理工大学 | Non-refrigeration infrared focal plane imaging system capable of removing background in real time |
-
2016
- 2016-03-28 CN CN201610180188.1A patent/CN105869125A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101059459A (en) * | 2007-06-05 | 2007-10-24 | 北京理工大学 | Microscopic thermal imaging method and device thereof |
| CN102519603A (en) * | 2012-01-10 | 2012-06-27 | 北京理工大学 | Non-refrigeration infrared focal plane imaging system capable of removing background in real time |
Non-Patent Citations (9)
| Title |
|---|
| JIANXIONG WU 等: "Research of infrared image optimization algorithm in optical read-out IR imaging", 《INTERNATIONAL SYMPOSIUM ON ADVANCED OPTICAL MANUFACTURING AND TESTING TECHNOLOGIES》 * |
| TENG CHENG 等: "Performance of an optimized substrate-free focal plane array for optical readout uncooled infrared detector", 《JOURNAL OF APPLIED PHYSICS》 * |
| TENG CHENG 等: "Uncooled Infrared Imaging Using a Substrate-Free Focal-Plane Array", 《IEEE ELECTRON DEVICE LETTERS》 * |
| 吴健雄: "基于无基底焦平面阵列的红外图像复原算法", 《实验力学》 * |
| 安毓英 等: "《光电子技术》", 30 April 2011, 电子工业出版社 * |
| 程腾: "无基底焦平面阵列的红外成像性能分析", 《物理学报》 * |
| 程腾: "无基底焦平面阵列的红外成像性能分析", 《红外与激光工程》 * |
| 蒋兴凯: "基于无基底焦平面阵列红外热像仪的理论模型分析", 《物理学报》 * |
| 陈扬 等: "《MATLAB 6.X 图形编程与图像处理》", 31 October 2002, 西安电子科技大学出版社 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110073185A (en) * | 2016-10-21 | 2019-07-30 | 瑞柏丽恩光子股份有限公司 | Mobile gas and chemicals image camera |
| US11044423B2 (en) | 2016-10-21 | 2021-06-22 | Rebellion Photonics, Inc. | Mobile gas and chemical imaging camera |
| CN110073185B (en) * | 2016-10-21 | 2022-02-18 | 瑞柏丽恩光子股份有限公司 | Mobile gas and chemical imaging camera |
| CN107862670A (en) * | 2017-11-30 | 2018-03-30 | 电子科技大学 | A kind of image recovery method for infrared imaging electrical crosstalk |
| CN107862670B (en) * | 2017-11-30 | 2021-04-30 | 电子科技大学 | Image restoration method for infrared imaging electric crosstalk |
| CN111445411A (en) * | 2020-03-26 | 2020-07-24 | 深圳数联天下智能科技有限公司 | Image denoising method and device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TW201601536A (en) | Performance parameter measurement method for infrared focal plane array module and defective pixel detection method | |
| CN104168429A (en) | Multi-aperture multi-band high-resolution-ratio imaging device and method | |
| CN105869125A (en) | Infrared image enhancement algorithm based on optical readout infrared chip | |
| Valenzuela et al. | Face recognition on a smart image sensor using local gradients | |
| Tisse et al. | An information-theoretic perspective on the challenges and advances in the race toward 12µm pixel pitch megapixel uncooled infrared imaging | |
| Lee et al. | Motion blur kernel rendering using an inertial sensor: Interpreting the mechanism of a thermal detector | |
| Bowers et al. | Unpolarized calibration and nonuniformity correction for long-wave infrared microgrid imaging polarimeters | |
| CN113884234A (en) | Complementary single-pixel centroid detection system and method | |
| Gordiyenko et al. | A multi-element thermal imaging system based on an uncooled bolometric array | |
| Becker et al. | Uniformity correction of CMOS image sensor modules for machine vision cameras | |
| Zhang et al. | Modified Two-Point Correction Method for Wide-Spectrum LWIR Detection System | |
| Caulfield et al. | Spatial oversampling in imaging sensors: Benefits in sensitivity and detection | |
| Leja et al. | Mathematical Model and Synthetic Data Generation for Infra-Red Sensors | |
| Lu et al. | Design and Performance Analysis of the Highly Sensitive Deep Vacuum Cooling sCMOS Imaging System for Highly Sensitive Detection of Space Targets | |
| Dewitte et al. | A Multispectral Camera Suite for the Observation of Earth’s Outgoing Radiative Energy | |
| Poonnen et al. | Digital readout integrated circuit for high dynamic range infrared imaging | |
| Gross et al. | Low light level CMOS sensor for night vision systems | |
| dos Santos et al. | Effects of the integration time on the performance of longwave infrared sensors | |
| Shi et al. | Two-point multi-section linear nonuniformity correction of IRFPA based on FPGA | |
| Cai et al. | Exploring Video Denoising in Thermal Infrared Imaging: Physics-inspired Noise Generator, Dataset and Model | |
| Wang et al. | Lightweight and Real-Time Infrared Image Processor Based on FPGA | |
| Plummer et al. | Investigation of speckle imagery spectral estimation challenges for modulation transfer function measurements | |
| Driggers et al. | A holistic approach to high performance infrared system design | |
| Li et al. | High dynamic range in infrared image using super-framing framework | |
| dos Santos et al. | Importance of frame rate during longwave infrared focal plane array evaluation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160817 |
|
| RJ01 | Rejection of invention patent application after publication |