WO2014173464A1 - Noise correction in an infra-red imaging system - Google Patents

Noise correction in an infra-red imaging system Download PDF

Info

Publication number
WO2014173464A1
WO2014173464A1 PCT/EP2013/058803 EP2013058803W WO2014173464A1 WO 2014173464 A1 WO2014173464 A1 WO 2014173464A1 EP 2013058803 W EP2013058803 W EP 2013058803W WO 2014173464 A1 WO2014173464 A1 WO 2014173464A1
Authority
WO
WIPO (PCT)
Prior art keywords
correction
pixel values
values
image
pixel
Prior art date
Application number
PCT/EP2013/058803
Other languages
French (fr)
Inventor
Christel-Loïc TISSE
Yanpeng CAO
Original Assignee
Mtech Imaging Pte Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mtech Imaging Pte Ltd filed Critical Mtech Imaging Pte Ltd
Priority to PCT/EP2013/058803 priority Critical patent/WO2014173464A1/en
Publication of WO2014173464A1 publication Critical patent/WO2014173464A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/61Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/20Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from infrared radiation only
    • H04N23/23Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from infrared radiation only from thermal infrared radiation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/67Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response
    • H04N25/671Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response for non-uniformity detection or correction

Definitions

  • the present disclosure relates to a method and processing device for noise correction in an image, and in particular to noise correction in an image captured by an infra- red imaging device.
  • Infra-red (IR) imaging devices permit thermal images of a scene to be captured. Such devices have a broad range of applications, including night vision and thermography.
  • IR imaging device comprise a cooling system, which is typically a cryogenic cooler.
  • IR imaging devices with such cooling systems tend to be bulky and expensive .
  • Uncooled IR imaging devices are a class of IR imager without any cooling system. Instead, a focal plane array (FPA) of the device operates at the ambient temperature. Microbolometers are an example of an uncooled IR imager.
  • FPA focal plane array
  • a technical difficulty in uncooled IR imagers is compensating for fixed pattern noise (FPN) in the captured image.
  • FPN fixed pattern noise
  • FPN is heavily dependent on the FPA temperature, making it difficult to perform effective noise correction across a broad range of operating temperatures.
  • a method of noise correction in an image having a plurality of pixel values captured by a focal plane array sensitive to infra-red light comprising: performing by a processing device a first correction of each of said pixels values of said image to generate modified pixel values based on an offset and/or gain value dependent on a temperature of said focal plane array; determining a correction value for each pixel value; and performing by said processing device a second correction to at least partially correct intensity imbalance across said image by applying said correction values to said modified pixel values.
  • said correction values are based on said modified pixel values or on modified pixel values of a previously captured image.
  • performing said second correction comprises: calculating, based on at least some of said modified pixel values, at least one parameter of a correction function for determining said correction values; and using said correction function to generate each of said correction values and applying said correction values to said modified pixel values to provide corrected output pixel values .
  • said correction function is based on a 2-dimensional spatial model.
  • the 2-dimensional spatial model is a bivariate polynomial.
  • the bivariate polynomial model is of the following form:
  • x represents the horizontal coordinates of the pixel values
  • y represents the vertical coordinates of the pixel values
  • calculating said at least one parameter comprises generating gradient values for at least some of said modified pixel values, and determining a value of said at least one parameter such that gradients of the correction function match said generated gradient values.
  • calculating said at least one parameter comprises minimizing the following error function ⁇ ( ⁇ ) :
  • x represents the horizontal coordinates of the pixel values
  • y represents the vertical coordinates of the pixel values
  • y are weightings applied to pixel value x
  • dl/dx are the gradients in the horizontal direction of said modified pixel values
  • dl/dy are the gradient in the vertical direction of said modified pixel values
  • Qx,y are pixel-location related terms corresponding to the gradient in the horizontal direction of said correction function
  • Rx,y are pixel-location related terms corresponding to the gradient in the vertical direction of said correction function .
  • said at least one parameter ⁇ of the mathematical model is calculated based on the following equation:
  • x represents the horizontal coordinates of the pixel values
  • y represents the vertical coordinates of the pixel values
  • W p and to W p are weightings applied to pixel value x
  • y 3l]_/dx to ⁇ /dx are the gradients in the horizontal direction of the modified pixel values
  • 3l]_/dy to ⁇ /dy are the gradients in the vertical direction of the modified pixel values
  • Q]_ to Qp are pixel- related terms corresponding to the gradients in the horizontal direction of the correction model
  • R]_ to Rp are pixel-related terms corresponding to the gradients in the vertical direction of correction model.
  • said focal plane array is sensitive to thermal infra-red, and said focal plane array is a microbolometer .
  • said offset and/or gain values are selected based on a temperature reading associated with the focal plane array.
  • said offset and/or gain values are selected based on a temperature value determined based on said pixel values of said image and/or pixel values of a previous image captured by said focal plane array.
  • said offset and/or gain values comprise both a gain value and an offset value associated with each of said pixel values.
  • said offset value is determined by applying a polynomial function to the temperature of the focal plane array.
  • the method further comprises determining, during a calibration phase of said focal plane array, coefficient values of said polynomial function based on a plurality of measurements of pixel values under controlled conditions.
  • a computer readable medium storing a computer program that, when executed by a processing device, causes the above method to be implemented.
  • a processing device for performing noise correction in an image having a plurality of pixel values captured by a focal plane array sensitive to infra-red light, the processing device being configured to: perform a first correction of each of said pixels values of said image to generate modified pixel values based on an offset and/or gain value dependent on a temperature of said focal plane array; determine an correction value for each pixel value; and perform a second correction to at least partially correct intensity imbalance across said image by applying said correction values to said modified pixel values.
  • said image is a frame of a video sequence captured by said focal plane array, and wherein said correction values are based on modified pixel values generated for a previous frame of said video sequence.
  • an infra-red imaging device comprising: a focal plane array sensitive to infra-red light; and the above processing device.
  • Figure 1 is a block diagram illustrating an IR imaging device according to an example embodiment
  • Figure 2 illustrates a pixel circuit of a microbolometer according to an example embodiment
  • Figure 3 is a block diagram representing a method of performing noise correction of an IR image according to an example embodiment
  • Figures 4A and 4B are graphs illustrating gain and offset variations plotted against FPA temperature for one pixel of an IR imager according to an example embodiment.
  • Figure 5 is a flow diagram illustrating operations in a method of noise correction in an IR image according to an example embodiment .
  • Figure 1 illustrates an IR imaging device 100 comprising a focal plane array (FPA) 102 sensitive to infra-red light.
  • the device 100 is for example an uncooled IR imaging device, and the FPA 102 is for example a microbolometer, a pyrometer, or other type of IR sensitive array.
  • FPA focal plane array
  • the optics module 104 for example comprises optics elements such as one or more lenses and/or a diaphragm, and one or more mechanical elements such as a lens housing (not illustrated in Figure 1) .
  • the optics module 104 further comprises a shutter.
  • An output 106 of the FPA 102 which is for example a serial interface, is coupled to a processing device 108.
  • the processing device 108 comprises circuitry for image processing, for example spatial denoising, strip noise correction and defective pixel correction, and for FPN (fixed pattern noise) correction.
  • the processing device is for example implemented by an FPGA (field programmable gate array) , a DSP (digital signal processor) , an ASIC (application specific integrated circuit) or other hardware implementation.
  • the processing device 108 may comprise one or more processors configured to execute instructions from an instruction memory for implementing some or all of the image processing functions. While in Figure 1 the processing device 108 is integral with the IR imaging device 100, in alternative embodiments the processing device 108 could form part of a separate computing device coupled to the FPA 102 via a suitable wired or wireless interface .
  • the processing device 108 is for example coupled to an image memory 110 storing captured images, and to a display 112 for displaying the captured images.
  • the display 112 is for example a touch screen providing a user interface.
  • the FPA 102 is adapted to capture a video stream comprising a sequence of frames
  • the memory 110 is a buffer for temporarily storing the frames prior to display on the display 112.
  • the FPA provides a video stream at a rate of between 30 and 60 FPS (frames per second) .
  • the processing device 108 is also coupled to a memory 114 storing parameters for FPN correction.
  • the parameters are for example stored in one or more look-up tables.
  • the parameter memory 114 may be implemented in a same memory device as the image memory 110.
  • the parameter memory 114 stores gain values A x ⁇ y indicating a gain to be applied to each pixel value [x,y] of the image and offset values E>x,y indicating offsets to be applied to each pixel value [x,y].
  • the parameter memory 114 for example stores a correction function C Xj y for correcting intensity imbalance across the image. At least certain parameters of the correction function are calculated based on the pixel values, as will be described in more detail below.
  • a temperature reading T ] _ of the FPA 102 may be provided, to the processing device 108, by a temperature sensor mounted on the FPA 102.
  • the temperature reading for example has a precision of around 0.1°C.
  • the FPA temperature can also be estimated by processing of the image pixel values.
  • the FPA 102 typically comprises a 640 by 480, or 1024 by 768 array of pixels. Of course, many other array sizes would be possible.
  • the pixels of the FPA 102 are sensitive to long-wave IR light, such as light with a wavelength of between 7 and 13 ⁇ .
  • FIG 2 illustrates an example of one pixel 200 of the array of pixels forming the FPA 102 of Figure 1, in the case that the FPA 102 is a microbolometer .
  • the pixel 200 comprises a layer 202 suspended over a silicon substrate 204, for example at a distance of around 2
  • the layer 202 is formed of an IR absorbing material, having the property that its resistance is modified by the presence of IR radiation. Such a change in the electrical properties of layer 202 can be detected by appropriate circuitry in a readout module formed in the substrate 204.
  • the layer 202 is supported above the substrate 204 by a pair of arms 206 that also form electrodes connecting the layer 202 to the readout circuit of the substrate 204.
  • the arms 206 are for example connected to respective sides of the layer 202 by gold contacts 208.
  • a reflective layer 210 is for example positioned on the surface of the substrate 204 directly under the layer 202 such that any IR radiation that passes through layer 202 will be reflected back, thereby increasing the absorption rate.
  • the space between these parts is for example maintained at a partial vacuum.
  • FIG. 2 illustrates just one example of the structure of a pixel of the FPA 102. It will be apparent to those skilled in the art that there are many alternative structures that could be used.
  • Figure 3 is a block diagram representing a method of performing noise correction of pixel values S x ⁇ y of an IR image for example read from the FPA 102 of Figure 1, where x corresponds to the coordinate in the horizontal direction in the image, and y corresponds to the coordinate in the vertical direction in the image.
  • orientation dependent terms used in relation to images such as "vertical” and "horizontal", apply when the image is displayed on a display in a vertical plane.
  • Each of the pixel values S x ⁇ y is a digital value, for example of between 8 and 32 bits in length.
  • an ADC analog to digital converter
  • forming part of the processing device 108 or of the FPA 102 is used to convert analog signals from the pixels of the FPA 102 into digital pixel values S x ⁇ y.
  • a gain and offset correction is applied as represented by a multiplication operation 302 followed by an addition operation 304 applied to the pixel values S Xi y to generate modified pixel values Z' x ⁇ y.
  • a gain and offset correction is applied as represented by a multiplication operation 302 followed by an addition operation 304 applied to the pixel values S Xi y to generate modified pixel values Z' x ⁇ y.
  • the order of these operations could be reversed.
  • Such high frequency FPN results at least partially from variations between the pixels of the FPA 102, which have differing behaviour depending on characteristics such as the exact dimensions and conductivity of the layers of IR absorbing material in each pixel. Furthermore, the high frequency FPN is also to dependent on the temperature of the FPA 102.
  • the gain is applied based on temperature dependent gain values A X; y(T ] _) , and the offset is applied based on temperature dependent offset values B x ⁇ y(T ] _).
  • the temperature T ] _ represents the temperature of the FPA 102, which is for example obtained by a temperature reading made by a temperature sensor of the FPA 102 as shown in Figure 1.
  • a temperature estimation may be determined by analysing one or more images captured by the FPA 102.
  • the optics module 104 includes a shutter, this may be used as an alternative or in addition to a temperature reading or estimation to obtain reference pixel values while the shutter is closed allowing a calculation of the temperature dependent gain and offset values.
  • the gain and offset values A x? y(T]_) , B x,y( T l) at a plurality of temperature control points are for example stored in lookup tables, and are determined during a calibration phase of the FPA 102.
  • the imaging device 100 is for example placed inside a temperature chamber.
  • the temperature within the chamber is varied in order to apply a relatively broad range of FPA temperatures to the imaging device 100, for example a range from -5°C to 65°C.
  • the FPA temperature is for example kept stable for a 15 minute time period.
  • Two video sequences each for example of 200 frames or more, are then captured by pointing the imaging device 100 towards each of two blackbody radiation sources at different temperatures, for example at 20 °C and 50 °C respectively.
  • a two-point calibration procedure based on these two video sequences is then performed to calculate, for each temperature control point and for each pixel of the FPA 102, corresponding gain and offset values.
  • Figure 4A is a graph illustrating an example of the gain measurements for a given pixel of the FPA with coordinates [x ] _, y ] _] plotted for temperature control points at 5°C intervals over a temperature range of -5°C to 65°C.
  • the present inventors have found that the gain value A x ⁇ y(T ] _) for each pixel is relatively insensitive to temperature changes. Therefore, the memory 114 of Figure 1 for example stores a separate look-up table for each temperature control point, each look-up table indicating the gain value A x ⁇ y for each pixel calculated during the calibration phase as described above.
  • the multiplication operation 302 of Figure 3 is for example performed by selecting a lookup table corresponding to the temperature control point closest to the temperature T ] _ of the FPA 102, and multiplying the pixel values ⁇ >x r y by the corresponding gain values A x ⁇ y from the selected lookup table.
  • an interpolation between the gain values in two lookup tables can be used to provide a more accurate gain value.
  • a hysteresis loop can be implemented for switching between gain values. For example, as the FPA temperature T ] _ decreases, a switch between the gain values of a lookup table LUT1 to a lookup table LUT2 may be made when the temperature goes below a threshold I l ⁇ However, if the temperature increases again, the switch back to lookup table LUT1 is for example only performed when the temperature reaches a threshold T-
  • Figure 4B is a graph illustrating an example of the offset measurement for a given pixel of the FPA with coordinates [x]_, y]_] plotted for temperature control points at 5°C intervals over a temperature range of -5°C to 65°C.
  • the present inventors have found that the offset value of each pixel changes in a relatively smoothly curve as a function of the FPA temperature. Therefore, the variations in the offset values with temperature are for example modelled by a polynomial, such as a second order polynomial of the form:
  • first or third order could be used as the model for the offset values B x ⁇ y.
  • various mathematical techniques can be used to compute the coefficients of the first, second or third order polynomial model representing the offset B X/ y(T]_) .
  • the Vandermonde matrix can be constructed, Lagrange polynomial fitting can be performed, Newton interpolation can be applied, or Neville's algorithm can be used.
  • the optics module 104 includes a shutter
  • this shutter can for example be used to periodically recalibrate the offset values B x ⁇ y(T]_).
  • various mathematical techniques such as Lagrange polynomial fitting, that can be used to re-compute, in other words update, the coefficients.
  • intensity imbalance across the image is corrected by an addition operation 306 of an offset determined by a correction function C x ⁇ y.
  • the correction function ⁇ ⁇ , ⁇ could instead be applied by a multiplication operation of the pixel value by a weighting factor for example determined by:
  • the intensity imbalance is for example the result of temperature fluctuations of elements in the optics module 104, such as one or more lenses, a lens housing, or other mechanical parts.
  • This type of FPN tends to be of a relatively low frequency, meaning that there is a relatively smooth change in this noise component across the image. It should be noted that, while such intensity imbalance is the result of temperature fluctuations of the optics module, it is not well correlated to the temperature of any one optics element, and therefore correction using a technique like the one used for the high frequency FPN is not appropriate.
  • the correction function C Xj y is at least partially determined based on the modified pixel values Z' x ⁇ y as will now be explained.
  • the correction function C x ⁇ y is based on a 2-dimensional spatial model, and one or more parameters of the model are calculated based on the modified pixel values.
  • the correction function C x ⁇ y is based on a bivariate polynomial model of degree N, and for example has the following form:
  • x represents the horizontal coordinates of the pixel values
  • y represents the vertical coordinates of the pixel values
  • the value of N is for example chosen based on a desired precision of the noise estimation, and there is a trade-off between a higher value of N leading to greater precision, and a lower value of N leading to reduced complexity.
  • the vector ⁇ can be represented as follows:
  • the parameter vector ⁇ can be calculated based on this penalty function in closed form, i.e. it can be directly calculated as a series of matrix operations in a single iteration.
  • the following series of matrix operations can be performed:
  • p are weightings applied to pixel value x,y, 3l ] _/dx to ⁇ /dx are the gradients in the horizontal direction of the modified pixel values, 3l ] _/dy to ⁇ /dy are the gradients in the vertical direction of the modified pixel values, Q ] _ to Qp are the pixel-location related terms corresponding to the gradients in the horizontal direction of the correction model and R ] _ to Rp are the pixel-location related terms corresponding to the gradients in the vertical direction of correction model.
  • the weighting coefficient associated with the gradient of each pixel value in the x and y directions are chosen to exclude the relatively high gradient values resulting from features in the scene.
  • any IR image of a scene is likely to have regions that are naturally lighter and darker, with relatively high gradients between these regions. Therefore, if a gradient component Vi 3 ⁇ 4y is mainly due to the presence of distinct objects or edge features, the corresponding weighting W x ⁇ y is assigned a small or zero value so that this gradient has a low or no influence in the penalty function minimization operation.
  • the weighting function can be computed as: where o is the standard variance and controls the rate of decline.
  • a value of o of around 1 provides good results, implying that gradients with magnitudes greater than 1 are more likely to be the result of scene objects or edges.
  • a similar weighting mechanism can be applied to reduce the influence of other types of undesired signals.
  • the number of pixels P used for calculating the parameters of the correction function C Xj y is for example a reduced set of the total number of pixels of the FPA 102. For example, pixels at ten-pixel intervals in the x and y directions are selected, leading to a reduction by a factor 100 of the number of pixels to be considered.
  • Figure 5 is a flow diagram illustrating operations in a method of noise correction in an IR image.
  • a first correction of each pixel value of the IR image is performed to generate modified pixel values based on gain and/or offset values A x ⁇ y (T]_) , B x,y( T l) dependent on the FPA temperature.
  • correction values C Xj y are determined based on the modified pixel values.
  • these correction values C Xj y could be determined based on modified pixel values generated for a previous image.
  • a second correction is performed to correct intensity imbalance across the IR image by applying the correction values to the modified pixel values.
  • a correction function C Xj y based on a bivariate polynomial model of degree N is merely one advantageous example, and there are alternative 2-dimensional spatial functions that could be used to model the intensity imbalance of pixel values across the IR image.
  • the correction function could be based on a standard bivariate polynomial model having the following form:
  • the correction function could be based on a modified bivariate polynomial model, for example similar to the above model, but with the x ⁇ and/or y2 terms omitted if their influence is insignificant.
  • the correction function could be based on a 2-dimensional Gaussian model of the following form:
  • C XiY A exp(- (a(x - x 0 ) 2 + 2b ⁇ x - x 0 )(y - y 0 ) + c(y - y 0 ) 2 )))
  • the correction function could be based on a model comprising the sum or product of two or more 2-dimensional spatial functions (f]_, f2, f3 ⁇ ⁇ ⁇ ) , to provide a model in one of the following forms:
  • a penalty function based on gradients is used to calculate the parameters ⁇ of the correction function.
  • the gradient provides one characteristic permitting low frequency intensity imbalance in the image to be distinguished from the scene image.
  • low-frequency intensity imbalance in the image can be decoupled from the scene image using a separate processing step. Once such an estimate for the low-frequency intensity imbalance across the image has been extracted, the parameters of the correction function can be fitted directly based on this estimate.
  • the gradient function could be used more generally for fitting, using a penalty function, the characteristics of a correction function based on any model f (x,y) to the corresponding characteristics of the low-frequency FPN as follows :
  • penalty function based on other suitable characteristics could be used.
  • the penalty function could be based on the gradient orientations, on histograms, or on multi-scale responses as follows:
  • ori(f(x, y)) - ori(FPN)
  • Multi _ scale(f(x, y)) ⁇ Multi _ scale(FPN) An advantage of the embodiments described herein is that effective noise correction of an IR image can be achieved without the need of a temperature reference, such as a shutter. In particular, both high-frequency and low-frequency FPN can be corrected by two main correction operations as described herein.
  • a 2-dimensional spatial function f (x,y) is fitted to the low-frequency intensity non-uniformity caused by temperature fluctuations of the optics module.
  • the fitting is for example performed directly on FPN decoupled from the image scene, or indirectly on suitable characteristics, such as gradients, of the FPN.
  • a further advantage is that, in the case of a video sequence, the first operations 302 and 304 of Figure 3 can be applied to pixel values with no frame delay, as the correction parameters can be computed as soon as the FPA temperature is available.
  • the parameters of the correction function applied in the second operation 306 of Figure 3 are for example calculated during the time interval between one frame and the next, leading to only a single frame delay.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Toxicology (AREA)
  • Radiation Pyrometers (AREA)

Abstract

The invention concerns a method of noise correction in an image having a plurality of pixel values captured by a focal plane array sensitive to infra-red light, the method comprising: performing by a processing device a first correction of each of said pixels values of said image to generate modified pixel values (Z'x,y) based on an offset and/or gain value (Ax,y(T1), Bx,y(T1)) dependent on a temperature (T1) of said focal plane array; determining a correction value (Cx,y) for each pixel value; and performing by said processing device a second correction to at least partially correct intensity imbalance across said image by applying said correction values to said modified pixel values.

Description

NOISE CORRECTION IN AN INFRA-RED IMAGING SYSTEM
FIELD OF THE PRESENT DISCLOSURE
The present disclosure relates to a method and processing device for noise correction in an image, and in particular to noise correction in an image captured by an infra- red imaging device.
BACKGROUND
Infra-red (IR) imaging devices permit thermal images of a scene to be captured. Such devices have a broad range of applications, including night vision and thermography.
Some types of IR imaging device comprise a cooling system, which is typically a cryogenic cooler. IR imaging devices with such cooling systems tend to be bulky and expensive .
Uncooled IR imaging devices are a class of IR imager without any cooling system. Instead, a focal plane array (FPA) of the device operates at the ambient temperature. Microbolometers are an example of an uncooled IR imager.
A technical difficulty in uncooled IR imagers is compensating for fixed pattern noise (FPN) in the captured image. Indeed, in many applications it is desirable to be able to detect surface temperatures in the scene with a precision as low as 0.1°C. FPN is heavily dependent on the FPA temperature, making it difficult to perform effective noise correction across a broad range of operating temperatures. Furthermore, there is the added problem of spatial non-uniformity in the image resulting from temperature fluctuations of optical elements, such as lenses, positioned along the optical path.
There is thus a need for an effective, fast and low cost solution for performing noise correction in images captured by an IR imaging device.
SUMMARY
It is an aim of embodiments of the present description to at least partially address one or more needs in the prior art .
According to one aspect, there is provided a method of noise correction in an image having a plurality of pixel values captured by a focal plane array sensitive to infra-red light, the method comprising: performing by a processing device a first correction of each of said pixels values of said image to generate modified pixel values based on an offset and/or gain value dependent on a temperature of said focal plane array; determining a correction value for each pixel value; and performing by said processing device a second correction to at least partially correct intensity imbalance across said image by applying said correction values to said modified pixel values.
According to one embodiment, said correction values are based on said modified pixel values or on modified pixel values of a previously captured image.
According to one embodiment, performing said second correction comprises: calculating, based on at least some of said modified pixel values, at least one parameter of a correction function for determining said correction values; and using said correction function to generate each of said correction values and applying said correction values to said modified pixel values to provide corrected output pixel values .
According to one embodiment, said correction function is based on a 2-dimensional spatial model. According to one embodiment, the 2-dimensional spatial model is a bivariate polynomial.
According to one embodiment, the bivariate polynomial model is of the following form:
i=Nj=i 1 = 1 j=0
where is said at least one parameter, x represents the horizontal coordinates of the pixel values, and y represents the vertical coordinates of the pixel values.
According to one embodiment, calculating said at least one parameter comprises generating gradient values for at least some of said modified pixel values, and determining a value of said at least one parameter such that gradients of the correction function match said generated gradient values.
According to one embodiment, calculating said at least one parameter comprises minimizing the following error function Ε(γ) :
E(V =
Figure imgf000004_0001
where P is the number of pixels, x represents the horizontal coordinates of the pixel values, y represents the vertical coordinates of the pixel values, ^xlY and ^x,y are weightings applied to pixel value x,y, dl/dx are the gradients in the horizontal direction of said modified pixel values, dl/dy are the gradient in the vertical direction of said modified pixel values, Qx,y are pixel-location related terms corresponding to the gradient in the horizontal direction of said correction function and Rx,y are pixel-location related terms corresponding to the gradient in the vertical direction of said correction function .
According to one embodiment, said at least one parameter γ of the mathematical model is calculated based on the following equation:
γ = (MTWM) lMTWG
where :
Figure imgf000005_0001
and where P is the number of pixels, x represents the horizontal coordinates of the pixel values, y represents the vertical coordinates of the pixel values, to Wp and to Wp are weightings applied to pixel value x,y, 3l]_/dx to θΐρ/dx are the gradients in the horizontal direction of the modified pixel values, 3l]_/dy to θΐρ/dy are the gradients in the vertical direction of the modified pixel values, Q]_ to Qp are pixel- related terms corresponding to the gradients in the horizontal direction of the correction model and R]_ to Rp are pixel-related terms corresponding to the gradients in the vertical direction of correction model.
According to one embodiment, said focal plane array is sensitive to thermal infra-red, and said focal plane array is a microbolometer .
According to one embodiment, said offset and/or gain values are selected based on a temperature reading associated with the focal plane array.
According to one embodiment, said offset and/or gain values are selected based on a temperature value determined based on said pixel values of said image and/or pixel values of a previous image captured by said focal plane array.
According to one embodiment, said offset and/or gain values comprise both a gain value and an offset value associated with each of said pixel values.
According to one embodiment, said offset value is determined by applying a polynomial function to the temperature of the focal plane array. According to one embodiment, the method further comprises determining, during a calibration phase of said focal plane array, coefficient values of said polynomial function based on a plurality of measurements of pixel values under controlled conditions.
According to another aspect, there is provided a computer readable medium storing a computer program that, when executed by a processing device, causes the above method to be implemented.
According to a further aspect, there is provided a processing device for performing noise correction in an image having a plurality of pixel values captured by a focal plane array sensitive to infra-red light, the processing device being configured to: perform a first correction of each of said pixels values of said image to generate modified pixel values based on an offset and/or gain value dependent on a temperature of said focal plane array; determine an correction value for each pixel value; and perform a second correction to at least partially correct intensity imbalance across said image by applying said correction values to said modified pixel values.
According to one embodiment, said image is a frame of a video sequence captured by said focal plane array, and wherein said correction values are based on modified pixel values generated for a previous frame of said video sequence.
According to yet a further aspect, there is provided an infra-red imaging device comprising: a focal plane array sensitive to infra-red light; and the above processing device.
BRIEF DESCRIPTION OF THE DRAWINGS
The foregoing and other features and advantages will become apparent from the following detailed description of embodiments, given by way of illustration and not limitation with reference to the accompanying drawings, in which:
Figure 1 is a block diagram illustrating an IR imaging device according to an example embodiment;
Figure 2 illustrates a pixel circuit of a microbolometer according to an example embodiment; Figure 3 is a block diagram representing a method of performing noise correction of an IR image according to an example embodiment;
Figures 4A and 4B are graphs illustrating gain and offset variations plotted against FPA temperature for one pixel of an IR imager according to an example embodiment; and
Figure 5 is a flow diagram illustrating operations in a method of noise correction in an IR image according to an example embodiment .
DETAILED DESCRIPTION
Figure 1 illustrates an IR imaging device 100 comprising a focal plane array (FPA) 102 sensitive to infra-red light. The device 100 is for example an uncooled IR imaging device, and the FPA 102 is for example a microbolometer, a pyrometer, or other type of IR sensitive array.
Light enters the imaging device 100 and is focused on the FPA 102 via an optics module 104. The optics module 104 for example comprises optics elements such as one or more lenses and/or a diaphragm, and one or more mechanical elements such as a lens housing (not illustrated in Figure 1) . In some embodiments, the optics module 104 further comprises a shutter.
An output 106 of the FPA 102, which is for example a serial interface, is coupled to a processing device 108. The processing device 108 comprises circuitry for image processing, for example spatial denoising, strip noise correction and defective pixel correction, and for FPN (fixed pattern noise) correction. The processing device is for example implemented by an FPGA (field programmable gate array) , a DSP (digital signal processor) , an ASIC (application specific integrated circuit) or other hardware implementation. Alternatively or additionally, the processing device 108 may comprise one or more processors configured to execute instructions from an instruction memory for implementing some or all of the image processing functions. While in Figure 1 the processing device 108 is integral with the IR imaging device 100, in alternative embodiments the processing device 108 could form part of a separate computing device coupled to the FPA 102 via a suitable wired or wireless interface .
The processing device 108 is for example coupled to an image memory 110 storing captured images, and to a display 112 for displaying the captured images. The display 112 is for example a touch screen providing a user interface. In some embodiments, the FPA 102 is adapted to capture a video stream comprising a sequence of frames, and the memory 110 is a buffer for temporarily storing the frames prior to display on the display 112. For example, the FPA provides a video stream at a rate of between 30 and 60 FPS (frames per second) .
The processing device 108 is also coupled to a memory 114 storing parameters for FPN correction. As will be described in more detail below, the parameters are for example stored in one or more look-up tables. In some embodiments, the parameter memory 114 may be implemented in a same memory device as the image memory 110. For example, the parameter memory 114 stores gain values Ax^y indicating a gain to be applied to each pixel value [x,y] of the image and offset values E>x,y indicating offsets to be applied to each pixel value [x,y]. Furthermore, the parameter memory 114 for example stores a correction function CXjy for correcting intensity imbalance across the image. At least certain parameters of the correction function are calculated based on the pixel values, as will be described in more detail below.
As illustrated by a dashed line 116 in Figure 1, a temperature reading T]_ of the FPA 102 may be provided, to the processing device 108, by a temperature sensor mounted on the FPA 102. The temperature reading for example has a precision of around 0.1°C. As explained in more detail below, the FPA temperature can also be estimated by processing of the image pixel values.
The FPA 102 typically comprises a 640 by 480, or 1024 by 768 array of pixels. Of course, many other array sizes would be possible. In some embodiments, the pixels of the FPA 102 are sensitive to long-wave IR light, such as light with a wavelength of between 7 and 13 μτα.
Figure 2 illustrates an example of one pixel 200 of the array of pixels forming the FPA 102 of Figure 1, in the case that the FPA 102 is a microbolometer . The pixel 200 comprises a layer 202 suspended over a silicon substrate 204, for example at a distance of around 2 |im. The layer 202 is formed of an IR absorbing material, having the property that its resistance is modified by the presence of IR radiation. Such a change in the electrical properties of layer 202 can be detected by appropriate circuitry in a readout module formed in the substrate 204. The layer 202 is supported above the substrate 204 by a pair of arms 206 that also form electrodes connecting the layer 202 to the readout circuit of the substrate 204. The arms 206 are for example connected to respective sides of the layer 202 by gold contacts 208. A reflective layer 210 is for example positioned on the surface of the substrate 204 directly under the layer 202 such that any IR radiation that passes through layer 202 will be reflected back, thereby increasing the absorption rate. To increase thermal insulation between the layer 202 and the substrate 204, the space between these parts is for example maintained at a partial vacuum.
Figure 2 illustrates just one example of the structure of a pixel of the FPA 102. It will be apparent to those skilled in the art that there are many alternative structures that could be used.
Figure 3 is a block diagram representing a method of performing noise correction of pixel values Sx^y of an IR image for example read from the FPA 102 of Figure 1, where x corresponds to the coordinate in the horizontal direction in the image, and y corresponds to the coordinate in the vertical direction in the image. Throughout the present description, orientation dependent terms used in relation to images, such as "vertical" and "horizontal", apply when the image is displayed on a display in a vertical plane. Each of the pixel values Sx^y is a digital value, for example of between 8 and 32 bits in length. For example, an ADC (analog to digital converter) forming part of the processing device 108 or of the FPA 102 is used to convert analog signals from the pixels of the FPA 102 into digital pixel values Sx^y.
During a first processing step, to correct high frequency FPN, a gain and offset correction is applied as represented by a multiplication operation 302 followed by an addition operation 304 applied to the pixel values SXiy to generate modified pixel values Z'x^y. In alternative embodiments the order of these operations could be reversed.
Such high frequency FPN results at least partially from variations between the pixels of the FPA 102, which have differing behaviour depending on characteristics such as the exact dimensions and conductivity of the layers of IR absorbing material in each pixel. Furthermore, the high frequency FPN is also to dependent on the temperature of the FPA 102.
The gain is applied based on temperature dependent gain values AX;y(T]_) , and the offset is applied based on temperature dependent offset values Bx^y(T]_). The temperature T]_ represents the temperature of the FPA 102, which is for example obtained by a temperature reading made by a temperature sensor of the FPA 102 as shown in Figure 1. Alternatively, it will be apparent to those skilled in the art that a temperature estimation may be determined by analysing one or more images captured by the FPA 102. In yet a further embodiment, if the optics module 104 includes a shutter, this may be used as an alternative or in addition to a temperature reading or estimation to obtain reference pixel values while the shutter is closed allowing a calculation of the temperature dependent gain and offset values.
The gain and offset values Ax?y(T]_) , Bx,y(Tl) at a plurality of temperature control points are for example stored in lookup tables, and are determined during a calibration phase of the FPA 102. During such a calibration phase, the imaging device 100 is for example placed inside a temperature chamber. The temperature within the chamber is varied in order to apply a relatively broad range of FPA temperatures to the imaging device 100, for example a range from -5°C to 65°C.
For each desired temperature control point, which are for example at 5°C intervals, the FPA temperature is for example kept stable for a 15 minute time period. Two video sequences, each for example of 200 frames or more, are then captured by pointing the imaging device 100 towards each of two blackbody radiation sources at different temperatures, for example at 20 °C and 50 °C respectively. A two-point calibration procedure based on these two video sequences is then performed to calculate, for each temperature control point and for each pixel of the FPA 102, corresponding gain and offset values.
Figure 4A is a graph illustrating an example of the gain measurements for a given pixel of the FPA with coordinates [x]_, y]_] plotted for temperature control points at 5°C intervals over a temperature range of -5°C to 65°C. The present inventors have found that the gain value Ax^y(T]_) for each pixel is relatively insensitive to temperature changes. Therefore, the memory 114 of Figure 1 for example stores a separate look-up table for each temperature control point, each look-up table indicating the gain value Ax^y for each pixel calculated during the calibration phase as described above. Thus the multiplication operation 302 of Figure 3 is for example performed by selecting a lookup table corresponding to the temperature control point closest to the temperature T]_ of the FPA 102, and multiplying the pixel values ^>xr y by the corresponding gain values Ax^y from the selected lookup table.
In some embodiments, an interpolation between the gain values in two lookup tables can be used to provide a more accurate gain value. Furthermore, in some embodiments a hysteresis loop can be implemented for switching between gain values. For example, as the FPA temperature T]_ decreases, a switch between the gain values of a lookup table LUT1 to a lookup table LUT2 may be made when the temperature goes below a threshold I l · However, if the temperature increases again, the switch back to lookup table LUT1 is for example only performed when the temperature reaches a threshold T-|- 2' which is a higher level than threshold T-|- l ·
Figure 4B is a graph illustrating an example of the offset measurement for a given pixel of the FPA with coordinates [x]_, y]_] plotted for temperature control points at 5°C intervals over a temperature range of -5°C to 65°C. The present inventors have found that the offset value of each pixel changes in a relatively smoothly curve as a function of the FPA temperature. Therefore, the variations in the offset values with temperature are for example modelled by a polynomial, such as a second order polynomial of the form:
Figure imgf000012_0001
= aXry x Ti + bXiY x Tx + cXiY ~1
Therefore, knowing the coefficients ax^y, bx,y and cx,y is sufficient to allow the offset Bx^y(T]_) to be calculated for any temperature T]_ of the FPA 102. The memory 114 of Figure 1 for example stores these three coefficients in three corresponding look-up tables.
Rather than a second order polynomial, other orders of polynomial, such as first or third order, could be used as the model for the offset values Bx^y. Furthermore, assuming that, for a given pixel, offset values for three or more FPA temperature control points are known, various mathematical techniques can be used to compute the coefficients of the first, second or third order polynomial model representing the offset BX/y(T]_) . For example, the Vandermonde matrix can be constructed, Lagrange polynomial fitting can be performed, Newton interpolation can be applied, or Neville's algorithm can be used.
Furthermore, if the optics module 104 includes a shutter, this shutter can for example be used to periodically recalibrate the offset values Bx^y(T]_). Indeed, in the case that shutter-based calibration is used to provide new data points, there are various mathematical techniques, such as Lagrange polynomial fitting, that can be used to re-compute, in other words update, the coefficients.
Referring again to Figure 3, during a second processing step, intensity imbalance across the image is corrected by an addition operation 306 of an offset determined by a correction function Cx^y. In alternative embodiments, the correction function ^χ,γ could instead be applied by a multiplication operation of the pixel value by a weighting factor for example determined by:
Figure imgf000013_0001
The intensity imbalance is for example the result of temperature fluctuations of elements in the optics module 104, such as one or more lenses, a lens housing, or other mechanical parts. This type of FPN tends to be of a relatively low frequency, meaning that there is a relatively smooth change in this noise component across the image. It should be noted that, while such intensity imbalance is the result of temperature fluctuations of the optics module, it is not well correlated to the temperature of any one optics element, and therefore correction using a technique like the one used for the high frequency FPN is not appropriate.
The correction function CXjy is at least partially determined based on the modified pixel values Z'x^y as will now be explained. For example, the correction function Cx^y is based on a 2-dimensional spatial model, and one or more parameters of the model are calculated based on the modified pixel values.
For example, the correction function Cx^y is based on a bivariate polynomial model of degree N, and for example has the following form:
Figure imgf000013_0002
where x represents the horizontal coordinates of the pixel values, y represents the vertical coordinates of the pixel values, and are parameters of the correction function to be calculated, which form a vector having a number of elements fixed by the degree N. The value of N is for example chosen based on a desired precision of the noise estimation, and there is a trade-off between a higher value of N leading to greater precision, and a lower value of N leading to reduced complexity. The vector γ can be represented as follows:
Y = (Υι,ο, Υο,ι, -YwYw-u■■■ΙΟ,ΝΪ ~3
In one embodiment, a weighted least squares approach is used to calculate the parameter γ by minimizing the following penalty function: E(j) =
Figure imgf000014_0001
f x, (vlXf - VCXry ~4 where P is the total number of pixel values to be considered when estimating the parameter γ, x^y are weighting values for each pixel, lXrY is the gradient at pixel (x,y) and cXrY is the gradient of the correction function, which can be expressed, in the x and y directions, as:
Figure imgf000014_0002
Ά i=Nj=i
° i=i j=o
where Qx,y anc^ ¾,y are pixel-location related terms of the correction function corresponding to:
QXiY = (l,0,2x, y,0, ...
Figure imgf000014_0003
... o) ; and
RXiY = (θ,Ι,Ο, x,2y, ...0, ... Ny"'1)
By substituting these pixel-location related terms of the correction function into the penalty function, the penalty E (γ) can be expressed as:
1 = 1
Advantageously, the parameter vector γ can be calculated based on this penalty function in closed form, i.e. it can be directly calculated as a series of matrix operations in a single iteration. For example, the following series of matrix operations can be performed:
γ = (MTWM)~lMTWG ~8 here :
Figure imgf000015_0001
and where W to Wp and W to Wy
p are weightings applied to pixel value x,y, 3l]_/dx to θΐρ/dx are the gradients in the horizontal direction of the modified pixel values, 3l]_/dy to θΐρ/dy are the gradients in the vertical direction of the modified pixel values, Q]_ to Qp are the pixel-location related terms corresponding to the gradients in the horizontal direction of the correction model and R]_ to Rp are the pixel-location related terms corresponding to the gradients in the vertical direction of correction model.
The weighting coefficient associated with the gradient of each pixel value in the x and y directions are chosen to exclude the relatively high gradient values resulting from features in the scene. In particular, any IR image of a scene is likely to have regions that are naturally lighter and darker, with relatively high gradients between these regions. Therefore, if a gradient component Vi¾y is mainly due to the presence of distinct objects or edge features, the corresponding weighting Wx^y is assigned a small or zero value so that this gradient has a low or no influence in the penalty function minimization operation. The present inventors have observed that, as the magnitude of the gradient caused by spatial non-uniformity is insignificant, the weighting function can be computed as:
Figure imgf000015_0002
where o is the standard variance and controls the rate of decline. For example, the present inventors have found that a value of o of around 1 provides good results, implying that gradients with magnitudes greater than 1 are more likely to be the result of scene objects or edges. A similar weighting mechanism can be applied to reduce the influence of other types of undesired signals.
The number of pixels P used for calculating the parameters of the correction function CXjy is for example a reduced set of the total number of pixels of the FPA 102. For example, pixels at ten-pixel intervals in the x and y directions are selected, leading to a reduction by a factor 100 of the number of pixels to be considered.
Figure 5 is a flow diagram illustrating operations in a method of noise correction in an IR image.
In an operation 502, a first correction of each pixel value of the IR image is performed to generate modified pixel values based on gain and/or offset values Ax^y (T]_) , Bx,y(Tl) dependent on the FPA temperature.
In a subsequent operation 504, correction values CXjy are determined based on the modified pixel values. Alternatively, these correction values CXjy could be determined based on modified pixel values generated for a previous image.
In a subsequent operation 506, a second correction is performed to correct intensity imbalance across the IR image by applying the correction values to the modified pixel values.
A correction function CXjy based on a bivariate polynomial model of degree N is merely one advantageous example, and there are alternative 2-dimensional spatial functions that could be used to model the intensity imbalance of pixel values across the IR image.
For example, the correction function could be based on a standard bivariate polynomial model having the following form:
CXiy = Yjx + J2y + y3xy + J4x2y + ...
Alternatively, the correction function could be based on a modified bivariate polynomial model, for example similar to the above model, but with the x^ and/or y2 terms omitted if their influence is insignificant. Alternatively, the correction function could be based on a 2-dimensional Gaussian model of the following form:
CXiY = A exp(- (a(x - x0)2 + 2b{x - x0)(y - y0) + c(y - y0)2)) As yet a further alternative, the correction function could be based on a model comprising the sum or product of two or more 2-dimensional spatial functions (f]_, f2, f3 · · · ) , to provide a model in one of the following forms:
CX, y = y) + k' Y) + ···
Figure imgf000017_0001
Furthermore, in the example embodiment described above with relation to equation 7, a penalty function based on gradients is used to calculate the parameters γ of the correction function. In particular, the gradient provides one characteristic permitting low frequency intensity imbalance in the image to be distinguished from the scene image.
In alternative embodiments, low-frequency intensity imbalance in the image can be decoupled from the scene image using a separate processing step. Once such an estimate for the low-frequency intensity imbalance across the image has been extracted, the parameters of the correction function can be fitted directly based on this estimate.
The gradient function could be used more generally for fitting, using a penalty function, the characteristics of a correction function based on any model f (x,y) to the corresponding characteristics of the low-frequency FPN as follows :
Figure imgf000017_0002
Alternatively, a penalty function based on other suitable characteristics could be used. For example, the penalty function could be based on the gradient orientations, on histograms, or on multi-scale responses as follows:
ori(f(x, y)) -» ori(FPN)
hist(f(x, y)) → hist(FPN)
Multi _ scale(f(x, y)) → Multi _ scale(FPN) An advantage of the embodiments described herein is that effective noise correction of an IR image can be achieved without the need of a temperature reference, such as a shutter. In particular, both high-frequency and low-frequency FPN can be corrected by two main correction operations as described herein.
Advantageously, a 2-dimensional spatial function f (x,y) is fitted to the low-frequency intensity non-uniformity caused by temperature fluctuations of the optics module. The fitting is for example performed directly on FPN decoupled from the image scene, or indirectly on suitable characteristics, such as gradients, of the FPN.
A further advantage is that, in the case of a video sequence, the first operations 302 and 304 of Figure 3 can be applied to pixel values with no frame delay, as the correction parameters can be computed as soon as the FPA temperature is available. The parameters of the correction function applied in the second operation 306 of Figure 3 are for example calculated during the time interval between one frame and the next, leading to only a single frame delay.
Having thus described at least one illustrative embodiment, various alterations, modifications and improvements will readily occur to those skilled in the art.
For example, it will be apparent to those skilled in that while it is described that both an offset and a gain are applied to the pixel values, in some embodiments it may be sufficient to apply only an offset or only a gain.
Furthermore, it will be apparent to those skilled in the art that the various features described in relation to each of the embodiments could be combined, in alternative embodiments, in any combination.

Claims

1. A method of noise correction in an image having a plurality of pixel values captured by a focal plane array (102) sensitive to infra-red light, the method comprising:
performing by a processing device a first correction of each of said pixels values of said image to generate modified pixel values (Z'x,y) based on an offset and/or gain value (Ax^y(T]_), BX;y(T]_) ) dependent on a temperature (T]_) of said focal plane array;
determining a correction value (CXry) for each pixel value; and
performing by said processing device a second correction to at least partially correct intensity imbalance across said image by applying said correction values to said modified pixel values.
2. The method of claim 1, wherein said correction values are based on said modified pixel values or on modified pixel values of a previously captured image.
3. The method of claim 1 or 2, wherein performing said second correction comprises:
calculating, based on at least some of said modified pixel values, at least one parameter (γ) of a correction function (Cx^y) for determining said correction values; and
using said correction function to generate each of said correction values and applying said correction values to said modified pixel values to provide corrected output pixel values (Zx^ y) .
4. The method of claim 3, wherein said correction function is based on a 2-dimensional spatial model.
5. The method of claim 4, wherein the 2-dimensional spatial model is a bivariate polynomial.
6. The method of claim 5, wherein the bivariate polynomial model is of the following form:
i=N j = i 1=1 j=0 where is said at least one parameter, x represents the horizontal coordinates of the pixel values, and y represents the vertical coordinates of the pixel values.
7. The method of any of claims 3 to 6, wherein calculating said at least one parameter comprises generating gradient values for at least some of said modified pixel values, and determining a value of said at least one parameter such that gradients of the correction function match said generated gradient values.
8. The method of claim 7, wherein calculating said at least one parameter comprises minimizing the following error function E (γ) :
p
E(J)
2 = 1
where P is the number of pixels, x represents the horizontal coordinates of the pixel values, y represents the vertical coordinates of the pixel values, ^xlY and ^x,y are weightings applied to pixel value x, y, dl/dx are the gradients in the horizontal direction of said modified pixel values, dl/dy are the gradient in the vertical direction of said modified pixel values, Qx,y are pixel-location related terms corresponding to the gradient in the horizontal direction of said correction function and Rx,y are pixel-location related terms corresponding to the gradient in the vertical direction of said correction function .
9. The method of any of claims 3 to 8, wherein said at least one parameter γ of the mathematical model is calculated based on the following equation:
Y (MTWM) 1MTWG
where :
Figure imgf000020_0001
and where P is the number of pixels, x represents the horizontal coordinates of the pixel values, y represents the vertical coordinates of the pixel values, W to Wp and to Pi// are weightings applied to pixel value x,y, 3l]_/dx to θΐρ/dx are the gradients in the horizontal direction of the modified pixel values, 3l]_/dy to θΐρ/dy are the gradients in the vertical direction of the modified pixel values, Q]_ to Qp are pixel- related terms corresponding to the gradients in the horizontal direction of the correction model and R]_ to Rp are pixel-related terms corresponding to the gradients in the vertical direction of correction model.
10. The method of any of claims 1 to 9, wherein said focal plane array is sensitive to thermal infra-red.
11. The method of any of claims 1 to 10, wherein said focal plane array is a microbolometer .
12. The method of any of claims 1 to 11, wherein said offset and/or gain values (AXiy(T]_), Bx,y(Tl)) are selected based on a temperature reading associated with the focal plane array.
13. The method of any of claims 1 to 12, wherein said offset and/or gain values (AXiy(T]_), Bx,y(Tl)) are selected based on a temperature value determined based on said pixel values of said image and/or pixel values of a previous image captured by said focal plane array.
14. The method of claim 12 or 13, wherein said offset and/or gain values (AXiy(T]_), Bx,y(Tl)) comprise both a gain value and an offset value associated with each of said pixel values .
15. The method of claim 14, wherein said offset value is determined by applying a polynomial function to the temperature (T]_) of the focal plane array.
16. The method of claim 15, further comprising determining, during a calibration phase of said focal plane array, coefficient values of said polynomial function based on a plurality of measurements of pixel values under controlled conditions .
17. A computer readable medium storing a computer program that, when executed by a processing device, causes the method of any of claims 1 to 16 to be implemented.
18. A processing device for performing noise correction in an image having a plurality of pixel values captured by a focal plane array sensitive to infra-red light, the processing device being configured to:
perform a first correction of each of said pixel values of said image to generate modified pixel values (Z'Xiy) based on an offset and/or gain value (AXiy(T]_), Bx,y(Tl)) dependent on a temperature (T]_) of said focal plane array;
determine an correction value (CXry) for each pixel value; and
perform a second correction to at least partially correct intensity imbalance across said image by applying said correction values to said modified pixel values.
19. The processing device of claim 18, wherein said image is a frame of a video sequence captured by said focal plane array, and wherein said correction values are based on modified pixel values generated for a previous frame of said video sequence .
20. An infra-red imaging device comprising: a focal plane array sensitive to infra-red light; and the processing device of claim 18 or 19.
PCT/EP2013/058803 2013-04-26 2013-04-26 Noise correction in an infra-red imaging system WO2014173464A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/EP2013/058803 WO2014173464A1 (en) 2013-04-26 2013-04-26 Noise correction in an infra-red imaging system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2013/058803 WO2014173464A1 (en) 2013-04-26 2013-04-26 Noise correction in an infra-red imaging system

Publications (1)

Publication Number Publication Date
WO2014173464A1 true WO2014173464A1 (en) 2014-10-30

Family

ID=48184215

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2013/058803 WO2014173464A1 (en) 2013-04-26 2013-04-26 Noise correction in an infra-red imaging system

Country Status (1)

Country Link
WO (1) WO2014173464A1 (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3188469A1 (en) * 2016-01-04 2017-07-05 Sensors Unlimited, Inc. Gain normalization and non-uniformity correction
WO2018118801A1 (en) * 2016-12-20 2018-06-28 Seek Thermal, Inc. Thermography process for a thermal imaging system
CN109297604A (en) * 2018-09-28 2019-02-01 浙江兆晟科技股份有限公司 A kind of method and system obtaining thermal infrared imager two point correction scaling parameter
CN111426643A (en) * 2020-03-12 2020-07-17 北京中科锐景科技有限公司 Atmospheric nitrogen dioxide column concentration inversion stripe noise self-adaptive window correction method
CN112352419A (en) * 2018-06-08 2021-02-09 灵锐得公司 Apparatus and method for parasitic thermal compensation in an infrared camera
EP3642580A4 (en) * 2017-06-21 2021-03-17 Seek Thermal, Inc. Design, test, and operation of a small thermal imaging core
CN113724171A (en) * 2021-09-14 2021-11-30 浙江大华技术股份有限公司 Image correction method, image correction device, storage medium and electronic device
CN114630060A (en) * 2020-12-14 2022-06-14 菲力尔商业系统公司 Uncertainty measurement system and method related to infrared imaging
US11892356B2 (en) 2019-08-30 2024-02-06 Seek Thermal, Inc. Design, test, and operation of a small thermal imaging core

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1160550A1 (en) * 1999-01-14 2001-12-05 Matsushita Electric Industrial Co., Ltd. Infrared imaging device, vehicle with the same, and infrared image adjusting device
US20020117621A1 (en) * 2001-02-26 2002-08-29 Osamu Nakamura Infrared imaging apparatus
US20040200961A1 (en) * 2002-02-27 2004-10-14 Parrish William J. Microbolometer focal plane array systems and methods
EP1953509A1 (en) * 2007-01-30 2008-08-06 NEC Electronics Corporation Method and apparatus for compensating infrared sensor for temperature
US20120200714A1 (en) * 2009-10-22 2012-08-09 Christophe Minassian Method for correcting images output by a detector without temperature regulation and detector implementing such a method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1160550A1 (en) * 1999-01-14 2001-12-05 Matsushita Electric Industrial Co., Ltd. Infrared imaging device, vehicle with the same, and infrared image adjusting device
US20020117621A1 (en) * 2001-02-26 2002-08-29 Osamu Nakamura Infrared imaging apparatus
US20040200961A1 (en) * 2002-02-27 2004-10-14 Parrish William J. Microbolometer focal plane array systems and methods
EP1953509A1 (en) * 2007-01-30 2008-08-06 NEC Electronics Corporation Method and apparatus for compensating infrared sensor for temperature
US20120200714A1 (en) * 2009-10-22 2012-08-09 Christophe Minassian Method for correcting images output by a detector without temperature regulation and detector implementing such a method

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9992432B2 (en) 2016-01-04 2018-06-05 Sensors Unlimited, Inc. Gain normalization and non-uniformity correction
EP3188469A1 (en) * 2016-01-04 2017-07-05 Sensors Unlimited, Inc. Gain normalization and non-uniformity correction
CN110312919B (en) * 2016-12-20 2021-03-19 塞克热量股份有限公司 Thermal imaging process for thermal imaging system
WO2018118801A1 (en) * 2016-12-20 2018-06-28 Seek Thermal, Inc. Thermography process for a thermal imaging system
CN110312919A (en) * 2016-12-20 2019-10-08 塞克热量股份有限公司 Thermal imaging for thermal imaging system is handled
US10605668B2 (en) 2016-12-20 2020-03-31 Seek Thermal, Inc. Thermography process for converting signal to temperature in a thermal imaging system
US11353365B2 (en) 2017-06-21 2022-06-07 Seek Thermal, Inc. Design, test, and operation of a small thermal imaging core
EP3642580A4 (en) * 2017-06-21 2021-03-17 Seek Thermal, Inc. Design, test, and operation of a small thermal imaging core
US11774293B2 (en) 2017-06-21 2023-10-03 Seek Thermal, Inc. Design, test, and operation of a small thermal imaging core
CN112352419A (en) * 2018-06-08 2021-02-09 灵锐得公司 Apparatus and method for parasitic thermal compensation in an infrared camera
US11792536B2 (en) 2018-06-08 2023-10-17 Lynred Device and method for parasitic heat compensation in an infrared camera
CN109297604B (en) * 2018-09-28 2020-06-09 浙江兆晟科技股份有限公司 Method and system for acquiring two-point correction calibration parameters of thermal infrared imager
CN109297604A (en) * 2018-09-28 2019-02-01 浙江兆晟科技股份有限公司 A kind of method and system obtaining thermal infrared imager two point correction scaling parameter
US11892356B2 (en) 2019-08-30 2024-02-06 Seek Thermal, Inc. Design, test, and operation of a small thermal imaging core
CN111426643A (en) * 2020-03-12 2020-07-17 北京中科锐景科技有限公司 Atmospheric nitrogen dioxide column concentration inversion stripe noise self-adaptive window correction method
CN111426643B (en) * 2020-03-12 2022-12-23 北京中科锐景科技有限公司 Adaptive window correction method for strip noise based on atmospheric nitrogen dioxide column concentration inversion
CN114630060A (en) * 2020-12-14 2022-06-14 菲力尔商业系统公司 Uncertainty measurement system and method related to infrared imaging
CN113724171A (en) * 2021-09-14 2021-11-30 浙江大华技术股份有限公司 Image correction method, image correction device, storage medium and electronic device

Similar Documents

Publication Publication Date Title
WO2014173464A1 (en) Noise correction in an infra-red imaging system
US10110833B2 (en) Hybrid infrared sensor array having heterogeneous infrared sensors
US8208755B1 (en) Scene based non-uniformity correction systems and methods
US9208542B2 (en) Pixel-wise noise reduction in thermal images
JP4604033B2 (en) Radiometry using an uncooled microbolometer detector
US9843742B2 (en) Thermal image frame capture using de-aligned sensor array
EP2923187B1 (en) Hybrid infrared sensor array having heterogeneous infrared sensors
US20160065848A1 (en) Thermography for a thermal imaging camera
EP2898670B1 (en) Row and column noise reduction in thermal images
KR101827810B1 (en) Apparatus and method for generating nonuniformity correction data, and infrared camera
CN107836111B (en) System and method for enhanced dynamic range infrared imaging
US20200099825A1 (en) Error smoothing through global source non-uniformity correction
US20180063454A1 (en) Method and apparatus for correcting fixed pattern noise of an infrared image
GB2481090A (en) Thermal imaging camera for detection of two different sensitivities of radiation
US10600164B2 (en) Image adjustment based on locally flat scenes
Venkateswarlu et al. Nonuniformity compensation for IR focal plane array sensors
Tempelhahn et al. Development of a shutterless calibration process for microbolometer-based infrared measurement systems
KR101731287B1 (en) Method and apparatus for compensating output of infrared sensor
TW202146862A (en) Infrared imaging device capable of correcting images through temperature detection and image processing
US11539889B2 (en) Method for the noise optimization of a camera, in particular a handheld thermal imaging camera
JP7143558B2 (en) Infrared imaging device and program used therefor
Liu et al. Effective dark-corner noise removal for infrared images based on spatial-temporal features
Wolf et al. On-line nonuniformity and temperature compensation of uncooled IRFPAs using embedded digital hardware
JP2022030434A (en) Infrared image processor, infrared imaging system, and infrared image processing method
Wang et al. A new non-uniformity correction algorithm for IRFPA based on statistical properties of scene

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13718592

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13718592

Country of ref document: EP

Kind code of ref document: A1