WO2015179841A1 - Procédés et systèmes de suppression de turbulences atmosphériques dans des images - Google Patents

Procédés et systèmes de suppression de turbulences atmosphériques dans des images Download PDF

Info

Publication number
WO2015179841A1
WO2015179841A1 PCT/US2015/032302 US2015032302W WO2015179841A1 WO 2015179841 A1 WO2015179841 A1 WO 2015179841A1 US 2015032302 W US2015032302 W US 2015032302W WO 2015179841 A1 WO2015179841 A1 WO 2015179841A1
Authority
WO
WIPO (PCT)
Prior art keywords
coefficients
video
image blocks
noise
video image
Prior art date
Application number
PCT/US2015/032302
Other languages
English (en)
Inventor
Alessandro Foi
Vladimir Katkovnik
Pavlo Molchanov
Enrique SANCHEZ-MONGE
Original Assignee
Flir Systems, Inc.
Noiseless Imaging Oy Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Flir Systems, Inc., Noiseless Imaging Oy Ltd. filed Critical Flir Systems, Inc.
Priority to EP15728981.0A priority Critical patent/EP3146499B1/fr
Priority to CA2949105A priority patent/CA2949105C/fr
Priority claimed from US14/720,086 external-priority patent/US9811884B2/en
Publication of WO2015179841A1 publication Critical patent/WO2015179841A1/fr
Priority to IL248876A priority patent/IL248876A0/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/80—Geometric correction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20182—Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering

Definitions

  • Fig. 15A illustrates an example of an input video image frame captured by an infrared imaging sensor in accordance with an embodiment of the disclosure.
  • Various embodiments of methods and systems disclosed herein may be used to model random noise and FPN to suppress both types of noise in images (e.g., video or still images). More specifically, in one or more embodiments, methods and systems may permit effective suppression of noise even in images that have a prominent FPN component, by modeling noise more accurately to comprise both random noise and FPN components, estimating one or more noise parameters, filtering images based on motion-adaptive parameters, and/or performing other operations described herein.
  • filtering may be performed on spatiotemporal volumes, any one of which may be constructed by grouping image blocks (e.g., a fixed-size portion or patch of a video image frame) extracted from a sequence of video image frames along a motion trajectory.
  • Memory component 120 may comprise one or more various types of memory devices including volatile and non-volatile memory devices, such as RAM (Random Access
  • noise exhibited in still or video images captured by many image sensors is not unstructured noise. Rather, both the random component and FPN component may be correlated. That is, noise pixels in different spatial (e.g., different pixel coordinates) and temporal (e.g., in different frames) locations are not independent of one another, but rather are correlated with each other. Typical noise in video image data may therefore be referred to as "colored” noise, rather than "white” noise. Such characteristics may be readily observed in power spectral density (PSD) graphs of example noise as shown in Figs. 3A-3B. More specifically, Fig.
  • PSD power spectral density
  • correlations of noise may be analyzed and expressed with respect to other transforms than the Fourier transform, for example, with respect to the discrete cosine transform (DCT), various types of wavelet transforms, or other suitable transforms.
  • DCT discrete cosine transform
  • wavelet transforms various types of wavelet transforms, or other suitable transforms.
  • %PN ⁇ FPN ® TM > (Equation 3) wherein are white noise factors following independent and identically distributed (i.i.d.) Gaussian distributions such that:
  • an actual pattern of FPN in the video image frames may be dynamically estimated, in addition to or in place of various statistical parameters associated with the FPN (e.g., a PSD an d a standard deviation °TMof the FPN estimated as described herein).
  • the dynamically estimated FPN pattern may be subtracted from the video image frames, and from the resulting video image frames a PSD of the residual FPN (e.g., FPN remaining in the video image frames after the dynamically estimated FPN pattern is subtracted) and/or other noise may be estimated online (e.g., using the received video image frames) as opposed to being estimated offline (e.g., using calibration video 402).
  • a PSD of the residual FPN e.g., FPN remaining in the video image frames after the dynamically estimated FPN pattern is subtracted
  • other noise may be estimated online (e.g., using the received video image frames) as opposed to being estimated offline (e.g., using calibration video 402).
  • Such online estimation of the PSD of the residual FPN or other noise may enable noise filter
  • filtering operations may be adjusted based on which plane (e.g., DC plane or AC co-volume) the coefficients belong to, as further described herein.
  • AC co- volume 706 may be viewed as other remaining coefficients, which typically satisfy some type of orthogonal relationship with the coefficients in DC-plane 704. It should be noted that Fig. 7 is merely a visual presentation provided for purposes of explaining filtering operations on a 3-d spectrum, and as such, the depiction of the location, size, and/or shape of 3-D spectrum 702, DC plane 704, AC co-volume 706 should not be understood as limiting a resulting 3-D spectrum.
  • some embodiments of methods and systems disclosed herein may be included in or implemented as various devices and systems that capture and/or process video or still images impaired by noise (e.g., video or still images captured by infrared image sensors or other sensors operating at a low signal-to-noise ratio regime, and/or video or still images processed by conventional FPN compensation techniques) to beneficially improve image quality.
  • noise e.g., video or still images captured by infrared image sensors or other sensors operating at a low signal-to-noise ratio regime, and/or video or still images processed by conventional FPN compensation techniques
  • alpha-rooting of the 3-D spectrum coefficients at operation 1312 may also include sharpening alpha-rooting of the spatial features common to all image blocks in a same spatiotemporal volume to sharpen those spatial features, which may be performed by operating on the temporal-DC coefficients (e.g., the coefficients of DC plane 704).
  • contents of the image blocks may be sharpened, while at the same time suppressing the higher frequency components of atmospheric turbulence effects (e.g., blurring) in image blocks.
  • performing different types of filtering, enhancements, or other modifications on different spectral dimension as in this example is advantageously facilitated by the structure of the constructed spatiotemporal volumes and the corresponding 3-D spectra.
  • Non-transitory instructions, program code, and/or data can be stored on one or more non-transitory machine readable mediums. It is also contemplated that software identified herein can be

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

La présente invention concerne diverses techniques pour supprimer des distorsions dans des images, telles que des distorsions causées par des turbulences atmosphériques. Par exemple, des blocs d'image similaires d'une séquence d'images peuvent être identifiés et suivis le long de trajectoires de mouvement pour construire des volumes spatiotemporels. Les trajectoires de mouvement sont lissées pour estimer les positions véritables des blocs d'image sans déplacements aléatoires dus aux distorsions, et les trajectoires lissées sont utilisées pour agréger les blocs d'image dans leurs nouvelles positions estimées afin de reconstruire la séquence d'images avec les déplacements aléatoires supprimés. Le flou qui peut rester à l'intérieur de chaque bloc d'image des volumes spatiotemporels peut être supprimé par modification des volumes spatiotemporels d'une manière collaborative. Par exemple, une transformée de décorrélation peut être appliquée aux volumes spatiotemporels afin de supprimer le flou dans un domaine de transformée, tel que par une opération sur des racines alpha ou d'autres opérations appropriées sur les coefficients des volumes spectraux.
PCT/US2015/032302 2014-05-23 2015-05-22 Procédés et systèmes de suppression de turbulences atmosphériques dans des images WO2015179841A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP15728981.0A EP3146499B1 (fr) 2014-05-23 2015-05-22 Procédés et systèmes de suppression de turbulences atmosphériques dans des images
CA2949105A CA2949105C (fr) 2014-05-23 2015-05-22 Procedes et systemes de suppression de turbulences atmospheriques dans des images
IL248876A IL248876A0 (en) 2014-05-23 2016-11-09 Methods and systems for suppressing atmospheric turbulence in images

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201462002731P 2014-05-23 2014-05-23
US62/002,731 2014-05-23
US14/720,086 2015-05-22
US14/720,086 US9811884B2 (en) 2012-07-16 2015-05-22 Methods and systems for suppressing atmospheric turbulence in images

Publications (1)

Publication Number Publication Date
WO2015179841A1 true WO2015179841A1 (fr) 2015-11-26

Family

ID=53396573

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2015/032302 WO2015179841A1 (fr) 2014-05-23 2015-05-22 Procédés et systèmes de suppression de turbulences atmosphériques dans des images

Country Status (3)

Country Link
CA (1) CA2949105C (fr)
IL (1) IL248876A0 (fr)
WO (1) WO2015179841A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10726863B2 (en) 2015-04-27 2020-07-28 Otocon Inc. System and method for locating mobile noise source
TWI702595B (zh) * 2018-03-30 2020-08-21 維呈顧問股份有限公司 移動噪音源的檢測系統與方法

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
HUEBNER CLAUDIA S ED - HOLST GERALD C ET AL: "Turbulence mitigation of short exposure image data using motion detection and background segmentation", INFRARED IMAGING SYSTEMS: DESIGN, ANALYSIS, MODELING, AND TESTING XXIII, SPIE, 1000 20TH ST. BELLINGHAM WA 98225-6705 USA, vol. 8355, no. 1, 11 May 2012 (2012-05-11), pages 1 - 13, XP060002956, DOI: 10.1117/12.918255 *
KOSTADIN DABOV ET AL: "JOINT IMAGE SHARPENING AND DENOISING BY 3D TRANSFORM-DOMAIN COLLABORATIVE FILTERING", 1 January 2007 (2007-01-01), pages 1 - 8, XP055206767, Retrieved from the Internet <URL:http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.75.6490&rep=rep1&type=pdf> [retrieved on 20150807] *
LI ET AL: "Suppressing atmospheric turbulent motion in video through trajectory smoothing", SIGNAL PROCESSING, ELSEVIER SCIENCE PUBLISHERS B.V. AMSTERDAM, NL, vol. 89, no. 4, 1 April 2009 (2009-04-01), pages 649 - 655, XP025839421, ISSN: 0165-1684, [retrieved on 20081022], DOI: 10.1016/J.SIGPRO.2008.10.012 *

Also Published As

Publication number Publication date
CA2949105C (fr) 2020-03-24
IL248876A0 (en) 2017-01-31
CA2949105A1 (fr) 2015-11-26

Similar Documents

Publication Publication Date Title
US9811884B2 (en) Methods and systems for suppressing atmospheric turbulence in images
US9635220B2 (en) Methods and systems for suppressing noise in images
Li et al. Single image de-hazing using globally guided image filtering
Tripathi et al. Single image fog removal using anisotropic diffusion
US7970170B2 (en) Continuous extended range image processing
Yang et al. Fusion of infrared polarization and intensity images using support value transform and fuzzy combination rules
Gao et al. Sand-dust image restoration based on reversing the blue channel prior
KR101901602B1 (ko) 디지털 사진에서 노이즈를 제거하는 장치 및 방법
Tripathi et al. Single image fog removal using bilateral filter
US11416970B2 (en) Panoramic image construction based on images captured by rotating imager
Singh et al. Image dehazing using Moore neighborhood-based gradient profile prior
CN104021532B (zh) 一种红外图像的图像细节增强方法
EP2740100A1 (fr) Procédé et système de suppression de brouillard, de brume ou de brume sèche dans des images et des vidéos
CN102170574A (zh) 一种实时视频去雾处理系统
CA2949105C (fr) Procedes et systemes de suppression de turbulences atmospheriques dans des images
Gadnayak et al. A survey on image dehazing methods
Wang et al. A fast method of foggy image enhancement
CN109903253B (zh) 一种基于景深先验的公路交通视频去雾算法
Roy et al. A survey on visibility enhancement techniques in degraded atmospheric outdoor scenes
EP3146499B1 (fr) Procédés et systèmes de suppression de turbulences atmosphériques dans des images
Khandelwal et al. Elimination of Fog in Single Image Using Dark Channel Prior
Arigela et al. Enhancement of hazy color images using a self-tunable transformation function
Trivedi et al. Dark Channel Prior and Global Contrast Stretching based Hybrid Defogging Image Technique
Xu Novel Image Dehazing Algorithm Using Scene Segmentation and Open Channel Model
Mohanram et al. An optimized image enhancement of foggy images using gamma adjustment

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15728981

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 248876

Country of ref document: IL

ENP Entry into the national phase

Ref document number: 2949105

Country of ref document: CA

NENP Non-entry into the national phase

Ref country code: DE

REEP Request for entry into the european phase

Ref document number: 2015728981

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2015728981

Country of ref document: EP