EP2308227A1 - Procédé et dispositif de production d images à défauts réduits, haute résolution et contraste amélioré - Google Patents

Procédé et dispositif de production d images à défauts réduits, haute résolution et contraste amélioré

Info

Publication number
EP2308227A1
EP2308227A1 EP09775956A EP09775956A EP2308227A1 EP 2308227 A1 EP2308227 A1 EP 2308227A1 EP 09775956 A EP09775956 A EP 09775956A EP 09775956 A EP09775956 A EP 09775956A EP 2308227 A1 EP2308227 A1 EP 2308227A1
Authority
EP
European Patent Office
Prior art keywords
images
resolution
image
sequence
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP09775956A
Other languages
German (de)
English (en)
Inventor
Klaus Schertler
Falk Schubert
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Airbus Defence and Space GmbH
Original Assignee
EADS Deutschland GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by EADS Deutschland GmbH filed Critical EADS Deutschland GmbH
Publication of EP2308227A1 publication Critical patent/EP2308227A1/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G06T3/4061—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution by injecting details from different spectral ranges
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/21—Indexing scheme for image data processing or generation, in general involving computational photography
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20208—High dynamic range [HDR] image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the invention relates to a method for generating error-reduced high-resolution and contrast-enhanced images from image sequences with low-resolution images, which are obtained by an image sensor with adjustable acquisition parameters.
  • Imaging sensors such as camera systems, whether in the visible or IR range, are subject to limitations both in terms of finite image resolution (number of pixels) and finite dynamic range (contrast range). Primarily, these quality factors are defined by technical and economic constraints. Further boundary conditions result from the type of system in which the imaging sensor system is implemented. Thus, for example, in flying platforms, the image resolution and the dynamic range of the sensors can not be increased arbitrarily, since this significantly increases both the data volume generated and the requirements for any downstream data transmission devices.
  • a number of techniques are known which aim to increase the resolution and / or dynamic range of images produced by imaging sensors of a given number of pixels, both of individual images, and image sequences, such as motion pictures.
  • image processing of individual images such as brightness normalization, contrast enhancement, sharpening, noise reduction methods are known in which several images of a sequence of images (image stack) of a common processing, a so-called "fusion" are subjected to from low-resolution images of an image sequence or get several quality-enhanced images or a quality-enhanced image sequence.
  • HDR High Dynamic Range Reconstruction Processing
  • Another known type of image processing aims to process redundant information contained in a plurality of low-resolution images contained in an image sequence or a picture stack in such a way that a higher resolution and / or a reduction of errors is obtained.
  • the recording parameters for example the exposure, etc.
  • This type of image quality enhancement is known as superresolution processing and is described in detail in, for example, M. Gevrekci, BK Gunturk, Super-Resolution Approaches For Photometrically Various Image Sequences, ICASSP, 2007, and T. Pham, Spatial Adaptive in Super -Resolution of Under-Sampled Image Sequences, PhD Thesis, Delft University of Technology, 2006 or S.
  • HDR high-dynamic-range reconstruction processing
  • SR super-resolution processing
  • the object of the invention is to provide methods and apparatus for an improved generation of reduced-error high-resolution and contrast-enhanced images or image sequences from a low-resolution image sequence.
  • the invention is achieved by a method for producing error-reduced high-resolution and contrast-enhanced images according to claim 1.
  • the invention provides a method for generating error-reduced high-resolution and contrast-enhanced images from images of low-resolution image sequences obtained by an image sensor with adjustable acquisition parameters.
  • the image sequences of the low-resolution images are subjected to combined super-resolution processing and high-dynamic-range reconstruction processing to obtain error-reduced high-resolution and contrast-enhanced images on the basis of redundant and complementary image information contained in the low-resolution images to obtain.
  • the images of a sequence of low resolution images are subjected to superresolution processing in which redundant information portions of the low resolution images are processed to increase the resolution.
  • the sensor parameters are selectively varied from image to image.
  • the exposure time can be varied from image to image as a sensor parameter.
  • the sensor parameters can be kept unchanged.
  • the first sequence and the second sequence are different sequences of low resolution images.
  • the first sequence and the second sequence may alternately follow each other in blocks.
  • first sequence and the second sequence may be nested.
  • the images of the first sequence are fused by high-dynamic-range reconstruction processing to one or more images with increased contrast
  • the image or images with increased contrast used for superresolution processing to obtain one or more reduced-resolution images of increased resolution.
  • the first fusion step for HDR reconstruction processing and the second fusion step for SR processing are performed sequentially.
  • the two fusion steps can also be processed integrated in a common algorithm.
  • image sequences of low-resolution, high-resolution and contrast-enhanced images are again generated from the image sequences of the low-resolution images.
  • the processing of the low resolution images may include image registration to match the used images of a sequence.
  • the image registration can be made with respect to a reference image from the sequence of low-resolution images.
  • the generation of the sequence of low resolution images for the superresolution processing can take place permanently, the generation of the images of the sequence of low resolution images for the high dynamic range reconstruction processing can be initiated by automatic or manual control respectively.
  • the invention also provides a device for carrying out the above method.
  • Such a device can in particular be software-controlled.
  • FIG. 1 shows by way of example a fusion of redundant image information with the method of super-resolution by processing an image stack with redundant image information
  • FIG. 2 shows, by way of example, a fusion of complementary image information by means of high-dynamic-range reconstruction processing of a picture stack with complementary image information
  • FIG. 3 shows a schematic representation of the generation of alternating successive subsequences of images with redundant and complementary information components for generating error-reduced high-resolution and contrast-enhanced images from image sequences with low-resolution images according to an exemplary embodiment of the invention
  • FIG. 4 is a schematic representation of a fusion of successive sequences for high-dynamic-range reconstruction processing and super-resolution processing in order to obtain a reduced-error, high-resolution and contrast-enhanced image, wherein a high-dynamic range
  • Reconstruction processing image is used as a reference image for a registration, according to an embodiment of the invention.
  • FIG. 5 shows a schematic illustration of the generation of a result-reduced image sequence containing high-resolution and contrast-enhanced images of fusion of alternating successive sequences for superresolution processing and high-dynamic-range reconstruction processing.
  • both redundant and complementary bit information of an image sensor with adjustable acquisition parameters are obtained in order to then fuse them.
  • the sensor parameters are understood to mean those adjustable parameters of the sensor whose variation is suitable for generating complementary image information, ie image information that can not be derived from a single image, but only from a sequence or a stack of images with each varied sensor parameters.
  • Relevant in the sense of the invention is in particular the parameter of the exposure time of the sensor or the corresponding physical parameters such as the integration time of the sensor. on the pixels (picture elements) of the sensor or the choice of recording sensitivities within the sensor.
  • the sensor parameter or parameters are selected such that the low-resolution image sequence generated by the imaging sensor system contains both complementary and redundant information components.
  • FIG. 1 shows a fusion of redundant image information by superresolution processing from an image stack, here two images represented, with redundant image information to a resulting image with increased resolution.
  • FIG. 2 shows by way of a corresponding example a fusion of complementary image information (overexposure, underexposure) by means of high-dynamic-range reconstruction processing of an image stack, here two images, with complementary image information to a resulting image with increased contrast.
  • the redundant and complementary information components within a total sequence of images are represented by alternately successive partial sequences of recorded images, namely a sequence R for the redundant component and a sequence K for the complementary component.
  • the number of individual images within the R or K sequence can be specified.
  • the subsequences differ in that in the embodiment shown in FIG. 3 in the R-sequence for the super-resolution processing, the sensor parameters remain unchanged, ie apart from scenic changes such as changes in illumination or movement of object and / or sensor, it is So this is redundant multiple measurements.
  • one or more sensor parameters are selectively varied from image to image in the K sequence used for high-dynamic-range reconstruction processing. If the exposure time or the corresponding physical variable is varied from image to image as the sensor parameter to be changed, then the K sequence corresponds to an exposure series. In this case, the differently exposed images also contain different and thus complementary image information.
  • result image or result image sequence The fusion of the redundant and complementary information portions in the evaluation or processing of the image sequence leads to a reconstruction of contrast-enhanced high-resolution and reduced-error images or image sequences, referred to as result image or result image sequence.
  • Fig. 4 shows, in one embodiment, the fusion of successive R and K sequences to one or more result images.
  • the fusion takes place in order to produce a result image HDR-SR in two steps.
  • a fusion step 1 the images of a K sequence are fused by variation of one or more sensor parameters by high-dynamic-range reconstruction processing to form a contrast-enhanced image HDR.
  • An error reduction for example noise suppression and increase of the resolution then takes place in a fusion step 2 by super-resolution processing.
  • the result image HDR previously obtained from the first fusion step is explicitly used in order to obtain improved contrast information within the superresolution processing.
  • the result image obtained from the R sequence plus the HDR image is the image HDR-SR, which is a high-resolution, high-contrast, error-reduced image. All fusion steps may implicitly include an image registration that serves to match the individual images of a sequence.
  • this image registration takes place relative to a preselected reference image.
  • all other images are transformed, for example perspectively altered or distorted, so that they are optimally aligned with the reference image.
  • each of the images may serve an R or a K sequence.
  • FIG. 5 shows how, by executing the two fusion steps described above according to an exemplary embodiment, the fusion steps 1 and 2 are carried out successively for all individual images of the R sequence and the K sequence as a reference image, so that the full image sequence is enhanced in contrast, highly resolved and reduced in error which is shown as a result image sequence of HDR-SR images in FIG. 5.
  • the described fusion steps 1 and 2 can also be carried out integrated into a common fusion step instead of successively according to an alternative embodiment.
  • the generation of the R-sequence can be made permanent and that of the K-sequence by automatic or manual control, for example, only if the reconstruction of a contrast-enhanced high-resolution and reduced-error frame or a biopsy sequence appears necessary.
  • other or additional sensor parameters for generating the K sequence can also be varied, for example focusing or zooming.
  • One advantage of the invention is that contrast-enhanced, high-resolution and reduced-error images can be generated with existing sensors of comparatively low resolution. It is only the controllability of a sensor parameter, such as the exposure time to provide.
  • the data streams generated by the imaging sensor system are not enlarged, so that existing data transmission devices, for example from the imaging sensor system to an image processing device, can be used unchanged.
  • the reconstruction of the quality-enhanced image sequence can be done, for example, in the case of a flying sensor platform (reconnaissance aircraft, drone), for example after the data transmission in a ground station.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)
  • Studio Devices (AREA)

Abstract

L’invention concerne un procédé de production d’images à défauts réduits, haute résolution et contraste amélioré à partir de séquences d’images comprenant des images à plus faible résolution qui sont produites avec des paramètres d’enregistrement ajustables. Selon l’invention, les séquences d’images (K, R) à plus faible résolution sont soumises à un traitement combiné de super-résolution (SR) et de reconstruction à plage dynamique élevée (HDR) pour obtenir des images à défauts réduits, haute résolution et contraste amélioré sur la base d’informations d’image redondantes et complémentaires obtenues dans les images à plus faible résolution.
EP09775956A 2008-07-25 2009-07-11 Procédé et dispositif de production d images à défauts réduits, haute résolution et contraste amélioré Withdrawn EP2308227A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102008034979A DE102008034979B4 (de) 2008-07-25 2008-07-25 Verfahren und Einrichtung zur Erzeugung von fehlerreduzierten hochauflösenden und kontrastverbesserten Bildern
PCT/DE2009/000971 WO2010009703A1 (fr) 2008-07-25 2009-07-11 Procédé et dispositif de production d’images à défauts réduits, haute résolution et contraste amélioré

Publications (1)

Publication Number Publication Date
EP2308227A1 true EP2308227A1 (fr) 2011-04-13

Family

ID=41211745

Family Applications (1)

Application Number Title Priority Date Filing Date
EP09775956A Withdrawn EP2308227A1 (fr) 2008-07-25 2009-07-11 Procédé et dispositif de production d images à défauts réduits, haute résolution et contraste amélioré

Country Status (4)

Country Link
US (1) US8849059B2 (fr)
EP (1) EP2308227A1 (fr)
DE (1) DE102008034979B4 (fr)
WO (1) WO2010009703A1 (fr)

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012004908A (ja) * 2010-06-17 2012-01-05 Sony Corp 画像処理装置、および画像処理方法、並びにプログラム
US8466976B2 (en) 2010-11-03 2013-06-18 Eastman Kodak Company Digital camera providing high dynamic range images
US8462221B2 (en) * 2010-11-03 2013-06-11 Eastman Kodak Company Method for producing high dynamic range images
US8675957B2 (en) 2010-11-18 2014-03-18 Ebay, Inc. Image quality assessment to merchandise an item
JP2012186593A (ja) * 2011-03-04 2012-09-27 Sony Corp 画像処理装置、および画像処理方法、並びにプログラム
DE102011121334B4 (de) * 2011-12-16 2016-08-18 Testo Ag Messanordnung und korrespondierendes Bildaufbereitungsverfahren
DE102013211286A1 (de) 2013-06-17 2014-12-18 Carl Zeiss Industrielle Messtechnik Gmbh Verfahren zur Vermessung eines Werkstücks mit einem optischen Sensor
CN105931179B (zh) * 2016-04-08 2018-10-26 武汉大学 一种联合稀疏表示与深度学习的图像超分辨率方法及系统
CN109919841B (zh) * 2019-01-24 2022-12-27 重庆邮电大学 一种用于高动态范围图像联合上采样的导向图的合成方法
EP3835543B1 (fr) 2019-12-12 2023-02-22 Services Pétroliers Schlumberger Outil de diametrage multidoigts
EP4036369A3 (fr) * 2019-12-12 2022-12-07 Services Pétroliers Schlumberger Procédés d'imagerie modulaire de trou de forage
CN113992861B (zh) * 2020-07-27 2023-07-21 虹软科技股份有限公司 一种图像处理方法及图像处理装置
CN112634160A (zh) * 2020-12-25 2021-04-09 北京小米松果电子有限公司 拍照方法及装置、终端、存储介质
DE102021124153A1 (de) * 2021-09-17 2023-03-23 Homag Plattenaufteiltechnik Gmbh Verfahren und Vorrichtung zur Qualitätsprüfung einer Kante eines plattenförmigen Werkstücks

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0954376A (ja) * 1995-06-09 1997-02-25 Pioneer Electron Corp 立体表示装置
US20060291750A1 (en) * 2004-12-16 2006-12-28 Peyman Milanfar Dynamic reconstruction of high resolution video from low-resolution color-filtered video (video-to-video super-resolution)
US7412107B2 (en) * 2004-12-17 2008-08-12 The Regents Of The University Of California, Santa Cruz System and method for robust multi-frame demosaicing and color super-resolution
US8666196B2 (en) * 2005-01-19 2014-03-04 The United States Of America As Represented By The Secretary Of The Army System and method for super-resolution imaging from a sequence of color filter array (CFA) low-resolution images
US8577184B2 (en) * 2005-01-19 2013-11-05 The United States Of America As Represented By The Secretary Of The Army System and method for super-resolution imaging from a sequence of color filter array (CFA) low-resolution images
US7777790B2 (en) * 2005-01-27 2010-08-17 Technion Research & Development Foundation Ltd. Acquisition of image sequences with enhanced resolution
KR20080034508A (ko) 2005-08-08 2008-04-21 요셉 러브너 적응형 노출 제어
WO2009034688A1 (fr) * 2007-09-10 2009-03-19 Nikon Corporation Procédé et programme de traitement d'image

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO2010009703A1 *

Also Published As

Publication number Publication date
US8849059B2 (en) 2014-09-30
US20110182528A1 (en) 2011-07-28
DE102008034979A1 (de) 2010-01-28
DE102008034979B4 (de) 2011-07-07
WO2010009703A1 (fr) 2010-01-28

Similar Documents

Publication Publication Date Title
EP2308227A1 (fr) Procédé et dispositif de production d images à défauts réduits, haute résolution et contraste amélioré
DE69434657T2 (de) System und Verfahren zur elektronischen Bildstabilisierung
DE69333846T2 (de) Verfahren zur mischung von bildern und vorrichtung hierzu.
DE602004009702T2 (de) Verfahren und System zur automatischen Reduzierung von Aliasing-Artefakten
DE102020131265A1 (de) Segmentieren von video-rahmen unter verwendung eines neuronalen netzes mit verringerter auflösung und von masken aus vorhergehenden rahmen
DE112016004731T5 (de) Erweitern von Mehrfachansicht-Bilddaten mit synthetischen Objekten unter Verwendung von IMU und Bilddaten
DE112016005905T5 (de) Verfahren und System zum Verschmelzen erfasster Messungen
DE102004049676A1 (de) Verfahren zur rechnergestützten Bewegungsschätzung in einer Vielzahl von zeitlich aufeinander folgenden digitalen Bildern, Anordnung zur rechnergestützten Bewegungsschätzung, Computerprogramm-Element und computerlesbares Speichermedium
EP2791896B1 (fr) Procédé de génération d'images superésolues à résolution améliorée et dispositif de mesure associé
EP2649802A1 (fr) Procédé et dispositif de traitement de données d'image de deux capteurs appropriés à la capture d'images qui font partie d'un système de capteurs stéréoscopique
DE202016007867U1 (de) Steuerung des Sichtlinienwinkels einer Bildverarbeitungsplattform
DE102010053458A1 (de) Verfahren zur Aufbereitung von IR-Bildern und korrespondierende Wärmebildkamera
WO2009082990A1 (fr) Procédé et dispositif de production d'images multivue en temps réel
DE60031014T2 (de) Verfahren zur bewegungsschätzung zwischen zwei bildern mit verwaltung der wendung einer masche und entsprechendes kodierungsverfahren
DE4037739C1 (fr)
DE102006006449A1 (de) Verfahren zum Betrieb einer Röntgendiagnostikeinrichtung zur Erzeugung hochaufgelöster Bilder
DE112021003347T5 (de) Bildverarbeitungsvorrichtung, bildanzeigesystem, -verfahren und -programm
DE102005010119A1 (de) Verfahren zum Betrieb einer Röntgendiagnostikeinrichtung
AT520839B1 (de) Verfahren zur Erstellung einer Bildstapel-Datenstruktur
DE102020207695A1 (de) Tiefenschätzung aus Monokamerabildern mittels künstlicher Intelligenz
DE102009017135A1 (de) Verfahren, Vorrichtung und Computerprogramm zur Videodatenanalyse
DE102009001518B4 (de) Verfahren zur Erzeugung einer HDR-Videobildfolge
WO2019206539A1 (fr) Procédé de détermination d'information d'augmentation pour une image
DE102011010863A1 (de) Verfahren zur Kalibrierung eines an einem Fahrzeug angeordneten Sensors
DE102010006522A1 (de) Verfahren zur Analyse von korrespondierenden Bildpunkten in Bilddatensätzen

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20110128

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR

AX Request for extension of the european patent

Extension state: AL BA RS

17Q First examination report despatched

Effective date: 20110621

DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20120327