WO2010103109A1 - Procédé de production d'une séquence d'images vidéo hdr - Google Patents

Procédé de production d'une séquence d'images vidéo hdr Download PDF

Info

Publication number
WO2010103109A1
WO2010103109A1 PCT/EP2010/053200 EP2010053200W WO2010103109A1 WO 2010103109 A1 WO2010103109 A1 WO 2010103109A1 EP 2010053200 W EP2010053200 W EP 2010053200W WO 2010103109 A1 WO2010103109 A1 WO 2010103109A1
Authority
WO
WIPO (PCT)
Prior art keywords
images
image
image sequence
exposure
value
Prior art date
Application number
PCT/EP2010/053200
Other languages
German (de)
English (en)
Inventor
Kilian Jacob
Peter Rieder
Markus Schu
Original Assignee
Trident Microsystems, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Trident Microsystems, Inc. filed Critical Trident Microsystems, Inc.
Publication of WO2010103109A1 publication Critical patent/WO2010103109A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/73—Circuitry for compensating brightness variation in the scene by influencing the exposure time
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/50—Control of the SSIS exposure
    • H04N25/57—Control of the dynamic range
    • H04N25/58—Control of the dynamic range involving two or more exposures

Definitions

  • the present invention relates to a method for generating an HDR video image sequence.
  • HDR video image sequences are video image sequences in which the individual images have a high dynamic range (HDR) compared to conventional video images.
  • HDR images can be generated by taking a plurality of images with different exposure times from a given camera position, the image information of which is then superimposed.
  • dark details of the photographed environment are displayed in high contrast
  • pictures with low exposure time for example, bright details of the imaged surroundings appear rich in contrast.
  • a method for generating an HDR video image sequence is described, for example, in US Pat. No. 6,993,2200 B2.
  • the object of the present invention is to provide an improved method for generating an HDR video signal sequence and a device suitable for carrying out this method.
  • the method according to the invention for generating a video image sequence with a high dynamic range comprises: generating a plurality of image sequences each having a plurality of images with different exposures; and generating a respective one of the individual images of a sequence of images.
  • This Method is evaluated prior to generating the images of a first image sequence movement in at least one temporally preceding second image sequence and a movement value is determined. The exposures of the individual images of the first image sequence are adjusted depending on the movement value.
  • the exposure of an image is affected by both the exposure time, which is exposed when the image is taken, and the size of the aperture.
  • a different exposure of the individual images can thus be achieved by a variation of the exposure time and / or by a variation of the aperture.
  • a "higher exposure" of one image compared to another image is present if a higher exposure time and / or a larger aperture were used to capture one image than for the other image.
  • enlarging the aperture results in a reduction in depth of field, and increasing the exposure time results in motion blur.
  • the exposure can be varied over both parameters, there is a possibility of optimization with regard to adaptation or change of the exposure with regard to the least possible motion blur or the greatest possible depth of field.
  • FIG. 1 illustrates three video image sequences recorded with different exposure times.
  • FIG. 2 schematically illustrates the generation of a
  • HDR video image sequence from the three video image sequences according to FIG. 1.
  • Figure 3 illustrates a video image sequence comprising temporally successive images generated at different exposure times.
  • FIG. 4 illustrates the generation of a plurality of video signal sequences with the same phase of motion from the video signal sequence according to FIG. 3.
  • FIG. 5 illustrates a method for adjusting the exposure time for the generation of a video image.
  • FIG. 6 illustrates an example of a device for generating a plurality of video image sequences with motion-dependent exposure times.
  • FIG. 7 illustrates a dependence of the exposure time on a movement value according to a first example.
  • FIG. 8 illustrates a dependence of the exposure time on a movement value according to a second example.
  • Figure 9 illustrates an example of an apparatus for generating a video image sequence having images with motion-dependent exposure times.
  • FIG. 10 illustrates a dependence of the exposure time on a movement value during the duration of a picture sequence.
  • FIG. 1 diagrammatically illustrates a plurality of-in the example three-video image sequences S1, S2, S3 which each have a number of temporally successive video images. These individual video image sequences form the basis for the generation of an HDR video image sequence.
  • the video images of the different video sequences are recorded with different exposures. For purposes of explanation, for example, it is assumed that video images of a first video image sequence S1 with a first exposure, video images of a second video image sequence S2 with a second exposure, and video images of a third video image sequence S3 with a third exposure are captured, the first exposure being greater than second exposure and wherein the second exposure is greater than the third exposure.
  • Exposure can be varied by one or more of the following parameters: exposure time, aperture and auxiliary lighting. Exposure is increased if, with the same aperture and the same additional illumination, the exposure time is increased, if the aperture is increased with the same exposure time and the same additional illumination, or if the intensity of the additional illumination is increased with the same exposure time and the same aperture.
  • the additional lighting can be realized for example by a lamp with two or more brightness levels. In order to be able to vary the brightness from image to image, the luminaire should have low delay times or persistence times. This can be achieved for example by a luminaire based on light emitting diodes (LEDs). Such a lamp includes, for example, one or more LEDs, which are turned on or off depending on the desired brightness of the additional lighting.
  • This exposure B is the exposure referred to below.
  • This exposure B is a function of the exposure time explained T B , the aperture A and the brightness Z of an additional illumination, so that in general:
  • the video image sequences shown schematically in FIG. 1 each have the same motion phases. This is illustrated in FIG. 1 in that in each case one image of each of the video image sequences has the same recording time. These recording times are designated in FIG. 1 by to, to ⁇ T, to ⁇ 2T. T in this case denotes a time interval between the recording times of the individual images of each of the video image sequences. Pictures of the individual video image sequences S1, S2, S3, which have the same recording time, are referred to below as an image sequence with the same recording time.
  • the images of an image sequence with the same recording time can be superimposed in a well-known manner to an HDR image. If one carries out such an overlay method for all image sequences with the same recording time, one obtains an HDR video image sequence. Such a procedure is shown schematically in FIG.
  • FIG. 2 an overlay unit is shown in FIG. 2, to which the individual video image sequences S1, S2, S3 are fed and which is designed to superimpose the image information of the image sequences with the same recording time and thus to generate an HDR video image sequence S.
  • a method for superimposing individual images of an image sequence with the same recording time into an HDR image and devices for carrying out such a process are basically knows, so that can be dispensed with further explanations.
  • Image sequences with the same recording time can be produced in that a plurality of receiving devices, such as cameras, are provided, which generate from the same camera Posi tion ⁇ images with different exposure times to the same recording time points.
  • a plurality of receiving devices such as cameras
  • Such a procedure is complicated insofar as several cameras are needed.
  • Image sequences with the same recording time can also be generated by taking the individual images of a sequence chronologically from a camera position and then performing a motion-compensated interpolation, by means of which images with the same motion phase from the temporally successive images with different exposure time be generated .
  • FIG. 4 schematically shows a device for generating video image sequences S1, S2, S3 having the same motion phases from the video image sequence S 'shown in FIG. 3, which comprises temporally successive images with different exposure times.
  • a video image sequence S 'with temporally successive images having different exposure times can be generated using a single imaging device, wherein the exposure time is changed from image to image.
  • the frame rate of this video image sequence S ' is higher than the frame rate of the video image sequences S1, S2, S3 shown in FIG.
  • This frame rate of the video image sequence S ' corresponds to the product of the desired frame rate of the later HDR video image sequence and the number of images whose image information is to be superposed onto an HDR image.
  • Figure 3 illustrated sequence S ' can be evenly spaced in time.
  • the superimposition of three images into one HDR image is only an example. Of course, only two or even more than three images that were generated with different exposure times can be superimposed to an HDR image.
  • FIG. 4 schematically shows a device for generating video image sequences S1, S2, S3 having the same motion phases from the video image sequence S 'shown in FIG.
  • This device comprises a motion estimation and interpolation unit 21, 22, 23 for each of the video image sequences S1, S2, S3 to be generated.
  • Each of these motion estimation and interpolation units is supplied with a subimage sequence of the image sequence S '.
  • These partial image sequences are formed by a multiplexer 20 in time division multiplexing the images of the input image sequence S 'alternately with the individual motion estimation and interpolation units 21, 22, 23, such that the first motion estimation and interpolation unit 21 only has the images, for example supplied with the first exposure time are supplied; the second motion estimation and interpolation unit 22 is supplied with only the images generated at the second exposure time; and that the third motion estimation and interpolation unit 23 is supplied only with the images generated at the third exposure time.
  • the individual motion estimation and interpolation units 21, 22, 23 are designed to perform a motion estimation and, using this motion estimation, to generate image sequences that have the same phase of motion.
  • Such motion estimation and interpolation methods and devices for carrying out such motion estimation and interpolation methods are known in principle, so that further explanations on this can be dispensed with.
  • the exposures of the images of the image sequence resulting in an HDR image are provided be superimposed, depending on a movement value.
  • the motion value at a given point in time at which an image is generated or recorded is dependent on movement that is present in images of the image sequence that were taken before this point in time.
  • FIGS. 1 and 3 denotes a time at which at least one image of an image sequence is to be recorded, the images of which are later superimposed to form an HDR image.
  • FIG. 1 in which a plurality of image sequences are generated simultaneously with the same movement phase, a plurality of images are generated at this time to.
  • FIG. 3 in which the image sequences with images to be overlaid are generated in temporal succession, only one image is generated at the instant to.
  • a movement value which is dependent on a movement value of the video image sequence before the time t0.
  • the determination of the movement value comprises the comparison of at least two previously recorded temporally successive images. If, in accordance with the example according to FIG. 1, there are several image sequences S1, S2, S3 with the same motion phases, one of these image sequences can be selected to determine the motion value and the motion value can be determined by comparing two temporally successive images of the selected image sequence. The temporally successive images can follow each other directly in time. However, there may be more pictures between the two pictures to compare.
  • images with the same phase of motion are, for example, images whose time of recording is: to ⁇ ki • Ti • T, where i ⁇ 0 or images for their recording time: t o -k 2 -TiT, with i > 0 or images for their time of admission applies: to ⁇ k3 • Ti • T, with i ⁇ 0th
  • each pixel has associated therewith three pixel values: a first pixel value for the color red (R); a second pixel value for the color green (G); and a third pixel value for the color blue (B).
  • each pixel value is also associated with three pixel values: a luminance value (Y); and two chrominance values (U, V).
  • a comparison of two temporally successive images for determining the movement value includes, for example, the determination of a distance measure.
  • This distance measure is a measure of the difference between the individual pixels of the associated pixel values.
  • the determination of the distance measure comprises, for example, the determination of the difference between the pixel values of those pixels which are located in the same positions in the two images, the magnitude of this difference and the summation of these differences.
  • the result obtained is also called the sum of the absolute values of the differences (SAD, Sum of Absolute Differences).
  • SAD Sum of Absolute Differences
  • the distance measure in this case is the higher, the more the image contents of the two images to be compared differ, the more moving contents are thus present in the two images to be compared.
  • any other measures of distance such as even powers of the pixel differences, may be used instead of the amounts of the differences.
  • the comparison of two temporally successive images may comprise the evaluation of only one pixel value associated with each pixel, e.g. only the red value, the green value or the blue value or the y value, the u-value or the v-value.
  • the method according to the invention it is provided in particular to set a difference between the exposures of the individual pictures of a picture sequence depending on the movement value such that the more moving contents the previously generated pictures contain, the lower this difference.
  • This is based on the consideration that with moving content a high depth of field or high dynamics of the displayed images are hardly perceived by the viewer anyway.
  • the image sequence S 'according to FIG. 3 and in which the generation of the images to be superimposed comprises a motion estimation (cf. Figure 4)
  • the overlaying of images with different exposure times greatly falsifies the image impression in the case of an erroneous motion estimation.
  • the dispersion of the exposures applied to the images of an image sequence is determined by the difference between the highest and the lowest exposure, for example a longest and a shortest exposure time or a largest and smallest aperture.
  • the scattering is determined by the difference between the highest and the lowest exposure, for example a longest and a shortest exposure time or a largest and smallest aperture.
  • it is intended to reduce the number of different exposures used with increasing movement. In extreme cases, if a high degree of motion is present in the previously recorded images, the same exposure can be used for all images of the image sequence.
  • the exposure as a function of the detected movement it is provided in another example to vary the number of images of an image sequence as a function of the detected movement. For example, it is intended to increase the number of images of a sequence of images when there is a high degree of movement.
  • the exposures of these individual images may differ, and the variation of the exposure may be achieved by varying one of the aforementioned parameters. If the individual images of a sequence are recorded chronologically in succession and the exposure is varied by varying the exposure times for the individual images, then the maximum permissible exposure time pro reduces as the number of images per image sequence increases Image, if a given time is available for taking pictures of a sequence.
  • FIG. 1 An example of a device for generating a plurality of video image sequences with the same motion phase and motion-dependent exposure time is shown in FIG.
  • This device comprises three recording devices or cameras 31, 32, 33, which are located at a common camera position. Each of these recording devices has an adjustable exposure time that is dependent on one of the recording device 31, 32, 33 supplied exposure signal El, E2, E3. This exposure time can be adjusted frame by frame.
  • Each of the recording devices 31, 32, 33 has an output at which the image sequence generated by the respective recording device is available.
  • These image sequences can be supplied to an overlay unit 1 (shown in dashed lines), which generates an HDR image sequence from these image sequences S1, S2, S3.
  • At least one of the image sequences S1, S2, S3 generated by the recording devices 31, 32, 33 is supplied with a motion detection unit which is designed to detect motion in the already recorded images of the image sequence, for example by comparing two temporally successive at least one of the images image sequences.
  • the movement value S41 is available at the output of this movement determination unit 41.
  • This movement value S41 is supplied to an exposure setting unit 43 which, depending on this movement value, provides exposure signals E1, E2, E3 which Exposure times of the individual recording devices 31, 32, 33 generates.
  • the exposure setting unit 43 is configured to set fixed predetermined different exposure values when the motion value S41 is below a predetermined threshold value S41o and to set equal exposure values when the motion value S41 is above a predetermined threshold value.
  • a procedure is shown schematically in FIG. 7, in which the exposure values E1, E2, E3 are shown as a function of the movement value S41.
  • the greater the motion components S41 the larger the motion components S41 have compared to each other for determining the motion value S41.
  • FIG. 9 shows an example of a device for generating a sequence of images with temporally successive images with different exposure time.
  • This device comprises a recording device 34 which has an exposure time which can be set by an exposure signal E.
  • the image signal S ' is available at an output of this device.
  • This image signal S ' is fed to a motion detection unit 44, which is designed to detect motion in the previously recorded image sequence and to generate a motion value S44 taking into account this movement.
  • This movement value S44 is supplied to an exposure setting unit 42.
  • This exposure setting unit 42 is configured to generate the exposure signal E. Unlike the device explained with reference to FIG. 9, the movement setting unit 42 according to FIG.
  • FIG. 9 generates time-sequentially different exposure values E, and in each case one exposure value for each image of a picture sequence to be generated by the recording device 34.
  • the timing of the exposure values generated by this exposure setting unit 42 is schematically shown in FIG. T in FIG. 10 designates the time interval between two images of the later HDR image sequence. The multiple images with different exposure times are generated within this period of time T.
  • El in Fig. 10 denotes a first exposure value, E2 a second exposure value, and E3 a third exposure value each applied to an image of an image sequence, the individual images of that image sequence then being superposed on an HDR image.
  • the image with the highest exposure time El is taken first, then the image with the next lower exposure time E2 and then the image with the lowest exposure time E3.
  • this order is basically arbitrary.
  • the individual exposure values E1, E2, E3 depend on the movement value S44.
  • the dependence of the exposure values E1, E2, E3 on the movement value S44 corresponds, for example, to the dependencies of the exposure values E1, E2, E3 of the movement value S41 explained above with reference to FIGS.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

L'invention concerne un procédé de production d'une séquence d'images vidéo avec plage dynamique élevée, comprenant : - production de plusieurs séquences d'images composées d'au moins une image; et production à chaque fois d'au moins une image à partir des différentes images d'une séquence d'images. Avant la production des images d'une première séquence d'images, le mouvement est analysé dans au moins une deuxième séquence d'images, temporellement antérieure, afin de déterminer une valeur de mouvement. Les expositions des différentes images de la première séquence d'images et/ou le nombre des images de la première séquence d'images sont sélectionnés en fonction de la valeur de mouvement.
PCT/EP2010/053200 2009-03-12 2010-03-12 Procédé de production d'une séquence d'images vidéo hdr WO2010103109A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102009001521.3 2009-03-12
DE102009001521A DE102009001521B4 (de) 2009-03-12 2009-03-12 Verfahren zur Erzeugung einer HDR-Videobildfolge

Publications (1)

Publication Number Publication Date
WO2010103109A1 true WO2010103109A1 (fr) 2010-09-16

Family

ID=42470617

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2010/053200 WO2010103109A1 (fr) 2009-03-12 2010-03-12 Procédé de production d'une séquence d'images vidéo hdr

Country Status (2)

Country Link
DE (1) DE102009001521B4 (fr)
WO (1) WO2010103109A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102015116026A1 (de) 2015-09-22 2017-03-23 JENETRIC GmbH Vorrichtung und Verfahren zur direkten optischen Bildaufnahme von Dokumenten und / oder lebenden Hautbereichen ohne abbildende optische Elemente

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0977432A2 (fr) * 1998-07-28 2000-02-02 Olympus Optical Co., Ltd. Dispositif de prise de vues
US6993200B2 (en) 2001-11-20 2006-01-31 Sony Corporation System and method for effectively rendering high dynamic range images
EP1909492A1 (fr) * 2006-10-02 2008-04-09 Arnold & Richter Cine Technik GmbH & Co. Betriebs KG Caméra cinématographique numérique

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3630905B2 (ja) * 1997-02-28 2005-03-23 キヤノン株式会社 撮像方法及び撮像装置
KR20080034508A (ko) * 2005-08-08 2008-04-21 요셉 러브너 적응형 노출 제어

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0977432A2 (fr) * 1998-07-28 2000-02-02 Olympus Optical Co., Ltd. Dispositif de prise de vues
US6993200B2 (en) 2001-11-20 2006-01-31 Sony Corporation System and method for effectively rendering high dynamic range images
EP1909492A1 (fr) * 2006-10-02 2008-04-09 Arnold & Richter Cine Technik GmbH & Co. Betriebs KG Caméra cinématographique numérique

Also Published As

Publication number Publication date
DE102009001521B4 (de) 2012-12-27
DE102009001521A1 (de) 2010-09-23

Similar Documents

Publication Publication Date Title
DE3907118C2 (de) Schaltungsanordnung zum Ausdehnen eines Steuersignals bei Videosignalverarbeitung
DE102010062496B4 (de) Verfahren und Vorrichtung zum Verarbeiten von Bildinformationen zweier zur Bilderfassung geeigneter Sensoren eines Stereo-Sensor-Systems
DE102016220757A1 (de) Verfahren und Vorrichtung zur Materialbahnbeobachtung und Materialbahninspektion
DE112012004352B4 (de) Stereobildgebungsvorrichtung
DE102012201696A1 (de) Bildverarbeitungsvorrichtung, zugehöriges Steuerverfahren, Programm und Aufzeichnungsmedium
EP3253045B1 (fr) Procédé de reconnaissance et de correction d'erreurs d'image et unité de traitement d'images et programme informatique associé
EP3634003A1 (fr) Procédé et dispositif de synchronisation temporelle de la transmission optique des données dans l'espace libre
EP2017784A2 (fr) Procédé de traitement d'une suite d'images à l'aide d'images vidéo successives destinées à l'amélioration de la résolution spatiale
WO2016008953A1 (fr) Procédé de représentation d'une scène en trois dimensions sur un moniteur auto-stéréoscopique
DE10236207A1 (de) Verfahren zur bewegungsvektorgestützten Interpolation eines Bildpunktes eines Zwischenbildes einer Bildfolge
DE102010014733B4 (de) Chromakeyverfahren und Chromakeyvorrichtung zur Aufnahme und Bildbearbeitung von Kamerabildern
DE102009026983A1 (de) Bildverarbeitungsverfahren mit einer Bewegungsschätzung und Bildverarbeitungsanordnung
EP2105883A1 (fr) Procédé destiné au test d'un vecteur de mouvement
DE102009001521B4 (de) Verfahren zur Erzeugung einer HDR-Videobildfolge
DE102009001518B4 (de) Verfahren zur Erzeugung einer HDR-Videobildfolge
DE102011117654B4 (de) Verfahren zum Betreiben einer Bildverarbeitungseinrichtung sowie entsprechende Bildverarbeitungseinrichtung
DE2360562A1 (de) Verfahren zum gewinnen von videosignalen und vorrichtung zur durchfuehrung des verfahrens
DE10327576A1 (de) Verfahren und Vorrichtung zur bewegungsvektorgestützten Bildpunktinterpolation
DE102008036279B4 (de) Verfahren zur Ermittlung eines Bewegungsvektor zu einem Bildblock eines Zwischenbilds
DE4327733C1 (de) Bewegungsdetektor
DE102009057724B4 (de) Bilderfassungsvorrichtung und Verfahren zum Reduzieren von Bewegungsunschärfe
DE202016008887U1 (de) Bildverarbeitungseinheit und Computerprogramm zur Erkennung und Korrektur von Bildfehlern
DE102004051161A1 (de) Verfahren zur Bilddarstellung unter Verwendung einer sequentiellen Farbdarstellung
DE102022002637B3 (de) Kamerasystem und Verfahren zu dessen Betrieb
DE19649651C1 (de) Verfahren und Schaltungsanordnung zum Erzeugen einer Folge von progressiven Bildern

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10713801

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 10713801

Country of ref document: EP

Kind code of ref document: A1