EP1449357A1 - Procede et systeme de combinaison de sequences video avec alignement spatio-temporel - Google Patents

Procede et systeme de combinaison de sequences video avec alignement spatio-temporel

Info

Publication number
EP1449357A1
EP1449357A1 EP01946591A EP01946591A EP1449357A1 EP 1449357 A1 EP1449357 A1 EP 1449357A1 EP 01946591 A EP01946591 A EP 01946591A EP 01946591 A EP01946591 A EP 01946591A EP 1449357 A1 EP1449357 A1 EP 1449357A1
Authority
EP
European Patent Office
Prior art keywords
images
composite representation
given
background
synchronizing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP01946591A
Other languages
German (de)
English (en)
Other versions
EP1449357A4 (fr
Inventor
Paolo Prandoni
Martin Vetterli
Serge Ayer
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ecole Polytechnique Federale de Lausanne EPFL
Businger Peter A
Original Assignee
Ecole Polytechnique Federale de Lausanne EPFL
Businger Peter A
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ecole Polytechnique Federale de Lausanne EPFL, Businger Peter A filed Critical Ecole Polytechnique Federale de Lausanne EPFL
Publication of EP1449357A1 publication Critical patent/EP1449357A1/fr
Publication of EP1449357A4 publication Critical patent/EP1449357A4/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/272—Means for inserting a foreground image in a background image, i.e. inlay, outlay
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2624—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of whole input images, e.g. splitscreen

Definitions

  • the present invention relates to visual displays and, more specifically, to time-dependent visual displays.
  • a composite video sequence can be generated which includes visual elements from each of the given sequences, suitably synchronized and represented in a chosen focal plane. For example, given two video sequences with each showing a different contestant individually racing the same down-hill course, the composite sequence can include elements from each of the given sequences to show the contestants as if racing simultaneously. Further applications are in track and field, diving, horseback riding and golf, for close comparison between contestants.
  • a composite video sequence can be made also by similarly combining one or more video sequences with one or more different sequences such as audio sequences, for example.
  • generating a composite representation from given images can be facilitated by referring to a common background against which foreground objects are imaged. The composite representation can be formed by suitably blending together of foreground and background, and with re-scaling and/or re-framing for optimized presentation.
  • Fig. 1 is a block diagram for synchronization and alignment processing.
  • Figs. 2 A and 2B are schematics of different downhill skiers passing before a video camera.
  • Figs. 3 A and 3B are schematics of images recorded by the video camera, corresponding to Figs. 2A and 2B.
  • Fig. 4 is a schematic of Figs. 2A and 2B combined.
  • Fig. 5 is a schematic of the desired video image, with the scenes of Fig. 3 A and 3B projected in a chosen focal plane.
  • Fig. 6 is a frame from a composite video sequence which was made with a prototype implementation of the invention.
  • Fig. 7 is a block diagram of processing for building a database of sequences registered on a manifold.
  • Fig. 8 is a block diagram for re-framing in generating a target sequence.
  • Fig. 9 is a schematic illustrating spatial indexing on a cylinder. Detailed Description
  • a video sequence may be defined as a sequence of image fields, each containing the visual information pertaining to the portion of the physical world seen by a camera at contiguous, discrete time instants.
  • spatial information which is related to the imaged physical features, and to the instantaneous conditions of the camera such as displacement, position, aperture, pan angle and tilt angle, for example.
  • Such information can be understood as a composite spatial index, representing a virtual camera frame and corresponding to a unique finite region of a multidimensional indexing manifold. Conversely, from a selected finite region of the manifold a spatial index can be derived which need not be unique.
  • Each field of the video sequence can be indexed further based on temporal information including a time index associated with each field, which can be used for temporal synchronization of different sequences.
  • Combined spatial and temporal indexing can be interpreted as being on a multidimensional manifold which is used as a global coordinate system for all possible frames which can be captured by the camera within the range of its physical limitations.
  • the invention can be appreciated in analogy with 2-dimensional (2D) "morphing", i.e. the smooth transformation, deformation or mapping of one image, II, into another, 12, in computerized graphics.
  • morphing leads to a video sequence which shows the transformation of II into 12, e.g., of an image of an apple into an image of an orange, or of one human face into another.
  • the video sequence is 3-dimensional, having two spatial and a temporal dimension. Parts of the sequence may be of special interest, such as intermediate images, e.g. the average of two faces, or composites, e.g. a face with the eyes from II and the smile from 12.
  • morphing between images can be appreciated as a form of merging of features from the images.
  • the invention is concerned with a more complicated task, namely the merging of two video images, especially in video sequences.
  • the morphing or mapping from one sequence to another leads to 4-dimensional data which cannot be displayed easily.
  • any intermediate combination, or any composite sequence leads to a new video sequence.
  • Of particular interest is the generation of a new video sequence combining elements from two or more given sequences, with suitable spatio-temporal alignment or synchronization, and projection into a chosen focal plane.
  • video sequences obtained from two contestants having traversed a course separately can be time-synchronized by selecting the frames corresponding to the start of the race.
  • the sequences may be synchronized for coincident passage of the contestants at a critical point such as a slalom gate, for example.
  • the chosen focal plane may be the same as the focal plane of the one or the other of the given sequences, or it may be suitably constructed yet different from both.
  • the video sequences synchronized can be further aligned spatially, e.g. to generate a composite sequence giving the impression of the contestants traversing the course simultaneously.
  • spatial alignment can be performed on a frame-by-frame basis.
  • the view in an output image can be extended to include background elements from several sequential images.
  • Fig. 1 shows two image sequences IS 1 and IS2 being fed to a module 1 1 for synchronization into synchronized sequences IS 1 ' and IS2'.
  • the sequences IS1 and 1S2 may have been obtained for two contestants in a down-hill racing competition, and they may be synchronized by the module 1 1 so that the first frame of each sequence corresponds to its contestant leaving the starting gate.
  • the synchronized sequences are fed to a module 12 for background-foreground extraction, as well as to a module 13 for camera coordinate transformation estimation.
  • the module 12 For each of the image sequences, the module 12 yields a weight-mask sequence (WMS), with each weight mask being an array having an entry for each pixel position and differentiating between the scene of interest and the background/foreground.
  • WMS weight-mask sequence
  • the generation of the weight mask sequence involves computerized searching of images for elements which, from frame to frame, move relative to the background.
  • the module 13 yields sequence parameters SP1 and SP2 including camera angles of azimuth and elevation, and camera focal length and aperture among others. These parameters can be determined from each video sequence by computerized processing including interpolation and matching of images.
  • a suitably equipped camera can furnish the sequence parameters directly, thus obviating the need for their estimation by computerized processing.
  • a camera can include sensors for tilt, roll and pan angles and record their instantaneous readings along with each image taken.
  • the weight-mask sequences WMS1 and WMS2 are fed to a module 13 for "alpha-layer" sequence computation.
  • the alpha layer is an array which specifies how much weight each pixel in each of the images should receive in the composite image.
  • sequence parameters SP1 and SP2 as well as the alpha layer are fed to a module 15 for projecting the aligned image sequences in a chosen focal plane, resulting in the desired composite image sequence.
  • a module 15 for projecting the aligned image sequences in a chosen focal plane, resulting in the desired composite image sequence.
  • Fig. 2 A shows a skier A about to pass a position marker 21, with the scene being recorded from a camera position 22 with a viewing angle ⁇ (A).
  • the position reached by A may be after an elapse of t(A) seconds from A's leaving the starting gate of a race event.
  • Fig. 2B shows another skier, B, in a similar position relative to the marker 21, and with the scene being recorded from a different camera position 23 and with a different, more narrow viewing angle ⁇ (B).
  • the position of skier B corresponds to an elapse of t(A) seconds from B leaving the starting gate.
  • skier B has traveled farther along the race course as compared with skier A.
  • Figs. 3A and 3B show the resulting respective images.
  • Fig. 4 shows a combination with Figs. 2A and 2B superposed at a common camera location.
  • Fig. 5 shows the resulting desired image projected in a chosen focal plane, affording immediate visualization of skiers A and B as having raced jointly for t(A) seconds from a common start.
  • Fig. 6 shows a frame from a composite image sequence generated by a prototype implementation of the technique, with the frame corresponding to a point of intermediate timing.
  • the value of 57.84 is the time, in seconds, that it took the slower skier to reach the point of intermediate timing, and the value of +0.04 (seconds) indicates by how much he is trailing the faster skier.
  • the prototype implementation of the technique was written in the "C" programming language, for execution on a SUN Workstation or a PC, for example.
  • Dedicated firmware or hardware can be used for enhanced processing efficiency, and especially for signal processing involving matching and interpolation.
  • background and foreground can be extracted using a suitable motion estimation method.
  • This method should be "robust", for background/foreground extraction where image sequences are acquired by a moving camera and where the acquired scene contains moving agents or objects.
  • Required also is temporal consistency, for the extraction of background/foreground to be stable over time.
  • temporal filtering can be used for enhanced temporal consistency.
  • background/foreground extraction Based on determinations of the speed with which the background moves due to camera motion, and the speed of the skier with respect to the camera, background/foreground extraction generates a weight layer which differentiates between those pixels which follow the camera and those which do not. The weight layer will then be used to generate an alpha layer for the final composite sequence.
  • Temporal alignment involves the selection of corresponding frames in the sequences, according to a chosen criterion. Typically, in sports racing competitions, this is the time code of each sequence delivered by the timing system, e.g. to select the frames corresponding to the start of the race. Other possible time criteria are the time corresponding to a designated spatial location such as a gate or jump entry, for example.
  • Spatial alignment is effected by choosing a reference coordinate system for each frame and by determining a camera coordinate transformation between the reference system and the corresponding frame of each sequence. Such a determination may involve position estimating based on image contents and/or measuring, e.g. using the global positioning system (GPS). A determination may be unnecessary when camera data such as camera position, viewing direction and focal length are recorded along with the video sequence.
  • the reference coordinate system is chosen as one of the given sequences, namely the one to be used for the composite sequence.
  • spatial alignment may be on a single-frame or multiple-frame basis.
  • alignment uses one frame from each of the sequences.
  • the method for estimating the camera coordinate transformation needs to be robust.
  • the masks generated in background/foreground extraction can be used.
  • temporal filtering can be used for enhancing the temporal consistency of the estimation process.
  • This technique allows free choice of the field of view of every frame in the scene, in contrast to the single-frame technique where the field of view has to be chosen as the one of the reference frame.
  • the field and/or angle of view of the composite image can be chosen such that all competitors are visible.
  • Video displays are of interest for TV broadcasting as well as for on-demand services, for example.
  • the latter may allow for user interaction, e.g in choosing camera angle, zooming, choice of viewpoint, and choice of contestants whose performance a user may wish to compare.
  • Such a service may be provided by an Internet-based sports site and may include enhancements such as graphing of virtual trajectories, marking of spatial locations for performance comparison among contestants, and stroboscoping of fast events which involves displaying an event as "frozen" in space, by a series of overlapping snapshots taken at short intervals of time.
  • a composite video sequence made in accordance with the invention is apparent from Fig. 6, namely for determining differential time between two runners at any desired location of a race. This involves simple counting of the number of frames in the sequence between the two runners passing the location, and multiplying by the time interval between frames.
  • a composite sequence can be broadcast over existing facilities such as network, cable and satellite TV, and as video on the Internet, for example.
  • Such sequences can be offered as on-demand services, e.g. on a channel separate from a strictly real-time main channel.
  • a composite video sequence can be included as a portion of a regular channel, displayed as a corner portion, for example.
  • composite video sequences can be used in sports training and coaching. And, aside from sports applications, there are potential industrial applications such as car crash analysis, for example. It is understood that composite sequences may be higher-dimensional, such as composite stereo video sequences.
  • one of the given sequences is an audio sequence to be synchronized with a video sequence.
  • the technique can be used to generate a voice-over or "lip-synch" sequence of actor A speaking or singing with the voice of B.
  • dynamic programming techniques can be used for synchronization.
  • the spatio-temporal realignment method can be applied in the biomedical field as well. For example, after orthopedic surgery, it is important to monitor the progress of a patient's recovery. This can be done by comparing specified movements of the patient over a period of time. In accordance with an aspect of the invention, such a comparison can be made very accurately, by synchronizing start and end of the movement, and aligning the limbs to be monitored in two or more video sequences.
  • Another application is in car crash analysis. The technique can be used for precisely comparing the deformation of different cars crashed in similar situations, to ascertain the extent of the difference. Further in car crash analysis, it is important to compare effects on crash dummies. Again, in two crashes with the same type of car, one can precisely compare how the dummies are affected depending on configuration, e.g. of safety belts.
  • Temporal indexing of video sequences can be performed in different ways, depending on their content and the blending goal.
  • video sequences can be temporally synchronized according to their timing information, usually a chronometric counter which is reset at the start of each run. After synchronization, spatial alignment of the sequences results in a single video sequence showing at each instant in time the relative position of contestants, thus highlighting trajectory and speed differences between contestants.
  • video sequences can be aligned with respect to spatially derived information, e.g. the instant of a competitor's crossing a pre- selected line in the environment as in high-jump and other track-and-field events.
  • Some sequences may consist only of static background information, needing no temporal synchronization but only spatial indexing for use in blending.
  • background information may be provided by a camera scan of the empty race field, taken prior to the sport event.
  • Spatial indexing of video sequences can be effected by hardware or software.
  • Hardware can include camera sensors which measure its instantaneous physical status and provide corresponding data along with the visual information in recorded fields.
  • Software can provide for robust estimation techniques for inferring the relative displacement of the camera, from a sequence of recorded visual fields.
  • Fig. 7 illustrates how a database of sequences registered on a manifold can be constructed.
  • a pre-processor 71 assembles video information 72 and camera parameters 73 if available. The assembled data and, unless a flag 74 is set to identify the video information 72 as static background information, synchronization information 75 are furnished to a manifold projection module 76 which produces a database update 77 for the database 78.
  • Blending of temporally and spatially indexed sequences can be effected as follows: First the two or more original sequences selected for blending are analyzed in terms of their spatial indexing. From the indexing information, an extended background is reconstructed using the global indexing on the manifold of the background sequences. The original sequences are now synchronized, and a new target sequence is formed by spatially aligning the original, synchronized sequences with their common background on a field-by-field basis. Such spatial alignment can be effected by a robust camera motion estimation technique, e.g. including explicit indexing of each field on the global manifold.
  • the final target sequence is obtained by blending the visual information of the original, spatio-temporally aligned sequences and of the background information on a field-by-field basis over a suitably defined viewing area.
  • the viewing area can be determined automatically and so as to ensure that all objects of interest in the original sequences appear in the same field of view in the blended sequence. Special viewing needs can be accommodated by operator-controlled re-framing, for example.
  • Blending can be effected by reference to an established alpha layer, i.e. a relative weight array prescribing the relative contributions of each original field to the blended field.
  • the alpha layer is determined based on information about the original sequences, e.g. the location of active foreground areas obtained by a robust foreground/background extraction method.
  • Series of target frames can be user-definable for effecting various video processing manipulations, e.g. slow motion and re-framing.
  • video processing manipulations e.g. slow motion and re-framing.
  • two ski race sequences which have little or no overlap with each other but share a common background can be integrated into a common field of view, based on background information.
  • enhancements can be included such as virtual trajectories or reference lines embedded in the background information, so that such trajectories or lines automatically will be properly positioned when the background is combined with an event sequence.
  • enhancements can be included such as virtual trajectories or reference lines embedded in the background information, so that such trajectories or lines automatically will be properly positioned when the background is combined with an event sequence.
  • stroboscopic still images including several images of an athlete in the course of his trajectory, e.g. in a broad-jump event, by using background information and blending of camera fields selected according to their time index from the beginning of the jump.
  • Stills as captured by a video camera will be called video fields.
  • the image captured in each field relates to the world around the camera via a set of parameters, including the geographical coordinates of the camera, three direction angles formed by the camera with respect to a chosen Cartesian reference system and usually called pan, tilt and roll angles, the camera aperture or zoom, and several physical parameters related to camera components, e.g. the lens, photosensitive elements and shutter speed.
  • Some of these camera parameters are fixed, while others vary under control by the cameraman in the course of a shoot.
  • a camera can furnish such parameters directly; otherwise they can be estimated computationally on the basis of motion characteristics of a recorded video sequence, using one of a number of known robust motion estimation techniques and mapping to the global reference system.
  • the camera-movement parameter values are delimited by mechanical camera limitations.
  • the parameter ranges define a multidimensional manifold which can be used to spatially index a video sequence produced by the camera.
  • a suitable projection surface or manifold e.g. a cylindrical or spherical surface centered at the camera location.
  • a region of the surface then represents a view from the camera position.
  • Fig. 9 illustrates indexing on a cylinder, of two video sequences of frame regions 91, . . ., 91' and 92, . . ., 92'. Shown further is a region 93 which corresponds to a desired view for combining the video sequences.
  • indexing on a cylinder involves recording azimuth and elevation. For indexing on a sphere, azimuth and declination can be used.
  • a temporal index can be associated for temporal alignment or synchronization between different sequences representing different events in the same environment.
  • the index can be based on choice of a suitable starting instant. A sequence which only represents background will not require temporal indexing for blending with an action sequence.
  • Background sequences can be spatially indexed on a suitably dimensioned manifold which can be taken as the reference system for the camera. Any other sequence, of an event against the background, can be projected on the same manifold to obtain a sequence of manifold coordinates which can be brought in correspondence with a series of fields in the indexed background sequences.
  • the visual information in this series of background fields can be stitched together using a robust image stitching and mosaicing technique, defining an extended background image for the sequence.
  • the width and extent of the background image can be modified readily.
  • Image processing techniques can be applied to the background information prior to forming a target sequence, including the drawing of virtual trajectories and targets, color coding of image areas, and image enhancement among others.
  • a new target sequence is formed, composed of a number of contiguous target fields. This is effected by selecting, for each target field, a number of fields in the original sequences according to a chosen criterion.
  • the visual information in the original fields is then suitably warped and blended with the visual information of their common reconstructed background to form each target field.
  • Spatial alignment can be effected by aligning the selected original fields with the reconstructed background. This operation relies on robust camera motion estimation software and/or hardware and may employ the same spatial indexing techniques as described above.
  • a reference system is selected for the common background, serving as multidimensional manifold as described above, and the original sequence frames are mapped onto this reference system by means of suitable warping techniques.
  • Synchronization is achieved by selecting those fields of the original sequences whose time indices, match a desired target time index. Suitable criteria include: (i) Spatio-temporal realignment of two or more sequences with extended background reconstruction, with one of the original sequences chosen as a reference sequence. For each field in the reference sequence the target indices are computed by selecting the fields in the other original sequences so that their time indices match. The selected fields are then spatially aligned with their reconstructed background visual information.
  • blending can be effected by processing as follows:
  • Object Motion Estimation For each original sequence a background/foreground extraction is performed, using a robust background-foreground estimation method. Robustness is called for here and throughout in the interest of processing image sequences acquired with a moving camera and containing moving persons and/or objects. Similarly called for is temporal consistency, i.e. foreground- background extraction is stable over time. As both the camera and people are moving according to physical properties, e.g. constant speed or acceleration, temporal filtering can be used for improving temporal consistency. Background-foreground extraction is aimed at generating a weight layer for distinguishing the portions of the original fields which follow the camera motion from those which do not. The weight layer will then be used in generating an alpha layer for the final composite sequence. (ii) Selection of the Viewing Area. For each target field, a viewing area is defined on the extended reconstructed background according to a chosen viewing criterion. Suitable criteria include:
  • Frame retrieval modules 83 and 84 furnish frames to respective blending modules 85 and 86 which in turn forward the respective blended background and foreground sequences to a final blending module 87 for blending into the target sequence output.
  • Re-framing can be used to advantage further, to give an impression of zoom-in or zoom-out. This is of particular interest in case of motion along the line of sight, as in ski- jump events, for example.
  • time re-scaling can stretch or compress time in an output video as compared with the original video, linearly or in any desired monotonic fashion. For example, for a more immediate comparison at critical points, of two participants in a triple-jump sports event for example, their videos can be synchronized so that their consecutive touch-downs appear as simultaneous.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Circuits (AREA)
  • Studio Devices (AREA)

Abstract

A partir de deux séquences vidéo (IS1, IS2), il est possible de générer une séquence vidéo composite qui comprend des éléments visuels de chacune desdites séquences, synchronisées de manière adaptée et représentées dans un plan focal choisi (figure 1). Par exemple, à partir de deux séquences vidéo présentant chacune un concurrent différent participant individuellement à la même course de descente, il est possible de générer une représentation composite contenant des éléments de chacune des séquences données dans le but de présenter les concurrents comme s'ils participaient simultanément à la course. La génération d'une représentation composite peut être facilitée si l'on utilise un arrière plan commun sur lequel les objets d'avant plan sont présentés. Il est possible de former la représentation composite en mélangeant de manière appropriée l'avant plan et l'arrière plan et en procédant à une mise à l'échelle et/ou à un recadrage aux fins de l'obtention d'une présentation optimisée.
EP01946591A 2001-06-19 2001-06-19 Procede et systeme de combinaison de sequences video avec alignement spatio-temporel Withdrawn EP1449357A4 (fr)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2001/019741 WO2002104009A1 (fr) 2001-06-19 2001-06-19 Procede et systeme de combinaison de sequences video avec alignement spatio-temporel

Publications (2)

Publication Number Publication Date
EP1449357A1 true EP1449357A1 (fr) 2004-08-25
EP1449357A4 EP1449357A4 (fr) 2006-10-04

Family

ID=21742661

Family Applications (1)

Application Number Title Priority Date Filing Date
EP01946591A Withdrawn EP1449357A4 (fr) 2001-06-19 2001-06-19 Procede et systeme de combinaison de sequences video avec alignement spatio-temporel

Country Status (2)

Country Link
EP (1) EP1449357A4 (fr)
WO (1) WO2002104009A1 (fr)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1247255A4 (fr) 1999-11-24 2007-04-25 Dartfish Sa Coordination et combinaison de sequences video avec normalisation spatiale et temporelle
US7268804B2 (en) * 2003-04-04 2007-09-11 Stmicroelectronics, Inc. Compound camera and method for synthesizing a virtual image from multiple input images
DE102004042166A1 (de) * 2004-08-31 2006-03-16 MAX-PLANCK-Gesellschaft zur Förderung der Wissenschaften e.V. Bildverarbeitungseinrichtung und entsprechendes Betriebsverfahren
US8396321B1 (en) 2007-04-25 2013-03-12 Marvell International Ltd. Method and apparatus for processing image data from a primary sensor and a secondary sensor
DE102009007639B4 (de) * 2009-02-05 2011-05-26 Kurt Wallerstorfer Vorrichtung zur Bildaufzeichnung eines Sportlers auf einer Rennstrecke
US8963949B2 (en) 2009-04-22 2015-02-24 Qualcomm Incorporated Image selection and combination method and device
BR112013006763B1 (pt) * 2010-09-20 2022-10-18 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V Método para diferenciar entre o fundo e o primeiro plano de um cenário em imagens gravadas por uma câmara eletrônica, método para substituir um fundo de uma imagem de um cenário e sistema de visualização eletrônica

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0669758A1 (fr) * 1993-09-06 1995-08-30 Kabushiki Kaisha Oh-Yoh Keisoku Kenkyusho Processeur d'images variant dans le temps et dispositif d'affichage
WO1999037088A1 (fr) * 1998-01-16 1999-07-22 Ecole Polytechnique Federale De Lausanne (Epfl) Procedes et systemes de combinaison de sequences video a alignement spatio-temporel
WO2001039130A1 (fr) * 1999-11-24 2001-05-31 Dartfish Ltd. Coordination et combinaison de sequences video avec normalisation spatiale et temporelle

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3622618C1 (de) * 1986-07-05 1987-05-21 Willy Bogner Verfahren zur gleichzeitigen Darstellung mindestens zweier zeitlich nacheinander ablaufender Ereignisse im TV sowie Vorrichtung zur Durchfuehrung dieses Verfahrens
GB8825160D0 (en) * 1988-10-27 1988-11-30 Strong S D Production of video recordings
AT400087B (de) * 1990-11-08 1995-09-25 Bauer Fritz Verfahren zur aufnahme von nacheinander erfolgenden ähnlichen bewegungsafbläufen mittels eines bildaufnahmegerätes

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0669758A1 (fr) * 1993-09-06 1995-08-30 Kabushiki Kaisha Oh-Yoh Keisoku Kenkyusho Processeur d'images variant dans le temps et dispositif d'affichage
WO1999037088A1 (fr) * 1998-01-16 1999-07-22 Ecole Polytechnique Federale De Lausanne (Epfl) Procedes et systemes de combinaison de sequences video a alignement spatio-temporel
WO2001039130A1 (fr) * 1999-11-24 2001-05-31 Dartfish Ltd. Coordination et combinaison de sequences video avec normalisation spatiale et temporelle

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of WO02104009A1 *

Also Published As

Publication number Publication date
EP1449357A4 (fr) 2006-10-04
WO2002104009A1 (fr) 2002-12-27

Similar Documents

Publication Publication Date Title
EP1055324B1 (fr) Procedes et systemes de combinaison de sequences video a alignement spatio-temporel
US8675021B2 (en) Coordination and combination of video sequences with spatial and temporal normalization
US7843510B1 (en) Method and system for combining video sequences with spatio-temporal alignment
EP1287518B1 (fr) Stroboscopie automatisee de sequences video
Yow et al. Analysis and presentation of soccer highlights from digital video
US6072504A (en) Method and apparatus for tracking, storing, and synthesizing an animated version of object motion
US7868914B2 (en) Video event statistic tracking system
CN108886583A (zh) 用于通过数据网络向多个用户提供虚拟的摇摄-倾斜-缩放，ptz，视频功能的系统和方法
JP2009505553A (ja) ビデオストリームへの視覚効果の挿入を管理するためのシステムおよび方法
US9087380B2 (en) Method and system for creating event data and making same available to be served
Cavallaro et al. Augmenting live broadcast sports with 3D tracking information
EP1128668A2 (fr) Méthode et appareil pour améliorer la transmission d'évènements en direct par la superposition d'animations basées sur des évènements réels
Pidaparthy et al. Keep your eye on the puck: Automatic hockey videography
RU2602792C2 (ru) Сравнение движущихся объектов на основе вектора движения
WO2002104009A1 (fr) Procede et systeme de combinaison de sequences video avec alignement spatio-temporel
Inamoto et al. Free viewpoint video synthesis and presentation from multiple sporting videos
CN114302234B (zh) 一种空中技巧快速包装方法
JP2009519539A (ja) イベントデータを作成し、これをサービス提供可能な状態にするための方法及びシステム
AU2003268578B2 (en) Method and System for Combining Video Sequences With Spatio-temporal Alignment
WO2005006773A1 (fr) Procede et systeme de combinaison de sequences video avec alignement spatio-temporel
MXPA00007221A (en) Method and system for combining video sequences with spacio-temporal alignment
Grimson et al. Immersive Sporting Events
KR20080097403A (ko) 이벤트 데이터를 생성하고 이벤트 데이터가 제공될 수 있게하는 방법 및 시스템
KR20070008687A (ko) 출력 이미지 생성하기

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20040119

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR

AX Request for extension of the european patent

Extension state: AL LT LV MK RO SI

REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1069947

Country of ref document: HK

A4 Supplementary search report drawn up and despatched

Effective date: 20060901

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20100105

REG Reference to a national code

Ref country code: HK

Ref legal event code: WD

Ref document number: 1069947

Country of ref document: HK