WO2002003707A2 - Codage d'images video base modeles - Google Patents

Codage d'images video base modeles Download PDF

Info

Publication number
WO2002003707A2
WO2002003707A2 PCT/US2001/019956 US0119956W WO0203707A2 WO 2002003707 A2 WO2002003707 A2 WO 2002003707A2 US 0119956 W US0119956 W US 0119956W WO 0203707 A2 WO0203707 A2 WO 0203707A2
Authority
WO
WIPO (PCT)
Prior art keywords
human face
images
sequence
shoulders
movement
Prior art date
Application number
PCT/US2001/019956
Other languages
English (en)
Other versions
WO2002003707A3 (fr
Inventor
Somnath Sentupta
Rama Suryanarayana
Original Assignee
Intel Corporation
Indian Institute Of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Intel Corporation, Indian Institute Of Technology filed Critical Intel Corporation
Priority to AU2001271382A priority Critical patent/AU2001271382A1/en
Publication of WO2002003707A2 publication Critical patent/WO2002003707A2/fr
Publication of WO2002003707A3 publication Critical patent/WO2002003707A3/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/001—Model-based coding, e.g. wire frame

Definitions

  • the present disclosure is related to video coding of images and, more particularly,
  • video image coding is typically employed due, at least in part,
  • Image coding for
  • visual communications is one example where improved coding in terms of compression
  • Video image coding is
  • FIG. 1 is a block diagram illustrating an embodiment of a technique for model-
  • FIG. 2 is a schematic diagram illustrating a geometric approximation that may be
  • FIG. 3 is the schematic diagram of FIG. 2 showing selected points and global
  • FIG. 1 is a block diagram illustrating an embodiment 100 of model-based video
  • Another operation for this particular embodiment includes global motion
  • Global motion estimation is also employed to update
  • the 3D model is also employed, illustrated by block 270, and is updated, illustrated
  • block 260 just as block 130 updates block 140 on the transmitting side or near end of
  • block 220 provides estimated global motion parameters that may be transmitted across
  • transmission medium 200 to be applied for image synthesis such as illustrated by block
  • model-based video image coding is
  • a third aspect of this particular embodiment is local processing, as illustrated by
  • local processing is to capture local motion or movement of the human face, such as
  • portions of the human face are transmitted across transmission medium 200 for decoding
  • processing illustrated by block 250.
  • processing illustrated by block 250.
  • model- based image video coding such as for images of a human face
  • model is applied to the human face for at least one of the images or frames in the
  • Facial expression coding is applied to the sequence of
  • facial expression coding is applied to estimate movement of the eyes
  • human face may be reconstructed by decoding selected windows or regions of the
  • the image background in this embodiment, including the
  • the 3D model employed is based at least in part on facial
  • the edges are detected using the Sobel operator, although
  • the face contour is represented by a set of
  • the top part of the human head is
  • part is represented as a semi-ellipse, as illustrated in FIG. 2, for example.
  • FIG. 2 For example.
  • the eyes are represented by two ellipses.
  • the parameters of these ellipses are represented by two ellipses.
  • an ellipse represents the mouth or lips and a fit is determined by
  • geometric shapes represent relevant facial features as
  • geometric shapes provide rough contours for application of an active
  • a model may be fit to the scene face. Specifically, in this
  • global and local transformations may be applied to the model so that it fits
  • point 1 is the left corner point of the left eye
  • point 2 is the right corner point and the right eye contour and point 3 is the
  • Point 4 is the top most point of the mouth contour
  • points 5 and 6 are points of intersection of the line passing through points 1 and 2 (the
  • the global transformation involves a translation and rotation in three dimensions
  • the scale factor S ⁇ is estimated from a ratio of the distance from point 3 to point 4 between the model and the measured features.
  • a single or first image is
  • the position of the face in the projection is estimated by matching
  • Local transformation refines the model to the face in the scene. This may be
  • the facial action coding system FACS
  • FACS describes a set of basic actions (called action units or AUs) performable
  • the facial expression may then be parameterized as a linear combination
  • the decoder receiving the signal information from
  • the facial expression produced may not in
  • Wire-Frame Adaptation including Photometric Effects for Model-Based Coding of Facial Image Sequences," by G. Bozdagi, A.M. Tekalp and L. Onural, IEEE Transactions on
  • the "clipped" regions of the image of the face are transmitted
  • the decoder transforms the
  • the facial expressive segments are "pasted" into the face.
  • the facial expression As previously indicated, in this particular embodiment, the facial expression
  • encapsulate eyes for example, are chosen based upon prior knowledge of vertices
  • DCT may be applied to these windows or regions of the human face for coding
  • the error is coded because this coding may typically be accomplished satisfactorily with only a few
  • the 2D model is employed to fit the shoulder image, similar to the approach described
  • [T] is [T ⁇ T ⁇ Tz] ⁇ , the values in the forgoing matrix
  • [r nim ] is a 3-by-3 matrix of angular velocities
  • wire frame model is collapsed onto the image frame by means of orthographic projection
  • the initial wire frame model is rotated and translated according to
  • the first frame is mapped into the corresponding triangle by means of decimation or
  • one embodiment may be in hardware, whereas another
  • embodiment may be in software. Likewise, an embodiment may be in firmware, or any combination thereof.
  • one embodiment may comprise an article
  • Such a storage medium such as, for example, a CD-ROM,
  • model-based video image in accordance with the invention, such as, for

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

L'invention concerne dans l'un de ses modes de réalisation un procédé de codage d'une image du visage et des épaules d'une personne. L'invention concerne également plusieurs approches d'exécution ou de réalisation de ce codage.
PCT/US2001/019956 2000-06-30 2001-06-21 Codage d'images video base modeles WO2002003707A2 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2001271382A AU2001271382A1 (en) 2000-06-30 2001-06-21 Model-based video image coding

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US60899100A 2000-06-30 2000-06-30
US09/608,991 2000-06-30

Publications (2)

Publication Number Publication Date
WO2002003707A2 true WO2002003707A2 (fr) 2002-01-10
WO2002003707A3 WO2002003707A3 (fr) 2002-05-23

Family

ID=24438919

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2001/019956 WO2002003707A2 (fr) 2000-06-30 2001-06-21 Codage d'images video base modeles

Country Status (2)

Country Link
AU (1) AU2001271382A1 (fr)
WO (1) WO2002003707A2 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115170388A (zh) * 2022-07-28 2022-10-11 西南大学 人物线稿生成方法、装置、设备及介质

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5978030A (en) * 1995-03-18 1999-11-02 Daewoo Electronics Co., Ltd. Method and apparatus for encoding a video signal using feature point based motion estimation

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5978030A (en) * 1995-03-18 1999-11-02 Daewoo Electronics Co., Ltd. Method and apparatus for encoding a video signal using feature point based motion estimation

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
BOVE V M: "OBJECT-ORIENTED TELEVISION" SMPTE JOURNAL, SMPTE INC. SCARSDALE, N.Y, US, vol. 104, no. 12, 1 December 1995 (1995-12-01), pages 803-807, XP000543848 ISSN: 0036-1682 *
DEFU CAI ET AL: "Several key problems in model-based image sequence compression by using interframe AUs correlation" PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP) AUSTIN, NOV. 13 - 16, 1994, LOS ALAMITOS, IEEE COMP. SOC. PRESS, US, vol. 3 CONF. 1, 13 November 1994 (1994-11-13), pages 409-413, XP010146202 ISBN: 0-8186-6952-7 *
EISERT P ET AL: "Rate-distortion-efficient video compression using a 3-D head model" IMAGE PROCESSING, 1999. ICIP 99. PROCEEDINGS. 1999 INTERNATIONAL CONFERENCE ON KOBE, JAPAN 24-28 OCT. 1999, PISCATAWAY, NJ, USA,IEEE, US, 24 October 1999 (1999-10-24), pages 217-221, XP010368702 ISBN: 0-7803-5467-2 *
SCHRODER K ET AL: "Combined description of shape and motion in an object based coding scheme using curved triangles" PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING. (ICIP). WASHINGTON, OCT. 23 - 26, 1995, LOS ALAMITOS, IEEE COMP. SOC. PRESS, US, vol. 3, 23 October 1995 (1995-10-23), pages 390-393, XP010197107 ISBN: 0-7803-3122-2 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115170388A (zh) * 2022-07-28 2022-10-11 西南大学 人物线稿生成方法、装置、设备及介质

Also Published As

Publication number Publication date
WO2002003707A3 (fr) 2002-05-23
AU2001271382A1 (en) 2002-01-14

Similar Documents

Publication Publication Date Title
Peng et al. Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans
Li et al. Image sequence coding at very low bit rates: a review
JP4335449B2 (ja) 表情の３ｄジオメトリ、色、およびシェーディングを取り込んで表すための方法およびシステム
Eisert et al. Analyzing facial expressions for virtual conferencing
Guenter et al. Making faces
US6600786B1 (en) Method and apparatus for efficient video processing
Malassiotis et al. Object-based coding of stereo image sequences using three-dimensional models
JPH08205149A (ja) ビデオ信号の符号化方法
Buck et al. Model-based image sequence coding
Malassiotis et al. Coding of video-conference stereo image sequences using 3D models
Eisert et al. Facial expression analysis for model-based coding of video sequences
WO2000064167A1 (fr) Procede et appareil permettant un traitement video efficace
WO2002003707A2 (fr) Codage d'images video base modeles
Pei et al. Global motion estimation in model-based image coding by tracking three-dimensional contour feature points
Girod Image sequence coding using 3D scene models
JPH0714208B2 (ja) 動画像伝送方式
GB2320839A (en) Encoding facial movement in a 3D model-based image coding system
US7046728B1 (en) Method of video coding the movement of a human face from a sequence of images
Ström Model-based head tracking and coding
Huang et al. 3-D face modeling and its applications
Kompatsiaris et al. Flexible 3D motion estimation and tracking for multiview image sequence coding
JPH08149461A (ja) 動画像処理装置とその方法
Tekalp et al. Two-versus three-dimensional object-based video compression
US20230300336A1 (en) V3C Patch Remeshing For Dynamic Mesh Coding
Huang et al. 3D model-based video coding: Computer vision meets computer graphics

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
AK Designated states

Kind code of ref document: A3

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A3

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase in:

Ref country code: JP