WO2009073415A3 - Temporal image prediction - Google Patents

Temporal image prediction Download PDF

Info

Publication number
WO2009073415A3
WO2009073415A3 PCT/US2008/084395 US2008084395W WO2009073415A3 WO 2009073415 A3 WO2009073415 A3 WO 2009073415A3 US 2008084395 W US2008084395 W US 2008084395W WO 2009073415 A3 WO2009073415 A3 WO 2009073415A3
Authority
WO
WIPO (PCT)
Prior art keywords
frame
frames
temporal image
motion estimate
reference frame
Prior art date
Application number
PCT/US2008/084395
Other languages
French (fr)
Other versions
WO2009073415A2 (en
WO2009073415A4 (en
Inventor
Christopher J Orlick
Richard Webb
Scott Miller
Jerome D Shields
Original Assignee
Dolby Lab Licensing Corp
Christopher J Orlick
Richard Webb
Scott Miller
Jerome D Shields
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dolby Lab Licensing Corp, Christopher J Orlick, Richard Webb, Scott Miller, Jerome D Shields filed Critical Dolby Lab Licensing Corp
Priority to US12/745,502 priority Critical patent/US8687693B2/en
Priority to EP08856921A priority patent/EP2223530A2/en
Priority to JP2010536084A priority patent/JP2011505756A/en
Priority to CN200880118271XA priority patent/CN101878650B/en
Publication of WO2009073415A2 publication Critical patent/WO2009073415A2/en
Publication of WO2009073415A3 publication Critical patent/WO2009073415A3/en
Publication of WO2009073415A4 publication Critical patent/WO2009073415A4/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/223—Analysis of motion using block-matching
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/573—Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0135—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes
    • H04N7/014—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes involving the use of motion vectors

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

For frames sequentially functioning as a reference frame in a video sequence frame set, a motion estimate set is accessed. One motion estimate characterizes motion associated with pixels of each region of the reference frame in relation to regions of one frame of the set of frames, which is temporally displaced in time with respect to other frames in the set of frames. An additional motion estimate characterizes motion associated with pixels of each reference frame region in relation to a second frame of the set, which is temporally displaced from the one frame and other frames of the frame set. A temporal image prediction set, corresponding to the first and additional motion estimate, is predicted, based on an alignment of the reference frame regions over the frame set. The temporal image predictions are blended and a temporal predictor is generated over the frame set based on the blending.
PCT/US2008/084395 2007-11-30 2008-11-21 Temporal image prediction WO2009073415A2 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US12/745,502 US8687693B2 (en) 2007-11-30 2008-11-21 Temporal image prediction
EP08856921A EP2223530A2 (en) 2007-11-30 2008-11-21 Temporal image prediction
JP2010536084A JP2011505756A (en) 2007-11-30 2008-11-21 Temporal image detection
CN200880118271XA CN101878650B (en) 2007-11-30 2008-11-21 Temporal image prediction method and system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US99167707P 2007-11-30 2007-11-30
US60/991,677 2007-11-30

Publications (3)

Publication Number Publication Date
WO2009073415A2 WO2009073415A2 (en) 2009-06-11
WO2009073415A3 true WO2009073415A3 (en) 2009-07-30
WO2009073415A4 WO2009073415A4 (en) 2009-09-17

Family

ID=40524522

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2008/084395 WO2009073415A2 (en) 2007-11-30 2008-11-21 Temporal image prediction

Country Status (5)

Country Link
US (1) US8687693B2 (en)
EP (1) EP2223530A2 (en)
JP (1) JP2011505756A (en)
CN (1) CN101878650B (en)
WO (1) WO2009073415A2 (en)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011050998A1 (en) * 2009-10-29 2011-05-05 Thomas Sikora Method and device for processing a video sequence
US8588535B2 (en) 2010-09-15 2013-11-19 Sharp Laboratories Of America, Inc. Methods and systems for estimation of compression noise
US8600188B2 (en) 2010-09-15 2013-12-03 Sharp Laboratories Of America, Inc. Methods and systems for noise reduction and image enhancement
US8538193B2 (en) * 2010-09-28 2013-09-17 Sharp Laboratories Of America, Inc. Methods and systems for image enhancement and estimation of compression noise
US8532429B2 (en) 2010-09-28 2013-09-10 Sharp Laboratories Of America, Inc. Methods and systems for noise reduction and image enhancement involving selection of noise-control parameter
US8711940B2 (en) * 2010-11-29 2014-04-29 Mediatek Inc. Method and apparatus of motion vector prediction with extended motion vector predictor
KR20130050403A (en) * 2011-11-07 2013-05-16 오수미 Method for generating rrconstructed block in inter prediction mode
WO2013161561A1 (en) * 2012-04-27 2013-10-31 株式会社安川電機 Time chart creation device, controller, machine element control system, time chart creation method, program and information storage medium
KR20140105103A (en) * 2013-02-21 2014-09-01 삼성전자주식회사 Method, apparatus and medical imaging system for tracking motion of organ
KR102453803B1 (en) * 2015-09-10 2022-10-12 삼성전자주식회사 Method and apparatus for processing image
US10257449B2 (en) * 2016-01-05 2019-04-09 Nvidia Corporation Pre-processing for video noise reduction
GB2557622A (en) * 2016-12-12 2018-06-27 V Nova Int Ltd Motion compensation techniques for video
JP6326180B1 (en) * 2017-05-30 2018-05-16 オリンパス株式会社 Image processing device
CN108549069A (en) * 2018-03-08 2018-09-18 哈尔滨工程大学 A kind of warping under imperfect waveguid converts modification method
US11481979B2 (en) * 2018-04-23 2022-10-25 The Regents Of The University Of Colorado, A Body Corporate Mobile and augmented reality based depth and thermal fusion scan
US11640668B2 (en) * 2021-06-10 2023-05-02 Qualcomm Incorporated Volumetric sampling with correlative characterization for dense estimation

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5057921A (en) * 1989-03-31 1991-10-15 Thomson Consumer Electronics Process and device for temporal image interpolation, with corrected movement compensation
US20040252230A1 (en) * 2003-06-13 2004-12-16 Microsoft Corporation Increasing motion smoothness using frame interpolation with motion analysis
WO2005004492A2 (en) * 2003-06-25 2005-01-13 Thomson Licensing S.A. Method and apparatus for weighted prediction estimation using a displaced frame differential
EP1855474A1 (en) * 2006-05-12 2007-11-14 Sony Deutschland Gmbh Method for generating an interpolated image between two images of an input image sequence

Family Cites Families (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2311184A (en) 1996-03-13 1997-09-17 Innovision Plc Motion vector field error estimation
EP0817499A3 (en) 1996-06-28 2002-05-22 Matsushita Electric Industrial Co., Ltd. Image coding method using extrapolated pixels in insignificant areas of blocks
US6249318B1 (en) 1997-09-12 2001-06-19 8×8, Inc. Video coding/decoding arrangement and method therefor
JP3855522B2 (en) 1999-02-23 2006-12-13 松下電器産業株式会社 Video converter
US7412002B2 (en) 2000-04-28 2008-08-12 Texas Instruments Incorporated Image preprocessing with selective lowpass filtering from motion compensation
JP2002281505A (en) 2001-03-16 2002-09-27 Toshiba Corp Moving image compressor, data preparing method for cut detection for the same, cut position data preparing method, cut detector and cut detecting method for the same
EP1404136B1 (en) 2001-06-29 2018-04-04 NTT DoCoMo, Inc. Image encoder, image decoder, image encoding method, and image decoding method
JP4163618B2 (en) 2001-08-28 2008-10-08 株式会社エヌ・ティ・ティ・ドコモ Video encoding / transmission system, video encoding / transmission method, encoding apparatus, decoding apparatus, encoding method, decoding method, and program suitable for use in the same
CN1744719A (en) 2002-04-09 2006-03-08 Lg电子株式会社 Block prediction method
KR100508798B1 (en) 2002-04-09 2005-08-19 엘지전자 주식회사 Method for predicting bi-predictive block
JP3807342B2 (en) 2002-04-25 2006-08-09 三菱電機株式会社 Digital signal encoding apparatus, digital signal decoding apparatus, digital signal arithmetic encoding method, and digital signal arithmetic decoding method
US20040190615A1 (en) 2002-05-22 2004-09-30 Kiyofumi Abe Moving image encoding method, moving image decoding method, and data recording medium
US20040001546A1 (en) 2002-06-03 2004-01-01 Alexandros Tourapis Spatiotemporal prediction for bidirectionally predictive (B) pictures and motion vector prediction for multi-picture reference motion compensation
US20060098886A1 (en) 2003-01-10 2006-05-11 Koninklijke Philips Electronics N.V. Efficient predictive image parameter estimation
KR20060105409A (en) 2005-04-01 2006-10-11 엘지전자 주식회사 Method for scalably encoding and decoding video signal
US7469069B2 (en) 2003-05-16 2008-12-23 Samsung Electronics Co., Ltd. Method and apparatus for encoding/decoding image using image residue prediction
JP4815107B2 (en) 2003-07-16 2011-11-16 三星電子株式会社 Lossless video encoding / decoding method and apparatus using color plane prediction
WO2005022919A1 (en) 2003-08-26 2005-03-10 Thomson Licensing S.A. Method and apparatus for decoding hybrid intra-inter coded blocks
US7894526B2 (en) 2004-02-27 2011-02-22 Panasonic Corporation Motion estimation method and moving picture coding method
EP1617672A1 (en) * 2004-07-13 2006-01-18 Matsushita Electric Industrial Co., Ltd. Motion estimator/compensator including a 16-bit 1/8 pel interpolation filter
KR100888963B1 (en) 2004-12-06 2009-03-17 엘지전자 주식회사 Method for scalably encoding and decoding video signal
KR100723403B1 (en) 2005-02-28 2007-05-30 삼성전자주식회사 A prediction image generating method and apparatus using using single coding mode among color components, and an image and video encoding/decoding method and apparatus using it
KR100703770B1 (en) 2005-03-25 2007-04-06 삼성전자주식회사 Video coding and decoding using weighted prediction, and apparatus for the same
FR2889004B1 (en) 2005-07-22 2007-08-24 Canon Kk METHOD AND DEVICE FOR PROCESSING A SEQUENCE OF DIGITAL IMAGES WITH SPATIAL SCALABILITY OR QUALITY
JP4236654B2 (en) 2005-09-13 2009-03-11 株式会社東芝 Video encoding apparatus and method
US8059719B2 (en) 2005-09-16 2011-11-15 Sony Corporation Adaptive area of influence filter
US7894522B2 (en) 2005-09-16 2011-02-22 Sony Corporation Classified filtering for temporal prediction
US7894527B2 (en) 2005-09-16 2011-02-22 Sony Corporation Multi-stage linked process for adaptive motion vector sampling in video compression
US7957466B2 (en) 2005-09-16 2011-06-07 Sony Corporation Adaptive area of influence filter for moving object boundaries
WO2007044556A2 (en) 2005-10-07 2007-04-19 Innovation Management Sciences, L.L.C. Method and apparatus for scalable video decoder using an enhancement stream
KR100873636B1 (en) 2005-11-14 2008-12-12 삼성전자주식회사 Method and apparatus for encoding/decoding image using single coding mode
FR2894421B1 (en) 2005-12-07 2008-01-18 Canon Kk METHOD AND DEVICE FOR DECODING A VIDEO STREAM CODE FOLLOWING A HIERARCHICAL CODING
EP2595381B1 (en) 2006-04-28 2016-05-18 NTT DoCoMo, Inc. Image predictive decoding device
JP2007300380A (en) * 2006-04-28 2007-11-15 Ntt Docomo Inc Image predictive encoding device, image predictive encoding method, image predictive encoding program, image predictive decoding device, image predictive decoding method, and image predictive decoding program

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5057921A (en) * 1989-03-31 1991-10-15 Thomson Consumer Electronics Process and device for temporal image interpolation, with corrected movement compensation
US20040252230A1 (en) * 2003-06-13 2004-12-16 Microsoft Corporation Increasing motion smoothness using frame interpolation with motion analysis
WO2005004492A2 (en) * 2003-06-25 2005-01-13 Thomson Licensing S.A. Method and apparatus for weighted prediction estimation using a displaced frame differential
EP1855474A1 (en) * 2006-05-12 2007-11-14 Sony Deutschland Gmbh Method for generating an interpolated image between two images of an input image sequence

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
CAROTTI E S G ET AL: "Low-complexity lossless video coding via adaptive spatio-temporal prediction", PROCEEDINGS 2003 INTERNATIONAL CONFERENCE ON IMAGE PROCESSING. ICIP-2003. BARCELONA, SPAIN, SEPT. 14 - 17, 2003; [INTERNATIONAL CONFERENCE ON IMAGE PROCESSING], NEW YORK, NY : IEEE, US, vol. 2, 14 September 2003 (2003-09-14), pages 197 - 200, XP010670274, ISBN: 978-0-7803-7750-9 *
JELENA KOVACEVIC ET AL: "Deinterlacing by Successive Approximation", IEEE TRANSACTIONS ON IMAGE PROCESSING, IEEE SERVICE CENTER, PISCATAWAY, NJ, US, vol. 6, no. 2, 1 February 1997 (1997-02-01), XP011026110, ISSN: 1057-7149 *
JIANNING ZHANG ET AL: "A Novel Spatial-Temporal Position Prediction Motion-Compensated Interpolation for Frame Rate Up-Conversion", ADVANCES IN MULTIMEDIA INFORMATION PROCESSING - PCM 2006 LECTURE NOTES IN COMPUTER SCIENCE;;LNCS, SPRINGER, BERLIN, DE, vol. 4261, 1 January 2006 (2006-01-01), pages 870 - 879, XP019047776, ISBN: 978-3-540-48766-1 *
KOTO S-I ET AL: "Adaptive Bi-predictive video coding using temporal extrapolation", PROCEEDINGS 2003 INTERNATIONAL CONFERENCE ON IMAGE PROCESSING. ICIP-2003. BARCELONA, SPAIN, SEPT. 14 - 17, 2003; [INTERNATIONAL CONFERENCE ON IMAGE PROCESSING], NEW YORK, NY : IEEE, US, vol. 3, 14 September 2003 (2003-09-14), pages 829 - 832, XP010669962, ISBN: 978-0-7803-7750-9 *
TUBARO S ET AL: "MOTION FIELD ESTIMATORS AND THEIR APPLICATION TO IMAGE INTERPOLATION", MOTION ANALYSIS AND IMAGE SEQUENCE PROCESSING, XX, XX, 1 January 1993 (1993-01-01), pages 153 - 187, XP002150854 *

Also Published As

Publication number Publication date
US8687693B2 (en) 2014-04-01
CN101878650A (en) 2010-11-03
CN101878650B (en) 2013-07-10
WO2009073415A2 (en) 2009-06-11
EP2223530A2 (en) 2010-09-01
JP2011505756A (en) 2011-02-24
US20120281758A1 (en) 2012-11-08
WO2009073415A4 (en) 2009-09-17

Similar Documents

Publication Publication Date Title
WO2009073415A3 (en) Temporal image prediction
WO2009045021A3 (en) Method and apparatus for inter prediction encoding/decoding an image using sub-pixel motion estimation
WO2009051668A3 (en) Methods and apparatus for video encoding and decoding geometrically partitioned bi-predictive mode partitions
WO2010041856A3 (en) A method and an apparatus for processing a video signal
WO2012015649A3 (en) Joint coding of bi-directional motion vectors in gpb frames
WO2008057563A3 (en) Advanced deinterlacer for high-definition and standard-definition video
WO2007118197A3 (en) Electronic video image stabilization
WO2008074857A3 (en) Method for decoding a block of a video image
WO2009034492A3 (en) Method, apparatus, and system for line-based motion compensation in video image data
WO2008085533A3 (en) Block information adjustment techniques to reduce artifacts in interpolated video frames
WO2008014288A3 (en) Adaptive video frame interpolation
EP1978750A3 (en) Method and system for processing multiview videos for view synthesis using skip and direct modes
WO2011126309A3 (en) Method and apparatus for video encoding and method and apparatus for video decoding
EP4135329A3 (en) Method for encoding and decoding image information and device using same
WO2008149840A1 (en) Image prediction encoding device, image prediction decoding device, image prediction encoding method, image prediction decoding method, image prediction encoding program, and image prediction decoding program
TW200737987A (en) Adaptive GOP structure in video streaming
MY164266A (en) Method and device for encoding intra prediction mode for image prediction unit, and method and device for decoding intra prediction mode for image prediction unit
EP1793611A3 (en) Method and system for synthesizing multiview videos
WO2008112426A3 (en) System and method for video recommendation based on video frame features
WO2007124360A3 (en) Image stabilization method
WO2012144829A3 (en) Method and apparatus for encoding and decoding motion vector of multi-view video
WO2009089032A3 (en) Methods and apparatus for illumination compensation of intra-predicted video
EP1936994A3 (en) Weighted prediction video encoding
WO2011126278A3 (en) Method and apparatus for encoding and decoding video
TW200631408A (en) Spatio-temporal adaptive video de-interlacing

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 200880118271.X

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08856921

Country of ref document: EP

Kind code of ref document: A2

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2010536084

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2008856921

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 12745502

Country of ref document: US