EP0648052A1 - Verfahren und Vorrichtung zur Bewegungsauswertung mit Blockübereinstimmung - Google Patents

Verfahren und Vorrichtung zur Bewegungsauswertung mit Blockübereinstimmung Download PDF

Info

Publication number
EP0648052A1
EP0648052A1 EP94113494A EP94113494A EP0648052A1 EP 0648052 A1 EP0648052 A1 EP 0648052A1 EP 94113494 A EP94113494 A EP 94113494A EP 94113494 A EP94113494 A EP 94113494A EP 0648052 A1 EP0648052 A1 EP 0648052A1
Authority
EP
European Patent Office
Prior art keywords
block
pixel
motion
vectors
motion vectors
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP94113494A
Other languages
English (en)
French (fr)
Other versions
EP0648052B1 (de
Inventor
Michael Knee
Michel Kerdranvat
Andrew Hackett
Nadine Bolender
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Technicolor SA
Original Assignee
Thomson Multimedia SA
Thomson Consumer Electronics Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Thomson Multimedia SA, Thomson Consumer Electronics Inc filed Critical Thomson Multimedia SA
Priority to EP19940113494 priority Critical patent/EP0648052B1/de
Publication of EP0648052A1 publication Critical patent/EP0648052A1/de
Application granted granted Critical
Publication of EP0648052B1 publication Critical patent/EP0648052B1/de
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/144—Movement detection
    • H04N5/145—Movement estimation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation

Definitions

  • the present invention relates to a method and to an apparatus for motion estimation using block matching.
  • Block matching is well known as a robust and intuitively simple method of motion estimation for television pictures.
  • One important parameter in block matching is the block size. Large blocks give more reliable motion estimation than small blocks, particularly in the presence of noise on the input picture, but they produce a coarser motion vector field. If the goal of the motion estimation is to provide a motion compensated prediction, for example in bit rate reduction applications, a suitable compromise can usually be found, bearing in mind the fact that neither reliability nor a fine motion vector field are necessary conditions for good performance since occasional errors can be tolerated.
  • the inventive solutions are slightly different, but are all based on block matching using large blocks, followed by post-processing in which the boundaries between motion vectors are fixed more precisely or 'localized'.
  • the picture is divided into large rectangular blocks LB (x b , y b ), where x b is a horizontal coordinate increasing from left to right and y b a vertical coordinate increasing from top to bottom of the picture.
  • the block size is supposed to be M pixels by N lines. Typical block sizes are 8x8, 16x8 or 16x16.
  • the block matching may be carried out either on a subsampled image as described in EP93 402059 of the applicant or on the original, and may be either conventional or 'two-sided', wherein the block to be interpolated is matched with a previous and a subsequent picture, as described in EP93 402187 of the applicant.
  • the block matching process yields an error value E(v x ,v y ) for each candidate motion vector (v x ,v y ).
  • the motion vector chosen for the block is the one for which the error is a minimum.
  • the aim of the localization process is to produce a separate motion vector for each pixel.
  • the motion vector will be one of four possibilities: the vector calculated for the block containing the pixel and the vectors of the nearest blocks horizontally, vertically and diagonally, as illustrated in figure 1.
  • the pixels for which the choice will be made from a given set of vectors form an offset block, shown in the figure by a broken line.
  • localized motion vectors are chosen from the vectors V1, V2, V3 and V4 of the corresponding adjacent large blocks.
  • the inventive method is suited for motion estimation using block matching, wherein a motion vector related to each block is calculated and wherein for any pixel of the current block, a pixel motion vector is calculated using four motion vectors, that is the motion vector of the current block and the motion vectors of the three adjacent blocks, whereby
  • a first embodiment of the inventive apparatus for motion estimation using block matching wherein a motion vector related to each block of a picture signal is calculated, includes:
  • a second embodiment of the inventive apparatus for motion estimation using block matching wherein a motion vector related to each block of a picture signal is calculated, includes:
  • a third embodiment of the inventive apparatus for motion estimation using block matching wherein a motion vector related to each block of a picture signal is calculated, includes:
  • This embodiment works by calculating an estimated error for each pixel, for each of the four motion vectors from which the choice will be made, and then choosing the vector for which the estimated error is a minimum.
  • the estimated error is calculated by linear interpolation between the block matching errors produced for each of the four blocks concerned, taking into account the position of the pixel in question relative to the centre of each block.
  • the motion vector is V and the errors from each of the four blocks are E1(V), E2(V), E3(V) and E4(V)
  • FIG. 2 is a block diagram of a possible hardware implementation.
  • Large-block matching means LBM receiving the input signal INP produce for each candidate motion vector and for each block errors E, which are stored in an error memory EM, together with block motion vectors BV, which are stored in a block vector memory BVM and for which the error is a minimum in each block.
  • Motion vectors BV4 from the memory BVM are used in sets of four to control (C) via error memory control means EMC access to the corresponding error values over four blocks which are stored in error memory EM.
  • C control
  • EMC error memory control means
  • These errors E4 form the input to an error interpolator EI which calculates estimated errors EE, pixel by pixel.
  • This embodiment works by first performing the 'block matching' operation on each pixel using a block size of one pixel only and comparing only the four vectors allowed for the pixel. This can be thought of as giving each pixel a 'vote' for one of the four vectors. On its own, this operation introduces an unacceptable number of spurious vectors, so it is followed by an adjustment of the boundaries between motion vectors.
  • An example of the operation of this embodiment is given in figure 3.
  • the number of choices for vectors from the two blocks on the left (V1 and V3, result of counting rows RCR) is counted.
  • row count RC row count RC
  • the decision is made that the 'a' leftmost pixels in the row will be given motion vectors V1 or V3 and the (M-a) rightmost pixels V2 or V4 (vector assignment in rows VAR).
  • the number of choices for the upper pair of vectors (V1 and V2, result of counting columns RCC) is counted (column count CC). If the result is 'b', it is decided that the 'b' uppermost pixels in the column will be given motion vectors V1 or V2 and the (N-b) lowest pixels V3 or V4 (vector assignment in columns VAC).
  • every pixel has two decisions which together define a unique motion vector for the pixel (final vector assignment FVA) by forming areas in which both decisions (VAR and VAC) contain the same kind of vector.
  • FVA final vector assignment
  • FIG. 4 is a block diagram of a possible hardware implementa tion of this embodiment.
  • Large-block matching means LBM receiving the input signal INP produce for each block block vectors BV, which are stored in a block vector memory BVM.
  • Single-pixel matching means SPM receive the input signal INP and evaluate sets of four block vectors BV4 received from block vector memory BVM.
  • the resulting single-pixel vectors PV (the 'votes') are stored in vote count means CV and are reassigned in a vector assigner VA using the block vectors BV from memory BVM, according to the counting algorithm described above, to provide the final output pixel vectors OPV.
  • This method also retains good noise performance and is quite simple in hardware because the secondary, single-pixel search requires only four comparisons per pixel.
  • the process of counting votes across rows and columns can be generalized to take into account the magnitudes of the errors resulting from the single-pixel 'block matching' process.
  • This method also uses matching of smaller blocks, but here a compromise is made between the reliability of larger blocks and the need for a pixel-based motion vector field.
  • Block matching is performed using small blocks, typically 2x2, again comparing only the four vectors relevant to the small block (or 'subblock').
  • the resulting motion vector field is much more reliable than that resulting from a search using single pixels, but there is a slight penalty in that the localization is coarser.
  • a subblock size of 2x2 there is some benefit in following the subblock search with post-processing to remove the occasional spurious result, especially if the input picture is noisy.
  • One possibility is a median filter, using a window of 3 subblocks by 3 subblocks, on each component of the motion vector separately.
  • FIG. 5 is a block diagram of a possible implementation.
  • Largeblock matching means LBM receiving the input signal INP produce for each block block vectors BV, which are stored in a block vector memory BVM.
  • Subblock matching means SBM receive the input signal INP and evaluate sets of four vectors BV4 received from block vector memory BVM.
  • the resulting subblock vectors SBV which have been selected in SBM with respect to the minimum subblock error are median filtered in filter means MF to produce the final output subblock vectors OSBV.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
EP19940113494 1993-09-08 1994-08-30 Verfahren und Vorrichtung zur Bewegungsauswertung mit Blockübereinstimmung Expired - Lifetime EP0648052B1 (de)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP19940113494 EP0648052B1 (de) 1993-09-08 1994-08-30 Verfahren und Vorrichtung zur Bewegungsauswertung mit Blockübereinstimmung

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
EP93402188 1993-09-08
EP93402188 1993-09-08
EP93117661 1993-11-02
EP93117661 1993-11-02
EP19940113494 EP0648052B1 (de) 1993-09-08 1994-08-30 Verfahren und Vorrichtung zur Bewegungsauswertung mit Blockübereinstimmung

Publications (2)

Publication Number Publication Date
EP0648052A1 true EP0648052A1 (de) 1995-04-12
EP0648052B1 EP0648052B1 (de) 2000-03-01

Family

ID=27235233

Family Applications (1)

Application Number Title Priority Date Filing Date
EP19940113494 Expired - Lifetime EP0648052B1 (de) 1993-09-08 1994-08-30 Verfahren und Vorrichtung zur Bewegungsauswertung mit Blockübereinstimmung

Country Status (1)

Country Link
EP (1) EP0648052B1 (de)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0780795A1 (de) 1995-12-22 1997-06-25 THOMSON multimedia Bewegungsschätzungsverfahren
EP0883298A2 (de) * 1997-06-04 1998-12-09 Hitachi, Ltd. Vorrichtung zur Umsetzung von Bildsignalen sowie Fernsehempfänger
WO1999026416A2 (en) * 1997-11-17 1999-05-27 Koninklijke Philips Electronics N.V. Motion-compensated predictive image encoding and decoding
WO1999026417A2 (en) * 1997-11-17 1999-05-27 Koninklijke Philips Electronics N.V. Motion-compensated predictive image encoding and decoding
EP1104970A1 (de) * 1998-06-25 2001-06-06 Hitachi, Ltd. Verfahren und vorrichtung zur umwandlung der zahl von videorahmen von bildsignalen
EP1585337A2 (de) * 2004-03-29 2005-10-12 Sony Corporation Bildverarbeitungsvorrichtung und -verfahren, Speichermedium und Programm
EP1921865A2 (de) * 2006-11-10 2008-05-14 Tandberg Television ASA Erhalten eines Bewegungsvektors bei einer blockbasierten Bewegungsschätzung innerhalb eines Bewegungsvektorensatzes

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5162907A (en) * 1990-09-28 1992-11-10 Sony Broadcast & Communications Limited Motion dependent video signal processing

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5162907A (en) * 1990-09-28 1992-11-10 Sony Broadcast & Communications Limited Motion dependent video signal processing

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0780795A1 (de) 1995-12-22 1997-06-25 THOMSON multimedia Bewegungsschätzungsverfahren
EP0883298A2 (de) * 1997-06-04 1998-12-09 Hitachi, Ltd. Vorrichtung zur Umsetzung von Bildsignalen sowie Fernsehempfänger
EP0883298A3 (de) * 1997-06-04 2000-03-29 Hitachi, Ltd. Vorrichtung zur Umsetzung von Bildsignalen sowie Fernsehempfänger
WO1999026416A2 (en) * 1997-11-17 1999-05-27 Koninklijke Philips Electronics N.V. Motion-compensated predictive image encoding and decoding
WO1999026417A2 (en) * 1997-11-17 1999-05-27 Koninklijke Philips Electronics N.V. Motion-compensated predictive image encoding and decoding
WO1999026417A3 (en) * 1997-11-17 1999-07-22 Koninkl Philips Electronics Nv Motion-compensated predictive image encoding and decoding
WO1999026416A3 (en) * 1997-11-17 1999-07-29 Koninkl Philips Electronics Nv Motion-compensated predictive image encoding and decoding
EP1104970A4 (de) * 1998-06-25 2005-08-03 Hitachi Ltd Verfahren und vorrichtung zur umwandlung der zahl von videorahmen von bildsignalen
EP1104970A1 (de) * 1998-06-25 2001-06-06 Hitachi, Ltd. Verfahren und vorrichtung zur umwandlung der zahl von videorahmen von bildsignalen
EP1585337A2 (de) * 2004-03-29 2005-10-12 Sony Corporation Bildverarbeitungsvorrichtung und -verfahren, Speichermedium und Programm
EP1585337A3 (de) * 2004-03-29 2006-02-08 Sony Corporation Bildverarbeitungsvorrichtung und -verfahren, Speichermedium und Programm
CN100366051C (zh) * 2004-03-29 2008-01-30 索尼株式会社 图像处理设备和方法
EP1796044A3 (de) * 2004-03-29 2009-09-16 Sony Corporation Vorrichtung und Verfahren zur Bildverarbeitung, Aufzeichnungsmedium und Programm
KR101141467B1 (ko) 2004-03-29 2012-05-04 소니 주식회사 화상 처리 장치 및 방법, 및 기록 매체
US8503531B2 (en) 2004-03-29 2013-08-06 Sony Corporation Image processing apparatus and method, recording medium, and program
EP1921865A2 (de) * 2006-11-10 2008-05-14 Tandberg Television ASA Erhalten eines Bewegungsvektors bei einer blockbasierten Bewegungsschätzung innerhalb eines Bewegungsvektorensatzes
EP1921865A3 (de) * 2006-11-10 2010-04-21 Ericsson AB Erhalten eines Bewegungsvektors bei einer blockbasierten Bewegungsschätzung innerhalb eines Bewegungsvektorensatzes

Also Published As

Publication number Publication date
EP0648052B1 (de) 2000-03-01

Similar Documents

Publication Publication Date Title
US6438170B1 (en) Method and apparatus for motion estimation using block matching
Lukac Adaptive color image filtering based on center-weighted vector directional filters
EP0395271B1 (de) Bewegungsabhängige Videosignalverarbeitung
US6285711B1 (en) Block matching-based method for estimating motion fields and global affine motion parameters in digital video sequences
US6240211B1 (en) Method for motion estimated and compensated field rate up-conversion (FRU) for video applications and device for actuating such method
US5917936A (en) Object detecting system based on multiple-eye images
EP1766970B1 (de) Verfahren und vorrichtung zur entschachtelung von verschachteltem video
US7701508B2 (en) Accurate motion detection for the combination of motion adaptive and motion compensation de-interlacing applications
EP0677958A2 (de) Abtastumsetzung mit Bewegungsadaptation unter Verwendung einer richtungsabhängigen Interpolation der Kanten
US20060098737A1 (en) Segment-based motion estimation
US6343100B1 (en) Motion-vector detecting device
US5635994A (en) Method of making a hierarchical estimate of image motion in a television signal
JP3055438B2 (ja) ３次元画像符号化装置
US20070236601A1 (en) Iterative method of interpolating image information values
EP0395264A2 (de) Bewegungsabhängige Videosignalverarbeitung
US20090060042A1 (en) System and method for motion vector collection based on k-means clustering for motion compensated interpolation of digital video
GB2283385A (en) Motion compensated video signal processing
WO2007051993A1 (en) Video motion detection
EP0395263A2 (de) Bewegungsabhängige Videosignalverarbeitung
EP0395266B1 (de) Bewegungsabhängige Videosignalverarbeitung
EP0648052A1 (de) Verfahren und Vorrichtung zur Bewegungsauswertung mit Blockübereinstimmung
US20170085912A1 (en) Video sequence processing
WO2007051983A1 (en) Video motion detection
JPH06326976A (ja) 動き補償ビデオ信号処理方式
US8830394B2 (en) System, method, and apparatus for providing improved high definition video from upsampled standard definition video

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): DE FR GB IT

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: THOMSON MULTIMEDIA

17P Request for examination filed

Effective date: 19950918

17Q First examination report despatched

Effective date: 19961216

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: THOMSON MULTIMEDIA

GRAG Despatch of communication of intention to grant

Free format text: ORIGINAL CODE: EPIDOS AGRA

GRAG Despatch of communication of intention to grant

Free format text: ORIGINAL CODE: EPIDOS AGRA

GRAH Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOS IGRA

GRAH Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOS IGRA

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): DE FR GB IT

ITF It: translation for a ep patent filed

Owner name: JACOBACCI & PERANI S.P.A.

REF Corresponds to:

Ref document number: 69423166

Country of ref document: DE

Date of ref document: 20000406

ET Fr: translation filed
PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed
REG Reference to a national code

Ref country code: GB

Ref legal event code: IF02

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20070828

Year of fee payment: 14

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080830

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20090820

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20100901

Year of fee payment: 17

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 69423166

Country of ref document: DE

Effective date: 20110301

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110301

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20110830

Year of fee payment: 18

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20120430

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110831

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20120830

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120830