EP1981269A2 - Entschachtelungsvideo - Google Patents

Entschachtelungsvideo Download PDF

Info

Publication number
EP1981269A2
EP1981269A2 EP08154351A EP08154351A EP1981269A2 EP 1981269 A2 EP1981269 A2 EP 1981269A2 EP 08154351 A EP08154351 A EP 08154351A EP 08154351 A EP08154351 A EP 08154351A EP 1981269 A2 EP1981269 A2 EP 1981269A2
Authority
EP
European Patent Office
Prior art keywords
field
pixel
new
motion vector
derived
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP08154351A
Other languages
English (en)
French (fr)
Other versions
EP1981269A3 (de
Inventor
Dan Eley
Thomas Huns
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Snell Advanced Media Ltd
Original Assignee
Snell and Wilcox Ltd
Snell Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Snell and Wilcox Ltd, Snell Ltd filed Critical Snell and Wilcox Ltd
Publication of EP1981269A2 publication Critical patent/EP1981269A2/de
Publication of EP1981269A3 publication Critical patent/EP1981269A3/de
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0117—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving conversion of the spatial resolution of the incoming video signal
    • H04N7/012—Conversion between an interlaced and a progressive signal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/144—Movement detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/144—Movement detection
    • H04N5/145—Movement estimation

Definitions

  • This invention concerns spatio-temporal interpolation of images.
  • Two to one interlaced scanning in which the spatial sampling grid is offset vertically by half the vertical sample pitch on alternate temporal samples, is a very common method of reducing the bandwidth of television images. Now that more modern (often transform-based) data compression methods are available and electron-beam-scanned displays are less common, the use of interlaced scanning is becoming less attractive.
  • De-interlacing can form part of many video processes; by de-interlacing at the input to a process, that process can be made easier. Examples include standards conversion and re-scaling.
  • each converted output image of the sequence thus has twice as many vertical samples (scanning lines) as each original input image; and, pixel data values are available for all vertical sample positions and temporal sample points.
  • de-interlacing This conversion is a spatio-temporal interpolation process and it is generally known as "de-interlacing". It has been found that best subjective results are obtained by de-interlacing systems which take into account the motion of portrayed objects: either by motion adaptation, in which movement is detected and the interpolation changed as a result; or, by using motion compensation, in which the change in position of objects between consecutive fields is measured, and used to "compensate" the positions of pixels in one field to the positions that the objects they portray would occupy in a different field. These positional changes are commonly described by two-dimensional "motion vectors" and in typical motion compensated processes one or more vectors are associated with each image pixel.
  • phase correlation blocks of contiguous pixels are transformed into the spatial frequency domain and the phases of the spatial frequency components are correlated between co-located blocks in consecutive fields.
  • block matching the values of groups of contiguous blocks of pixels are compared with similar, but spatially shifted, groups of pixels in adjacent fields so as to find the shift vector which gives the best match of the pixel values.
  • match error is evaluated as a sum of the magnitudes of pixel value differences over a block of pixels.
  • the said first motion vector is derived from motion measurement between the said field to be de-interlaced and the preceding temporally adjacent field; and, the said second motion vector is derived from motion measurement between the said field to be de-interlaced and the next temporally adjacent field.
  • the first and second new-pixel values may be combined by addition, with appropriate scaling factors related to or determined by the respective confidence values.
  • the said confidence value is derived from the height of a peak in a phase-correlation surface.
  • the said confidence value is derived from the match-error in a block-matching process.
  • new pixel values derived by combination of said first and second new-pixel values in dependence on comparison of vector confidence values are adaptively combined with other new pixel values derived by one or more alternative methods.
  • apparatus for de-interlacing a video field in which new pixels to be added to the said field are derived from temporally adjacent fields comprising a field delay arrangement for making available a next field, a current field and a previous field; a motion estimator for forming a first motion vector from the previous field to the current field with a first motion vector confidence value and a second motion vector from the next field to the current field with a second motion vector confidence value; a first pixel shifter for forming a first new-pixel value from a pixel of the previous field and the first motion vector; a second pixel shifter for forming a second new-pixel value from a pixel of the next field and the second motion vector; and a combiner receiving the first and second motion vector confidence values and serving to combine the said first and second new-pixel values in dependence upon a comparison of the respective confidence values as to obtain a value for a new pixel to be added to the said field to be de-interlaced.
  • the apparatus further comprises a confidence processor serving to compare the first and second motion vector confidence values to generate a multi-valued parameter k, said combiner forming a weighted sum of the first and second new-pixel values governed said parameter k.
  • the parameter k is not a binary value simply adapting or switching from one value to the other.
  • An important application of embodiments of this invention lies in de-interlacing video material which combines true video material with video material obtained from film in 2:2 or 3:2 processes. Changes in mode can occur between fields and modes can also be mixed within a field.
  • a video originating logo may, for example, be imposed on film originating material.
  • the ability of the parameter k to vary from field to field and also in preferred arrangements from pixel to pixel, is therefore important.
  • the processing combines pixels directly - in the manner of a finite impulse response filter (FIR) - without the use of recursion.
  • FIR finite impulse response filter
  • Another important feature of aspects of this invention is that the motion measurement is conducted directly upon the input interlaced fields.
  • the confidence measures of such vectors are therefore strong indicators of the nature of the video material. Any material processing of the input interlaced material (such as for example a preliminary de-interlacing filter) would likely degrade the relationship between the confidence in the vector and the nature of the material and specifically the film/video mode of the material.
  • Figure 1 shows an example of a de-interlacing system according to the invention.
  • An input interlaced video signal (1) is required to be de-interlaced; i.e. for every input field an additional set of pixels is required located at vertical positions mid way between the existing pixels and representing the same point in time as the existing pixels.
  • the input sequence of interlaced fields (1) is delayed in two, cascaded field delays (2) (3). If the output (4) from the first field delay (2) is regarded as the "current field”, then the un-delayed input (1) can be considered the "next field”, and the output (5) from the second field delay (3) can be considered to be the "previous field”.
  • a motion estimator (6) compares the current field (4) with the next field (1) so as to obtain motion vectors (8). These vectors define, for each pixel of the current field (4), the equivalent position of the object portrayed by that pixel in the next field (1). These vectors (8) are therefore “forward vectors” for the current field (4).
  • a similar motion estimator (7) derives “backward vectors” (9) from comparison with the previous field (5).
  • a pixel shifter (10) uses the motion vectors (8) to find values for additional pixels at intermediate vertical positions in the current field (4) by shifting appropriate pixels of the next field (1) to the positions of the additional pixels.
  • the vectors (8) were derived for the existing pixels of the current field (4) it will be necessary to find equivalent vectors at the locations of the new pixels to be created. This can be done by interpolation of the vector field; in most cases "zero order interpolation" can be used so that the vector from a vertically adjacent position is used.
  • the value of each new pixel is set to the value of the existing pixel in the next field (1) whose location (relative to the new pixel location) is determined by the motion vector. Where the location determined by the motion vector does not correspond exactly with that of an existing pixel, a weighted sum of two or more existing pixel values can be used to obtain the necessary value according to well-known subpixel interpolation techniques.
  • a similar pixel shifter (11) uses the backward vectors (9) to derive a second set of new pixel values (co-located with the new pixels created by the pixel-shifter (10)) from the values of the pixels of the previous field (5) by shifting objects portrayed in the previous field (5) to their positions at the time of the current field (4).
  • the pixels of the next field (1) and the previous field (5) will be located vertically between the pixels of the current field (4). Depending on the characteristics of the video signal, it may sometimes be preferable to use the pixels derived from the next field (1) and at other times it may be preferable to use the pixels derived from the previous field (5).
  • the preferred source of additional pixels may not be the same throughout a particular field; this will frequently be the case when different video sources, having different temporal sampling characteristics, have been combined.
  • motion vectors are determined by a correlation process between pixels of adjacent fields, and the maximum value of a correlation parameter is used to determine the relevant motion vector.
  • a two-dimensional correlation surface is obtained and peaks in the surface represent candidate motion vectors; the co-ordinates of the locations of the peaks correspond to the components of vectors. The heights of the peaks give a measure of confidence that the corresponding vector is accurate.
  • a match-error value is obtained and the error value corresponding to an output motion vector can be used to calculate a measure of confidence for that vector.
  • the match error can be input to a function which returns a high confidence value for small match errors and returns a low confidence value for large match errors.
  • each of the vectors comprising the sets of motion vectors (8) (9) is accompanied by a respective confidence value.
  • each of these sets of vectors has been used to create a respective set of new pixel values: one set from the next field (1); and, a second set from the previous field (5).
  • a set of additional pixels (13) to be used to de-interlace the current field (4) is derived by combining co-located pairs of pixels from the pixel shifter (10) and the pixel shifter (11) in a cross-fader (12).
  • the cross-fader (12) combines each pixel from the pixel shifter (10) with the corresponding co-located pixel from the pixel shifter (11) according to the fader-control parameter K derived in a confidence processor (14).
  • the confidence processor (14) compares the vector confidence values of the vectors used to create the respective shifted pixels. Where the confidence of the forward vector (8) is higher than the confidence of the backward vector (9), a value of K greater than one half is output, and the contribution of the pixel from the shifter (10) to the output pixel (13) is greater than the contribution from the shifter (11).
  • the sets of output pixels (13) from the cross-fader (12) can be combined with the respective sets of pixels of the current field (4) so as to obtain a sequence of de-interlaced fields in which each output field has twice the vertical resolution of each input field.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Graphics (AREA)
  • Television Systems (AREA)
  • Color Television Systems (AREA)
EP08154351A 2007-04-11 2008-04-10 Entschachtelungsvideo Withdrawn EP1981269A3 (de)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
GB0706988A GB2448336A (en) 2007-04-11 2007-04-11 De-interlacing video using motion vectors

Publications (2)

Publication Number Publication Date
EP1981269A2 true EP1981269A2 (de) 2008-10-15
EP1981269A3 EP1981269A3 (de) 2009-11-11

Family

ID=38091179

Family Applications (1)

Application Number Title Priority Date Filing Date
EP08154351A Withdrawn EP1981269A3 (de) 2007-04-11 2008-04-10 Entschachtelungsvideo

Country Status (3)

Country Link
US (1) US8421918B2 (de)
EP (1) EP1981269A3 (de)
GB (1) GB2448336A (de)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2197152A (en) 1986-10-31 1988-05-11 British Broadcasting Corp Interpolating lines of video signals
EP0735748A2 (de) * 1995-03-27 1996-10-02 AT&T Corp. Verfahren und Vorrichtung zum Umsetzen einer nach dem Zeilensprungverfahren abgetasteten Videobildfolge in einem Bildfolge mit progressiver Abtastung
US5682205A (en) * 1994-08-19 1997-10-28 Eastman Kodak Company Adaptive, global-motion compensated deinterlacing of sequential video fields with post processing
WO2006075178A1 (en) 2005-01-14 2006-07-20 Snell & Wilcox Limited Image processing
WO2008035063A2 (en) * 2006-09-18 2008-03-27 Snell & Wilcox Limited Method and apparatus for interpolating an image

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5886745A (en) * 1994-12-09 1999-03-23 Matsushita Electric Industrial Co., Ltd. Progressive scanning conversion apparatus
US6269484B1 (en) * 1997-06-24 2001-07-31 Ati Technologies Method and apparatus for de-interlacing interlaced content using motion vectors in compressed video streams
AU2001247574A1 (en) * 2000-03-27 2001-10-08 Teranex, Inc. Temporal interpolation of interlaced or progressive video images
US7116372B2 (en) * 2000-10-20 2006-10-03 Matsushita Electric Industrial Co., Ltd. Method and apparatus for deinterlacing
US6940557B2 (en) * 2001-02-08 2005-09-06 Micronas Semiconductors, Inc. Adaptive interlace-to-progressive scan conversion algorithm
KR20050049680A (ko) * 2003-11-22 2005-05-27 삼성전자주식회사 노이즈 감쇠장치 및 디인터레이싱 장치
EP1714482A1 (de) * 2004-02-04 2006-10-25 Koninklijke Philips Electronics N.V. Bewegungskompensierte entschachtelung mit filmmodusanpassung
GB2411784B (en) * 2004-03-02 2006-05-10 Imagination Tech Ltd Motion compensation deinterlacer protection
US7953293B2 (en) * 2006-05-02 2011-05-31 Ati Technologies Ulc Field sequence detector, method and video device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2197152A (en) 1986-10-31 1988-05-11 British Broadcasting Corp Interpolating lines of video signals
US5682205A (en) * 1994-08-19 1997-10-28 Eastman Kodak Company Adaptive, global-motion compensated deinterlacing of sequential video fields with post processing
EP0735748A2 (de) * 1995-03-27 1996-10-02 AT&T Corp. Verfahren und Vorrichtung zum Umsetzen einer nach dem Zeilensprungverfahren abgetasteten Videobildfolge in einem Bildfolge mit progressiver Abtastung
WO2006075178A1 (en) 2005-01-14 2006-07-20 Snell & Wilcox Limited Image processing
WO2008035063A2 (en) * 2006-09-18 2008-03-27 Snell & Wilcox Limited Method and apparatus for interpolating an image

Also Published As

Publication number Publication date
US8421918B2 (en) 2013-04-16
GB0706988D0 (en) 2007-05-16
US20080252779A1 (en) 2008-10-16
EP1981269A3 (de) 2009-11-11
GB2448336A (en) 2008-10-15

Similar Documents

Publication Publication Date Title
US7057665B2 (en) Deinterlacing apparatus and method
US9185431B2 (en) Motion detection device and method, video signal processing device and method and video display device
EP0393906B1 (de) Interpolation eines Videosignals
JP2001054075A (ja) 画像信号の動き補償走査変換回路
KR20040009967A (ko) 디인터레이싱장치 및 방법
CN100534131C (zh) 具有改进的运动向量选择的运动向量估算器和估算方法
EP1721458A1 (de) Verringern von artefakten bei der abtastratenumsetzung von bildsignalen durch kombinieren von interpolation und extrapolation von bildern
US5485224A (en) Motion compensated video signal processing by interpolation of correlation surfaces and apparatus for doing the same
CN101167353A (zh) 帧频变换装置及影像显示装置
EP1723786B1 (de) Bewegungskompensations-entschachtelerschutz
EP1592249A1 (de) Umkehr-Filmmodus-Extrapolation
US20080259207A1 (en) Motion Compensated De-Interlacing with Film Mode Adaptation
EP2619977B1 (de) Verfahren und vorrichtung zur zeilenentflechtung von videodaten
US8421918B2 (en) De-interlacing video
KR100594780B1 (ko) 영상변환 장치 및 그 방법
GB2312806A (en) Motion compensated video signal interpolation
KR20070043703A (ko) 비월 비디오 이미지에서의 움직임 추정
Park et al. Covariance-based adaptive deinterlacing method using edge map
KR100382650B1 (ko) 비디오신호 처리 시스템에서의 스케일 움직임 정보를이용한 움직임 검출 방법 및 장치와 이를 이용한 데이터보간 방법 및 장치
Park et al. Adaptive weighting scheme for edge-based line interpolation
Fan Deinterlacing system and technology overview and discussion
KR20050015189A (ko) 위상 정정 필드를 이용한 적응적 디-인터레이싱 방법 및그 장치와, 이를 구현하기 위한 프로그램이 기록된 기록매체
EP1665780A1 (de) Robuste entschachtelung von videosignalen
EP1665781A1 (de) Robuste entschachtelung von videosignalen
KR100728914B1 (ko) 영상의 속성을 이용한 디인터레이싱 장치 및 그 방법

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA MK RS

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: SNELL LIMITED

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA MK RS

17P Request for examination filed

Effective date: 20100225

17Q First examination report despatched

Effective date: 20100506

AKX Designation fees paid

Designated state(s): DE FR GB

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: SNELL ADVANCED MEDIA LIMITED

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20181221

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20190501