WO1993017525A1 - Motion compensated video image processing - Google Patents
Motion compensated video image processing Download PDFInfo
- Publication number
- WO1993017525A1 WO1993017525A1 PCT/GB1993/000403 GB9300403W WO9317525A1 WO 1993017525 A1 WO1993017525 A1 WO 1993017525A1 GB 9300403 W GB9300403 W GB 9300403W WO 9317525 A1 WO9317525 A1 WO 9317525A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- producing
- assigning
- technique
- image processing
- motion compensated
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
- H04N7/0135—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes
- H04N7/014—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes involving the use of motion vectors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/112—Selection of coding mode or of prediction mode according to a given display mode, e.g. for interlaced or progressive display mode
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
- H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/587—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/144—Movement detection
- H04N5/145—Movement estimation
Definitions
- This invention relates to video image processing and in particular to motion compensated image processing.
- Various motion compensation techniques may be used when it is desired to calculate a frame intermediate of two frames in a video sequence. This may be necessary in slow motion simulation or in standards conversion. The positions of moving objects within these fields are calculated with reference to a plurality of the other fields in the sequence.
- An example of such a motion compensation system is given in our British Patent Application No. 9111348.0.
- Each elementary area may be as small as a pixel or may comprise a block of many pixels.
- Phase correlation has been found to work very well for linear movements in objects and is very rarely confused by detail in the scene. It has not been found, however, to be particularly effective when used on rotating objects or on scenes where small picture elements move in different directions.
- 'block matching' The other commonly used technique for motion estimation is known as 'block matching'.
- n pixels by m pixels is compared with a corresponding block in a previous field and with each of a plurality of equally sized blocks within a predetermined range in that previous field. For a pair of interleaved video fields this range might be +/- 8 field lines vertically and +/- 16 pixels horizontally.
- the block which gives the best match to the original block is then used to derive a motion vector for that block.
- a method for motion compensated image processing comprising the steps of producing vectors assigned to elementary areas of input video fields from at least two fields in a sequence of video images by means of a first technique and producing vectors assigned to elementary areas of input video fields by means of a second technique in response to an error signal produced by the first technique.
- a motion compensated image processing system which uses a phase correlation technique to derive motion vectors.
- the motion estimation comprises phase correlating the input images and from this deriving a phase correlation surface.
- the peaks in this correspond to motion peaks and a list of a predetermined number of these above a certain threshold is selected as a trial menu for assigning vectors to elementary areas of the picture for deriving an. output field.
- a vector is assigned to an elementry area when its match error (calculated by applying the vector to an elementary area of an input field and summing the differences at a second input field ⁇ is below a certain level.
- a block matcher is used on that picture area and a best fit vector derived. If the match error for this vector is lower than, that for the phase correlation derived vector and preferably also below the match error then it is assigned to that picture area.
- the combined system uses the strengths of each method, a correctly assigned vector from the phase correlation has a very high confidence of being true motion and hence a dual system would have a priority for building pictures by this method. Where no good assignment can be made from the phase correlation technique the system will look at the match error of a block match derived vector.
- the dual system will exploit the natural strengths of each technique and be less complex and more certain than current methods.
- a simulation using the parameters of a currently available block match integrated circuit confirms that the technique works for those regions of the picture that phase correlation finds difficult.
- the work to date has been carried out using a block size of 16 pixels and 8 field lines.
- the range of detected motions are approx +/- 16 pixels and +/- 8 field lines. Over this range of linear displacements either technique will work.
- the phase correlator 4 is of the type described in British Patent No. 2188510B and produces a correlation surface 8 at its output. This forms the input to a peak hunter circuit 10 which searches the correlation surface for the highest values thereon. These correspond to the peak motion vectors between a pair of input fields and these trial vectors are provided on outputs V 1 -V n which form inputs to assignment units 12. Typically the five highest peaks will be used as the set of trial vectors provided at the outputs.
- Each assignment circuit 12 receives a single vector and tests each elementary area of an input field with that vector and produces an error signal in dependence on the error that vector generates in deriving one field from that input field. These errors 14 are provided in parallel to an error detector 16 for each elementary area in turn.
- the minimum error signal detected by the error detector 16 is provided at an output coupled to an input A of a thresholding circuit 18.
- a second input B receives a threshold level. If the error A ⁇ B then a control signal for a vector selection 20 is generated which will then select the best assigned vector as determined by the error detection 20 for assignment to that particular elementary area of an input video field.
- thresholding circuit 26 the operation of which will be described below.
- the second thresholding circuit 26 receives an error signal from the block matcher circuit 6.
- This block matcher circuit operates on the same input video fields 2 as the phase correlator 4. It generates a best block match, vector 30 which is provided at an input of the vector selection circuit 20.
- the block match error C is compared with a threshold D. If the error C ⁇ D then an output is generated which is coupled to the second output of AND gate 22. Thus if the phase correlation generates an error A greater than threshold B and the block matcher produces an error C less than a threshold D a logical 1 will be generated at the output of AND gate 22. This output is used to control selection circuit 20 to assign the best block match vector to the particular elementary area.
- the block matches are overlapped 2:1 in both directions and the best vector assigned to the central 1/4 area of the block. It is also proposed to determine an estimate of subpixel movements by matching a 'V shape to the lower match error in the distortion map from the block matcher.
- the vectors (probably only three plus a pan vector) from the phase correlator are assigned in a well known manner. A good match gives high confidence of being true motion. If the match is above some threshold then the block matcher is left to assign. If the block matcher also shows a high match error then a fallback method is used as before. This would comprise assiging a zero vector to that elementary area. In essence the block matcher is a more sophisticated fallback mode that will allow the processor to cope with most kinds of motion.
- the match error is normalised by dividing it by the luminance gradient for the picture element in question. It is then compared with the threshold level to decide whether or not to use the block matcher.
- the reason for this normalisation is that a good match error for a picture with a lot of motion in it will be substantially different to a good match error for a picture with very little motion in it.
- the normalisation by dividing by the luminance gradient will enable the same threshold level to be used for each picture.
- the luminance gradient used for normalisation is the greatest value of luminance gradient at a picture element. It is calculated from a combination of the vertical and horizontal gradients.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Image Analysis (AREA)
- Television Systems (AREA)
- Color Television Systems (AREA)
Abstract
Description
Claims
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GB9417278A GB2279200B (en) | 1992-02-26 | 1993-02-26 | Motion compensated video image processing |
DE69320461T DE69320461T2 (en) | 1992-02-26 | 1993-02-26 | Motion compensated video image processing |
EP93904265A EP0628234B1 (en) | 1992-02-26 | 1993-02-26 | Motion compensated video image processing |
JP5514672A JPH07504304A (en) | 1992-02-26 | 1993-02-26 | Motion compensated video image processing method and apparatus |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GB929204117A GB9204117D0 (en) | 1992-02-26 | 1992-02-26 | Video image processing |
GB9204117.7 | 1992-02-26 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO1993017525A1 true WO1993017525A1 (en) | 1993-09-02 |
Family
ID=10711092
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/GB1993/000403 WO1993017525A1 (en) | 1992-02-26 | 1993-02-26 | Motion compensated video image processing |
Country Status (8)
Country | Link |
---|---|
US (1) | US5940145A (en) |
EP (1) | EP0628234B1 (en) |
JP (1) | JPH07504304A (en) |
CN (1) | CN1078340A (en) |
AU (1) | AU3572293A (en) |
DE (1) | DE69320461T2 (en) |
GB (2) | GB9204117D0 (en) |
WO (1) | WO1993017525A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0695096B1 (en) * | 1994-07-27 | 2000-03-22 | Sony Corporation | Apparatus for detecting motion vector |
Families Citing this family (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH07250328A (en) * | 1994-01-21 | 1995-09-26 | Mitsubishi Electric Corp | Moving vector detector |
KR100224752B1 (en) * | 1997-06-11 | 1999-10-15 | 윤종용 | Target tracking method and apparatus |
US6496184B1 (en) * | 1998-11-30 | 2002-12-17 | William T. Freeman | Method for inferring scenes from test images and training data using probability propagation in a markov network |
US6473460B1 (en) * | 2000-03-31 | 2002-10-29 | Matsushita Electric Industrial Co., Ltd. | Method and apparatus for calculating motion vectors |
US6665450B1 (en) * | 2000-09-08 | 2003-12-16 | Avid Technology, Inc. | Interpolation of a sequence of images using motion analysis |
JP4106874B2 (en) * | 2001-02-05 | 2008-06-25 | ソニー株式会社 | Image processing apparatus and method, and recording medium |
US7043058B2 (en) * | 2001-04-20 | 2006-05-09 | Avid Technology, Inc. | Correcting motion vector maps for image processing |
US7545957B2 (en) * | 2001-04-20 | 2009-06-09 | Avid Technology, Inc. | Analyzing motion of characteristics in images |
US7194676B2 (en) | 2002-03-01 | 2007-03-20 | Avid Technology, Inc. | Performance retiming effects on synchronized data in an editing system |
AU2003265075A1 (en) * | 2002-10-22 | 2004-05-13 | Koninklijke Philips Electronics N.V. | Image processing unit with fall-back |
US7136506B2 (en) * | 2003-03-03 | 2006-11-14 | Lockheed Martin Corporation | Correlation based in frame video tracker |
US7751482B1 (en) * | 2004-02-27 | 2010-07-06 | Vbrick Systems, Inc. | Phase correlation based motion estimation in hybrid video compression |
US8000392B1 (en) | 2004-02-27 | 2011-08-16 | Vbrick Systems, Inc. | Phase correlation based motion estimation in hybrid video compression |
US8265145B1 (en) | 2006-01-13 | 2012-09-11 | Vbrick Systems, Inc. | Management and selection of reference frames for long term prediction in motion estimation |
US8488007B2 (en) * | 2010-01-19 | 2013-07-16 | Sony Corporation | Method to estimate segmented motion |
US8285079B2 (en) * | 2010-03-19 | 2012-10-09 | Sony Corporation | Method for highly accurate estimation of motion using phase correlation |
US8896759B2 (en) | 2010-10-26 | 2014-11-25 | Sony Corporation | Method to increase the accuracy of phase correlation motion estimation in low-bit-precision circumstances |
US8842735B2 (en) * | 2010-10-26 | 2014-09-23 | Sony Corporation | Method to improve detection of motion with phase correlation |
US8792559B2 (en) | 2010-10-26 | 2014-07-29 | Sony Corporation | Method to improve accuracy and reliability of motion estimated with phase correlation |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0322956A1 (en) * | 1987-12-22 | 1989-07-05 | Koninklijke Philips Electronics N.V. | Video encoding and decoding using an adpative filter |
US4980764A (en) * | 1988-06-24 | 1990-12-25 | Etat Francais (Cnet) | Method for the encoding of data for assistance in the reconstruction of a sub-sampled moving electronic image |
WO1991014340A1 (en) * | 1990-03-15 | 1991-09-19 | Thomson Consumer Electronics S.A. | Digital image processing including block edges filtering |
Family Cites Families (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1987005769A1 (en) * | 1986-03-19 | 1987-09-24 | British Broadcasting Corporation | Tv picture motion measurement |
GB2222500A (en) * | 1988-09-05 | 1990-03-07 | Philips Electronic Associated | Picture motion measurement |
FR2638874B1 (en) * | 1988-11-10 | 1994-07-01 | Thomson Csf | METHOD FOR ESTIMATING THE MOVEMENT OF AT LEAST ONE TARGET IN A SUITE OF IMAGES, AND DEVICE FOR CARRYING OUT SAID METHOD |
US5081531A (en) * | 1989-01-11 | 1992-01-14 | U.S. Philips Corporation | Method and apparatus for processing a high definition television signal using motion vectors representing more than one motion velocity range |
GB2231747B (en) * | 1989-04-27 | 1993-07-07 | Sony Corp | Motion dependent video signal processing |
GB2231745B (en) * | 1989-04-27 | 1993-07-07 | Sony Corp | Motion dependent video signal processing |
US5043807A (en) * | 1989-05-23 | 1991-08-27 | Zenith Electronics Corporation | Three dimensional composite video motion detection |
US5072293A (en) * | 1989-08-29 | 1991-12-10 | U.S. Philips Corporation | Method of estimating motion in a picture signal |
JP2860702B2 (en) * | 1990-10-16 | 1999-02-24 | シャープ株式会社 | Motion vector detection device |
US5175618A (en) * | 1990-10-31 | 1992-12-29 | Victor Company Of Japan, Ltd. | Compression method for interlace moving image signals |
JPH04207788A (en) * | 1990-11-30 | 1992-07-29 | Sony Corp | Band compression device |
JP3103894B2 (en) * | 1991-02-06 | 2000-10-30 | ソニー株式会社 | Apparatus and method for correcting camera shake of video data |
JP2677312B2 (en) * | 1991-03-11 | 1997-11-17 | 工業技術院長 | Camera work detection method |
US5278663A (en) * | 1991-06-28 | 1994-01-11 | Samsung Electronics Co. Ltd. | Method for compensating the vibration of an image and device therefor in a video camera |
JP2507204B2 (en) * | 1991-08-30 | 1996-06-12 | 松下電器産業株式会社 | Video signal encoder |
US5227878A (en) * | 1991-11-15 | 1993-07-13 | At&T Bell Laboratories | Adaptive coding and decoding of frames and fields of video |
SE469412B (en) * | 1992-04-13 | 1993-06-28 | Dv Sweden Ab | MAKE ADAPTIVE ESTIMATES UNUSUAL GLOBAL IMAGE INSTABILITIES IN IMAGE SEQUENCES IN DIGITAL VIDEO SIGNALS |
GB9217098D0 (en) * | 1992-08-12 | 1992-09-23 | British Broadcasting Corp | Derivation of studio camera position and motion from the camera image |
EP0697788A3 (en) * | 1994-08-19 | 1997-03-26 | Eastman Kodak Co | Adaptive, global-motion compensated deinterlacing of sequential video fields with post processing |
-
1992
- 1992-02-26 GB GB929204117A patent/GB9204117D0/en active Pending
-
1993
- 1993-02-25 CN CN93101564.2A patent/CN1078340A/en active Pending
- 1993-02-26 AU AU35722/93A patent/AU3572293A/en not_active Abandoned
- 1993-02-26 JP JP5514672A patent/JPH07504304A/en active Pending
- 1993-02-26 WO PCT/GB1993/000403 patent/WO1993017525A1/en active IP Right Grant
- 1993-02-26 DE DE69320461T patent/DE69320461T2/en not_active Expired - Fee Related
- 1993-02-26 EP EP93904265A patent/EP0628234B1/en not_active Expired - Lifetime
- 1993-02-26 GB GB9417278A patent/GB2279200B/en not_active Expired - Fee Related
-
1997
- 1997-07-21 US US08/897,380 patent/US5940145A/en not_active Expired - Lifetime
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0322956A1 (en) * | 1987-12-22 | 1989-07-05 | Koninklijke Philips Electronics N.V. | Video encoding and decoding using an adpative filter |
US4980764A (en) * | 1988-06-24 | 1990-12-25 | Etat Francais (Cnet) | Method for the encoding of data for assistance in the reconstruction of a sub-sampled moving electronic image |
WO1991014340A1 (en) * | 1990-03-15 | 1991-09-19 | Thomson Consumer Electronics S.A. | Digital image processing including block edges filtering |
Non-Patent Citations (1)
Title |
---|
Signal Processing: Image Communication, Volume 3, 1991, (Colchester), H.M. Ahmad Fadzil et al, "VIDEO SUBBAND VQ AT 64 kbit/s USING SHORT-KERNEL FILTER BANKS WITH AN IMPROVED MOTION ESTIMATION TECHNIQUE" * |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0695096B1 (en) * | 1994-07-27 | 2000-03-22 | Sony Corporation | Apparatus for detecting motion vector |
Also Published As
Publication number | Publication date |
---|---|
CN1078340A (en) | 1993-11-10 |
EP0628234B1 (en) | 1998-08-19 |
GB9204117D0 (en) | 1992-04-08 |
EP0628234A1 (en) | 1994-12-14 |
DE69320461D1 (en) | 1998-09-24 |
JPH07504304A (en) | 1995-05-11 |
AU3572293A (en) | 1993-09-13 |
US5940145A (en) | 1999-08-17 |
DE69320461T2 (en) | 1998-12-24 |
GB2279200A (en) | 1994-12-21 |
GB2279200B (en) | 1995-09-13 |
GB9417278D0 (en) | 1994-10-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US5940145A (en) | Method and apparatus for motion compensated image processing using two separate motion measurement techniques | |
US5436674A (en) | Method of detecting motion vector, apparatus therefor, and picture signal processing system utilizing the apparatus | |
US5748247A (en) | Refinement of block motion vectors to achieve a dense motion field | |
EP0181215B1 (en) | Apparatus for detecting motion of television images | |
EP0513047B1 (en) | Three-frame technique for analyzing two motions in successive image frames dynamically | |
US5742710A (en) | Computationally-efficient method for estimating image motion | |
US7010039B2 (en) | Motion estimator for reduced halos in MC up-conversion | |
US4663665A (en) | TV system conversion apparatus | |
WO1992019068A1 (en) | A method for estimating motion contents in video signals | |
JPS6152084A (en) | Highly efficient encoder | |
CA2212013A1 (en) | Method for estimating the location of an image target region from tracked multiple image landmark regions | |
EP0722252B1 (en) | Boundary matching motion estimation apparatus | |
KR20050089886A (en) | Background motion vector detection | |
JPS61269475A (en) | Detection system for dynamic vector | |
EP0734175A1 (en) | Hierarchical motion vector estimation using multiple block sizes | |
KR20060083978A (en) | Motion vector field re-timing | |
KR20050012766A (en) | Unit for and method of estimating a motion vector | |
EP0632915B1 (en) | A machine method for compensating for non-linear picture transformations, e.g. zoom and pan, in a video image motion compensation system | |
EP0367310B1 (en) | Picture motion measurement | |
WO2004082294A1 (en) | Method for motion vector determination | |
JPH09182077A (en) | Image encoding method and device | |
US6044114A (en) | Method of image segmentation | |
WO1995024096A1 (en) | Motion estimation | |
JPS6225587A (en) | Detector circuit for moving vector | |
JPH06350972A (en) | Motion detector for picture signal |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AU BG BR CA CZ FI GB HU JP KR NO RU SK US |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): AT BE CH DE DK ES FR GB GR IE IT LU MC NL PT SE |
|
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
WWE | Wipo information: entry into national phase |
Ref document number: 1993904265 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref country code: US Ref document number: 1994 290969 Date of ref document: 19940929 Kind code of ref document: A Format of ref document f/p: F |
|
WWP | Wipo information: published in national office |
Ref document number: 1993904265 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: CA |
|
WWG | Wipo information: grant in national office |
Ref document number: 1993904265 Country of ref document: EP |