EP1958451B1 - Motion vector field correction - Google Patents

Motion vector field correction Download PDF

Info

Publication number
EP1958451B1
EP1958451B1 EP06821541.7A EP06821541A EP1958451B1 EP 1958451 B1 EP1958451 B1 EP 1958451B1 EP 06821541 A EP06821541 A EP 06821541A EP 1958451 B1 EP1958451 B1 EP 1958451B1
Authority
EP
European Patent Office
Prior art keywords
motion vector
image
occlusion
region
regions
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Not-in-force
Application number
EP06821541.7A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP1958451A2 (en
Inventor
Rui F.C. Guerreiro
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Entropic Communications LLC
Original Assignee
Entropic Communications LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Entropic Communications LLC filed Critical Entropic Communications LLC
Priority to EP06821541.7A priority Critical patent/EP1958451B1/en
Publication of EP1958451A2 publication Critical patent/EP1958451A2/en
Application granted granted Critical
Publication of EP1958451B1 publication Critical patent/EP1958451B1/en
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/144—Movement detection
    • H04N5/145—Movement estimation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H04N19/521—Processing of motion vectors for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/553—Motion estimation dealing with occlusions

Definitions

  • the invention relates to a motion vector correction unit for correcting a motion vector field comprising motion vectors of respective regions of an image.
  • the invention further relates to an image processing apparatus comprising such a motion vector correction unit, to a method of correcting a motion vector field comprising motion vectors of respective regions of an image and to a computer program product to be loaded by a computer arrangement, comprising instructions for correcting a motion vector field comprising motion vectors of respective regions of an image.
  • a motion vector of a region typically indicates a direction and a magnitude of movement of the region from an image to the next or from the next to the previous image in a sequence of images.
  • a motion vector field comprising a set of motion vectors can be used to substantially define all the motions in an input image.
  • the motion vector field is useful in predicting the next image from the previous image in a sequence of images involving motion of objects in a scene, for e.g. a video sequence.
  • a motion estimation unit being designed to estimate motion vectors based on comparing pixel values of pairs of images is known from the article " True-motion estimation with 3D-recursive Search Block Matching", by G. de. Haan et. al in IEEE Transactions on Circuits and Systems for Video Technology, vol. 3, no.5, October 1993, pages 368-379 .
  • Other motion vector estimation units are also possible.
  • a motion estimation unit estimating motion between two consecutive images of a video sequence cannot perform well in occlusion regions, as it is typical for these regions that some parts of background information are available in either of the two images.
  • An estimated motion vector field therefore, might comprise erroneous motion vectors caused by covering and/or uncovering regions.
  • Estimating and assigning the correct motion vectors to occlusion regions i.e. correcting the motion vector field of an image is an area of active research.
  • correct assignment of motion vector is meant, assigning the motion vector of the background object to the covering region in the previous image and assigning the motion vector of the background object to the uncovering region in the next image.
  • Assigning the motion vector of the foreground object to the uncovering region in the previous image and assigning the motion vector of the foreground object to the covering region in the next image are also considered as correct assignment.
  • a wrongly assigned motion vector can cause artifacts such as 'halo effect' at the boundaries of objects, when a next image is predicted from the previous image or interpolated from two consecutive images for e.g. in temporal up conversion.
  • a method and unit for determining the correct motion vectors in occlusion regions of a current image by choosing appropriate motion vectors estimated from the current to the previous and from the current to the next image is known in prior art.
  • An embodiment of such a method is described in a patent application published under number WO 03/067523 .
  • a method of assigning a motion vector estimated from the current to the previous image for a covering area in the current image and assigning a motion vector estimated from the current to the next image to an uncovering area in the current image is described. Accordingly, three consecutive images are required for assigning a correct motion vector to an occlusion region of an image resulting in a higher latency compared to motion estimation and correction with two consecutive images. Higher latency is undesirable in some consumer applications for e.g. television wherein sound has to be synchronous to the video stream.
  • Motion compensating interpolation considering occluding, appearing and disappearing areas describes a method of correcting a motion vector in an occlusion region by using interpolation from neighbouring motion vectors.
  • the motion vector correction unit comprises:
  • the motion vector correction unit is capable of correcting an estimated motion vector field by using two consecutive images.
  • correcting a motion vector is meant replacing the erroneous motion vector previously assigned to the occlusion region with the most probably correct motion vector.
  • correcting motion vector field means replacing the motion vector of the covering region by the motion vector of the background object in the previous image and replacing the motion vector of the uncovering region by the motion vector of the background object in the current image.
  • covering region occurring in the current image, the motion vector of the foreground object has to be assigned.
  • An estimated motion vector field with detected occlusion regions is made available at the input of the motion vector correction unit according to the invention.
  • the motion vectors being computed for an occlusion region typically comprise a motion vector which corresponds with the movement of the foreground, i.e. the foreground motion vector and a motion vector which corresponds with the movement of the background i.e. the background motion vector.
  • the invention provides a solution to this problem by providing an analyzer for analyzing the image contents of an occlusion region and a number of non-occlusion regions in the neighborhood of the occlusion region to extract an image property value of each region.
  • An evaluator computes similarity values based on the image property value of the occlusion region with each one of the non-occlusion regions.
  • a selector selects the best matching non-occlusion region based on the similarity values.
  • a vector replacement unit replaces the erroneous motion vector of the occlusion region by the motion vector of the particular non-occlusion region which is selected by the selector.
  • the invention is based on the insight that the image contents of the covering region in an image substantially matches with the image contents of the background object of the previous image and the image properties of the uncovering region substantially matches with the background regions of the current image. Therefore it is possible to select the best matching non-occlusion region within a predefined neighborhood of an occlusion region.
  • a vector replacement unit replaces the motion vector of the occlusion region by motion vectors of the selected non-occlusion region.
  • the motion vector correction unit according to the invention correctly assigns the background motion vector to the covering region in the previous image and the background motion vector to the uncovering region in the current image.
  • Motion vector estimation, occlusion detection and classification can be done on a pair of images.
  • the motion vector correction unit typically requires two consecutive images of a sequence of images and therefore works with reduced latency compared to the prior art unit that requires three consecutive images.
  • the analyzer is arranged to obtain a texture parameter as an image property value
  • An image property value indicating a texture measure of a region can be obtained by analyzing the contents of the region.
  • a similarity value based on the texture content of the image is a useful measure for comparing regions with aim of selecting the best matching region. Any one of the texture property measures such as DCT coefficients, Hadamard transform coefficients or statistical measures such as average, variance, density peaks and/or the direction and/or the strength of detected edges are preferably used as similarity values.
  • the analyzer is arranged to obtain a luminance parameter as an image property value.
  • An image property value based on the luminance values of pixels can be easily computed from a region.
  • Luminance values of pixels represented in one of the known formats such as RGB, HIS or YUV can be used for comparison.
  • a simple and useful luminance property of a region is average hue and/or color saturation.
  • the neighborhood is a spatial neighborhood.
  • a set of non-occlusion regions in the neighborhood of the occlusion region is selected as candidate regions for contributing the correct motion vector. It is highly probable that most similar non-occlusion region can be found within a spatial neighborhood of the occlusion region.
  • the motion vector correction unit is arranged to locate one of the number of non-occlusion regions in the spatial neighborhood of the occlusion region on basis of a combination of a motion vector of the occlusion region and a further motion vector of the occlusion region for analyzing, evaluating and selecting the particular non-occlusion region.
  • the selection of motion vectors being considered for the occlusion region comprises a background motion vector and a foreground motion vector V 1 and V 2 .
  • temporal neighborhood is meant the neighborhood considered in an image temporally adjacent to the image under consideration. That means, the previous or next image from a sequence of images.
  • a number of non-occlusion regions from a temporally adjacent image are considered. Hence, the non-occlusion regions may not be available at the same coordinates, but in a spatial neighborhood of the temporal image due to motion. It is expected that image characteristics of such regions will be substantially close to the region under consideration.
  • the motion vector correction unit is arranged to locate one of the number of non-occlusion regions in the temporal neighborhood comprising the image and a further image, the image and the further image being consecutive images of a sequence of images, on basis of a motion vector of the occlusion region for analyzing, evaluating and selecting the particular non-occlusion region.
  • the background motion vector and the foreground motion vector V 1 and V 2 typically calculated for the occlusion region are projected back to the temporally adjacent image to select a temporal neighborhood.
  • the requirement to select a region outside the occlusion region, being in the temporal neighborhood of the occlusion region is thus fulfilled.
  • the selected regions may be found typically in the spatial neighborhood of the temporal image.
  • the evaluator is arranged to evaluate further similarity values which are based on a second image property value of the occlusion region and corresponding image property values of the respective non-occlusion regions, based upon a request from the selector.
  • the advantage of this method is that if the set of similarity values is insufficient to select a particular non-occlusion region due to substantially equal values, a second set of similarity values can be calculated by the system, thereby avoiding ambiguity of selection of the particular non-occlusion region.
  • the image processing apparatus comprises:
  • the motion vector correction unit receives two consecutive images of an image sequence, analyzes the image contents of neighboring non-occlusion regions of an occlusion region and extract image property values and based on the similarity values computed from the image property value, selects a non-occlusion region.
  • the motion vector of the selected non-occlusion region is assigned to the occlusion region thereby correcting the estimated motion vector field.
  • the image processing apparatus is arranged to carry out motion compensated processing of the sequence of images based on the corrected motion vector field.
  • the image processing apparatus may be arranged to support one or more of the following types of image processing:
  • the image processing apparatus may comprise additional components for e.g. a display device for displaying the output images or it may supply images to a connected display.
  • the image processing apparatus can support one or more of the following types of consumer and professional apparatus by incorporating additional means: TV, a set top box, a VCR/VCP, a satellite tuner, DVD player/recorder.
  • the image processing apparatus comprises storage means like a hard-disk or means for storage on removable media such as optical disks.
  • the image processing apparatus might also be a system being applied by a film studio or broadcaster or re-transmitter.
  • This object of the invention is achieved that the method of correcting a motion vector field of an image comprising motion vectors of respective regions of an image, the method comprises:
  • the method according to the invention corrects motion vector field from the image under consideration. For estimating the motion vector field, finding the occlusion regions and classifying them, typically two consecutive images can be used. Thus the method can carry out the motion vector correction process typically by using two images and hence works with a reduced latency compared to the prior art method.
  • Modifications of the motion vector correction unit may correspond to modifications and variations thereof of the image processing apparatus, the method and the computer program product described.
  • Fig 1A schematically shows a pair of consecutive images 100, 102 comprising a moving object 104.
  • Two consecutive input images 100 and 102 are shown with an elliptically shaped object 104 at a first point in time n-1 and a subsequent point in time n , respectively.
  • the vertical coordinates correspond with axis 110 and the horizontal coordinates correspond with the axis 130.
  • the temporal axis 120 corresponding to time is also shown.
  • the elliptically shaped object 104 moves from left to right in the foreground as indicated by the trajectory 124. Simultaneously, the background moves in the opposite direction with a trajectory as indicated by the arrow 126.
  • Fig. 1B schematically shows a 2-D representation of the situation depicted in Fig. 1A.
  • Fig. 1B is rotated such that the temporal axis 120 and the horizontal axis 130 are visible.
  • the moving foreground object 104 is shown as black rectangles in images 100 and 102.
  • the background object vector 126 is assigned with a velocity V 1 and the foreground object vector 124 is assigned with a velocity V 2 .
  • Motion vector fields M n and M n -1 have been computed for image 100 and image 102 respectively.
  • the occlusion areas shown as gray triangles appear due to the movement of the background and the foreground objects. In these areas, motion vectors are not typically correct since the information is not present in one of the two images.
  • Fig. 2 schematically shows an embodiment of the motion vector correction unit according to the invention.
  • the motion vector correction unit 200 comprises an analyzer 210, and evaluator 220, a selector 230 and a motion vector replacement unit 240.
  • a sequence of images is supplied at the input connector 201.
  • the motion vector field and the classified occlusion regions are also supplied for respective regions at the input 201.
  • a motion vector field can be estimated for group of pixels or regions by comparing regions of two images.
  • One such motion estimation unit is described in the article " True motion estimation with 3-D Recursive Search Block Matching" by G. de Haan et. Al. in IEEE Transactions on circuits and systems for video technology, vol. 3, no. 5. October 1993, pages 368-379 .
  • a motion vector belongs to a region or a block of pixels under consideration.
  • the book “Video processing for multimedia systems”, by G. de. Haan, University Press Eindhoven, 2000, ISBN 90-9014015-8, chapter 4 describes method for detection of occlusion regions and for the covering/uncovering classification.
  • a set of motion vectors comprising e.g. a foreground motion vector and a background motion vector are typically computed.
  • the motion vector correction unit is provided to detect a background motion vector and to assign it to the corresponding region, e.g. a covering region in the previous image and detect a background motion vector and assign it to the corresponding region, e.g. an uncovering region in the current image.
  • the estimated motion vector field and classified occlusion regions are made available as inputs to the motion vector correction unit according to the invention.
  • some of the regions may not converge to a particular motion vector.
  • the invention maybe useful in resolving the assignment of a most probably correct motion vector in such cases.
  • the analyzer 210 is arranged to analyze the image contents of an occlusion region and a number of non-occlusion regions in the neighborhood of the occlusion region and compute image property values of respective regions.
  • An image property value is computed from the pixel values in the spatial domain or in a transformed domain such as frequency domain.
  • the image property can be a texture parameter or a luminance parameter.
  • Luminance parameter can be derived from one or more of the components of intensity values of the pixels of the region represented in some standard formats, for e.g. RGB, HIS or YUV.
  • An image property measure can be derived from one or more of the intensity values such as, average of one or more of the Y, U and V components, variance of Y or U or V components.
  • Similarity value based on texture can be obtained from histogram comparison of Y or YUV, Discreet Cosine transform coefficients or Fourier transform coefficients.
  • Another way of computing a texture parameter is by modeling the texture in the region using a stochastic model and estimating the parameters of the model.
  • the evaluator 220 is arranged to receive the image property values of the respective regions and compute a similarity measure of the image property value of the occlusion region with the image property value of each one of the non-occlusion regions.
  • the evaluator 220 may employ a matching method or a difference method for obtaining a similarity value.
  • a selection of a number of image property values for each region may be used in evaluating a similarity value a region. If more than one image property value is used in the computation of similarity value it may be a comparative measure such as sum of absolute differences or sum of squared differences.
  • the selector 230 is arranged to select a particular non-occlusion region best matching with the occlusion region, based on the corresponding similarity value. If the similarity value is based on an error measure, a minimum value of the similarity measure corresponds to the best matching non-occlusion region. If the similarity measure is a matching score such as a dot product of vectors, a maximum value corresponds to the best matching non-occlusion region. If the selector is unable to find the best matching non-occlusion regions with the available similarity value, it sends a request 250 for recalculation of a new set of similarity values based on further image property measures to the evaluator.
  • the vector replacement unit 240 replaces the motion vector of the occlusion region by the motion vector of the selected particular non-occlusion region.
  • the motion vector belongs to the region or block of pixels under consideration.
  • the corrected motion vector field of the input image is provided at the output connector 241 for control of further processing.
  • Fig. 3 schematically shows an illustration of non-occlusion regions in a spatial neighborhood of an occlusion region. Images 100 and 102 at time instances n -1 and n are shown with a moving object 104. The foreground object velocity 124 is shown as V 2 and the background object velocity 126 is shown as V 1 . After the occlusion regions are detected, one of the occlusion region e.g. R 0 may have vector V 2 assigned to it or the region R 0 is yet to be assigned with one of the vectors from the candidate vectors V 1 and V 2 . Typically, while estimating the motion vectors, two motion vectors can become strong competitors to a single region and the iterative process may not converge.
  • a spatial neighborhood is selected with respect to an occlusion region, the neighborhood comprises regions that obviously fall outside the occlusion region R 0 and within the image boundaries.
  • One method of selecting such a spatial neighborhood is to combine the two motion vectors V 1 and V 2 available for the occlusion region V 2 and project the combination back to the image that two non-occlusion regions are selected as candidate regions for analyzing the contents.
  • the vector V 2 is projected back to reach a background region R 3 in the image 100 and from the background region, another vector V 1 is identified.
  • region R 2 is identified as one of the candidate regions whose motion vector can be used to replace the wrongly assigned vector of region R 0 .
  • V c ⁇ 2 V 2 - V 1
  • V 1 - V 2 combination identifies region R 2 and V 2 - V 1 combination identifies region R 1 for further analysis.
  • R 1 happens to be a foreground region. Both these regions are analyzed, image property values computed and the region that matches best with occlusion region is selected.
  • the best matching region may be e.g. the region R 2 . Therefore the motion vector of region R 2 being the background motion vector, which replaces the erroneous motion vector of the uncovering region R 0 .
  • correction of motion vector of an uncovering region in image 102 is depicted, an identical scheme can be applied to correct the motion vector of a covering region in image 100 as well.
  • a skilled person by following the same procedure can assign the correct e.g. background motion vector to the covering region in the previous image. It also follows that in case of covering region occurring in the current image, the most probably correct e.g. the foreground motion vector can be assigned by a similar scheme.
  • Fig. 4 schematically shows an illustration of non-occlusion regions located in a temporal neighborhood of an occlusion region.
  • the images 102 and 100 with a moving object 104 at time instances n and n -1 are shown.
  • Non-occlusion regions in the temporal neighborhood can be considered for analysis of image contents.
  • Region R 0 is the occlusion area. Initially, region R 0 may be wrongly assigned e.g. with vector V 2 or, at the time of estimation, two candidate vectors V 1 and V 2 may be identified but one of the two may not be assigned due to uncertainty.
  • V 1 and V 2 candidate regions in image 100 may be identified.
  • Motion vectors V 1 and V 2 are projected back to image 100 to identify two regions in the temporal neighborhood namely R 1 and R 2 .
  • the image is temporal but the regions are spatially adjacent.
  • Image contents of R 0 , R 1 and R 2 are analyzed, respective image property values computed and similarity values are computed to select the matching non-occlusion region for uncovering region R 0 . It is expected that the image property value of region R 2 matches well with that of R 0 .
  • an alternate embodiment of identifying candidate regions may be implemented.
  • the selector When an occlusion region is substantially similar to a non-occlusion region in the spatial or temporal neighborhood, there is a possibility of selecting a wrong non-occlusion region by the selector.
  • similarity values may be substantially equal to each other and the selector may encounter a problem in selecting a particular non-occlusion region.
  • the motion vector correction unit provides two schemes to resolve such ambiguous situations. In the first scheme, the selector sends a feedback signal 250 to the evaluator 220 to evaluate a new set of similarity values based on further image property measures of each one of the occlusion and non-occlusion regions. The new set of similarity values are expected to have a better discrimination for deciding the best matching non-occlusion region.
  • a new set of non-occlusion regions can be selected by extending the foreground and background vectors. Each vector is incremented by a pre-decided fraction of its respective value. The increase in vector will result in different sets of non-occlusion regions being selected as candidate regions. The increase is as small as possible in order to preserve spatial correlation, but large enough so that a new non-occlusion region can be identified in which it is expected that the computed image property value gives a similarity value for better discrimination and selection.
  • the vectors may be incremented iteratively until a discriminatory similarity value is obtained.
  • the vector correction unit 240 for replacing the motion vector of the occlusion region may use a previously selected non-occlusion region. Similarly, in case of covering region occurring in the current image, assignment of most probably correct e.g. foreground motion vector can be carried out by following a similar procedure.
  • FIG. 5 schematically shows an illustration of an image processing apparatus according to the invention comprising:
  • Fig. 6 schematically shows the method according to the invention.
  • a sequence of images are received along with their motion vector fields and classified occlusion regions in the analysis step 610.
  • the output of analysis step 610 comprises image property values of respective occlusion and non-occlusion regions and these values are used in the evaluation step 620 for computing similarity values.
  • a best matching non-occlusion region is selected based on the similarity values in the selection step 630.
  • the motion vector of the selected particular non-occlusion region in the vector replacement step 640 replaces the motion vector of the occlusion region.
  • the selector 630 sends a request 650 to the evaluator in order to evaluate a second set of similarity values based on a second image property value of the respective regions.
  • the computer program product can be handled in a standard comprised or detachable storage, e.g. flash memory or hard disk.
  • the computer program product may be embedded in a device such as an integrated circuit or a computing machine as embedded software or kept pre-loaded or loaded from one of the standard storage or memory devices.
  • the computer program product may be presented in any one of the known codes such as machine level codes or assembly languages or higher level languages and made to operate on any of the available platforms such as hand-held devices or personal computers or servers.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)
  • Television Systems (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
EP06821541.7A 2005-11-30 2006-11-24 Motion vector field correction Not-in-force EP1958451B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP06821541.7A EP1958451B1 (en) 2005-11-30 2006-11-24 Motion vector field correction

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
EP05111497 2005-11-30
PCT/IB2006/054422 WO2007063465A2 (en) 2005-11-30 2006-11-24 Motion vector field correction
EP06821541.7A EP1958451B1 (en) 2005-11-30 2006-11-24 Motion vector field correction

Publications (2)

Publication Number Publication Date
EP1958451A2 EP1958451A2 (en) 2008-08-20
EP1958451B1 true EP1958451B1 (en) 2014-03-19

Family

ID=38007133

Family Applications (1)

Application Number Title Priority Date Filing Date
EP06821541.7A Not-in-force EP1958451B1 (en) 2005-11-30 2006-11-24 Motion vector field correction

Country Status (5)

Country Link
EP (1) EP1958451B1 (es)
JP (1) JP2009533887A (es)
CN (1) CN101322409B (es)
ES (1) ES2460541T3 (es)
WO (1) WO2007063465A2 (es)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11102501B2 (en) 2015-08-24 2021-08-24 Huawei Technologies Co., Ltd. Motion vector field coding and decoding method, coding apparatus, and decoding apparatus

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2063636B1 (en) * 2006-09-15 2012-12-12 Panasonic Corporation Video processing device and video processing method
US8208065B2 (en) * 2008-07-30 2012-06-26 Cinnafilm, Inc. Method, apparatus, and computer software for digital video scan rate conversions with minimization of artifacts
EP2224740A1 (en) 2009-02-27 2010-09-01 Nxp B.V. Detecting occlusion
EP2224738A1 (en) 2009-02-27 2010-09-01 Nxp B.V. Identifying occlusions
JP2011223493A (ja) * 2010-04-14 2011-11-04 Canon Inc 画像処理装置および画像処理方法
CN103685866B (zh) * 2012-09-05 2016-12-21 杭州海康威视数字技术股份有限公司 视频稳像方法及其装置
ITTO20121073A1 (it) * 2012-12-13 2014-06-14 Rai Radiotelevisione Italiana Apparato e metodo per la generazione e la ricostruzione di un flusso video

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2279531B (en) * 1993-06-24 1997-07-16 Sony Uk Ltd Motion compensated image interpolation
JPH09130803A (ja) * 1995-10-27 1997-05-16 Fujitsu Ltd 背景画像復元方法および装置
JP2000270337A (ja) * 1999-03-18 2000-09-29 Telecommunication Advancement Organization Of Japan 動き検出装置及び方法
WO2001088852A2 (en) * 2000-05-18 2001-11-22 Koninklijke Philips Electronics N.V. Motion estimator for reduced halos in motion compensared picture rate up-conversion
JP2005517251A (ja) * 2002-02-05 2005-06-09 コーニンクレッカ　フィリップス　エレクトロニクス　エヌ　ヴィ ピクセル群の動きベクトルを推定する方法及びユニット
JP4070104B2 (ja) * 2002-10-16 2008-04-02 独立行政法人情報通信研究機構 動画像符号化復号方法及び動画像符号化装置並びに動画像復号装置
JP2007504700A (ja) * 2003-09-02 2007-03-01 コーニンクレッカ　フィリップス　エレクトロニクス　エヌ　ヴィ 閉塞検出に基づく画素の時間的補間
EP1665806A1 (en) * 2003-09-17 2006-06-07 Koninklijke Philips Electronics N.V. Motion vector field re-timing

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11102501B2 (en) 2015-08-24 2021-08-24 Huawei Technologies Co., Ltd. Motion vector field coding and decoding method, coding apparatus, and decoding apparatus

Also Published As

Publication number Publication date
CN101322409A (zh) 2008-12-10
WO2007063465A2 (en) 2007-06-07
WO2007063465A3 (en) 2007-11-15
CN101322409B (zh) 2011-08-03
EP1958451A2 (en) 2008-08-20
ES2460541T3 (es) 2014-05-13
JP2009533887A (ja) 2009-09-17

Similar Documents

Publication Publication Date Title
KR101135454B1 (ko) 특정 이미지의 특정 픽셀 값 결정 방법, 픽셀 값 결정 유닛, 이미지 처리 장치 및 컴퓨터 판독 가능한 저장 매체
KR100973429B1 (ko) 배경 움직임 벡터 선택기, 업-변환 유닛, 이미지 처리 장치, 배경 움직임 벡터 선택 방법 및 컴퓨터 판독 가능한 기록 매체
EP1958451B1 (en) Motion vector field correction
US7949205B2 (en) Image processing unit with fall-back
US20050180506A1 (en) Unit for and method of estimating a current motion vector
US20050226462A1 (en) Unit for and method of estimating a motion vector
EP1444837B1 (en) Occlusion detector for and method of detecting occlusion areas
US20050163355A1 (en) Method and unit for estimating a motion vector of a group of pixels
US20170085912A1 (en) Video sequence processing
US8102915B2 (en) Motion vector fields refinement to track small fast moving objects
WO2006013510A1 (en) De-interlacing
JP4597282B2 (ja) 画像情報変換装置、変換方法および表示装置
WO2005091625A1 (en) De-interlacing

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20080630

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

17Q First examination report despatched

Effective date: 20090309

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: TRIDENT MICROSYSTEMS (FAR EAST) LTD.

111L Licence recorded

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU LV MC NL PL PT RO SE SI SK TR

Name of requester: KONINKLIJKE PHILIPS ELECTRONICS N.V., NL

Effective date: 20120220

DAX Request for extension of the european patent (deleted)
RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: ENTROPIC COMMUNICATIONS, INC.

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20130502

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20131029

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602006040772

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: H04N0007260000

Ipc: H04N0019000000

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

111L Licence recorded

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU LV MC NL PL PT RO SE SI SK TR

Name of requester: KONINKLIJKE PHILIPS ELECTRONICS N.V., NL

Effective date: 20120220

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 19/00 20140101AFI20140212BHEP

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 658350

Country of ref document: AT

Kind code of ref document: T

Effective date: 20140415

Ref country code: CH

Ref legal event code: PK

Free format text: ERGAENZUNG LIZENZEINTRAG: NICHT AUSSCHLIESSLICHE LIZENZ

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602006040772

Country of ref document: DE

Effective date: 20140430

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2460541

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20140513

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140319

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20140319

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 658350

Country of ref document: AT

Kind code of ref document: T

Effective date: 20140319

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140319

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140319

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140319

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140319

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140319

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140319

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140719

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140619

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140319

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140319

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140319

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140319

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140319

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140319

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602006040772

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140721

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140319

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20141126

Year of fee payment: 9

Ref country code: FR

Payment date: 20141118

Year of fee payment: 9

Ref country code: GB

Payment date: 20141127

Year of fee payment: 9

26N No opposition filed

Effective date: 20141222

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20141125

Year of fee payment: 9

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602006040772

Country of ref document: DE

Effective date: 20141222

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140319

Ref country code: LU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20141124

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140319

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20141130

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20141130

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20141124

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140620

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20151124

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20151124

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140319

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20061124

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20160729

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20151124

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20151130

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20161123

Year of fee payment: 11

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20151125

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602006040772

Country of ref document: DE

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20180626

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180602