# USRE44012E1 - Methods for motion estimation with adaptive motion accuracy - Google Patents

Methods for motion estimation with adaptive motion accuracy Download PDF## Info

- Publication number
- USRE44012E1 USRE44012E1 US13289902 US201113289902A USRE44012E US RE44012 E1 USRE44012 E1 US RE44012E1 US 13289902 US13289902 US 13289902 US 201113289902 A US201113289902 A US 201113289902A US RE44012 E USRE44012 E US RE44012E
- Authority
- US
- Grant status
- Grant
- Patent type
- Prior art keywords
- motion vector
- motion
- searching
- criteria
- best
- Prior art date
- Active

## Links

## Images

## Abstract

## Description

This application is a Continuation Reissue Application of copending U.S. application Ser. No. 11/984,661, filed on Nov. 20, 2007. U.S. application Ser. No. 11/984,661 is a Reissue of U.S. application Ser. No. 09/615,791, filed on Jul. 13, 2000, now U.S. Pat. No. 6,968,008, which claims the benefit of priority of U.S. Provisional Application No. 60/146,102, filed on Jul. 27, 1999. The entire contents of all of the above applications are incorporated herein by reference.

This application claims the benefit of Provisional Application No. 60/146,102, filed Jul. 27, 1999.

The present invention relates generally to a method of compressing or coding digital video with bits and, specifically, to an effective method for estimating and encoding motion vectors in motion-compensated video coding.

In classical motion estimation the current frame to be encoded is decomposed into image blocks of the same size, typically blocks of 16×16 pixels, called “macroblocks.” For each current macroblock, the encoder searches for the block in a previously encoded frame (the “reference frame”) that best matches the current macroblock. The coordinate shift between a current macroblock and its best match in the reference frame is represented by a two-dimensional vector (the “motion vector”) of the macroblock. Each component of the motion vector is measured in pixel units.

For example, if the best match for a current macroblock happens to be at the same location, as is the typical case in stationary background, the motion vector for the current macroblock is (0,0). If the best match is found two pixels to the right and three pixels up from the coordinates of the current macroblock, the motion vector is (2,3). Such motion vectors are said to have integer pixel (or “integer-pel” or “full-pel”) accuracy, since their horizontal X and vertical Y components are integer pixel values. In _{1}=(1,1) represents the full-pel motion vector for a given current macroblock.

Moving objects in a video scene do not move in integer pixel increments from frame to frame. True motion can take any real value along the X and Y directions. Consequently, a better match for a current macroblock can often be found by interpolating the previous frame by a factor N×N and then searching for the best match in the interpolated frame. The motion vectors can then take values in increments of 1/N pixel along X and Y and are said to have 1/N pixel (or “1/N-pel”) accuracy.

In “Response to Call for Proposals for H.26L,” ITU-Telecommunications Standardization Sector, Q.15/SG16, doc. Q15-F-11, Seoul, Nov. 98, and “Enhancement of the Telenor proposal for H.26L,” ITU-Telecommunications Standardization Sector, Q.15/SG16, doc. Q15-G-25, Monterey, Feb. 99, Gisle Bjontegaard proposed using ⅓-pel accurate motion vectors and cubic-like interpolation for the H26L video coding standard (the “Telenor encoder”). To do this, the Telenor encoder interpolates or “up-samples” the reference frame by 3×3 using a cubic-like interpolation filter. This interpolated version requires nine times more memory than the reference frame. At a given macroblock, the Telenor encoder estimates the best motion vector in two steps: the encoder first searches for the best integer-pel vector and then the Telenor encoder searches for the best ⅓-pixel accurate vector V_{1/3 }near V_{1}. Using _{1/3}=(VX, VY)=(1+⅓,1). The Telenor encoder has several problems. First, it uses a sub-optimal fast-search strategy and a complex cubic filter (at all stages) to compute the ⅓-pel accurate motion vectors. As a result, the computed motion vectors are not optimal and the memory and computation requirements are very expensive. Further, the Telenor encoder uses an accuracy of the effective rate-distortion criteria that is fixed at ½-pixel and, therefore, does not adapt to select better motion accuracies. Similarly, the Telenor encoder variable-length code (“VLC”) table has an accuracy fixed at ⅓-pixel and, therefore, is not adapted and interpreted differently for different accuracies.

Most known video compression methods estimate and encode motion vectors with ⅓-pixel accuracy, because early studies suggested that higher or adaptive motion accuracies would increase computational complexity without providing additional compression gains. These early studies, however, did not estimate the motion vectors using optimized rate-distortion criteria, did not exploit the convexity properties of such criteria to reduce computational complexity, and did not use effective strategies to encode the motion vectors and their accuracies.

One such early study was Bernd Girod's “Motion-Compensating Prediction with Fractional-Pel Accuracy,” IEEE Transactions on Communications, Vol. 41, No. 4, pp. 604-612, April 1993 (the “Girod work”). The Girod work is the first fundamental analysis on the benefits of using sub-pixel motion accuracy for video coding. Girod used a simple, hierarchical strategy to search for the best motion vector in sub-pixel space. He also used simple mean absolute difference (“MAD”) criteria to select the best motion vector for a given accuracy. The best accuracy was selected using a formula that is not useful in practice since it is based on idealized assumptions, is very complex, and restricts all motion vectors to have the same accuracy within a frame. Finally, Girod focused only on prediction error energy and did not address how to use bits to encode the motion vectors.

Another early study was Smita Gupta's and Allen Gersho's “On Fractional Pixel Motion Estimation,” Proc. SPIE VCIP, Vol. 2094, pp. 408-419, Cambridge, November 1993 (the “Gupta work”). The Gupta work presented a method for computing, selecting, and encoding motion vectors with sub-pixel accuracy for video compression. The Gupta work disclosed a formula based on mean squared error (“MSE”) and bilinear interpolation, used this formula to find an ideal motion vector, and then quantized such vector to the desired motion accuracy. The best motion vector for a given accuracy was found using the sub-optimal MSE criteria and the best accuracy was selected using the largest decrease in difference energy per distortion bit, which is a greedy (sub-optimal) criteria. A given motion vector was coded by first encoding that vector with ½-pel accuracy and then encoding the higher accuracy with refinement bits. Coarse-to-fine coding tends to require significant bit overhead.

In “On the Optimal Motion Vector Accuracy for Block-Based Motion-Compensated Video Coders,” Proc. IST/SPIE Digital Video Compression: Algorithms and Technologies, pp. 302-314, San Jose, February 1996 (the “Ribas work”), Jordi Ribas-Corbera and David L. Neuhoff, modeled the effect of motion accuracy on bit rate and proposed several methods to estimate the optimal accuracies that minimize bit rate. The Ribas work set forth a full-search approach for computing motion vectors for a given accuracy and considered only bilinear interpolation. The best motion vector was found by minimizing MSE and the best accuracy was selected using some formulas derived from a rate-distortion optimization. The motion vectors and accuracies were encoded with frame-adaptive entropy coders, which are complex to implement in real-time applications.

In “Proposal for a new core experiment on prediction enhancement at higher bitrates,” ISO/IEC JTC1/SC29/WG11 Coding of Moving Pictures and Audio, MPEG 97/1827, Sevilla, February 1997 and “Performance Evaluation of a Reduced Complexity Implementation for Quarter Pel Motion Compensation,” ISO/IEC JTC1/SC29/WG11 Coding of Moving Pictures and Audio, MPEG 97/3146, San Jose, January 1998, Ulrich Benzler proposed using ¼-pel accurate motion vectors for the video sequence and more advanced interpolation filters for the MPEG4 video coding standard. Benzler, however, used the Girod's fast-search technique to find the ¼-pel motion vectors. Benzler did consider different interpolation filters, but proposed a complex filter at the first stage and a simpler filter at the second stage and interpolated one macroblock at a time. This approach does not require much cache memory, but it is computationally expensive because of its complexity and because all motion vectors are computed with ¼-pel accuracy for all the possible modes in a macroblock (e.g., 16×16, four-8×8, sixteen-4×4, etc.) and then the best mode is determined. Benzler used the MAD criteria to find the best motion vector which was fixed to ¼-pel accuracy for the whole sequence, and hence he did not address how to select the best motion accuracy. Finally, Benzler encoded the motion vectors with a variable-length code (“VLC”) table that could be used for encoding ½ and ¼pixel accurate vectors.

The references discussed above do not estimate the motion vectors using optimized rate-distortion criteria and do not exploit the convexity properties of such criteria to reduce computational complexity. Further, these references do not use effective strategies to encode motion vectors and their accuracies.

One preferred embodiment of the present invention addresses the problems of the prior art by computing motion vectors of high pixel accuracy (also denoted as “fractional” or “sub-pixel” accuracy) with a minor increase in computation.

Experiments have demonstrated that, by using the search strategy of the present invention, a video encoder can achieve significant compression gains (e.g., up to thirty percent in bit rate savings over the classical choices of motion accuracy) using similar levels of computation. Since the motion accuracies are adaptively computed and selected, the present invention may be described as adaptive motion accuracy (“AMA”).

One preferred embodiment of the present invention uses fast-search strategies in sub-pixel space that smartly searches for the best motion vectors. This technique estimates motion vectors in motion-compensated video coding by finding a best motion vector for a macroblock. The first step is searching a first set of motion vector candidates in a grid of sub-pixel resolution of a predetermined square radius centered on V_{1 }to find a best motion vector V_{2}. Next, a second set of motion vector candidates in a grid of sub-pixel resolution of a predetermined square radius centered on V_{2 }is searched to find a best motion vector V_{3}. Then, a third set of motion vector candidates in a grid of sub-pixel resolution of a predetermined square radius centered on V_{3 }is searched to find the best motion vector of the macroblock.

In an alternate preferred embodiment the present invention, a technique for estimating high-accurate motion vectors may use different interpolation filters at different stages in order to reduce computational complexity.

Another alternate preferred embodiment of the present invention selects the best vectors and accuracies in a rate-distortion (“RD”) sense. This embodiment uses rate-distortion criteria that adapts according to the different motion accuracies to determine both the best motion vectors and the best motion accuracies.

Still further, another alternate preferred embodiment of the present invention encodes the motion vector and accuracies with an effective VLC approach. This technique uses a VLC table that is interpreted differently at different coding units, according to the associated motion vector accuracy.

The foregoing and other objectives, features, and advantages of the invention will be more readily understood upon consideration of the following detailed description of the invention, taken in conjunction with the accompanying drawings.

**114** of

The methods of the present invention are described herein in terms of the motion accuracy being modified at each image block. These methods, however, may be applied when the accuracy is fixed for the whole sequence or modified on a frame-by-frame basis. The present invention is also described as using Telenor's video encoders (and particularly the Telenor encoder) as described in the Background of the Invention. Although described in terms of Telenor's video encoders, the techniques described herein are applicable to any other motion-compensated video coder.

Most video coders use motion vectors with half pixel (or “½-pel”) accuracy and bilinear interpolation. The first version of Telenor's encoder also used ½-pel motion vectors and bilinear interpolation. The latest version of Telenor's encoder, however, incorporated ⅓-pel vectors and cubic-like interpolation because of the additional compression gains. Specifically, at a given macroblock, Telenor's encoder estimates the best motion vector in two steps shown in _{1 }(**100**. Second, the Telenor encoder searches for the best ⅓-pixel accurate vector V_{1/3 }(_{1 } **102**. This second step is shown graphically in _{1}. In _{1/3}=(V_{x}, V_{y})=(1+⅓, 1).

The technology of the present invention allows the encoder to choose between any set of motion accuracies (for example, ½, ⅓, and ⅙-pel accurate motion vectors) using either a full search strategy or a fast search strategy.

Full-Search AMA Search Strategy

As shown in _{1 }(**104**. In the second step of the full-search AMA, the encoder searches for the best ⅙-pixel accurate vector V_{1/6 }(_{1 } **106**. In other words, the full-search AMA modifies the second step of the Telenor's process so that the encoder also searches for motion vector candidates in other sub-pixel locations in the velocity space. The objective is to find the best motion vector in the grid, i.e., the vector that points to the block (in the interpolated reference frame) that best matches the current macroblock. Although the full-search strategy is computationally complex since it searches 120 sub-pixel candidates, it shows the full potential of this preferred method of the present invention.

A critical issue in the motion vector search is the choice of a measure or criterion for establishing which block is the best match for the given macroblock. In practice, most methods use either the mean squared error (“MSE”) or mean absolute difference (“MAD”) criteria. The MSE between two blocks consists of subtracting the pixel values of the two blocks, squaring the pixel differences, and then taking the average. The MAD difference between two blocks is a similar distortion measure, except that the absolute value of the pixel differences is computed instead of the squares. If two image blocks are similar to each other, the MSE and MAD values will be small. If, however, the image blocks are dissimilar, these values will be large. Hence, typical video coders find the best match for a macroblock by selecting the motion vector that produces either the smallest MSE or the smallest MAD. In other words, the block associated to the best motion vector is the one closest to the given macroblock in an MSE or MAD sense.

Unfortunately, the MSE and MAD distortion measures do not take into account the cost in bits of actually encoding the vector. For example, a given motion vector may minimize the MSE, but it may be very costly to encode with bits, so it may not be the best choice from a coding standpoint.

To deal with this, advanced encoders such as those described by Telenor use rate-distortion (“RD”) criteria of the type “distortion+L*Bits” to select the best motion vector. The value of “distortion” is typically the MSE or MAD, “L” is a constant that depends on the compression level (i.e., the quantization step size), and “Bits” is the number of bits required to code the motion vector. In general, any RD criteria of this type would work with the present invention. However, in the present invention “Bits” include the bits needed for encoding the vector and those for encoding the accuracy of the vector. In fact, some candidates can have several “Bits” values, because they can have several accuracy modes. For example, the candidate at location (½, −½) can be thought of having ½ or ⅙-pixel accuracy.

Fast-Search AMA Search Strategy

As shown in **1**, which is centered on V_{1 } **108**. V_{2 }is then set to denote the candidate that has the smallest RD cost (i.e., the best of the eight previous vectors and V_{1}) **110**. Next, the encoder checks the eight motion vector locations in a grid of ⅙-pixel resolution of square radius **1** that is now centered on V_{2 } **112**. If V_{2 }has the smallest RD cost **114**, the encoder stops its search and selects V_{2 }as the motion vector for the block. Otherwise, V_{3 }is set to denote the best motion vector of the eight **116**. The encoder then searches for a new motion vector candidate in the grid of ⅙-pixel resolution of square radius **1** that is centered on V_{3 } **118**. It should be noted that some of the candidates in this grid have already been tested and can be skipped. The candidate with the smallest RD cost in this last step is selected as the motion vector for the block **120**.

Experimental data has shown that, on average, this simple fast search strategy typically checks the RD cost of about eighteen locations in sub-pixel space (ten more than Telenor's search strategy), and hence the overall computational complexity is only moderately increased.

The experimental data discussed below in connection with

Alternate embodiments of the invention replace one or more of the steps **108**-**120**. These embodiments have also been effective and have further reduced the number of motion vector candidates to check in the sub-pixel velocity space.

**112** is replaced by one of three possible scenarios. First, if the best motion vector candidate from step **110** is at the center of V_{1 }(the “integer-pel vector”) **130**, then the encoder checks three candidates of ⅓-pel accuracy between the center vector and the ½-pel location with the next lowest RD cost **132**. Second, if the best motion vector candidate from step **110** is a corner vector **134**, then, the encoder checks the four vector candidates of ⅓-pel accuracy that are closest to such corner **136**. Third, if the best motion vector candidate from step **110** is between two corners **138**, then, the encoder determines which of these two corners has lower RD cost and checks the four vector candidates of ⅓-pel accuracy that are closest to the line between such corner and the best candidate from step **110** **140**. It should be noted that in implementing this process step **138** may be unnecessary because if V_{2 }is neither at the center or a corner vector, then it would necessarily be between two corners. If the encoder is set to find motion vectors with ⅓-pixel accuracy, **114**.

Computation And Memory Savings

Because step **108** checks only motion vector candidates of ½-pixel accuracy, the computation and memory requirements for the hardware or software implementation are significantly reduced. To be specific, in a smart implementation embodiment of this fast-search the reference frame is interpolated by 2×2 in order to obtain the RD costs for the ½-pel vector candidates. A significant amount of fast (or cache) memory for a hardware or software encoder is saved as compared to Telenor's approach that needed to interpolate the reference frame by 3×3. In comparison to the Telenor encoder, this is a cache memory savings of 94, or a factor of 2.25. The few additional interpolations can be done later on a block-by-block basis.

Additionally, since the interpolations in step **108** are used to direct the search towards the lower values of the RD cost function, a complex filter is not needed for these interpolations. Accordingly, computation power may be saved by using a simple bilinear filter for step **108**.

Also, other key coding decisions such as selecting the mode of a macroblock (e.g., 16×16, four-8×8, etc.) can be done using the ½-pel vectors because such decisions do not benefit significantly from using higher accuracies. Then, the encoder can use a more complex cubic filter to interpolate the required sub-pixel values for the few additional vector candidates to check in the remaining steps. Since the macroblock mode has already been chosen, these final interpolations only need to be done for the chosen mode.

Use of multiple-filters obtained computation savings of over twenty percent in running time on a Sparc Ultra 10 Workstation in comparison to Telenor's approach, which uses a cubic interpolation all the time. Additionally, the fast-memory requirements were reduced by nearly half. Also, there was little or no loss in compression performance. Comparing one preferred embodiment of the fast-search, Benzler's technique requires about 70 interpolations per pixel in the Telenor encoder and the present invention requires only about 7 interpolations per pixel.

Coding The Motion Vector And Accuracies With Bits

Once the best motion vector and accuracy are determined, the encoder encodes both the motion vector and accuracy values with bits. One approach is to encode the motion vector with a given accuracy (e.g., half-pixel accuracy) and then add some extra bits for refining the vector to the higher motion accuracy. This is the strategy suggested by B. Girod, but it is sub-optimal in a rate-distortion sense.

In one preferred embodiment of the present invention, the accuracy of the motion vector for a macroblock is first encoded using a simple code such as the one given in Table 1. Any other table with code lengths {1, 2, 2} could be used as well. The bit rate could be further reduced using a typical DPCM approach.

TABLE 1 | ||

VLC table to indicate the accuracy mode for a given macroblock. | ||

Code | Motion Accuracy | |

1 | ½-pel | |

01 | ⅓-pel | |

11 | ⅙-pel | |

Observe that this code is the fourth entry (code number 3) of H26L's VLC table in [6]. |

Next, the value of the vector/s in the respective accuracy space is encoded. These bits can be obtained from entries of a single VLC table such as the one used in the H26L codec. The key idea is that these bits are interpreted differently depending on the motion accuracy for the macroblock. For example, if the motion accuracy is ⅓ and the code bits for the X component of the difference motion vector are 00001

^{1}, the X component of the vector is Vx=⅔. If the accuracy is ½, such code corresponds to Vx=1. Observe that this code is the fourth entry (code number 3) of H26L's VLC table in [6].

Compared to the Benzler method for encoding the motion vectors with a variable length code (“VLC”) table that could be used for encoding ½and ¼pixel accurate vectors, the method of the present invention can be used for encoding vectors of any motion accuracy and the table can be interpreted differently at each frame and macroblock. Further, the general method of the present invention can be used for any motion accuracy, not necessarily those that are multiples of each other or those that are of the type 1/n (with n an integer). The number of increments in the given sub-pixel space is simply counted and the bits in the associated entry of the table is used as the code.

From the decoder's viewpoint, once the motion accuracy is decoded, the motion vector can also be easily decoded. After that, the associated block in the previous frame is reconstructed using a typical 4-tap cubic interpolator. There is a different 4-tap filter for each motion accuracy.

The AMA does not increase decoding complexity, because the number of operations needed to reconstruct the predicted block are the same, regardless of the motion accuracy.

Experimental Results

TABLE 2 | |||||

Description of the Experiments | |||||

Video sequence | FIG. # | Resolution | Frame rate | ||

Container | FIG. 8 | QCIF | 10 | ||

News | FIG. 9 | QCIF | 10 | ||

Mobile | FIG. 10 | QCIF | 10 | ||

FIG. 11 | SIF | 15 | |||

Garden | FIG. 12 | QCIF | 15 | ||

Tempete | FIG. 13 | SIF | 15 | ||

FIG. 14 | QCIF | 15 | |||

Paris Shaked | FIG. 15 | QCIF | 10 | ||

The video sequences are commonly used by the video coding community, except for “Paris Shaked.” The latter is a synthetic sequence obtained by shifting the well-known sequence “Paris” by a motion vector whose X and Y components take a random value within [−1,1]. This synthetic sequence simulates small movements caused by a hand-held camera in a typical video phone scene.

Comparison Of Full-Search And Fast-Search AMA

The experimental results shown in

Combining AMA And Multiple Reference Frames

In the plot shown in

The experiments show that the gains with AMA add to those obtained using multiple reference frames. The gain from AMA in the one-reference case can be measured by comparing the curve labeled with a “+” (Telenor AMA+c+1 r) with the curve labeled with an “x” (Telenor ⅓+1r), and the gain in the five-reference case can be measured between the curve labeled with a “diamond” (Telenor AMA+c+5r) with the curve labeled with a “*” (Telenor ⅓+5r).

It should be noted that the present invention may be implemented at the frame level so that different frames could use different motion accuracies, but within a frame all motion vectors would use the same accuracy. Preferably in this embodiment the motion vector accuracy would then be signaled only once at the frame layer. Experiments have shown that using the best, fixed motion accuracy for the whole frame should also produce compression gains as those presented here for the macroblock-adaptive case.

In another frame-based embodiment the encoder could do motion compensation on the entire frame with the different vector accuracies and then select the best accuracy according to the RD criteria. This approach is not suitable for pipeline, one-pass encoders, but it could be appropriate for software-based or more complex encoders. In still another fame-based embodiment, the encoder could use previous statistics and/or formulas to predict what will be the best accuracy for a given frame (e.g., the formulas set forth in the Ribas work or a variation thereof can be used). This approach would be well-suited for one-pass encoders, although the performance gains would depend on the precision of the formulas used for the prediction.

The terms and expressions which have been employed in the foregoing specification are used therein as terms of description and not of limitation, and there is no intention, in the use of such terms and expressions, of excluding equivalents of the features shown and described or portions thereof, it being recognized that the scope of the invention is defined and limited only by the claims that follow.

## Claims (28)

_{1 }to find a best motion vector V

_{2 }using a first criteria;

_{2 }to find a best motion vector V

_{3 }using a second criteria;

_{3 }to find said best motion vector of said macroblock using a third criteria; and

_{1 }to find a best motion vector V

_{2 }further comprising the step of searching a first set of eight motion vector candidates in a grid of ½-pixel resolution of square radius

**1**centered on V

_{1 }to find a best motion vector V

_{2}.

_{2 }to find a best motion vector V

_{3 }further comprising the step of searching a second set of eight motion vector candidates in a grid of ⅙-pixel resolution of square radius

**1**centered on V

_{2 }to find a best motion vector V

_{3}.

_{2 }as the motion vector for the macroblock if V

_{2 }has the smallest rate-distortion cost and skipping step (c) of

_{3 }to find said best motion vector of said macroblock further comprising the step of searching a third set of eight motion vector candidates in a grid of ⅙-pixel resolution of square radius

**1**centered on V

_{3 }to find said best motion vector of said macroblock.

_{3 }to find said best motion vector of said macroblock further comprising the step of skipping motion vector candidates of said third set of motion vector candidates that have already been tested.

_{2 }to find a best motion vector V

_{3 }further comprising the steps of:

_{2 }and a ½-pel location with the next lowest rate-distortion cost if V

_{2 }is at the center;

_{2 }if V

_{2 }is a corner vector; and

_{2 }is between two corners vectors.

_{1 }using a first criteria to find a best motion vector V

_{2 }using a first filter to do a first interpolation;

_{2 }using a second criteria to find a best motion vector V

_{3 }using a second filter to do a second interpolation; and

_{3 }using a third criteria to find said best motion vector of said macroblock using a third filter to do a third interpolation;

_{1 }using a first criteria to find a best motion vector V

_{2};

_{2 }using a second criteria to find a best motion vector V

_{3}; and

_{3 }using a third criteria to find said best motion vector of said macroblock;

_{1 }to find a best motion vector V

_{2};

_{2 }to find a best motion vector V

_{3};

_{3 }to find said best motion vector of said macroblock, and

_{2 }as the motion vector for the macroblock if V

_{2 }has the smallest rate-distortion cost and skipping step (c).

## Priority Applications (4)

Application Number | Priority Date | Filing Date | Title |
---|---|---|---|

US14610299 true | 1999-07-27 | 1999-07-27 | |

US09615791 US6968008B1 (en) | 1999-07-27 | 2000-07-13 | Methods for motion estimation with adaptive motion accuracy |

US11984661 USRE45014E1 (en) | 1999-07-27 | 2007-11-20 | Methods for motion estimation with adaptive motion accuracy |

US13289902 USRE44012E1 (en) | 1999-07-27 | 2011-11-04 | Methods for motion estimation with adaptive motion accuracy |

## Applications Claiming Priority (1)

Application Number | Priority Date | Filing Date | Title |
---|---|---|---|

US13289902 USRE44012E1 (en) | 1999-07-27 | 2011-11-04 | Methods for motion estimation with adaptive motion accuracy |

## Related Parent Applications (1)

Application Number | Title | Priority Date | Filing Date | |
---|---|---|---|---|

US09615791 Reissue US6968008B1 (en) | 1999-07-27 | 2000-07-13 | Methods for motion estimation with adaptive motion accuracy |

## Publications (1)

Publication Number | Publication Date |
---|---|

USRE44012E1 true USRE44012E1 (en) | 2013-02-19 |

# Family

## ID=47683004

## Family Applications (1)

Application Number | Title | Priority Date | Filing Date |
---|---|---|---|

US13289902 Active USRE44012E1 (en) | 1999-07-27 | 2011-11-04 | Methods for motion estimation with adaptive motion accuracy |

## Country Status (1)

Country | Link |
---|---|

US (1) | USRE44012E1 (en) |

## Cited By (2)

Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|

US9371099B2 (en) | 2004-11-03 | 2016-06-21 | The Wilfred J. and Louisette G. Lagassey Irrevocable Trust | Modular intelligent transportation system |

USRE46468E1 (en) * | 1999-07-27 | 2017-07-04 | Sharp Kabushiki Kaisha | Methods for motion estimation with adaptive motion accuracy |

## Citations (28)

Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|

JPS6442295A (en) | 1987-08-10 | 1989-02-14 | Seiko Epson Corp | Memory card mounting structure |

US4864393A (en) | 1987-06-09 | 1989-09-05 | Sony Corp. | Motion vector estimation in television images |

US4937666A (en) | 1989-12-04 | 1990-06-26 | Bell Communications Research, Inc. | Circuit implementation of block matching algorithm with fractional precision |

EP0420653A2 (en) | 1989-09-29 | 1991-04-03 | Victor Company Of Japan, Ltd. | Motion picture data coding/decoding system having motion vector coding unit and decoding unit |

JPH04264889A (en) | 1991-02-19 | 1992-09-21 | Victor Co Of Japan Ltd | Inter-motion-compensating-frame encoder |

JPH0795585A (en) | 1993-09-17 | 1995-04-07 | Sony Corp | Moving vector detector |

US5408269A (en) | 1992-05-29 | 1995-04-18 | Sony Corporation | Moving picture encoding apparatus and method |

US5489949A (en) | 1992-02-08 | 1996-02-06 | Samsung Electronics Co., Ltd. | Method and apparatus for motion estimation |

JPH08116532A (en) | 1994-10-14 | 1996-05-07 | Graphics Commun Lab:Kk | Image decoding system and device therefor |

US5610658A (en) | 1994-01-31 | 1997-03-11 | Sony Corporation | Motion vector detection using hierarchical calculation |

GB2305569A (en) | 1995-09-21 | 1997-04-09 | Innovision Res Ltd | Motion compensated interpolation |

US5623313A (en) | 1995-09-22 | 1997-04-22 | Tektronix, Inc. | Fractional pixel motion estimation of video signals |

JPH09153820A (en) | 1995-11-29 | 1997-06-10 | Sharp Corp | Encoding/decoding device |

US5682205A (en) | 1994-08-19 | 1997-10-28 | Eastman Kodak Company | Adaptive, global-motion compensated deinterlacing of sequential video fields with post processing |

US5694179A (en) | 1994-12-23 | 1997-12-02 | Electronics And Telecommunications Research Institute | Apparatus for estimating a half-pel motion in a video compression method |

US5754240A (en) | 1995-10-04 | 1998-05-19 | Matsushita Electric Industrial Co., Ltd. | Method and apparatus for calculating the pixel values of a block from one or two prediction blocks |

US5767907A (en) | 1994-10-11 | 1998-06-16 | Hitachi America, Ltd. | Drift reduction methods and apparatus |

WO1998041011A1 (en) | 1997-03-12 | 1998-09-17 | Matsushita Electric Industrial Co., Ltd. | Hdtv downconversion system |

US5844616A (en) | 1993-06-01 | 1998-12-01 | Thomson Multimedia S.A. | Method and apparatus for motion compensated interpolation |

DE19730305A1 (en) | 1997-07-15 | 1999-01-21 | Bosch Gmbh Robert | A method for generating an improved video signal during the motion estimation of picture sequences, in particular of a predicted signal for moving pictures with motion-compensating prediction |

JPH1146364A (en) | 1997-07-28 | 1999-02-16 | Victor Co Of Japan Ltd | Motion compensation coding and decoding device, and coding and decoding device |

JPH1155673A (en) | 1997-07-31 | 1999-02-26 | Victor Co Of Japan Ltd | Motion vector decoder, coding method and decoding method |

US5987181A (en) | 1995-10-12 | 1999-11-16 | Sharp Kabushiki Kaisha | Coding and decoding apparatus which transmits and receives tool information for constructing decoding scheme |

US6005509A (en) | 1997-07-15 | 1999-12-21 | Deutsches Zentrum Fur Luft-Und Raumfahrt E.V. | Method of synchronizing navigation measurement data with S.A.R radar data, and device for executing this method |

EP1073276A2 (en) | 1999-07-27 | 2001-01-31 | Sharp Kabushiki Kaisha | Methods for motion estimation with adaptive motion accuracy |

US6249318B1 (en) | 1997-09-12 | 2001-06-19 | 8×8, Inc. | Video coding/decoding arrangement and method therefor |

US6269174B1 (en) * | 1997-10-28 | 2001-07-31 | Ligos Corporation | Apparatus and method for fast motion estimation |

US6714593B1 (en) | 1997-10-21 | 2004-03-30 | Robert Bosch Gmbh | Motion compensating prediction of moving image sequences |

## Patent Citations (34)

Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|

US4864393A (en) | 1987-06-09 | 1989-09-05 | Sony Corp. | Motion vector estimation in television images |

JPS6442295A (en) | 1987-08-10 | 1989-02-14 | Seiko Epson Corp | Memory card mounting structure |

EP0420653A2 (en) | 1989-09-29 | 1991-04-03 | Victor Company Of Japan, Ltd. | Motion picture data coding/decoding system having motion vector coding unit and decoding unit |

US5105271A (en) | 1989-09-29 | 1992-04-14 | Victor Company Of Japan, Ltd. | Motion picture data coding/decoding system having motion vector coding unit and decoding unit |

US4937666A (en) | 1989-12-04 | 1990-06-26 | Bell Communications Research, Inc. | Circuit implementation of block matching algorithm with fractional precision |

JPH04264889A (en) | 1991-02-19 | 1992-09-21 | Victor Co Of Japan Ltd | Inter-motion-compensating-frame encoder |

US5489949A (en) | 1992-02-08 | 1996-02-06 | Samsung Electronics Co., Ltd. | Method and apparatus for motion estimation |

US5408269A (en) | 1992-05-29 | 1995-04-18 | Sony Corporation | Moving picture encoding apparatus and method |

US5844616A (en) | 1993-06-01 | 1998-12-01 | Thomson Multimedia S.A. | Method and apparatus for motion compensated interpolation |

JPH0795585A (en) | 1993-09-17 | 1995-04-07 | Sony Corp | Moving vector detector |

US5610658A (en) | 1994-01-31 | 1997-03-11 | Sony Corporation | Motion vector detection using hierarchical calculation |

US5682205A (en) | 1994-08-19 | 1997-10-28 | Eastman Kodak Company | Adaptive, global-motion compensated deinterlacing of sequential video fields with post processing |

US5767907A (en) | 1994-10-11 | 1998-06-16 | Hitachi America, Ltd. | Drift reduction methods and apparatus |

JPH08116532A (en) | 1994-10-14 | 1996-05-07 | Graphics Commun Lab:Kk | Image decoding system and device therefor |

US5694179A (en) | 1994-12-23 | 1997-12-02 | Electronics And Telecommunications Research Institute | Apparatus for estimating a half-pel motion in a video compression method |

US20010017889A1 (en) | 1995-09-21 | 2001-08-30 | Timothy John Borer | Motion compensated interpolation |

GB2305569A (en) | 1995-09-21 | 1997-04-09 | Innovision Res Ltd | Motion compensated interpolation |

US5623313A (en) | 1995-09-22 | 1997-04-22 | Tektronix, Inc. | Fractional pixel motion estimation of video signals |

US5754240A (en) | 1995-10-04 | 1998-05-19 | Matsushita Electric Industrial Co., Ltd. | Method and apparatus for calculating the pixel values of a block from one or two prediction blocks |

US5987181A (en) | 1995-10-12 | 1999-11-16 | Sharp Kabushiki Kaisha | Coding and decoding apparatus which transmits and receives tool information for constructing decoding scheme |

JPH09153820A (en) | 1995-11-29 | 1997-06-10 | Sharp Corp | Encoding/decoding device |

WO1998041011A1 (en) | 1997-03-12 | 1998-09-17 | Matsushita Electric Industrial Co., Ltd. | Hdtv downconversion system |

US6005509A (en) | 1997-07-15 | 1999-12-21 | Deutsches Zentrum Fur Luft-Und Raumfahrt E.V. | Method of synchronizing navigation measurement data with S.A.R radar data, and device for executing this method |

DE19730305A1 (en) | 1997-07-15 | 1999-01-21 | Bosch Gmbh Robert | A method for generating an improved video signal during the motion estimation of picture sequences, in particular of a predicted signal for moving pictures with motion-compensating prediction |

WO1999004574A1 (en) | 1997-07-15 | 1999-01-28 | Robert Bosch Gmbh | Interpolation filtering method for accurate sub-pixel motion assessment |

US7224733B1 (en) | 1997-07-15 | 2007-05-29 | Robert Bosch Gmbh | Interpolation filtering method for accurate sub-pixel motion assessment |

US6205176B1 (en) | 1997-07-28 | 2001-03-20 | Victor Company Of Japan, Ltd. | Motion-compensated coder with motion vector accuracy controlled, a decoder, a method of motion-compensated coding, and a method of decoding |

JPH1146364A (en) | 1997-07-28 | 1999-02-16 | Victor Co Of Japan Ltd | Motion compensation coding and decoding device, and coding and decoding device |

JPH1155673A (en) | 1997-07-31 | 1999-02-26 | Victor Co Of Japan Ltd | Motion vector decoder, coding method and decoding method |

US6249318B1 (en) | 1997-09-12 | 2001-06-19 | 8×8, Inc. | Video coding/decoding arrangement and method therefor |

US6714593B1 (en) | 1997-10-21 | 2004-03-30 | Robert Bosch Gmbh | Motion compensating prediction of moving image sequences |

US6269174B1 (en) * | 1997-10-28 | 2001-07-31 | Ligos Corporation | Apparatus and method for fast motion estimation |

EP1073276A2 (en) | 1999-07-27 | 2001-01-31 | Sharp Kabushiki Kaisha | Methods for motion estimation with adaptive motion accuracy |

US6968008B1 (en) | 1999-07-27 | 2005-11-22 | Sharp Laboratories Of America, Inc. | Methods for motion estimation with adaptive motion accuracy |

## Non-Patent Citations (21)

Title |
---|

Bernd Girod, Motion-Compensating Prediction with Fractional-Pel Accuracy, IEEE Transactions on Communications, vol. 41, No. 4, pp. 604-612, (Apr. 1993). |

Chan et al., "Review of Block Matching Based Motion Estimation Algorithms for Video Compression," Electrical and Computer Engineering, Canadian Conference on Vancouver, BC, Canada 14-17, Sep. 14, 1993, New York, NY, USA, IEEE, pp. 151-154, XP010117942. |

Ebrahimi et al., "A video codec based on perceptually derived and localized wavelet transform for mobile applications," Signal Processing Theories and Applications, Brussels, Aug. 24-27, 1992, vol. 3, pp. 1361-1364, XP000356495. |

Enhancement for the Telenor proposal for H.26L, ITU-Telecommunications Standardization Section, Q. 15/SG16, doc. Q15-G-25. Monterey. (Feb. 1999). |

Extended European Search Report, dated May 24, 2011, for European Application No. 10013511.0. |

Jordi Ribas-Corbera and David L. Neuhoff, On the Optimal Motion Vector Accuracy for Block-Based Motion-Compensated Video Coders, Proc. IST/SPIE Digital Video Compression: Algorithms and Technologies, pp. 302-314, San Jose, (Feb. 1996). |

Joshi et al., "Lossy Encoding of Motion Vectors Using Entropy-Constrained Vector Quantization," IEEE Comp. Soc. Press, US, Proceedings of the International Conference on Image Processing (ICIP). Washington, Oct. 23-26, 1995, vol. 3, Los Alamitos, pp. 109-112, XP010197142. |

Lee, "Rate-Distortion Optimized Motion Smoothing for MPEG-2 Encoding," Proceedings of the International Conference on Image Processing (ICIP), Los Alamitos, CA, vol. 2, Jan. 1, 1997, pp. 45-48, XP000914163. |

Netravali et al., "A Codec for HDTV," IEEE Transactions on Consumer Electronics, IEEE Service Center, New York, NY, US, vol. 38. No. 3, Aug. 1, 1992, pp. 325-340, XP000311862. |

Ohm, "Motion-Compensated 3-D Subband Coding with Multiresolution Representation of Motion Parameters," IEEE, Proceedings of the International Conference on Image Processing (ICIP), Nov. 13-16, 1994, vol. 3, Conf. 1, pp. 250-254, XP010146425. |

Pang et al., "Optimum Loop Filter in Hybrid Coders", IEEE Transactions on Circuits and Systems for Video Technology, vol. 4, No. 2, pp. 158-167, Apr. 1994, XP000489688. |

Response to Call for Proposals for H.26L, ITU-Telecommunications Standardization Section, Q.15/SG16.doc Q15-F-11, Seoul, (Nov. 1998). |

Response to Call for Proposals for H.26L, ITU—Telecommunications Standardization Section, Q.15/SG16.doc Q15-F-11, Seoul, (Nov. 1998). |

Shen et al., "Adaptive motion accuracy (AMA) in Telenor's proposal", ITU-Telecommunications Standardization Sector, Study Group 16, Video Coding Experts Group (Question 15), Q15-H-20, Eighth Meeting, Berlin, Aug. 3-6, 1999, XP030002958. |

Shen et al., "Adaptive motion accuracy (AMA) in Telenor's proposal", ITU—Telecommunications Standardization Sector, Study Group 16, Video Coding Experts Group (Question 15), Q15-H-20, Eighth Meeting, Berlin, Aug. 3-6, 1999, XP030002958. |

Smita Gupta and Allen Gersho, On Fractional Pixel Motion Estimation, Proc. SPIE VCIP, vol. 2094, pp. 408-419, Cambridge, (Nov. 1993). |

Ulrich Benzler, Performance Evaluation of a Reduced Complexity Implementation for Quarter Pel Motion Compensation, ISO/IEC JTC1/SC29/WG11 Coding of Moving Pictures and Audio, MPEG 97/3146. San Jose, (Jan. 1998). |

Ulrich Benzler, Proposal for a new core experiment on prediction enhancement at higher bitrates, ISO/IEC JTC1/SC29/VVG11 Coding of Moving Pictures and Audio, MEPEG 97/1827, Sevilla, (Feb. 1997). |

Wedi, "Results of core experiment on Adaptive Motion Accuracy (AMA) with 1/2, 1/4 and 1/8-pel accuracy," ITU Study Group 16-Video Coding Experts Group, May 16, 2000, pp. 1-9, XP002301984. |

Wedi, "Results of core experiment on Adaptive Motion Accuracy (AMA) with 1/2, 1/4 and 1/8-pel accuracy," ITU Study Group 16—Video Coding Experts Group, May 16, 2000, pp. 1-9, XP002301984. |

Xiaoming Li and Cesar Gonzales, A Locally Quadratic Model of the Motion Estimation Error Criterion Function and Its Application to Subpixel Interpolations. IEEE Transactions on Circuits and Systems for Video Technology, vol. 6, No. 1, (Feb. 1996). |

## Cited By (2)

Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|

USRE46468E1 (en) * | 1999-07-27 | 2017-07-04 | Sharp Kabushiki Kaisha | Methods for motion estimation with adaptive motion accuracy |

US9371099B2 (en) | 2004-11-03 | 2016-06-21 | The Wilfred J. and Louisette G. Lagassey Irrevocable Trust | Modular intelligent transportation system |

## Similar Documents

Publication | Publication Date | Title |
---|---|---|

US5786860A (en) | High speed block matching for bi-directional motion vector estimation | |

US7336720B2 (en) | Real-time video coding/decoding | |

US7599438B2 (en) | Motion vector block pattern coding and decoding | |

US7577198B2 (en) | Number of reference fields for an interlaced forward-predicted field | |

Lim et al. | Text description of joint model reference encoding methods and decoding concealment methods | |

US7110459B2 (en) | Approximate bicubic filter | |

US20040114688A1 (en) | Device for and method of estimating motion in video encoder | |

US20090129472A1 (en) | Method and Apparatus for Performing Motion Estimation | |

US7747094B2 (en) | Image encoder, image decoder, image encoding method, and image decoding method | |

US7620106B2 (en) | Joint coding and decoding of a reference field selection and differential motion vector information | |

US6757330B1 (en) | Efficient implementation of half-pixel motion prediction | |

US20080212678A1 (en) | Computational reduction in motion estimation based on lower bound of cost function | |

US20030128761A1 (en) | Image and video coding | |

US20090022220A1 (en) | Method and apparatus for enhanced video coding | |

US7280597B2 (en) | System and method for determining coding modes, DCT types and quantizers for video coding | |

US20050201633A1 (en) | Method, medium, and filter removing a blocking effect | |

US20030112864A1 (en) | Method for sub-pixel value interpolation | |

US20030072374A1 (en) | Method for motion vector estimation | |

Flierl et al. | Generalized B pictures and the draft H. 264/AVC video-compression standard | |

US7606308B2 (en) | Signaling macroblock mode information for macroblocks of interlaced forward-predicted fields | |

US7317839B2 (en) | Chroma motion vector derivation for interlaced forward-predicted fields | |

US20040120401A1 (en) | Motion estimation engine with parallel interpolation and search hardware | |

US7852936B2 (en) | Motion vector prediction in bi-directionally predicted interlaced field-coded pictures | |

RU2310231C2 (en) | Space-time prediction for bi-directional predictable (b) images and method for prediction of movement vector to compensate movement of multiple images by means of a standard | |

US20050053142A1 (en) | Hybrid motion vector prediction for interlaced forward-predicted fields |