CN101090491B - Be used for the block-based motion estimation algorithm of the enhancing of video compress - Google Patents

Abstract

A kind of method, system and software are proposed, for obtaining the piece of first image similar with the piece of the second image (" reference picture "). Piece to the first image is sequentially processed, and is the each multiple position candidate that obtain in the second image, and is each position candidate assessment cost function. Each position candidate in the second image is to be replaced by the motion vector separately of the piece from the first image. In a first aspect of the present invention, described cost function is the function of the motion vectors of the following piece (also there is no the piece of the first processed image) of the first image. In second aspect present invention, described motion vector is provided by positional value, and described positional value is not all both full-pixel space, half-pix space or 1/4 pixel space.

Description

Be used for the block-based motion estimation algorithm of the enhancing of video compress

Related application

The application's request in the priority of the U.S. Provisional Patent Application 60/691,181 of on June 17th, 2005 application,At this, its entirety is incorporated herein by reference.

Technical field

The present invention relates generally to the method and system for data signal compression, coding and expression, and more definitelySay, the present invention relates to use the method and system of multiframe estimation (ME). The invention further relates to a kind of computer programProduct, such as recording medium, the programmed instruction that carrying can be read by computing equipment, to make described computing equipment carry out basisA kind of method of the present invention.

Background technology

Due to the huge size of the raw digital video data (or image sequence) that used by modern multimedia application, mustMust compress to can transmit and store these data this data. There are many important video compression standards, compriseISO/IECMPEG-1, MPEG-2, MPEG-4 standard and ITU-TH.261, H.263, standard H.264. ISO/IECMPEG-1/2/4 standard apply to widely show business with distribution film, comprise the digital video of video compact disc or VCD (MPEG-1)Broadcast, digital video disk or digital multi-purpose CD or DVD (MPEG-2), recordable DVD (MPEG-2), digital videoBroadcast, DVB or DVB (MPEG-2), video request program or VOD (MPEG-2), high-definition television in USOr HDTV (MPEG-2) etc. MPEG-4 standard is more advanced than MPEG-2, can under lower bit rate, realize high-qualityAmount video, this make its be very suitable for internet, digital wireless network (for example 3G network), MMS (Multimedia Message Service) (fromThe MMS standard of 3GPP) etc. on video stream. MPEG-4 is by fine definition DVD of future generation (HD-DVD) standard and MMS markAccurate institute accepts. ITU-TH.261/3/4 standard design is for low deferred telegram phonovision and video conferencing system. In early days H.261Standard design is at p^*Under 64kbit/s, work, p=1,2 .., 31. H.263 the standard in later stage is extremely successful, is used widelyIn advance TV conference system, and for the video stream of broadband network and wireless network, wherein wireless network comprisesMMS (Multimedia Message Service) (MMS) in 2.5G and 3G network and other networks. Newest standards, H.264 (also referred to as MPEG-4Version 10, or MPEG-4AVC) be the video compression standard of current state-of-the-art. It is so powerful so that MPEG decisionDevelop jointly with the ITU-T in the framework of JointVideoTeam (JVT). H.264 new standard is called in ITU-T,And be known as MPEG-4 advanced video coding (MPEG-4AVC), or MPEG-4 version 10. H.264 for HD-DVD standard,Directly video broadcasting (DVB) standard and may be used for MMS standard. Based on H.264, currently developing and be called audiovisual in ChinaThe relevant criterion of standard (AVS). AVS1.0 is designed for high-definition television (HDTV). AVS-M is designed for mobile application.H.264 have and exceed MPEG-1/2/4 and the H.261/3 target of standard and subjective video quality. Except using, integer 4x4 is discreteCosine transform (DCT) replaces beyond traditional 8x8DCT, and H.263 the basic coding class of algorithms H.264[1] is similar to or MPEG-4, and also have extra feature, comprise the inter-frame forecast mode of I frame, for the multiple block size of motion estimation/compensation andMultiple reference frame, for deblocking effect (in-in 1/4th pixel precisions of estimation, loopLoopdeblocking) wave filter, context adaptive binary arithmetic coding (contextadaptivebinaryArithmeticcoding), etc.

Estimation is the core of most of video compression standards (such as MPEG-1/2/4 and H.261/3/4), itsThe redundancy that employs one's time to the best advantage, therefore its performance directly affect video coding system compression efficiency, well as subjective video quality andCoding rate.

Estimate in (BMME) at block matching motion, in ME, the most general to the distortion between current block and reference blockMeasurement be absolute difference and (SAD), for N × N piece, be defined as

$\mathrm{SAD}(\mathrm{mvx},\mathrm{mvy})=\underset{m=0,n=0}{\overset{N-1}{\mathrm{\Σ}}}|F_t(x+m,y+n)-F_{t-1}(x+m+\mathrm{mvx},y+n+\mathrm{mvy})|$

Wherein F_tPresent frame, F_t-1Be standard frame, (mvx, mvy) represents current motion vector (MV). For width=X,Highly=Y, and the frame of block size=N × N, in hunting zone ±WIn need to assess to search optimum fortune to SADThe search point sum of dynamic vector equals:

$\left(\frac{X}{N}\right)\left(\frac{Y}{N}\right){(2W+1)}^2,$

For X=352, Y=288, in the situation of N=16 and W=32, it equals 1673100. This is at Video codingIn device, consume the huge numerical value of huge computing capability. Having proposed many fast algorithms [2]-[9] reduces in ME and searches forThe number of point, for example three step search (TSS) [11], 2D logarithm search [12], new three step search (NTSS) [3], MVFAST[7],And PMVFAST[2]. MVFAST and PMVFAST are better than first three algorithm significantly, because they use intermediate value motion vector pre-Survey device and depart from ME as search center implementation center, reduced thus the number of the position of MV coding by smooth motion vector field.

PMVFAST algorithm (it is the important improvement to MVFAST and other fast algorithm, and thereby by mpeg standard[10] accept) consider at first one group of MV fallout predictor, comprise intermediate value, zero point, the left side, top, top-right and previous common position(co-located) MV fallout predictor. Fig. 1 for example understand current block, left, top block, upper right square, upper right square andThe position of right (it is the following piece ＂ of ＂, that is, processed piece after current block). It calculates the SAD cost of each prediction. ?In development afterwards, PMVFAST is modified to calculate RD (rate distortion) cost [13] and substitute the following cost letter of useThe SAD cost of number:

J(m，λ_motion)＝SAD(s，c(m))+λ_motion(R(m-p))(1)

Wherein s is raw video signal, and c is reference video signal, and m is current MV, and p is the intermediate value MV prediction of current blockDevice, λ_motionBe Lagrange multiplier, R (m-p) represents the position for coded motion information. Next step in PMVFASTBe the MV fallout predictor of selecting to have minimum cost, and it is large to carry out number of executions according to the value of the minimum cost obtaining from MV fallout predictorDiamond search (diamondsearch) or little diamond search.

In definition current video coding standard, independent still major issue is to use sub-pix pr-set vector, comprises half pictureElement, 1/4 pixel or 1/8 pixel motion vector even, it provides the describing more accurately of motion, and can carryThe PSNR gain of the about 1dB estimating for whole pixel motion. Adopt half-pixel accuracy, motion vector can adopt equidistant positionValue, such as 0.0,0.5,1.0,1.5,2.0 etc. Adopt 1/4 pixel precision, motion vector can adopt such as 0.00,0.25,0.50,0.75,1.00,1.25,1.50,1.75,2.00 etc. and so on positional value. Adopt 1/8 pixel precision, motion vector canTo adopt such as 0.000,0.125,0.250,0.375,0.500,0.625,0.750,0.875,1.000,1.125,1.250,1.375,1.500,1.625,1.750,1.875,2.000 etc. and so on positional value.

As everyone knows, motion vector distribution trends towards off-centring, this means that motion vector trends towards approaching in the extremeIn (0,0). In Fig. 6 (a), this situation has been shown, its be depicted as (0,0) MV use completely search (FS) algorithmMotion vector distribution in Foreman sequence. In addition,, as shown in Fig. 6 (b), motion vector distribution is also to median prediction device (intermediate valueMV) skew, it is the intermediate value of left of motion vector, top block and upper right square shown in Figure 1. In addition, as Fig. 6 (c) instituteShow, motion vector is also to institute in the adjacent motion vectors in present frame (leftMV, topMV, topRightMV) and previous frameMotion vector (preMV) skew arranging, as shown in Fig. 6 (d). These can be all for the motion vector of current vector canThe fallout predictor of considering, and they can be used to PMVFAST.

Summary of the invention

The present invention seeks to be provided for the new and useful technology of estimation, it is applicable to data signal and pressesContracting, coding and the method and system of expressing.

Particularly, the present invention manages to provide new and useful effective exercise estimation technique, and it can for example be applied inMPEG-1, MPEG-2, MPEG-4, H.261, H.263, H.264 or in AVS or other associated video coding standard.

The realization of first aspect present invention based on such: although the estimation of PMVFAST algorithm is with compared with front technologyReally there is advantage, but it is not best. In principle, for the each frame in video, existence makes whole on the wholeMotion vector field { the m of the RD Least-cost of individual frame_i，j，i＝0..M-1，j＝0..N-1}：

$\mathrm{total}\_\mathrm{RD}\_\mathrm{Cost}=\underset{i=0}{\overset{M-1}{\mathrm{\Σ}}}\underset{j=0}{\overset{N-1}{\mathrm{\Σ}}}[\mathrm{SAD}(s_{i,j},c\left(m_{i,j}\right))+{\mathrm{\λ}}_{i,j}\left(R(m_{i,j}-p_{i,j})\right)]---\left(2\right)$

Wherein (i, j) is illustrated in (i, j) the individual piece in the frame that comprises M × N piece. For fixing Q_p(it is to quantize ginsengNumber), λ_i，j=λ=constant, and

p_i，j＝median(m_i，j-1，m_i-1，j，m_i-1，j+1)(3)

But, consider that whole RD costs of whole frame need exponential computational complexity simultaneously, this is unpractiaca.Thereby PMVFAST and other algorithm known are each only considers the only RD cost of a piece, instead of whole pieces in a frame.

Particularly, MVFAST or PMVFAST do not consider that this causes in the time deriving with respect to the motion vector of current blockThe variation in intermediate value MV fallout predictor of next piece. This can affect the level and smooth of whole motion vector field.

In general, a first aspect of the present invention is by improving the cost definition of PMVFAST and motion predictor candidateSelect and proposed a kind of new ME algorithm. Particularly, for each current block of the first image (current block can be 16 ×16,16 × 8,8 × 16,8 × 8,4 × 8,8 × 4,4 × 4 or other rectangle length, even non-rectangle), according to a cost functionSelect the second image (reference picture) similar, described cost function comprises two: (i) current block and the not phase of similarSeemingly measure the item of (for example SAD, SAE), and (ii) conduct is at least the prediction of the motion vector of the following piece to the first imageThe item of function.

Particularly, the algorithm proposing (will not be located by comprising current intermediate value MV fallout predictor and comprising future so farReason) the estimation intermediate value MV fallout predictor of encoding block, making to improve sports ground flatness becomes possibility.

Many variations of the present invention are possible. Particularly, described can have any size and any shape.

Can there be multiple the second images (being multiple reference values) and described search can be included in all the second imagesPosition candidate.

In addition, can by form together larger region in described the first image and adopt and determined by coded digitalMultiple sub-blocks that the coded sequence of justice is encoded are carried out described new cost function. These sub-blocks need not have formed objects orShape.

The present invention has on the other hand, and it can or independently use with first aspect present invention combination. In general, thisBright second aspect propose when to the first image (current block can be 16 × 16,16 × 8,8 × 16,8 × 8,4 × 8,8 × 4,4 ×4, or other rectangle size, even non-rectangle) when current block is encoded, use selected, have from known technologyThe motion vector of the positional value (i.e. assembly separately in two axis directions) of selecting in the different class value of middle used valueEncode.

Consider a possible motion vector predictor: (0,0). And the routine techniques of whole pixel allows motion vector to adoptSuch as-2.0 ,-1.0,0,1.0, the positional value of 2.0 grades, second aspect present invention proposes amendment close to one of described fallout predictorGroup possible position value. For the positional value 1.0 that approaches most 0, we can use another location value such as 0.85, can thereby makeThe positional value allowing will comprise-2.0 ,-0.85,0,0.85,2.0 etc. Its advantage is, in statistics, motion vector becomesTo close to 0. And, thereby by selecting the position close to 0 more, we will approach true motion vector more, andThereby can provide the better motion compensation that can cause high compression efficiency.

Thereby, in particular expression of the present invention, can be chosen as at least one direction of principal axis and select one group of possible positionValue, thus make them cannot all write Lm, wherein m=-..., 2 ,-1,0,1,2..., and L is constant (for example 1 pictureElement interval, 1/2 pixel separation, or 1/4 pixel separation); , positional value is inhomogeneous. Particularly, can be chosen as at least oneIndividual direction of principal axis is selected one group of possible position value, thereby makes them cannot all write m/n, wherein m=-..., and 2 ,-1,0,1,2 ... and n is 1 or 2 power.

Note, a second aspect of the present invention is not limited to chosen position value from one group of locus heterogeneous value;Compared with traditional positional value group, be also not limited to and only select two positional values close to zero. As example, at thisIn another example of bright second aspect, positional value 2.0 becomes 1.9, thereby admissible positional value will comprise-1.9，- 0.85，0，0.85，1.9Deng.

Thereby in the interchangeable particular expression of second aspect present invention, (be at least one described in direction) selects oneGroup possible position value, can write as LA to comprise_mOne or more positional values of m/n, wherein m=-..., 2 ,-1,0,1,2.. n is 1 or 2 power, L is constant (for example 1 pixel separation, 1/2 pixel separation, or 1/4 pixel separation), and A_mTo be less than1 is at least still 0.75 value (optionally different for different m values), is at least more preferably 0.80, and optimumSelection of land is at least 0.85.

We have found that A_mOptimum value depend on video.

An advantage of the specific embodiment of second aspect present invention is, the motion vector that they produce can adoptThe form type code identical with conventional algorithm encoded, and makes except the regular code of this positional value should be interpreted as respectively this embodimentWith possible position value outside. For example, if the positional value being used by specific embodiment be-1.9，-0.85，0，0.85，1.9Deng, the regular code of positional value 1.0 should be interpreted as 0.85, and the regular code of positional value 2.0 should be interpreted as 1.9, etc.

Method according to a second aspect of the invention can comprise the following steps: definition of search region, and in the described field of searchThe multiple position candidate of definition in territory, described multiple position candidate comprise by the new positional value of second aspect present invention definedOne group of multiple position. These positional values be from key position (for example, (0,0) motion vector position, motion vectors, etc.)The corresponding shift value of the position candidate at place. For each candidate motion vector, our calculation cost function, described cost letterNumber is the similarity measurement (example between the piece at the described candidate motion vector place in current block and the second image in the first imageAs SAD, SAE) function. Alternatively, it can also be the function of following motion vector: described candidate motion vector, current pre-Survey motion vector, and alternatively, as first aspect present invention, one or more future anticipation motion vectors. For example, optionalGround, as first aspect present invention, can given this cost function.

Brief description of the drawings

Now will only the embodiment of the present invention be described with reference to the following drawings as example, wherein:

Fig. 1 shows current block, left, top block, upper right square, upper right square and right;

Fig. 2 shows the exemplary distribution of Diff, its be | m_i，j+1—p_i，j+1| with | m_i，j—p_i，j+1| between difference;

Fig. 3 a shows the way of search of the huge diamond search using as first embodiment of the invention;

Fig. 3 b shows the way of search of the huge diamond search of the amendment as used in first embodiment of the invention;

Fig. 4 shows the way of search of the little diamond search as used in first embodiment of the invention;

Fig. 5 a and 5b compare the flatness in the MV territory of PMVFAST and first embodiment of the invention;

Fig. 6 shows configuring in (a) (0,0) MV, (b) intermediate value MV, (c) adjacent MV and previous frame in present frameBottom-right MV (PreBottomRightMV) in previous frame of MV, (d) use that search (FS) algorithm completely obtainsMotion vector distribution in Foreman sequence;

Fig. 7 is the flow chart of first embodiment of the invention.

Detailed description of the invention

First embodiment of the invention has adopted many features of PMVFAST algorithm, but is by considering several contiguous blocksInstead of only a piece comes PMVFAST (and other existing algorithm) in addition improved. Equation (2) and (3) illustrate, current blockThe selection of MV directly affects the RD cost of contiguous block, and described contiguous block comprises right (or (i, j+1) piece), lower-left square (or(i+1, j-1) piece), and lower block (or (i+1, j) piece). This is because current MV will affect these contiguous blocksTherefore and then affect the optimum movement vector of those pieces the MV that predicts, and. These are " future " pieces, because when processing current blockTime, also they are not carried out to estimation. We cannot side by side calculate with current block the optimal movement arrow of these following piecesAmount, because we need to calculate in whole frame according to equation (2) optimum movement vector of all simultaneously, this will answer very muchAssorted.

As an alternative, in order to evaluate the implication of selection of current MV of the current block on right or (i, j+1) piece,We can be that the RD cost function of the current block of equation (1) increases by one:

R(|m_i，j+1-p_i，j+1|)(4)

Wherein, m_i，j+1The optimum movement vector of right of the current MV based on current block, and p_i，j+1Based on currentThe intermediate value MV fallout predictor of right of the current MV of piece,

p_i，j+1＝median(m_i，j，m_i-1，j+1，m_i-1，j+2)(5)

But, the m in equation (4)_i，j+1Unknown, because the estimation of right (following piece) is not also carried out. SoAnd we notice, | m_i，j+1-p_i，j+1| can pass through | m_i，j-p_i，j+1| approximate well. Allow Diff be | m_i，j+1-p_i，j+1|-|m_i，j-p_i，j+1|. We test and study the distribution of Diff to many video measurement orders. Fig. 2 has shown Foreman orderThe probability density function (probability-distribution function) of row, as shown in Figure 2, typical result is seriously partial to zero. This means this twoIndividual amount in most of situation, be essentially identical (identical in about 70% situation, and about 23% in the situation that onlyDiffer 1). This means, | m_i，j-p_i，j+1| be | m_i，j+1-p_i，j+1| good approximate. Therefore, R (| m_i，j+1-p_i，j+1|) canBy W*R (| m_i，j-p_i，j+1|) be similar to wherein W > 0. Equally, we can add addition Item to generation to lower-left square and lower blockIn valency.

For (i, j) piece, allow medianMV represent to provide intermediate value MV fallout predictor by equation 3. Allow FmedianMV represent byThe following intermediate value MV fallout predictor (for the intermediate value MV fallout predictor of right) that equation (5) provides. Thereby FmedianMV is MV candidateFunction. At this, the first embodiment is called as " the strengthening motion vectors field adaptive random search " for (i, j) piece(E-PMVFAST). The step of this embodiment is as follows.

Each step of embodiment is as follows, is shown in Fig. 7.

To any candidate MV, as the cost of giving a definition.

cost(MV)＝SAD+λ*[w*R(MV-medianMV)+(1-w)*R(MV-FmedianMV)]

(6)

1. calculate the cost of three motion vector predictor: (i) intermediate value MV fallout predictor (" medianMV "), (ii) rightEstimated motion vector (" futureMV "), it is defined as follows:

FutureMV ≡ median (TopMV, TopRightMV, TopRightRightMV) and (iii) from past pieceThe MV fallout predictor of (" pastMV "), it is previous common position MV (" PreMV ") and is away from the previously bottom-right of medianMVIn MV (" PreBottomRightMV ") one,

pastMV≡argmax_{MV∈{PreMv，PreBottomRightMV}}{abs(MV-medianMV)}

Note, (ii) can by for another contiguous following piece (such as lower left, bottom, and/or bottom-rightPiece) estimated motion vector supplement or replace.

Shall also be noted that in item (iii), previous bottom-right MV can be predicted by the previous MV for another contiguous blockDevice supplements or replaces.

Note (ii) and (iii) independent aspects of shape cost invention.

For example, if above any one MV fallout predictor unavailable border of frame (), skips this fallout predictor.

2., if the minimum cost of motion vector predictor is less than threshold value T1, stops search and forward step 7 to. No, the motion vector of selecting to have a minimum cost is as currentMV (current MV) and forward next step to. Note 3 fortuneThe cost of dynamic vector fallout predictor can adopt predefined procedure to calculate that (for example medianMV, is then futureMV, is thenPastMV), and at any time,, if the cost of any motion vector predictor is less than specific threshold, search may stopOnly and forward step 7 to.

3. carry out an iteration of directed little diamond search around currentMV. The directed little diamond search of explained laterConcept.

4., if minimum cost is less than threshold value T2, stops search and forward step 7 to. Otherwise, select to have minimum generationThe motion vector of valency is as currentMV and forward next step to.

If (currentMV=medianMV) and current minimum cost be less than threshold value T3, carry out little rhombus and searchRope and forward step 7 to.

6., if video is not interlacing scan, carry out large diamond search, as shown in Fig. 3 (a); Otherwise, carry out as Fig. 3(b) the large diamond search of the amendment shown in. In each step in these steps, to each index point assessment cost of rhombusFunction.

7. select to have the MV of minimum cost.

In our experiment, find that the value about 0.8 of w is effective.

Explain now the step of directed little diamond search. Suppose, centerMV is current search center, and MV1, MV2,MV3 and MV4 be four around search point, as shown in Figure 4. For each MVi calculates R (MVi-medianMV). If R is (MVi-MedianMV) < R (centerMV-medianMV), calculates SAD and the cost of MVi. Otherwise, ignore this MVi. Selection hasThe MV of low-cost is as currentMV. Note, the concept of directed square search is considered to new, and structure cost invention oneIndividual independent aspects, its need not with combine to carry out by the concept of futureMV.

The step of the large diamond search of large diamond search and amendment is identical, but described search be for Fig. 3 (a) andAll set of the point illustrating respectively in 3 (b) complete.

We consider the multiple possible variant of embodiment within the scope of the present invention now.

First, note, for different pieces, the weight coefficient w in cost function can be different. In addition, optionalSelecting ground, can be different for w described in different MV candidates. In particular, the definition of w can be depended on such as MV candidate and isWhether the no X-axis component close to medianMV and/or futureMV or MV candidate or Y-axis component be with described FmedianMV'sThe situation that X-axis component or Y-axis component are identical and so on.

In addition, cost function can be not restricted to the form of equation (6). It can be (for example to comprise distortion measurement itemSAD, distortion quadratic sum (SSD), average deviation distortion (MAD), MSD etc.) and to current block and some contiguous block (for example consideredRight, lower block, lower-left piece etc.) encode any function of item of necessary position of motion vector.

In addition,, in step 1, the definition of futureMV is not limited to the form that above step 1 provides. For futureMVTwo possible replacements definition be:

futureMV≡median(leftMV，TopRightMV，TopRightRightMV)

futureMV≡median(medianMV，TopRightMV，TopRightRightMV)

In addition, in the above step 1 of expressing, pastMV be chosen as possible MV in previous frame (preMV withPreBottomRlghtMV) farthest one of distance medianMV in list collection. But the MV list that will consider can be wrappedContaining two above possible MV (for example preMV, preLeftMV, preRightMV, preTopMV, preTopLeftMV,PreTopRightMV, preBottomMB, preBottomLeftMV, preBottomRightMV, etc.). In addition, from oneThe MV of above previous coding frame can be included in that in described list, (if for example present frame is frame N, described list can compriseFrame N-1, N-2, N-3 ...). If present frame is B frame, the list of previous coding frame can comprise following P frame.

In addition,, in step 1, selecting pastMV is distance reference MV (medianMV in step 1) possibility farthestMV. Other is also fine with reference to MV, comprises leftMV or TopMV or TopRightMV or certain combination. From possible MVList in other method of selecting be also possible.

In step 2, the cost of described 3 motion vector predictor obtains according to certain predefined order.Possible predefined order comprises:

A) medianMV, follows by futureMV, follows by pastMV

B) medianMV, follows by pastMV, follows by futureMV

C) futureMV, follows by medianMV, follows by pastMV

D) futureMV, follows by pastMV, follows by medianMV

E) pastMV, follows by medianMV, follows by futureMV

F) pastMV, follows by futureMV, follows by medianMV

In addition, although express as above, in step 3, carried out an iteration of directed little diamond search, can applyMore than one iteration.

Analog result

We provide the analog result of embodiment E-PMVFAST now. Described embodiment is embedded into H.264 reference softwareJM9.3[13] in, and use various QP, video sequence, resolution ratio and hunting zone to simulate it. Table 1 (a-c) and 2(a-c) some typical analog results are shown. PSNR (ratio of peak signal and noise) changes and BR (bit rate) variation isWith respect to the search PSNR of (FS) and the variation of bit rate completely. Analog result illustrates, the bit rate of the E-PMVFAST proposingTrend towards with search and PMVFAST are similar completely with PSNR, but on large-scale video sequence and bit rate, E-PMVFAST trends towards than PMVFAST fast about 40%. A key character of E-PMVFAST is, its motion vector field trendIn level and smooth in the extreme, thereby make the motion vector can be than other fast moving algorithm for estimating moving of indicated object more exactlyMoving.

In Fig. 5 (a) and 5 (b), the image in left side illustrates the motion vector that (as short-term) obtained by PMVFAST algorithm, and the imaging on right side illustrates the motion vector of the identical imaging obtaining by described embodiment. The motion vector of E-PMVFASTField is more level and smooth than the motion vector field of PMVFAST significantly, particularly in irised out region. For perception conversion is compiledCode (perceptualtrans-coding), speed control, multiple block-sized motion estimation, multiple standards frame estimation etc.In video motion content classify, level and smooth sports ground is very useful.

The analog result of form 1a-foremanCIF sequence

The analog result of form 1b-CoastguardCIF sequence

The analog result of form 1c-HallCIF sequence

The analog result of form 2a-foremanQCIF sequence

The analog result of form 2b-AkiyoQCIF sequence

The analog result of form 2c-CoastguardQCIF sequence

We turn to the second embodiment of the present invention now, and it shows a second aspect of the present invention.

As mentioned above, conventional integer pixel completely allows motion vector in each direction, to adopt-2.0 ,-1.0,0,1.0,2.0 etc. positional value. In second embodiment of the invention, select the possible position value close to fallout predictor. For approaching most 0Positional value, we can use (replace 1.0) another location value, such as 0.85, thereby the positional value of permission can be wrappedDraw together-2.0,-0.85，0，0.85, 2.0 etc. This advantage is, in statistics, motion vector tends to close to 0. Therefore, pass throughSelect more to approach 0 position, we can be more close to real motion vector, and thereby can provide and can causeThe better motion compensation of high compression efficiency. Similarly, can change other positional value. For example, positional value 2.0 can be changedBe 1.9, thereby the positional value allowing can be comprised-1.9，-0.85, 0,0.85,1.9 etc. Except 1.0 coding motion is vowedAmount position should be interpreted as 0.85, and 2.0 encoding motion vector position should be interpreted as beyond 1.9, the change proposingAdvantage be to use same movement vector code.

Half-pixel accuracy allows motion vector to adopt such as 0.0,0.5,1.0,1.5,2.0 etc. and so on positional value. IThese positional values of proposed amendments, particularly those are close to the positional value of fallout predictor. For the positional value that is in close proximity to 00.5, we advise using a different value. For example, a kind of possibility is to use 0.4 to replace 0.5. In other words, positional valueTo comprise 0.0,0.4, 1.0,1.5,2.0. Similarly, other positional value can be modified. For example, positional value 1.0 can changeBe 0.9, thereby make this group reposition value will comprise 0.0,0.4,0.95,1.5,2.0 etc. Equally, this can contribute to carryHigh compression efficiency. Similarly, can revise other positional value to improve compression efficiency. But changing this position can be remarkableGround causes the higher computational efficiency at both places of encoder. Conventionally, most of compression efficiency gain comes from positionThe value of putting is changed into close to fallout predictor.

1/4 pixel precision allows motion vector to adopt such as 0.00,0.25,0.50,0.75,1.00 etc. and so on positionValue. We can location revision value, and those are close to the positional value of fallout predictor especially. For example, we can be revised as them0.00,0.20,0.47,0.73,0.99 etc.

Note, the method proposing allows us between each integer position value, to select the positional value of any amount. ExampleAs, between positional value 0 and 1, half-pixel accuracy is used 1 positional value, and { 0.5}, 1/4 pixel precision uses 3 positional values0.25,0.50,0.75}, and 7 positional values of 1/8 pixel precision use 0.125,0.250,0.375,0.500,0.625,0.750,0.875}. The method proposing allows us between 0 and 1, to select any N positional value. For example, we can select N=2 values are such as 0.3 and 0.6.

The inhomogeneous sub-pix estimation proposing and compensation are not each regions that must be applied to each frame.On the contrary, some position can be incorporated in head, with indicate each region (for example sheet) for described frame of video its whether byOpen or close. In addition, it can directly apply to existing standard in the situation that changing without any grammer, because canTo apply identical motion vector code.

Adopt H.264JM82 software to simulate proposed inhomogeneous sub-pix estimation and compensation, andIts result is shown in above form, and wherein QP represents quantization parameter. This simulation in x and y direction use location value (...-1 ,-0.75 ,-0.5 ,-0.15,0,0.15,0.5,0.75,1..). , with the standard method that uses 1/4 pixel space positional valueCompare, only revised-0.25 and the positional value at+0.25 place. Except using new positional value, described algorithm is at itHis aspect is identical with known H.264 canonical algorithm. Shown at table, the second embodiment is realizing similar PSNR'sMeanwhile, reduced significantly bit rate. Do not need amendment grammer H.264.

Although be only several embodiments of the present invention described above, many within the scope of the present invention variations are all possible.

For example, the description of the invention more than providing is the piece of the fixed size of the P frame for having a reference frame.But the present invention can be applied to the piece with multiple sub-block size, and described must be not non-overlapped. Can haveMore than one reference frame, and reference frame can be the past with respect to present frame or in the future any of video sequence.

For video, a pictorial element (pixel) can have one or more components, such as luminance component, redness,Green, blue (RGB) component, YUV component, YCrCb component, infrared component, X-ray or other component. Each point of pictorial elementAmount is to be expressed as digital symbol, and described numeral can be natural number, integer, real number or even plural number. NaturalIn situation, they can be 12,8, or any other bit resolution. Although being the rectangle having, the pixel in video adoptsThe 2 dimension samples in sample grid and uniform sampling cycle, but described sampling grid is not to be necessary for rectangle and described sampling periodNot that to be necessary for be uniform.

Industrial applicibility

Each embodiment of the present invention is suitable for by MPEG-1, MPEG-2, MPEG-4, H.261, H.263, H.264, AVSOther can be modified to include above it associated video coding standard or quick, low delay and the low-cost software of methodRealize with hardware. Possible application comprises DVB (ground, satellite, wired), digital camera, numeral canCamcorder (camcorder), digital VTR, Set Top Box, personal digital assistant (PDA), can use multimedia honeycombFormula phone (2.5G, 3G and more than), video conferencing system, video on-demand system, WLAN devices, bluetooth applications, networkVideo streaming services device in server, low or bandwidth applications, video code interpreter are (from a format conversion to another latticeFormula) and other television communications system etc.

Bibliography

Disclosure below with reference to document is introduced in this comprehensively:

[1]JointVideoTeamofITU-TandISO/1ECJTC1，“DraftITU-TRecommendationandFinalDraftinternationalStandardofJointVideoSpecification(ITU-TRec.H.264|ISO/IEC14496-10AVC)”，docunmentJVT-G050r1，may2003

[2]A.M.Tourapis，O.C.AuandM.L.Liou，“PredictiveMotionVectorFieldAdaptiveSearchTechnique(PMVFAST)”，ISO/IECJTC1/SC29/WG11MPEG2000，Noordwijkerhout，ML，March’2000

[3]R.LI，B.ZengandM.L.Liou，“Anewthree-stepsearchalgorithmforblockmotionestimation”，OnCircuitsandSystemsforVideoTechnology，vol4，no4，pp438-42，Aug’94

[4] Z.L.He and M.L.Liou, " Ahighperformancefastsearchalgorithmforblockmatchingmotionestimation”，IEEETrans.onCircuitsandSystemsforVld&o2Technology，vol.7，no5，pp826-8，Oct’97

[5]A.M.Tourapis，O.C.Au，andM.L.Liou，“FastMotionEstimationusingCircularZonalSearch”，Proc.ofSPIESym.OfVisualComm.&ImaggProcessin，vol2，pp.1496-1504，Jan.25-27，‘99

[6]A，M.Tourapis，O.C.Au，M.LLiou，G.Shen，andI.Ahmad，“OptimizingtheMpeg-4Encoder-AdvancedDiamondZonalSearch”，inPros.of2000IEEEInter.Sym.onCircuitsandSystems，Geneva，Switzerland，May，2000

[7]K.K.MaandP.I.Hosur，“PerformanceReportofMotionVectorFieldAdaptiveSearchTechnique(MVFAST)”，inISO/IECJTC1/SC29/WG11MPEG99/m81，Noordwijkerhout，NLMar’00

[8]A.M.Tourapis，O.C.Au，andM.L.Liou，“FastBlock-MatchingMotionEstimationusingPredictiveMotionVectorFieldAdaptiveSearchTechnique(PMVFAST)”，inISO/IEC/JTC1/SC29/WG11MPEG2000/M5866，Noordwijkerhout，NL，Mar’00

[9]ImplementationStudyGroup，“Experimentalconditionsforevaluatingencodermotionestimationalgorithms”，inISO/IECJTC1/SC29/WG11MPEG99/n3141，Hawaii，USA，Dec’99

[10]“MPEG-4OptimizationModelVersion1.0”，inISO/IECJTC1/SC29/WG11MPEG2000/n3324，Noordwijkerhout，NL，Mar’00

[11]T，Koga，K.linuma，A.Hirano，Y.lijima，andT.Ishlguro，“Motioncompensatedinterframecodingforvideoconferencing”Proc.Nat.Telecommun.Conf.，NewOrleans，LA，pp.G.5.3.1-G.5.3.5，Dec’81。

[12]J.R.JainandA.K.Jain，“Displacementmeasurementanditsapplicationininterframeimagecoding”，JEEETrans.OnCommunications，vol.COM-29，pp.1799-808，Dec’81

[13]JVTreferencesoftwareJM9.2forJVT/H.264FRext

Claims (8)

1. one kind is that each in a series of block of pixels in the first image is selected the similar pixel separately in the second imageThe method of piece, described method comprises:

For the current block in described a series of block of pixels of described the first image, the multiple position candidate in described the second imageBetween select, described multiple position candidate comprise by two defined position candidate of positional value on two direction of principal axisGroup, at least one positional value in described positional value is inhomogeneous for described two axial at least one direction of principal axis.

2. the method for claim 1, wherein each in the group of described position candidate is worth with each by relevant positionFrom motion vector be associated, described motion vector is different from motion vectors.

3. the method for claim 1, wherein each in the group of described position candidate is worth with each by relevant positionFrom motion vector be associated, described motion vector is different from (0,0) motion vector.

4. method as claimed in claim 1, wherein, described positional value can be write DA_mM/n, wherein m=-..., 2 ,-1,0,1,2 ...,N is 1 or 2 power, and D is constant, and at least one the value of m A_mTo be less than 1 value, and for all values of m0.75≤A_m＜1。

5. method as claimed in claim 4, wherein, each in the group of described position candidate is worth with each by relevant positionFrom motion vector be associated, described motion vector is different from motion vectors.

6. method as claimed in claim 4, wherein, each in the group of described position candidate is worth with each by relevant positionFrom motion vector be associated, described motion vector is different from (0,0) motion vector.

7. method as claimed in claim 4, wherein, for all values of m, A_mValue be at least 0.85.

8. one kind is that each in a series of block of pixels in the first image is selected the similar pixel separately in the second imageThe system of piece, described system comprises processor, described processor is configured to, and is described a series of block of pixels of described the first imageIn current block, between the multiple position candidate in described the second image, select, described multiple position candidate comprise byThe group of two defined position candidate of positional value on two direction of principal axis, at least one positional value in described positional value forDescribed two axial at least one direction of principal axis are inhomogeneous.