CN100377599C - A fast sub-picture element movement estimating method - Google Patents

Abstract

The present invention relates to a fast sub-picture element movement estimating method which belongs to the field of image processing. On the basis of integral picture element movement estimation, the present invention directly calculates the position of the optimal sub-picture element. Thus, searching points of sub-picture element movement estimation are decreased to less than two; the point-by-point search comparison of a traditional method is avoided, so the searching speed is raised, and the memory cost desired by interpolating sub-picture element points is approach to zero. By experimental results, the minimal searching price is used for obtaining the equivalent effect of full search in the method.

Description

A kind of fast sub-picture element movement estimating method

Technical field

The present invention relates to a kind of method that improves sub-pix estimation speed, reduces storage overhead in the sub-pix motion estimation process, belong to technical field of image processing.

Background technology

Over nearly 10 years, in video compression, obtained extensive use based on the encoding scheme of discrete cosine transform and motion compensation, and brought a series of international standards into, as H.263, MPEG-4 and JVT etc.In this scheme, estimation is to eliminate effective ways redundant between frame of video, but because it adopts the search of interframe block-by-block relatively, and amount of calculation is huge, becoming influences the key technology of video compression performance obstacle.Estimation is estimated by whole pixel motion and sub-pix estimation two parts are formed.Whole pixel motion estimates that directly with coded frame be reference frame, the blocks and optimal matching blocks of the whole Pixel-level of search; The sub-pix estimation then by the value of non-Direct Sampling point (sub-pix point) in the interpolation estimation reference frame, as reference, is further searched for the blocks and optimal matching blocks of sub-pixel on the basis that whole pixel motion is estimated.Experiment shows, carries out the sub-pix estimation on the basis that whole pixel motion is estimated, can obviously improve the motion compensation effect, so the sub-pix estimation has become the effective ways that improve image compression rate.

The sub-pix estimation that generally adopts in the video compression standard is an all direction search method at present, promptly earlier entire image is carried out interpolation, on this basis, the whole pixel motion of search is estimated 8 1/2 pixels around the optimum point, obtain the optimum point of 1/2 Pixel-level, so analogize, the search of each grade sub-pix is its 8 next stage sub-pix points on every side of center search with the optimum point of upper level search all.(or claim 1/2 for the K level ^K) the sub-pix estimation, then each current block need carry out the search of 8 * K piece, and the image increase after the interpolation simultaneously is 2 of input picture ^2KDoubly, need in the cataloged procedure to increase by 2 ^2K-1 times interpolation memory space.This all direction search method not only computational complexity is high but also also be exponential growth with the increase interpolation storage overhead of sub-pixel precision.

For solving the problem that exists in the sub-pix all direction search method, many fast algorithms have been produced.These current methods are generally dwindled the hunting zone to reduce search point from aspects such as search strategy, end conditions.Generally speaking still there is big gap in its search precision with full search, and search speed also needs further raising; In addition, a large amount of interpolation storage overheads also are insoluble problems.In the practical application, especially at the limited applied environment of storage resources, a kind of good sub-picture element movement estimating method must be considered search precision, operational efficiency and storage overhead simultaneously.

Summary of the invention

The object of the present invention is to provide a kind of fast sub-picture element movement estimating method.This method is according to the optimum sub-pixel location under the contiguous whole pixel value direct derivation arbitrary accuracy, thus the storage overhead of having avoided the method for traditional point by point search comparison to bring.

For realizing above-mentioned goal of the invention, the present invention adopts following technical scheme:

A kind of fast sub-picture element movement estimating method comprises according to whole pixel motion and estimates to obtain whole pixel blocks and optimal matching blocks, it is characterized in that further may further comprise the steps:

Step: represent current block and the Minimum Mean Square Error of whole pixel blocks and optimal matching blocks sub-pix piece on every side and the position of corresponding optimum sub-pix piece thereof with current block, whole pixel blocks and optimal matching blocks and contiguous whole block of pixels thereof, and calculate the level under the arbitrary accuracy, the position of vertical optimum sub-pix piece respectively according to the relation between the whole block of pixels of current block, whole pixel blocks and optimal matching blocks and vicinity thereof;

Step 2: according to the sub-pixel precision that adopts in the concrete application, the optimum sub-pixel location that obtains in the step 1 is carried out approximate processing, obtain the near-optimization position under the designated precision;

Step 3: near-optimization position on the level that obtains respectively in the step 2 and the vertical direction is synthesized, obtain two-dimentional near-optimization position;

Described two-dimentional near-optimization position is if be in sub-pixel location, with its corresponding blocks as a candidate blocks; And from step 2, obtain on the horizontal direction and vertical direction on the near-optimization sub-pixel location in choose the residual error reckling, its be in sub-pixel location then with its corresponding blocks as another candidate blocks;

Described candidate blocks is searched for coupling, and compare, get optimum piece as final result with described whole pixel motion estimated result.

Described step 1 comprises step:

(11) calculate current block and whole pixel motion respectively and estimate minimum match error between the sub-pix piece of blocks and optimal matching blocks and arranged on left and right sides, and get minimum value among both as horizontal least residual;

(12) calculate current block and whole pixel motion respectively and estimate minimum match error between the sub-pix piece of the upper and lower both sides of blocks and optimal matching blocks, and get minimum value among both as vertical least residual.

Described step 1 further may further comprise the steps:

(13), calculate optimum sub-pixel location under its corresponding arbitrary accuracy respectively according to level that obtains in step (11) and (12) and the minimum match error on the vertical direction.

Described minimum match error be Minimum Mean Square Error and minimum absolute difference and in a kind of.

Pass between the Minimum Mean Square Error of described current block and sub-pix piece and described current block, whole pixel blocks and optimal matching blocks and the contiguous whole block of pixels thereof is:

$\mathrm{Min}\left(D_{c_{i,j}{s}_{i,j}}\right)=\frac{4D_{r_{i,j}{r}_{i+1,j}}{D}_{c_{i,j}{r}_{i,j}}-{(D_{c_{i,j}{r}_{i,j}}-D_{c_{i,j}{r}_{i+1,j}}-D_{r_{i,j}{r}_{i+1,j}})}^2}{{4D}_{r_{i,j}{r}_{i+1,j}}}$

Pass between the position of described optimum sub-pix piece and current block, whole pixel blocks and optimal matching blocks and the contiguous whole block of pixels thereof is: $m_h=\frac{D_{c_{i,j}{r}_{i+1,j}}-D_{c_{i,j}{r}_{i,j}}}{2D_{r_{i,j}{r}_{i+1,j}}}+\frac{1}{2}$

Wherein shape such as D _{Dx, yd ' x, y}The symbolic representation image in top-left position point be d _{X, y,}D ' _{X, y}Two pieces absolute difference and, c _{I, j}Be the point on the current block, r _{I, j}Be the point on the whole pixel blocks and optimal matching blocks, Min (D _{Ci, jsi, j}) Minimum Mean Square Error between expression sub-pix piece and the current block, m _hThe expression optimal location.

The optimum sub-pixel location to obtaining in the step 1 in the described step 2 is carried out approximate processing, comprising: when specifying search precision is 1/2 ^KThe time, the optimal location m that obtains in one dimension near-optimization position m ' and the step 1 _hBetween close and to be: m '=roumd (m _b* 2 ^K)/2 ^k, wherein, roumd () table rounds up, and obtains the near-optimization position thus.

On two-dimentional near-optimization position and level and vertical direction, choose the candidate blocks that is in sub-pixel location the near-optimization sub-pixel location in the described step 3, comprising:

1) if two-dimentional near-optimization position corresponding blocks be in sub-pixel location, then with it as a candidate blocks.

2) minimum absolute difference of determined level direction near-optimization position and vertical direction near-optimization position correspondence and, if the minimum absolute difference of horizontal direction near-optimization position correspondence and less, whether then further determined level direction near-optimization position is in sub-pixel location, if then selected level direction near-optimization position corresponding blocks is a candidate blocks; If whether the minimum absolute difference of vertical direction near-optimization position correspondence and less then further judges vertical direction near-optimization position in sub-pixel location, if then selected vertical direction near-optimization position corresponding blocks is a candidate blocks.

Step 3 also comprises: by the value of each sub-pix point in the interpolation calculation candidate blocks, and calculate this piece and current block minimum absolute difference and, and estimate the minimum absolute difference that obtains with whole pixel motion and compare, with the correspondence position of smaller value among both as the motion vector that finally searches.

The intermediate object program that the present invention utilizes whole pixel motion to estimate is directly determined 0-2 optimum sub-pix candidate blocks by calculating, and at this enterprising line search in candidate blocks basis.Theory analysis shows, is 1/2 for search precision ^KThe sub-pix estimation, compare with the sub-pix all direction search method, search block is counted rate of descent up to (more than 8 * K-2)/8 * K.Experimental result shows that during 1/4 sub-pix estimation, search block is counted rate of descent more than 90%, and picture quality and compression bit rate all do not have significant change.Simultaneously, the interpolation storage overhead in the search procedure of the present invention levels off to zero, and this is very important for the video encoder of realizing on special chip, DSP.

Description of drawings

Fig. 1 is the flow chart of sub-picture element movement estimating method of the present invention;

Fig. 2 is the pixel distribution schematic diagram of two-dimentional sub-pix motion;

Fig. 3 is decomposed into the schematic diagram that estimation is carried out in motion in one dimension to two dimensional motion;

Fig. 4 is the schematic diagram of one dimension best fit approximation position.

Embodiment

The present invention according to the optimum sub-pixel location under the contiguous whole pixel value direct derivation arbitrary accuracy, thereby has proposed a kind of fast sub-picture element movement estimating method that calculates based on optimal location on the basis that whole pixel motion is estimated.

Because the sub-pixel location point is not a Direct Sampling, but by contiguous whole pixel sampling point interpolation, therefore the position that has a sub-pix piece of Optimum Matching with current block can directly be derived by the whole pixel value around it fully, with avoid in the current method point by point search relatively.

Below in conjunction with description of drawings implementation of the present invention.The overall flow of clearly having represented the method for the invention among Fig. 1, that is: the optimum sub-pixel location of one dimension on elder generation's difference calculated level and the vertical direction; According to the precision of setting, the optimum sub-pixel location of one dimension is carried out approximate processing again, obtain the near-optimization position on level and the vertical direction respectively; Last again according to the near-optimization choice of location candidate piece on level and the vertical direction, finish search.Be described in detail as follows below:

Step 1:, calculate the level under the arbitrary accuracy, the position of vertical optimum sub-pix piece respectively according to the relation of current block and whole pixel blocks and optimal matching blocks and adjacent interblock thereof.

How to specify calculated level or vertical optimum sub-pixel location below in conjunction with Fig. 2, Fig. 3.In Fig. 2 and Fig. 3, with the whole pixel of " ● " expression; " X " expression sub-pix point.Each point is the upper left point of its place piece among Fig. 3, R _{I, j}For whole pixel motion is estimated (IME) blocks and optimal matching blocks, R _{I+1, j}, R _{I-1, j}Be its level two lateral extents be 1 pixel adjacent piece, R _{I, j+1}, R _{I, j+1}Be its vertical two lateral extents adjacent piece that is 1 pixel.

In the method for the invention, establish that the point on current to be searched is c in the coded frame _{I, j}(i ∈ (0, M), j ∈ (0, N)), M wherein, N represents level, the vertical direction pixel count that search block comprises respectively; In the reference frame that whole pixel motion obtains after estimating on the blocks and optimal matching blocks each point be r _{I, j}(i ∈ (0, M), j ∈ (0, N)); Each point r on the blocks and optimal matching blocks _{I, j}Between the sub-pix piece that obtains of interpolation be s _{I, j}, and S _{I, j}Satisfy formula: s _{I, j}=mr _{I, j}+ (1+m) r _{I+1, j}Wherein, m is a location parameter.

By mathematical derivation, the sub-pix piece of a side and the mean square deviation between the current block can finally be expressed as through abbreviation on IME blocks and optimal matching blocks level or the vertical direction:

$D_{c_{i,j}{s}_{i,j}}=D_{r_{i,j}{r}_{i+1,j}}{m}^2+(D_{c_{i,j}{r}_{i,j}}-D_{c_{i,j}{r}_{i+1,j}}-D_{r_{i,j}{r}_{i+1,j}})m+D_{c_{i,j}{r}_{i+1,j}}---\left(1\right)$

To (2) formula differentiate, and according to the character of quadratic function, the Minimum Mean Square Error of sub-pix piece and current block is as can be known:

$\mathrm{Min}\left(D_{c_{i,j}{s}_{i,j}}\right)=\frac{{4D}_{r_{i,j}{r}_{i-1,j}}{D}_{c_{i,j}{r}_{i,j}}-{(D_{c_{i,j}{r}_{i,j}}-D_{c_{i,j}{r}_{i+1,j}}-D_{r_{i,j}{r}_{i+1,j}})}^2}{{4D}_{r_{i,j}{r}_{i+1,j}}}---\left(2\right)$

Optimum sub-pixel location under its corresponding arbitrary accuracy is:

$m_h=\frac{D_{c_{i,j}{r}_{i+1,j}}-D_{c_{i,j}{r}_{i,j}}}{2D_{r_{i,j}{r}_{i-1,j}}}+\frac{1}{2}---\left(3\right)$

Wherein shape such as D _{Dx, d ' x, y}The symbolic representation image in top-left position point be d _{X, y}, d ' _{X, y}Two pieces absolute difference and, c _{I, j}Be the last point of current block, r _{I, j}Be the point on the whole pixel blocks and optimal matching blocks, Min (D _{Ci, jsi, j}) Minimum Mean Square Error between expression sub-pix piece and the current block, m _hThe expression optimal location.

It is to be noted: estimation usually with mean square deviation (MSE) or its reduced form absolute difference sum (SAD) as matching criterior, in above-mentioned mathematical analysis, get the MSE matching criterior and analyze.Usually replace mean square deviation MSE with absolute difference and sad value for reducing operand in the actual motion estimation procedure.Therefore be appreciated that method of the present invention also can adopt the SAD matching criterior similarly.In this case, the MSE value replaces with corresponding sad value in formula (2), (3).

According to the mathematical formulae that above-mentioned mathematical derivation obtained, in step 1, need the concrete implementation step of carrying out as follows:

(1) calculates current block and R respectively by formula (2) _{I, j}And arranged on left and right sides sub-pix piece minimum absolute difference and, and get minimum value among both as horizontal least residual, note is made S _h

(2) calculate current block and R respectively by formula (2) _{I, j}Upper and lower both sides sub-pix piece minimum absolute difference and, and get minimum value among both as vertical least residual, note is made S _v

(3) calculate S respectively by formula (3) _h, S _vOptimum sub-pixel location h, v under the corresponding arbitrary accuracy.

Step 2: according to the sub-pixel precision that adopts in the concrete application, the optimum sub-pixel location that obtains in the step 1 is carried out approximate processing, obtain the near-optimization position under the designated precision.

Step 1 obtains the optimal location under the arbitrary accuracy.But in the practical application, after estimation (ME) reached certain precision, the reduction of residual error coefficient was very little, and motion vector takies than multi-bit, thereby caused final number of coded bits no longer significantly to descend.Therefore, general 1/2,1/4 or 1/8 precision that adopts in the practical application adopts 1/2 pixel as the ME of MPEG-4, and JVT then can be accurate to 1/8 pixel.For match m in actual applications with international standard _hValue should be consistent with used search precision (1/2,1/4 or 1/8) in the standard.

By formula (1), the absolute difference and the D of the sub-pix piece around current block and the whole pixel blocks and optimal matching blocks _{Ci, jsi, j}Be the second-degree parabola function of sub-pixel location m, by parabolical symmetrical monotonicity as can be known: when specifying search precision is 1/2 ^KThe time, the nearest sub-pixel location of distance calculation optimal location is a best fit approximation, obtains the computing formula of the one dimension near-optimization position m ' under the designated precision in view of the above:

m′＝round(m _b×2 ^K)/2 ^K (4)

Wherein round represents to round up, m _hBe the optimal location that obtains in the step 1.As shown in Figure 2, when adopt 1/4 pixel precision, sub-pixel location obtain when being 9/16 minimum absolute difference and, be 1/2 with 9/16 nearest 1/4 sub-pixel location this moment, therefore 1/2 is the best fit approximation position.

According to the optimal location h, the v that obtain in the step 1 and the search precision of appointment, calculate near-optimization position h ' and v ' under this search precision according to formula (4).

Step 3: with the synthetic two-dimentional near-optimization position that obtains, near-optimization position on level in the step 2 and the vertical direction.From synthetic two-dimentional near-optimization position that obtains and abovementioned steps, choose the candidate blocks that is in sub-pixel location in the near-optimization sub-pixel location on level and the vertical direction, and candidate blocks is searched for, obtain motion estimation result.

Calculate the near-optimization position on level, vertical two the one dimension directions in the step 2 respectively, consider that two dimensional motion can regard the synthetic of componental movement on level, vertical two the one dimension directions as, among the present invention with this level, the synthetic corresponding two-dimensional position piece in upright position as a candidate blocks.In addition, the position of considering this candidate blocks is a kind of approximation, for the precision that improves ME again with horizontal optimal location piece S _h, residual error S in the vertical optimal location piece _vMiddle smaller's corresponding blocks is as another candidate blocks.The piece that is in sub-pixel location in two candidate blocks is searched for coupling, and compare, get optimum piece as final result with whole pixel Search Results.As seen, the present invention is reduced to two or still less with the candidate piece, therefore significantly reduced expense, but search speed is but very fast.The concrete steps of step 3 are as follows:

(1) put the candidate blocks set for empty, if point (h ', v ') be in sub-pixel location, its corresponding blocks is added the candidate blocks set;

(2) if S _h≤ S _vAnd point (h ', 0) be in sub-pixel location, then incite somebody to action (h ', 0) set of corresponding blocks adding candidate blocks; Change (4);

(3) if S _h＞S _vAnd point (0, v ') is in sub-pixel location, then incite somebody to action (v ', 0) set of corresponding blocks adding candidate blocks;

(4) each piece in the candidate blocks set is carried out computings such as interpolation, search, determine final result.That is: the value by each sub-pix point in the interpolation calculation candidate blocks, and calculate itself and current block absolute difference and, relatively its minimum absolute difference of estimating with whole pixel motion with, with the correspondence position of minimum value among both as the motion vector that finally searches.

More than disclosed only be specific embodiments of the invention, according to thought provided by the invention, those skilled in the art can think and variation, all should fall within the scope of protection of the present invention.

Claims (9)

1. a fast sub-picture element movement estimating method comprises according to whole pixel motion and estimates to obtain whole pixel blocks and optimal matching blocks, it is characterized in that further may further comprise the steps:

Step 1: represent current block and the Minimum Mean Square Error of whole pixel blocks and optimal matching blocks sub-pix piece on every side and the position of corresponding optimum sub-pix piece thereof with current block, whole pixel blocks and optimal matching blocks and contiguous whole block of pixels thereof, and calculate the level under the arbitrary accuracy, the position of vertical optimum sub-pix piece respectively according to the relation between the whole block of pixels of current block, whole pixel blocks and optimal matching blocks and vicinity thereof;

Step 2: according to the sub-pixel precision that adopts in the concrete application, the optimum sub-pixel location that obtains in the step 1 is carried out approximate processing, obtain the near-optimization position under the designated precision;

Step 3: near-optimization position on the level that obtains respectively in the step 2 and the vertical direction is synthesized, obtain two-dimentional near-optimization position;

Described two-dimentional near-optimization position is if be in sub-pixel location, with its corresponding blocks as a candidate blocks; And from step 2, obtain on the horizontal direction and vertical direction on the near-optimization sub-pixel location in choose the residual error reckling, its be in sub-pixel location then with its corresponding blocks as another candidate blocks;

Described candidate blocks is searched for coupling, and compare, get optimum piece as final result with described whole pixel motion estimated result.

2. fast sub-picture element movement estimating method as claimed in claim 1 is characterized in that: described step 1 comprises step:

(11) calculate minimum match error between the sub-pix piece of current block and whole pixel blocks and optimal matching blocks and arranged on left and right sides respectively, and get minimum value among both as horizontal least residual;

(12) calculate minimum match error between the sub-pix piece of current block and the upper and lower both sides of whole pixel blocks and optimal matching blocks respectively, and get minimum value among both as vertical least residual.

3. fast sub-picture element movement estimating method as claimed in claim 2 is characterized in that: described step 1 further may further comprise the steps:

(13), calculate optimum sub-pixel location under its corresponding arbitrary accuracy respectively according to level that obtains in step (11) and (12) and the minimum match error on the vertical direction.

4. fast sub-picture element movement estimating method as claimed in claim 2 is characterized in that:

Described minimum match error be Minimum Mean Square Error and minimum absolute difference and in a kind of.

5. fast sub-picture element movement estimating method as claimed in claim 1 is characterized in that:

Pass between the Minimum Mean Square Error of described current block and sub-pix piece and described current block, whole pixel blocks and optimal matching blocks and the contiguous whole block of pixels thereof is:

$\mathrm{Min}\left(D_{c_{i,j}{s}_{i,j}}\right)=\frac{{4D}_{r_{i,j}{r}_{i+1,j}}{D}_{c_{i,j}{r}_{i,j}}-{(D_{c_{i,j}{r}_{i,j}}-D_{c_{i,j}{r}_{i+1,j}}-D_{r_{i,j}{r}_{i+1,j}})}^2}{{4D}_{r_{i,j}{r}_{i+1,j}}}$

Pass between the position of described optimum sub-pix piece and current block, whole pixel blocks and optimal matching blocks and the contiguous whole block of pixels thereof is: $m_b=\frac{D_{c_{i,j}{r}_{i+1,j}}-D_{c_{i,j}{r}_{i,j}}}{{2D}_{r_{i,j}{r}_{i+1,j}}}+\frac{1}{2}$

Wherein shape such as D _{Dx, yd ' x, y}The symbolic representation image in top-left position point be d _{X, y}, d ' _{X, y}Two pieces absolute difference and, c _{I, j}Be the point on the current block, r _{I, j}Be the point on the whole pixel blocks and optimal matching blocks, Min (D _{Ci, jxi, j}) Minimum Mean Square Error between expression sub-pix piece and the current block, m _bThe expression optimal location.

6. fast sub-picture element movement estimating method as claimed in claim 1 is characterized in that:

The optimum sub-pixel location to obtaining in the step 1 in the described step 2 is carried out approximate processing, comprising: when specifying search precision is 1/2 ^KThe time, the optimal location m that obtains in one dimension near-optimization position m ' and the step 1 _bBetween close and to be: m '=round (m _b* 2 ^K)/2 ^K, wherein, round () table rounds up, and obtains the near-optimization position thus.

7. fast sub-picture element movement estimating method as claimed in claim 1 is characterized in that:

On two-dimentional near-optimization position and level and vertical direction, choose the candidate blocks that is in sub-pixel location the near-optimization sub-pixel location in the described step 3, comprising:

1) if two-dimentional near-optimization position corresponding blocks be in sub-pixel location, then with it as a candidate blocks.

2) minimum absolute difference of determined level direction near-optimization position and vertical direction near-optimization position correspondence and, if the minimum absolute difference of horizontal direction near-optimization position correspondence and less, whether then further determined level direction near-optimization position is in sub-pixel location, if then selected level direction near-optimization position corresponding blocks is a candidate blocks; If whether the minimum absolute difference of vertical direction near-optimization position correspondence and less then further judges vertical direction near-optimization position in sub-pixel location, if then selected vertical direction near-optimization position corresponding blocks is a candidate blocks.

8. fast sub-picture element movement estimating method as claimed in claim 7 is characterized in that:

Described step 3 also comprises: by the value of each sub-pix point in the interpolation calculation candidate blocks, and calculate this piece and current block minimum absolute difference and, and estimate the minimum absolute difference that obtains with whole pixel motion and compare, with the correspondence position of smaller value among both as the motion vector that finally searches.

Images