CN1492687A - Quick full pixel movement evaluation method of multiple kinds of modules - Google Patents

Abstract

This invention relates to quick integral picture elements motion estimation method for multiple kinds of block modes, firstly, searching interms of small block mode to determine larger block mode search sphere by small block mode motion vectors to search in light of the large mode till finishing the integral picture element search. This method not only secures the original rate distortion character of the coder but also reduces computations in large scale related to various block modes to increase coding speed greatly, mainly solving problems on multiblock mode motion estimation method in H264 video coder not limitted on H264 international standard, it can be used together with the present subpicture elements search method and RDO block mode decisive models.

Description

The fast whole pixel motion estimating method of multiple block mode

Technical field

The invention belongs to the field of video encoding of information processing, be primarily aimed at the multiple block mode motion compensation process of up-to-date proposition in the video coding international standard H.264, propose the new correlation of utilizing various block modes and ended the rapid motion estimating method of (half stop) criterion midway, under the condition that guarantees the original rate distortion of video coding, improved the search speed of various block modes greatly, and algorithm is simple, helps software and hardware and realizes.

Background technology

At present, it is ripe that video coding technique has been tending towards, international standard H.261, H.263, H.264, the formulation of MPEG-1, MPEG-2, MPEG-4 etc. promoted application of multimedia technology greatly.Its code efficiency of H.264 video encoding standard that formulation is soon finished improved greatly than former coding standard, thereby extensively was subjected to the concern of industry.

The international standard of above video coding all adopts hybrid encoding frame, promptly takes all factors into consideration factors such as prediction, conversion, entropy coding and improves code efficiency.Prediction comprises infra-frame prediction and inter prediction, can remove the spatial redundancy of intraframe coding behind the infra-frame prediction, and the prediction of interframe then can be removed temporal redundancy.

Because the temporal continuity of moving image, remove temporal redundancy and can obtain bigger compression efficiency, so remove the redundancy of interframe, be the focus of video coding research by method for estimating always.Traditional method for estimating is based on the piece coupling, and this is relevant with coding method.Existing video encoding standard all is based on DCT (or class DCT) conversion, usually a two field picture is encoded by 16 * 16 image block.During such as macro block M in a certain P frame of coding, the piece R that search is mated the most in the certain limit of reference frame relevant position is as the reference block of macro block M, current macro M is deducted remaining matrix after reference macroblock R can obtain estimation, this remaining matrix is promptly obtained the code stream behind this macroblock coding through conversion, quantification, entropy coding.

Will use matching criterior when the search match block, the most general adaptation function is SAD (Sum ofAbsolute Difference) function at present, and it is defined as follows: $\mathrm{SAD}(x,y,n)=\underset{i=0}{\overset{M-1}{\mathrm{\Σ}}}\underset{j=0}{\overset{N-1}{\mathrm{\Σ}}}\mathrm{abs}(f(i,j,t)-f(i-x,j-y,t-n))$ Formula (1)

Here hypothesis adopts multi-reference frame, and wherein the match block size is M * N, H.263, MPEG-4 has 16 * 16 and 8 * 8 two kinds of block modes, H.264 comprise 7 kinds of piece match patterns, as: 16 * 16,16 * 8,8 * 16,8 * 8,8 * 4,4 * 8,4 * 4, referring to accompanying drawing 1.Wherein: f (i, j, t) the expression t brightness value of current image frame piece coordinate position constantly, function abs () represent absolute value, SAD (x, y, n) represent motion vector be (x, y) and the estimation cost function of reference frame when being n.Suppose that the current block matrix is C, (n) relative current image block motion vector is that ((x, y n) are then remaining matrix E for x, reference block matrix y) to P in n reference frame of expression for x, y

E (x, y, n)=C-P (x, y, n) formula (2)

H.264 adopted and the similar hybrid encoding frame of former coding standard, different is the new technology that all adopted on prediction, conversion, entropy coding, has further improved code efficiency.H.264 the inter prediction technology mainly comprise seven kinds of block modes (as Fig. 1) of motion compensation, various block modes motion vector prediction, be accurate to sub-pixel interpolation of 1/4 etc.Experiment confirm motion compensation process H.264 is to its code efficiency raising effect maximum, and adopt the whole pixel motion compensation of the block mode of different sizes, can improve about 15% code efficiency than the pattern that only adopts 16X16, if combine, can obtain higher code efficiency with existing rate-distortion optimization (RDO) block mode decision model.

But the high efficiency of its coding is to be cost with complexity that increase to calculate.If each block mode separate searches, even adopt the fastest searching method at present, its amount of calculation is also well imagined.Want to reach real-time coding, too high to System Hardware Requirement, cost may improve exponentially.If adopt full search method, and every kind of block mode carries out separate searches, and its whole pixel search complexity will increase more than 7 times.Thereby reduce the different masses pattern search amount of calculation will to bring into play H.264 the advantage of code efficiency play a part to cause close important.Be to improve the speed of H.264 whole pixel search, must take all factors into consideration the correlation between method such as its integer transform, quantization method, vector forecasting and each block mode thereof.

Summary of the invention

The present invention is on existing method for estimating basis, at H.264 waiting video encoding standard to propose a kind of method that adapts to multiple block mode estimation.Under the condition that guarantees original encoding rate distorted characteristic, greatly reduced operand, improved speed of coding greatly, and algorithm thought has been simple, be easy to realize.This method is not only applicable to H.264 international standard, and visual concrete condition is used in combination with sub-pix searching method and rate-distortion optimization (RDO) block mode decision model in coding.

The present invention is from the correlation of various block modes, in conjunction with the prediction of starting point and carry out the search of various block modes by method such as withdrawing from advance.The technical solution used in the present invention is as follows:

The fast whole pixel motion estimating method of multiple block mode is characterized in that comprising following four parts: the detection of determining to reach skip macroblock (skipped macroblock) of starting point; The rhombus band searching method of smallest blocks pattern; The searching method of other block mode; The search cutoff criterion, the various piece performing step is as follows:

A. the detection of determining to reach skip macroblock (skipped macroblock) of starting point:

Starting point is set at the position that may become skip macroblock, and whether detect this macro block in starting point be skip macroblock.As this macro block is skip macroblock, then no longer advances piece match search and macroblock coding, directly carries out the piece match search and the coding of next macro block; Otherwise begin next step search;

B. the rhombus ring searching method of smallest blocks pattern

This proposes two kinds of rhombus ring searching methods in step: rhombus ring ring diffusion search method and based on the rhombus ring ring diffusion search method of correction for direction.These two kinds of methods as the region of search, and are sorted out the rhombus ring with diamond-shaped area as follows:

N＝abs(MV_X)+abs(MV_Y)

During search, press the order of rhombus ring, the search of a rhombus ring of rhombus ring ground, up to midway by or finish search in the region of search, second method has proposed a kind of rhombus ring ring diffusion search method based on correction for direction, correction for direction is by the statistics sad value, progressively abandon the search of maximum statistics sad value direction, up to the match block that searches the sad value minimum, speed is more previous fast, but precision is a little bit poorer slightly; Midway by meeting following one of them condition:

1)N _cur-N _min＞＝2

N _CurBe current search rhombus ring number, N _MinMinimum SAD place rhombus ring number,

2) reference block of current search rhombus ring SAD minimum meets the 4th part search cutoff criterion;

C. the searching method of other block mode: other block mode search method is based on by a kind of searching method of smaller piece pattern to relatively large pattern search, determine the upper left corner, the upper right corner, the lower left corner, the lower right corner of relatively large pattern search scope by the smaller piece motion vector, promptly determine the moving-square search scope of its search, and search for entirely or search fast in this scope, reduce thus to reduce amount of calculation than the method for the hunting zone of large model;

D. search for cutoff criterion: the reference of this search cutoff criterion is international standard integer transform and quantization method H.264, with 4 * 4 block sizes serves as to detect unit, by to the remaining matrix of each estimation of 4 * 4 through integer transform and after proofreading and correct, according to its 16 coefficients whether all less than threshold value H1, determine whether each 4 * 4 block search ends, this cutoff criterion is used for the detection of first step skip macroblock (skipped macroblock) and the detection that the second step smallest blocks pattern search is ended.

Below above four parts are further described:

Determining of the detection of skip macroblock (skipped macroblock) and starting point

H.264 the standard code skip macroblock must meet following three criterions:

1) 16 * 16 block mode.

2) as shown in Figure 2, the A piece is a current coding macro block, and L piece and T piece are respectively the left side and top smallest blocks pattern.If arbitrary motion vector level of L piece and T piece with vertical all be zero, i.e. null vector, and A piece motion vector is a null vector; If all non-vanishing vector of the motion vector of L piece and T piece, and A piece motion vector is the predicted vector of its 16 * 16 block mode.

3) the remaining matrix behind the inter prediction is 0 after integer transform and quantification entirely.

Whether the first step of the present invention at first detects is skip macroblock.As be skip macroblock, so whole macro block need not to carry out match search and macroblock coding again, directly enters the estimation and the coding of next macro block; Otherwise, carry out the search of second step.The search concrete steps are as follows:

1) at first with above-mentioned criterion 2) determine that the A piece does exercises vector as the starting point of estimation.

2) obtain the starting point place behind inter prediction remaining matrix, adopt this to illustrate that the search of the 4th part detects this remnants matrix by (half stop) criterion and whether meets above skip macroblock criterion.

3) as meeting the skip macroblock criterion, then withdraw from the current macro estimation; Otherwise begin the search of second step.

2. the rhombus ring search method of smallest blocks pattern

The smallest blocks pattern may be 4 * 4 block modes, also may be 8 * 8 block mode.What be worth emphasizing is, the accuracy of smallest blocks pattern search often directly has influence on the search precision of other block mode, so adopt the higher searching method of search precision as far as possible, we propose two kinds of rhombus ring search methods, and its feature is as follows:

(1) hunting zone no longer is traditional square scope, and directly adopts the rhombus scope, has simplified the software and hardware realization;

(2) Gu Ding search pattern and good extensibility are easy to the realization of software and hardware;

(3) since H.264 motion vector coding adopt UVLC variable length coding method, motion vector residual error data (MVD) is big more, its code length is long more.Rhombus ring search method adopt from than the long motion vector of short code position to strategy than the long motion vector location finding of long code, the easier match block that searches near the current macro position.This also is the characteristics of a maximum of this method.

Rhombus ring search method can be-7 ~+7 diamond search scope among the figure referring to accompanying drawing 3, and wherein a is the search starting point, and the rhombus ring is determined by following formula under the searching position:

N=abs (MV_X)+abs (MV_Y) formula 3

Suppose that rhombus ring search starting point a is the 0th a rhombus ring, b ~ h is respectively the 1st ~ 7 a rhombus ring, by that analogy.N is rhombus ring numbering in the above formula, and MV_X, MV_Y are respectively the level and the vertical component of motion vector.The other numeral rhombus ring direction of letter in the accompanying drawing 3, as b1, b2, b3, b4 represent the four direction of b ring.

1) rhombus ring ring diffusion search method

The region of search that the front has been introduced the rhombus ring substantially is provided with, and the simplest rhombus ring search method is carried out according to the following steps:

(1) searching position a after finishing skip macroblock and detecting, as does not meet the skip macroblock criterion, then begins to search for match block (as shown in Figure 3) along the b loop wire.

(2) finish b ring search after, as current search to minimum sad value in the b ring, detect 4 * 4 of minimum sad value and whether meet cutoff criterion (this illustrate that the 4th part set forth), as meet and then stop this 4 * 4 block search, begin next search of 4 * 4; As do not meet, then begin the search of c ring.

(3) down search of ring so one by one, when meeting following formula or having searched for the regulation hunting zone, this 4 * 4 block search stops, next 4 * 4 block search of beginning:

N _Cur-N _Min＞=2 formula 4

N _CurBe current search rhombus ring number, N _MinMinimum SAD place rhombus ring number.

2) the rhombus ring ring based on correction for direction spreads search method

Last method sequence more for static block or that mostly be mass movement can search match block rapidly and accurately, but less to moving mass, and the sequence of the difficult prediction of motion, then to consume a lot of time, based on last a kind of method, a kind of searching method is faster proposed here.Step is as follows:

(1) as above method search b, c, d rhombus ring, and calculate total sad value of four direction respectively

SAD1 _Sum, SAD2 _Sum, SAD3 _Sum, SAD4 _Sum, wherein:

SAD1 _sum＝SAD _b1+SAD _c1+SAD _d1

SAD2 _sum＝SAD _b2+SAD _c2+SAD _d2

SAD3 _Sum=SAD _B3+ SAD _C3+ SAD _D3Formula 5

SAD4 _Sum=SAD _B4+ SAD _C4+ SAD _D4SAD _B1, SAD _B2, SAD _B3, SAD _B4Be respectively the total sad value of b ring four direction, SAD _C1, SAD _C2, SAD _C3, SAD _C4Be respectively the total sad value of c ring four direction, SAD _D1, SAD _D2, SAD _D3, SAD _D4Be respectively the total sad value of d ring four direction.As at search b, c, when satisfying as first method cut-off condition in the d process, search ends; Otherwise compare SAD1 _SumSAD2 _SumSAD3 _SumSAD4 _SumFour values, and find out maximum and mark maximum place direction, the match block search of this direction abandoned at next step.

(2) also do not stop as search, continue the search of other three directions of e rhombus ring, and these three all SAD of direction are added its respective direction SADX _Sum(X represents certain direction), when f rhombus ring is searched for, abandon maximum SADX _SumThe search of direction, so search is gone down, and in above search procedure, the search cut-off method of pressing first method is by search, up to having only a direction.Do not stop as search, press

This direction is finished search.

More than two kinds of methods all based on a hypothesis: the SAD minimum value near the predicted value place is optimum matching point.Second method is also based on following hypothesis: be less than other sad value near SAD around the minimum sad value.Set up for these two hypothesis of most situations, also sometimes and be false.Experiment is found, though can not find optimum Match sometimes, can guarantee original rate distortion characteristic to the full extent when guaranteeing higher search speed.

More than the characteristics of two kinds of method maximums be that independence is better in each step and between the step, easily Parallel Implementation helps hardware and DSP and realizes.Traditional searching method adopts misalignment (center bias) method more, as diamond search (ds) (Diamond search), and hexagon search method (hexagon search) etc., correlation is big between its each step, is difficult for Parallel Implementation.

3. the search of other block mode

This search method adopts the strategy from the smaller piece pattern to relatively large pattern, progressively search.Is that example is set forth at this with smallest blocks pattern 4 * 4.After finishing in the macro block 16 smallest blocks pattern searches, the search of beginning 4 * 8 block modes.Form by two 4 * 4 for each 4 * 8, can produce this rectangular search scope (as shown in Figure 4) of 4 * 8 thus, suppose two 4 * 4 motion vectors be respectively (MV0_X, MV0_Y), (MV1_X, MV1_Y) hunting zone produces formula following (as shown in Figure 4):

The LT coordinate is: (min (MV0_X, MV1_X), min (MV0_Y, MV1_Y))

The LB coordinate is: (min (MV0_X, MV1_X), max (MV0_Y, MV1_Y)) formula 6

The RT coordinate is: (max (MV0_X, MV1_X), min (MV0_Y, MV1_Y))

The RB coordinate is: (max (MV0_X, MV1_X), max (MV0_Y, MV1_Y))

Usually, because image spatially moves and has certain continuity, the piece in same moving object particularly, the hunting zone that is drawn is minimum thus, but the utmost point finds match block exactly, therefore can in this scope, adopt full search, can't be too big to the search speed influence, can certainly adopt search strategy more fast.

Finish after the search of current macro 4 * 8 block modes, can adopt similar approach to finish 8 * 4, the search of bigger block mode such as 8 * 8 grades has greatly reduced the amount of calculation that match block is searched for.

Finish after the search of various block modes, cooperate other sub-pix search method again, and rate distortion (RD0) model is finished the decision-making of the best macro block (mb) type of current macro.

4. search is by (half stop) criterion

The search cutoff criterion is used in the detection and the second step smallest blocks pattern search of first step skip macroblock to be ended.The detection of first step skip macroblock is that unit is judged with one 4 * 4 piece, detects up to finishing 16 pieces.Do not meet cutoff criterion as one of them piece, promptly withdraw from detection, carry out next step search.Second step, each rhombus ring was once searched for by detecting during the smallest blocks pattern search.

Because H.264 adopt the integer transform of class DCT, the conversion meeting brings certain distortion, and adopt quantization method, conversion coefficient is compensated.In order to simplify the detection method that quantizes the back zero coefficient, at first to proofread and correct the coefficient after its conversion, formula is as follows:

Y=(CEC ^T)  T formula 7

Wherein: $T=\left[\begin{array}{cccc}0.256& 0.160& 0.256& 0.160\\ 0.160& 0.100& 0.160& 0.100\\ 0.256& 0.160& 0.256& 0.160\\ 0.160& 0.100& 0.160& 0.100\end{array}\right],$ $C=\left[\begin{array}{cccc}1& 1& 1& 1\\ 2& 1& -1& -2\\ 1& -1& -1& 1\\ 1& -2& 2& -1\end{array}\right]$

T is unified correction matrix, and E is 4 * 4 a luminance block prediction back residual matrix, and C is a class dct transform matrix, and  representing matrix corresponding element multiplies each other, and Y is and proofreaies and correct the back matrix.At first obtaining quantification back value is the preceding correction coefficient H1 of 1 pairing quantification, and with its threshold value as detection.During promptly less than the H1 value, then it is considered as zero coefficient.

H1=0.256 * (2 ^Q_bits-qp_const)/quant_coeff[qp_rem] formula 8 wherein: qp_per=QP/6, qp_rem=QP%6, q_bits=15+qp_per, qp_const=(2 ^Q-bits)/6.The QP maximum is 51, and qp_rem is an integer 0 ~ 5, quant_coeff[6]=13107,11916,10082,9362,8192,7282}.

Search is carried out as follows by detecting:

1) as stated above, 4 * 4 matrix of differences are carried out integer transform, and correction matrix;

2) whether detect 16 coefficients one by one greater than threshold value H1,, then withdraw from case the coefficient greater than H1 is arranged.Search cutoff criterion implementation basis:

1) H.264 integer transform has been simplified computational complexity to a great extent, carries out integer transform at this, as long as not too frequent judgement can not increase too many amount of calculation;

2) can not be too frequent by judging, judge once that as each searching position its motion calculation amount is too big, a general rhombus ring is once judged;

3) in the testing process, also can be long by ignoring some strokes, its value (Level) is 1 coefficient after quantizing, thereby improves coding rate and code efficiency.

Characteristics of the present invention and effect:

The present invention proposes the whole fast pixel motion estimating method of the multiple block mode of a kind of adaptation.This method has guaranteed existing encoding rate distorted characteristic to the full extent, has greatly reduced the amount of calculation of estimation.Its search precision is determined by smallest blocks pattern search precision; Its estimation speed is relevant with the encoder target bit rate, and target bit rate is low more usually, and average quantisation parameter is big more, and estimation speed is fast more.Be used in combination with rate-distortion optimization (RDO) macro block (mb) type decision model with the sub-pix searching method, obtain the optimal encoding code-rate distortion property.The invention is not restricted to H.264 international standard, can consider that concrete condition suitably uses.

Description of drawings:

Fig. 1 is seven kinds of inter motion compensation block modes of standard code schematic diagram H.264;

Fig. 2 is standard code 16 * 16 block mode motion vector prediction schematic diagrames H.264;

Fig. 3 is a rhombus ring method for estimating hunting zone schematic diagram;

Fig. 4 is other block mode hunting zone schematic diagram except that the smallest blocks pattern.

Embodiment

The whole pixel motion estimating searching method that is adapted to multiple block mode that the present invention proposes mainly is applicable to the video encoding standard with multiple block mode, as H.264 waiting, mainly carries out as follows:

1. the detection of determining to reach skip macroblock (skipped macroblock) of starting point

Starting point is at first determined in search, begins search with the position that may become skip macroblock (skipped macroblock) as starting point.This step is relevant with video encoding standard.But as be defined as skip macroblock, then no longer carry out the piece match search with coding, continue the search of next macro block, otherwise begin the search in second step.

2. the rhombus ring search method of smallest blocks pattern

This step is the key of estimation, and its search precision directly influences next step other block mode search precision.Piece is more little, and often coupling is difficult more, the block mode of 4 * 4 sizes particularly, and minimum sad value match block is best matching blocks not necessarily also.This step the invention provides two searching methods, and promptly rhombus ring ring diffusion search method and based on the rhombus ring ring diffusion search method of correction for direction can cooperate the search cutoff criterion, accurately finishes the coupling of 4 * 4 block modes.

3. the search of other block mode

Its piece matched position of relatively large pattern is relevant with the smaller piece pattern, thus finish the smaller piece pattern search earlier, then by relatively large pattern search.Big block mode is searched in relevant little block mode scope.Because its scope is less usually, can search for entirely in its scope.Also can consider concrete condition, in this scope, adopt fast algorithm.

4. search is by (half stop) criterion

The search that the search cutoff criterion is mainly used in the detection of first step skip macroblock and the second step smallest blocks pattern ends.Cutoff criterion is with reference to H.264 integer transform and quantization method, this method serves as to detect unit with one 4 * 4 piece, and residual volume is made integer transform after at first will estimating, proofreaies and correct conversion coefficient then, correction matrix and threshold value H1 are compared one by one, then withdraw from detection as coefficient greater than threshold value H1.

This cutoff criterion accuracy of judgement but operand is bigger should not judge all by each search point that as adopting rhombus ring search method, general rhombus ring search is finished the back and done to end judgement.

The present invention has tested on the H.264 encoder of inventor's independent development and has passed through.Following technology during H.264 this encoder has adopted:

1) reference frame number: 1

2) frame type: I frame, P frame

3) rate-distortion optimization (RD0)

4) CAVLC entropy coding

5) whole pixel motion estimation range: 32

6) estimation block mode: comprise 7 kinds of block modes

7) 1/4 pixel search precision

By this The Application of Technology, on the PC of 1GHz CPU, to foreman, mobile, paris, CIF sequential codings such as bus can reach 6-10 frame/second, can reach 20-40 frame/second to QCIF sequences such as foreman.Under the condition of multimedia extensions such as MMX of no use, SSE instruction, the QCIF sequence can realize real-time coding on PC.And encoded image has kept original rate distortion characteristic.

Claims (1)

1. the fast whole pixel motion estimating method of many kinds of block modes is characterized in that comprising following four parts: the detection of determining to reach skip macroblock (skipped macroblock) of starting point; The rhombus band searching method of smallest blocks pattern; The searching method of other block mode; The search cutoff criterion, the various piece performing step is as follows:

A. the detection of determining to reach skip macroblock (skipped macroblock) of starting point:

Starting point is set at the position that may become skip macroblock, and whether detect this macro block in starting point be skip macroblock.As this macro block is skip macroblock, then no longer advances piece match search and macroblock coding, directly carries out the piece match search and the coding of next macro block; Otherwise begin next step search;

B. the rhombus ring searching method of smallest blocks pattern

This proposes two kinds of rhombus ring searching methods in step: rhombus ring ring diffusion search method and based on the rhombus ring ring diffusion search method of correction for direction.These two kinds of methods as the region of search, and are sorted out the rhombus ring with diamond-shaped area as follows:

N＝abs(MV_X)+abs(MV_Y)

During search, press the order of rhombus ring, a rhombus ring of rhombus ring ground search, up to midway by or finish search in the region of search.Second method has proposed a kind of rhombus ring ring diffusion search method based on correction for direction, correction for direction is by the statistics sad value, progressively abandons the search of maximum statistics sad value direction, up to the match block that searches the sad value minimum, speed is more previous fast, but precision is a little bit poorer slightly;

Midway by meeting following one of them condition:

1)N _cur-N _min＞＝2

N _CurBe current search rhombus ring number, N _MinMinimum SAD place rhombus ring number,

2) reference block of current search rhombus ring SAD minimum meets the 4th part search cutoff criterion;

C. the searching method of other block mode: other block mode search method is based on by a kind of searching method of smaller piece pattern to relatively large pattern search, determine the upper left corner, the upper right corner, the lower left corner, the lower right corner of relatively large pattern search scope by the smaller piece motion vector, promptly determine the moving-square search scope of its search, and search for entirely or search fast in this scope, reduce thus to reduce amount of calculation than the method for the hunting zone of large model;

D. search for cutoff criterion: the reference of this search cutoff criterion is international standard integer transform and quantization method H.264, with 4 * 4 block sizes serves as to detect unit, by to the remaining matrix of each estimation of 4 * 4 through integer transform and after proofreading and correct, according to its 16 coefficients whether all less than threshold value H1, determine whether each 4 * 4 block search ends, this cutoff criterion is used for the detection of first step skip macroblock (skipped macroblock) and the detection that the second step smallest blocks pattern search is ended.

Images