CN101198065A - Frame and intraframe coding mode selection method - Google Patents

Abstract

The invention provides a selection method of an encode mode between interframe and inframe, which includes the following steps: (1) after the selection of prediction model of the interframe motion compensation is completed and the interframe model cost JInter is calculated, an Intra cost Ji of four square blocks at the top of a micro block is calculated, and image reconstructions of the four square blocks are completed, the i represents the number of luminance block, wherein, I is more than or equal to 0 and less than or equal to 3; (2) relative index SADBE of the inner space of the micro block are calculated; (3) a predicted value JInter of the Intra cost is calculated, according to the Intra cost of the four square blocks at the top of the micro block, an offset QP0(QP) of mode selection and the relative index of the inner space of the micro block; (4) whether the predicted Intra cost JInter is larger than the calculated Inter cost JInter or not is judged, if yes, the encode mode of the micro block is determined to be an Inter mode, and the method is completed.

Description

The system of selection of a kind of interframe and intra-frame encoding mode

Technical field

The present invention relates to a kind of coding mode selection algorithm in the real-time video coding.

Background technology

Video coding algorithm comprises intraframe coding and has two kinds of patterns of interframe encode of motion compensation interframe prediction.The interframe encoding mode (back is called the Inter pattern) that has the motion compensation interframe prediction has impayable advantage for intra-frame encoding mode (back is called the Intra pattern) on compression ratio.Therefore, adopt the Inter pattern to obtain high compression ratio in picture frame one-level and 16 * 16 image blocks (being called macro block) one-level, be exactly P frame and Inter macro block as far as possible.Only when initial time (comprising that the absolute initial time of sequence and incision are constantly) and needs are corrected or dealt with transmission error, use I frame (coding mode of picture frame one-level is Intra), interframe block when it fails to match with need topography's off resonance correction or use Intra macro block (coding mode of macro block one-level is Intra) when refreshing.

Wherein, it fails to match causes by blocking for interframe block.At this moment, there is not corresponding relation between present frame partial image region and the reference frame image or to deserved incorrect, the distortion performance of Inter pattern is also poorer than Intra pattern.Though the interframe block of Same Scene coupling is successful in most cases, it fails to match that situation only occupies the minority for such piece, if the untimely Intra pattern that makes into will cause topography's quality degradation, as smear and residual effect occur.Common solution is when carrying out P frame coding each macro block to be carried out the trial of Inter pattern and Intra pattern simultaneously, calculate the distortion performance amount of measuring separately, if can realize Lagrangian cost J=D+ λ * R that code check R and distortion factor D optimize simultaneously, therefrom elimination factor distortion performance pattern preferably.

H.264Intra 4 * 4 brightness infra-frame predictions that pattern comprised require to utilize preceding adjacent 4 * 4 reconstructed image data generation forecast value of encoding and decoding order.And if only if after previous 4 * 4 reconstructed image generates, and could begin to handle back one 4 * 4.So, in order to obtain the Lagrangian cost J of 4 * 4 Intra patterns, must be to the Intra macroblock coding process of each 4 * 4 luminance block complete.Its subsidiary Intra pattern-coding result who generates only is defined as under the situation of Intra pattern just useful at macro-block coding pattern at last.And the chance that the Intra macro block occurs in the P frame is very low.In most cases, Intra pattern-coding result becomes hash and is dropped.

H.263TMN12 interframe and frame mode selection algorithm are much simple: only need obtain the auto-variance of macro block, the absolute value and the SAD that just can be used as Intra cost and Inter prediction residue compare, and do not need to carry out the computing of any Intra coding.Unfortunate this method can not be in use to H.264 coding.Because H.264Intra pattern has comprised the H.263Intra unexistent plane of delineation spatial prediction of pattern, must compare with Inter prediction residue with intra prediction.

Document Fast inter mode select, JVT of ITU-T VCEG and ISO/IEC MPEG, Document JVT-I020, San Diego, USA, September 2003, a kind of interframe and frame mode selection algorithm are disclosed: after finishing the Inter model selection, relatively relativity of time domain and spatial correlation determine it is to carry out the Intra pattern to attempt according to comparative result, determine that still macro-block coding pattern is the Inter pattern.This algorithm definition square spatial correlation index ABE is poor (boundary line one side original image value of the pixel value of the crossing boundary pixel of the adjacent macroblocks before current macro and the coded sequence, use on one side the reconstructed image value in addition) mean value, definition square relativity of time domain index AR is the average length of best Inter pattern down conversion coefficient code stream.If AR＜kABE (k is predefined positive number), relativity of time domain is greater than spatial correlation, and the forced coding pattern is the Inter pattern, the trial that can skip the Intra pattern; Otherwise spatial correlation needs to carry out the Intra pattern and attempts greater than relativity of time domain.

Practical application shows that this algorithm can reduce the chance that the Intra pattern is attempted significantly, effectively reduces operand, simultaneously to the influence of picture quality very little (＜0.1dB).Can comprise that this algorithm is relatively needing to generate Inter mode conversion coefficient code stream before relativity of time domain and the spatial correlation by improvements, the process of twice generation conversion coefficient code stream may appear in each macro block like this, has the leeway of further simplifying processing procedure and minimizing operand; This algorithm can only be used for the not limited coding situation of infra-frame prediction, can not be used for the limited coding situation of infra-frame prediction.

The not limited situation of infra-frame prediction allows to go to predict current macro with the border pixel values of any macro block (comprising Inter macro block and Intra macro block) adjacent in the same slice, the limited situation of infra-frame prediction only allows to go to predict current macro with the border pixel values of Intra macro block adjacent in the same slice, does not allow to go to predict current macro with the border pixel values of adjacent Inter macro block.H.264 the not limited and limited two kinds of the encoding options of infra-frame prediction have been defined.The infra-frame prediction efficient that infra-frame prediction is not limited and Intra mode compression be than higher, and the limited code stream fault-tolerance of infra-frame prediction is stronger.Unless because neighboring macro-blocks is the Intra macro block entirely, otherwise the image reconstruction of current I ntra macro block was not relevant with the reconstructed image of past frame when infra-frame prediction was limited, may transmit the mistake image of past frame; The reconstructed image of the image reconstruction of current I ntra macro block and past frame was irrelevant fully when infra-frame prediction was limited, can play the effect of picture refreshing.And only consider with the frame mode selection algorithm between fast frame in the prior art and can only be applied to the not limited coding situation of infra-frame prediction.

The fault-tolerance of video source code stream is the key performance of multimedia communication product.Especially for the H.323 video conference terminal of the IP network transmitting multimedia data that guarantees by no QoS, more apparent outstanding by the meaning of the limited Intra macro block updating topography mistake of a small amount of infra-frame prediction in the code stream.Be applicable to that the limited coding situation of infra-frame prediction omits meaningless processing by self adaptation simultaneously, the method that effectively reduces the quick selection interframe of operand and frame mode becomes the active demand of commercialization.

Summary of the invention

The technical problem to be solved in the present invention provides a kind of interframe and mode selection method in frame, in the limited the encoding option of the infra-frame prediction of strong fault tolerance, can not only refresh topography's mistake, and omit meaningless processing by the trial self adaptation of abort frame internal schema ahead of time, further reduce operand, thereby finish fast between macroblock frame and the selection of frame mode, improve coding rate.

The present invention proposes the system of selection of a kind of interframe and intra-frame encoding mode, may further comprise the steps:

(1) finishes the selection of inter motion compensation predictive mode and trying to achieve inter-frame mode cost J _InterAfter, the Intra cost J of 4 squares in computing macro block top _i, and finishing the image reconstruction of these 4 squares, described i represents the numbering of luminance block, 0≤i≤3;

(2) computing macro block inner space correlation index S AD _BE

(3) according to the Intra cost of 4 squares in macro block top, the side-play amount QP of model selection ₀(QP) and interior macroblocks spatial coherence index, calculate Intra cost predicted value

(4) judge the Intra cost that dopes

Whether greater than the Inter cost J that has tried to achieve _Inter, if determine that then macro-block coding pattern is the Inter pattern, finishes.

Further, said method also can have following characteristics: in the step (4), if the Intra cost that dopes

Less than the Inter cost J that has tried to achieve _Inter, execution in step (5),

(5) 4 * 4 squares calculate the Intra cost J of 12 4 * 4 remaining squares one by one _i, and the Intra cost J that adds up and obtain macro block _Intra

(6) judge the J as a result that this time adds up _IntraWhether greater than J _Inter, if, determine that macro-block coding pattern is the Inter pattern, finish, otherwise, continue execution in step (7);

(7) judge whether to calculate the Intra cost of all 16 squares in the macro block,, finish if the coding mode of current macro is set to Intra, otherwise, execution in step (5) continued.

Further, said method also can have following characteristics: the method for calculating each square Intra cost in the step (1) comprises:

(a) select 4 * 4 infra-frame prediction modes, a kind of prediction mode of elimination factor distortion performance the best, generation forecast image;

(b) according to raw image data-predicted image data=residual error data, further absolute value and the SAD by the side-play amount+residual error data of 4 * 4 intra prediction modes calculates the Intra cost.

Further, said method also can have following characteristics: in the step (2), and computing macro block inner space correlation index S AD _BEMethod comprise:

${\mathrm{SAD}}_{\mathrm{BE}}=\underset{i=0}{\overset{15}{\mathrm{\Σ}}}|Y_{\_{\mathrm{org}}_i}-Y_{\_r{\mathrm{ec}}_i}|$

Wherein, Y_org _iBe the original image pixels value of current macro the 4th row i row, Y_rec _iIt is the reconstructed image pixel value of current macro the 3rd row i row.

Further, said method also can have following characteristics: calculate Intra cost predicted value in the step (2)

Method comprise:

${\hat{J}}_{\mathrm{Intra}}={\mathrm{QP}}_0\left(\mathrm{QP}\right)+\underset{i=0}{\overset{3}{\mathrm{\Σ}}}{J}_i+12*{\mathrm{SAD}}_{\mathrm{BE}}$

Described QP ₀(QP) be a constant.

Further, said method also can have following characteristics: described method is applicable to that infra-frame prediction is limited, comprise the video coding of the macro-block coding pattern of brightness 4 * 4 infra-frame predictions simultaneously.

Compared with prior art, technique effect of the present invention is to omit meaningless processing by the trial self adaptation of doing sth. in advance abort frame intraprediction encoding pattern under the limited the encoding option of the infra-frame prediction of strong fault tolerance, thereby finish fast between macroblock frame and the selection of frame mode, significantly improve coding rate.

Description of drawings

Fig. 1 (a) is the coded sequence figure of prior art regulation.

Fig. 1 (b) is the coded sequence figure in the embodiment of the invention.

Fig. 2 calculates interior macroblocks spatial coherence schematic diagram among the embodiment.

Fig. 3 is the method flow diagram among the embodiment.

Embodiment

Under the limited situation of infra-frame prediction, have only adjacent Intra macro block just can provide infra-frame prediction.Obviously, the distortion performance of Intra pattern can not be assessed with the boundary pixel value difference of current macro in the prior art and adjacent macroblocks.

In order to select Intra pattern in interframe Inter pattern and the frame fast, the present invention is adjusted into the processing sequence shown in Fig. 1 (b) with the coded sequence of 16 4 * 4 luminance block in the macro block of regulation among Fig. 1 (a).Like this after handling 44 * 4 squares that are positioned at the macro block top, computing macro block volume inside correlation index immediately, be pixel value between the lastrow (being the 4th row among Fig. 2) of next line (the 3rd row among Fig. 2) and 4,5,6,7 original images of 0,1,2,3 reconstructed image in the calculating chart 1 (b) absolute value difference with, and with this absolute value difference be Intra cost predicted value.By relatively Inter cost and Intra cost prediction value decision are to continue the trial of Intra pattern or stop Intra pattern trial (determining that the Inter pattern is best macro-block coding pattern).So just can after handling 44 * 4, stop trial of Intra pattern and Intra cost earlier and calculate, reduce operand.

Certainly, for compatibility and the interoperability that keeps code stream, the concrete processing method of each 4 * 4 square is still carried out according to standard fully, comprises 4 * 4 square space adjacency judgements, macro block code flow structure etc.Just 44 * 4 the coded sequence of changing, the entropy coding order that interior macroblocks is 4 * 4 still as before.

Below in conjunction with accompanying drawing embodiments of the invention are elaborated.As shown in Figure 3, this method may further comprise the steps:

Step 110 is finished the selection of inter motion compensation predictive mode and is being tried to achieve inter-frame mode cost J _InterAfter, to 44 * 4 squares in macro block top, promptly among Fig. 1 (b) 0,1,2,3, calculate its Intra cost, the Intra cost of 4 squares is respectively J ₀, J ₁, J ₂, J ₃, and finish the image reconstruction of these 4 squares;

The Intra cost of existing each square of calculating is also finished the method for image reconstruction, may further comprise the steps:

(a) select 4 * 4 infra-frame prediction modes, from H.264 standard provided a kind of prediction mode of elimination factor distortion performance the best 9 kinds of prediction mode altogether, generation forecast image;

(b) according to raw image data-predicted image data=residual error data, further absolute value and the SAD by the side-play amount+residual error data of 4 * 4 intra prediction modes calculates the Intra cost;

(c) residual error coefficient coding is generated conversion coefficient Run-Level symbol, wherein step comprise orthogonal transform, quantification, word scanning generate Run-Level symbol, inverse quantization, the local reconstructed image of generation encoder with Run-Length Coding.

Supplementary notes, said process is compared with complete H.2644 * 4 intraframe predictive coding process, only poor one step of entropy coding.Even under the requirement that reduces the model selection operand to greatest extent, also only can omit this step.

Step 120 is according to the method for calculating square spatial correlation index ABE, computing macro block inner space correlation index S AD _BE,

${\mathrm{SAD}}_{\mathrm{BE}}=\underset{i=0}{\overset{15}{\mathrm{\Σ}}}|Y_{\_{\mathrm{org}}_i}-Y_{\_r{\mathrm{ec}}_i}|$

Wherein, Y_org _iBe the original image pixels value of current macro the 4th row i row among Fig. 2, Y_rec _iIt is the reconstructed image pixel value of current macro the 3rd row i row among Fig. 2.

The method of this step is referring to document Fast inter mode select, JVT of ITU-T VCEG andISO/IEC MPEG, and Document JVT-I020, San Diego, USA, September 2003.

Step 130 is calculated Intra cost predicted value

${\hat{J}}_{\mathrm{Intra}}={\mathrm{QP}}_0\left(\mathrm{QP}\right)+\underset{i=0}{\overset{3}{\mathrm{\Σ}}}{J}_i+12*{\mathrm{SAD}}_{\mathrm{BE}}$

In the following formula,

Be frame mode cost predicted value, QP ₀(QP) being the side-play amount of model selection, is a constant, describes in detail to see H.264 Joint Model, J ₀, J ₁, J ₂, J ₃Be the frame mode cost at 44 * 4 at the top of 16 * 16 macro blocks, SAD _BEBe interior macroblocks spatial coherence index.

Under the limited situation of infra-frame prediction, the i.e. constrained_intra_pred_flag=1 that concentrates of image parameter H.264, though can not be before the Intra pattern be attempted according to macro block between the correlation trial of skipping the Intra pattern, but can processing finish time of 44 * 4 luminance block at the top of interior macroblocks according to interior macroblocks dependency prediction Intra cost, thereby insignificant processing is omitted in the trial of possible premature termination Intra pattern.

Step 140 is judged the Intra cost that dopes

Whether greater than the Inter cost J that has tried to achieve _Inter, if, execution in step 150, otherwise, execution in step 160;

Step 150 stops the Intra cost immediately and calculates, and determines that macro-block coding pattern is the Inter pattern, finishes;

For the Inter piece that occurs the chance maximum in the P frame, therefore corresponding relativity of time domain stops the continuation trial of unnecessary Intra pattern herein always greater than spatial correlation with high probability.As for remaining sub-fraction Inter pattern situation, also can be by progressively being got rid of the Intra pattern in the accurate Intra cost of 12 4 * 4 luminance block of macro block remainder is calculated with the comparison of Inter cost.

Step 160, continue with the Inter cost be the premature termination amount of attempting one by one 4 * 4 squares calculate the Intra cost J of 12 4 * 4 remaining squares _i(among Fig. 1 (b) 4～15), and the Intra cost J that adds up and obtain macro block _Intra,

$J_{\mathrm{Intra}}=\underset{i=0}{\overset{n}{\mathrm{\Σ}}}{J}_i$

Wherein, n≤15.

Step 170 is judged the J as a result that this time adds up _IntraWhether greater than J _Inter, if, execution in step 150, otherwise, execution in step 180 continued;

Step 180 judges whether to calculate the Intra cost of all 16 squares in the macro block, if, execution in step 190, otherwise, execution in step 160;

Step 190, the coding mode of current macro is set to Intra, finishes.

Macro block mode Intra just when handling 16 squares at last, coding result becomes useful data.The Intra macro block at P frame proportion seldom.Therefore, the Intra pattern operand attempting being spent significantly reduces.Also show in actual applications and both significantly reduced operand, guarantee the effect of picture quality and distortion performance again, the speed that improves P frame Intra pattern-coding reaches 32.12%, improves total processing speed 2.63%; The cost of being paid is to increase code check maximum 1.27%, or the maximum decline 0.077dB of picture quality.

Claims (6)

1. the system of selection of interframe and intra-frame encoding mode may further comprise the steps:

(1) finishes the selection of inter motion compensation predictive mode and trying to achieve inter-frame mode cost J _InterAfter, the Intra cost J of 4 squares in computing macro block top _i, and finishing the image reconstruction of these 4 squares, described i represents the numbering of luminance block, 0≤i≤3;

(2) computing macro block inner space correlation index S AD _BE

(3) according to the Intra cost of 4 squares in macro block top, the side-play amount QP of model selection ₀(QP) and interior macroblocks spatial coherence index, calculate Intra cost predicted value

(4) judge the Intra cost that dopes

Whether greater than the Inter cost J that has tried to achieve _Inter, if determine that then macro-block coding pattern is the Inter pattern, finishes.

2. method according to claim 1 is characterized in that, in the step (4), if the Intra cost that dopes

Less than the Inter cost J that has tried to achieve _Inter, execution in step (5),

(5) 4 * 4 squares calculate the Intra cost J of 12 4 * 4 remaining squares one by one _i, and the Intra cost J that adds up and obtain macro block _Intra

(6) judge the J as a result that this time adds up _IntraWhether greater than J _Inter, if, determine that macro-block coding pattern is the Inter pattern, finish, otherwise, continue execution in step (7);

(7) judge whether to calculate the Intra cost of all 16 squares in the macro block,, finish if the coding mode of current macro is set to Intra, otherwise, execution in step (5) continued.

3. method according to claim 1 is characterized in that, the method for calculating each square Intra cost in the step (1) comprises:

(a) select 4 * 4 infra-frame prediction modes, a kind of prediction mode of elimination factor distortion performance the best, generation forecast image;

(b) according to raw image data-predicted image data=residual error data, further absolute value and the SAD by the side-play amount+residual error data of 4 * 4 intra prediction modes calculates the Intra cost.

4. method according to claim 1 is characterized in that, in the step (2), and computing macro block inner space correlation index S AD _BEMethod comprise:

$\mathrm{SA}{D}_{\mathrm{BE}}=\underset{i=0}{\overset{15}{\mathrm{\Σ}}}|Y_{\_\mathrm{or}{g}_i}-Y_{\_\mathrm{re}{c}_i}|$

Wherein, Y_org _iBe the original image pixels value of current macro the 4th row i row, Y_rec _iIt is the reconstructed image pixel value of current macro the 3rd row i row.

5. as method as described in the claim 4, it is characterized in that, calculate Intra cost predicted value in the step (2)

Method comprise:

${\hat{J}}_{\mathrm{Intra}}=Q{P}_0\left(\mathrm{QP}\right)+\underset{i=0}{\overset{3}{\mathrm{\Σ}}}{J}_i+12*\mathrm{SA}{D}_{\mathrm{BE}}$

Described QP ₀(QP) be a constant.

6. method according to claim 1 is characterized in that:

Described method is applicable to that infra-frame prediction is limited, comprise the video coding of the macro-block coding pattern of brightness 4 * 4 infra-frame predictions simultaneously.

Images