CN101783951A - Method for rapidly confirming video coding inter-frame predicting mode based on human vision system - Google Patents

Abstract

The invention discloses a method for rapidly confirming a video coding inter-frame predicting mode based on a human vision system, which is characterized by comprising the following executing steps of: firstly, carrying out HVS filtration on an input reference frame image and a current frame image; then dividing a filtered image into macro-blocks and confirming a dividing way of the MB as one of five modes of SKIP, 16*16, 16*8, 8*16 and 8*8 according to an average value and a variance of each MB in an HVS filtered image; if the confirmed type is 8*8, calculating average values and variances of 8*8, 4*8, 8*4 and 4*4 and finally confirming a concrete corresponding mode of 8*8, 4*8, 8*4 and 4*4; and carrying out relevant motion estimation and motion compensation on the basis of confirmation of the inter-frame predicting mode.

Description

Video coding inter-frame predicting mode fast determination method based on the human visual system

Technical field

The present invention relates to inter-frame forecast mode in the video encoding standard and divide or determine method, be specifically related to a kind ofly carry out H.264/AVC definite method of video coding inter-frame predicting mode according to human visual system (HVS) characteristic.

Background technology

Inter-frame forecast mode in the video coding is owing to relate to estimation, so the complexity of calculating is very high, especially in video coding H.264/AVC, inter-frame forecast mode needs to determine a kind of pattern of optimum from a large amount of candidates' inter-frame forecast mode before determining.This coded system needs a large amount of computings,

Summary of the invention

The purpose of this invention is to provide a kind of method and carry out optimum H.264/AVC video coding inter-frame predicting mode selection according to HVS filtering, after determining optimization model, carry out estimation, such estimation and motion compensation process only carry out at selected pattern, and other non-selected pattern is not carried out estimation, therefore the execution efficient of encoder can be improved effectively, especially in the occasion of real-time coding, more its superiority can be embodied.

For reaching above purpose, the present invention takes following technical scheme to be achieved:

A kind of video coding inter-frame predicting mode fast determination method based on the human visual system is characterized in that, comprises the steps:

At first execution in step 01, and reference frame image and current frame image are carried out obtaining filtered residual image PRD based on the relevant filtering of human visual system's characteristic; While execution in step 06: current frame image is carried out intra prediction mode select, to determine a kind of intra prediction mode and respective coding cost RDC thereof of the best;

Execution in step 02 then, and residual image PRD is divided into the macro block MB of 16x16 and calculates average and the variance of each macro block MB;

Next execution in step 03, each macro block MB carried out inter-frame forecast mode divide, and MB is divided into SKIP pattern, a 16x16 piece, two 16x8 pieces, two 8x16 pieces or four 8x8 pieces; When the mode division of described macro block MB is the SKIP pattern, then skip to step 08 automatically; When the mode division of described macro block MB is 16x16,16x8, perhaps a kind of in three kinds of patterns of 8x16, skips steps 04 automatically then, execution in step 05, step 07 are to 08; When the mode division of described macro block MB is the 8x8 pattern, then execution in step 04;

Step 04 pair 8x8 block mode is divided into again: a 8x8 piece, two 8x4 pieces, two 4x8 pieces or four 4x4 pieces;

Next execution in step 05, and determined partition mode in step 03 and 04 is carried out estimation and motion compensation, with the motion vector MV of definite final divided block, and the coding cost RDC of calculating macro block MB under above-mentioned mode division;

Execution in step 07 then, selects current macro MB to adopt in the frame or inter-frame forecast mode according to the coding cost RDC of determined inter-frame forecast mode of step 05 and the determined intra prediction mode of step 06;

Last execution in step 08 is carried out entropy coder and is formed final coded-bit output.

In the said method, described average and the variance to each macro block of step 02 calculated and carried out according to following formula, to the size piece f that is MxN (m, average μ n) (M, N) and variance δ (M N) is:

$\mathrm{\μ}(M,N)=\frac{1}{N\×M}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}f(m,n)---\left(1\right)$

$\mathrm{\δ}(M,N)=\frac{1}{N\×M}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{(\mathrm{\μ}-f(m,n))}^2---\left(2\right)$

In the formula: M, the value of N is 4,8 or 16.

The described MB with a 16x16 of step 03 is divided into SKIP, 16x16,16x8, the basis of 8x16 and 8x8 pattern as shown in the formula:

Mode in the formula _16x16Represent the type that final determined MB divides, JND _ThRepresent the threshold value that corresponding 16x16 macro block MB sets;

Step 04 is described to be divided into 8x8 again with a 8x8 piece, 8x4, the basis of 4x8 and 8x8 pattern as shown in the formula:

In the formula: Mode _8x8Represent the type that final determined MB divides, JND _Th ⁸The threshold value that expression 8x8 macro block MB sets.

Compared with prior art, H.264/AVC method of the present invention can embed in the codec, therefore can not destroy code flow structure H.264/AVC, has good versatility.Can be with H.264/AVC codec be compatible mutually at present.In addition, QP and JND have been provided among the present invention _ThBetween mapping relations, can realize coding not changing under the situation of coding configuration like this, therefore have ease for use.

Description of drawings

The present invention is described in further detail below in conjunction with the drawings and the specific embodiments.

Fig. 1 is the steps flow chart block diagram of the video coding inter-frame predicting mode fast determination method based on HVS of the present invention.

Embodiment

As shown in Figure 1, the present invention is based on HVS carries out video coding H.264/AVC and determines that fast the method for inter-frame forecast mode comprises following execution in step:

At first execution in step 01, and reference frame and current encoded frame (P frame or B frame) are carried out obtaining filtered residual image (being called for short PRD) based on the relevant filtering of human visual system's characteristic (being called for short HVS filtering); Execution in step 06 is carried out intra prediction mode to current frame image and is selected simultaneously, to determine a kind of intra prediction mode and respective coding cost RDC thereof of the best;

Execution in step 02 then, and the filtered residual image PRD of HVS is divided into the macro block (be called for short MB) of 16x16 and calculates average and the variance of each MB;

Next execution in step 03, each MB carried out inter-frame forecast mode select, and MB is divided into: unit 030 pairing SKIP (promptly motion vector and the residual error of current MB not being encoded) pattern, or unit 031 a pairing 16x16 piece; Or unit 032 pairing two 16x8 pieces; Or unit 033 pairing two 8x16 pieces; Or a kind of in unit 034 pairing four 8x8 pieces.

If the mode division of a MB is the pairing SKIP pattern in Unit 030 in step 03, then skips to step 08 automatically and carry out this pattern of entropy coding and carry out code stream output; If the pattern that MB divides is 16x16,16x8, perhaps a kind of in three kinds of patterns of 8x16, skips steps 04 automatically then, execution in step 05 is to step 08; If the pattern that MB divides is a pairing 8x8 pattern in the unit 034, then need pattern execution in step 04 to these 4 8x8 pieces to determine its more meticulous pattern once more.Four 8x8 pieces being divided in step 04 pair Unit 034 are divided into again: unit 041 a pairing 8x8 piece; Or unit 042 pairing two 8x4 pieces; Or unit 043 pairing two 4x8 pieces; Or a kind of in unit 044 pairing four 4x4 pieces;

Next execution in step 05, current MB is carried out estimation and motion compensation according to determined partition mode in step 03 and 04, with the motion vector MV of definite final divided block, and the code check of calculating coding of current MB under above-mentioned mode division and distortion factor optimization relation function RDC;

Execution in step 07 then, in the frame and inter-frame forecast mode select, select current MB to adopt in the frame or inter-frame forecast mode according to the RDC of determined intra prediction mode in determined inter-frame forecast mode in the step 05 and 06 step.

Last execution in step 08 is carried out entropy coder and is formed final coded-bit output.

In above-mentioned steps 01, the method for prediction reference frame and current reference frame being done HVS filtering adopts known method.This partial content has detailed report in document (X.Zhang, W.Lin, and P.Xue, " Improvedestimation for just-noticeable visual distortion, " Signal Process, vol.85, pp.795-808,2005.).Through obtaining a residual image PRD after the HVS filtering based on the vision rate.

The average of each macro block and variance are calculated according to formula (1) and formula (2) among the residual image PRD in above-mentioned steps 02.To the size piece f that is MxN (m, average μ n) (M, N) and variance δ (M, N)

$\mathrm{\μ}(M,N)=\frac{1}{N\×M}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}f(m,n)---\left(1\right)$

$\mathrm{\δ}(M,N)=\frac{1}{N\×M}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{(\mathrm{\μ}-f(m,n))}^2---\left(2\right)$

In the formula: M, the value of N is 4,8,16, corresponding size range that MB may divide, as 16x16,16x8,8x16,8x8,8x4,4x8,4x4.In above-mentioned steps 03, be unit 030 with the concrete mode division of a MB according to the average in the above-mentioned formula and variance, unit 031, unit 032, a kind of in unit 033 and the unit 034 pairing 5 kinds of patterns.MB with a 16x16 among the present invention is divided into SKIP, 16x16, and 16x8, the basis of 8x16 and 8x8 pattern are formula (3)

Mode in formula (3) _16x16Represent the type that final determined MB divides.If when macro block satisfy simultaneously average be 0 and variance be that 0 constraints (also is μ (16,16)=0﹠amp; δ (16,16)=0) then the type of this MB is defined as pairing SKIP pattern in Unit 030.If the average of a MB be 0 but variance be less than or equal to a preset threshold JND _Th(also be μ (16,16)=0﹠amp; δ (16,16)≤JND _Th), determine that then the type of this MB is a pairing 16x16 pattern in Unit 031.If when macro block satisfy simultaneously average be not 0 and variance greater than JND _ThConstraints (also be μ (16,16) ≠ 0﹠amp; δ (16,16)＞JND _Th), need make careful type to this MB this moment and divide, the type of division comprises following three kinds of patterns: 032 two 16x8 in unit, 034 4 8x8 patterns of 033 two 8x16 in unit and unit.If δ ^*(16,8), δ ^*(8,16) and δ ^*(8,8) represent two 16x8 respectively, variance maximum in two 8x16 and four the 8x8 patterns.If δ ^*The maximum of (16,8) above-mentioned three kinds of careful divisions (also is δ ^*(16,8)=max{ δ ^*(16,8), δ ^*(8,16), δ ^*(8,8) }), then this MB is divided into pairing two 16x8 block modes in the unit 032.If δ ^*Maximum in (8,16) above-mentioned three kinds of careful divisions (also is δ ^*(8,16)=max{ δ ^*(16,8), δ ^*(8,16), δ ^*(8,8) }), then this MB is divided into pairing two 8x16 block modes in the unit 033.If δ ^*Maximum in (8,8) above-mentioned three kinds of careful divisions (also is δ ^*(8,8)=max{ δ ^*(16,8), δ ^*(8,16), δ ^*(8,8) }), then this MB is divided into pairing four 8x8 block modes in the unit 034.

In step 04, this each 8x8 piece is divided into following four patterns: a 8x8 block mode in the unit 041, two 8x4 block modes in the unit 042, two 4x8 block modes in the unit 043 and a kind of pattern in four 4x4 block modes in the unit 043.Among the present invention a 8x8 piece is divided into 8x8,8x4, the basis of 4x8 and 8x8 pattern are formula (4)

Mode in formula (4) _8x8Represent the type that final determined MB divides.If the average of a 8x8 piece be 0 but variance be less than or equal to a preset threshold JND _Th ⁸(also be

) determine that then the type of this 8x8 piece is 8x8.If when 8x8 piece satisfy simultaneously average be not 0 and variance greater than JND _Th ⁸Constraints (also be ), need make careful type to this 8x8 piece this moment and divide, the type of division comprises following several modes: two 4x8, two 8x4 and four 4x4 patterns.If δ ^*(4,8), δ ^*(8,4) and δ ^*(4,4) represent two 4x8 respectively, variance maximum in two 8x4 and four the 4x4 patterns.If δ ^*Maximum in (8,4) above-mentioned three kinds of careful divisions (also is δ ^*(8,4)=max{ δ ^*(4,8), δ ^*(8,4), δ ^*(4,4) }), then this 8x8 piece is divided into two 8x4 block modes.If δ ^*Maximum in (4,8) above-mentioned three kinds of careful divisions (also is δ ^*(4,8)=max{ δ (4,8), δ (8,4), δ ^*(4,4) }), then this 8x8 piece is divided into two 4x8 block modes.If δ ^*Maximum in (4,4) above-mentioned three kinds of careful divisions (also is δ ^*(4,4)=max{ δ ^*(4,8), δ ^*(8,4), δ ^*(4,4) }), then this 8x8 piece is divided into 4 4x4 block modes.

Through step 03 and 04 pattern is divided, after partition mode, carry out motion-estimation step 05, the sub-piece of each division is made estimation respectively and drawn its corresponding motion vector.After mode division that obtains a MB and corresponding motion vector, can obtain motion compensation residual information afterwards.The coding cost RDC that calculating is drawn under the inter-frame forecast mode under the above-mentioned division.

Wherein step 05,07,08 and H.264/AVC the processing mode in the standard is in full accord is not changed in the present invention.

JND in above-mentioned steps 03 and 04 _ThAnd JND _Th ⁸Effect and coding standard H.264/AVC in QP (quantization parameter value) identical.Set among the present invention ${\mathrm{JND}}_{\mathrm{th}}^8={\mathrm{JND}}_{\mathrm{th}}/4$ , by adjusted value JND _ThJust can reach and realize the controlled purpose of video quality.For with original H.264/AVC video encoding standard in the video quality method to set up consistent, provided JND among the present invention _ThAnd the corresponding mapping relations table of comparisons between the QP value.

Table 1.JND _ThAnd the corresponding mapping relations table of comparisons between the QP value

??QP	??40	??36	??32	??28	??24	??20	??16	??12
									??JND th	??2	??1.5	??1.2	??1.0	??0.7	??0.2	??0.08	??0.03

We are to football, foreman, and mobile, the performance comparison that five video test sequence of bus and soccer are drawn under the situation of same parameter configuration is as shown in table 2.Δ Y wherein _PSNRBe that method among the present invention is with respect to the PSNR gain of the method in H.264/AVC.And Speedup to be the inventive method save situation with respect to the time in H.264/AVC.

ΔY _PSNR＝HVS _PSNR-JM _PSNR

$\mathrm{Speedup}=\frac{{\mathrm{Time}}_{\mathrm{JM}}-{\mathrm{Time}}_{\mathrm{HVS}}}{{\mathrm{Time}}_{\mathrm{JM}}}\×100\%$

From the contrast experiment of table 2 as can be seen the method the present invention coding parameter be provided with identical in, the lifting that not only has coding quality can also promote speed of coding simultaneously effectively.

The method among table 2. the present invention and the performance comparison of encoder (JM16.0) H.264/AVC

??Sequence	??football	??foreman	??mobile	??bus	??soccer
						??ΔY PSNR(dB)	??0.691	??0.302	??0.363	??0.214	??0.167
??Speedup(％)	??71.54	??78.34	??88.15	??89.34	??80.63

Through the above description of the embodiments, those skilled in the art can be well understood to the present invention and can realize by the mode that software adds essential hardware platform, can certainly all implement by hardware.

Below only be the preferable embodiment of the present invention: as changing the HVS filter, the threshold value of using instead in the model selection is determined method, QP and JND _ThBetween mapping relations etc., but protection scope of the present invention is not limited thereto, and anyly is familiar with those skilled in the art in the technical scope that the present invention discloses, the variation that can expect easily or replacement all may fall within protection scope of the present invention.

Claims (5)

1. the video coding inter-frame predicting mode fast determination method based on the human visual system is characterized in that, comprises the steps:

At first execution in step 01, and reference frame image and current frame image are carried out obtaining filtered residual image PRD based on the relevant filtering of human visual system's characteristic; While execution in step 06: current frame image is carried out intra prediction mode select, to determine a kind of intra prediction mode and respective coding cost RDC thereof of the best;

Execution in step 02 then, and residual image PRD is divided into the macro block MB of 16x16 and calculates average and the variance of each macro block MB;

Next execution in step 03, each macro block MB carried out inter-frame forecast mode divide, and MB is divided into SKIP pattern, a 16x16 piece, two 16x8 pieces, two 8x16 pieces or four 8x8 pieces; When the mode division of described macro block MB is the SKIP pattern, then skip to step 08 automatically; When the mode division of described macro block MB is 16x16,16x8, perhaps a kind of in three kinds of patterns of 8x16, skips steps 04 automatically then, execution in step 05, step 07 are to 08; When the mode division of described macro block MB is the 8x8 pattern, then execution in step 04;

Step 04 pair 8x8 block mode is divided into again: a 8x8 piece, two 8x4 pieces, two 4x8 pieces or four 4x4 pieces;

Next execution in step 05, and determined partition mode in step 03 and 04 is carried out estimation and motion compensation, with the motion vector MV of definite final divided block, and the coding cost RDC of calculating macro block MB under above-mentioned mode division;

Execution in step 07 then, selects current macro MB to adopt in the frame or inter-frame forecast mode according to the coding cost RDC of determined inter-frame forecast mode of step 05 and the determined intra prediction mode of step 06;

Last execution in step 08 is carried out entropy coder and is formed final coded-bit output.

2. the video coding inter-frame predicting mode fast determination method based on the human visual system as claimed in claim 1, it is characterized in that, described average and the variance calculating to each macro block of step 02 is carried out according to following formula, to the size piece f (m that is MxN, n) average μ (M, N) and variance δ (M N) is:

In the formula: M, the value of N is 4,8 or 16.

3. the video coding inter-frame predicting mode fast determination method based on the human visual system as claimed in claim 1 is characterized in that, the described MB with a 16x16 of step 03 is divided into SKIP, 16x16,16x8, the basis of 8x16 and 8x8 pattern as shown in the formula:

Mode in the formula _16x16Represent the type that final determined MB divides, JND _ThRepresent the threshold value that corresponding 16x16 macro block MB sets.

4. the video coding inter-frame predicting mode fast determination method based on the human visual system as claimed in claim 1 is characterized in that, step 04 is described to be divided into 8x8 again with a 8x8 piece, 8x4, the basis of 4x8 and 8x8 pattern as shown in the formula:

In the formula: Mode _8x8Represent the type that final determined MB divides; JND _Th ⁸The threshold value that expression 8x8 macro block MB sets.

5. the video coding inter-frame predicting mode fast determination method based on the human visual system as claimed in claim 3 is characterized in that, the described JND of step 03 _ThBe corresponding mapping relations between the quantization parameter value QP in the coding standard H.264/AVC.

Images