CN101783951B - Method for rapidly confirming video coding inter-frame predicting mode based on human vision system - Google Patents

Abstract

The invention discloses a method for rapidly confirming a video coding inter-frame predicting mode based on a human vision system, which is characterized by comprising the following executing steps of: firstly, carrying out HVS filtration on an input reference frame image and a current frame image; then dividing a filtered image into macro-blocks and confirming a dividing way of the MB as one of five modes of SKIP, 16*16, 16*8, 8*16 and 8*8 according to an average value and a variance of each MB in an HVS filtered image; if the confirmed type is 8*8, calculating average values and variances of 8*8, 4*8, 8*4 and 4*4 and finally confirming a concrete corresponding mode of 8*8, 4*8, 8*4 and 4*4; and carrying out relevant motion estimation and motion compensation on the basis of confirmation of the inter-frame predicting mode.

Description

Video coding inter-frame predicting mode fast determination method based on the human visual system

Technical field

The present invention relates to inter-frame forecast mode in the video encoding standard and divide or confirm method, be specifically related to a kind ofly carry out H.264/AVC definite method of video coding inter-frame predicting mode according to human visual system (HVS) characteristic.

Background technology

Inter-frame forecast mode in the video coding is owing to relate to estimation; So the complexity of calculating is very high; Especially in video coding H.264/AVC, inter-frame forecast mode need be confirmed a kind of pattern of optimum from a large amount of candidates' inter-frame forecast mode before confirming.This coded system needs a large amount of computings,

Summary of the invention

The purpose of this invention is to provide a kind of method and carry out optimum H.264/AVC video coding inter-frame predicting mode selection according to HVS filtering; After confirming optimization model, carry out estimation; Such estimation and motion compensation process only carry out to selected pattern; And other non-selected pattern is not carried out estimation, and therefore can improve the execution efficient of encoder effectively, especially in the occasion of real-time coding, more can embody its superiority.

For reaching above purpose, the present invention takes following technical scheme to be achieved:

A kind of video coding inter-frame predicting mode fast determination method based on the human visual system is characterized in that, comprises the steps:

At first execution in step 01, and reference frame image and current frame image are carried out obtaining filtered residual image PRD based on the relevant filtering of human visual system's characteristic; Execution in step 06 is carried out intra prediction mode to current frame image and is selected simultaneously, to confirm a kind of intra prediction mode and respective coding cost RDC thereof of the best;

Execution in step 02 then, and residual image PRD is divided into the macro block MB of 16x16 and calculates average and the variance of each macro block MB;

Next execution in step 03, each macro block MB carried out inter-frame forecast mode divide, and MB is divided into SKIP pattern, a 16x16 block mode, two 16x8 block modes, two 8x16 block modes or four 8x8 block modes; When the mode division of said macro block MB is the SKIP pattern, then skip to step 08 automatically; When the mode division of said macro block MB is 16x16,16x8, perhaps a kind of in three kinds of patterns of 8x16, skips steps 04 automatically then, execution in step 05, step 07 are to 08; When the mode division of said macro block MB is the 8x8 pattern, then execution in step 04;

Step 04 pair 8x8 block mode is divided into again: a 8x8 piece, two 8x4 pieces, two 4x8 pieces or four 4x4 pieces;

Next execution in step 05, and determined partition mode in step 03 and the step 04 is carried out estimation and motion compensation, with the motion vector MV of definite final divided block, and the coding cost RDC of calculating macro block MB under above-mentioned mode division;

Execution in step 07 then, selects current macro MB to adopt in the frame or inter-frame forecast mode according to the coding cost RDC of the determined inter-frame forecast mode of step 05 and the coding cost RDC of the determined intra prediction mode of step 06;

H.264/AVC last execution in step 08 is carried out entropy coder and is formed final coded-bit output.

In the said method, said average and the variance to each macro block of step 02 calculated and carried out according to following formula, to the size piece f that is MxN (m, average μ n) (M, N) with variance δ (M N) is:

$\mathrm{\μ}(M,N)=\frac{1}{N\×M}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}f(m,n)---\left(1\right)$

$\mathrm{\δ}(M,N)=\frac{1}{N\×M}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{\left(\mathrm{\μ}-f(m,n)\right)}^2---\left(2\right)$

In the formula: M, the value of N is 4,8,16, (m, n) the horizontal ordinate of expression position.

The said MB with a 16x16 of step 03 is divided into the SKIP pattern, the 16x16 pattern, the 16x8 pattern, the basis of 8x16 pattern and 8x8 pattern as shown in the formula:

Mode in the formula _16x16Represent the type that final determined MB divides, JND _ThRepresent the threshold value that corresponding 16x16 macro block MB sets; δ ^*(16,8), δ ^*(8,16) and δ ^*(8,8) are represented two 16x8 respectively, variance maximum in two 8x16 and four the 8x8 patterns.

Step 04 is said to be divided into the 8x8 pattern again with a 8x8 piece, the 8x4 pattern, the basis of 4x8 pattern and 8x8 pattern as shown in the formula:

In the formula: Mode _8x8Represent the type that final determined MB divides,

Represent the threshold value that corresponding 8x8 macro block MB sets, δ ^*(4,8), δ ^*(8,4) and δ ^*(4,4) are represented two 4x8 respectively, variance maximum in two 8x4 and four the 4x4 patterns.

Compared with prior art, H.264/AVC method of the present invention can embed in the codec, therefore can not destroy code flow structure H.264/AVC, has good versatility.Can be with H.264/AVC codec be compatible mutually at present.In addition, QP and JND have been provided among the present invention _ThBetween mapping relations, like this can be under the situation that does not change code allocation, realize coding, therefore have ease for use.

Description of drawings

Below in conjunction with accompanying drawing and embodiment the present invention is done further detailed description.

Fig. 1 is the steps flow chart block diagram of the video coding inter-frame predicting mode fast determination method based on HVS of the present invention.

Embodiment

As shown in Figure 1, the present invention is based on HVS and carry out video coding H.264/AVC and confirm that fast the method for inter-frame forecast mode comprises following execution in step:

At first execution in step 01, and reference frame and current encoded frame (P frame or B frame) are carried out obtaining filtered residual image (being called for short PRD) based on the relevant filtering of human visual system's characteristic (being called for short HVS filtering); Execution in step 06 is carried out intra prediction mode to current frame image and is selected simultaneously, to confirm a kind of intra prediction mode and respective coding cost RDC thereof of the best;

Execution in step 02 then, and the filtered residual image PRD of HVS is divided into the macro block (be called for short MB) of 16x16 and calculates average and the variance of each MB;

Next execution in step 03, each MB carried out inter-frame forecast mode select, and MB is divided into: unit 030 pairing SKIP (promptly motion vector and the residual error of current MB not being encoded) pattern, or unit 031 a pairing 16x16 piece; Or unit 032 pairing two 16x8 pieces; Or unit 033 pairing two 8x16 pieces; Or a kind of in unit 034 pairing four 8x8 pieces.

If the mode division of a MB is the pairing SKIP pattern in Unit 030 in step 03, then skips to step 08 automatically and carry out this pattern of entropy coding and carry out code stream output; If the pattern that MB divides is 16x16,16x8, perhaps a kind of in three kinds of patterns of 8x16, skips steps 04 automatically then, execution in step 05 is to step 08; If the pattern that MB divides is a pairing 8x8 pattern in the unit 034, then need pattern execution in step 04 to these 4 8x8 pieces to confirm its more meticulous pattern once more.Four 8x8 pieces being divided in step 04 pair Unit 034 are divided into again: unit 041 a pairing 8x8 piece; Or unit 042 pairing two 8x4 pieces; Or unit 043 pairing two 4x8 pieces; Or a kind of in unit 044 pairing four 4x4 pieces;

Next execution in step 05; Current MB is carried out estimation and motion compensation according to determined partition mode in the step 03 and 04; With the motion vector MV of definite final divided block, and the code check and the distortion factor optimum relations function R DC of calculating coding of current MB under above-mentioned mode division;

Execution in step 07 then, selects with inter-frame forecast mode in the frame, selects current MB to adopt in the frame or inter-frame forecast mode according to the RDC of determined intra prediction mode in determined inter-frame forecast mode in the step 05 and 06 step.

Last execution in step 08 is carried out entropy coder and is formed final coded-bit output.

In above-mentioned steps 01, the method for prediction reference frame and current reference frame being done HVS filtering adopts known method.This partial content has detailed report in document (X.Zhang, W.Lin, and P.Xue, " Improvedestimation for just-noticeable visual distortion, " Signal Process, vol.85, pp.795-808,2005.).Through obtaining a residual image PRD after the HVS filtering based on the vision rate.

The average of each macro block and variance are calculated according to formula (1) and formula (2) among the residual image PRD in above-mentioned steps 02.To the size piece f that is MxN (m, average μ n) (M, N) with variance δ (M, N)

$\mathrm{\μ}(M,N)=\frac{1}{N\×M}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}f(m,n)---\left(1\right)$

$\mathrm{\δ}(M,N)=\frac{1}{N\×M}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{\left(\mathrm{\μ}-f(m,n)\right)}^2---\left(2\right)$

In the formula: M, the value of N is 4,8,16, corresponding size range that MB possibly divide, like 16x16,16x8,8x16,8x8,8x4,4x8,4x4.In above-mentioned steps 03, be unit 030 with the concrete mode division of a MB according to the average in the above-mentioned formula and variance, unit 031, unit 032, a kind of in unit 033 and the unit 034 pairing 5 kinds of patterns.MB with a 16x16 among the present invention is divided into SKIP, 16x16, and 16x8, the basis of 8x16 and 8x8 pattern are formula (3)

Mode in formula (3) _16x16Represent the type that final determined MB divides.If when macro block satisfy simultaneously average be 0 with variance be 0 constraints (also being μ (16,16)=0& δ (16,16)=0) then the type of this MB confirm as pairing SKIP pattern in Unit 030.If the average of a MB be 0 but variance be less than or equal to a preset threshold JND _Th(also be μ (16,16)=0& δ (16,16)≤JND _Th), confirm that then the type of this MB is a pairing 16x16 pattern in Unit 031.If when macro block satisfy simultaneously average be not 0 with variance greater than JND _ThConstraints (also be μ (16,16) ≠ 0& δ (16,16)＞JND _Th), need make careful type to this MB this moment and divide, the type of division comprises following three kinds of patterns: 032 two 16x8 in unit, 034 4 8x8 patterns of 033 two 8x16 in unit and unit.If δ ^*(16,8), δ ^*(8,16) and δ ^*(8,8) are represented two 16x8 respectively, variance maximum in two 8x16 and four the 8x8 patterns.If δ ^*The maximum of (16,8) above-mentioned three kinds of careful divisions (also is δ ^*(16,8)=max{ δ ^*(16,8), δ ^*(8,16), δ ^*(8,8) }), then this MB is divided into pairing two 16x8 block modes in the unit 032.If δ ^*Maximum in (8,16) above-mentioned three kinds of careful divisions (also is δ ^*(8,16)=max{ δ ^*(16,8), δ ^*(8,16), δ ^*(8,8) }), then this MB is divided into pairing two 8x16 block modes in the unit 033.If δ ^*Maximum in (8,8) above-mentioned three kinds of careful divisions (also is δ ^*(8,8)=max{ δ ^*(16,8), δ ^*(8,16), δ ^*(8,8) }), then this MB is divided into pairing four 8x8 block modes in the unit 034.

In step 04, this each 8x8 piece is divided into following four patterns: a 8x8 block mode in the unit 041, two 8x4 block modes in the unit 042, two 4x8 block modes in the unit 043 and a kind of pattern in four 4x4 block modes in the unit 043.Among the present invention a 8x8 piece is divided into 8x8,8x4, the basis of 4x8 and 8x8 pattern are formula (4)

Mode in formula (4) _8x8Represent the type that final determined MB divides.If a 8x8 block variance with mean 0 but less than or equal to a set threshold

(ie

) is determined that the 8x8 block of type 8x8.If when 8x8 piece satisfy simultaneously average be not 0 with the constraints (also be ) of variance greater than

; Need make careful type to this 8x8 piece this moment divides; The type of dividing comprises following several modes: two 4x8, two 8x4 and four 4x4 patterns.If δ ^*(4,8), δ ^*(8,4) and δ ^*(4,4) are represented two 4x8 respectively, variance maximum in two 8x4 and four the 4x4 patterns.If δ ^*Maximum in (8,4) above-mentioned three kinds of careful divisions (also is δ ^*(8,4)=max{ δ ^*(4,8), δ ^*(8,4), δ ^*(4,4) }), then this 8x8 piece is divided into two 8x4 block modes.If δ ^*Maximum in (4,8) above-mentioned three kinds of careful divisions (also is δ ^*(4,8)=max{ δ (4,8), δ (8,4), δ ^*(4,4) }), then this 8x8 piece is divided into two 4x8 block modes.If δ ^*Maximum in (4,4) above-mentioned three kinds of careful divisions (also is δ ^*(4,4)=max{ δ ^*(4,8), δ ^*(8,4), δ ^*(4,4) }), then this 8x8 piece is divided into 4 4x4 block modes.

Pattern is divided with 04 through step 03, after partition mode, carry out motion-estimation step 05, the sub-piece of each division is made estimation respectively and drawn its corresponding motion vector.After mode division that obtains a MB and corresponding motion vector, can obtain motion compensation residual information afterwards.The coding cost RDC that calculating is drawn under the inter-frame forecast mode under the above-mentioned division.

Wherein step 05,07,08 and H.264/AVC the processing mode in the standard is in full accord is not changed in the present invention.

JND in above-mentioned steps 03 and 04 _ThWith

Effect and coding standard H.264/AVC in QP (quantization parameter value) identical.Set among the present invention

Through adjusted value JND _ThJust can reach and realize the controlled purpose of video quality.For with original H.264/AVC video encoding standard in the video quality method to set up consistent, provided JND among the present invention _ThAnd the corresponding mapping relations table of comparisons between the QP value.

Table 1.JND _ThAnd the corresponding mapping relations table of comparisons between the QP value

QP	40	36	32	28	24	20	16	12
									JND th	2	1.5	1.2	1.0	0.7	0.2	0.08	0.03

We are to football, foreman, and mobile, the performance comparison that five video test sequence of bus and soccer are drawn under the situation of same parameter configuration is as shown in table 2.Δ Y wherein _PSNRBe that method among the present invention is with respect to the PSNR gain of the method in H.264/AVC.And Speedup to be the inventive method save situation with respect to the time in H.264/AVC.

ΔY _PSNR＝HVS _PSNR-JM _PSNR

$\mathrm{Speedup}=\frac{{\mathrm{Time}}_{\mathrm{JM}}-{\mathrm{Time}}_{\mathrm{HVS}}}{{\mathrm{Time}}_{\mathrm{JM}}}\×100\%$

From the contrast experiment of table 2 can find out the present invention method coding parameter be provided with identical in, the lifting that not only has coding quality can also promote speed of coding simultaneously effectively.

The method among table 2. the present invention and the performance comparison of encoder (JM16.0) H.264/AVC

Sequence	football	foreman	mobile	bus	soccer
						ΔY PSNR(dB)	0.691	0.302	0.363	0.214	0.167
Speedup(％)	71.54	78.34	88.15	89.34	80.63

Through the description of above execution mode, those skilled in the art can be well understood to the present invention and can realize by the mode that software adds essential hardware platform, can certainly all implement through hardware.

More than be merely the preferable embodiment of the present invention: as changing the HVS filter, the threshold value of using instead in the model selection is confirmed method, QP and JND _ThBetween mapping relations etc., but protection scope of the present invention is not limited thereto, any technical staff who is familiar with the present technique field is in the technical scope that the present invention discloses, the variation that can expect easily or replacement all possibly fall within protection scope of the present invention.

Claims (5)

1. the video coding inter-frame predicting mode fast determination method based on the human visual system is characterized in that, comprises the steps:

At first execution in step 01, and reference frame image and current frame image are carried out obtaining filtered residual image PRD based on the relevant filtering of human visual system's characteristic; Execution in step 06 is carried out intra prediction mode to current frame image and is selected simultaneously, to confirm a kind of intra prediction mode and respective coding cost RDC thereof of the best;

Execution in step 02 then, and residual image PRD is divided into the macro block MB of 16x16 and calculates average and the variance of each macro block MB;

Next execution in step 03, each macro block MB carried out inter-frame forecast mode divide, and MB is divided into SKIP pattern, a 16x16 block mode, two 16x8 block modes, two 8x16 block modes or four 8x8 block modes; When the mode division of said macro block MB is the SKIP pattern, then skip to step 08 automatically; When the mode division of said macro block MB is 16x16,16x8, perhaps a kind of in three kinds of patterns of 8x16, skips steps 04 automatically then, execution in step 05, step 07 are to 08; When the mode division of said macro block MB is the 8x8 pattern, then execution in step 04;

Step 04 pair 8x8 block mode is divided into again: a 8x8 piece, two 8x4 pieces, two 4x8 pieces or four 4x4 pieces;

Next execution in step 05, and determined partition mode in step 03 and the step 04 is carried out estimation and motion compensation, with the motion vector MV of definite final divided block, and the coding cost RDC of calculating macro block MB under above-mentioned mode division;

Execution in step 07 then, selects current macro MB to adopt in the frame or inter-frame forecast mode according to the coding cost RDC of the determined inter-frame forecast mode of step 05 and the coding cost RDC of the determined intra prediction mode of step 06;

H.264/AVC last execution in step 08 is carried out entropy coder and is formed final coded-bit output.

2. the video coding inter-frame predicting mode fast determination method based on the human visual system as claimed in claim 1; It is characterized in that; Said average and the variance to each macro block of step 02 calculated and carried out according to following formula, to a size be MxN piece f (m, average μ (M n); N) and variance δ (M N) is:

In the formula: M, the value of N is 4,8,16, (m, n) the horizontal ordinate of expression position.

3. the video coding inter-frame predicting mode fast determination method based on the human visual system as claimed in claim 1; It is characterized in that the said MB with a 16x16 of step 03 is divided into the SKIP pattern, the 16x16 pattern; The 16x8 pattern, the basis of 8x16 pattern and 8x8 pattern as shown in the formula:

Mode in the formula _16x16Represent the type that final determined MB divides, JND _ThRepresent the threshold value that corresponding 16x16 macro block MB sets; δ ^*(16,8), δ ^*(8,16) and δ ^*(8,8) are represented two 16x8 respectively, variance maximum in two 8x16 and four the 8x8 patterns.

4. the video coding inter-frame predicting mode fast determination method based on the human visual system as claimed in claim 1; It is characterized in that; Step 04 is said to be divided into the 8x8 pattern again with a 8x8 piece, the 8x4 pattern, the basis of 4x8 pattern and 4x4 pattern as shown in the formula:

In the formula: Mode _8x8Represent the type that final determined MB divides; Represent the threshold value that corresponding 8x8 macro block MB sets, δ ^*(4,8), δ ^*(8,4) and δ ^*(4,4) are represented two 4x8 respectively, variance maximum in two 8x4 and four the 4x4 patterns.

5. the video coding inter-frame predicting mode fast determination method based on the human visual system as claimed in claim 3 is characterized in that, the described JND of step 03 _ThBe corresponding mapping relations between the quantization parameter value QP in the coding standard H.264/AVC.