CN104219526B - HEVC rate distortion optimization algorithm based on just-noticeable perception quality judging criterion - Google Patents

Abstract

The invention relates to an HEVC rate distortion optimization algorithm based on the just-noticeable perception quality judging criterion. The algorithm is characterized by including analyzing the motion models and static texture features of each macroblock of each frame, acquiring the perception quality type of the current macroblock, and acquiring the image salient region; calculating a just-noticeable distortion threshold on the basis of a visual salient region; calculating the perception quality on the basis of a just-noticeable distortion model; performing rate distortion optimization on the basis of the perception quality on the basis of the just-noticeable distortion model. The algorithm is reasonable in design, the HEVC rate distortion optimization is performed by the just-noticeable perception quality judging criterion, defects that MSE (mean-square errors) are uses for measuring the video distortion estimation standard can be overcome, and the final encoding effect can meet the visual subjective perception quality better; meanwhile, more noises can be tolerated on the premise that the subjective quality is maintained, unnecessary perception redundancy is removed, the compressing efficiency is improved, and the code rate of encoded files is decreased.

Description

Based on the HEVC rate-distortion optimization algorithms that can just examine perceived quality decision rule

Technical field

It is especially a kind of based on the HEVC that can just examine perceived quality decision rule the invention belongs to technical field of video coding Rate-distortion optimization algorithm.

Background technology

In recent years, constantly the carrying to video quality demands with the fast-developing of national economy, the progress of technology and people Height, high definition/ultra high-definition video coding technique is used as industry such as future home movie theatre, digital broadcast television, Internet video, high-definition movies The basic core technology of business turns into industry focus of attention.For high definition/ultra high-definition video communication, existing video encoding standard In compression ratio also a certain distance is compared with the application demand of reality.Therefore, International Organization for standardization ISO/IEC (MPEG) and ITU-T starts the planning of generation digital video compression standard --- efficient Video coding (High Efficiency Video Coding, HEVC), target is that compression efficiency is enhanced about more than once on the basis of H.264/AVC top grade.

Efficient Video coding (HEVC) is in a total of two links of loop filtering process：Block-eliminating effect filtering and self adaptation Sampling point compensates SAO.Wherein, self adaptation sampling point compensation SAO can be further classified as banding compensation (Band Offset, BO) and side Edge compensates (Edge offset, EO) two major classes.Edge compensation algorithm (EO) is carried out mainly for the profile of each object in image Compensation is, it is necessary to selection one kind is carried out from four class adjacent encoder blocks of level, vertical, left-leaning unity slope and Right deviation unity slope The comparing of the value of current pixel point and the value of two neighboring pixel.Banding backoff algorithm (BO) be mainly used in in image each Color and lines information inside object are compensated, and the division of its compensation type is based entirely on pixel amplitude in itself, That is image pixel intensities are divided into 32 grades by HEVC from 0 to maximum, by the selection of rate-distortion optimization, wherein 4 companies The pixel compensation of continuous grade will finally write code stream.

HEVC encoders according to picture material, using the method for rate-distortion optimization, frame in and interframe it is numerous can modeling Optimal coding mode is chosen in formula.Although to a certain extent, the judgement of rate distortion model can make the cataloged procedure become complicated, It is that, just because of the application of rate-distortion optimization technology, encoder can as far as possible obtain optimum prediction information, so as to ensure that Picture quality, improves the overall performance of encoder.

The rate distortion framework of conventional video coding is including being carried out using mean square error MSE as distortion value including HEVC Calculate.Although in most cases MSE can reflect the real quality of image, it is that MSE reflects not still to have certain situation Out, for example salt-pepper noise can produce huge interference to picture, and the MSE of whole two field picture may not be very big.

The content of the invention

It is an object of the invention to overcome the deficiencies in the prior art, there is provided a kind of reasonable in design, subjective visual quality it is high and Can remove it is more perception redundancies based on the HEVC rate-distortion optimization algorithms that can just examine perceived quality decision rule.

The present invention solves existing technical problem and takes following technical scheme to realize：

It is a kind of based on the HEVC rate-distortion optimization algorithms that can just examine perceived quality decision rule, comprise the following steps：

Step 1, before the coding side of efficient video codec carries out mode adjudging, analyze each macro block in each frame Motor pattern and static textural characteristics, obtain the perceived quality type of current macro, and obtain figure according to different motion states As marking area；

Step 2, according to specific image region calculate the proper of view-based access control model salient region examine distortion threshold；

Step 3, calculated based on can just examine distortion model according to the proper distortion threshold of examining of view-based access control model salient region Perceived quality；

Step 4, basis carry out rate-distortion optimization based on the perceived quality that can just examine distortion model；

The step 1 perceived quality type is obtained using following Mathematical Modeling：

In formula,It is the perceived quality type of current macro, Normal Pattern are normal perceived quality type, Aliased Pattern are distortion-aware quality type, and Hysteresis Pattern are delayed perception quality type, Background is static perceived quality type,WithRespectively normal condition, distortion status, inactive state go out Existing probability,ForWithVector form,It is status attribute descriptor, SS represents inactive state.

And, described salient region of image is the macro block and Hysteresis Pattern of Normal Pattern types The macro block of type combines the region of composition.

And, the proper distortion threshold of examining of the step 2 view-based access control model salient region is calculated according to the following equation：

In above-mentioned formula, FJND examines distortion threshold, T for the proper of view-based access control model salient region_Basic(k, n, i, j), F_Lum、F_Contrast、F_TemporalAnd F_FoveaBe respectively basic threshold value, intensity modifier value, contrast correction value, time-domain correction value and Marking area correction value.

And, the step 3 is based on just examining the perceived quality of distortion model by two kinds of distortions of PSNR-HA and PSNR-HMA Criterion is weighted and obtained, and its computational methods is as follows：

(1) for the reference block A and distortion block B that give, the difference of the two is calculated WithIt is respectively The average value of reference block A and distortion block B coefficients；

(2) correction matrix C=B+Delt is obtained；

(3) correction factor is calculated

(4) revised macro block D=C × ρ is calculated；

(5) calculating can just examine the revised distortion value MSE of distortion model_HVS, computational methods are as follows：

Wherein coeff_o(i, j) and coeff_d(i, j) represents the pixel value of reference block and reconstructed block correspondence position, jnd respectively (i, j) then represents that the proper of correspondence position of second calculating examines distortion threshold；

(6) if M₁＞ M₂Then

Finally give

MSE_HVS=M₁+Delt²×coef3

In above formula, coef1, coef2, coef3 represent the modifying factor to perceptual error, are taken respectively according to experiment experience 0.002nd, 0.25 and 0.04；

(7) PSNR-HA is obtained according to the above-mentioned perceived quality distortion being calculated：

For coloured image, M is Y, and mono- luminance component of Cr, Cb, two perceptual distortions of color components weight what is obtained Average distortion, such as following formula are calculated：

M=(M_Y+M_Cb×coef4+M_Cr×coef4)/(1+2×coef4)

M_Y、M_Cb、M_CrIt is respectively a luminance component Y, two perceptual distortions of color component Cb, Cr, weight coefficient coef4 It is 0.5；

The amendment of PSNR-HMA is to calculate MSE_HVSWhen do corresponding amendment with can just examine distortion model.

And, the method that the step 4 carries out rate-distortion optimization is：Perceived quality will can just be examined based on salient region of image Molten bonding is closed on R- λ models, is obtained such as drag：

J is cost function in above formula, then λ=dQ/dR, for each macro block, calculation cost function：

j₁=q₁(QP₁)-λ·r₁(QP₁)for coding block #1

j₂=q₂(QP₂)-λ·r₂(QP₂)for coding block #2

……

j_N=q_N(QP_N)-λ·r_N(QP_N)for coding block #N

In above formula, QP is quantization step, and the coding mode for obtaining making max { J } minimum is compared by search.

Advantages and positive effects of the present invention are：

The present invention is reasonable in design, and it to use and carry out HEVC rate-distortion optimizations, energy based on just can examining perceived quality decision rule Enough overcome the shortcomings of mean square error MSE as measurement video distortion evaluation criterion so that final encoding efficiency more conforms to people The subjective perceptual quality of eye, meanwhile, the video after coding can tolerate more noises on the premise of subjective quality is not reduced, The unnecessary perception redundancy of removal, so as to improve compression efficiency, reduces the code check of file after coding.

Brief description of the drawings

Fig. 1 is general frame figure of the invention；

Fig. 2 is the video interception that embodiment is given；

Fig. 3 is the notable figure obtained after processing Fig. 2.

Specific embodiment

The embodiment of the present invention is further described below in conjunction with accompanying drawing.

It is a kind of based on the HEVC rate-distortion optimization algorithms that can just examine perceived quality decision rule, as shown in figure 1, including following Step：

Step 1, before the coding side of efficient video codec carries out mode adjudging, analyze each macro block in each frame Motor pattern and static textural characteristics, draw the perceived quality type of current macro, and obtain figure according to different motion states As marking area.

In this step, for each coding unit CU of each frame, and each of which divided block, initially set up a set of Complete status attribute descriptor(representing the state of k-th macro block in a frame of t), it is as follows：

Wherein

In formula, NS, AS, SS represent normal condition, distortion status, inactive state respectively,Represent motion vector,It is Represent current block residual energy and texture energy than ratio,K-th macro block in a frame of t is represented, q is corresponding Quantization parameter N × N is the size of macro block.

With vectorForm represent status attribute descriptorIt is as follows：

WhereinThree values correspond to three kinds of states situation about not occurring respectively, i.e., 0 represents do not occur, and 1 represents occur.

At a time,Respectively represent NS, AS and SS occur probability weight, three use to Amount formRepresent；WithThe probability that state occurs in three is represented, vector form is expressed asIt is as follows：

Wherein

η is a constant more than 1, represents iterative rate；

Each macro block probability vector is drawn according to above-mentioned parameter, the perceived quality type according to belonging to following criterions judge itIt is as follows：

Background in formula represents that picture still is motionless, is background parts；

Think that only belonging to two kinds of situations of Normal Pattern and Hysteresis Pattern both attracted in this method The notice of people, can differentiate the details of clear picture again.Belong to region that the macro block of the both of which combines i.e. significantly Property region.Fig. 2 gives a video interception, and Fig. 3 is the notable figure obtained by this step.

Step 2, according to specific image region calculate the proper of view-based access control model salient region examine distortion threshold.

This method can attract the notice of beholder for moving region in video content, and the visual sensitivity of human eye As vision central fovea is outwards gradually reduced, so saliency region can just be examined into distortion model with tradition mutually tying Close, further excavate video-aware redundancy, compression efficiency is improved on the premise of viewing quality is ensured.

The proper distortion threshold FJND that examines of view-based access control model salient region is calculated according to the following equation：

In above-mentioned formula, T_Basic(k, n, i, j), F_Lum、F_Contrast、F_TemporalAnd F_FoveaIt is respectively basic threshold value, brightness The threshold value of correction value, contrast correction value, time-domain correction value and marking area correction value, wherein former three for still image That is modifying factor, behind two modifying factors for video.The calculating of each factor is respectively：

(1) basic threshold value T_Basic(k, n, i, j)：

In formula, (n, i, j) represents n-th piece (i, j) individual position, and s is the parameter for illustrating the accumulative effect in space, empirically Value takes 0.25；A, b, c, r are that constant is respectively equal to 1.33,0.11,0.18 and 0.6.φ_iAnd φ_jBe dct transform normalization because Sonω_ijIt is frequency

(2) intensity modifier value F_LumFor：

In formula,Represent the mean flow rate of the macro block.

(3) contrast correction value F_ContrastFor：

There are the different types of macro block F of tri- kinds of texture for plane, edge_ContrastCalculated according to above-mentioned formula.It is first First, all of marginal point in image is found out using candy operators；Then, calculating edge dot density function ρ_edge=∑ edge/N²； Then the value of ψ is calculated according to following two formula：

So can be obtained by the contrast correction factor.

(4) time-domain correction value F_TemporalFor：

F in above formula_tIt is temporal frequency, f_sIt is spatial frequency.

(5) marking area correction value F_FoveaFor：

κ (bg (k, i, j)) is background luminance function in above formula, and the background luminance function is as follows：

f_m(v, e)=min (f_c(e), f_d(v))

f_cIt is to be determined by contrast sensitivity function, maximum 1.0 is set to herein；f_dIt is display cut-off frequency, is set to The half of monitor resolution.Parameter e represents the distance of the saliency regional center that the position obtains to Part I.

Step 3, calculated based on can just examine distortion model according to the proper distortion threshold of examining of view-based access control model salient region Perceived quality.

The present invention evaluates BMMF come instead of in original coding framework using based on the perceived quality that can just examine distortion model The calculating of distortion, perceived quality evaluation BMMF is calculated as follows and obtains：

Wherein x is the index for adjudicating matrix, that is, belong to which macro block；Y represents the type belonging to the macro block, and Class1 is represented Flat, type 2 represents edge, and type 3 represents texture,WithReference macroblock is represented respectively and rebuilds macro block；Q_xyRepresent that x is grand Value of the block under y type evaluation criterions；

Q in above-mentioned formula is based on the perceived quality (perceived quality decision rule) that can just examine distortion model, the perception Quality is obtained by two kinds of distortion criterion weightings of PSNR-HA and PSNR-HMA；Then represent that distortion evaluation of estimate must be weighted flat , w_xyIt is weight.For PSNR-HA and PSNR-HMA, this method it is existing can just examine distortion model on the basis of done as follows Amendment, specific modification method includes procedure below：

(1) for the reference block A and distortion block B that give, the difference of the two is calculated WithIt is respectively The average value of reference block A and distortion block B coefficients；

(2) correction matrix C=B+Delt is obtained；

(3) correction factor is calculated

(4) revised macro block D=C × ρ is calculated；

(5) calculating can just examine the revised distortion value MSE of distortion model_HVS, it is as follows：

Wherein coeff_o(i, j) and coeff_d(i, j) represents the pixel value of reference block and reconstructed block correspondence position respectively.jnd (i, j) then represents that the proper of correspondence position of second calculating examines distortion threshold, is that human eye can be examined more than the distortion of this thresholding, And the distortion for being less than this thresholding is imperceptible human eye, perception redundancy so can be at utmost excavated；

(6) if M₁＞ M₂Then

Finally give

MSE_HVS=M₁+Delt²×coef3

In above-mentioned two formula, coef1, coef2, coef3 represent the modifying factor to perceptual error, according to experiment experience 0.002,0.25 and 0.04 is taken respectively；

(7) PSNR-HA can be obtained according to the above-mentioned perceived quality distortion being calculated, it is as follows：

For coloured image, M is Y, and mono- luminance component of Cr, Cb, two perceptual distortions of color components weight what is obtained Average distortion, such as following formula are calculated：

M=(M_Y+M_Cb×coef4+M_Cr×coef4)/(1+2×coef4)

M_Y、M_Cb、M_CrIt is respectively a luminance component Y, two perceptual distortions of color component Cb, Cr, weight coefficient coef4 It is 0.5.

The amendment of PSNR-HMA is similar with PSNR-HA, is to calculate MSE_HVSWhen accordingly repaiied with can just examine distortion model and do Just, do not repeating herein.

Step 4, basis carry out rate-distortion optimization based on the perceived quality that can just examine distortion model, find optimal coding mould Formula.

Traditional rate-distortion optimization model is to be based on R-D models, and R- λ models are employed in HEVC, and it is than R-D model It is more accurate.To obtain can just examining perceived quality based on salient region of image in step 3 on the basis of R- λ models below and melt Enter wherein, obtain more conforming to the model of subjective quality, it is as follows：

J is cost function in above formula, then λ=dQ/dR.For each macro block, calculation cost function：

j₁=q₁(QP₁)-λ·r₁(QP₁)for coding block #1

j₂=q₂(QP₂)-λ·r₂(QP₂)forcoding block #2

……

j_N=q_N(QP_N)-λ·r_N(QP_N)for coding block #N

In above formula, QP is quantization step, and the target of optimization is to compare the volume for obtaining making max { J } minimum by seemingly searching for Pattern.

A test is done below as the method for the present invention, experiment effect of the invention is illustrated.

Test environment：Visual Studio 2010；

Cycle tests：Select the video test sequence of three kinds of sizes as follows from HEVC officials cycle tests：

832x480：BQMall, Basketball-Drill

1280x720：Johnny, FourPeople

1920x1080：Basketball-Drive, BQterrace

Test result is as follows：

The Y-PSNR of table one (dB)

The code check of table two (kbps)

The subjective testing standard of table three

The subjective test results of table four

Experiment conclusion

This experimental result chooses six HEVC standard cycle tests.According to table one it can be seen that and HEVC reference softwares HM10.0 compared to Y-PSNR (PSNR) under same sequence identical quantization parameter (QP) value will low 1.13dB~4.12dB, this says Bright this technology can tolerate more noises on the premise of Subjective video quality holding is constant.Identical QP values can be obtained according to table two Under, smaller bit rate can be obtained using the present invention compared to original HEVC, improve compression ratio.

In subjective test, original HM10.0 compression videos are placed on left side, compression video of the invention is put in right side, please 30 observers are tested according to the standard of table three, and the result of table four is obtained after statistical average.It can be seen that pressure of the invention Contracting video is generally better than HEVC primitive technologies in supervisor's quality, especially under conditions of larger QP values；And in small QP values Due to the distortion very little after primitive technology compression, so the two subjective difference can be ignored substantially.

By objective and subjective experiment, test result shows compared with original HEVC coding techniques, the present invention can gram Mean square error MSE is taken as the deficiency for weighing video distortion evaluation criterion, tolerates more on the premise of subjective quality is not reduced Noise, remove unnecessary perception redundancy, so as to improve compression efficiency, reduce the code check of file after coding.

It is emphasized that embodiment of the present invention is illustrative, rather than limited, therefore present invention bag The embodiment for being not limited to described in specific embodiment is included, it is every by those skilled in the art's technology according to the present invention scheme The other embodiment for drawing, also belongs to the scope of protection of the invention.

Claims (5)

1. it is a kind of based on the HEVC rate-distortion optimization algorithms that can just examine perceived quality decision rule, it is characterised in that including following step Suddenly：

Step 1, before the coding side of efficient video codec carries out mode adjudging, analyze the motion of each macro block in each frame Pattern and static textural characteristics, obtain the perceived quality type of current macro, and it is aobvious to obtain image according to different motion states Write region；

Step 2, according to specific image region calculate the proper of view-based access control model salient region examine distortion threshold；

Step 3, calculated based on can just examine the perception of distortion model according to the proper distortion threshold of examining of view-based access control model salient region Quality；

Step 4, basis carry out rate-distortion optimization based on the perceived quality that can just examine distortion model；

The step 1 perceived quality type is obtained using following Mathematical Modeling：

${\mathrm{mp}}_k^t=\left\{\begin{array}{cc}NormalPattern& if\max \left\{P_k^t\right\}=p_{0,k}^t,s_k^t\≠SS\\ AliasedPattern& if\max \left\{P_k^t\right\}=p_{1,k}^t,s_k^t\≠SS\\ HysteresisPattern& if\max \left\{P_k^t\right\}\≠p_{2,k}^t,s_k^t=SS\\ Background& if\max \left\{P_k^t\right\}=p_{2,k}^t,s_k^t=SS\end{array}\right.$

In formula,It is the perceived quality type of current macro, Normal Pattern are normal perceived quality type, Aliased Pattern is distortion-aware quality type, and Hysteresis Pattern are delayed perception quality type, and Background is quiet Only perceived quality type,WithThe probability that respectively normal condition, distortion status, inactive state occur,ForWithVector form,It is status attribute descriptor, SS represents inactive state.

2. according to claim 1 based on the HEVC rate-distortion optimization algorithms that can just examine perceived quality decision rule, it is special Levy and be：Described salient region of image is the macro block and Hysteresis Pattern types of Normal Pattern types Macro block combines the region of composition.

3. according to claim 1 based on the HEVC rate-distortion optimization algorithms that can just examine perceived quality decision rule, it is special Levy and be：The proper distortion threshold of examining of the step 2 view-based access control model salient region is calculated according to the following equation：

In above-mentioned formula, FJND examines distortion threshold, T for the proper of view-based access control model salient region_Basic(k,n,i,j)、F_Lum、 F_Contrast、F_TemporalAnd F_FoveaBe respectively basic threshold value, intensity modifier value, contrast correction value, time-domain correction value and significantly Region correction value.

4. according to claim 1 based on the HEVC rate-distortion optimization algorithms that can just examine perceived quality decision rule, it is special Levy and be：The perceived quality that the step 3 is based on just examining distortion model is added by two kinds of distortion criterions of PSNR-HA and PSNR-HMA Weigh and obtain, its computational methods is as follows：

(1) for the reference block A and distortion block B that give, the difference of the two is calculated WithIt is respectively reference block The average value of A and distortion block B coefficients；

(2) correction matrix C=B+Delt is obtained；

(3) correction factor is calculated

(4) revised macro block D=C × ρ is calculated；

(5) calculating can just examine the revised distortion value MSE of distortion model_HVS, computational methods are as follows：

Wherein coeff_o(i, j) and coeff_d(i, j) represents the pixel value of reference block and reconstructed block correspondence position respectively, jnd (i, J) then represent that the proper of correspondence position of second calculating examines distortion threshold；

(6) if M₁＞ M₂Then

Finally give

MSE_HVS=M₁+Delt²×coef3

In above formula, coef1, coef2, coef3 represent the modifying factor to perceptual error, take 0.002 respectively according to experiment experience, 0.25 and 0.04；

(7) PSNR-HA is obtained according to the above-mentioned perceived quality distortion being calculated：

$PSNR-HA=10{\log }_{10}\left(\frac{{255}^2}{M}\right)$

For coloured image, M is Y, mono- luminance component of Cr, Cb, and it is average that the perceptual distortion weighting of two color components is obtained Distortion, such as following formula are calculated：

M=(M_Y+M_Cb×coef4+M_Cr×coef4)/(1+2×coef4)

M_Y、M_Cb、M_CrIt is respectively a luminance component Y, two perceptual distortions of color component Cb, Cr, weight coefficient coef4 is 0.5；

The amendment of PSNR-HMA is to calculate MSE_HVSWhen do corresponding amendment with can just examine distortion model.

5. according to claim 1 based on the HEVC rate-distortion optimization algorithms that can just examine perceived quality decision rule, it is special Levy and be：The method that the step 4 carries out rate-distortion optimization is：The conjunction of perceived quality molten bonding will can be just examined based on salient region of image On R- λ models, obtain such as drag：

$\frac{dJ}{dR}=\frac{dQ}{dR}-\mathrm{\λ}=0$

J is cost function in above formula, then λ=dQ/dR, for each macro block, calculation cost function：

$\begin{array}{c}{j}_1=q_1\left({\mathrm{QP}}_1\right)-\mathrm{\λ}\·r_1\left({\mathrm{QP}}_1\right)forcodingblock\#1\\ j_2=q_2\left({\mathrm{QP}}_2\right)-\mathrm{\λ}\·r_2\left({\mathrm{QP}}_2\right)forcodingblock\#2\\ \mathrm{......}\\ j_N=q_N\left({\mathrm{QP}}_N\right)-\mathrm{\λ}\·r_N\left({\mathrm{QP}}_N\right)forcodingblock\#N\\ \max \left\{J\right\}=\max \left({\Σ}_{n=1}^N{q}_n\right({\mathrm{QP}}_n)-\mathrm{\λ}\·{\Σ}_{n=1}^N{r}_i({\mathrm{QP}}_n\left)\right)\end{array}$

In above formula, QP is quantization step, and the coding mode for obtaining making max { J } minimum is compared by search.