CN103237223B

CN103237223B - LCU based on entropy divides fast

Info

Publication number: CN103237223B
Application number: CN201310169748.XA
Authority: CN
Inventors: 张萌萌; 曲建峰; 白慧慧
Original assignee: North China University of Technology
Current assignee: North China University of Technology
Priority date: 2013-05-10
Filing date: 2013-05-10
Publication date: 2016-01-13
Anticipated expiration: 2033-05-10
Also published as: CN103237223A

Abstract

A kind of maximum coding unit (LCU) division methods based on efficient video coding (HEVC), comprising: perform second quantization, to reduce Pixel-level to LCU; Calculate the entropy of the CU likely marked off in described LCU; According to the entropy of calculated CU, division rule and threshold value determine whether described CU should be divided by continuation.

Description

LCU based on entropy divides fast

Joint study

The application is by North China University of Tech and the joint study of information institute of Beijing Jiaotong University, and obtain following fund assistance: state natural sciences fund (No.61103113, No.60903066), Beijing institution of higher education talent directly under the jurisdiction of a municipal government teaches by force in-depth planning item (PHR201008187); Jiangsu Province's Natural Science Fund In The Light (BK2011455), Beijing's Natural Science Fund In The Light (No.4102049), Ministry of Education beginning teacher fund (No.20090009120006); Country 973 plan (2012CB316400), central colleges and universities' basic research fund (No.2011JBM214).

Technical field

The present invention relates to image processing field, more specifically, relate to and based on entropy, Fast Segmentation is carried out to LCU in efficient video coding HEVC.

Background technology

In April, 2010, the international video encoding standard of Liang great organizes VCEG and MPEG to set up video compression associating group JCT-VC (JointcollaborativeTeamonVideoCoding), together develop efficient video coding HEVC (Highefficiencyvideocoding) standard, it is also referred to as H.265.HEVC standard main target is the raising H.264/AVC realizing significantly code efficiency with previous generation standard, especially for high-resolution video sequence.Its target is that code check reduces to 50% of H.264 standard under same video quality (PSNR).

With regard to the current stage, HEVC still continues to use the hybrid encoding frame H.264 just starting to adopt.Interframe and intraframe predictive coding: the correlation eliminating time-domain and spatial domain.Transition coding: transition coding is carried out to eliminate spatial coherence to residual error.Entropy code: eliminate statistical redundancy.HEVC in hybrid encoding frame, will put forth effort the new coding tools of research or technology, improve video compression efficiency.

At present, the new features of many codings proposed in the discussion of JCT-VC tissue, likely can add in HEVC standard, and the concrete document of each discussion can be from http:// wftp3.itu.intobtain.

In HEVC, provide larger coding unit (CU) and quad-tree structure more flexibly.The size of the CU in HEVC has 64x64 (LCU), 32x32,16x16, and8x8.The CU of this 64x64 is wherein defined by maximum coding unit (LCU), each LCU can recurrence split into 4 equal-sized CU, until minimum CU (8x8).In order to find optimized CU splitting scheme, encoder must consider all dividing condition.Fig. 2 illustrates a LCU as an example and how to be divided into CU.The process of this traversal is called rate-distortion optimization RDO (RateDistortionOptimization), and RDO process can scan the CU combination that all possible CU goes to select to bring minimum rate distortion costs.This larger CU is adopted to be a good scheme for improving the validity of coding section.Fig. 3 A and 3B respectively illustrates a pictures by the result H.264 divided with HEVC, and from Fig. 3 B, we can find out that larger CU makes encoder can have the coding of emphasis: the region that amount of information is very little can be divided into relatively large CU; The region contained much information can be divided into relatively little CU.But larger CU can bring a negative effect, because the coding side of HEVC needs to travel through all possible CU, the amount of calculation of this ergodic process is too large, and computation burden large is like this inappropriate for much needing the application of real-time coding.

The application with reference to each document following, and these documents are incorporated herein thus, using the part as this paper disclosure.

[1]Woo-JinHan.ImprovedVideoCompressionEfficiencyThroughFlexibleUnitRepresentationandCorrespondingExtensionofCodingTools.CircuitsandSystemsforVideoTechnology.Dec.2010，20，1709-1720.

[2]Il-KooKim.HM7：HighEfficiencyVideoCoding(HEVC)TestModel7EncoderDescription.JCT-VC9thMeeting，Geneva，CN，27April-7May，2012.

[3]BenjaminBross.Highefficiencyvideocoding(HEVC)textspecificationdraft7，JCT-VC5thMeeting，Geneva，March，2011

[4]Bossen，F.VideoCodingUsingaSimplifiedBlockStructureandAdvancedCodingTechniques，CircuitsandSystemsforVideoTechnology.Dec.2010，20，1667-1675.

[5]Cassa，M.B.FastratedistortionoptimizationfortheemergingHEVCstandard.PictureCodingSymposium(PCS)，2012.7-9May2012，493-496.

[6]Woo-JinHan.ImprovedVideoCompressionEfficiencyThroughFlexibleUnitRepresentationandCorrespondingExtensionofCodingTools.CircuitsandSystemsforVideoTechnology.Dec.2010.20，12，1709-1720.

[7]G.J.Sullivan，T.Wiegand，“Rate-distortionoptimizationforvideocompression”，IEEESignalProcessingMagazine，vol.15，6，pp.74-90，Nov.1998.

[8]Ihara，Shunsuke.Informationtheoryforcontinuoussystems.1993WorldScientific.p.2.

[9]FrankBossen，2011，Commontestconditionsandsoftwarereferenceconfigurations，JCT-VC-F900，JCT-VC6thMeeting，Torino，July，2011，14-22.

Summary of the invention

When studying HEVC, applicant notices and is limited to based on RDO for all researchs of LCU segmentation at present, and do not consider that other technologies whether can be adopted to carry out alternative RDO algorithm realizes preferably LCU segmentation and lower amount of calculation, thus reach the compromise between LCU segmentation quality and calculated load.

In research process of the present invention, applicant notices that the amount of information of CU and these CU marked off by RDO process itself has much relations.Fig. 2 illustrates the correlation of segmentation LCU and amount of information as an example.Because Shannon entropy defines the average unpredictability of stochastic variable, this is equivalent to its amount of information.

So applicant proposes to replace splitting the RDO process in LCU based on the technology of Shannon entropy.

According to an aspect, provide a kind of maximum coding unit (LCU) division methods based on efficient video coding (HEVC), comprising:

Second quantization is performed, to reduce Pixel-level to LCU;

Calculate the entropy of the CU likely marked off in described LCU;

According to the entropy of calculated CU, division rule and threshold value determine whether described CU should be divided by continuation, wherein:

If the entropy of one of them CU is very little, then this CU does not need to divide again, or

If the entropy of this CU is very large, then this CU needs to continue to be divided, or

If the entropy of this CU is close to the mean value of the entropy of the described CU likely marked off, then this CU can not divide again.

According on the other hand, provide a kind of computer program, comprise the computer-readable medium with computer instruction stored thereon, described computer instruction makes processor perform following operation:

Second quantization is performed, to reduce Pixel-level to LCU;

Calculate the entropy of the CU likely marked off in described LCU;

According to another aspect, provide a kind of device, comprising:

For performing second quantization, to reduce the module of Pixel-level to LCU;

For calculating the module of the entropy of the CU likely marked off in described LCU;

For the entropy according to calculated CU, division rule and threshold value determine whether described CU should be continued the module of division, wherein:

According to another aspect, provide a kind of encoder based on HEVC, it comprises:

CU dispenser, it is configured to:

Second quantization is performed, to reduce Pixel-level to LCU;

Calculate the entropy of the CU likely marked off in described LCU;

Accompanying drawing explanation

Fig. 1 shows an embodiment of the encoder block diagram of HEVC.

Fig. 2 shows in HEVC the schematic diagram utilizing quaternary tree progressively to split LCU.

Fig. 3 A and 3B respectively illustrates a pictures by the result H.264 divided with HEVC.

Fig. 4 shows the high level flow chart embodying principle of the present invention.

Fig. 5 A shows according to one embodiment of present invention, for the method flow diagram split LCU based on entropy.

Fig. 5 B shows according to one embodiment of present invention, for the device block diagram split LCU based on entropy.

Fig. 6 compares with the result figure that the LCU based on entropy of the present invention is split based on the LCU segmentation of RDO algorithm.

Embodiment

With reference now to accompanying drawing, various scheme is described.In the following description, in order to make an explanation, set forth multiple detail to provide the thorough understanding to one or more scheme.But, obviously, also these schemes can be realized when there is no these details.

As used in this specification, term " assembly ", " module ", " system " etc. are intended to refer to the entity relevant to computer, such as but not limited to, the combination of hardware, firmware, hardware and software, software, or executory software.Such as, assembly can be but be not limited to: the process run on a processor, processor, object, executable (executable), execution thread, program and/or computer.For example, the application program run on the computing device can be assembly with this computing equipment.One or more assembly can be positioned at executive process and/or execution thread, and assembly can be positioned on a computer and/or be distributed on two or more platform computers.In addition, these assemblies can perform from the various computer-readable mediums with various data structures stored thereon.Assembly can communicate by means of local and/or remote process, such as according to the signal with one or more packet, such as, come from by means of another component interaction in signal and local system, distributed system and/or with the data by means of signal and the mutual assembly of other system on the network of such as internet and so on.

Fig. 1 shows the general configuration figure of the video encoder that efficient video coding (HEVC) realizes.The encoder architecture of HEVC is roughly the same with H.264 used encoder architecture, mainly further research, improvement are carried out for the algorithm used in modules, especially for high-resolution video sequence, its target improved is that code check reduces to 50% of H.264 standard under same video quality (PSNR).

Because the encoder architecture of HEVC is roughly the same with H.264 used encoder architecture, therefore do not obscure the present invention, the overall architecture in Fig. 1 is not described in the application, and the LCU only paid close attention to based on HEVC standard is split.

As described above, when studying HEVC, applicant notices and is limited to based on RDO for all researchs of LCU segmentation at present, and do not consider that other technologies whether can be adopted to carry out alternative RDO algorithm realizes preferably LCU segmentation and lower amount of calculation, thus reach the compromise between LCU segmentation quality and calculated load.

Therefore, the technology that applicant proposed based on Shannon entropy replaces splitting the RDO process in LCU.Key of the present invention is: find the relation between CU and the entropy of these CU be selected.Based on this point, target of the present invention is that the CU that makes to be marked off by the present invention and optimized CU have maximum similarity.

First, the formula of entropy is as follows:

H (x) = - Σ_{i = 0}^{i = 1} p_{1} \log_{2} p_{1}

①

In this formula, H (x) is entropy, p _ithe probability that represent pixel level i occurs, j is pixel progression.

In order to obtain the amount of information of all CU, we will calculate the entropy of all possible CU in LCU.

But before the computation, preferably carry out the process of a second quantization, the sum of Pixel-level is become little as far as possible.As shown from the above formula, when Pixel-level sum j diminishes, amount of calculation can be reduced.

Usually, the computer digit for each pixel value represents normally 1-2 byte, such as 8 digital videos or 10 digital videos.Therefore, it is contemplated that adopt SIMD instruction to carry out right shift (such as, N position) to the pixel value of the multiple pixel of expression implements this second quantization simultaneously.A hypothesis of this second quantization is: when carrying out second quantization, the minimizing of the amount of information in image LCU and the minimizing of total pixel technique roughly proportional, or its monotonic function.Therefore, even lost a large amount of information after carrying out second quantization to LCU, but the loss of this information can not cause appreciable impact to the determination of the optimum segmentation of LCU.Therefore, mainly can split the figure place of every pixel in the compromise of quality and calculated load and original video based on LCU to the determination of the total Pixel-level quantity after second quantization.Such as, when every pixel 256 Pixel-level, can be every pixel 32 Pixel-level by its second quantization, i.e. above N=3.

In another implementation, no matter in original video, every pixel has how many positions, is all every pixel 32 Pixel-level by its second quantization.

In addition, another object of this second quantization removes background noise, namely serves as one and remove background noise filter.

Next step, will calculate the entropy of all CU that may occur in a LCU.In LCU, one has 85 CU that may be divided out, comprising: the CU of 1 64x64,4 32x32,16 16x16,32 8x8 sizes.Next, the probability that often kind of pixel value occurs in a CU is added up.These probability can be used in the calculating of entropy.The formula of calculating probability is as follows:

p_{i} = \frac{n_{i}}{N} (i = 0,1, . . . . . . 31)

②

Wherein, N is the number of pixel in this CU, n _irepresent that pixel value is the number of the pixel of i.

After drawing probability, we will use formula 3. to calculate the pixel value of each CU.Draw the foundation that 85 entropy will divide as CU.

H (x) = - Σ_{i = 0}^{i = 1} p_{i} \log_{2} p_{i}, (i = 31)

③

Based on the entropy obtained in the theory of Part II and part A, inventor do a lot of work to find CU entropy and CU divide between contact.

By these research, inventor has found the criterion of these segmentations LCU:

The entropy of a.CU is very little, and so this CU probably can not continue division and goes down.

The entropy of b.CU is very large, and so this CU probably will continue to be divided down.

The entropy of c.CU is close to the mean value of all 85 entropy, and so this CU appears in optimum division result possibly.

On the basis of these rules, inventor has attempted a lot of threshold values, has found optimum threshold values:

If a. the entropy of CU is less than 1.2, so this CU does not need to divide again.

If b. the entropy of CU is greater than 3.5, so this CU will continue division and goes down.

If c. CU entropy average entropy ± scope of 0.15 within, so this CU can not divide again.

Through experiment, compared to HM8.0, the algorithm that applicant proposes has the reduction of 40.9% on the scramble time, and BD-rate have lost 3.82%, PSNR and reduces 0.117%.

In this experiment, inventor to use formula 4. to carry out the minimizing of calculation code time how many.

ΔT = \frac{T_{HBVC} - T_{Proposed}}{T_{HBVC}}

④

Wherein, T _hBVCthe scramble time of HM8.0, T _proposedrepresent and this algorithm is joined the scramble time after HM8.0.Δ T represents the result reducing the scramble time.

Its experimental result is as shown in table 1:

Table 1. experimental result

Fig. 6 compares with the result figure that the LCU based on entropy of the present invention is split based on the LCU segmentation of RDO algorithm.As we know from the figure, algorithm in this paper and the optimized division based on RDO algorithm have the matching degree of about 80%.

In step 501, second quantization is performed, to reduce Pixel-level to LCU.

In step 503, the entropy of the CU likely marked off in described LCU is calculated.Such as, as described above, when 32 Pixel-level, the entropy of 85 CU is altogether calculated.

In step 505, according to the entropy of calculated CU, division rule and threshold value determine whether described CU should be divided by continuation.If the entropy of one of them CU is very little, then this CU does not need to divide again, if or the entropy of this CU very large, then this CU needs to continue to be divided, if or the entropy of this CU is close to the mean value of the entropy of the described CU likely marked off, then this CU can not divide again.

In a preferred embodiment, if the entropy of a CU is less than 1.2, then think that the entropy of this CU is very little and do not need to divide again, if the entropy of a CU is greater than 3.5, then think that the entropy of this CU is very large and need to continue to be divided, if the entropy of a CU the entropy of the described CU likely marked off mean value ± scope of 0.15 within, then this CU entropy close to the entropy of the described CU likely marked off mean value and can not divide again.

Fig. 5 B shows according to one embodiment of present invention, for the device block diagram split LCU based on entropy.The function that its modules realizes is consistent with the method step in Fig. 5 A, does not therefore repeat them here.

The above embodiment of the present invention all can be embodied as the encoder based on HEVC, particularly intra-prediction encoder wherein.Should can as shown in Figure 1 based on internal structure of the encoder of HEVC, and infra-frame prediction choice box wherein and infra-frame prediction frame constitute described intra-prediction encoder.It should be appreciated by those skilled in the art that this decoder can be implemented as software, hardware and/or firmware.

When implemented in hardware, video encoder by general processor, digital signal processor (DSP), application-specific integrated circuit (ASIC) (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components or can be designed to its combination in any performing function described herein, realizes or performs.General processor can be microprocessor, but alternatively, this processor also can be the processor of any routine, controller, microcontroller or state machine.Processor also can be implemented as the combination of computing equipment, such as, and the combination of the combination of DSP and microprocessor, the combination of multi-microprocessor, one or more microprocessor and DSP kernel or other this kind of structure any.In addition, at least one processor can comprise and can operate to perform above-mentioned one or more step and/or one or more modules of operation.

When realizing video encoder with hardware circuits such as ASIC, FPGA, it can comprise the various circuit blocks being configured to perform various function.Those skilled in the art can carry out these circuit of Design and implementation in every way according to the various constraintss applied over the whole system, realize various function disclosed in this invention.

Although aforementioned open file discusses exemplary arrangement and/or embodiment, it should be noted that when not deviating from the scope of the scheme of the description defined by claims and/or embodiment, many changes and amendment can be made at this.And, although describe in the singular or the described scheme that requires and/or the key element of embodiment, also it is contemplated that the situation of plural number, be limited to odd number unless expressly stated.In addition, any scheme and/or all or part of of embodiment can with other scheme and/or all or part of of embodiment be combined, unless indicated different arbitrarily.

Claims

1., based on a maximum coding unit LCU division methods for efficient video coding (HEVC), comprising:

Second quantization is performed, to reduce Pixel-level to LCU;

Calculate the entropy of the coding unit CU likely marked off in described LCU;

If the entropy of this CU is close to the mean value of the entropy of the described CU likely marked off, then this CU can not divide again,

Wherein, if the entropy of this CU is less than 1.2, then think that the entropy of this CU is very little, if the entropy of this CU is greater than 3.5, then think that the entropy of this CU is very large, if the entropy of this CU the entropy of the described CU likely marked off mean value ± scope of 0.15 within, then the entropy of this CU is close to the mean value of the entropy of the described CU likely marked off.

2. the method for claim 1, wherein described is by representing what right shift N position realized to the computer digit of each pixel value of expression to LCU execution second quantization.

3. method as claimed in claim 2, wherein, N is 3.

4. the Pixel-level after the method for claim 1, wherein reducing is every pixel 32 Pixel-level.

5. the maximum coding unit LCU based on efficient video coding (HEVC) divides a device, comprising:

For performing second quantization, to reduce the module of Pixel-level to LCU;

For calculating the module of the entropy of the coding unit CU likely marked off in described LCU;

6., based on an encoder of efficient video coding (HEVC), it comprises:

Coding unit CU dispenser, it is configured to:

Second quantization is performed, to reduce Pixel-level to maximum coding unit LCU;

Calculate the entropy of the CU likely marked off in described LCU;