CN110022477B

CN110022477B - Method for quickly selecting inter-frame prediction mode based on CUTree

Info

Publication number: CN110022477B
Application number: CN201910248674.6A
Authority: CN
Inventors: 张昊; 向广
Original assignee: Central South University
Current assignee: Central South University
Priority date: 2019-03-29
Filing date: 2019-03-29
Publication date: 2021-02-19
Anticipated expiration: 2039-03-29
Also published as: CN110022477A

Abstract

The invention provides a CUTree-based inter-frame prediction mode rapid selection method, and belongs to the technical field of video coding and decoding. The method aims at the value of the weighting processing of the CUTree on the SATD, normalizes the value and uses the normalized value as the condition of mode selection of the CU in interframe prediction, reduces unnecessary coding time, improves coding efficiency, is simple and easy to implement, and is beneficial to popularization in other coding standards.

Description

Method for quickly selecting inter-frame prediction mode based on CUTree

Technical Field

The invention belongs to the technical field of video coding and decoding, and particularly relates to a method for quickly selecting an inter-frame prediction mode based on a CUTree.

Background

Video coding means to convert a file of a video signal into another file format by some compression means, so that the bandwidth usage is reduced during the signal transmission process, and the video signal is efficiently transmitted. High Efficiency Video Coding (HEVC) is a new Video compression standard developed by the joint collaboration on Video (JCT-VC) group established in 2012 by MPEG and VCEG. HEVC is superior to h.264 in performance, and the compression rate of HEVC can reach 2 times that of h.264 under the same video quality. After videos such as movies and animation films are compressed by HEVC (high efficiency video coding), not only is the flow consumption greatly reduced when a mobile phone user watches online videos, but also the downloading speed is faster, the image quality is basically not influenced, the online watching is smoother, and the mobile phone user is not easy to get stuck.

HEVC employs Coding Tree Units (CTUs), one CTU consisting of a luma CTB and 2 chroma CTBs and additional syntax elements. The CTU is divided into four Coding Units (CU) of smaller size according to a quadtree-like structure, and can be divided into 4 layers at most. I.e. from 64 x 64 up to 8 x 8. A CU also consists of one luma coding block CB and two chroma coding blocks CB and additional syntax elements. The CU may further be further decomposed into Prediction Units (PUs) and Transform Units (TUs), which are basic units for performing Prediction operations, so that the coding is more convenient, concise and clear. The intra prediction module of HEVC uses a multi-angle prediction mode with a divided block as a basic unit, and aims to reduce the influence caused by spatial correlation in an image. The intra mode selection direction of HEVC is extended by 9 modes of h.264 to 35. Meanwhile, in the inter-frame mode selection of HEVC, besides referring to variable CU size, high-precision motion estimation is also adopted, and even for motion parameters, three coding modes are designed: inter prediction mode for direct coding, Skip Inter prediction after improvement, and Merge Inter prediction mode.

In HEVC, a video signal sequence is encoded in a Group of pictures (GOP) unit, and each frame is divided into a series of slices (independent units for encoding). One slice is divided into a plurality of CTUs, the CTUs are divided into CUs, the CUs are basic units shared by links such as HEVC intra/inter prediction, quantization transformation and entropy coding, the supportable coding size is 64 x 64 at most and 8 x 8 at most, and the encoder can reasonably select the size of the CUs according to different picture contents, picture sizes and application requirements, so that the optimization of a larger degree is obtained. In addition, a CU takes a CTU as a root node, and a CU can also serve as a root node of a PU tree and a TU tree.

The PU defines all prediction modes of the coding unit, all information related to prediction is defined in the prediction unit part, and the function of the PU is to acquire relevant information of a prediction process. For example, the direction of intra prediction, the partition of inter prediction, motion vector prediction, and the index of inter prediction reference picture all belong to the category of the prediction unit. For intra prediction of an image, the size of a prediction unit PU is the same as that of the associated coding unit CU, and the prediction unit PU can be divided into N × N prediction units PU only when the coding unit CU is at the lowest layer, i.e., 0 layer. Therefore, PU sizes range from 4 × 4 to 64 × 64. For inter prediction of an image, there are 8, 4 symmetric modes (2N × 2N, 2N × N, N × 2N, N × N) and 4 asymmetric modes (2N × nU, 2N × nD, nL × 2N, nR × 2N) of a prediction unit, where 2N × nU and 2N × nD are divided at a ratio of upper and lower 1:3, 3:1, respectively, and nL × 2N and nR × 2N are divided at a ratio of left and right 1:3, 3:1, respectively. The SKIP mode is one of the MERGE modes, and the MERGE mode only needs to encode the motion parameter set index, and the SKIP mode does not need to encode the residual information in addition to the motion parameter set index. The prediction residual obtained from the prediction unit PU is transformed appropriately to convert the distributed distribution of the image energy in the spatial domain into the centralized distribution in the transform domain, so as to achieve the purpose of removing spatial redundancy.

TU is a basic unit that independently performs transform and quantization, and its size is flexible. The size depends on the CU mode, allowing TUs to be recursively partitioned in a quadtree across multiple PUs within one CU. For a 2N × 2N CU, a flag bit determines whether it is divided into 4N × N TUs. The TU may adaptively select an optimal mode according to a local variation characteristic of the prediction residual. The large-block TU mode can better concentrate energy, and the small-block TU mode can preserve more image detail. The flexible partitioning structure can fully compress the transformed residual energy to further improve the coding gain.

For the HEVC configured normally, during encoding, all PU partition modes, i.e., 8 partition modes, of a current CU need to be traversed, and one with the minimum Rate Distortion cost is found through Rate Distortion Optimization (RDO), so as to determine whether the current CU needs to be partitioned into PUs and how to partition. This means that RDO needs to evaluate all CUs of the CTU, i.e. PU and TU of each CU, and in addition, when performing inter prediction, each prediction unit partition mode needs to perform motion estimation, and finally, in order to achieve the best coding performance, the workload is huge and it takes a long time to obtain the best partition mode.

Disclosure of Invention

In order to solve the problems that in the prior art, in order to achieve the best coding performance, the workload of HEVC is huge, and the time for obtaining the best partition mode is long, one of the embodiments of the present invention aims to provide a method for quickly selecting an inter-frame prediction mode based on a CUTree.

In order to achieve the above purpose, one of the embodiments of the present invention adopts the following technical solutions:

a method for quickly selecting an inter-frame prediction mode based on a CUTree comprises the following steps:

(1) acquiring a flag bit flag _ satdcast value of a current CU depth layer d;

(2) judging whether the CUs are all in the image: if yes, performing the step (3); if not, directly entering the division of the next depth layer d + 1; namely, the current CU is divided into 4 sub-CUs from a lower quadtree;

(3) judging whether to select the Merge mode or the Skip mode: if yes, calculating the RDcost of the Merge mode and the Skip mode in the inter-frame prediction, and selecting the mode with smaller RDcost from the Merge mode and the Skip mode as a temporary optimal mode; if not, skipping the selection of the Merge mode and the Skip mode by the current depth, and directly performing the step (4);

(4) judging whether to divide the depth of the next CU: if so, performing quadtree division on the CU at the next depth d + 1; if not, directly performing the step (5);

(5) judging whether to judge eight inter-frame modes: if yes, carrying out mode selection of eight inter-frame modes; if not, directly dividing the next depth;

(6) and obtaining the optimal prediction mode of the current CU.

The terms are explained in detail first, as shown in fig. 1, where the CU depth is a CU quadtree partition diagram, the numbers indicated by subscripts are the depths of current CUs, and the depths corresponding to CU blocks with sizes of 64 × 64, 32 × 32, 16 × 16, and 8 × 8 are 0, 1, 2, and 3, respectively, for example, CU0 is partitioned to obtain 4 CUs 1, where CU0 is referred to as a parent CU of CU1, and CU1 is a child CU of CU 0.

Preferably, in step (1), if the CU depth layer is a 0 layer, when the CU depth layer is a P frame, and the SATD value is <15000, the flag _ satdcost is assigned to 0; 15000< SATD value <20000, the flag _ satdcost is assigned as 1; when the SATD value is more than 20000, the flag _ satdcost is assigned to 2; when the frame is an I frame and the SATD value is less than 15000, the flag _ satdcost is assigned to 0; 15000< SATD value <30000, flag _ satdcost is assigned to 1; when the SATD value is more than 30000, the flag _ satdcost is assigned to be 2; when the frame is a B frame and the SATD value is less than 10000, the flag _ satdcost is assigned to be 0; when 10000< SATD value <20000, the flag _ satdcost is assigned as 1; when the SATD value is >20000, the flag _ satdcost is assigned to 2.

Preferably, if the CU depth layer is 1 layer, when the CU depth layer is a P frame, the flag _ satdcost is assigned to 0 when the SATD value is < 4000; when the SATD value is more than 4000 and less than 10000, the flag _ satdcost is assigned to be 1; when the SATD value is more than 10000, the flag _ satdcost is assigned to be 2; when the frame is an I frame and the SATD value is less than 4000, the flag _ satdcost is assigned to be 0; when the SATD value is 4000< 8000, the flag _ satdcost is assigned to be 1; when the SATD value is more than 8000, the flag _ satdcost is assigned to be 2; when the frame is a B frame and the SATD value is less than 2000, the flag _ satdcost is assigned to be 0; when the SATD value is 2000< 5000, the flag _ satdcost is assigned to be 1; when the SATD value is >5000, the flag _ satdcost is assigned to 2.

Preferably, in step (1), if the CU depth layer is 2 layers, when the CU depth layer is a P frame and the SATD value is <1000, the flag _ satdcost is assigned to 0; when the SATD value is 1000< 3000, the flag _ satdcost is assigned to 1; when the SATD value is more than 3000, the flag _ satdcost is assigned to be 2; when the frame is an I frame and the SATD value is less than 1000, the flag _ satdcost is assigned to be 0; when the SATD value is 1000< 3000, the flag _ satdcost is assigned to 1; when the SATD value is more than 3000, the flag _ satdcost is assigned to be 2; when the frame is a B frame and the SATD value is less than 1000, the flag _ satdcost is assigned to be 0; when the SATD value is 1000< 3500, the flag _ satdcost is assigned to 1; with SATD value >3500, flag _ satdcost is assigned a value of 2.

Preferably, in step (1), if the CU depth layer is 3 layers, the flag _ satdcost is assigned to 3.

Preferably, the division into the next depth layer d +1 in step (2) refers to a division from the current CU to the next quadtree into 4 sub-CUs.

Preferably, the determination condition of step (3) is (flag _ satdcost ═ 2| | flag _ satdcost ═ 1) & & mightNotSplit.

In the step (3), the judgment is performed according to the flag bits flag _ satdcost and mightNotSplit. Wherein: mightNotSplit is a flag bit that the CU is not divided any more downward, and flag _ satdcost is a normalized SATD value weighted by the curree.

Preferably, the judgment condition of the step (4) is mightSplit & (flag _ satdcost | (0)).

In the step (4), the flag bits are mightSplit and flag _ satdcost. Wherein: mightSplit is a flag bit that the CU can continue to be divided, and flag _ satdcost is a normalized SATD value weighted by the CUTree.

Preferably, the judgment condition of the step (5) is (mightNotSplit & & flag _ santdcost ═ 2).

In the step (5), the flag bits are mightSplit and flag _ satdcost. Wherein: mightSplit is a flag bit that the CU can continue to be divided, and flag _ satdcost is a normalized SATD value weighted by the CUTree.

In the method for quickly selecting the inter-frame prediction mode based on the curree, based on the influence of the curree on the SATD in the aspect of rate control, on the basis of the influence, normalization is performed on the Sum of Absolute Transformed (SATD) value after the curree weighting processing, and then the normalized Sum of Absolute Transformed (SATD) value is taken as one of the judgment conditions for mode selection: if the SATD value is larger, the CU is indicated to have larger influence on the quality of the subsequent frame, and a mode selection method with higher complexity is selected; if the SATD value is smaller, the influence of the CU on the quality of the subsequent frames is smaller, a mode selection method with lower complexity is selected, and under the condition that the subjective quality reduction of the video is negligible, the calculation complexity of an encoder is reduced, the encoding time is reduced, and the encoding efficiency is improved.

The curree in the encoder x265 takes into account the MBTree in x264, and when it calculates qp in rate control, records, for each CU, the number of times that the CU is referred to in the process of predicting a certain number of frames (one GOP) forward to calculate the qp value of the CU, and through its corresponding relationship table, takes a smaller value for the CU with a large number of reference times and takes a larger value for the CU with a small number of reference times. In the embodiment of the invention, the application place of the CUTree is expanded, and the value obtained by weighting the SATD value by using the CUTree is applied to mode selection in inter-frame prediction.

The cuttree tracks the CU of the current propagation frame through the adjacent frame, if the current CU is referred to more points, bits are more distributed to the current CU, and if the current CU is referred to less points, bits are less distributed to the current CU, and the coding efficiency is improved through the method.

In HEVC, the SATD value is calculated by performing motion estimation on a motion vector to obtain a residual MV square, performing Hadamard transform on the residual MV square, and finally adding all coefficients of the transformed square to obtain the SATD value. In the encoder x265, the intra SATD value for each frame is calculated by down-sampling each frame 1/2 to calculate the intra SATD value for each 8 × 8 block size.

A Hadamard (Hadamard) matrix is an orthogonal square matrix composed of basic elements. The Hadamard transform is a typical non-sinusoidal transform, and is based on a Hadamard matrix, and for data of an image, the data can be concentrated to the upper left corner of the matrix after the Hadamard transform, that is, the data are concentrated to the upper left corner of the matrix after the Hadamard transform. Thus, the image compression device plays a good role in image information compression.

In HEVC, the encoder uses a Hadamard transform of order 4 and a Hadamard transform of order 8 to compute the SATD. When the SATD is calculated, if the square matrix of the residual MV obtained after the motion vector passes through motion estimation is X and the Hadamard matrix is H, the SATD is calculated by the following formula:

when it calculates a 4X 4 block X₄]For the SATD value of (a), two-dimensional Hadamard transformation is performed: w₄＝[H₄]x[X₄]x[H₄]Then to W it₄All coefficients are taken as absolute values and then summed.

For 8X 8 block X₈]The SATD value of (1) is calculated by first comparing [ X [ ]₈]Performing a Hadamard transform: w₈＝[H₈]x[X₈]x[H₈]Then calculate the W₈All coefficients are taken as absolute values and then summed.

In view of the MBTree in x264, when the CU in encoder x265 calculates qp in rate control, for each CU, the number of times that the CU is referred to in the process of predicting a certain number of frames (one GOP) ahead is recorded, and the qp value of the CU is calculated through its corresponding relationship table. The invention expands the application place of the CUTree, and applies the value obtained by weighting the SATD value by the CUTree to the mode selection in the interframe prediction.

The effect of the CUTree on SATD is weighted by its qpcutree offset value. This value represents the effect of the reference frame on the current frame.

qpCuTreeOffset[cuIndex]＝qpAqOffset[cuIndex]-m_cuTreeStrength*log2_ratio。

Wherein: m _ cutetestrength is 5.0 (1.0-qCompress), qCompress is the compression factor of the quantizer, which has a default value of 0.6, but when it applies CUTree, it has a value of 1. qpAqOffset [ cuIndex ] is the QP offset value for each 1/2 downsampled 8 × 8CU block. log2_ ratio is the scale factor.

And carrying out normalization processing on the weighted SATD value to obtain a flag bit flag _ satdcost.

The specific method comprises the following steps:

(s1) mode selection for CU _64 with depth 0, i.e., CTU with size 64 × 64.

a. And acquiring the value of the flag bit flag _ satdcast with the current depth of 0.

b. And judging whether the CU _64 is positioned in the image, if so, proceeding to the next step, and if not, directly entering the step 2, namely the next depth division, namely dividing the current 64 × 64-sized CU _64 into 4 32 × 32-sized sub CU _ 32. The key judgment conditions are as follows:

(numPartitions>＝16||！bMVType)

c. it is determined whether to select either the Merge mode or the Skip mode. And judging according to the flag bits flag _ satdcost and mightNotSplit. Wherein: mightNotSplit is a flag bit of the CU _64 which is not divided down any more, and flag _ satdcost is a normalized SATD value weighted by the curree: and if the condition is true, calculating the RDcost of the Merge mode and the Skip mode in the inter-frame prediction, and selecting the mode with the smaller RDcost from the two modes as the temporary Best mode. And if the mightNotSplit is false, skipping the selection of the Merge mode and the Skip mode by the current depth and directly performing the step d. The key judgment conditions are as follows:

(flag_satdcost＝＝2||flag_satdcost＝＝1)&&mightNotSplit

d. and judging whether the depth of the next CU is divided or not, wherein the flag bits are mightSplit and flag _ satdcost. Wherein: mightSplit is a flag bit for continuously dividing the CU _64, and flag _ satdcost is a normalized SATD value weighted by the curree, if the condition is true, the CU _64 is subjected to next-depth quadtree division, and the step 2 is entered, that is, the CU _64 is divided into 4 sub-CUs _ 32. If the conditions are false, the next step 1.e is carried out directly. The key judgment conditions are as follows:

mightSplit&&(flag_satdcost！＝0)

e. it is judged whether or not to make judgment of eight kinds of Inter modes. By its flag bit mightNotSplit and

and f, flag _ satdcost, if the condition is true, performing eight modes of Inter mode selection to find out Best mode, and otherwise, directly performing the step 2. The key judgment conditions are as follows:

(mightNotSplit&&flag_satdcost＝＝2)

(s2) when the 1 st and d th steps are true, that is, when the flag bits mightSplit and flag _ satdcost of the 1 st step are both true, the mode selection is performed with the next depth of 1, that is, the CU is divided into four 32 × 32 CU _32 from CU _64, and the mode selection of the first CU _32_1 is calculated first.

b. And judging whether the CU is in the image, if so, carrying out the next step 2.c, and if not, directly entering the step 3, namely the next depth division, and dividing the CU which is currently the CU _32_1 into 4 sub-CUs _16 with the size of 16 multiplied by 16. The key judgment conditions are as follows:

(numPartitions>＝16||！bMVType)

c. it is determined whether to select either the Merge mode or the Skip mode. And judging according to the flag bits flag _ satdcost and mightNotSplit: and if the condition is true, calculating a Merge mode and a Skip mode in the inter-frame prediction, judging the Merge mode and the Skip mode according to the RDcost values of the Merge mode and the Skip mode, and selecting the mode with the smaller RDcost from the Merge mode and the Skip mode as the temporary Best mode. And if the mightNotSplit is false, skipping the selection of the Merge mode and the Skip mode by the current depth, and directly performing the step 2. d. The key judgment conditions are as follows:

(flag_satdcost＝＝2||flag_satdcost＝＝1)&&mightNotSplit

d. and judging whether the depth of the next CU is divided or not, wherein the flag bits are mightSplit and flag _ satdcost: if mightSplit is true and flag _ satdcost is equal to 1, the CU is subjected to next depth quadtree partitioning, and step 3 is entered, i.e., the CU is partitioned into four sub-CUs. If mightSplit is false or flag _ satdcost is not equal to 1, the next step 2.e is directly proceeded to. The key judgment conditions are as follows:

mightSplit&&(flag_satdcost！＝0)

e. it is determined whether or not eight kinds of Inter modes of the current CU _32_1 block are determined. Through the flag bits mightNotSplit and the flag _ satdcost, if the mightNotSplit is true and the flag _ satdcost is equal to 2, the mode selection of the Inter mode is carried out to find out the Best mode, otherwise, the step 3 is directly carried out. The key judgment conditions are as follows:

(mightNotSplit&&flag_satdcost＝＝2)

(s3) when step 2 and d are true, that is, when the flags mightSplit and flag _ satdcost of step 2 are both true, the mode selection is performed with the next depth of 2, that is, the CU is divided into four 16 × 16 CUs _16 from CU _32_1, and the mode selection of the first CU _16_1 is calculated first, and the current CU size is 16 × 16.

a. And acquiring the value of the flag bit flag _ satdcost with the current depth of 2.

b. And judging whether the CUs are all positioned in the image, if so, performing the step 3.c, and if not, directly entering the step 4, namely, performing the next depth division, and dividing the current 16 × 16-sized CU _16_1 into 4 8 × 8-sized sub-CUs _ 8. The key judgment conditions are as follows:

(numPartitions>＝16||！bMVType)

c. it is determined whether to select either the Merge mode or the Skip mode. And judging according to the flag bits flag _ satdcost and mightNotSplit: and if the condition is true, calculating a Merge mode and a Skip mode in the inter-frame prediction, judging the Merge mode and the Skip mode according to the RDcost values of the Merge mode and the Skip mode, and selecting the mode with the smaller RDcost from the Merge mode and the Skip mode as the temporary Best mode. And if the mightNotSplit is false, skipping the selection of the Merge mode and the Skip mode by the current depth, and directly performing the step 3. d. The key judgment conditions are as follows:

(flag_satdcost＝＝2||flag_satdcost＝＝1)&&mightNotSplit

d. and judging whether the depth of the next CU is divided or not, wherein the flag bits are mightSplit and flag _ satdcost: if mightSplit is true and flag _ satdcost is equal to 1, the CU is subjected to next depth quadtree partitioning, and step 4 is entered, i.e., the CU is partitioned into four sub-CUs. If mightSplit is false or flag _ satdcost is not equal to 1, the next step 3.e is directly proceeded to. The key judgment conditions are as follows:

mightSplit&&(flag_satdcost！＝0)

e. it is judged whether or not to make judgment of eight kinds of Inter modes. And selecting an inter-frame prediction mode to find out the Best mode if the mightNotSplit is true and the flag _ satdcost is equal to 2 through the flag bits mightNotSplit and the flag _ satdcost, otherwise, directly entering the step 4. The key judgment conditions are as follows:

(mightNotSplit&&flag_satdcost＝＝2)

(s4) when step 3.d is true, i.e. when the flags mightSplit and flag _ satdcost of step 3 are both true, it performs the mode selection with the next depth of 3, i.e. its CU _16_1 is divided into four 8 × 8 CUs _8 from the CU with size 16 × 16, and the mode selection of the first CU _8_1 is calculated first.

a. And acquiring the value of the flag bit flag _ satdcost with the current depth of 3.

b. And judging whether the CU is positioned in the image, if so, performing the step b, and if not, entering the step 5, namely, performing the mode selection judgment of the second 16 multiplied by 16 sub CU _16_2 of the third step. The key judgment conditions are as follows:

(numPartitions>＝16||！bMVType)

c. it is determined whether to select either the Merge mode or the Skip mode. And judging according to the flag bits flag _ satdcost and mightNotSplit: and if the condition is true, calculating a Merge mode and a Skip mode in the inter-frame prediction, judging the Merge mode and the Skip mode according to the RDcost values of the Merge mode and the Skip mode, and selecting the mode with the smaller RDcost from the Merge mode and the Skip mode as the temporary Best mode. And if the mightNotSplit is false, skipping the selection of the Merge mode and the Skip mode by the current depth, and directly performing the step 4. d. The key judgment conditions are as follows:

(flag_satdcost＝＝2||flag_satdcost＝＝1)&&mightNotSplit

d. it is judged whether or not to make other eight kinds of Inter modes. And selecting an inter-frame prediction mode to find out the Best mode if the mightNotSplit is true and the flag _ satdcost is equal to 2 through the flag bits mightNotSplit and the flag _ satdcost, otherwise, directly entering the step 5. The key judgment conditions are as follows:

(mightNotSplit&&flag_satdcost＝＝2)

(s5) the mode selection of CU _8_2 of the second 8 x 8 block is computed, and the best mode for CU _8_2 is found by going through 4.a, 4.b, 4.c in step 4.

(s6) the mode selection of CU _8_2 of the third 8 x 8 block is computed, and the best mode for CU _8_3 is found by going through 4.a, 4.b, 4.c in step 4.

(s7) calculate the mode selection for CU _8_2 of the fourth 8 x 8 block, go through 4.a, 4.b, 4.c in step 4, and find the best mode for CU _8_ 4.

(s8) calculate the mode selection of CU _16_2 for the second 16 × 16 block, go through steps 3, 4, 5, 6, and 7 to find the best mode for CU _16_ 2.

(s9) calculating the mode selection of CU _16_3 of the third 16 × 16 block, and traversing steps 3, 4, 5, 6 and 7 to find the best mode of CU _16_ 3.

(s10) calculate the mode selection of CU _16_4 of the fourth 16 × 16 block, go through steps 3, 4, 5, 6, and 7 to find the best mode of CU _16_ 4.

(s11) calculate the mode selection of CU _32_2 for the second 32 × 32 block, go through steps 2, 3, 4, 5, 6, and 7 to find the best mode for CU _32_ 2.

(s12) calculate the mode selection of CU _32_3 of the third 32 × 32 block, go through steps 2, 3, 4, 5, 6, and 7 to find the best mode of CU _32_ 3.

(s13) calculate the mode selection of CU _32_4 of the fourth 32 × 32 block, go through steps 2, 3, 4, 5, 6, and 7 to find the best mode of CU _32_ 4.

(s14) obtaining the best prediction mode for the current CTU.

The embodiment of the invention has the beneficial effects

The method for quickly selecting the inter-frame prediction mode based on the CUTree in the embodiment of the invention is characterized in that the value of the weighted SATD of the CUTree is normalized and used as the condition for mode selection when the CU performs inter-frame prediction, thereby reducing unnecessary coding time and improving coding efficiency.

The method provided by one embodiment of the invention is simple and easy to implement, and is beneficial to popularization in other coding standards.

Drawings

Fig. 1 is a schematic diagram of CU partitioning.

Fig. 2 is a flow chart of a fast inter-frame prediction mode selection method based on the curree, where the determination condition a is: whether the number of blocks of which the current CU is 4 x 4 is more than 16; the judgment condition B is as follows: mightNotSplit & (flag _ satdcost ═ 2| | flag _ satdcost ═ 1); the judgment condition C is as follows: whether the current Jmerge is smaller than Jskip; the judgment condition D is as follows: mightSplit & (flag _ satdcost | ═ 0); the judgment condition E is as follows: mightNotSplit & (flag _ satdcost ═ 2); the judgment condition F is as follows: whether Jbest1 is less than Jinter; j refers to the rate-distortion cost value of the current CU; mightsplit is a flag bit for continuously dividing the current CU; migntnotsplit is a flag bit that the current CU is not divided any more; flag _ satdcost is the normalized SATD value.

Detailed Description

The following are specific examples of the present invention, and the technical solutions of the present invention will be further described with reference to the examples, but the present invention is not limited to the examples.

Example 1

(1) acquiring a flag bit flag _ satdcast value of a current CU depth layer d;

(3) judging whether to select the Merge mode or the Skip mode: if yes, calculating the RDcost of the Merge mode and the Skip mode in the inter-frame prediction, and selecting the mode with the smaller RDcost from the Merge mode and the Skip mode as a temporary optimal mode; if not, skipping the selection of the Merge mode and the Skip mode by the current depth, and directly performing the step (4);

(6) and obtaining the optimal prediction mode of the current CU.

In the step (1), if the CU depth layer is a 0 layer, when the frame is a P frame and the SATD value is <15000, the flag _ satdcost is assigned to 0; 15000< SATD value <20000, the flag _ satdcost is assigned as 1; when the SATD value is more than 20000, the flag _ satdcost is assigned to 2; when the frame is an I frame and the SATD value is less than 15000, the flag _ satdcost is assigned to 0; 15000< SATD value <30000, flag _ satdcost is assigned to 1; when the SATD value is more than 30000, the flag _ satdcost is assigned to be 2; when the frame is a B frame and the SATD value is less than 10000, the flag _ satdcost is assigned to be 0; when 10000< SATD value <20000, the flag _ satdcost is assigned as 1; when the SATD value is >20000, the flag _ satdcost is assigned to 2.

If the CU depth layer is a 1 layer, when the frame is a P frame and the SATD value is less than 4000, the flag _ satdcost is assigned to be 0; when the SATD value is more than 4000 and less than 10000, the flag _ satdcost is assigned to be 1; when the SATD value is more than 10000, the flag _ satdcost is assigned to be 2; when the frame is an I frame and the SATD value is less than 4000, the flag _ satdcost is assigned to be 0; when the SATD value is 4000< 8000, the flag _ satdcost is assigned to be 1; when the SATD value is more than 8000, the flag _ satdcost is assigned to be 2; when the frame is a B frame and the SATD value is less than 2000, the flag _ satdcost is assigned to 0; when the SATD value is 2000< 5000, the flag _ satdcost is assigned to be 1; when the SATD value is >5000, the flag _ satdcost is assigned to 2.

In the step (1), if the depth layer of the CU is 2 layers, when the frame is a P frame and the SATD value is less than 1000, the flag _ satdcost is assigned to be 0; when the SATD value is 1000< 3000, the flag _ satdcost is assigned to 1; when the SATD value is more than 3000, the flag _ satdcost is assigned to be 2; when the frame is an I frame and the SATD value is less than 1000, the flag _ satdcost is assigned to be 0; when the SATD value is 1000< 3000, the flag _ satdcost is assigned to 1; when the SATD value is more than 3000, the flag _ satdcost is assigned to be 2; when the frame is a B frame and the SATD value is less than 1000, the flag _ satdcost is assigned to be 0; when the SATD value is 1000< 3500, the flag _ satdcost is assigned to 1; with SATD value >3500, flag _ satdcost is assigned a value of 2.

In the step (1), if the CU depth layer is 3 layers, the flag _ satdcost is assigned to 3.

The division into the next depth layer d +1 in the step (2) refers to dividing the current CU into 4 sub-CUs from the current CU to the next quadtree.

The judgment condition in the step (3) is (flag _ satdcost ═ 2| | | flag _ satdcost ═ 1) & & mightNotSplit.

The judgment condition of the step (4) is mightSplit & (flag _ satdcost | (0)).

The judgment condition in the step (5) is (mightNotSplit & & flag _ satdcost ═ 2).

As shown in fig. 2, the determination condition a is: whether the number of blocks of which the current CU is 4 x 4 is more than 16; the judgment condition B is as follows: mightNotSplit & (flag _ satdcost ═ 2| | flag _ satdcost ═ 1); the judgment condition C is as follows: whether the current Jmerge is smaller than Jskip; the judgment condition D is as follows: mightSplit & (flag _ satdcost | ═ 0); the judgment condition E is as follows: mightNotSplit & (flag _ satdcost ═ 2); the judgment condition F is as follows: whether Jbest1 is less than Jinter; j refers to the rate-distortion cost value of the current CU; mightsplit is a flag bit for continuously dividing the current CU; migntnotsplit is a flag bit that the current CU is not divided any more; flag _ satdcost is the normalized SATD value.

Example 2

For convenience, the following describes in detail the fast gear of the present invention based on the open source coding software x265 with reference to an example. Here, for inter prediction, it is calculated by taking CTU as a basic unit to perform inter mode selection, that is, starting from the largest CU with a depth of 0, that is, CTU (64 × 64). The size of the test sequence is 412 × 240, (6 × 64+32) × (4 × 64), that is, 7 × 4 × 28 CTUs with size 64 × 64 are required, and the first CTU of the I frame is taken for specific explanation.

The method comprises the following specific steps:

(1) mode selection for CU _64 with a depth of 0, i.e., a CU with a size of 64 × 64, is performed:

a. obtaining a flag _ satdcost of the SATD value with the current depth of 0: flag _ satdcost is 1;

b. and judging whether the CU _64 is located in the image at present, wherein the judgment conditions are as follows: (numcandidates > -16 | | | | | bMVType), where numcandidates is the number of 4 × 4 blocks of the current CU, bMVType is whether the MV is re-referenced or not,

numPartitions＝256，

if numcolours are more than 16, entering the next step;

c. judging whether to select the Merge mode and the Skip mode, wherein flag bits of the Merge mode and the Skip mode are flag _ satdcost and mightNotSplit: (flag _ satdcost ═ 2| | flag _ satdcost ═ 1) & & mightNotSplit, with the judgment condition false, jumping out of the judgment of the Merge mode and Skip mode, and proceeding to the next step;

d. and judging whether the depth of the next CU is divided or not, wherein the flag bits are mightSplit and flag _ satdcos: mightSplit & & (flag _ satdcost | ═ 0), with the condition true, the CU with the entry depth of 1 makes the mode selection, i.e. is divided into 4 sub-CUs of 32 × 32 size by CTU quad tree of 64 × 64 size;

(2) make the mode selection of the first CU _32_ 1:

a. obtaining a flag _ satdcost of the SATD value with the current depth of 1: flag _ satdcost is 1;

b. determine whether the current CU _32_1 is inside the image: the judgment conditions are as follows: (numselections > -16 | | | | | bMVType),

numPartitions＝64，

if numcolours are more than 16, entering the next step;

c. judging whether to judge the Merge mode and the Skip mode, wherein flag bits of the Merge mode and Skip mode are flag _ satdcost and mightNotSplit: (flag _ satdcost ═ 2) & & mightNotSplit, with the judgment condition of false, jumping out of the judgment of the Merge mode and the Skip mode, and carrying out the next step;

d. and judging whether the depth of the next CU is divided or not, wherein the flag bits are mightSplit and flag _ satdcost:

mightSplit & & (flag _ satdcost | ═ 0), with the condition true, the CU with the entry depth of 2 makes the mode selection, i.e. is divided into 4 sub-CUs of size 16 × 16 by CU quadtree of size 32 × 32;

(3) make the mode selection of the first CU _16_ 1:

a. obtaining a flag _ satdcost of the SATD value with the current depth of 2: flag _ satdcost is 1;

b. whether the CU _32_1 is in the image or not is judged, and the judgment conditions are as follows: (numselections > -16 | | | | | bMVType),

numPartitions＝16，

if numcolours are more than 16, entering the next step;

d. judging whether to divide the depth of the next CU or not, wherein the flag bits are mightSplit and flag _ satdcost,

mightSplit & & (flag _ satdcost | ═ 0), with the condition true, the CU with entry depth of 2 makes the mode selection, i.e. is divided into 4 sub-CUs of size 8 × 8 by CU quadtree of size 16 × 16;

(4) making the mode selection of the first CU _8_ 1:

a. obtaining a flag _ satdcost of the SATD value with the current depth of 3: flag _ satdcost is 3;

b. whether the CU _8_1 is in the image is judged, and the judgment conditions are as follows: (numselections > - | | | | bMVType), where |! The bMVType is true, and the next step is carried out;

c. judging whether to judge the Merge mode and the Skip mode, wherein flag bits of the Merge mode and Skip mode are flag _ satdcost and mightNotSplit: (flag _ satdcost ═ 2| | flag _ satdcost ═ 3) & & mightNotSplit, with the judgment condition true, enter the judgment of the Merge mode and Skip mode;

d. finding out the best prediction mode in the Merge mode and the Skip mode of the current CU _8_ 1;

firstly, setting a Skip mode as a Bestmode, and setting a Merge mode as a Tempmode;

calculating the rate distortion cost RDcost of the Skip mode, namely Jskip, and taking the Jskip as the RDcost of the optimal mode;

and then calculating the rate-distortion cost RDcost of the Merge mode: jmerge;

judging the RDcost size of the two modes;

if Jskip > Jmerge, the Merge mode is taken as the optimal prediction mode;

taking Skip mode as the best prediction mode if Skip < merge;

if Jskip > Jmerge of the current CU _8_1, the Merge mode is taken as the optimal prediction mode;

e. judging whether the next CU depth is divided or not, wherein the flag bits are mightSplit and flag _ satdcast, the mightSplit & (flag _ satdcast! ═ 0), and if the condition is false, directly carrying out the next step;

f. judging whether to judge the eight kinds of Intra modes, wherein the flag bits are mightNotSplit and

(mightNotSplit & & flag _ satdcost ═ 2), and since the flag _ satdcost is 3, the selection of the mode is directly skipped;

(5) make the mode selection of the second CU _8_ 2:

b. judging whether the CU _8_1 is located inside the image, wherein the main judgment conditions are as follows:

d. finding the best prediction mode of Merge mode and Skip mode of the current CU _8_ 2:

and then calculating the rate-distortion cost RDcost of the Merge mode: jmerge;

judging the RDcost size of two modes:

if Jskip > Jmerge, the Merge mode is taken as the optimal prediction mode;

taking Skip mode as the best prediction mode if Skip < merge;

if Jskip > Jmerge of the current CU _8_2, the Merge mode is taken as the optimal prediction mode;

(6) make the mode selection of the third CU _8_ 3:

b. judging whether the CU _8_3 is located inside the image, wherein the main judgment conditions are as follows:

c. judging whether to judge the Merge mode and the Skip mode, wherein flag bits of the Merge mode and Skip mode are flag _ satdcost and mightNotSplit:

(flag _ satdcost ═ 2| | flag _ satdcost ═ 3) & & mightNotSplit, with the judgment condition true, enter the judgment of the Merge mode and Skip mode;

d. finding the best prediction mode of Merge mode and Skip mode of the current CU _8_ 3:

and then calculating the rate-distortion cost RDcost of the Merge mode: jmerge;

judging the RDcost size of two modes:

if Jskip > Jmerge, the Merge mode is taken as the optimal prediction mode;

taking Skip mode as the best prediction mode if Skip < merge;

(7) making a mode selection of a fourth CU _8_ 4:

b. judging whether the CU _8_4 is located inside the image, wherein the main judgment conditions are as follows:

d. finding the best prediction mode of Merge mode and Skip mode of the current CU _8_ 4:

and then calculating the rate-distortion cost RDcost of the Merge mode: jmerge;

judging the RDcost size of two modes:

if Jskip > Jmerge, the Merge mode is taken as the optimal prediction mode;

taking Skip mode as the best prediction mode if Skip < merge;

if Jskip > Jmerge of the current CU _8_4, taking the Merge mode as the optimal prediction mode;

(8) at this time, a first CU _32 divided downwards by the current CTU, a first CU _16 divided downwards, and prediction modes of four CU _8 divided downwards are obtained, then mode selection of a second CU _16_2 is carried out, and the optimal mode of the current CU block is found out by traversing the steps 4, 5, 6 and 7;

(9) making mode selection of a third CU _16_3, and finding out the best mode of the current CU block by traversing the steps 4, 5, 6 and 7;

(10) making mode selection of a fourth CU _16_4, and finding out the best mode of the current CU block by traversing the 4 th, 5 th, 6 th and 7 th steps;

(11) at this time, a prediction mode of a first CU _32 divided downwards by the current CTU is obtained, then mode selection of a second CU _32_2 is carried out, and the optimal mode of the current CU block is found out by traversing the 3 rd, 4 th, 5 th, 6 th, 7 th, 8 th, 9 th and 10 th steps;

(12) making mode selection of a third CU _32_3, and finding out the best mode of the current CU block by traversing the 3 rd, 4 th, 5 th, 6 th, 7 th, 8 th, 9 th and 10 th steps;

(13) making mode selection of a fourth CU _32_4, and finding out the best mode of the current CU block by traversing the 3 rd, 4 th, 5 th, 6 th, 7 th, 8 th, 9 th and 10 th steps;

(14) obtaining a current CTU partitioning mode and an optimal prediction mode under the corresponding depth;

(15) the loop of mode prediction of the current CTU ends.

Claims

1. A method for quickly selecting an inter-frame prediction mode based on a CUTree is characterized by comprising the following steps:

(1) acquiring a flag _ satdcast value of a flag bit of a current CU depth layer d, wherein the flag _ satdcast value is a normalized SATD value weighted by a CUTree;

(2) judging whether the CUs are all in the image: if yes, performing the step (3); if not, directly entering the division of the next depth layer d +1, namely dividing the current CU into 4 sub-CUs from a downward quad-tree; the judgment condition is (numcandidates > =16| | | | | | bMVType), where numcandidates are the number of 4 × 4 blocks of the current CU, and bMVType is whether the MV is re-referenced;

(3) judging whether to select the Merge mode or the Skip mode: if yes, calculating the RDcost of the Merge mode and the Skip mode in the inter-frame prediction, and selecting the mode with smaller RDcost from the Merge mode and the Skip mode as a temporary optimal mode; if not, skipping the selection of the Merge mode and the Skip mode by the current depth, and directly performing the step (4); the judgment condition is (flag _ satdcost = =2| flag _ satdcost = =1) & & mightNotSplit, wherein mightNotSplit is a flag bit of the CU which is not divided downwards any more, and RDcost is rate distortion cost;

(4) judging whether to divide the depth of the next CU: if so, performing quadtree division on the CU at the next depth d + 1; if not, directly performing the step (5); the judgment condition is mightSplit & (flag _ satdcost! = 0), wherein the mightSplit is a flag bit of the CU which can be continuously divided;

(5) judging whether to judge eight inter-frame modes: if yes, carrying out mode selection of eight inter-frame modes; if not, directly dividing the next depth; the judgment condition is (mightNotSplit & & flag _ santdcost = = 2);

(6) obtaining the optimal prediction mode of the current CU;

in the step (1): if the CU depth layer is a 0 layer,

when the SATD value is less than 15000 in the case of P frame, the flag _ satdcost is assigned to 0; 15000< SATD value <20000, the flag _ satdcost is assigned as 1; when the SATD value is more than 20000, the flag _ satdcost is assigned to 2;

when the frame is an I frame and the SATD value is less than 15000, the flag _ satdcost is assigned to 0; 15000< SATD value <30000, flag _ satdcost is assigned to 1; when the SATD value is more than 30000, the flag _ satdcost is assigned to be 2;

when the frame is a B frame and the SATD value is less than 10000, the flag _ satdcost is assigned to be 0; when 10000< SATD value <20000, the flag _ satdcost is assigned as 1; when the SATD value is more than 20000, the flag _ satdcost is assigned to 2;

in the step (1): if the CU depth layer is a 1 layer,

when the frame is a P frame and the SATD value is less than 4000, the flag _ satdcost is assigned to be 0; when the SATD value is more than 4000 and less than 10000, the flag _ satdcost is assigned to be 1; when the SATD value is more than 10000, the flag _ satdcost is assigned to be 2;

when the frame is an I frame and the SATD value is less than 4000, the flag _ satdcost is assigned to be 0; when the SATD value is 4000< 8000, the flag _ satdcost is assigned to be 1; when the SATD value is more than 8000, the flag _ satdcost is assigned to be 2;

when the frame is a B frame and the SATD value is less than 2000, the flag _ satdcost is assigned to be 0; when the SATD value is 2000< 5000, the flag _ satdcost is assigned to be 1; when the SATD value is more than 5000, the flag _ satdcost is assigned to be 2;

in the step (1): if the CU depth layer is 2 layers,

when the frame is a P frame and the SATD value is less than 1000, the flag _ satdcost is assigned to be 0; when the SATD value is 1000< 3000, the flag _ satdcost is assigned to 1; when the SATD value is more than 3000, the flag _ satdcost is assigned to be 2;

when the frame is an I frame and the SATD value is less than 1000, the flag _ satdcost is assigned to be 0; when the SATD value is 1000< 3000, the flag _ satdcost is assigned to 1; when the SATD value is more than 3000, the flag _ satdcost is assigned to be 2;

when the frame is a B frame and the SATD value is less than 1000, the flag _ satdcost is assigned to be 0; when the SATD value is 1000< 3500, the flag _ satdcost is assigned to 1; when the SATD value is more than 3500, the flag _ satdcost is assigned as;

in the step (1): if the CU depth layer is 3 layers, the flag _ satdcost is assigned to 3.