KR102084802B1 - Adaptive PU mode decision method of HEVC encoder - Google Patents

Abstract

A PU mode selection method for HEVC fast coding is provided. The HEVC method according to the embodiment of the present invention skips the INTER SMP mode when the prediction mode of neighboring CUs of the current CU is INTER or SKIP mode and the split size is 2Nx2N, and when the prediction mode is INTER or SKIP mode, the INTRA 2Nx2N mode and INTRA Skip NxN mode. As a result, coding speed can be improved by using best PU mode information of spatially neighboring CUs, and memory usage and data loading time can be reduced by excluding RDCost between best PU mode information and CU depth of temporally co-located CUs. have.

Description

Adaptive PU mode decision method of HEVC encoder

The present invention relates to efficient encoding of an image, and more particularly, a method for enabling efficient fast encoding of an HEVC standard image by using optimal PU mode information of neighboring CUs to determine an optimal PU mode for a current CU. It is about.

1) High Efficiency Video Coding (HEVC)

HEVC is a new video coding standard developed after H.264 / AVC and provides approximately 50% compression improvement over H.264 / AVC. The coding structure of HEVC is very similar to H.264 / AVC, but there are two differences.

First, video data is compressed using a highly flexible quadtree partition structure. The quadtree partition structure allows HEVC to compress images of various resolutions into optimal partitions. The compression structure of HEVC is composed of three coding units (CUs), prediction units (PUs), and transform units (TUs).

A CU is similar to a macroblock of H.264 / AVC and represents a basic unit for compression. In HEVC, the size of a CU may have various sizes different from the fixed macroblock size (16 × 16) of H.264 / AVC. In HEVC, CU size can range from 8x8 to 64x64. The largest CU (largest CU: LCU) and the smallest CU (smallest CU: SCU) are described in the sequence parameter set (SPS). In addition, while one or two PUs are included in one CU, the TU may have various sizes as another quadtree structure in one CU.

The second difference compared to H.264 / AVC is that new coding tools are included in the HEVC standard for coding efficiency. New intra prediction directions have been added, including large transforms (16x16, 32x32), asymmetric motion partitions (AMP), and sample adaptive offset (SAO).

FIG. 1 illustrates a block diagram of an HEVC encoder. In particular, in the case of a PU, all PU modes are searched for a CU depth to be encoded unless a fast encoding method of Early_SKIP and CBF_Fast is applied. Therefore, searching for all PU modes increases coding time and is inefficient. Therefore, there is a need to increase the speed of encoding by reducing the unnecessary PU mode search and the calculation process generated according to the characteristics of the image.

As a result, the encoding process of HEVC is more complicated than that of H.264 / AVC, and thus an implementation method for fast encoding is required.

2) HEVC-based coded block determination method

HEVC uses a quad-tree structure to recursively divide a CU into smaller units according to the hierarchy. In order to determine the optimal CU size, the block size having the smallest cost for each PU size and TU size in each CU is examined. In this case, the PU is divided into prediction blocks of various shapes to investigate the cost. The TU splits into a recursive tree structure to investigate the cost for each size. Here, cost is a value quantified according to QP taking into account both image quality and compression ratio. For example, relatively low cost means that high compression is possible at the same image quality or high quality is possible at the same compression ratio.

2 shows a method of determining an optimal PU size and TU size from a CU of any size. In order to find a CU having an optimal cost, the cost is sequentially calculated according to a recursive quadtree structure from the largest LCU (large coding unit) to the smallest SCU (small coding unit). Each CU size calculates all the prediction costs for the various modes of the PU each time, and when performing the cost survey for each PU, the TU cost survey is also performed. The TU may be recursively divided into four subblocks.

In the encoding process of HEVC, it can be seen that the number of combinations of PUs and TUs is very large according to the CU, and a lot of calculation processes are required to determine the optimal CU, PU, and TU. In order to solve this problem, the following high speed methods have been developed.

① Early_SKIP

If one of the CUs first examines the INTER 2Nx2N mode and then the Coded Block Flag (CBF) value is 0 and the Motion vector difference (MVD) has the value (0,0), the other type of PU mode is not examined. Partition the CU to continue CU investigation at the next depth. Here, CBF is a flag indicating the presence or absence of a non-zero residual signal, and MVD is a difference value of motion information with neighboring blocks or co-located blocks of a previous frame.

② CBF_Fast

In the fast PU decision method according to the value of the CBF (CBF Fast Mode Setting; CFM), when the CBF of the INTER_2Nx2N PU is 0, other types of remaining PU modes are not searched.

③ Early_CU

As a method of early terminating the investigation process of the CU, if the optimal mode is determined as the SKIP mode during the investigation of the PU of the CU, recursive CU partitioning into lower blocks is not performed.

While such HEVC speed-up methods have emerged, coding time is still consumed for use in a real-time environment. Therefore, it is essential to design a new HEVC structure and to develop a high-speed method in addition to the existing high-speed methods.

The prior art of early selecting the PU mode in the HEVC encoder is as follows.

3.1) Prior Art I

Fast mode decision scheme using upperBestMode and UpperIntraCost of the upper depth in consideration of the CU depth level. This method uses the best PU mode information (: upperBestMode) of the upper CU depth and the RDCost information (: UpperIntraCost) of the INTRA mode of the upper CU depth to search for the best PU mode of the current CU depth.

3 shows two pieces of information (BestMode, IntraCost) of a higher CU depth used in a current CU depth to terminate a PU mode search early.

4 shows the entire process of fast PU mode decision. This method has a total of three conditions and if the corresponding conditions are satisfied, the PU mode search of the current CU depth is terminated.

① Satisfies condition 1 ((depth CU0 of current CU) && (best PU mode of upper CU depth is SKIP) && (best PU mode of current CU is SKIP / MERGE))

-PU mode skipped: Do not perform INTER NxN, INTER SMP, INTER AMP, INTRA 2Nx2N, INTRA NxN.

② Simultaneously satisfy conditions 2 and 3 ((depth of current CU> 0) && (best PU mode of upper CU depth is INTER) && (best PU mode of current CU is INTER AMP) && (divide best PU of current CU depth) Size is 2Nx2N) && (best RDCost of current CU depth <INTRA RDCost / 5 of higher CU depth))

-PU mode skipped: Do not perform INTRA 2Nx2N, INTRA NxN.

This method does not perform this method when the current CU depth is 0 because the best PU information of the upper CU depth must be used.

PU mode search skip structure using the correlation of the CU depth has two major disadvantages.

(1) The threshold value obtained by dividing the INTRA RDCost of the upper CU depth by an arbitrary value is not suitable for the fast coding method because the accuracy is lower depending on the image.

(2) Since the INTRA RDCost value of the upper CU depth must be stored in memory, it is not suitable as a method of speeding up because of an increase in memory usage and encoding delay.

3.2) Prior Art II

In order to determine the best PU mode of the current CU depth, a correlation between spatially / temporally existing CUs and upper CU depths is used.

5 shows various correlations that a current CU has. Here, CU1, CU2, and CU3 represent CUs that exist in the periphery around the current CU, and CU4 and CU5 represent CUs that exist in time. CU6 and CU7 are CUs having the upper depth of the current CU. Of these, the CUs having the same best PU mode with the same probability as the best PU mode of the current CU are CU1, CU2, and CU3 that exist spatially in the periphery.

6 shows a temporal hierarchical structure and a coding order between frames when encoding in a random access configuration. Level 1 is 8 as the frame distance, and Level 2, Level 3, and Level 4 have 4, 2, and 1 frame distances, respectively. In this case, the smaller the frame distance is, the higher the probability that the best PU mode of the current CU and the CU existing in time becomes the same.

This method first defines Equation (1) below as a prediction set with reference to FIG.

(1)

(One)

Then, various weight factors are defined according to the characteristics of the image area. The weight factor is defined as shown in the table of FIG. 7 using the property that a large PU block should be used in the homogeneous region and a small PU block in the case of complex or rich texture. (: INTER NXN, HM7 Excluded because it does not

In the table of FIG. 7, γ represents a weight factor of the PU mode according to the characteristics of the image region. Next, define block motion complexity (BMC) as shown in equation (2).

(2)

(2)

Where N is the number of CUs, w (i, l) is the weight factor function of each mode, and γ _i is the weight factor for the mode of CU _i . k _i is a parameter to check whether CU _i exists and has a value of 1 if present and 0 if not present. w (i, l) is defined as equations (3) and (4).

(3)

(3)

(4)

(4)

이다. w_i의 default value는 도 8의 표에, 프레임 거리 차에 따른 weight factor는 도 9의 표와 같다.Equation (3) is an adaptive weighting factor function. Where w _i is the weight factor of the adjacent CU

to be. The default value of w _i is shown in the table of FIG. 8, and the weight factor according to the frame distance difference is shown in the table of FIG. 9.

The l parameter of the adaptive weighting factor function is a value of the temporal hierarchical level of FIG. 6 and has one of 1,2,3,4 when the GOP is 8. Equation (4) is a weight of spatial, temporal or depth correlation, T _level is 0.02, Tk _i is a control value, CU ₄ , CU ₅ is 1, CU ₆ is -2, and the other CUs are 0. T _t also has a value of 0.5 as a control value.

Using this defined BMC value, an arbitrary threshold value is determined as in Equation (5).

(5)

(5)

In Equation (5), T ₁ is 1 and T ₂ is 3. If the BMC value satisfies condition 1, only the SKIP and INTER 2Nx2N modes are performed as slow homogeneous motion. If the condition 2 is satisfied, the SKIP, INTER 2Nx2N and INTER SMP modes are performed as medium motion. To perform.

The PU decision structure having such a threshold has two major disadvantages.

(1) Due to the large number of parameters that need to be changed according to the characteristics of the image area, and the accuracy of the threshold is inferior, performance is inferior.

(2) Since there is a parameter whose value is defined to determine the threshold value and there is a lot of information to be considered (spatial, temporal, upper CU depth information), the complexity increases and the fast coding performance decreases.

HEVC has a more complicated coding process than H.264 / AVC, so a new method for fast coding is expected to be extensively researched and developed. The structure of skipping a specific PU mode is also one of the promising methods.

Since information used for prediction of the best PU mode of the current CU is important for effective skipping of the PU mode, a search for a scheme for allowing the HEVC encoder to select a PU mode early is required.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to provide a method for selecting a PU mode of a CU to be encoded early with only best PU mode information of spatially neighboring CUs.

According to an embodiment of the present invention, a high efficiency video coding (HEVC) method includes: determining a prediction mode and a split size of neighboring CUs of a current CU; And skipping an INTER SMP (INTER Nx2N, INTER 2NxN) mode when the prediction mode is INTER or SKIP mode and the split size is 2Nx2N.

The HEVC method according to an embodiment of the present invention may further include skipping an INTRA 2Nx2N mode and an INTRA NxN mode when the prediction mode is an INTER or SKIP mode.

In addition, the determining may be performed when the depth of the current CU is 0 or 1.

The determining may be performed when the frame including the current CU is not an I frame and the CTB including the current CU is not positioned on the first upper line and the first left line of the frame.

In addition, the peripheral CUs may include at least one of a base-decoded Left CU, Above CU, and Above-Left CU.

On the other hand, HEVC method according to another embodiment of the present invention, the step of identifying the prediction mode of the neighboring CUs of the current CU; And skipping an INTRA 2Nx2N mode and an INTRA NxN mode if the prediction mode is an INTER or SKIP mode.

As described above, according to the embodiments of the present invention, the early prediction method of the PU mode for improving the speed of the HEVC encoder may improve the coding speed by using the best PU mode information of the spatially neighboring CU.

In addition, while the prior arts utilize threshold values using correlations between best PU mode information of temporally co-located CUs, best PU mode information of spatially neighboring CUs, and CU depth, based on the current CU, embodiments of the present invention. According to the present invention, memory usage and data retrieval time can be reduced by excluding RDCost between best PU mode information and CU depth of a temporally co-located CU.

1 is an HEVC encoder block diagram,
2 is a view showing a recursive CU, PU, TU determination process,
3 is a diagram illustrating a method of using encoding information of an upper CU depth in a current CU depth;
4 is a diagram illustrating a fast PU mode determination process;
5 is a diagram illustrating various correlations of a current CU;
6 is a diagram illustrating a frame hierarchy and coding order in a random access configuration;
7 is a table showing the definition of a weight factor for each PU mode;
8 is a table showing a weight factor (default setting) of CUs,
9 is a table showing weight factors (when frame distance is 1 and 2) of CUs,
10 is a diagram illustrating neighboring CUs referred to in predicting a best PU mode of a CU to be encoded;
11 is a table illustrating PU mode skip according to depth information of a current CU and best PU mode information of neighboring CUs;
12 is a flowchart of a PU mode search range determination structure of a CU;
FIG. 13 is a diagram for describing a method of processing a range prediction of a CU depth in units of frames. FIG.

Hereinafter, with reference to the drawings will be described the present invention in more detail.

The method of early terminating the PU mode prediction for speeding up the HEVC encoder according to the embodiment of the present invention basically starts at the CU level. FIG. 10 illustrates a method of using best PU mode information of spatially neighboring CUs (peripheral CUs) to early predict a best PU mode (optimal PU mode) of a CU depth to be currently encoded.

Spatially neighboring CUs are highly correlated with the current CU and have a high probability of having the same best PU mode. Therefore, in order to predict the PU mode of the current CU early, it is necessary to obtain the best PU mode information of neighboring CUs (left CU, Above-Left CU, Above CU) of the current CU. At this time, the prediction mode information and the split size information which are the best PU mode information of the base coded Left CU, Above CU, Above-Left CU specified in the table of FIG. 11 are used.

12 is a flowchart illustrating an overall flowchart of an early prediction method of a PU mode at a specific CU depth for speed improvement of an HEVC encoder. As shown in FIG. 12, it can be seen that a corresponding PU mode is skipped according to two conditions shown in the table of FIG. 11.

Condition ① in the table shown in FIG. 11 uses prediction mode information and split size information of a best PU mode of neighboring CUs. If the prediction mode of the best PU of the Above CU, Left CU, Above-Left CU is INTER or SKIP mode and the split size is 2Nx2N, the INTER SMP mode is skipped.

Condition ② of the table shown in FIG. 11 uses only prediction mode information of the best PU mode of neighboring CUs. In order to predict skip of the INTRA mode, the partition size information is not necessary. If the best PU mode prediction information of the Left CU, Above CU, Above-Left CU is INTER or SKIP mode, the INTRA 2Nx2N mode and the INTRA NxN mode are skipped.

The two conditions of the table shown in FIG. 11 apply only when the depth of the current CU is 0 (: 64x64) or 1 (: 32x32), and searches for all PU modes for the remaining CU depth.

In addition, when the frame is an I frame, the above method is not applied because the INTRA PU modes must be performed.

As illustrated in FIG. 12, early_SKIP, CBF_Fast, and Early_CU, which are fast encoding methods, are applied to an embodiment of the present invention. In order to determine the best PU mode of the current CU early, INTER 2Nx2N process is performed and Early SKIP and CBF Fast conditions are compared. If Early_SKIP and CBF_Fast conditions are true, the best PU mode of the CU is INTER 2Nx2N. If false, SKIP mode is executed.

After the execution of the SKIP mode, if the depth of the current CU, condition ① of the table shown in FIG. 11 is 0 or 1, and the prediction mode of the best PU mode of neighboring CUs satisfies INTER or SKIP and the split size satisfies 2Nx2N, the INTER SMP (: INTER 2NxN, INTER Nx2N) mode is skipped and if not satisfied, INTER SMP mode is executed.

Next, if the depth of the current CU, condition ② of the table shown in FIG. 11, is 0 or 1, and the prediction mode of the best PU mode of neighboring CUs satisfies INTER or SKIP, the INTRA 2Nx2N mode and the INTRA NxN mode are skipped. If not satisfied, it performs INTRA 2Nx2N mode and INTRA NxN mode.

Finally, RDO (Rate Distortion Optimization) is performed only on the searched PU modes to find the best PU mode of the CU to be currently encoded, and determines the best PU mode of the current CU depth.

FIG. 13 illustrates a PU mode early prediction method of a CU currently encoded in frame units. Since the method proposed in the embodiment of the present invention predicts the PU mode search range of the current CU using the information of the best PU mode of the spatially neighboring CU, the best PU mode information of neighboring CUs is necessary. Accordingly, in order to obtain best PU mode information of neighboring CUs, the CTBs of the first line and the CTBs of the first left line in the frame do not apply the method proposed in the embodiment of the present invention as shown in FIG. Therefore, the CTBs of the first top line and the first left line search for all PU modes.) Because the CTBs of the first line and the CTBs of the left line have the best PU mode information to be used for accurate prediction of the PU mode. (: There is no best PU mode information for Above, Above-Left, Left CUs)

The early prediction method of the PU mode for speed improvement of the HEVC encoder according to the embodiment of the present invention may improve the coding speed by using the best PU mode information of the spatially neighboring CU. This is because the prior arts utilize the threshold value using the correlation between the best PU mode information of the temporally co-located CU, the best PU mode information of the spatially neighboring CU, and the CU depth around the current CU, while the best of the temporally co-located CU is used. By excluding RDCost between PU mode information and CU depth, memory usage and data loading time can be reduced.

In addition, while the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, but the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

Above-Left CU Above CU
Left CU current CU
INTER SMP
INTRA 2Nx2N INTRA NxN

Claims (6)

Determining a prediction mode and a split size of neighboring CUs of the current CU; And
And skipping an INTER SMP mode when the depth of the current CU is 0 or 1, the prediction mode is an INTER or SKIP mode, and the split size is 2N × 2N. 2.
The method according to claim 1,
If the depth of the current CU is 0 or 1 and the prediction mode is INTER or SKIP mode, skipping an INTRA 2Nx2N mode and an INTRA NxN mode.
delete The method according to claim 1,
The grasping step,
And the frame including the current CU is not an I frame, and the CTB including the current CU is not positioned at the first upper line and the first left line of the frame.
The method according to claim 1,
The surrounding CUs,
HEVC method characterized in that it comprises at least one of the sub-subscribed Left CU, Above CU and Above-Left CU.
Determining a prediction mode of neighboring CUs of the current CU; And
If the depth of the current CU is 0 or 1 and the prediction mode is INTER or SKIP mode, skipping an INTRA 2Nx2N mode and an INTRA NxN mode.

Images