KR20200141703A - Video signal encoding method and apparatus and video decoding method and apparatus - Google Patents

Abstract

Provided are a method and device for encoding a video using an intra prediction mode signaling and encoding method. Therefore, the coding efficiency of a video signal is improved.

Description

Video signal processing method and apparatus TECHNICAL FIELD [Video signal encoding method and apparatus and video decoding method and apparatus]

The present invention relates to a video signal processing method and apparatus.

Video images are compression-encoded by removing spatio-temporal redundancy and inter-view redundancy, which can be stored in a format suitable for a storage medium or transmitted through a communication line.

The present invention is to improve the coding efficiency of a video signal.

In order to solve the above problems, the present invention provides a video encoding method and apparatus using an intra prediction mode signaling method.

The video signal processing method and apparatus according to the present invention can improve video signal coding efficiency through the intra prediction mode signaling method.

One Basic coding block structure

A picture is divided into a plurality of basic coding units (Coding Tree Units, hereinafter, CTU) as shown in Figure 1 .

Picture 1

It is configured not to overlap with other CTUs. For example, in the entire sequence, the CTU size can be set to 128x128, and any one of 128x128 to 256x256 can be selected and used in picture units.

A coding block/coding unit (CU) may be generated by hierarchically dividing the CTU. Prediction and transformation may be performed in units of coding units, and become a basic unit that determines a prediction coding mode. The prediction encoding mode represents a plurality of methods of generating a predicted image, such as intra prediction (intra prediction, hereinafter, intra prediction), inter prediction (hereinafter, inter prediction), or combined prediction. Can be mentioned. Specifically, for example, a prediction block may be generated by using at least one of intra prediction, inter prediction, and composite prediction in a coding unit unit. In the inter prediction mode, when the reference picture indicates the current picture, a prediction block may be generated in an area within the current picture that has already been decoded. Because a prediction block is generated using a reference picture index and a motion vector, it can be included in inter prediction. Intra prediction is a method of generating a prediction block using information of the current picture, inter prediction is a method of generating a prediction block using information of another picture that has already been decoded, and composite prediction is a method of mixing inter prediction and intra prediction. This is how to use.

The CTU can be divided (partitioned) into a quad tree, binary tree or triple tree as shown in Figure 2 . The divided block can be further divided into a quad tree, binary tree, or triple tree. The method of dividing the current block into 4 square partitions is called quad tree partitioning, the method of dividing the current block into 2 irregular partitions is called binary tree partitioning, and the method of dividing the current block into 3 irregular partitions It is defined as binary tree partitioning.

Binary partitioning in the vertical direction (SPLIT_BT_VER in Figure 2 ) is called vertical binary tree partitioning, and binary tree partitioning in the horizontal direction (SPLIT_BT_HOR in Figure 2 ) is defined as horizontal binary tree partitioning.

Triple-tree partitioning in the vertical direction (SPLIT_TT_VER in Figure 2 ) is called vertical triple-tree partitioning, and triple-tree partitioning in the horizontal direction (SPLIT_TT_HOR in Figure 2 ) is defined as horizontal triple-tree partitioning.

The number of additional divisions is called partitioning depth, and the maximum value of the partitioning depth for each sequence or picture/tile set can be set differently, and different partitioning depths according to the partitioning tree type (quad tree/binary tree/triple tree) It can be set to have, and a syntax indicating this can be signaled.

Picture 2

The divided coding block may be further divided by a method such as quad partitioning, binary partitioning, or multi-partitioning to configure a coding unit, or may configure a coding unit without further partitioning.

Figure 3

As shown in Figure 3 , a coding unit can be hierarchically configured for one CTU, and a coding unit can be partitioned using at least one of a coding unit using binary tree partitioning, quad tree partitioning/triple tree partitioning. This method is defined as multi tree partitioning.

A coding unit generated by dividing an arbitrary coding unit having a partitioning depth of k is referred to as a lower coding unit, and the partitioning depth is k+1. A coding unit having a partitioning depth k including a lower coding unit having a partitioning depth k+1 is referred to as an upper coding unit.

The partitioning type of the current coding unit may be limited according to the partitioning type of the upper coding unit and/or the partitioning type of the coding units surrounding the current coding unit.

Here, the partitioning type represents an indicator indicating which partitioning is used among binary tree partitioning, quad tree partitioning/triple tree partitioning.

2 How to generate predictive image

In video encoding, a prediction image can be generated by a plurality of methods, and a method of generating a prediction image is called a prediction encoding mode.

The prediction encoding mode may include an intra prediction encoding mode, an inter prediction encoding mode, a current picture reference encoding mode, or a combined prediction mode.

The inter prediction coding mode is a prediction coding mode that generates a prediction block (prediction image) of the current block using information of a previous picture, and the intra prediction coding mode is a prediction that generates a prediction block using samples adjacent to the current block. It is called an encoding mode. A prediction block may be generated using an image that has already been reconstructed of the current picture, and this is defined as a current picture reference mode or an intra block copy mode.

A prediction block may be generated using at least two or more of an inter prediction encoding mode, an intra prediction encoding mode, or a current picture reference encoding mode, which is referred to as a combined prediction mode.

2.1 Intra prediction coding method

In intra prediction, as shown in Fig. 4 , an already coded boundary sample around the current block is used to generate intra prediction, and this is defined as an intra reference sample.

The average value of the intra reference samples is predicted by setting the value of all samples in the prediction block (DC mode), or the vertical direction prediction sample generated by weighted prediction of the horizontal direction prediction sample and the vertical direction reference sample generated by performing horizontal reference weighted prediction After generation, a prediction sample may be generated by weighted prediction of a horizontal direction prediction sample and a vertical direction prediction sample (Planar mode), or intra prediction may be performed using a directional intra prediction mode.

Figure 4

As shown in the left figure of Figure 5 , intra prediction can be performed using 33 directions (a total of 35 intra prediction modes), and 65 directions (a total of 67 intra prediction modes) as shown in the right figure of Figure 5 May be. When directional intra prediction is used, an intra reference sample (reference reference sample) may be generated in consideration of the direction of the intra prediction mode, and intra prediction may be performed therefrom.

An intra reference sample on the left side of the coding unit is called a left intra reference sample, and an intra reference sample on the upper side of the coding unit is called an upper intra reference sample.

When directional intra prediction is performed, as shown in Table 1, an intra direction parameter (intraPredAng), which is a parameter indicating a prediction direction (or a prediction angle), may be set according to the intra prediction mode. Table 3 below is only an example based on a directional intra prediction mode having a value of 2 to 34 when 35 intra prediction modes are used. It goes without saying that the prediction directions (or prediction angles) of the directional intra prediction modes are further subdivided, so that more than 33 directional intra prediction modes can be used.

Picture 5

Table 1

Figure 6

When intraPredAng is negative (for example, when the intra prediction mode index is between 11 and 25), as shown in Figure 6 , the left intra reference sample and the upper intra reference sample are configured in 1D according to the angle of the intra prediction mode. It can be reconstructed as a one-dimensional reference sample (Ref_1D).

Figure 7

When the intra prediction mode index is between 11 and 18, a one-dimensional reference sample can be generated in a counterclockwise direction from an intra reference sample located on the upper right side of the current block to an intra reference sample located on the lower left side as shown in Figure 7 .

In other modes, a one-dimensional reference sample may be generated using only the upper side intra reference sample or the left side intra reference sample.

Figure 8

When the intra prediction mode index is between 19 and 25, a one-dimensional reference sample can be generated in a clockwise direction from an intra reference sample located at the lower left side of the current block to an intra reference sample located at the right side of the current block, as shown in Figure 8 .

A weight-related parameter ifact applied to at least one reference sample determined based on the reference sample determination index iIdx and iIdx may be derived as in Equations (1) to (2) below. iIdx and i _fact are variably determined according to the slope of the directional intra prediction mode, and a reference sample specified by iIdx may correspond to an integer pel.

iIdx = (y+1) * P _ang /32 (One)

i _fact = [(y+1) * P _ang ] & 31 (2)

The prediction image may be derived by specifying at least one one-dimensional reference sample for each prediction sample. For example, a position of a one-dimensional reference sample that can be used to generate a prediction sample may be specified in consideration of a slope value of a directional intra prediction mode. Each prediction sample may have a different directional intra prediction mode. A plurality of intra prediction modes may be used for one prediction block. The plurality of intra prediction modes may be represented by a combination of a plurality of non-directional intra prediction modes, a combination of one non-directional intra prediction mode and at least one directional intra prediction mode, or a plurality of directional intra prediction modes. It can also be expressed as a combination of modes. A different intra prediction mode may be applied for each predetermined sample group in one prediction block. A predetermined sample group may consist of at least one sample. The number of sample groups may be variably determined according to the size/number of samples of the current prediction block, or may be a fixed number pre-set in the encoder/decoder independently of the size/number of samples of the prediction block.

Specifically, for example, the position of the one-dimensional reference sample may be specified using the reference sample determination index iIdx.

When the slope of the intra prediction mode cannot be expressed using only one one-dimensional reference sample according to the slope of the intra prediction mode, a first prediction image may be generated by interpolating adjacent one-dimensional reference samples as shown in Equation (3). When the angular line according to the slope/angle of the intra prediction mode does not pass the reference sample located in the integer pel, the first prediction image may be generated by interpolating the reference sample adjacent to the left/right or above/below the corresponding angular line. . The filter coefficient of the interpolation filter used at this time may be determined based on i _fact . For example, the filter coefficient of the interpolation filter may be derived based on a distance between a fractional pel located on an angular line and a reference sample located on the integer pel.

P(x,y) = ((32-i _fact )/32)* Ref_1D(x+iIdx+1) + (i _fact /32)* Ref_1D(x+iIdx+2) (3)

When the slope of the intra prediction mode can be expressed with only one one-dimensional reference sample (when i _fact is 0), the first prediction image can be generated as shown in Equation (4) below.

P(x,y) = Ref_1D(x+iIdx+1) (4)

2.2 Wide-angle intra prediction coding method

The prediction angle of the directional intra prediction mode may be set between 45 and -135 as shown in Figure 9 .

Figure 9

When the intra prediction mode is performed in the amorphous coding unit, a disadvantage of predicting a current sample from an intra reference sample far from the current sample instead of an intra reference sample close to the current sample may occur due to a predefined prediction angle.

For example, as shown in the left figure of Figure 10 , in a coding unit whose width is larger than the height of the coding unit (hereinafter, a horizontal coding unit), intra prediction is performed at a far distance L instead of a close sample T. I can. As another example, as shown in the right figure of Figure 10, in a coding unit whose height is greater than the width of the coding unit (hereinafter, the vertical coding unit), the intra-distance sample T from the distant sample T is You can make predictions.

Figure 10

In the amorphous coding unit, intra prediction may be performed at a prediction angle wider than a predefined prediction angle, and this is defined as a wide angle intra prediction mode.

내지

의 예측 각도를 가질 수 있으며, 기존 인트라 예측 모드에서 사용된 각도를 벗어나는 예측 각도를 와이드 앵글 각도라고 정의 한다. The wide angle intra prediction mode

To

It can have a prediction angle of, and a prediction angle that deviates from the angle used in the existing intra prediction mode is defined as a wide angle angle.

In the left figure of Fig. 10 , sample A in the horizontal direction coding unit can be predicted from the intra reference sample T using the wide-angle intra prediction mode.

In the right figure of Fig. 10 , sample A in the vertical direction coding unit can be predicted from the intra reference sample L using the wide-angle intra prediction mode.

N+M intra prediction modes may be defined by adding M wide angle angles to N existing intra prediction modes. Specifically, as shown in Table 2, a total of 95 intra prediction modes may be defined by adding 28 wide angle angles to 67 intra modes.

The intra prediction mode that the current block can use may be determined according to the shape of the current block. For example, 65 directional intra prediction modes may be selected from among 95 directional intra prediction modes based on at least one of a size of a current block, an aspect ratio (eg, a ratio of width and height), and a reference line index.

Table 2

The intra prediction mode angle shown in Table 2 may be adaptively determined based on at least one of a shape of a current block and a reference line index. For example, the intraPredAngle of Mode 15 may be set to have a larger value when the current block is a square than when the current block is an amorphous. Alternatively, the intraPredAngle of Mode 75 may be set to have a larger value when a non-adjacent reference line is selected than when an adjacent reference line is selected.

When using the wide-angle intra prediction mode, the length of the upper intra reference sample can be set to 2W+1 and the length of the left intra reference sample can be set to 2H+1, as shown in Figure 11 .

Figure 11

In the case of using wide-angle intra prediction, when the intra prediction mode of the wide-angle intra prediction mode is encoded, the number of intra prediction modes increases, and thus encoding efficiency may be lowered. The wide-angle intra prediction mode can be encoded by replacing the existing intra prediction mode that is not used in the wide-angle intra prediction mode, and the replaced prediction mode is called a wide-angle replacement mode. The wide angle replacement mode may be an intra prediction mode in a direction opposite to the wide angle intra prediction mode.

Specifically, for example, when 35 intra predictions are used as shown in Figure 12 , wide-angle intra prediction mode 35 can be encoded using intra prediction mode 2, which is a wide-angle replacement mode, and wide-angle intra prediction mode 36 is used for wide-angle replacement. It can be encoded in intra prediction mode 3, which is a mode.

Figure 12

The replacement mode and number may be differently set according to the shape of the coding block or the ratio of the height to the width of the coding block. Specifically, for example, Table 3 shows intra prediction modes used according to the ratio of the width and height of a coding block.

Table 3

2.3 Multi-line intra prediction coding method

As shown in Figure 13 below, intra prediction may be performed on a plurality of intra reference lines, which is called a multi-line intra prediction coding method.

Fig. 13

Intra prediction may be performed by selecting any one of a plurality of intra reference lines composed of an adjacent intra parallel line and a non-adjacent intra reference line, and this is called a multi-line intra prediction method. The non-adjacent intra reference line is a first non-adjacent intra reference line (non-adjacent reference line index 1), a second non-adjacent intra reference line (non-adjacent reference line index 2), and a third non-adjacent intra reference line (non-adjacent reference line). It can be composed of index 3). Only some of the non-adjacent intra reference lines may be used. For example, only a first non-adjacent intra reference line and a second non-adjacent intra reference line may be used, or only a first non-adjacent intra reference line and a third non-adjacent intra reference line may be used.

An intra reference line index (intra_luma_ref_idx), a syntax specifying a reference line used for intra prediction, may be signaled in units of coding units.

Specifically, when using the adjacent intra reference line, the first non-adjacent intra reference line, and the third non-adjacent intra reference line, intra_luma_ref_idx may be defined as shown in Table 4 below.

Table 4

When a non-adjacent intra reference line is used, it may be set not to use a non-directional intra prediction mode. That is, when a non-adjacent intra reference line is used, it may be limited not to use the DC mode or the planar mode.

The number of samples belonging to the non-adjacent intra reference line may be set to be larger than the number of samples of the adjacent intra parallel line. Also, the number of samples of the i+1th non-adjacent intra reference line may be set to be greater than the number of samples of the ith non-adjacent intra reference line. The difference between the number of upper samples of the i-th non-adjacent intra reference line and the number of upper samples of the i-1th non-adjacent intra reference line may be expressed as a reference sample number offset offsetX[i].

offsetX[1] represents a difference value between the number of upper samples of the first non-adjacent intra reference line and the number of upper samples of the adjacent intra reference line. The difference between the number of left samples of the i-th non-adjacent intra reference line and the number of left samples of the i-1th non-adjacent intra reference line may be expressed as a reference sample number offset offsetY[i]. OffsetY[1] represents a difference between the number of left samples of the first non-adjacent intra reference line and the number of left samples of the adjacent intra reference line.

A non-adjacent intra reference line with an intra reference line index of i may be composed of an upper non-adjacent reference line refW + offsetX[i], a left non-adjacent reference line refH+ offsetY[i], and an upper left sample. The non-adjacent intra reference line The number of included samples may consist of refW + refH + offsetX[i] + offsetY[i] +1.

refW =2* nTbW (5)

refH =2*　*nTbH (6)

In equations (5) to (6), nTbW represents the width of the coding unit, nTbH represents the height of the coding unit, and whRatio can be defined as in the following equation (7).

whRatio = Min(log2(nTbW/nTbH),2) (7)

In the multi-line intra prediction coding method, when a non-adjacent intra reference line is used, the wide-angle intra mode may be set not to be used. Alternatively, if the MPM mode of the current coding unit is a wide-angle intra mode, the multi-line intra prediction coding method may be set not to be used.

In this case, a non-adjacent intra reference line with an intra reference line index of i may consist of an upper non-adjacent reference line W + H + offsetX[i], a left non-adjacent reference line H + W + offsetY[i], and an upper left sample. , The number of samples belonging to the non-adjacent intra reference line may be composed of 2W + 2H + offsetX[i] + offsetY[i] +1, and the values of offsetX[i] and offsetY[i] may vary depending on the whRatio value. have.

For example, if the whRatio value is greater than 1, the offsetX[i] value can be set to 1 and the offsetY[i] value can be set to 0. If the whRatio value is less than 1, the offsetX[i] value can be set to 0, offsetY[ i] You can also set the value to 1.

2.4 Sub-partition intra prediction coding and decoding method

Figure 14

As shown in Fig. 14, after intra prediction is performed in units of coding units, and after partitioning the coding units in units of sub-blocks, transformation and quantization may be performed, respectively, and this is called a sub-partition intra coding method.

In the sub-partition intra coding method, all sub-blocks in the coding unit may be restricted to use the same intra prediction mode.

Alternatively, in the sub-partition intra coding method, at least one intra prediction mode among sub-blocks in the coding unit may be different from other sub-blocks. For example, the intra prediction mode of the nth subblock may be derived by adding/subtracting an offset in the intra prediction mode of the n-1th subblock. The offset may be predefined in the encoder/decoder, or may be derived according to the shape and size of the coding unit, the intra prediction mode, and the shape and size of the sub-partition. Alternatively, information for determining the offset may be signaled through a bitstream.

Alternatively, some of the plurality of sub-blocks may use a directional intra prediction mode, and others may use a non-directional prediction mode. In this case, only when the directional prediction mode of the first sub-block is a predefined intra prediction mode, the non-directional prediction mode may be used in the second sub-block.

Alternatively, in the sub-partition intra coding method, an intra prediction mode of sub-blocks in a coding unit may be selected from predefined candidates. Here, the predefined candidate may include at least one of a horizontal intra prediction mode, a vertical intra prediction mode, a diagonal intra prediction mode (eg, an upper left, an upper right, and a lower left intra prediction mode), and a non-directional prediction mode. .

Alternatively, in the sub-partition intra coding method, the intra prediction mode of sub-blocks in the coding unit may be derived only from MPM candidates.

Alternatively, the use of the sub-partition intra coding method may be restricted according to the intra prediction mode of the coding unit. For example, when the intra prediction mode of the coding unit is a non-directional prediction mode (eg, Planar or DC), or when a predefined directional prediction mode (eg, horizontal, vertical or diagonal direction), the sub-partition intra coding method is used. Can be set not to be. Alternatively, when the intra prediction mode of the coding unit is an intra prediction mode in which a neighboring sub-partition cannot be utilized as a reference sample (eg, a diagonal direction or a wide-angle intra prediction mode), the sub-partition intra coding method may be set not to be used. I can.

Alternatively, the use of the sub-partition intra coding method may be limited according to the size or shape of the coding unit. For example, when the height-width ratio of the coding unit is greater than or equal to the threshold value, or when at least one of the width or height of the current block is less than or equal to the threshold value, the sub-partition intra encoding method may be set not to be used.

Different quantization parameter values may be used for each subblock, and a difference between a quantization parameter value of a previous subblock and a quantization parameter value of a current subblock may be encoded.

The intra sub-partition can be divided into a vertical partition or a horizontal partition. The vertical partition is generated by dividing the coding block using at least one vertical line, and the horizontal partition may be generated by dividing the coding block using at least one horizontal line.

The number of partitions may be determined according to the size/type of the coding unit. For example, in a 4x4 coding unit, a sub-partition is not divided, in an 8x4 to 4x8 coding unit, it can be divided into two sub-partitions, and in other coding units, it can be divided into four partitions.

Alternatively, information indicating the number, size, or shape of partitions may be signaled through a bitstream. The size or shape of the partition may be determined according to the number of partitions determined by signaled information, or the number of partitions may be determined according to the size/type of the partition determined by signaled information.

As shown in Figure 15, the coding unit can be divided into a vertical partition or a horizontal partition. An intra partition type flag (intra_subpart_type_flag) indicating whether to divide into a vertical partition or a horizontal partition may be transmitted through a bitstream. If the intra_subPart_type_flag value is 1, it indicates that a sub-partition can be configured by dividing into horizontal partitions, and if the intra_subPart_type_flag value is 0, it indicates that a sub-partition can be configured by dividing into vertical partitions.

Alternatively, based on the size/shape of the coding unit, the shape of the intra sub-partition may be derived. For example, it is possible to derive the intra sub-partition type without signaling syntax in the bitstream by using the shape ratio (whRatio) of the coding unit.

For example, if whRatio is less than or equal to a specific value N (hereinafter, a positive sub-partition threshold), a vertical partition is used, and whRatio is less than a specific value -N (hereinafter, a negative sub-partition threshold). In the case of or, you can use horizontal partitions. For example, the sub-partition threshold may be set to a predefined value such as 1 to 2 to 3, or a value defined in units of a sequence.

Alternatively, the type of the intra sub-partition may be determined according to the intra prediction mode of the coding unit. For example, when the intra prediction mode of the coding unit is in the horizontal direction, vertical partitioning (or horizontal partitioning) is applied, and when the intra prediction mode of the coding unit is in the vertical direction, horizontal partitioning (or vertical partitioning) is applied. I can.

Figure 15

Alternatively, any one of the width or height of the sub-partition may be limited so as not to be smaller than a predefined value. The predefined values may be 2, 4, 8, and the like. For example, if the predefined value is 4, as shown in the upper figure of Fig. 15, the 4x8 coding unit may limit the use of vertical partitioning without signaling the sub-partition type. As another example, as shown in the lower figure of Fig. 15, the 8x4 type coding unit may limit the use of horizontal partitioning without signaling the sub-partition type.

Fig 16

Specifically, for example, if the positive sub-partition threshold is 2, as shown in the upper figure of Figure 16, the coding unit with whRatio of 1 can divide the coding unit using a horizontal partition, and the lower figure of Figure 16 Likewise, the coding unit having whRatio of 2 may divide the coding unit using a vertical partition.

Figure 17

For another example, if the negative sub-partition threshold is 2, as shown in the left figure of Figure 17, the coding unit with whRatio of -1 can divide the coding unit using a vertical partition, and the right figure of Figure 17 As described above, a coding unit having a whRatio of -2 may divide the coding unit using a horizontal partition.

Intra prediction of the sub-partition may be performed based on a reference sample derived from a neighboring sample neighboring the sub-partition. For example, intra prediction of the second sub-partition may be performed using a reference sample included in a first sub-partition adjacent to the top or left of the second sub-partition. Here, the reference sample may be derived from a reconstructed sample of the first sub-partition. Accordingly, parallel intra prediction may not be applied between sub-partitions. That is, after encoding/decoding of the first sub-partition is completed, intra prediction for the second sub-partition may be performed.

In the sub-partition intra coding method, it may be restricted not to use the PDPC. Alternatively, when the sub-partition intra coding method is used, PDPC may be applied only to some sub-blocks. As an example, the PDPC may be applied only to the sub-blocks in contact with the upper boundary or the left boundary of the coding unit, or the PDPC may be applied only to the sub-blocks in contact with the lower boundary or the right boundary.

2.6 Matrix-based intra prediction coding method

Fig. 18

Figure 19

Figure 20

Fig. 21

As shown in Figs. 18 to 21, an upper boundary smoothing sample bdry _red ^top and a left boundary smoothing sample bdry _red ^left can be generated by down-sampling neighboring samples adjacent to the current coding block. The upper boundary smoothing sample may be generated by averaging the N upper neighboring samples, and the left boundary smoothing sample may be generated by averaging the N left neighboring samples. The upper boundary smoothing sample and the left boundary smoothing sample can be defined as the boundary smoothing sample bdry _red .

The sampling rate (ie, the value of N) may be determined based on at least one of the size, shape, or intra prediction mode of the current coding block. The sampling rate for the upper boundary smoothing sample may be determined based on the width of the coding block, and the sampling rate for the left boundary smoothing sample may be determined based on the height of the coding block.

For example, according to the width or height of the coding block, the number of neighboring samples to be smoothed may be different. When the width or height of the coding block is 4, a boundary smoothing sample bdry _Red may be generated by smoothing two neighboring samples. As another example, when the width or height of the coding block is 16, the boundary smoothing sample bdry _Red may be generated by smoothing four neighboring samples.

When the coding block is amorphous in which the width is greater than the height, the downsampling rate of left neighboring samples may be smaller than the downsampling rate of upper neighboring samples. Conversely, when the coding block is amorphous whose height is greater than the width, a downsampling rate of upper neighboring samples may be smaller than a downsampling rate of left neighboring samples.

As shown in Figs. 18 to 21, an upper boundary smoothing sample bdry _red ^top and a left boundary smoothing sample bdry _red ^left can be generated by down-sampling neighboring samples adjacent to the current coding block. The upper boundary smoothing sample may be generated by averaging the N upper neighboring samples, and the left boundary smoothing sample may be generated by averaging the N left neighboring samples. The upper boundary smoothing sample and the left boundary smoothing sample can be defined as the boundary smoothing sample bdry _red .

The sampling rate (ie, the value of N) may be determined based on at least one of the size, shape, or intra prediction mode of the current coding block. The sampling rate for the upper boundary smoothing sample may be determined based on the width of the coding block, and the sampling rate for the left boundary smoothing sample may be determined based on the height of the coding block.

For example, according to the width or height of the coding block, the number of neighboring samples to be smoothed may be different. When the width or height of the coding block is 4, a boundary smoothing sample bdry _Red may be generated by smoothing two neighboring samples. As another example, when the width or height of the coding block is 16, the boundary smoothing sample bdry _Red may be generated by smoothing four neighboring samples.

When the coding block is amorphous in which the width is greater than the height, the downsampling rate of left neighboring samples may be smaller than the downsampling rate of upper neighboring samples. Conversely, when the coding block is amorphous whose height is greater than the width, a downsampling rate of upper neighboring samples may be smaller than a downsampling rate of left neighboring samples.

(8)

(8)

As shown in Equation (8), a sub-sampled prediction value predMip can be generated using mWeight[iMode] and vBias[iMode] on the boundary smoothing sample bdry _Red .

The matrices MWeight[iMode] and vBias[iMode] may be determined based on the intra prediction mode of the current coding block. For example, if the intra prediction mode is 0 to 3, MWeight[0] may be used, and if the intra prediction mode is 4 to 7, MWeight[1] may be used.

In Figures 19-21, the grayed out samples represent locations corresponding to predMIP. By performing linear interpolation or up-sampling the sub-sampled prediction value predMIP, a final prediction image pred for the current coding block may be generated.

This method is called a matrix-based intra prediction coding method. In a coding block in which the matrix-based intra prediction coding method is used, it may be limited not to use the secondary transform or RST. Alternatively, when primary transforming a coding block using a matrix-based intra prediction coding method, a predefined transform core may be used. Alternatively, for a coding block in which the matrix-based intra prediction coding method is used, the use of transform_skip may be restricted.

A syntax intra_mip_flag indicating whether the matrix-based intra prediction coding method is used in the coding block may be signaled.

If the intra_mip_flag value is 1, it indicates that the matrix-based intra prediction coding method is used in the coding block, and if the intra_mip_flag value is 0, it indicates that the matrix-based intra prediction coding method is not used in the coding block.

If the intra_mip_flag value is 1, a flag intra_mip_mpm_flag indicating whether the prediction mode (hereinafter, matrix intra prediction mode) of the matrix-based intra prediction encoding method is an mpm mode may be signaled through a bitstream. If the intra_mip_mpm_flag value is 1, it indicates that the matrix intra prediction mode is any one of the MPM modes, and if the intra_mip_mpm_flag value is 0, it indicates that the matrix intra prediction mode is not the MPM mode.

The intra prediction mode of the left neighboring block of the current coding block is referred to as candMipModeA, and the intra prediction mode of the upper neighboring block of the current coding block is referred to as candMipModeB.

If the intra prediction mode of the left neighboring block is not available, the candMipModeA value may be set to -1, and if the intra prediction mode of the upper neighboring block is not available, the candMipModeB value may be set to -1. If the size or SizeID of the current coding block and the neighboring block are not the same, it may be determined that the intra prediction mode of the neighboring block is not available. SizeID can be determined according to the size of the block.

When both candMipModeA and candMipModeB are -1, the MPM candidate may be set to a predefined default mode (hereinafter, referred to as MIP MPM default mode).

The MIP MPM default mode may be set differently according to the size of the block.

Table 5 shows the default mode for each SizeID. SizeID 0 indicates a case where the size of a coding block is 4x4, and SizeID 1 indicates a case where the size of a coding block is 4x8 or 8x4. SizeID 2 represents a case where the size of the coding block is 8x8 or the width/height is greater than 8. For example, as shown in Table 5, if the coding block is 4x4, candModeList[0] is set to 17, candModeList[1] is set to 34, and candModeList[2] is set to 5, and if the coding block is 8x4 to 4x8, candModeList[0] is 0, candModeList[1] is 7, candModeList[2] is set to 16, in other cases, candModeList[0] is set to 1, candModeList[1] is set to 4, and candModeList[2] is set to 6.

Table 5

Alternatively, one MIP MPM default mode as shown in Table 6 may be used regardless of the size of the coding block.

Table 6

When candMipModeA and candMipModeB are the same, or when any one of candMipModeA and candMipModeB is -1, the MPM candidate can be derived as shown in Equations (9) to (11) below.

candMipModeList[　0　] = (　candMipModeA　!=　-1　)　　? candMipModeA　　:　　candMipModeB (9)

candMipModeList[　1　] = mipMpmCand[　sizeId　][　1　] (10)

candMipModeList[　2　] = mipMpmCand[　sizeId　][　2　] (11)

If candMipModeList[0] has the same value as mipMpmCand[　sizeId　][　0　], candMipModeList[1] and candMipModeList[2] can be derived using equations (10) to (11). CandMipModeList[1] and candMipModeList[2] can be derived using (12) to (13).

candMipModeList[　1　] = mipMpmCand[　sizeId　][　0　] (12)

candMipModeList[　2　] = (　candMipModeList[　0　]　!=　mipMpmCand[　sizeId　][　1　]　)　　?

mipMpmCand[　sizeId　][　1　]　　:　　mipMpmCand[　sizeId　][　2　] (13)

The mipMpmCand of formulas (9) to (13) can also be derived from Tables 5 to 6. That is, a plurality of MIP MPM default modes or one MIP MPM default mode may be used.

Information indicating whether the MIP intra prediction mode is a default mode may be signaled through a bitstream. The information indicates whether the MIP intra prediction mode is a specific intra prediction mode or whether the MIP intra prediction mode is a default mode. The specific intra prediction mode may be a planar, DC, horizontal direction prediction mode, vertical direction prediction mode, diagonal direction prediction mode, or a directional mode having a predetermined index value. The default mode may include at least one intra prediction mode of planar, DC, vertical direction, horizontal direction, and diagonal direction.

For example, a flag intra_mip_not_planar_flag indicating whether the MIP intra prediction mode is a planar mode or a flag intra_mip_not_default_flag indicating whether the MIP intra prediction mode is a default mode may be signaled.

When intra_mip_not_default_flag indicates that the MIP intra prediction mode is a default mode, an index for specifying any one of the default modes may be further signaled.

Alternatively, the specific direction may be predefined by an encoder and a decoder. Alternatively, information specifying a specific direction may be signaled through a sequence, picture, or slice header. For example, when a specific direction N is determined through a sequence parameter header, an intra_mip_not_N_flag may be signaled at a block level. The intra_mip_not_N_flag indicates whether the intra prediction mode of the current block is N.

Table 7 shows an example in which a flag intra_mip_not_planar_flag indicating whether the matrix intra prediction mode is a planar mode is signaled.

If the intra_mip_not_planar_flag value is 1, it indicates that the matrix intra prediction mode is not the planar mode, and if the intra_mip_not_planar_flag value is 0, it indicates that the matrix intra prediction mode is the planar mode.

Table 7

As shown in Table 8, when the intra_mip_flag value is 1, the intra_mip_not_planar_flag may be signaled. When the MIP intra prediction mode is not planar (ie, when the intra_mip_not_planar_flag value is 1), the intra_mip_mpm_flag may be additionally signaled. When the value of Intra_mip_not_planar_flag is 1, the Planar mode is not included as an mpm candidate.

Table 8

In the prediction mode of the matrix-based intra prediction encoding method (hereinafter, matrix intra prediction mode), the intra prediction mode of the current block may be derived based only on the MPM mode. That is, an intra prediction mode that is not set as an MPM candidate cannot be used as an intra prediction mode of the current block.

Specifically, for example, as shown in Table 9, when the intra_mip_flag value is 1, the MIP MPM index may be signaled.

Alternatively, after determining whether the intra prediction mode of the current block is a specific mode, the MIP MPM index may be signaled only when the intra prediction mode of the current block is not a specific mode. In this case, only a specific mode and intra prediction modes set as MPM candidates may be used as intra prediction modes of the current block.

Table 9

2.6 Intra prediction mode signaling method

A prediction image of the coding block may be generated, and a residual signal obtained by differentiating the prediction image of the previous row from the prediction image of the current row or the prediction image of the previous column from the prediction image of the current column may be generated. Transform skip coefficients may be transmitted through a bitstream. This method is defined as intra bdpcm.

When intra bdpcm is used, it may be restricted to use a predefined intra prediction method. For example, it may be limited to use only the horizontal prediction mode or the vertical prediction mode.

When intra bdpcm is used, it may be restricted to use transform skip.

If the intra_bdpcm_flag value is 1, it indicates that intra bdpcm is performed in the coding block.

If the intra_bdpcm_dir_flag value is 0, it indicates that the BDPCM prediction direction is horizontal. Here, that the BDPCM prediction direction is horizontal indicates that a residual signal is generated by differentiating the prediction image of the previous row from the prediction image of the current row.

If the intra_bdpcm_dir_flag value is 1, it indicates that the BDPCM prediction direction is vertical. Here, that the BDPCM prediction direction is vertical indicates that a residual signal is generated by differentiating the prediction image of the previous column from the prediction image of the current row.

Regular intra mode can be defined. The regular intra mode indicates an intra prediction mode in which at least one of whether Matrix-based intra prediction is used, a reference sample line index, sub-partition intra prediction is used, intra bdpcm is used, or an intra prediction mode is set as a default value. In the case of regular intra mode, information indicating whether matrix-based intra prediction is used, information indicating a reference sample line index, a flag indicating whether sub-partition intra prediction is used, information for determining whether to use intra bdpcm or an intra prediction mode At least one of the encoding/decoding may be omitted.

For example, regular intra may represent a matrix-based intra prediction, a multi-line intra and sub-partition intra, and an intra prediction method that does not use intra bdpcm. That is, in regular intra, it refers to an intra prediction method based on one of a maximum of 67 intra prediction modes and reference samples adjacent to the current block.

As shown in Table 10, a flag regular_intra_flag indicating whether regular intra has been used in a coding block may be signaled. If the value of regular_intra_flag is 1, it indicates that regular intra is used in the coding block, and if the value of regular_intra_flag is 0, it indicates that regular intra is not used in the coding block.

When the regular_intra_flag value is 0, the intra_mip_flag, intra_luma_ref_idx, or intra_subpartitions_mode_flag, etc. may be signaled.

Table 10

Whether to allow the Regular_intra_flag may be determined based on the size/type of the current block. For example, only when the current block is a square and/or when the size of the current block is less than or equal to a threshold value, regular_intra_flag may be signaled.

2.6 Inter prediction coding and decoding method

A method of generating a prediction block (prediction image) of a block in a current picture using information of a previous picture is called an inter prediction encoding mode.

A prediction block may be generated from a specific block in a previous picture of the current block.

In the encoding step, the block with the smallest reconstruction error among the blocks in the previous picture can be selected by searching around a collocated block, and the x-axis difference and y-axis difference between the upper left sample of the current block and the upper left sample of the selected block Is defined as a motion vector, and can be signaled by transmitting it to a bitstream. A block generated through interpolation from a specific block of a reference picture specified by a motion vector is called a motion compensated predictor block.

As shown in Figure 22 , the collocated block represents the block of the picture with the same location and size of the current block and the upper left sample. The corresponding picture may be specified from the same syntax as the reference picture reference.

Figure 22

In the inter prediction encoding mode, a prediction block may be generated in consideration of the motion of an object.

For example, if you know how and in which direction an object in the previous picture has moved in the current picture, a prediction block (prediction image) can be generated by differentiating a block considering motion from the current block, and this is defined as a motion prediction block. .

A residual block may be generated by differentiating a motion prediction block or a corresponding prediction block from the current block.

When motion occurs in an object, if a motion prediction block is used rather than a corresponding prediction block, the energy of the residual block decreases, and compression performance may be improved.

This method of using a motion prediction block is called motion compensation prediction, and most inter prediction coding uses motion compensation prediction.

A value indicating in which direction and to what extent the object in the previous picture has moved in the current picture is called a motion vector. As the motion vector, motion vectors having different pixel precisions may be used in units of sequences, groups of tiles, or blocks. For example, the pixel precision of a motion vector in a specific block may be at least one of octor-pel, quarter-pel, half-pel, integer pel, and 4 integer pel.

As the inter prediction mode, an inter prediction method using translaton motion and an affine inter prediction method using affine motion may be selectively used.

2.6.1 Afine inter prediction coding and decoding method

There are many cases in which the motion of a specific object does not appear linearly in a video. For example, as shown in Figure 23, camera zoom (Zoom-in), zoom out (Zoom-out), rotation (roation), such as affine transformation, which enables the conversion to any type of affine motion of the image using the object If only translation motion vectors are used for motion, the motion of an object cannot be effectively expressed, and coding performance may be degraded.

Figure 23

Affine motion can be expressed as the following equation (14).

(14)

(14)

Expressing affine motion using a total of six parameters is effective for images with complex motion, but encoding efficiency may be degraded due to the large number of bits used to encode affine motion parameters.

Accordingly, affine motion can be simplified and expressed with four parameters, and this is defined as a four-parameter afine motion model. Equation (15) expresses affine motion with four parameters.

(15)

(15)

The four-parameter affine motion model may include motion vectors at two control points of the current block. The control point may include an upper left corner, an upper right corner, and a lower left corner of the current block. For example, 4-parameter affine motion model to the motion vector sv1 in the upper left sample of the coding unit as shown in the left figure of Figure 2. 4 (x0, y0) top right samples of the motion vector sv0 the coding unit in the (x1, y1) It can be determined by, and sv ₀ and sv ₁ are defined as affine seed vectors. Hereinafter, it is assumed that an affine seed vector sv ₀ located in the upper left corner is a first affine seed vector, and an affine seed vector sv ₁ located in the upper right corner is a second affine seed vector. In the 4-parameter affine motion model, it is possible to replace one of the 1/2 affine seed vectors with the affine seed vector located in the lower left corner.

Figure 24

The 6-parameter affine motion model is an affine motion model in which the motion vector sv ₂ of the residual control point (e.g., sample (x2,y2) in the lower left corner) is added to the 4-parameter affine motion model as shown in the right figure of Figure 24 . . Hereinafter, assuming that the affine seed vector sv ₀ located at the top left is the first affine seed vector, the affine seed vector sv ₁ located at the top right is the second affine seed vector, and the affine seed vector located at the bottom left Assume that sv ₂ is the third affine seed vector.

Information on the number of parameters for expressing affine motion may be encoded in the bitstream. For example, a flag indicating whether 6 parameters are used and a flag indicating whether 4 parameters are used may be encoded in a tile group, a tile, a coding unit, or a CTU unit. Accordingly, a 4-parameter affine motion model to a 6-parameter affine motion model may be selectively used in units of tile groups, coding units, or CTUs.

A motion vector can be derived for each sub-block of the coding unit using the affine seed vector as shown in Figure 25, and this is defined as an affine sub-block vector. _

Figure 25

The affine sub-block vector can also be derived as in Equation (16) below. Here, the reference sample position (x,y) of the sub-block may be a sample located at a corner of the block (eg, an upper left sample), or a sample (eg, a center sample) in which at least one of the x-axis or y-axis is centered. .

(16)

(16)

Motion compensation may be performed in units of coding units or sub-blocks within the coding unit using the afine sub-block vector, and this is defined as an afine inter prediction mode. In Equation (16), (x ₁ -x ₀ ) may have the same value as the width of the coding unit.

2.6.2 Merge mode coding method

Motion information (motion vector, reference picture index, etc.) of the current coding unit is not encoded, and can be derived from motion information of neighboring blocks. Motion information of any one of the neighboring blocks can be set as motion information of the current coding unit, and this is defined as a merge mode.

Neighboring blocks used for the remaining mode may be a picture 26 remaining candidate indices from 0 to 4 and the block are adjacent to the coding unit as the (current boundary with the abutting block of the coded unit), as shown in Figure 26, the remaining candidate index from 5 to 26 of the It may be a non-adjacent block.

Fig. 26

If the distance of the merge candidate to the current block exceeds a predefined threshold, it may be set as not available.

For example, a predefined threshold may be set to the height of the CTU (ctu_height) or ctu_height+N, and this is defined as a merge candidate available threshold. That is, the y-axis coordinate (y _i ) of the merge candidate and the y-axis coordinate difference (y ₀ ) (i.e., y _i- y ₀ ) of the upper left sample of the current coding unit (hereinafter, the reference sample of the current coding unit) are the merge candidate available threshold If it is larger than the value, the merge candidate may be set as not available. Here, N is a predefined offset value. Specifically, for example, N may be set to 16 or ctu_height may be set.

Fig. 27

When there are many merge candidates crossing the CTU boundary, many unusable merge candidates occur, and encoding efficiency may be lowered. Merge candidates existing above the coding unit (hereinafter, upper merge candidates) may be set as small as possible, and merge candidates existing at the left and lower sides of the coding unit (hereinafter, lower left merge candidates) may be set as many as possible.

As shown in Fig. 27, the difference between the y-axis coordinate of the current coding unit reference sample and the y-axis coordinate of the upper merge candidate may not exceed twice the height of the coding unit.

As shown in Figure 27, the difference between the x-axis coordinate of the current coding unit reference sample and the x-axis coordinate of the lower-left merge candidate may not exceed twice the width of the coding unit.

A merge candidate adjacent to the current coding unit is called an adjacent merge candidate, and a merge candidate that is not adjacent to the current coding unit is defined as a non-adjacent merge candidate.

A flag isAdjacentMergeflag indicating whether a merge candidate of the current coding unit is an adjacent merge candidate may be signaled.

If the isAdjacentMergeflag value is 1, it indicates that motion information of the current coding unit can be derived from an adjacent merge candidate, and if the isAdjacentMergeflag value is 0, it indicates that the motion information of the current coding unit can be derived from a non-adjacent merge candidate.

2.6.3 Inter decoding area merge method

Motion information (motion vector and reference picture index) of a coding unit already encoded by inter prediction in the current picture can be stored in a list having a predefined size, and this is defined as an inter-region motion information list.

Motion information (motion vector and reference picture index) in the inter-region motion information list is called an inter-decoding region merge candidate.

The inter decoding region merge candidate may be used as a merge candidate of the current coding unit, and this method is defined as an inter decoding region merge method.

When the tile group is initialized, the inter-region motion information list is empty, and when a partial region of a picture is encoded/decoded, it can be added to the inter-region motion information list. The initial inter decoding region merge candidate of the inter region motion information list may be signaled through the tile group header.

When the coding unit is encoded/decoded by inter prediction, motion information of the coding unit may be updated in the inter-region motion information list as shown in Fig. 28 . If the number of inter-decoding region merge candidates in the inter-region motion information list is the maximum value, the value with the smallest inter-region motion information list index (first, the inter-decoding region merge candidate in the inter-region motion information list) is removed. , The motion vector of the most recently encoded/decoded inter-region may be added as an inter-decoding region merge candidate.

Fig. 28

The motion information mvCand of the decoded coding unit may be updated in the inter-region motion information list HmvpCandList. At this time, if the motion information of the decoded coding unit is the same as any one of the motion information in the inter-region motion information list (when both the motion vector and the reference index are the same), the inter-region motion information list is not updated, or as shown in Figure 30 . The motion vector mvCand of the coding unit decoded together may be stored at the end of the inter-region motion information list. At this time, if the index of HmvpCandList that has the same motion information as mvCand is hIdx, HMVPCandList[i] can be set to HVMPCandList[i-1] for all i larger than hIdx as shown in Figure 30 . When sub-block merge candidates are used in the currently decoded coding unit, motion information of the representative sub-block in the coding unit may be stored in the inter-region motion information list.

As an example, the representative sub-block in the coding unit may be set as an upper left sub-block in the coding unit or as a middle sub-block in the coding unit as shown in FIG. 29 .

Fig. 29

The sub-block unit merge candidate can be derived as follows.

1. An initial shift vector (shVector) can be derived from a motion vector of a merge candidate block adjacent to the current block.

2. As shown in Equation (17), an initial shift vector can be added to the upper left sample (xSb,ySb) of the subblock in the coding unit to derive the shifted subblock having the upper left sample position (xColSb, yColSb).

(xColSb, yColSb) = (xSb + shVector[0]>> 4,ySb+shVector[1]>> 4) (17)

3. The motion vector of the collocated block corresponding to the center position of the sub-block containing (xColSb, yColSb) can be derived as the motion vector of the sub-block containing the upper left sample (xSb,ySb).

Fig. 30

A total of NumHmvp motion information (motion vector and reference picture index) can be stored in the inter-region motion information list, and NumHmvp is defined as the size of the inter-region motion information list.

The size of the inter-region motion information list may use a predefined value. The inter-region motion information list size may be signaled in the tile group header. For example, the size of the inter-region motion information list may be defined as 16 or 6.

In a coding unit that is inter prediction and has an affine motion vector, it may be limited not to have an inter-region motion information list.

Alternatively, in the case of inter prediction and having an affine motion vector, the affine sub-block vector may be added to the inter-region motion information list. In this case, the position of the sub-block may be set to the upper left or upper right, or the center sub-block.

Alternatively, the average motion vector value of each control point may be added to the inter-region merge candidate list.

When the motion vector MV0 derived by encoding/decoding a specific coding unit is the same as any one of the inter-decoding region merge candidates, MV0 may not be added to the inter-region motion information list. Alternatively, an existing inter-decoding region merge candidate having the same motion vector as MV0 may be deleted, and the index allocated to MV0 may be updated by newly including MV0 in the decoding region merge candidate.

In addition to the inter-region motion information list, an inter-region motion information long term list HmvpLTList may be configured. The inter-region motion information long term list size may be set to be the same as the inter-region motion information list size, or may be set to a different value.

The inter-region motion information long term list may be composed of an inter-decoding region merge candidate initially added to the tile group start position. After the inter-region motion information long term list is configured with all available values, the inter-region motion information list may be configured, or motion information in the inter-region motion information list may be set as motion information of the inter-region motion information long term list.

In this case, the inter-region motion information long-term list configured once may be updated again when no update is performed, a decoded region of the tile group is more than half of the entire tile group, or may be set to be updated every m CTU lines. The inter-region motion information list may be updated whenever the inter-region is decoded, or may be set to be updated in units of CTU lines.

The motion information and partition information or type of the coding unit may be stored in the inter-region motion information list. The inter-decoding region merging method may be performed using only inter-region merge candidates having a similar partition information and shape as the current coding unit.

Alternatively, an inter-region merge candidate list may be individually configured according to the block type. In this case, one of a plurality of inter-region merge candidate lists may be selected and used according to the shape of the current block.

Fig. 31

As shown in Figure 31 , it can be composed of an inter-area affine motion information list and an inter-area motion information list. When the decoded coding unit is in affine inter or afine merge mode, the first affine seed vector and the second affine seed vector may be stored in the inter-region afine motion information list HmvpAfCandList. Motion information in the inter-region affine merge candidate list is called an inter-decoding region affine merge candidate.

The merge candidates usable in the current coding unit may be configured as follows, and may have the same search order as the configuration order.

1. Spatial merge candidate (coding block adjacent merge candidate and coding block non-adjacent merge candidate)

2. Temporal merge candidate (merge candidate derived from previous reference picture)

3. Inter-decoding domain merge candidate

4. Inter decoding domain affine merge candidate

5. Zero motion merge candidate

First, the merge candidate list mergeCandList may be composed of a spatial merge candidate and a temporal merge candidate. The number of available spatial merge candidates and temporal merge candidates is defined as the number of available merge candidates (NumMergeCand). When the number of available merge candidates is less than the maximum allowable merge number, an inter-decoding region merge candidate may be added to the merge candidate list mergeCandList.

When adding the inter-region motion information list HmvpCandList to the merge candidate list mergeCandList, it may be checked whether motion information of the inter-decoding region merge candidate in the inter-region motion information list is the same as the motion information of the existing merge list mergeCandList. If the motion information is the same, it is not added to the merge list mergeCandList, and if the motion information is not the same, the inter-decoding region merge candidate may be added to the merge list mergeCandList.

Among the merge candidates of the inter-decoding area, a large index may be added to the merge candidate list mergeCandList, or a small index may be added to the merge candidate list mergeCandList.

The inter decoding region motion candidate can be used as a motion vector predictor (MVP) candidate of the current coding unit, and this method is defined as an inter decoding region motion information prediction method.

An inter-decoding region affine motion candidate can be used as a motion vector predictor (MVP) candidate of the current coding unit, and this method is defined as an inter-decoding region motion information affine prediction method.

Motion information predictor candidates usable in the current coding unit may be configured as follows, and may have the same search order as the configuration order.

1. Spatial motion predictor candidate (same as the coding block adjacent merge candidate and the coding block non-adjacent merge candidate)

2. Temporal merge candidate (motion predictor candidate derived from previous reference picture)

3. Inter-decoding domain merge candidate

4. Inter decoding domain affine merge candidate

5. Zero motion motion predictor candidate

2.7 Triangular prediction unit coding method

The coding unit can be divided into a plurality of prediction units. Specifically, the coding unit may be encoded as a plurality of prediction units using at least one line such as a vertical line, a horizontal line, or a diagonal line. Information on at least one of the number and angle of the number of lines dividing the coding unit may be signaled through the bitstream. Alternatively, at least one of the number of lines and angles for dividing the coding unit may be adaptively determined according to the intra prediction mode, the inter prediction mode, and the position of an available merge candidate of the coding unit. The coding unit may be divided into a plurality of prediction units, and intra prediction or inter prediction may be performed in units of the divided prediction units.

Fig. 4 shows an example of dividing a coding unit into two prediction units using a diagonal line. Dividing a coding unit into two prediction units using a diagonal line may be defined as diagonal partitioning. In Figure 4, it is shown that the coding unit is divided into two triangular shape prediction units having the same size. Unlike in the illustrated example, the coding unit may be divided into two prediction units having different sizes. For example, the upper left corner of the diagonal that divides the coding unit is set to meet the left boundary or the upper boundary of the coding unit other than the upper left corner of the coding block, or It can be set to cross the lower boundary.

The left figure in Figure 32 is called the left triangular partitioning, and the right figure in Figure 32 is called the right triangular partitioning. The prediction unit to which the upper left or lower left sample of the coding unit belongs is called a left triangular prediction unit, the prediction unit to which the upper right or lower right sample of the coding unit belongs is called a right triangular prediction unit, and the right triangular prediction unit or left triangular prediction unit is called It is collectively called a triangular prediction unit.

Fig. 32

In hardware implementation, when the motion type of the area is larger than that of the 64x64 data unit, there is a disadvantage that the 64x64 data unit must be accessed repeatedly. Accordingly, when at least one of the width or height of the coding unit is greater than 64 (eg, when at least one of the width or height is 128), the use of the triangular prediction unit may be restricted. Specifically, for example, in a 128xN coding unit or an Nx128 coding unit, as shown in Figure 33 , it may be restricted not to use a triangular prediction unit. Here, N represents a value less than or equal to 64.

Fig. 33

Alternatively, when at least one of the width or height of the coding block is greater than the threshold value, diagonal partitioning may not be allowed. Here, the threshold value may be a value predefined in the encoder/decoder, or information about the threshold value may be signaled through a bitstream.

Alternatively, it may be determined whether to allow diagonal partitioning according to the size of the Merge Estimation Region and the size of the coding block. For example, when the coding block is larger than the parallel processing region, diagonal partitioning may not be allowed.

Alternatively, it may be determined whether to allow diagonal partitioning according to the number of samples included in the coding unit. For example, when the number of samples included in the coding unit is M or less, or when the number of samples included in the coding unit is N or more, diagonal partitioning may not be allowed.

Depending on the type of coding unit, it is possible to determine whether to allow diagonal partitioning. Specifically, it may be limited to use the diagonal prediction unit coding method only when the width of the coding unit is a high ratio of the height of the coding unit or the coding unit type ratio (whRatio) is within a specific range. Here, the coding unit type ratio can be defined as (coding unit width (cbWsize): coding unit height ratio (cbHSize)) as shown in the following equation (18).

whRatio_CU = cbWSize/cbHSize (18)

The diagonal prediction unit encoding method may be limited to be used only when whRatio_CU is smaller than k or larger than 1/k. Specifically, for example, when the k value is set to 16, it may be limited not to use diagonal prediction unit coding in a coding unit of 64x4 size or 4x64.

Whether to allow diagonal partitioning may be determined according to a method of dividing a parent node of a coding unit that is a leaf node. For example, when the parent node is QT-divided, diagonal partitioning may be allowed for a coding unit that is a leaf node, while when the parent node is BT/TT-divided, diagonal partitioning may not be allowed for a coding unit that is a leaf node.

Alternatively, the triangular prediction unit may be used only in the square coding unit, and the triangular prediction unit may not be used in the amorphous coding unit.

Alternatively, information indicating whether diagonal partitioning is allowed in units of coding tree units, tiles, tile sets (tile groups), pictures or sequences may be signaled.

Or, when the coding unit is coded by intra prediction, when the coding unit is coded by inter prediction, or when the coding unit is a specific inter prediction mode (e.g., any one of merge mode, AMVP mode, ATMVP mode, or afine mode) Diagonal partitioning may be allowed only in the case of coded as.

Information indicating whether to apply diagonal partitioning to the coding unit may be signaled through a bitstream. For example, a triangle partition flag (merge_triangle_flag), which is a syntax indicating whether diagonal partitioning is applied to the coding unit, may be signaled in units of CU.

In addition, information indicating the direction of diagonal partitioning may be signaled through a bitstream. For example, a triangle partition type flag (merge_triangle_split_dir), a syntax indicating whether the left triangular partitioning is used or the right triangular partitioning is used in units of CU, may be signaled in units of CU. If the triangle_partion_type_flag value is 0, it indicates the left triangular partition, and if the merge_triangle_split_dir value is 1, it indicates the right triangular partition. Alternatively, information indicating the direction of diagonal partitioning may be signaled in units of coding tree units, in units of tiles, or in units of tile sets (tile groups). In this case, among the coding units included in the level at which the information is signaled, the coding units to which the diagonal partitioning is applied may have the same partition shape.

As another example, the triangular partitioning type of the coding unit may be determined based on the triangular partitioning type of a neighboring coding unit adjacent to the coding unit. For example, the triangular partitioning type of the coding unit may be determined to be the same as the triangular partitioning type of the neighboring coding unit. Here, the neighboring coding unit may include a neighboring block positioned adjacent to the diagonal direction of the current coding unit, and a neighboring block positioned adjacent to the top or left of the current coding unit.

Information indicating the direction of diagonal partitioning is signaled to the coding unit to which the first triangular partitioning is applied in the CTU, while the same diagonal partitioning direction as the first coding unit may be applied to the second and subsequent coding units to which the triangular partitioning is applied.

The left triangular prediction unit and the right triangular prediction unit may each have a motion vector and a reference picture index. The motion vector of the triangular prediction unit can be derived from a triangular merge candidate list consisting of a spatial merge candidate list and a temporal merge candidate list as shown in Figure 34 . It can have a maximum of M triangular merge candidates, and M is called a triangular merge candidate maximum value.

Triangular prediction units may use different merge candidate lists. As an example, the merge candidate list of the right triangular prediction unit may be constructed using the remaining merge candidates excluding the merge candidate indicated by the merge candidate index of the left triangular prediction unit among merge candidates of the left triangular prediction unit. The triangular merge candidate maximum values of the left triangular prediction unit and the right triangular prediction unit may be differently set. As an example, the left triangular prediction unit may have M triangular merge candidates, while the right triangular prediction unit may have M-1 triangular merge candidates excluding merge candidates indicated by the merge candidate index of the left triangular prediction unit.

As another example, triangular prediction units may share one merge candidate list.

The triangular merge candidates may be added to the merge candidate list in a predefined order. As an example, the following order among the merge candidates in Figure 34 may be used.

0 --> 1 --> 2 --> 3 --> 7 --> 4 --> 6

The candidate block selected from the triangular merge candidate list may be represented by a triangular merge candidate index (merge_triangle_idx). The triangular merge candidate index may consist of a left triangular merge candidate index (merge_triangle_idx0) and a right triangular merge candidate index (merge_triangle_idx1).

Fig. 34

Only one of the left triangular merge candidate index or the right triangular merge candidate index may be signaled, and the other triangular merge candidate index may be derived from the signaled merge candidate index.

As an example, only a left triangular merge candidate index may be signaled, and a left triangular merge candidate index and a neighboring merge candidate index may be derived as a right triangular merge candidate index.

Specifically, for example, if the left triangular merge candidate index is N, the right triangular merge candidate index may be derived as N+1. Only the left triangular merge candidate index can be encoded, the right triangular merge candidate index is not encoded, and can be derived from the left triangular merge candidate index.

When the left triangular merge candidate index N is the triangular merge candidate maximum value, the right triangular merge candidate index may be derived as N-1 or 0.

Alternatively, motion information of the right triangular prediction unit may be derived from a merge candidate having the same reference picture as the merge candidate indicated by the left triangular merge candidate index. Here, the same reference picture may represent a merge candidate in which at least one of the L0 reference picture and the L1 reference picture is the same. When there are a plurality of merge candidates having the same reference picture, either one may be selected based on whether bi-directional prediction or an index difference value with the merge candidate indicated by the left triangular merge candidate index.

The left triangular merge candidate index (merge_triangle_idx0) and the right triangular merge candidate index (merge_triangle_idx1) may be signaled, respectively. The motion information of the left triangular prediction unit is derived from the merge candidate signaled by the first merge candidate index and is derived from the right triangular merge candidate index. The motion information of the triangular prediction unit may be derived from the merge candidate signaled by the second merge candidate index.

In this case, the merge candidate indicated by the first merge candidate index of the left triangular prediction unit may be set not to be usable as a merge candidate of the right triangular prediction unit. Accordingly, the second merge candidate index may indicate any one of the remaining merge candidates excluding the merge candidate indicated by the first merge candidate index. As an example, when the value of the second merge candidate index is smaller than the value of the first merge candidate index, the motion information of the right triangular prediction unit is a merge having the same index as the second merge candidate index among merge candidates included in the merge candidate list. Can be derived from candidates. On the other hand, when the value of the second merge candidate index is the same as the value of the first merge candidate index or is greater than the first merge candidate index, the motion information of the right triangular prediction unit is the second merge candidate among merge candidates included in the merge candidate list. It can be derived from a merge candidate having an index of 1 greater than the candidate index.

2.8 Chroma sample sharing unit

Whether to allow partitioning of the coding block may be determined based on at least one of the number of chroma samples or the size/type of the chroma component coding block. Here, the size of the coding block may mean at least one of a width of a coding block, a height of a coding block, or a value calculated based on a product of a width and a height of the coding block.

As an example, a partitioning form of a coding block in which the number of chroma samples is smaller than a specific threshold (hereinafter, a chroma-allowable sample threshold) may not be allowed. Alternatively, a division type of a coding block in which the width or height of the chroma coding block is smaller than a specific threshold value may not be allowed.

Alternatively, when the partition types of the luma component and the chroma component are different (ie, when a separate tree is used), the partitioning form of the coding block in which the number of chroma samples is smaller than the chroma-permissible sample threshold may not be allowed.

Alternatively, whether to allow division of the chroma coding block may be determined based on the prediction mode and the chroma allowed sample threshold. As an example, in the intra prediction mode, a partitioning form of a coding block in which the number of chroma samples is smaller than a chroma-allowable sample threshold value may not be allowed. On the other hand, in the inter prediction mode, a partitioning form of a coding block in which the number of chroma samples is smaller than a chroma-allowable sample threshold may be allowed.

For example, when the chroma-allowable sample threshold is 16, the number of chroma samples may be greater than or equal to 16 to generate a chroma coding block. A coding block in which the number of chroma samples is the same as the chroma-permitted sample threshold may be defined as a chroma minimum unit coding block.

Additional partitioning may be allowed for the luma block corresponding to the chroma minimum unit coding block. Fig. 31 shows the partitioning pattern of luma block and chroma block. The thin solid line shows the division pattern of the luma block, and the thick solid line shows the division pattern of the chroma block. If the chroma-tolerant sample threshold is 16, a chroma coding block can be generated to include at least 16 samples as shown in Figure 35. As shown in Figure 35, the luma coding block corresponding to the chroma coding block can be further divided.

Fig. 35

Whether to apply the division of the luma coding block to the division of the chroma coding block may be determined based on whether the number of at least one chroma sample among the partitions generated as a result of the division of the luma coding block is less than the chroma-allowable sample threshold. As an example, assuming that the luma coding block corresponding to 32 chroma samples in Figure 35 is triple-tree partitioned, if the triple-tree partitioning is applied to the chroma coding block as well, a 2x4 type partition generated as a result of triple-tree partitioning Are included in the number of chroma samples smaller than the chroma-tolerant sample threshold. Accordingly, the triple tree division applied to the luma coding block may not be applied to the chroma coding block.

The chroma-allowable sample threshold may signal a syntax smallest_chroma_unit_minus4 indicating a chroma-allowable sample threshold at at least one of a picture parameter header, a slice header, or a CTU (Coding Tree Unit) unit through a bitstream. In this case, the chroma-allowable sample threshold may be set to 1 << (smallest_chroma_unit_minus4 +4).

For example, when the chroma-allowed sample threshold is 32, smallest_chroma_unit_minus4 may be set to 1.

Alternatively, a chroma acceptable sample threshold may be determined based on the color format.

In the chroma minimum unit coding block, at least one of a sub-partition intra prediction method (ISP), a position-based prediction sample correction method (PDPC), or a plurality of reference sample lines may be set not to be used.

3 Transform and quantization

An image obtained by subtracting the predicted image from the original image is called a residual image.

The residual image can be decomposed into binary frequency components through a two-dimensional transform such as a discrete cosine transform (DCT). Even if high-frequency components are removed from an image, there is a characteristic that significant distortion does not occur visually. If the value corresponding to the high frequency is decreased or set to 0, the compression efficiency can be increased while the visual distortion is not large.

Discrete sine transform (DST) may be used according to the size of the prediction block or the prediction mode. Specifically, for example, when the prediction block/coding block is in an intra prediction mode and the size of the prediction block/coding block is smaller than NxN, the DST transform may be used, and other prediction blocks/coding blocks may be set to use DCT.

DCT is a process that decomposes (transforms) an image into 2D frequency components using a cosine transform, and the frequency components at that time are expressed as a base image. For example, if DCT transformation is performed in an NxN block, N ² basic pattern components can be obtained. Performing DCT transformation is to obtain the size of each of the basic pattern components included in the original pixel block. The size of each basic pattern component is defined as a DCT coefficient.

In general, a discrete cosine transform (DCT) is mainly used for an image in which many non-zero components are distributed in a low frequency, and a Discrete Sine Transform (DST) may be used for an image in which a large number of high-frequency components are distributed.

DST represents a process of decomposing (converting) an image into 2D frequency components using sin transformation. A 2D image can be decomposed (transformed) into 2D frequency components using a transformation method other than DCT or DST transformation, and this is defined as 2D image transformation.

Among the residual images, a 2D image transformation may not be performed in a specific block, and this is defined as transform skip. Quantization can be applied after the transform skip. Whether to allow the transform skip may be determined based on at least one of the size or shape of the coding unit. As an example, the transform skip may be limited to be used only in coding units of a specific size or less. For example, it is possible to set the transform skip to be used only in blocks smaller than 32x32. Alternatively, it is also possible to restrict the use of transform skip to only a square block. Specifically, for example, transformation skip may be applied in units of 32x32, 16x16, 8x8, and 4x4 blocks.

DCT, DST, or 2D image transformation may be applied to an arbitrary block in the 2D image, and the transformation used in this case is defined as a first transformation. After performing the first transformation, transformation may be performed again in a partial region of the transformation block, and this is defined as a second transformation.

The first transform may use one of a plurality of transform cores. Specifically, for example, one of DCT2, DCT8, or DST7 may be selected and used in the transform block. Alternatively, the transform cores of the horizontal direction transformation and the vertical direction transformation of the transformation block may be individually determined. Accordingly, the horizontal direction transformation and the vertical direction transformation may be the same or different. Different transform cores may be used in horizontal and vertical transforms, and this is defined as a multiple transform core encoding method (Multiple Transform Selection, MTS).

Block units for performing the first transformation and the second transformation may be set differently. Specifically, for example, after performing the first transform in an 8x8 block of the residual image, the second transform may be performed for each 4x4 sub-block. For another example, after performing the first transformation in each 4x4 block, the second transformation may be performed in each 8x8 size block.

The residual image to which the first transform is applied is defined as a first transform residual image.

DCT, DST, or 2D image transformation may be applied to the first transform residual image, and the transform used in this case is defined as a second transform. The 2D image to which the second transformation is applied is defined as a second transformation residual image.

A sample value in the block after performing the first transformation and/or the second transformation or the sample value in the block after performing the transformation skip is defined as a transform coefficient. Quantization refers to the process of dividing a transform coefficient by a predefined value to reduce the energy of a block. A value defined to apply quantization to a transform coefficient is defined as a quantization parameter.

A predefined quantization parameter may be applied in units of sequences or blocks. Typically, a quantization parameter can be defined with a value between 1 and 64.

After performing transformation and quantization, inverse quantization and inverse transformation may be performed to generate a residual reconstructed image. A first reconstructed image may be generated by adding a prediction image to the residual reconstructed image.

3.1 Sub transform block coding method

Fig. 36

Fig. 37

Transformation may be performed on only a partial region within the coding unit as shown in Figs. 36 to 37, and this is defined as a sub-transform block coding method. A sub-transform block flag cu_sbt_flag indicating whether sub-transformation block coding is used may be transmitted through a bitstream. If cu_sbt_flag is 1, it indicates that the sub transform block encoding method is used.

The width of the coding unit/coding block is less than or equal to a predefined value (hereinafter, the maximum width of the sub-transformation block), and the height of the coding unit/coding block is less than the predefined value (hereinafter, the maximum height of the sub-transformation block), or When they are the same, the sub-transform block encoding method may be used.

For example, when sub-transform block coding is used, the coding unit can be divided into a horizontal direction binary partition or a vertical direction binary partition as shown in Figure 36. A binary partition on the left or upper side is referred to as a first sub-transformed binary partition, and a binary partition on the right or lower side is referred to as a second sub-transformed binary partition. Transformation and quantization may be performed only in one of the first sub-transform binary partition and the second sub-transform binary partition (hereinafter, referred to as transform binary partition), and this is referred to as a transform binary partition encoding method.

A flag cu_sbt_horizontal_flag indicating whether the coding unit is divided into a horizontal direction binary partition or a vertical direction binary partition may be transmitted through a bitstream.

If the cu_sbt_horizontal_flag value is 1, it indicates that it is divided into horizontal binary partitions, and if the cu_sbt_horizontal_flag value is 0, it indicates that it is divided into vertical binary partitions.

For another example, as shown in Figure 37, the coding unit can be divided into four quaternary partitions, which is called a quaternary transform partition coding method. A flag cu_sbt_quad_flag indicating whether the quaternary transform partition encoding method is used may be signaled through a bitstream.

The upper left quaternary partition is the first sub-transformation quaternary partition, the upper right quaternary partition is the second sub-transformation quaternary partition, the lower left quaternary partition is the third sub-transformation quaternary partition, and the lower right The quaternary partition in is referred to as the fourth sub-transformation quaternary partition.

Transformation and quantization may be performed only in one of the first to fourth sub-quaternary transform partitions (hereinafter, transformed quaternary partition).

A flag cu_sbt_pos_flag for specifying a transform quaternary partition or a transform binary partition may be signaled through a bitstream.

For example, if the quaternary transform coding method is used and the value of cu_sbt_pos_flag is 0, the transform quaternary partition may be set as the first sub transform quaternary partition.

As another example, if the quaternary transform partition encoding method is used and the cu_sbt_pos_flag value is 2, the transform quaternary partition may be set as the third sub transform quaternary partition.

For another example, if the transform binary partition encoding method is used and the cu_sbt_pos_flag value is 1, the transform binary partition may be set as the first sub transform binary partition.

Vertical direction conversion binary partitions can be allowed only when the width of the coding unit is greater than or equal to a specific value (hereinafter, referred to as vertical binary threshold), and if the height of the coding unit is greater than a specific value (hereinafter, referred to as the horizontal binary threshold) Only horizontal translation binary partitions can be allowed.

The quaternary transform partition encoding method may be used only when the width and height of the coding unit are greater than or equal to a specific value (hereinafter, referred to as the quaternary threshold), and cu_sbt_quad_flag may be transmitted through a bitstream.

For example, if the quaternary threshold is 16, the width of the coding unit is 16, and the height is 8, since the height of the coding unit is less than the quaternary threshold, the quaternary transform partition coding method is not used. , As shown in Table 11 , cu_sbt_quad_flag may not be signaled.

Table 11

3.2 Reduced secondary transform coding method

A secondary transform may be performed on a sub-region of a predefined size/shape at the upper left of the first transform block generated by performing the first transform in the NxN transform block. The size/shape of the sub-region may be a 4x4 or 8x8 square shape, or a 4x8 or 8x4 amorphous shape. The size/shape of the sub-region may be determined according to the size/shape of the first transform block. As an example, the size/shape of the sub-region may be a size/shape corresponding to any one partition when the first transform block is equally divided into four. Secondary transform can also be applied in a non-separable form, and this is defined as a non-separable secondary transform.

When performing a non-separable transform on a square sub-region NxN located at the upper left of the first transform block, the residual coefficients of the upper left NxN region may be rearranged into a matrix A having a size of N ² x1. By applying the N ² x N ² non-separable transform matrix T to the matrix A, a secondary transform matrix A'(N ² x 1 matrix) can be generated as shown in Equation (19).

A'= T*A (19)

The secondary transform matrix A'can be rearranged according to the size of the upper left NxN region.

Matrix A or partial matrix A'of matrix A (e.g., (N ² /2)x(N ² /2) or (N ² /2)x(N ² /2)) with kxl non-separable transform matrix T By applying it, a reduced secondary transform matrix A _R (kx1 matrix) can be generated as shown in Equation (20), and this is defined as a reduced secondary transform. k is N ² or It may be an integer less than N ² (eg, N ² /2, N ² /4 or N ² /4). l is N ² or It may be an integer less than N ² (eg, N ² /2, N ² /4 or N ² /4).

A _R = R*A (20)

The size of the non-separable transform matrix T or R may be determined based on at least one of the size/shape of the current block, a quantization parameter, or a transform scheme. Alternatively, information for specifying any one of a plurality of non-separable transform matrices may be signaled through a bitstream.

The (kx1) sized Reduced secondary transform matrix A _R may be rearranged into an upper left MxM block (M is an integer smaller than N) among the first transform blocks.

Figure 37 shows the flow chart of the Reduced secondary transform where N is 8 and k is 16. The reduced secondary transform matrix A _R of size (kx1) may be rearranged in the upper left 4x4 block among the first transform blocks.

If the size of the sub-region is smaller than the predefined value, it can be set not to apply the Reduced secondary transform. For example, in the upper left 4x4 area, it may be set not to apply the Reduced secondary transform.

Fig. 37

For example, as shown in Figure 38 , the second transformation may be performed using a 64x16 or 48x16 non-separable matrix.

Fig. 38

Alternatively, as shown in Figure 39 , the second transformation may be performed using a 32x16 non-separable matrix.

Fig. 39

Alternatively, you can limit the use of a 32x16 non-separable matrix only when using an ISP.

The ISP may be configured not to apply Secondary Transform or Reduced Secondary Transform (RST). Alternatively, it is possible to set to apply ST or RST only when the ISP is a 4xL or Lx4 sub-partition. Here, L can be set to an integer greater than 4.

In this case, a 32x16 RST matrix can be used as shown in Figure 39 .

When using a 4xL type sub-partition, a 32x16 RST matrix as shown in the left figure of Figure 40 can be used, and when using an Lx4 type sub-partition, a 32x16 RST matrix as shown in the right figure of Figure 40 can be used.

The sub-region to which the secondary transform or RST is applied may be determined in units of coding blocks to which the ISP is applied. As an example, it may be set to apply ST or RST only to a sub-partition located at the top or left of the sub-partitions. Alternatively, an area having a size of 4x8 or 8x4 located at the upper left of the coding block may be defined as a sub-region for applying ST or RST.

When the form of the intra block or the form of the sub-partition is 4xN or Nx4, the 4x8 or 8x4 subregion can be rearranged into a 32x1 matrix A _ISP .

By applying a 32 x 32 non-separable transform matrix T _ISP to matrix A _ISP , you can create a (reduced) secondary transform matrix A _R (32x1 matrix) as shown in Equation (21), which is based on ISP (Reduced) secondary transform. Is defined as

A _ISP = T _ISP *A (21)

ISP-based (Reduced) secondary transform matrix A _ISP may be rearranged into the upper left 4x8 or 8x4 block among the first transform blocks.

Among the combined prediction coding modes, under merge-intra combined prediction, the final prediction image by weighted prediction of the prediction image generated from the merge mode (hereinafter, the merge prediction image) and the prediction generated by the intra prediction method (hereinafter, the intra prediction image). Can be created. A flag indicating whether merge-intra-combination prediction has been used A merge-intra-combination prediction flag (ciip_flag) may be signaled through a bitstream. If the ciip_flag value is 1, a merge-intra prediction image may be generated by weighted prediction of a merge prediction image acquired based on specific motion information in merge_idx and an intra prediction image acquired based on an intra prediction mode.

Secondary transform or RST may be allowed even in a coding block having a ciip_flag value of 1. That is, even when the value of ciip_flag is 1, whether to perform secondary transform or RST may be adaptively determined according to the size, shape, or intra prediction mode of the current coding block. Alternatively, even when the value of ciip_flag is 1, a syntax indicating whether to perform secondary transform or RST may be signaled.

4 In-loop filtering

In-loop filtering is a technique that adaptively performs filtering on a decoded image in order to reduce the loss of information that occurs during quantization and encoding. A deblocking filter, a sample adaptive offset filter (SAO), and an adaptive loop filter (ALF) are examples of in-loop filtering.

A second reconstructed image may be generated by performing at least one of a deblocking filter, a sample adaptive offset (SAO), or an adaptive loop filter (ALF) on the first reconstructed image.

After applying the deblocking filter to the reconstructed image, SAO and ALF may be applied.

In the video encoding process, transformation and quantization are performed in units of blocks. A loss occurs during the quantization process, and a discontinuity occurs at the boundary of the reconstructed image. Discontinuous images appearing at the boundary of a block are defined as blocking artifacts.

The deblocking filter is a method of mitigating block quality deterioration (blocking artifact) occurring at a block boundary of a first image and enhancing encoding performance.

Figure 40

Block quality deterioration can be alleviated by performing filtering at the block boundary, and whether the block is coded in intra prediction mode as shown in Figure 40 , or whether the difference between the absolute values of the motion vectors of neighboring blocks is greater than a predefined threshold. A block filter strength (BS) value may be determined based on at least one of whether or not reference pictures of neighboring blocks are identical to each other. When the BS value is 0, filtering is not performed, and when the BS value is 1 or 2, filtering may be performed at a block boundary.

Since quantization is performed on a transform coefficient, a ringing artifact occurs at the edge of an object, or a pixel value increases or decreases by a certain value compared to the original because quantization is performed in the frequency domain. The SAO can effectively reduce the ringing phenomenon by adding or subtracting a specific offset in units of blocks in consideration of the pattern of the first reconstructed image. SAO is composed of an edge offset (hereinafter referred to as EO) and a band offset (BO) according to characteristics of a reconstructed image. The edge offset is a method of adding an offset to the current sample differently according to the surrounding pixel sample pattern. Band offset is to reduce coding errors by adding a certain value to a set of pixels with similar pixel brightness values in a region. By dividing the pixel brightness into 32 uniform bands, pixels having similar brightness values can be made into one set. For example, four adjacent bands can be grouped into one category. One category can be set to use the same offset value.

ALF (Adaptive Loop Filter) is a method of generating a second reconstructed image using one of predefined filters for a first reconstructed image or a reconstructed image that has subjected deblocking filtering to the first reconstructed image as shown in Equation (22). .

(22)

(22)

Fig. 41

In this case, the filter can be selected in units of pictures or units of CTU.

In the Luma component, you can select any of the 5x5, 7x7 or 9x9 diamond shapes as shown in Figure 41 below. In the chroma component, it may be limited to use only the 5x5 diamond shape.

5 Video coding structure

Fig. 42

Figure 42 is a diagram showing the video coding structure. The input video signal can be divided into a predetermined coding unit with the smallest rate distortion value. A prediction image may be generated by performing at least one of inter prediction, intra prediction, or complex prediction for each coding unit. In order to perform inter prediction in the encoding step, motion estimation may be performed. A residual image may be generated by differentiating the predicted image from the original image.

A transform coefficient may be generated by performing a first transform and/or a second transform on the residual image, or a transform skip coefficient may be generated by performing a transform skip. Quantization may be performed on a transform coefficient or a transform skip coefficient to generate a quantized transform coefficient or a quantized transform skip coefficient.

A reconstructed image may be generated by performing inverse quantization and inverse transformation on the quantized transform coefficient or the quantized transform skip coefficient. In-loop filtering may be performed on the reconstructed image to generate a filtered reconstructed image. This filtered reconstructed image becomes the final output image in the encoding step. The filtered reconstructed image is stored as a reference picture buffer and can be used as a reference picture in another picture.

Claims (1)

Video coding and decoding method using intra prediction mode signaling coding