KR20210030816A - Video encoding method and apparatus and video decoding method and apparatus - Google Patents

Abstract

A method for predictive encoding using tile, brick, and slice information encoding and a device thereof are provided. The method can improve the video signal coding efficiency.

Description

Video signal processing method and apparatus TECHNICAL FIELD OF THE INVENTION

The present invention relates to a video signal processing method and apparatus.

Video images are compression-encoded by removing spatio-temporal redundancy and inter-view redundancy, which can be stored in a format suitable for a storage medium or transmitted through a communication line.

The present invention is to improve the coding efficiency of a video signal.

In order to solve the above problems, the present invention provides a method and apparatus for encoding tile, brick, and slice information.

The video signal processing method and apparatus according to the present invention can improve video signal coding efficiency through a prediction method using tile, brick, and slice information encoding.

One Basic coding block structure

A picture is divided into a plurality of basic coding units (Coding Tree Unit, hereinafter, CTU) as shown in Figure 1.

Picture 1

The size of the CTU can be defined in units of pictures or video sequences, and each CTU is configured not to overlap with other CTUs. For example, the CTU size may be set to 128x128 in the entire sequence, and any one of 128x128 to 256x256 may be selected and used in picture units.

A coding block/coding unit (CU) may be generated by hierarchically dividing the CTU. Prediction and transformation may be performed in units of coding units, and become a basic unit for determining a prediction coding mode. The prediction encoding mode refers to a plurality of methods of generating a predicted image, such as intra prediction (intra prediction, hereinafter, intra prediction), inter prediction (hereinafter, inter prediction), or combined prediction. It can be mentioned. In more detail, for example, a prediction block may be generated by using at least one of intra prediction, inter prediction, and complex prediction in a coding unit unit. In the inter prediction mode, when the reference picture indicates the current picture, a prediction block may be generated in an area within the current picture that has already been decoded. Since a prediction block is generated using a reference picture index and a motion vector, it can be included in inter prediction. Intra prediction is a method of generating a prediction block using information of the current picture, inter prediction is a method of generating a prediction block using information of another picture that has already been decoded, and composite prediction is a method of mixing inter prediction and intra prediction. This is how you use it.

The CTU can be divided (partitioned) into a quad tree, binary tree or triple tree, as shown in Figure 2. The divided block can be further divided into a quad tree, binary tree, or triple tree. The method of dividing the current block into four square partitions is called quad-tree partitioning, the method of dividing the current block into two non-square partitions is called binary tree partitioning, and the method of dividing the current block into three non-square partitions is called quad-tree partitioning. It is defined as binary tree partitioning.

Binary partitioning in the vertical direction ( SPLIT_BT_VER in Figure 2 ) is called vertical binary tree partitioning, and binary tree partitioning in the horizontal direction ( SPLIT_BT_HOR in Figure 2 ) is defined as horizontal binary tree partitioning.

Triple-tree partitioning in the vertical direction ( SPLIT_TT_VER in Fig. 2 ) is called vertical triple-tree partitioning, and triple-tree partitioning in the horizontal direction ( SPLIT_TT_HOR in Fig. 2 ) is defined as horizontal triple-tree partitioning.

The number of additional divisions is called partitioning depth, and the maximum value of the partitioning depth for each sequence or picture/slice can be set differently. It can be set to have, and a syntax indicating this can be signaled.

Picture 2

The divided coding block may be further divided by a method such as quad partitioning, binary partitioning, or multi-partitioning to configure a coding unit, or may configure a coding unit without further partitioning.

Figure 3

As shown in Figure 3 , a coding unit can be hierarchically set for one CTU, and a coding unit can be partitioned using at least one of coding units using binary tree partitioning, quad tree partitioning/triple tree partitioning. This method is defined as multi tree partitioning.

A coding unit generated by dividing an arbitrary coding unit having a partitioning depth of k is referred to as a lower coding unit, and the partitioning depth is k+1. A coding unit having a partitioning depth k including a lower coding unit having a partitioning depth k+1 is referred to as an upper coding unit.

The partitioning type of the current coding unit may be limited according to the partitioning type of the upper coding unit and/or the partitioning type of the coding units surrounding the current coding unit.

Here, the partitioning type represents an indicator indicating which partitioning of binary tree partitioning, quad tree partitioning/triple tree partitioning is used.

2 How to generate predictive image

In video encoding, a predicted image can be generated by a plurality of methods, and a method of generating a predicted image is called a predictive encoding mode.

The prediction encoding mode may include an intra prediction encoding mode, an inter prediction encoding mode, a current picture reference encoding mode, or a combined prediction mode.

The inter prediction coding mode is a prediction coding mode that generates a prediction block (prediction image) of the current block using information of a previous picture, and the intra prediction coding mode is a prediction that generates a prediction block using samples adjacent to the current block. It is called an encoding mode. A prediction block may be generated using an image that has already been reconstructed of the current picture, and this is defined as a current picture reference mode or an intra block copy mode.

A prediction block may be generated by using at least two or more of an inter prediction encoding mode, an intra prediction encoding mode, or a current picture reference encoding mode, which is referred to as a combined prediction mode.

2.1 Intra prediction coding and decoding method

In intra prediction , as shown in Figure 4 , an already coded boundary sample around the current block is used to generate intra prediction, which is called an intra reference sample.

The average value of the intra reference sample is predicted by setting the value of all samples in the predicted block (DC mode), or the vertical direction predicted sample generated by weighted predicting the horizontal direction prediction sample and the vertical direction reference sample generated by performing horizontal reference weighted prediction. After generation, a prediction sample may be generated by weighted prediction of a horizontal direction prediction sample and a vertical direction prediction sample (Planar mode), or intra prediction may be performed using a directional intra prediction mode.

Figure 4

As shown in the left figure of Figure 5, intra prediction can be performed using 33 directions (total of 35 intra prediction modes), and 65 directions (total of 67 intra prediction modes) as shown in the right figure of Figure 5 May be. In the case of using directional intra prediction, an intra reference sample (reference reference sample) may be generated in consideration of the direction of the intra prediction mode, and intra prediction may be performed therefrom.

An intra reference sample on the left side of the coding unit is referred to as a left intra reference sample, and an intra reference sample on the upper side of the coding unit is referred to as an upper intra reference sample.

When directional intra prediction is performed, as shown in Table 1, an intra direction parameter (intraPredAng), which is a parameter indicating a prediction direction (or a prediction angle), may be set according to the intra prediction mode. Table 1 is only an example based on a directional intra prediction mode having a value of 2 to 34. It goes without saying that the prediction directions (or prediction angles) of the directional intra prediction modes are further subdivided so that more than 33 directional intra prediction modes can be used.

Picture 5

Table 1

Figure 6

When intraPredAng is negative (for example, when the intra prediction mode index is between 11 and 25), as shown in Figure 6 , intra reference samples located on the left and upper sides of the current block are 1D configured according to the angle of the intra prediction mode. It can be reconstructed as a reference sample (Ref_1D).

Figure 7

When the intra prediction mode index is between 11 and 18, a one-dimensional reference sample can be generated in a counterclockwise direction from an intra reference sample located on the upper right side of the current block to an intra reference sample located on the lower left side, as shown in Figure 7.

In other modes, a one-dimensional reference sample may be generated using only an upper side intra reference sample or a left side intra reference sample.

Figure 8

When the intra prediction mode index is between 19 and 25, a one-dimensional reference sample can be generated in a clockwise direction from an intra reference sample located at the lower left side of the current block to an intra reference sample located at the right side of the current block as shown in Figure 8.

A weight-related parameter ifact applied to at least one reference sample determined based on the reference sample determination index iIdx and iIdx may be derived as in Equations (1) to (2) below. iIdx and i _fact are variably determined according to the slope of the directional intra prediction mode, and a reference sample specified by iIdx may correspond to an integer pel.

iIdx = (y+1) * P _ang /32 (One)

i _fact = [(y+1) * P _ang ] & 31 (2)

A prediction image may be derived by specifying at least one one-dimensional reference sample for each prediction sample. For example, a position of a one-dimensional reference sample that can be used to generate a prediction sample may be specified in consideration of a slope value of a directional intra prediction mode. Each prediction sample may have a different directional intra prediction mode. A plurality of intra prediction modes may be used for one prediction block. A plurality of intra prediction modes may be expressed as a combination of a plurality of non-directional intra prediction modes, a combination of one non-directional intra prediction mode and at least one directional intra prediction mode, or a plurality of directional intra prediction modes. It can also be expressed as a combination of modes. Different intra prediction modes may be applied for each predetermined sample group in one prediction block. A predetermined sample group may consist of at least one sample. The number of sample groups may be variably determined according to the size/number of samples of the current prediction block, or may be a fixed number pre-set in the encoder/decoder independently of the size/number of samples of the prediction block.

Specifically, for example, the position of the one-dimensional reference sample may be specified using the reference sample determination index iIdx.

When the slope of the intra prediction mode cannot be expressed with only one one-dimensional reference sample according to the slope of the intra prediction mode, a first prediction image may be generated by interpolating adjacent one-dimensional reference samples as shown in Equation (3). When the angular line according to the slope/angle of the intra prediction mode does not pass the reference sample located in the integer pel, the first prediction image may be generated by interpolating the reference sample adjacent to the left/right or above/below the corresponding angular line. . The filter coefficient of the interpolation filter used at this time may be determined based on _{i fact.} For example, the filter coefficient of the interpolation filter may be derived based on a distance between a fractional pel located on an angular line and a reference sample located on the integer pel.

P(x,y) = ((32-i _fact )/32)* Ref_1D(x+iIdx+1) + (i _fact /32)* Ref_1D(x+iIdx+2) (3)

When the slope of the intra prediction mode can be expressed with only one one-dimensional reference sample (when the i _fact value is 0), the first prediction image can be generated as shown in Equation (4) below.

P(x,y) = Ref_1D(x+iIdx+1) (4)

2.2 Wide-angle intra prediction coding method

The prediction angle of the directional intra prediction mode may be set between 45 and -135 as shown in Figure 9.

Figure 9

When the intra prediction mode is performed in the amorphous coding unit, there may be a disadvantage of predicting a current sample from an intra reference sample far from the current sample instead of an intra reference sample close to the current sample due to a predefined prediction angle.

For example, as shown in the left figure of Fig. 10 , in a coding unit whose width is larger than the height of the coding unit (hereinafter, horizontal coding unit), intra prediction is performed at a far distance L instead of a close sample T. I can. As another example, as shown in the right figure of Figure 10, in a coding unit whose height is larger than the width of the coding unit (hereinafter, the vertical coding unit), the intra-distance sample T from the distant sample T instead of the close sample L You can make predictions.

Figure 10

In the amorphous coding unit, intra prediction may be performed at a prediction angle wider than a predefined prediction angle, and this is defined as a wide angle intra prediction mode.

내지

의 예측 각도를 가질 수 있으며, 기존 인트라 예측 모드에서 사용된 각도를 벗어나는 예측 각도를 와이드 앵글 각도라고 한다. The wide angle intra prediction mode is

To

It can have a prediction angle of, and a prediction angle that deviates from the angle used in the existing intra prediction mode is called a wide angle angle.

In the left figure of Figure 10 , sample A in the horizontal direction coding unit can be predicted from the intra reference sample T using the wide-angle intra prediction mode.

In the right figure of Figure 10 , sample A in the vertical direction coding unit can be predicted from the intra reference sample L using the wide-angle intra prediction mode.

N+M intra prediction modes may be defined by adding M wide angle angles to N existing intra prediction modes. Specifically, for example, as shown in Table 2, a total of 95 intra prediction modes may be defined by adding 28 wide angle angles to 67 intra modes.

The intra prediction mode that the current block can use may be determined according to the shape of the current block. For example, 65 directional intra prediction modes may be selected from among 95 directional intra prediction modes based on at least one of a size of a current block, an aspect ratio (eg, a ratio of width and height), and a reference line index.

The intra prediction mode angle may be adaptively determined based on at least one of a shape of a current block and a reference line index. For example, the intraPredAngle of Mode 15 may be set to have a larger value when the current block is a square than when the current block is an amorphous. Alternatively, the intraPredAngle of Mode 75 may be set to have a larger value when a non-adjacent reference line is selected than when an adjacent reference line is selected.

When using the wide-angle intra prediction mode, the length of the upper intra reference sample can be set to 2W+1, and the length of the left intra reference sample can be set to 2H+1, as shown in Figure 11.

Figure 11

In the case of using wide-angle intra prediction, when the intra prediction mode of the wide-angle intra prediction mode is encoded, the number of intra prediction modes increases, and thus encoding efficiency may be lowered. The wide-angle intra prediction mode can be encoded by replacing the existing intra prediction mode that is not used in the wide-angle intra prediction mode, and the replaced prediction mode is called a wide-angle replacement mode. The wide angle replacement mode may be an intra prediction mode in a direction opposite to the wide angle intra prediction mode.

Specifically, for example, when 35 intra predictions are used as shown in Figure 12 , wide-angle intra prediction mode 35 can be encoded with intra prediction mode 2, which is a wide-angle replacement mode, and wide-angle intra prediction mode 36 is used for wide-angle replacement. It can be encoded in intra prediction mode 3, which is a mode.

Figure 12

The replacement mode and number may be differently set according to the type of coding block or the ratio of the height to width of the coding block. Specifically, for example, Table 2 shows intra prediction modes used according to the ratio of the width and height of a coding block.

Table 2

2.3 Multi-line intra prediction coding method

As shown in Figure 13 below, intra prediction may be performed on a plurality of intra reference lines, which is referred to as a multi-line intra prediction coding method.

Fig. 13

Intra prediction may be performed by selecting any one of a plurality of intra reference lines composed of an adjacent intra parallel line and a non-adjacent intra reference line, and this is referred to as a multi-line intra prediction method. The non-adjacent intra reference line is a first non-adjacent intra reference line (non-adjacent reference line index 1), a second non-adjacent intra reference line (non-adjacent reference line index 2), and a third non-adjacent intra reference line (non-adjacent reference line). It can be composed of index 3). Only some of the non-adjacent intra reference lines may be used. For example, only a first non-adjacent intra reference line and a second non-adjacent intra reference line may be used, or only a first non-adjacent intra reference line and a third non-adjacent intra reference line may be used.

An intra reference line index (intra_luma_ref_idx), a syntax specifying a reference line used for intra prediction, may be signaled in units of coding units.

Specifically, when using an adjacent intra reference line, a first non-adjacent intra reference line, and a third non-adjacent intra reference line, intra_luma_ref_idx may be defined as shown in Table 4 below.

Table 4

When a non-adjacent intra reference line is used, it may be set not to use a non-directional intra prediction mode. That is, when a non-adjacent intra reference line is used, it may be restricted not to use the DC mode or the planar mode.

The number of samples belonging to the non-adjacent intra reference line may be set to be larger than the number of samples of the adjacent intra parallel line. Also, the number of samples of the i+1th non-adjacent intra reference line may be set to be greater than the number of samples of the ith non-adjacent intra reference line. The difference between the number of upper samples of the i-th non-adjacent intra reference line and the number of upper samples of the i-1th non-adjacent intra reference line may be represented by the reference sample number offset offsetX[i].

offsetX[1] represents a difference between the number of upper samples of the first non-adjacent intra reference line and the number of upper samples of the adjacent intra reference line. The difference between the number of left samples of the i-th non-adjacent intra reference line and the number of left samples of the i-1th non-adjacent intra reference line may be expressed as a reference sample number offset offsetY[i]. offsetY[1] represents a difference value between the number of left samples of the first non-adjacent intra reference line and the number of left samples of the adjacent intra reference line.

A non-adjacent intra reference line with an intra reference line index of i may be composed of an upper non-adjacent reference line refW + offsetX[i], a left non-adjacent reference line refH+ offsetY[i], and an upper left sample. The number of included samples may consist of refW + refH + offsetX[i] + offsetY[i] +1.

refW =2* nTbW (5)

refH =2*　*nTbH (6)

In equations (5) to (6), nTbW represents the width of the coding unit, nTbH represents the height of the coding unit, and whRatio can be defined as in the following equation (7).

whRatio = Min(log2(nTbW/nTbH),2) (7)

In the multi-line intra prediction coding method, when a non-adjacent intra reference line is used, the wide-angle intra mode may be set not to be used. Alternatively, if the MPM mode of the current coding unit is a wide-angle intra mode, the multi-line intra prediction coding method may be set not to be used.

In this case, a non-adjacent intra reference line with an intra reference line index of i may consist of an upper non-adjacent reference line W + H + offsetX[i], a left non-adjacent reference line H + W + offsetY[i], and an upper left sample. , The number of samples belonging to the non-adjacent intra reference line may be composed of 2W + 2H + offsetX[i] + offsetY[i] +1, and the values of offsetX[i] and offsetY[i] may vary depending on the whRatio value. have.

For example, if the whRatio value is greater than 1, you can set the offsetX[i] value to 1 and the offsetY[i] value to 0. If the whRatio value is less than 1, you can set the offsetX[i] value to 0, offsetY[ i] You can also set the value to 1.

2.4 Sub-partition intra prediction coding and decoding method

Fig. 14

As shown in Figure 14, intra prediction is performed in units of coding units, and after partitioning the coding units in units of sub-blocks, transformation and quantization can be performed, respectively, and this is performed by Intra sub-partition prediction (ISP). ).

In the sub-partition intra coding method, all sub-blocks in the coding unit may be restricted to use the same intra prediction mode.

Alternatively, in the sub-partition intra coding method, at least one intra prediction mode among sub-blocks in the coding unit may be different from other sub-blocks. For example, the intra prediction mode of the nth subblock may be derived by adding/subtracting an offset in the intra prediction mode of the n-1th subblock. The offset may be predefined in the encoder/decoder, or may be derived according to the shape and size of the coding unit, the intra prediction mode, and the shape and size of the sub-partition. Alternatively, information for determining the offset may be signaled through a bitstream.

Alternatively, some of the plurality of sub-blocks may use a directional intra prediction mode, and others may use a non-directional prediction mode. In this case, only when the directional prediction mode of the first sub-block is a predefined intra prediction mode, the non-directional prediction mode may be used in the second sub-block.

Alternatively, in the sub-partition intra coding method, an intra prediction mode of sub-blocks in a coding unit may be selected from predefined candidates. Here, the predefined candidate may include at least one of a horizontal intra prediction mode, a vertical intra prediction mode, a diagonal intra prediction mode (eg, an upper left, an upper right, and a lower left intra prediction mode), and a non-directional prediction mode. .

Alternatively, in the sub-partition intra coding method, the intra prediction mode of sub-blocks in the coding unit may be derived only from MPM candidates.

Alternatively, the use of the sub-partition intra coding method may be limited according to the intra prediction mode of the coding unit. For example, when the intra prediction mode of the coding unit is a non-directional prediction mode (eg, Planar or DC), or when a predefined directional prediction mode (eg, horizontal, vertical or diagonal direction), the sub-partition intra coding method is used. It can be set not to be. Alternatively, when the intra prediction mode of the coding unit is an intra prediction mode in which a neighboring sub-partition cannot be used as a reference sample (eg, a diagonal direction or a wide-angle intra prediction mode), the sub-partition intra coding method may be set not to be used. I can.

Alternatively, the use of the sub-partition intra coding method may be limited according to the size or shape of the coding unit. For example, when the height-width ratio of the coding unit is greater than or equal to the threshold value, or when at least one of the width or height of the current block is less than or equal to the threshold value, the sub-partition intra encoding method may be set not to be used.

Different quantization parameter values may be used for each subblock, and a difference between a quantization parameter value of a previous subblock and a quantization parameter value of a current subblock may be encoded.

The intra sub-partition can be divided into a vertical partition or a horizontal partition. The vertical partition may be generated by dividing the coding block using at least one vertical line, and the horizontal partition may be generated by dividing the coding block using at least one horizontal line.

The number of partitions may be determined according to the size/type of the coding unit. For example, a 4x4 coding unit may not divide a sub-partition, an 8x4 to 4x8 coding unit may be divided into two sub-partitions, and other coding units may be divided into four partitions.

Alternatively, information indicating the number, size, or shape of partitions may be signaled through a bitstream. According to the number of partitions determined by signaled information, the size or shape of the partition may be determined, or the number of partitions may be determined according to the size/type of the partition determined by signaled information.

As shown in Figure 15, the coding unit can be divided into a vertical partition or a horizontal partition. An intra partition type flag (intra_subpart_type_flag) indicating whether to divide into a vertical partition or a horizontal partition may be transmitted through a bitstream. If the intra_subPart_type_flag value is 1, it indicates that a sub-partition can be configured by dividing into horizontal partitions, and if the intra_subPart_type_flag value is 0, it indicates that a sub-partition can be configured by dividing into vertical partitions.

Alternatively, the shape of the intra sub-partition may be derived based on the size/shape of the coding unit. For example, it is possible to derive the intra sub-partition type without signaling the syntax in the bitstream by using the shape ratio (whRatio) of the coding unit.

For example, if whRatio is less than or equal to a specific value N (hereinafter, positive sub-partition threshold), a vertical partition is used, and whRatio is less than a specific value -N (hereinafter, negative sub-partition threshold). In the case of or, you can use horizontal partitions. As an example, the sub-partition threshold may be set to a predefined value such as 1 to 2 to 3, or may be set to a value defined in units of a sequence.

Alternatively, the shape of the intra sub-partition may be determined according to the intra prediction mode of the coding unit. For example, when the intra prediction mode of the coding unit is in the horizontal direction, vertical partitioning (or horizontal partitioning) is applied, and when the intra prediction mode of the coding unit is in the vertical direction, horizontal partitioning (or vertical partitioning) is applied. I can.

Fig. 15

2.5 Inter prediction coding method

A method of generating a prediction block (prediction image) of a block in a current picture using information of a previous picture is referred to as an inter prediction encoding mode.

A prediction block may be generated from a specific block in a previous picture of the current block.

In the encoding step, the block with the smallest reconstruction error among the blocks in the previous picture can be selected by searching around a collocated block, and the x-axis difference and y-axis difference between the upper left sample of the current block and the upper left sample of the selected block Is defined as a motion vector, and can be signaled by transmitting it to a bitstream. A block generated through interpolation from a specific block of a reference picture specified by a motion vector is called a motion compensated predictor block.

As shown in Fig. 16, the collocated block represents the block of the picture with the same location and size of the current block and the upper left sample. The corresponding picture may be specified from the same syntax as the reference picture reference.

Fig 16

A residual block may be generated by differentiating the motion compensation prediction block from the original image of the current block.

As the motion vector, motion vectors having different pixel precisions may be used in units of sequences, units of slices, or units of blocks. For example, the pixel precision of a motion vector in a specific block may be at least one of Octor-pel, Quarter-Pel, Half-Pel, Integer pel, and 4 Integer pel.

In the inter prediction mode, an inter prediction method using a translation motion and an affine inter prediction method using an affine motion may be selectively used.

2.5.1 Afine inter prediction coding and decoding method

There are many cases in which a motion of a specific object does not appear linearly in a video. For example, in an image using affine motion such as camera zoom-in, zoom-out, rotation, and affine transformation that enables transformation into arbitrary shapes, as shown in Figure 17, the object's When only the translation motion vector is used for motion, the motion of the object cannot be effectively expressed, and coding performance may be degraded.

Figure 17

Affine motion can be expressed as the following equation (8).

(8)

(8)

Expressing affine motion using a total of six parameters is effective for images with complex motion, but encoding efficiency may be degraded due to the large number of bits used to encode affine motion parameters.

Accordingly, affine motion can be simplified and expressed with four parameters, and this is defined as a four-parameter afine motion model. Equation (9) expresses affine motion with four parameters.

(9)

(9)

The four-parameter affine motion model may include motion vectors at two control points of the current block. The control point may include an upper left corner, an upper right corner, and a lower left corner of the current block. As an example, as shown in the left figure of Figure 18, the 4-parameter affine motion model is determined by the motion vector sv0 at the upper left sample (x0,y0) of the coding unit and the motion vector sv1 at the upper right sample (x1,y1) of the coding unit. It can be determined, and sv0 and sv1 are defined as affine seed vectors. Hereinafter, it is assumed that the affine seed vector sv0 located in the upper left corner is the first affine seed vector, and the affine seed vector sv1 located in the upper right corner is the second affine seed vector. In the 4-parameter affine motion model, it is also possible to replace one of the 1/2 affine seed vectors with the affine seed vector located in the lower left corner.

Fig. 18

The 6-parameter affine motion model is an affine motion model in which the motion vector sv2 of the residual control point (e.g., sample (x2,y2) in the lower left corner) is added to the 4-parameter affine motion model as shown in the right figure of Fig. 18. Hereinafter, the affine seed vector sv0 located at the upper left is assumed to be the first affine seed vector, the affine seed vector sv1 located at the upper right is assumed to be the second affine seed vector, and the affine seed vector sv2 located at the lower left is assumed. Assume that it is the third affine seed vector.

Information about the number of parameters for expressing affine motion may be encoded in the bitstream. For example, a flag indicating whether 6 parameters are used and a flag indicating whether 4 parameters are used may be encoded in slices, tiles, coding units, or CTU units. Accordingly, a 4-parameter affine motion model to a 6-parameter affine motion model may be selectively used in slices, coding units, or CTU units.

A motion vector can be derived for each sub-block of the coding unit using the affine seed vector, as shown in Fig. 19, and this is defined as an affine sub-block vector.

Figure 19

The affine sub-block vector can also be derived as in Equation (10) below. Here, the reference sample position (x,y) of the sub-block may be a sample located at a corner of the block (eg, an upper left sample), or a sample having at least one of the x-axis or y-axis at the center (eg, a center sample) .

(10)

(10)

Motion compensation may be performed in units of coding units or in units of sub-blocks within the coding unit by using the afine sub-block vector, and this is defined as an afine inter prediction mode. In Equation (10), (x ₁ -x ₀ ) may be the same as the width w of the coding unit, or may be set to w/2 or w/4.

The affine seed vector and the reference picture list of the current coding unit can be derived by using the afine motion vector (afine sub-block vector or afine seed vector) of the neighboring block of the current coding unit, and this is an afine merge mode. Is defined as.

2.5.2 Merge mode coding method

Motion information (motion vector, reference picture index, etc.) of the current coding unit is not encoded, and can be derived from motion information of neighboring blocks. Motion information of any one of the neighboring blocks can be set as motion information of the current coding unit, and this is defined as a merge mode.

Neighboring blocks used for the remaining mode may be a picture 20 remaining candidate indices from 0 to 4 and the block are adjacent to the coding unit as the (current boundary with the abutting block of the coded unit), as shown in Figure 20 the remaining candidate index from 5 to 26 of the It may be a non-adjacent block.

Fig. 20

When the distance of the merge candidate to the current block exceeds a predefined threshold, it may be set as not available.

For example, a predefined threshold may be set to the height of the CTU (ctu_height) or ctu_height+N, and this is defined as a merge candidate usable threshold. That is, the y-axis coordinate (y _{i ) of the merge candidate and the y-axis coordinate difference (y 0} ) (i.e., y _i- y ₀ ) of the upper left sample of the current coding unit (hereinafter, the reference sample of the current coding unit) are the merge candidate available threshold. If it is greater than the value, the merge candidate may be set as not available. Here, N is a predefined offset value. Specifically, for example, N may be set to 16 or ctu_height may be set.

2.5.3 Inter-decoding area merging method

Motion information (motion vector and reference picture index) of a coding unit that is already encoded by inter prediction in the current picture can be stored in a list having a predefined size, which is defined as an inter-region motion information list.

Motion information (motion vector and reference picture index) in the inter-region motion information list is called an inter-decoding region merge candidate.

The inter decoding region merge candidate can be used as a merge candidate of the current coding unit, and this method is defined as an inter decoding region merge method.

When the slice is initialized, the inter-region motion information list is empty, and when a partial region of the picture is encoded/decoded, it can be added to the inter-region motion information list. The initial inter-decoding region merge candidate of the inter-region motion information list may be signaled through the slice header.

When the coding unit is encoded/decoded by inter prediction, motion information of the coding unit may be updated in the inter-region motion information list as shown in Figure 21. When the number of inter-decoding region merge candidates in the inter-region motion information list is the maximum value, the value with the smallest inter-region motion information list index (firstly, the inter-decoding region merge candidate in the inter-region motion information list) is removed. , Motion information of the most recently encoded/decoded inter region may be added as an inter decoding region merge candidate.

Fig. 21

The motion information mvCand of the decoded coding unit may be updated in the inter-region motion information list HmvpCandList. At this time, if the motion information of the decoded coding unit is the same as any one of the motion information in the inter-region motion information list (when both the motion vector and the reference index are the same), the inter-region motion information list is not updated, or as shown in Figure 23. The motion information mvCand of the coding unit decoded together may be stored at the end of the inter-region motion information list. At this time, if the index of HmvpCandList that has the same motion information as mvCand is hIdx, HMVPCandList [i] can be set to HVMPCandList[i-1] for all i larger than hIdx as shown in Figure 23.

When sub-block merge candidates are used in the currently decoded coding unit, motion information of the representative sub-block in the coding unit may be stored in the inter-region motion information list.

As an example, the representative sub-block in the coding unit may be set as an upper left sub-block in the coding unit or as a middle sub-block in the coding unit, as shown in FIG. 22.

Fig. 22

Fig. 23

A total of NumHmvp motion information (motion vector and reference picture index) can be stored in the inter-region motion information list, and NumHmvp is defined as the size of the inter-region motion information list.

The size of the inter-region motion information list may use a predefined value. The size of the inter-region motion information list may be signaled in the slice header. For example, the size of the inter-region motion information list may be defined as 16 or 6.

In a coding unit that is inter prediction and has affine motion information, it may be limited not to have an inter-region motion information list.

Alternatively, in the case of inter prediction and an affine motion vector, the affine subblock vector may be added to the inter-region motion information list. In this case, the position of the sub-block may be set to the upper left or upper right, or the center sub-block.

Alternatively, the average motion vector value of each control point may be added to the inter-region merge candidate list.

When the motion vector MV0 derived by encoding/decoding a specific coding unit is the same as any one of the inter-decoding region merge candidates, MV0 may not be added to the inter-region motion information list. Alternatively, an existing inter-decoding region merge candidate having the same motion vector as MV0 may be deleted, and the index allocated to MV0 may be updated by newly including MV0 in the decoding region merge candidate.

In addition to the inter-region motion information list, an inter-region motion information long term list HmvpLTList may be configured. The size of the inter-region motion information long term list may be set equal to the size of the inter-region motion information list, or may be set to a different value.

The inter-region motion information long term list may be composed of an inter-decoding region merge candidate initially added to a slice start position. After the inter-region motion information long term list is configured with all available values, the inter-region motion information list may be configured, or motion information in the inter-region motion information list may be set as motion information of the inter-region motion information long term list.

At this time, the inter-region motion information long-term list configured once may be updated again when no update is performed, a decoded region among slices is more than half of all slices, or may be updated for every m CTU lines. The inter-region motion information list may be updated whenever the inter-region is decoded, or may be set to be updated in units of CTU lines.

The motion information and partition information or type of the coding unit may be stored in the inter-region motion information list. The inter-decoding region merging method may be performed using only inter-region merge candidates having a similar partition information and shape as the current coding unit.

Alternatively, an inter-region merge candidate list may be individually configured according to the block type. In this case, one of a plurality of inter-region merge candidate lists may be selected and used according to the shape of the current block.

Fig. 24

As shown in Figure 24 , it can be composed of an inter-area affine motion information list and an inter-area motion information list. When the decoded coding unit is in the afine inter or afine merge mode, the first affine seed vector and the second affine seed vector may be stored in the inter-region afine motion information list HmvpAfCandList. Motion information in the inter-region affine merge candidate list is called an inter-decoding region affine merge candidate.

The merge candidates usable in the current coding unit may be configured as follows, and may have the same search order as the configuration order.

1. Spatial merge candidate (coding block adjacent merge candidate and coding block non-adjacent merge candidate)

2. Temporal merge candidate (merge candidate derived from previous reference picture)

3. Inter-decoding domain merge candidate

4. Inter-decoding domain affine merge candidate

5. Zero motion merge candidate

3 Transform and quantization

An image obtained by subtracting the predicted image from the original image is called a residual image.

The residual image may be decomposed into binary frequency components through a two-dimensional transform such as a discrete cosine transform (DCT). Even if high-frequency components are removed from an image, there is a characteristic that significant distortion does not occur visually. If the value corresponding to the high frequency is decreased or set to 0, the compression efficiency can be increased while the visual distortion is not large.

Discrete sine transform (DST) may be used according to the size of the prediction block or the prediction mode. Specifically, for example, when the prediction block/coding block is in the intra prediction mode and the size of the prediction block/coding block is smaller than NxN, the DST transform may be used, and other prediction blocks/coding blocks may be set to use the DCT.

DCT is a process that decomposes (transforms) an image into 2D frequency components using cosine transformation, and the frequency components at that time are expressed as a base image. For example, if DCT transformation is performed in an NxN block, N ² basic pattern components can be obtained. Performing DCT transformation is to obtain the size of each of the basic pattern components included in the original pixel block. The size of each basic pattern component is defined as a DCT coefficient.

In general, a discrete cosine transform (DCT) is mainly used in an image in which many non-zero components are distributed in a low frequency, and a Discrete Sine Transform (DST) may be used in an image in which a large number of high-frequency components are distributed.

DST represents a process of decomposing (converting) an image into 2D frequency components using sin transformation. A 2D image can be decomposed (transformed) into a 2D frequency component using a transformation method other than DCT or DST transformation, and this is defined as 2D image transformation.

Among the residual images, a 2D image transformation may not be performed in a specific block, and this is defined as transform skip. Quantization can be applied after the transform skip. Whether to allow the transform skip may be determined based on at least one of the size or shape of the coding unit. As an example, the transform skip may be limited to be used only in coding units of a specific size or less. For example, it is possible to set the transform skip to be used only in blocks smaller than 32x32. Alternatively, it is possible to restrict the use of transform skip to only a square block. Specifically, for example, transformation skip may be applied in units of 32x32, 16x16, 8x8, and 4x4 blocks.

DCT, DST, or 2D image transformation may be applied to an arbitrary block in the 2D image, and the transformation used in this case is defined as a first transformation. After performing the first transformation, transformation may be performed again in a partial region of the transformation block, and this is defined as a second transformation.

The first transformation may use one of a plurality of transformation cores. Specifically, for example, one of DCT2, DCT8, or DST7 may be selected and used in the transform block. Alternatively, the transform cores of the horizontal direction transformation and the vertical direction transformation of the transformation block may be individually determined. Accordingly, the horizontal direction transformation and the vertical direction transformation may be the same or different. Different transform cores may be used in the horizontal direction transformation and the vertical direction transformation, and this is defined as a multiple transformation core encoding method. In the multi-transform core encoding method, a horizontal direction transformation core and a vertical direction transformation core are referred to as a transformation core set. For example, when the horizontal direction conversion core is DCT2 and the vertical direction conversion core is DST7, the conversion core set may be defined as {DCT2, DST7}.

Block units for performing the first transformation and the second transformation may be set differently from each other. Specifically, for example, after performing the first transform in an 8x8 block of the residual image, the second transform may be performed for each 4x4 sub-block. For another example, after performing the first transform in each 4x4 block, the second transform may be performed in each 8x8 block.

The residual image to which the first transform is applied is defined as a first transform residual image.

DCT, DST, or 2D image transformation may be applied to the first transform residual image, and the transform used in this case is defined as a second transform. The 2D image to which the second transformation is applied is defined as a second transformation residual image.

A sample value in the block after performing the first transformation and/or the second transformation or the sample value in the block after performing the transformation skip is defined as a transform coefficient. Quantization refers to the process of dividing a transform coefficient by a predefined value to reduce the energy of a block. A value defined to apply quantization to a transform coefficient is defined as a quantization parameter.

A predefined quantization parameter may be applied in units of sequences or blocks. Typically, a quantization parameter can be defined with a value between 1 and 63.

After performing transformation and quantization, inverse quantization and inverse transformation may be performed to generate a residual reconstructed image. A first reconstructed image may be generated by adding a predicted image to the residual reconstructed image.

Index information'tu_mts_idx' indicating the transformation type of the current block may be signaled through a bitstream. tu_mts_idx may refer to one of a plurality of transformation type candidates representing an example of a combination of a vertical direction transformation type and a horizontal direction transformation type. On the basis of the transformation type candidate specified by tu_mts_idx, whether to skip the vertical direction transformation and the horizontal direction transformation, and the vertical direction transformation core and the horizontal direction transformation core may be determined. The conversion core may include at least one of DCT2, DCT8, and DST7. As shown in Table 4, a transform type candidate may be set for each tu_mts_idx. Specifically, for example, as shown in Table 4, if tu_mts_idx is 0, DCT-II can be applied to both horizontal and vertical direction conversion, and if tu_mts_idx is 2, DCT-VIII is applied to horizontal direction conversion, and vertical direction conversion DST-VII can be applied to.

Table 4

In the case of using the sub-partition encoding method, a different set of transform cores may be used for each sub-partition, or the same set of transform cores may be used for each coding unit. In this case, only the first sub-partition may signal tu_mts_idx through the bitstream.

4 In-loop filtering

In-loop filtering is a technique that adaptively performs filtering on a decoded image in order to reduce the loss of information that occurs during quantization and encoding. A deblocking filter, a sample adaptive offset filter (SAO), and an adaptive loop filter (ALF) are examples of in-loop filtering.

A second reconstructed image may be generated by performing at least one of a deblocking filter, a sample adaptive offset (SAO), or an adaptive loop filter (ALF) on the first reconstructed image.

After applying the deblocking filter to the reconstructed image, SAO and ALF may be applied.

In the video encoding process, transformation and quantization are performed in units of blocks. A loss occurs during the quantization process, and a discontinuity occurs at the boundary of the reconstructed image. Discontinuous images appearing at the block boundary are defined as block quality deterioration (blocking artifact).

The deblocking filter is a method of mitigating block quality deterioration (blocking artifact) occurring at a block boundary of a first image and improving encoding performance.

Fig. 25

Block quality deterioration can be alleviated by performing filtering at the block boundary, and as shown in Figure 25 , whether the block is encoded in intra prediction mode, or whether the difference between the absolute values of the motion vectors of neighboring blocks is greater than a predefined threshold. A block filter strength (BS) value may be determined based on at least one of whether or not reference pictures of neighboring blocks are identical to each other. When the BS value is 0, filtering is not performed, and when the BS value is 1 or 2, filtering may be performed at a block boundary.

Since quantization is performed in the frequency domain, a ringing artifact occurs at the edge of an object, or a pixel value increases or decreases by a certain value compared to the original. The SAO can effectively reduce the ringing phenomenon by adding or subtracting a specific offset in units of blocks in consideration of the pattern of the first reconstructed image. SAO is composed of an edge offset (hereinafter referred to as EO) and a band offset (BO) according to characteristics of a reconstructed image. Edge offset is a method of adding an offset to a current sample differently according to a sample pattern of surrounding pixels. Band offset is to reduce coding errors by adding a certain value to a set of pixels with similar pixel brightness values in a region. By dividing the pixel brightness into 32 uniform bands, pixels having similar brightness values can be made into one set. For example, four adjacent bands can be grouped into one category. One category can be set to use the same offset value.

ALF (Adaptive Loop Filter) is a method of generating a second reconstructed image using one of predefined filters for a first reconstructed image or a reconstructed image that has subjected deblocking filtering to the first reconstructed image as shown in Equation (11). .

(10)

(10)

Fig. 26

In this case, the filter can be selected in units of pictures or units of CTU.

In the Luma component, you can select any of the 5x5, 7x7 or 9x9 diamond shapes as shown in Figure 26 below. In the chroma component, it may be limited to use only the 5x5 diamond shape.

5 Video coding structure

Fig. 27

Fig. 27 is a diagram showing the video coding structure. The input video signal can be divided into a predetermined coding unit with the smallest rate distortion value. A prediction image may be generated by performing at least one of inter prediction, intra prediction, or complex prediction for each coding unit. In order to perform inter prediction in the encoding step, motion estimation may be performed. A residual image may be generated by differentiating the predicted image from the original image.

A transform coefficient may be generated by performing a first transform and/or a second transform on the residual image, or a transform skip coefficient may be generated by performing a transform skip. Quantization may be performed on a transform coefficient or a transform skip coefficient to generate a quantized transform coefficient or a quantized transform skip coefficient.

A reconstructed image may be generated by performing inverse quantization and inverse transformation on the quantized transform coefficient or the quantized transform skip coefficient. In-loop filtering may be performed on the reconstructed image to generate a filtered reconstructed image. This filtered and reconstructed image becomes the final output image in the encoding step. The filtered reconstructed image is stored as a reference picture buffer and can be used as a reference picture in another picture.

6 Video encoding method using tiles and bricks

In realistic media applications such as panoramic video, 360-degree video or 4K/8K Ultra High Definition (UHD), high-resolution images of Full-HD level or higher are used. Parallelization is required for real-time and/or low-latency encoding/decoding of high-resolution images. A certain area of a picture may be independently encoded and/or decoded by using at least one screen division method among subpictures, slices, tiles, and bricks.

A block in a tile may refer to data in another tile and/or slice and/or a sub-picture, or may be set so as not to encode/decode an image by using it as a reference sample. For example, data in other tiles and/or slices and/or subpictures are not used as reference samples in intra prediction, or data in other tiles and/or slices and/or subpictures are candidates for motion vector derivation. It may be set not to be used as (eg, a merge candidate, an AMVP candidate, or an HVMP candidate) or not to use data in other tiles and/or slices and/or subpictures to calculate the context of a symbol.

A probability table of a context adaptive binary arithmetric coding (CABAC) context may be initialized for each tile unit. You can disable the loop filter at the tile boundary (the boundary between two neighboring tiles). Alternatively, information indicating whether a loop filter is used at the tile boundary may be encoded and decoded.

Fig. 28

The picture and/or sub-picture may be composed of one or at least one or more slices. The slice may be a raster-scan slice or a rectangular slice. One raster scan slice is defined as a group of one or more contiguous tile(s) when following the raster scan order. The figure on the left of Figure 28 shows a raster scan slice. Raster scan slices can take on a non-rectangular shape. The square-shaped slice is defined as at least one or more tiles having a rectangular and/or square shape. The figure on the right of Figure 28 shows a square-shaped slice.

The boundary of the slice may coincide with the picture boundary and/or the tile boundary. As an example, the boundary of the slice may be set so that the left boundary and the upper boundary are located at the picture boundary, or the boundary of the slice may be set to be located at the boundary of an already decoded tile.

Slices and/or tiles may be composed of at least one or more bricks.

The tile may have a square and/or rectangular shape. Alternatively, non-rectangular-shaped tiles can be allowed. Information indicating whether a non-rectangular shape tile exists may be encoded. The boundary of the tile may coincide with the picture boundary and/or the CTU boundary.

Information indicating whether a plurality of tiles exist in a picture may be signaled through a bitstream. When there are multiple tiles in a picture, as shown in Figure 29, the heights of tiles adjacent to the top or bottom may be the same (hereinafter, vertical tile set/tile row), or adjacent to the left or right. The tiles may be configured to have the same width (hereinafter, a horizontal tile set/tile column). At this time, information indicating whether the height of the previous picture and/or the row of the previous tile of the current picture is the same, and/or information indicating whether the width of the previous picture and/or the previous tile column of the current picture is the same as a signal through the bitstream. Can be ringed. For example, as shown in Table 5, all tile rows and/or tile columns of the current picture have the same height except for the tile row and/or tile column adjacent to the rightmost and lowermost boundaries of the current picture. And/or information indicating that they may have the same width may be signaled through the bitstream, and in this case, the height of the tile row and the width of the tile column may be signaled, respectively. The current tile row and/or tile column may have a different height and width from the adjacent previous tile row and/or tile column. In this case, height and width information for each of all tile rows and/or tile columns excluding tile rows and/or tile columns adjacent to the rightmost and lowermost boundaries of the current picture may be signaled. In this case, the height and width of the tile row and/or tile column described above may be expressed as an integer multiple of the luminance and/or color coding tree block (CTB), which are coding basic units. For example, if the width of an arbitrary tile column is 3, it can be seen that the width of an arbitrary tile is composed of 3 CTBs.

Fig. 29

Table 5

Information indicating the number of vertical tile sets in the current picture may be signaled using at least one of a picture parameter set, a sequence parameter set, or a slice header. As an example, the information may be the syntax num_tile_columns_minus1 shown in Table 5. In the example of Figure 29, the number of tile sets in the vertical direction is 4, and a value of the syntax num_tile_columns_minus1 representing this can be set to 3. In this case, a syntax tile_column_width_minus1[i] indicating the width of the i-th vertical tile set excluding the rightmost tile column may be signaled.

Information indicating the number of horizontal tile sets in a picture may be signaled using at least one of a picture parameter set, a sequence parameter set, or a slice header. As an example, the information may be the syntax num_tile_rows_minus1 shown in Table 5. In the example of Figure 29, the number of horizontal tile sets is 3, and a value of the syntax num_tile_rows_minus1 representing this can be set to 2. In this case, the syntax tile_row_height_minus1[i] indicating the height of the i-th horizontal tile set excluding the lowest tile row may be signaled.

Information indicating whether tiles have the same size may be signaled through a picture parameter set, a sequence parameter set, or a slice header. As an example, the information may be the same size tile flag uniform_tile_spacing_flag shown in Table 5. If the uniform_tile_spacing_flag value is 1, this indicates that the widths of each tile in the picture are equally divided, and the heights of the tiles are equally divided.

If the uniform_tile_spacing_flag value is 1, the syntax tile_cols_width_minus1 indicating the uniform width of the tile and the syntax tile_rows_height_minus1 indicating the uniform height of the tile may be signaled. Tiles other than tiles adjacent to the right/bottom boundary in the current picture may have the uniform width and the uniform height.

For example, if uniform_tile_spacing_flag is 1, the width of tiles other than tiles adjacent to the picture boundary may be set to (tile_cols_width_minus1+1)*CtbSizeWidth, and the height may be set to (tile_rows_hieight_minus1+1)*CtbSizeHeight.

The height of the tile at the picture boundary (tile_rows_height_minus1+1)*CtbSizeHeight or less, and the width may be (tile_cols_width_minus1+1)*CtbSizeWidth or less.

When the uniform_tile_spacing_flag value is 0, information indicating a width of a tile and information indicating a height of a tile may be encoded for each tile rows and columns, respectively. As an example, the syntax tile_column_width_minus1[i] indicating the width of the tile and the syntax tile_row_height_minus1[i] indicating the height of the tile may be signaled. For example, tile_column_width_minus1[i] represents the width of the i-th tile, and tile_row_height_minus1[i] represents the height of the i-th tile.

Fig. 30

In Table 5, split information of a tile may signal only a difference value using information signaled first as a predicted value. When signaling the difference value, the difference value including the sine (sign) can be signaled, or the sine (sign) flag and the absolute value of the difference value can be signaled.

For example, when tile information is signaled as in the order of Table 5, tile_rows_height_minus1 may transmit a difference value using tile_cols_width_minus1 as a predicted value.

As an example, when tile information is signaled as in the order of Table 5, num_tile_columns_minus1 may transmit a difference value using at least one of tile_cols_width_minus1 and/or tile_rows_height_minus1 as a predicted value.

For example, tile_column_width_minus1[i] may signal a difference value using at least one or more of the previously signaled tile encoding information as a prediction value, or may signal a difference value using tile_column_width_minus1[i-N] as a prediction value. In this case, N may be expressed as a positive integer greater than 0.

For example, tile_row_height_minus1[i] may signal a difference value using at least one or more of the previously signaled tile encoding information as a prediction value, or tile_row_height_minus1[i-N] may signal a difference value using a prediction value. In this case, N may be expressed as a positive integer greater than 0.

Tiles can be composed of at least one or more bricks. Bricks may be units of parallel processing, and may be encoded/decoded independently of each other. When encoding/decoding a block within a brick, it can be set not to refer to the encoding/decoding information of the other brick. For example, in encoding/decoding a block in a brick, data in the tab is not used as a reference sample for intra prediction, or data in the tab is a candidate for motion vector derivation (e.g., merge candidate, AMVP candidate or HVMP candidate), or data in tabs can be set not to be used for calculating the context of the symbol.

Alternatively, information indicating whether parallel processing between bricks is required may be encoded. According to the information, the availability of data included in the other brick may be determined when encoding/decoding a block included in the brick.

Bricks can be created by dividing tiles horizontally. The figure on the right of Figure 30 shows an example in which tiles located in the rightmost tile column are divided into a plurality of bricks with blue lines. When scanning tiles according to the raster scan sequence as shown in the right figure of Figure 30, for a tile containing multiple bricks, the next tile can be scanned after all the bricks in the tile are scanned. Also, when scanning a tile, as shown in the left figure of Figure 30, after scanning the CTB in the tile according to the raster scan order, the CTB in the next tile can be scanned according to the tile scanning order. The brick boundary can coincide with the CTU boundary. That is, one or more CTU rows in a slice and/or tile may be defined as one brick.

Table 6

The current picture is composed of a plurality of tiles, information indicating whether at least one tile is divided into bricks among the plurality of tiles, brick division information for each tile, and an arbitrary tile is divided into at least one or more bricks. If so, at least one of information indicating the number of bricks and information indicating the width and/or height of each brick may be encoded. For example, as shown in Table 6, a flag brick_spliitng_present_flag indicating that at least one tile can be divided into two or more bricks may be signaled through a bitstream.

If the brick_splitting_present_flag value is 1, it indicates that at least one tile is divided into two or more bricks, and if the brick_splitting_present_flag value is 0, there is no tile that is divided into two or more bricks in a picture or in a picture, subpicture, or slice referring to PPS. Represents.

As shown in Table 6, a flag brick_split_flag[i] indicating whether a tile with a tile ID of i is divided into two or more bricks may be signaled through a bitstream.

If the value of brick_split_flag[i] is 1, it indicates that the tile with tile ID i is divided into two or more bricks. If the value of brick_split_flag[i] is 0, it indicates that the tile with tile ID i is not divided into two or more bricks.

As shown in Table 6, if brick_split_flag[i] for a tile with a tile ID of i is 1 (i.e., a tile divided into two or more bricks), the syntax uniform_brick_spacing_flag indicating whether each brick in the tile is divided equally is used. Can signal. The syntax uniform_brick_spacing_flag indicates whether the heights of bricks in the tile are uniform.

If the uniform_brick_spacing_flag value is 1, it indicates that the heights of bricks in the tile are evenly divided.

If the uniform_brick_spacing_flag value is 1, the syntax brick_height_minus1 indicating the uniform height of bricks may be signaled, and at this time, bricks in the tile may have the same uniform height.

If uniform_brick_spacing_flag is 1, excluding bricks adjacent to the picture boundary, the tile may be divided into bricks having a height of (brick_height_minus1+1)*CtbSizeHeight.

If the uniform_brick_spacing_flag value is 0, the syntax num_brick_rows_minus1[i] indicating the number of bricks in the i-th tile and the syntax brick_row_height_minus1[i][j] indicating the height of the j-th brick in the i-th tile may be signaled.

The syntax brick_row_height_minus1[i] can signal as many as the number of bricks or the number of bricks less 1. For example, the last brick in the tile may be derived by dividing a value obtained by adding all the height values of the previous bricks from the height of the tile.

Tiles and bricks can be identified by brick index. A tile that is not included as bricks is considered to be one brick and may be assigned a brick index. For example, brick indexes may be allocated to tiles and bricks according to the raster scan order.

The syntax brick_height_minus1[i]+1 or brick_row_hegiht_minus1+1 indicating the height of the brick may be set smaller than the height of the tile including the brick index i (the number of vertical CTBs in the tile, hereinafter, rowHeight).

Specifically, for example, brick_height_minus1[i] to brick_row_height_minus1 may be set to have a value between 0 and rowHeight-2.

As shown in Table 6, a flag single_brick_per_slice_flag indicating whether each slice is composed of one brick may be signaled through a bitstream.

If single_brick_per_slice_flag is 1, it indicates that each slice referring to the PPS is composed of one brick, and if single_brick_per_slice_flag is 0, it indicates that at least one of each slice referring to the PPS is composed of two or more bricks.

Information indicating the slice type may be signaled through a bitstream. The information indicates whether the slice is a raster scan type or a rectangular shape type. For example, as shown in Table 6, a syntax rect_slice_flag indicating whether a rectangular slice is used may be signaled through a bitstream.

If rect_slice_flag is 1, it indicates that a rectangular slice is used, and if rect_slice_flag is 0, it indicates that a raster-scan slice is used.

When using a rectangular slice, a slice area can be defined using a brick index.

As an example, a rectangular slice region may be defined by using the upper left brick index information in the slice and the lower right brick index information within the slice.

Specifically, for example, the syntax top_left_brick_idx indicating the brick index of the upper left brick of the rectangular slice and the lower right brick index bottom_right_brick_idx of the rectangular slice may be signaled through a bitstream.

Table 7

For another example, as shown in Table 7 below, a syntax bottom_right_brick_idx indicating the index of the lower right brick may be signaled through a bitstream without signaling the index top_left_brick_idx of the upper left brick of the slice.

The variable TopLeftBrickIdx[sliceIdx] indicating the index of the upper left brick can also be derived as follows.

The variable TopLeftBrickIdx[　i　], NumBricksInSlice[　i　] and BricksToSliceMap[　j　], which specify the brick index of the brick located at the top left corner of the i-th slice, the number of bricks in the i-th slice and the mapping of bricks to slices, are derived as follows:

for( j = 0; i　　=　=　　0　　&&　　j <NumBricksInPic; j++)

BricksToSliceMap[　j　] = -1

NumBricksInSlice[　i　] = 0

for( j = bottom_right_brick_idx[　i　]; j　　>=　　0; j--) {

if(BrickColBd[　j　]　　<=

BrickColBd[　bottom_right_brick_idx[　i　]　]　　&&

BricksToSliceMap[　j　]　　=　=　　-1) {

TopLeftBrickIdx[　i　] = j

NumBricksInSlice[　i　]++

BricksToSliceMap[　j　] = i

}

}

Table 8

As another example, as shown in Table 8 below, a syntax top_left_brick_idx_delta indicating the index of the upper left brick may be signaled through a bitstream without signaling the index bottom_right_brick_idx of the lower right brick of the slice.

In Table 8, top_left_brick_idx_delta[i] is the difference between the index value of the first brick of the previous slice (slice index i-1, hereinafter (i-1) th slice) and the first brick of the current slice (slice index i, hereinafter i th slice). Represents.

The brick index top_left_brick_idx(i) of the upper left brick of the i-th slice can be derived by summing TopLeftBrickIdx(i-1) of the i-1 th slice to top_left_brick_idx_delta[i] as shown in Equation (12). The maximum number of bits of the syntax top_left_brick_idx_delta[i] may be Ceil (Log2( NumBricksInPic-TopLeftBrickIdx( i -1) -1) ). Here, Ceil(a) represents the smallest integer greater than or equal to a.

TopLeftBrickIdx(i) = TopLeftBrickIdx(i-1) +top_left_brick_idx_delta[i] (12)

The value of TopLeftBrickIdx(0) of the first slice (ie, a slice in which i is 0) may be set to 0. In the second slice (ie, a slice in which i is 0), top_left_brick_idx[i] may be encoded instead of top_left_brick_idx_delta[i].

You can set the slice ID for each brick using the following process. BricksToSliceMap[j] represents the slice ID of the brick index j, and BottomRightBrickIdx[i] represents the index of the lower right brick of the i-th slice.

The variable BottomRightBrickIdx[　i　], NumBricksInSlice[　i　] and BricksToSliceMap[　j　], which specify the brick index of the brick located at the top left corner of the i-th slice, the number of bricks in the i-th slice and the mapping of bricks to slices, are derived as follows:

for( j = 0; i　　=　=　　0　　&&　　j <NumBricksInPic; j++)

BricksToSliceMap[　j　] = -1

NumBricksInSlice[　i　] = 0

for( j = TopLeftBrickIdx(i); j　　<　　NumBricksInPic; j--) {

if( BrickColBd[　j　]　　>=　　BrickColBd[　TopLeftBrickIdx(i)　]　　&&

BrickColBd[　j　]　　<=　　BrickColBd[　TopLeftBrickIdx(i)]　　&&

BricksToSliceMap[　j　]　　=　=　　-1) {

BottromRightBrickIdx [　i　] = j

NumBricksInSlice[　i　]++

BricksToSliceMap[　j　] = i

}

}

}

Table 9

As shown in Table 9 below, the syntax bottom_right_brick_idx_delta[i] indicating the difference between the lower right brick index of the i-th slice and the lower right brick index of the i-1th slice may be signaled through a bitstream.

In Table 9, bottom_right_brick_idx_delta[i] is the index of the lower right brick of the previous slice (slice index i-1, hereinafter (i-1) th slice) and the lower right brick of the current slice (slice index i, hereinafter i th slice). Represents the difference value of the index.

The brick index bottomRightIdx[i] of the lower-right brick of the i-th slice can be derived by summing the BottomRightIdx(i-1) of the i-1 th slice to bottom_right_brick_idx_delta[i] as shown in Equation (13). The maximum number of bits of the syntax bottom_right_brick_idx_delta[i] may be Ceil (Log2( NumBricksInPic-BottomRightIdx( i -1)) ). Here, Ceil(a) represents the smallest integer greater than or equal to a.

BottomRightIdx[i] = BottomRightIdx[i-1] + bottom_right_brick_idx_delta[i] (13)

The BottomRightIdx[0] value of the first slice (ie, a slice in which i is 0) may be set as a bottom_right_brick_idx_delta[0] value.

You can set the slice ID for each brick using the following process. BricksToSliceMap[j] represents the slice ID of the brick index j, and BottomRightBrickIdx[i] represents the index of the lower right brick of the i-th slice.

The variable BottomRightBrickIdx[　i　], NumBricksInSlice[　i　] and BricksToSliceMap[　j　], which specify the brick index of the brick located at the top left corner of the i-th slice, the number of bricks in the i-th slice and the mapping of bricks to slices, are derived as follows:

for( j = 0; i　　=　=　　0　　&&　　j <NumBricksInPic; j++)

BricksToSliceMap[　j　] = -1

NumBricksInSlice[　i　] = 0

for( j = TopLeftBrickIdx(i); j　　<　　NumBricksInPic; j--) {

if( BrickColBd[　j　]　　>=　　BrickColBd[　BottomRightIdx[i]　]　　&&

BrickColBd[　j　]　　<=　　BrickColBd[　c[i]]　　&&

BricksToSliceMap[　j　]　　=　=　　-1) {

BottromRightBrickIdx [　i　] = j

NumBricksInSlice[　i　]++

BricksToSliceMap[　j　] = i

}

}

}

Table 10

Brick_idx_delta_sign_flag[i] in Table 10 is a syntax representing the sign value of the difference between the lower right brick index of the i-th slice and the lower right brick index of the i-1th slice.

When i is not 0, the lower right brick index BottomRightIdx[i] of the i-th slice can be derived as shown in the following equation (14).

BottomRightIdx[i-1]+ sign[i]* bottom_right_brick_idx_delta[i] (14)

If the brick_idx_delta_sign_flag[i] value is 1, sign[i] can be set to 1, and if the brick_idx_delta_sign_flag[i] value is 0, sign[i] can be set to -1.

Here, bottom_right_brick_idx_length_minus1 represents the bit length of the syntax bottom_right_brick_idx_delta[i].

loop_filter_across_bricks_enabled_flag is a flag indicating whether to disable the loop filter in the brick boundary (two neighboring tiles or the boundary of a brick).

Table 11

As shown in Table 11, when the single_tile_in_pic_flag value is 0, the uniform_tile_spcaing_flag value is 1, and the brick_splitting_present_flag value is 1, the number of signals to signal the syntax num_tiles_in_pic_minus1 indicating the number of tiles in the picture in the PPS without inducing the number of tiles in the picture have.

The number of tiles in a picture can be expressed as num_tiles_in_pic_minus1+1, and when the tile index is between 0 and num_tiles_in_pic_minus1, brick_split_flag indicating whether each tile is divided into bricks can be signaled.

Table 12

If the value of single_tile_in_pic_flag is 0, it indicates that more than one tile exists in the picture. That is, there are at least two or more tiles. When the single_tile_in_pic_flag value is 0, the uniform_tile_spcaing_flag value is 1, and the brick_splitting_present_flag value is 1, the syntax num_tiles_in_pic_minusN indicating the number of tiles in the picture in the PPS can be signaled here without inducing the number of tiles in the picture. It can represent an integer greater than or equal to 1. Specifically, for example, as shown in Table 12, num_tiles_in_pic_minus2 may be signaled.

The number of tiles in a picture can be expressed as num_tiles_in_pic_minus2+2, and when the tile index is between 0 and num_tiles_in_pic_minus2+1, brick_split_flag indicating whether each tile is divided into bricks can be signaled.

In Table 12, signaled_slice_id_flag is a syntax indicating whether the slice id of each slice is signaled.

slice_id[i] represents the slice id of the i-th slice.

signalled_slice_id_length_minus1 plus 1 is a syntax indicating the number of bits of a slice id (slice_id).

When the current picture is divided into a plurality of slices, the area of the i-th slice can be signaled through bottom_right_brick_idx_delta[i] and brick_idx_delta_sign_flag[i] as shown in Table 10. At this time, when brick_idx_delta_sign_flag[i] of all slices have the same sign (sign), the same sign (sign) information can be signaled only once. For example, if comm_brick_idx_delta_sign is true (1), all brick_idx_delta_sign_flag[i] have the same sign (sign) information, and at this time, comm_brick_idx_delta_sign_flag information is signaled to signal a positive sign (1) and/or a negative sign (0). can do.

For example, pos_sign_brick_delata_flag of Table 13 and/or neg_sign_brick_delata_flag of Table 14 may be signaled whether all sign (sign) flag information is positive and/or negative as shown in Table 13 and/or Table 14.

Table 13

As shown in Table 13, a flag pos_sign_brick_delta_flag indicating whether the brick_idx_delta_sign_flag[i] values in the picture are all 1 may be signaled.

Brick_idx_delta_sign_flag[i] may be signaled only when the pos_sign_brick_delta_flag value is 0.

If the pos_sign_brick_delta_flag value is 1, it indicates that all brick_idx_delta_sign_flag[i] values in the picture are 1.

If the pos_sign_brick_delta_flag value is 0, it indicates that the value of brick_idx_delta_sign_flag[i] is not 1 in at least one i (at least one tile) in the picture.

Table 14

As shown in Table 14, a flag neg_sign_brick_delta_flag indicating whether the brick_idx_delta_sign_flag[i] values in the picture are all 1 may be signaled.

Brick_idx_delta_sign_flag[i] may be signaled only when the neg_sign_brick_delta_flag value is 0.

If the neg_sign_brick_delta_flag value is 1, it indicates that all brick_idx_delta_sign_flag[i] values in the picture are 0.

If the value of neg_sign_brick_delta_flag is 1, it indicates that the value of brick_idx_delta_sign_flag[i] is not 0 in at least one i (at least one tile) in the picture.

In order to indicate the bit length of the syntax bottom_right_brick_idx_length (hereinafter, the lower right brick index delta bit length), the difference value may be signaled using information of a tile and/or slice and/or brick as a predicted value. When signaling the difference value, the difference value including the sine (sign) can be signaled, or the sine (sign) flag and the absolute value of the difference value can be signaled.

At this time, the tile information means that at least one of the number of tiles of the current and/or previous picture, the number of tile columns, and the number of tile rows is individually used as a prediction value, or a combination of at least one or more information (average value, The bit length is obtained from sum, product, maximum value, minimum value, and median value), and can be used as a predicted value of the current picture bottom_right_brick_idx_length bit length.

In this case, the slice information may be used as a predicted value of the bottom_right_brick_idx_length bit length of the current picture by obtaining a bit length from the number of current and/or previous picture slices.

Here, the brick information may be used as a predicted value of the bottom_right_brick_idx_length bit length of the current picture by obtaining a bit length from the number of bricks in the previous picture.

In this case, a combination of slice, tile, and brick information may be used as a predicted value of the bottom_right_brick_idx_length bit length of the current picture. For example, a bit length for a maximum value among the number of slices and/or tiles of the current and/or previous picture may be used as a predicted value of the bottom_right_brick_idx_length bit length of the current picture.

At this time, the bit length may be expressed as FLC (Fixed Length Code) length.

Table 15

As shown in Table 15, information of the syntax bottom_right_brick_idx_length_delta (hereinafter, the difference value of the binary bit length that can represent the range that the lower right brick index delta value can have) can use information on the number of tiles in a picture as a predicted value.

For example, a variable NumTileInPic representing the number of tiles in a picture or syntax num_tiles_in_pic_minus1, syntax num_tiles_in_pic_minus2, num_tile_columns_minus1, num_tile_columns_minus1, num_tile_columns_minus1, num_tile_rows_minus1 Can be used as.

For example, as shown in Table 15, bottom_right_brick_idx_length_delta indicating the bit length of bottom_right_brick_idx_delta[i] may be set as log2 (NumBricksInPic)-log2 (NumTilesInPic). Alternatively, bottom_right_brick_idx_length_delta may be set as log2 (NumBricksInPic)-log2 (num_tiles_in_pic_minus1). Alternatively, bottom_right_brick_idx_length_delta may be set as log2 (NumBricksInPic)-log2 (num_tiles_in_pic_minus2).

Table 16

As shown in Table 16, the information of the syntax bottom_right_brick_idx_length_delta (hereinafter, the difference value of the binary bit length that can represent the range that the lower-right brick index delta value can have) is the number of slices in the picture (variable NumSliceInPic indicating the number of slices in the picture, and /Or the syntax num_slices_in_pic_minus1 and/or the syntax num_slices_in_pic_minus2) may be used as a predicted value.

For example, a variable NumSliceInPic indicating the number of slices in a picture can be used as a predicted value of the delta bit length of the lower right brick index to indicate the delta bit length of the lower right brick index.

For example, as shown in Table 16, bottom_right_brick_idx_length_delta indicating the bit length of bottom_right_brick_idx_delta[i] may be set as log2 (NumBricksInPic)-log2 (NumSliceInPic). Alternatively, bottom_right_brick_idx_length_delta may be set as log2 (NumBricksInPic)-log2 (num_slices_in_pic_minus1). Alternatively, bottom_right_brick_idx_length_delta may be set as log2 (NumBricksInPic)-log2 (num_slices_in_pic_minus2).

Table 17

The syntax bottom_right_brick_idx_length_delta indicating the bit length of the syntax bottom_right_brick_idx_delta[i] can be derived using at least one of information indicating the number of slices in a picture (NumSliceInPic) and/or information indicating the number of tiles.

Since the number of bricks is equal to or greater than the number of tiles, and the number of bricks is equal to or greater than the number of slices, max (number of tiles, number of slices) can be used as a predicted value of bottom_right_brick_idx_length_delta. At this time, max(A, B) means a larger value among A and B, and the number of tiles and/or slices can use the explicitly signaled syntax value, or the variable NumTilesInPic and/or NumSliceInPic values. You can induce it.

For example, in Table 17, bottom_right_brick_idx_length_delta may be signaled as log2 (NumBricksInPic)-log2 (max (number of tiles, number of slices)).

Table 18

As shown in Table 18, information on the number of slices in a picture may be signaled using information on the number of tiles in a picture as a predicted value. For example, a syntax num_slices_in_pic_delta value representing a value obtained by dividing information on the number of tiles in a picture from the number of slices in a picture (hereinafter, tile slice difference) may be signaled, and at this time, a signal may be signaled as an absolute value of the tile slice difference. .

In this case, the syntax num_slices_in_pic_delta_sign_flag indicating the sign value of the tile slice difference may be signaled. If the num_slices_in_pic_delta_sign_flag value is 1, it indicates that the tile slice difference is positive, and if the num_slices_in_pic_delta_sign_flag value is 0, it indicates that the tile slice difference is negative.

Table 19

As shown in Table 19, information on the number of slices in a picture may be derived based on bottom_right_brick_idx_length_minus1.

1<< bottom_right_brick_idx_length_minus1 is called the number of induction of the lower right brick.

For example, a syntax num_slices_in_pic_delta value representing a value obtained by differentiating the number of lower-right bricks derived from the number of slices in the picture (hereinafter, the difference between the lower-right bricks of a tile) may be signaled. num_slices_in_pic_delta represents the absolute value of the tile slice difference.

In this case, the syntax num_slices_in_pic_delta_sign_flag indicating the sign value of the brick difference at the lower right of the tile may be signaled. When the value of num_slices_in_pic_delta_sign_flag is 1, it indicates that the difference at the bottom right of the tile is positive, and when the value of num_slices_in_pic_delta_sign_flag is 0, it indicates that the difference at the bottom right of the tile is negative.

Table 20

Table 21

The number of tiles in the picture is one, and may not be divided into bricks. The flag single_brick_in_pic_flag indicating this case may be signaled to the PPS as shown in Table 21.

If the single_brick_in_pic_flag value is 1, it indicates that the number of bricks in the picture is one, and if the single_brick_in_pic_flag value is 0, it indicates that the number of bricks in the picture is at least one.

Single_brick_in_pic_flag as shown in Table 21 Only when the value is 0, a single_tile_in_pic_flag may be signaled.

single_tile_in_pic_flag When the value is 0 or the single_brick_in_pic_flag value is 0, the uniform_tile_spacing_flag may be signaled.

single_tile_in_pic_flag If the value is 0 and the value of single_brick_in_pic_flag is 0, the syntax brick_splitting_present_flag, num_tiles_in_pic_minus1/ num_tiles_in_pic_minus2, brick_split_flag[ i ], uniform_brick_spacing_flag_us_minight_[i], brick_brick_us1, etc. It can be signaled.

single_tile_in_pic_flag When the value is 1 and the single_brick_in_pic_flag value is 0, as shown in Table 21, the syntax uniform_brick_spacing_flag[ 0 ], brick_height_minus1[ 0 ], num_brick_rows_minus2[ 0 ], brick_row_height_minus1[ 0 ][ j ], etc. can be signaled.

Table 22

As shown in Table 22, only when the single_tile_in_pic_flag value is 1, a single_brick_in_pic_flag may be signaled.

Table 23

As shown in Table 23, when the single_tile_in_pic_flag value is 1, the single_brick_in_pic_flag value is 0, and the single_brick_per_slice_flag value is 0, the syntax rect_slice_flag is not signaled, and the value can be set to 0.

As shown in Table 23, when the single_tile_in_pic_flag value is 1 and the single_brick_in_pic_flag value is 0, the syntax brick_idx_delta_sing_flag may not be signaled, and the value may be set to 0.

Table 24

As shown in Table 24, single_brick_in_pic_flag may be signaled. If there is only one tile in the picture, it can be set not to be further divided into bricks. Single_brick_in_pic_flag if there is only one tile in the picture When the value is set to 1 and there is more than one tile in the picture, the single_brick_in_pic_flag value may be set to 0.

As shown in Table 25, brick and/or slice information may be signaled depending on the case where single_tile_in_pic_flag is false or true.

Table 25

Claims (1)

Predictive coding method using tile, brick and slice information coding