KR20200141696A - Video signal encoding method and apparatus and video decoding method and apparatus - Google Patents

Abstract

Provided are a method and device for encoding a video using affine inter prediction encoding. Therefore, the coding efficiency of a video signal is improved.

Description

Video signal processing method and apparatus TECHNICAL FIELD [Video signal encoding method and apparatus and video decoding method and apparatus]

The present invention relates to a video signal processing method and apparatus.

Video images are compression-encoded by removing spatio-temporal redundancy and inter-view redundancy, which can be stored in a format suitable for a storage medium or transmitted through a communication line.

The present invention is to improve the coding efficiency of a video signal.

In order to solve the above problems, the present invention provides a video encoding method and apparatus using affine inter prediction encoding.

The video signal processing method and apparatus according to the present invention may improve video signal coding efficiency through affine inter prediction coding.

One Basic coding block structure

A picture is divided into a plurality of basic coding units (Coding Tree Units, hereinafter, CTU) as shown in Figure 1 .

Picture 1

It is configured not to overlap with other CTUs. For example, in the entire sequence, the CTU size can be set to 128x128, and any one of 128x128 to 256x256 can be selected and used in picture units.

A coding block/coding unit (CU) may be generated by hierarchically dividing the CTU. Prediction and transformation may be performed in units of coding units, and become a basic unit that determines a prediction coding mode. The prediction encoding mode represents a plurality of methods of generating a predicted image, such as intra prediction (intra prediction, hereinafter, intra prediction), inter prediction (hereinafter, inter prediction), or combined prediction. Can be mentioned. Specifically, for example, a prediction block may be generated by using at least one of intra prediction, inter prediction, and composite prediction in a coding unit unit. In the inter prediction mode, when the reference picture indicates the current picture, a prediction block may be generated in an area within the current picture that has already been decoded. Because a prediction block is generated using a reference picture index and a motion vector, it can be included in inter prediction. Intra prediction is a method of generating a prediction block using information of the current picture, inter prediction is a method of generating a prediction block using information of another picture that has already been decoded, and composite prediction is a method of mixing inter prediction and intra prediction. This is how to use.

The CTU can be divided (partitioned) into a quad tree, binary tree or triple tree as shown in Figure 2 . The divided block can be further divided into a quad tree, binary tree, or triple tree. The method of dividing the current block into 4 square partitions is called quad tree partitioning, the method of dividing the current block into 2 irregular partitions is called binary tree partitioning, and the method of dividing the current block into 3 irregular partitions It is defined as binary tree partitioning.

Binary partitioning in the vertical direction (SPLIT_BT_VER in Figure 2 ) is called vertical binary tree partitioning, and binary tree partitioning in the horizontal direction (SPLIT_BT_HOR in Figure 2 ) is defined as horizontal binary tree partitioning.

Triple-tree partitioning in the vertical direction (SPLIT_TT_VER in Figure 2 ) is called vertical triple-tree partitioning, and triple-tree partitioning in the horizontal direction (SPLIT_TT_HOR in Figure 2 ) is defined as horizontal triple-tree partitioning.

The number of additional divisions is called partitioning depth, and the maximum value of the partitioning depth for each sequence or picture/tile set can be set differently, and different partitioning depths according to the partitioning tree type (quad tree/binary tree/triple tree) It can be set to have, and a syntax indicating this can be signaled.

Picture 2

The divided coding block may be further divided by a method such as quad partitioning, binary partitioning, or multi-partitioning to configure a coding unit, or may configure a coding unit without further partitioning.

Figure 3

As shown in Figure 3 , a coding unit can be hierarchically configured for one CTU, and a coding unit can be partitioned using at least one of a coding unit using binary tree partitioning, quad tree partitioning/triple tree partitioning. This method is defined as multi tree partitioning.

A coding unit generated by dividing an arbitrary coding unit having a partitioning depth of k is referred to as a lower coding unit, and the partitioning depth is k+1. A coding unit having a partitioning depth k including a lower coding unit having a partitioning depth k+1 is referred to as an upper coding unit.

The partitioning type of the current coding unit may be limited according to the partitioning type of the upper coding unit and/or the partitioning type of the coding units surrounding the current coding unit.

Here, the partitioning type represents an indicator indicating which partitioning is used among binary tree partitioning, quad tree partitioning/triple tree partitioning.

2 How to generate predictive image

In video encoding, a prediction image can be generated by a plurality of methods, and a method of generating a prediction image is called a prediction encoding mode.

The prediction encoding mode may include an intra prediction encoding mode, an inter prediction encoding mode, a current picture reference encoding mode, or a combined prediction mode.

The inter prediction coding mode is a prediction coding mode that generates a prediction block (prediction image) of the current block using information of a previous picture, and the intra prediction coding mode is a prediction that generates a prediction block using samples adjacent to the current block. It is called an encoding mode. A prediction block may be generated using an image that has already been reconstructed of the current picture, and this is defined as a current picture reference mode or an intra block copy mode.

A prediction block may be generated using at least two or more of an inter prediction encoding mode, an intra prediction encoding mode, or a current picture reference encoding mode, which is referred to as a combined prediction mode.

2.1 Intra prediction coding method

In intra prediction, as shown in Fig. 4 , an already coded boundary sample around the current block is used to generate intra prediction, and this is defined as an intra reference sample.

The average value of the intra reference samples is predicted by setting the value of all samples in the prediction block (DC mode), or the vertical direction prediction sample generated by weighted prediction of the horizontal direction prediction sample and the vertical direction reference sample generated by performing horizontal reference weighted prediction After generation, a prediction sample may be generated by weighted prediction of a horizontal direction prediction sample and a vertical direction prediction sample (Planar mode), or intra prediction may be performed using a directional intra prediction mode.

Figure 4

As shown in the left figure of Figure 5 , intra prediction can be performed using 33 directions (a total of 35 intra prediction modes), and 65 directions (a total of 67 intra prediction modes) as shown in the right figure of Figure 5 May be. When directional intra prediction is used, an intra reference sample (reference reference sample) may be generated in consideration of the direction of the intra prediction mode, and intra prediction may be performed therefrom.

An intra reference sample on the left side of the coding unit is called a left intra reference sample, and an intra reference sample on the upper side of the coding unit is called an upper intra reference sample.

When directional intra prediction is performed, as shown in Table 1, an intra direction parameter (intraPredAng), which is a parameter indicating a prediction direction (or a prediction angle), may be set according to the intra prediction mode. Table 1 below is only an example based on a directional intra prediction mode having a value of 2 to 34 when 35 intra prediction modes are used. It goes without saying that the prediction directions (or prediction angles) of the directional intra prediction modes are further subdivided, so that more than 33 directional intra prediction modes can be used.

Picture 5

Table 1

Figure 6

When intraPredAng is negative (for example, when the intra prediction mode index is between 11 and 25), as shown in Figure 6 , the left intra reference sample and the upper intra reference sample are configured in 1D according to the angle of the intra prediction mode. It can be reconstructed as a one-dimensional reference sample (Ref_1D).

Figure 7

When the intra prediction mode index is between 11 and 18, a one-dimensional reference sample can be generated in a counterclockwise direction from an intra reference sample located on the upper right side of the current block to an intra reference sample located on the lower left side as shown in Figure 7 .

In other modes, a one-dimensional reference sample may be generated using only the upper side intra reference sample or the left side intra reference sample.

Figure 8

When the intra prediction mode index is between 19 and 25, a one-dimensional reference sample can be generated in a clockwise direction from an intra reference sample located at the lower left side of the current block to an intra reference sample located at the right side of the current block, as shown in Figure 8 .

A weight-related parameter ifact applied to at least one reference sample determined based on the reference sample determination index iIdx and iIdx may be derived as in Equations (1) to (2) below. iIdx and i _fact are variably determined according to the slope of the directional intra prediction mode, and a reference sample specified by iIdx may correspond to an integer pel.

iIdx = (y+1) * P _ang /32 (One)

i _fact = [(y+1) * P _ang ] & 31 (2)

The prediction image may be derived by specifying at least one one-dimensional reference sample for each prediction sample. For example, a position of a one-dimensional reference sample that can be used to generate a prediction sample may be specified in consideration of a slope value of a directional intra prediction mode. Each prediction sample may have a different directional intra prediction mode. A plurality of intra prediction modes may be used for one prediction block. The plurality of intra prediction modes may be represented by a combination of a plurality of non-directional intra prediction modes, a combination of one non-directional intra prediction mode and at least one directional intra prediction mode, or a plurality of directional intra prediction modes. It can also be expressed as a combination of modes. A different intra prediction mode may be applied for each predetermined sample group in one prediction block. A predetermined sample group may consist of at least one sample. The number of sample groups may be variably determined according to the size/number of samples of the current prediction block, or may be a fixed number pre-set in the encoder/decoder independently of the size/number of samples of the prediction block.

Specifically, for example, the position of the one-dimensional reference sample may be specified using the reference sample determination index iIdx.

When the slope of the intra prediction mode cannot be expressed using only one one-dimensional reference sample according to the slope of the intra prediction mode, a first prediction image may be generated by interpolating adjacent one-dimensional reference samples as shown in Equation (3). When the angular line according to the slope/angle of the intra prediction mode does not pass the reference sample located in the integer pel, the first prediction image may be generated by interpolating the reference sample adjacent to the left/right or above/below the corresponding angular line. . The filter coefficient of the interpolation filter used at this time may be determined based on i _fact . For example, the filter coefficient of the interpolation filter may be derived based on a distance between a fractional pel located on an angular line and a reference sample located on the integer pel.

P(x,y) = ((32-i _fact )/32)* Ref_1D(x+iIdx+1) + (i _fact /32)* Ref_1D(x+iIdx+2) (3)

When the slope of the intra prediction mode can be expressed with only one one-dimensional reference sample (when i _fact is 0), the first prediction image can be generated as shown in Equation (4) below.

P(x,y) = Ref_1D(x+iIdx+1) (4)

2.2 Wide-angle intra prediction coding method

The prediction angle of the directional intra prediction mode may be set between 45 and -135 as shown in Figure 9 .

Figure 9

When the intra prediction mode is performed in the amorphous coding unit, a disadvantage of predicting a current sample from an intra reference sample far from the current sample instead of an intra reference sample close to the current sample may occur due to a predefined prediction angle.

For example, as shown in the left figure of Figure 10 , in a coding unit whose width is larger than the height of the coding unit (hereinafter, a horizontal coding unit), intra prediction is performed at a far distance L instead of a close sample T. I can. As another example, as shown in the right figure of Figure 10, in a coding unit whose height is greater than the width of the coding unit (hereinafter, the vertical coding unit), the intra-distance sample T from the distant sample T is You can make predictions.

Figure 10

In the amorphous coding unit, intra prediction may be performed at a prediction angle wider than a predefined prediction angle, and this is defined as a wide angle intra prediction mode.

내지

의 예측 각도를 가질 수 있으며, 기존 인트라 예측 모드에서 사용된 각도를 벗어나는 예측 각도를 와이드 앵글 각도라고 정의 한다. The wide angle intra prediction mode

To

It can have a prediction angle of, and a prediction angle that deviates from the angle used in the existing intra prediction mode is defined as a wide angle angle.

In the left figure of Fig. 10 , sample A in the horizontal direction coding unit can be predicted from the intra reference sample T using the wide-angle intra prediction mode.

In the right figure of Fig. 10 , sample A in the vertical direction coding unit can be predicted from the intra reference sample L using the wide-angle intra prediction mode.

N+M intra prediction modes may be defined by adding M wide angle angles to N existing intra prediction modes. Specifically, as shown in Table 2, a total of 95 intra prediction modes may be defined by adding 28 wide angle angles to 67 intra modes.

The intra prediction mode that the current block can use may be determined according to the shape of the current block. For example, 65 directional intra prediction modes may be selected from among 95 directional intra prediction modes based on at least one of a size of a current block, an aspect ratio (eg, a ratio of width and height), and a reference line index.

Table 2

The intra prediction mode angle shown in Table 2 may be adaptively determined based on at least one of a shape of a current block and a reference line index. For example, the intraPredAngle of Mode 15 may be set to have a larger value when the current block is a square than when the current block is an amorphous. Alternatively, the intraPredAngle of Mode 75 may be set to have a larger value when a non-adjacent reference line is selected than when an adjacent reference line is selected.

When using the wide-angle intra prediction mode, the length of the upper intra reference sample can be set to 2W+1 and the length of the left intra reference sample can be set to 2H+1, as shown in Figure 11 .

Figure 11

In the case of using wide-angle intra prediction, when the intra prediction mode of the wide-angle intra prediction mode is encoded, the number of intra prediction modes increases, and thus encoding efficiency may be lowered. The wide-angle intra prediction mode can be encoded by replacing the existing intra prediction mode that is not used in the wide-angle intra prediction mode, and the replaced prediction mode is called a wide-angle replacement mode. The wide angle replacement mode may be an intra prediction mode in a direction opposite to the wide angle intra prediction mode.

Specifically, for example, when 35 intra predictions are used as shown in Figure 12 , wide-angle intra prediction mode 35 can be encoded using intra prediction mode 2, which is a wide-angle replacement mode, and wide-angle intra prediction mode 36 is used for wide-angle replacement. It can be encoded in intra prediction mode 3, which is a mode.

Figure 12

The replacement mode and number may be differently set according to the shape of the coding block or the ratio of the height to the width of the coding block. Specifically, for example, Table 3 shows intra prediction modes used according to the ratio of the width and height of a coding block.

Table 3

After generating the intra prediction image, the intra prediction image may be updated for each sample based on the sample position, and this is defined as a sample position-based intra-weighted prediction method (Position dependent prediction combination, PDPC).

2.3 Multi-line intra prediction coding method

As shown in Figure 13 below, intra prediction may be performed on a plurality of intra reference lines, which is called a multi-line intra prediction coding method.

Fig. 13

Intra prediction may be performed by selecting any one of a plurality of intra reference lines composed of an adjacent intra parallel line and a non-adjacent intra reference line, and this is called a multi-line intra prediction method. The non-adjacent intra reference line is a first non-adjacent intra reference line (non-adjacent reference line index 1), a second non-adjacent intra reference line (non-adjacent reference line index 2), and a third non-adjacent intra reference line (non-adjacent reference line). It can be composed of index 3). Only some of the non-adjacent intra reference lines may be used. For example, only a first non-adjacent intra reference line and a second non-adjacent intra reference line may be used, or only a first non-adjacent intra reference line and a third non-adjacent intra reference line may be used.

An intra reference line index (intra_luma_ref_idx), a syntax specifying a reference line used for intra prediction, may be signaled in units of coding units.

Specifically, when using the adjacent intra reference line, the first non-adjacent intra reference line, and the third non-adjacent intra reference line, intra_luma_ref_idx may be defined as shown in Table 4 below.

Table 4

When a non-adjacent intra reference line is used, it may be set not to use a non-directional intra prediction mode. That is, when a non-adjacent intra reference line is used, it may be limited not to use the DC mode or the planar mode.

The number of samples belonging to the non-adjacent intra reference line may be set to be larger than the number of samples of the adjacent intra parallel line. Also, the number of samples of the i+1th non-adjacent intra reference line may be set to be greater than the number of samples of the ith non-adjacent intra reference line. The difference between the number of upper samples of the i-th non-adjacent intra reference line and the number of upper samples of the i-1th non-adjacent intra reference line may be expressed as a reference sample number offset offsetX[i].

offsetX[1] represents a difference value between the number of upper samples of the first non-adjacent intra reference line and the number of upper samples of the adjacent intra reference line. The difference between the number of left samples of the i-th non-adjacent intra reference line and the number of left samples of the i-1th non-adjacent intra reference line may be expressed as a reference sample number offset offsetY[i]. OffsetY[1] represents a difference between the number of left samples of the first non-adjacent intra reference line and the number of left samples of the adjacent intra reference line.

A non-adjacent intra reference line with an intra reference line index of i may be composed of an upper non-adjacent reference line refW + offsetX[i], a left non-adjacent reference line refH+ offsetY[i], and an upper left sample. The non-adjacent intra reference line The number of included samples may consist of refW + refH + offsetX[i] + offsetY[i] +1.

refW =2* nTbW (5)

refH =2*　*nTbH (6)

In equations (5) to (6), nTbW represents the width of the coding unit, nTbH represents the height of the coding unit, and whRatio can be defined as in the following equation (7).

whRatio = log2(nTbW/nTbH) (7)

In the multi-line intra prediction coding method, when a non-adjacent intra reference line is used, the wide-angle intra mode may be set not to be used. Alternatively, if the MPM mode of the current coding unit is a wide-angle intra mode, the multi-line intra prediction coding method may be set not to be used.

In this case, a non-adjacent intra reference line with an intra reference line index of i may consist of an upper non-adjacent reference line W + H + offsetX[i], a left non-adjacent reference line H + W + offsetY[i], and an upper left sample. , The number of samples belonging to the non-adjacent intra reference line may be composed of 2W + 2H + offsetX[i] + offsetY[i] +1, and the values of offsetX[i] and offsetY[i] may vary depending on the whRatio value. have.

For example, if the whRatio value is greater than 1, the offsetX[i] value can be set to 1 and the offsetY[i] value can be set to 0. If the whRatio value is less than 1, the offsetX[i] value can be set to 0, offsetY[ i] You can also set the value to 1.

2.5 Inter prediction coding and decoding method

A method of generating a prediction block (prediction image) of a block in a current picture using information of a previous picture is called an inter prediction encoding mode.

A prediction block may be generated from a specific block in a previous picture of the current block.

In the encoding step, the block with the smallest reconstruction error among the blocks in the previous picture can be selected by searching around a collocated block, and the x-axis difference and y-axis difference between the upper left sample of the current block and the upper left sample of the selected block Is defined as a motion vector, and can be signaled by transmitting it to a bitstream. A block generated through interpolation from a specific block of a reference picture specified by a motion vector is called a motion compensated predictor block.

As shown in Figure 14 , the collocated block represents the block of the picture with the same location and size of the current block and the upper left sample. The corresponding picture may be specified from the same syntax as the reference picture reference.

Figure 14

In the inter prediction encoding mode, a prediction block may be generated in consideration of the motion of an object.

For example, if you know how and in which direction an object in the previous picture has moved in the current picture, a prediction block (prediction image) can be generated by differentiating a block considering motion from the current block, and this is defined as a motion prediction block. .

A residual block may be generated by differentiating a motion prediction block or a corresponding prediction block from the current block.

When motion occurs in an object, if a motion prediction block is used rather than a corresponding prediction block, the energy of the residual block decreases, and compression performance may be improved.

This method of using a motion prediction block is called motion compensation prediction, and most inter prediction coding uses motion compensation prediction.

A value indicating in which direction and to what extent the object in the previous picture has moved in the current picture is called a motion vector. As the motion vector, motion vectors having different pixel precisions may be used in units of sequences, groups of tiles, or blocks. For example, the pixel precision of a motion vector in a specific block may be at least one of octor-pel, quarter-pel, half-pel, integer pel, and 4 integer pel.

As the inter prediction mode, an inter prediction method using translaton motion and an affine inter prediction method using affine motion may be selectively used.

2.4.1 Afine inter prediction coding and decoding method

There are many cases in which the motion of a specific object does not appear linearly in a video. For example, as shown in Figure 15 , in an image using affine motion such as zoom-in, zoom-out, rotation, and affine transformation that enables transformation into arbitrary shapes, the object's If only translation motion vectors are used for motion, the motion of an object cannot be effectively expressed, and coding performance may be degraded.

Figure 15

Affine motion can be expressed as the following equation (8).

(8)

(8)

Expressing affine motion using a total of six parameters is effective for images with complex motion, but encoding efficiency may be degraded due to the large number of bits used to encode affine motion parameters.

Accordingly, affine motion can be simplified and expressed with four parameters, and this is defined as a four-parameter afine motion model. Equation (9) expresses afine motion with four parameters.

(9)

(9)

The four-parameter affine motion model may include motion vectors at two control points of the current block. The control point may include an upper left corner, an upper right corner, and a lower left corner of the current block. As an example, as shown in the left figure of Figure 16 , the 4-parameter affine motion model is determined by the motion vector sv0 at the upper left sample (x0,y0) of the coding unit and the motion vector sv1 at the upper right sample (x1,y1) of the coding unit. Can be determined, and sv ₀ and sv ₁ are defined as affine seed vectors. Hereinafter, it is assumed that an affine seed vector sv ₀ located in the upper left corner is a first affine seed vector, and an affine seed vector sv ₁ located in the upper right corner is a second affine seed vector. In the 4-parameter affine motion model, it is possible to replace one of the 1/2 affine seed vectors with the affine seed vector located in the lower left corner.

Fig 16

The 6-parameter affine motion model is an afine motion model in which the motion vector sv ₂ of the residual control point (e.g., sample (x2,y2) in the lower left corner) is added to the 4-parameter affine motion model as shown in the right figure of Figure 16 . . Hereinafter, assuming that the affine seed vector sv ₀ located at the top left is the first affine seed vector, the affine seed vector sv ₁ located at the top right is the second affine seed vector, and the affine seed vector located at the bottom left Assume that sv ₂ is the third affine seed vector.

Information on the number of parameters for expressing affine motion may be encoded in the bitstream. For example, a flag indicating whether 6 parameters are used and a flag indicating whether 4 parameters are used may be encoded in a tile group, a tile, a coding unit, or a CTU unit. Accordingly, a 4-parameter affine motion model to a 6-parameter affine motion model may be selectively used in units of tile groups, coding units, or CTUs.

A motion vector can be derived for each sub-block of the coding unit using the affine seed vector as shown in Figure 17, and this is defined as an affine sub-block vector. _

Figure 17

The affine sub-block vector can also be derived as in Equation (10) below. Here, the reference sample position (x,y) of the sub-block may be a sample located at a corner of the block (eg, an upper left sample), or a sample (eg, a center sample) in which at least one of the x-axis or y-axis is centered. .

(10)

(10)

Motion compensation may be performed in units of coding units or sub-blocks within the coding unit using the afine sub-block vector, and this is defined as an afine inter prediction mode. In Equation (10), (x ₁ -x ₀ ) may be the same as the width of the coding unit and the unit width w, or may be set to w/2 or w/4.

The affine seed vector and the reference picture list of the current coding unit can be derived by using the afine motion vector (afine subblock vector or afine seed vector) of the neighboring block of the current coding unit, and this is an afine merge mode. Is called.

Fig. 18

In the afine merge mode, at least one of the neighboring blocks of the current coding unit (e.g., A, B, C, D, E in Fig. 18) is an afine mode (afine inter prediction mode or afine merge mode). In the case of encoding, the affine seed vector of the current block can be derived from the affine motion vector of the neighboring block. A block encoded in the affine mode among neighboring blocks is called an affine neighboring block.

The search order of the neighboring blocks follows a predefined order such as A --> B --> C --> D --> E, and the affine seed of the current block from the neighboring block, which is the first affine in the search order. A vector may be derived or a neighboring block, which is an affine of one of the neighboring blocks, may be selected, and the index of the selected neighboring block may be encoded.

Figure 19

Alternatively, up to two affine seed vectors may be derived from neighboring blocks A0, A1, B0, B1, and B2 of the current coding unit. As an example, an affine seed vector can be derived from one of {A0,A1} among the neighboring blocks in Figure 19, and another affine seed vector can be derived from {B0, B1, B2}. This is called affine merge mode.

As shown in Figure 20, from the seed vectors nv ₀ = (nv _0x , nv _0y ) and nv ₁ = (nv _1x , nv _1y ) of the affine surrounding blocks, the third seed vector nv ₂ = (nv _2x , nv _2y ) can be derived. I can.

Figure 20

The third seed vector of the neighboring block of the affine can be derived as shown in Equation (11) below.

(11)

(11)

The seed vector of the current block can be derived using the first affine seed vector, the second affine seed vector, and the third seed vector of the neighboring block as shown in Equations (12) to (13) below.

(12)

(12)

(13)

(13)

Alternatively, the seed vector of the current block may be derived using two of the first affine seed vector, the second affine seed vector, and the third seed vector of the neighboring block.

Fig. 21

At least one of the first seed vector, the second seed vector, and the third seed vector of the neighboring block may be replaced with a motion vector of a sub-block located at the bottom of the sub-block of the affine neighboring block. As an example, as shown in Figure 21, the lower left sub-block or the lower center sub-block (hereinafter, the fourth affine sub-block) among blocks surrounding the affine, and the lower-right sub-block or the lower center sub-block (hereinafter, 5 Affine sub-block) can be used to specify a specific sample location, and an affine seed vector can be derived using a specific sample location. This specific sample position is called the position of the affine reference sample.

The positions of the fourth affine sub-block and the fifth affine sub-block may be set differently depending on whether the target block is in an upper CTU of the current block. For example, if the target block exists in the upper CTU of the current block (the upper CTU of the CTU to which the current block belongs), the 4th affine subblock is set as the lower center subblock, and the 5th affine sub The block can also be set as a lower-right sub-block. At this time, the difference between the fourth affine reference sample (the reference sample of the fourth affine sub-block) and the fifth affine reference sample (the reference sample of the fifth affine sub-block) may be set to be a power series (2 ⁿ ). For example, it can be set to be nbW/2. Here, nbW can also be set as the width of the surrounding block.

At least one of the first seed vector, the second seed vector, and the third seed vector of the neighboring block may be replaced with a motion vector of a sub-block located at the bottom of the sub-block of the affine neighboring block. As an example, as shown in Figure 21, the lower left sub-block or the lower center sub-block (hereinafter, the fourth affine sub-block) among blocks surrounding the affine, and the lower-right sub-block or the lower center sub-block (hereinafter, 5 Affine sub-block) can be used to specify a specific sample location, and an affine seed vector can be derived using a specific sample location. This specific sample position is called the position of the affine reference sample.

The positions of the fourth affine sub-block and the fifth affine sub-block may be set differently depending on whether the target block is in an upper CTU of the current block. For example, if the target block exists in the upper CTU of the current block (the upper CTU of the CTU to which the current block belongs), the 4th affine subblock is set as the lower center subblock, and the 5th affine sub The block can also be set as a lower-right sub-block. At this time, the difference between the fourth affine reference sample (the reference sample of the fourth affine sub-block) and the fifth affine reference sample (the reference sample of the fifth affine sub-block) may be set to be a power series (2 ⁿ ). For example, it can be set to be nbW/2. Here, nbW can also be set as the width of the surrounding block.

Figure 22

If the target block exists in the left CTU of the current block (the CTU to the left of the CTU to which the current block belongs), the fourth affine subblock is set as the lower left subblock, and the fifth affine subblock is set as the lower right subblock. May be.

Figure 23

For another example, if the target block exists in the upper CTU of the current block (the upper CTU of the CTU to which the current block belongs), the fourth affine sub-block is set as the lower left sub-block, as shown in Figure 23, and The in sub-block can also be set as the lower center sub-block. At this time, the difference between the fourth affine reference sample (the reference sample of the fourth affine sub-block) and the fifth affine reference sample (the reference sample of the fifth affine sub-block) may be set to be a power series (2 ⁿ ). For example, it can be set to be nbW/2. Here, nbW can also be set as the width of the surrounding block.

The affine reference sample position may be set to the upper left block or the lower right block of the sub-block, or may be set to a sample located at the center. Alternatively, the fourth affine sub-block and the fifth affine sub-block may be set to have different affine reference sample positions. Specifically, for example, as shown in the left figure of Figure 24, the affine reference sample position of the fourth affine sub-block (hereinafter, the fourth affine reference sample position) can be set as the upper left sample within the sub-block, and the fifth affine The affine reference sample position (hereinafter, the fifth affine reference sample position) of the in sub-block may be set as the right sample (x _i +1, y _i ) of the upper right sample (x _i , y _i ) in the sub-block.

In this case, the reference position of the fifth affine sub-block may be set to a right sample of the upper-right sample, and the motion vector of the fifth affine sub-block may be set as a motion vector of the upper-right sample. Alternatively, the motion vector of the right sample of the upper right sample may be set as the motion vector of the fifth affine sub-block.

For another example, as shown in the right figure of Figure 24, the 4th affine reference sample position is the left sample (x _i -1,y _i ) of the upper right sample (x _i ,y _i ) in the 4th affine subblock. Can be set. In this case, the reference position of the fourth affine sub-block may be set to a left sample of the upper left sample, and the motion vector of the fourth affine sub-block may be set as a motion vector of the upper left sample. Alternatively, the motion vector of the left sample of the upper left sample may be set as the motion vector of the fourth affine sub-block.

Figure 24

For another example, as shown in the left figure of Figure 25, the fourth affine reference sample position can be set as the lower left sample in the sub-block, and the fifth affine reference sample position is the lower right sample in the sub-block (x _j , y _It can be set to the right sample (x _j +1, y _j ) of _j ). In this case, the reference position of the fifth affine sub-block is a right sample of the lower-right sample, but the motion vector of the fifth affine sub-block may be set as a motion vector of the lower-right sample. Alternatively, the motion vector of the right sample of the lower right sample may be set as the motion vector of the fifth affine sub-block.

For another example, as shown in the left figure of Figure 25, the reference sample position of the fourth affine can be set as the left sample (x _j -1, y _j ) of the lower left sample (x _j , y _j ) in the subblock. , The fifth affine reference sample position may be set as the lower right sample within the sub-block.

In this case, the reference position of the fifth affine sub-block may be a left sample of the lower left sample, and the motion vector of the fifth affine sub-block may be set as a motion vector of the lower left sample. Alternatively, the motion vector of the left sample of the lower left sample may be set as the motion vector of the fifth affine sub-block.

Figure 25

For another example, as shown in the left figure of Figure 26, the 4th affine reference sample position can be set to a sample between the upper left sample and the lower left sample (hereinafter, left middle sample) in the subblock, and the 5th affine reference The sample position may be set as a right sample (xk+1, yk) of a sample (hereinafter, right middle sample) (x _k , y _k ) between the upper right sample and the lower right sample in the sub-block. In this case, the reference position of the fifth affine sub-block may be a right sample of the right middle sample, and the motion vector of the fifth affine sub-block may be set as a motion vector of the right middle sample. Alternatively, the motion vector of the right sample of the right middle sample may be set as the motion vector of the fifth affine sub-block.

For another example, as shown in the right figure of Figure 26, the fourth affine reference sample position can be set to the left sample (x _k -1, y _k ) of the left middle sample (x _k , y _k ), and the fifth The affine reference sample position can be set to the right middle sample.

Fig. 26

Alternatively, as shown in equations (14) to (15), the affine motion vector of the fourth affine sub-block/the fifth affine sub-block can be derived from the first seed vector and the second seed vector of the affine neighboring block. , This is called a fourth affine seed vector nv ₃ = (nv _3x , nv _3y ) and a fifth affine seed vector nv ₄ = (nv _4x , nv _4y ), respectively.

(14)

(14)

(15)

(15)

The first affine seed vector and the second affine seed vector of the current block may be derived as shown in Equations (16) to (17) using the fourth affine seed vector and the fifth affine seed vector.

(16)

(16)

(17)

(17)

In equations (16) to (17), x _n4 -x _n3 can be set to Wseed. Wseed is called the subseed vector width. The subseed vector width Wseed can be set to be a power series of 2 (2 ⁿ ).

If the subseed vector width Wseed is not a power series of 2 (2 ⁿ ), equations (16) to (17) can be applied after adding a specific offset so that Wseed is the power series of 2, or a bit shift operation instead of a division operation. Can also be replaced with

When present in the LCU boundary, the affine seed vector is not derived as in Equations (14) to (15), and the translation motion vector information at the position of the fourth affine reference sample is set as the fourth affine seed vector, The translation motion vector information at the position of the fifth affine reference sample may be set as the fifth affine seed vector.

Regardless of whether it exists in the LCU boundary, translation motion vector information at the i-th affine reference sample position may be set as the i-th affine seed vector. Here, the range of i can be set to 1 to 5.

Alternatively, the translation motion vector information at the fourth affine reference sample position may be set as the first affine seed vector, and the translation motion vector information at the fifth affine reference sample position may be set as the second affine seed vector. .

Fig. 27

Alternatively, if the affine reference sample does not belong to the affine neighboring block, motion vector information may be derived from the sample of the affine neighboring block closest to the affine reference sample, and this is called a modified affine merge vector derivation method.

Specifically, for example, as shown in Figure 27, when the fifth affine reference sample (X _n4 , y _n4 ) does not belong to the affine neighboring block, the left boundary sample of the fifth affine reference sample (that is, (X _n4- The translation motion vector information of the block including 1, y _n4 )) may be set as the fifth affine seed vector.

2.4.2 Weighted affine prediction coding method

Fig. 28

A plurality of division patterns can be applied to one coding block. For example, when dividing the coding block into a plurality of sub-blocks as shown in Figure 28, two patterns can be applied. A coding block obtained by dividing a sub-block by a pattern like pattern0 in Figure 28 is called a first pattern coding block, and a coding block by dividing a sub-block by pattern like pattern1 in Figure 28 is called a second pattern coding block. Pattern N and pattern N+1 are the number of horizontal lines dividing the coding block, the number of vertical lines, the height difference between the horizontal lines, the width difference between the vertical lines, the location of the first horizontal line, the location of the first vertical line, and the size of the sub-block created as a result of the division. / At least one of the shape and the number of sub-blocks may be different.

The split pattern applied to the first pattern coding block and the split pattern applied to the second pattern coding block may be predefined by an encoder and a decoder. Alternatively, information on a division pattern applied to the first pattern coding block and/or the second pattern coding block may be signaled through a bitstream. Alternatively, a division pattern applied to the second pattern coding block may be determined based on a division pattern applied to the first pattern coding block.

Using the afine inter prediction encoding method, the affine prediction image P0 for the first pattern coding block (hereinafter, the first pattern affine prediction image) and the afine prediction image P1 for the first pattern coding block (hereinafter, the second Pattern affine prediction image) can be derived.

The motion information used to derive the first pattern affine prediction image and the second pattern affine prediction image are the same. For example, motion vectors of sub-blocks included in each pattern coding block may be derived based on the same CPMVs.

Alternatively, motion information for inducing the first pattern affine prediction image and motion information for inducing the second pattern affine prediction image may be different. As an example, CPMV for inducing a first pattern affine prediction image is derived based on a first motion vector candidate, and CPMV for inducing a second pattern affine prediction image is derived based on a second motion vector candidate. I can.

The prediction sample of the coding block may be obtained by weighted prediction of the first pattern affine prediction image and the second pattern affine prediction image. This is defined as a weighted affine prediction coding mode. The weights used for weighted prediction are called weighted affine predictions.

The weighted affine prediction coding mode may be allowed only when the coding block is unidirectional prediction.

The coding block encoded in the weighted affine prediction coding mode may be set not to use the affine merge vector derivation method or the modified affine merge vector derivation method.

In the weighted affine prediction encoding mode, an affine merge vector derivation method or a modified affine merge vector derivation method may be used.

The weighted affine prediction value can be derived from neighboring blocks adjacent to the current coding block. As an example, when a neighboring block is encoded in a weighted affine prediction coding mode, a weighted affine prediction value of the neighboring block may be set as a weighted affine prediction value of the current coding block. Alternatively, the weighted affine prediction value of the current coding block may be determined based on the gbi_idx of the neighboring block.

Alternatively, information for determining the weighted affine prediction value may be signaled through the bitstream. The information may be an index indicating any one of a plurality of weight sets.

2.4.3 Merge mode coding method

Motion information (motion vector, reference picture index, etc.) of the current coding unit is not encoded, and can be derived from motion information of neighboring blocks. Motion information of any one of the neighboring blocks can be set as motion information of the current coding unit, and this is defined as a merge mode.

Neighboring blocks used for the remaining mode may be a picture 29 remaining candidate indices from 0 to 4 and the block are adjacent to the coding unit as the (current boundary with the abutting block of the coded unit), as shown in Figure 29, the remaining candidate index from 5 to 26 of the It may be a non-adjacent block.

Fig. 29

If the distance of the merge candidate to the current block exceeds a predefined threshold, it may be set as not available.

For example, a predefined threshold may be set to the height of the CTU (ctu_height) or ctu_height+N, and this is defined as a merge candidate available threshold. That is, the y-axis coordinate (y _i ) of the merge candidate and the y-axis coordinate difference (y ₀ ) (i.e., y _i- y ₀ ) of the upper left sample of the current coding unit (hereinafter, the reference sample of the current coding unit) are the merge candidate available threshold If it is larger than the value, the merge candidate may be set as not available. Here, N is a predefined offset value. Specifically, for example, N may be set to 16 or ctu_height may be set.

Fig. 30

When there are many merge candidates crossing the CTU boundary, many unusable merge candidates occur, and encoding efficiency may be lowered. Merge candidates existing above the coding unit (hereinafter, upper merge candidates) may be set as small as possible, and merge candidates existing at the left and lower sides of the coding unit (hereinafter, lower left merge candidates) may be set as many as possible.

As shown in Fig. 30, the difference between the y-axis coordinate of the current coding unit reference sample and the y-axis coordinate of the upper merge candidate may not exceed twice the height of the coding unit.

It is also possible to set so that the difference between the x-axis coordinates of the reference sample of the current coding unit and the x-axis coordinates of the lower left merge candidate as shown in Figure 30 does not exceed twice the width of the coding unit.

A merge candidate adjacent to the current coding unit is called an adjacent merge candidate, and a merge candidate that is not adjacent to the current coding unit is defined as a non-adjacent merge candidate.

A flag isAdjacentMergeflag indicating whether a merge candidate of the current coding unit is an adjacent merge candidate may be signaled.

If the isAdjacentMergeflag value is 1, it indicates that motion information of the current coding unit can be derived from an adjacent merge candidate, and if the isAdjacentMergeflag value is 0, it indicates that the motion information of the current coding unit can be derived from a non-adjacent merge candidate.

2.4.4 Inter decoding area merge method

Motion information (motion vector and reference picture index) of a coding unit already encoded by inter prediction in the current picture can be stored in a list having a predefined size, and this is defined as an inter-region motion information list.

Motion information (motion vector and reference picture index) in the inter-region motion information list is called an inter-decoding region merge candidate.

The inter decoding region merge candidate may be used as a merge candidate of the current coding unit, and this method is defined as an inter decoding region merge method.

When the tile group is initialized, the inter-region motion information list is empty, and when a partial region of a picture is encoded/decoded, it can be added to the inter-region motion information list. The initial inter decoding region merge candidate of the inter region motion information list may be signaled through the tile group header.

When the coding unit is encoded/decoded by inter prediction, motion information of the coding unit may be updated in the inter-region motion information list as shown in Fig. 31 . If the number of inter-decoding region merge candidates in the inter-region motion information list is the maximum value, the value with the smallest inter-region motion information list index (first, the inter-decoding region merge candidate in the inter-region motion information list) is removed. , The motion vector of the most recently encoded/decoded inter-region may be added as an inter-decoding region merge candidate.

Fig. 31

The motion vector mvCand of the decoded coding unit may be updated in the inter-region motion information list HmvpCandList. At this time, if the motion information of the decoded coding unit is the same as any one of the motion information in the inter-domain motion information list (in the case of all the motion vector and the reference index) there does not update the list of inter-area motion information, and figure 33 The motion vector mvCand of the coding unit decoded together may be stored at the end of the inter-region motion information list. At this time, if the index of HmvpCandList that has the same motion information as mvCand is hIdx, HMVPCandList [i] can be set to HVMPCandList[i-1] for all i larger than hIdx as shown in Figure 33 . When sub-block merge candidates are used in the currently decoded coding unit, motion information of the representative sub-block in the coding unit may be stored in the inter-region motion information list.

As an example, the representative subblock in the coding unit is shown in Figure 32 . Likewise, it may be set as the upper left sub-block in the coding unit or as the middle sub-block in the coding unit.

Fig. 32

Fig. 33

A total of NumHmvp motion information (motion vector and reference picture index) can be stored in the inter-region motion information list, and NumHmvp is defined as the size of the inter-region motion information list.

The size of the inter-region motion information list may use a predefined value. The inter-region motion information list size may be signaled in the tile group header. For example, the size of the inter-region motion information list may be defined as 16, may be defined as 6, or may be defined as 5.

In a coding unit that is inter prediction and has an affine motion vector, it may be limited not to have an inter-region motion information list.

Alternatively, in the case of inter prediction and having an affine motion vector, the affine sub-block vector may be added to the inter-region motion information list. In this case, the position of the sub-block may be set to the upper left or upper right, or the center sub-block.

Alternatively, the average motion vector value of each control point may be added to the inter-region merge candidate list.

When the motion vector MV0 derived by encoding/decoding a specific coding unit is the same as any one of the inter-decoding region merge candidates, MV0 may not be added to the inter-region motion information list. Alternatively, an existing inter-decoding region merge candidate having the same motion vector as MV0 may be deleted, and the index allocated to MV0 may be updated by newly including MV0 in the decoding region merge candidate.

In addition to the inter-region motion information list, an inter-region motion information long term list HmvpLTList may be configured. The inter-region motion information long term list size may be set to be the same as the inter-region motion information list size, or may be set to a different value.

The inter-region motion information long term list may be composed of an inter-decoding region merge candidate initially added to the tile group start position. After the inter-region motion information long term list is configured with all available values, the inter-region motion information list may be configured, or motion information in the inter-region motion information list may be set as motion information of the inter-region motion information long term list.

In this case, the inter-region motion information long-term list configured once may be updated again when no update is performed, a decoded region of the tile group is more than half of the entire tile group, or may be set to be updated every m CTU lines. The inter-region motion information list may be updated whenever the inter-region is decoded, or may be set to be updated in units of CTU lines.

The motion information and partition information or type of the coding unit may be stored in the inter-region motion information list. The inter-decoding region merging method may be performed using only inter-region merge candidates having a similar partition information and shape as the current coding unit.

Alternatively, an inter-region merge candidate list may be individually configured according to the block type. In this case, one of a plurality of inter-region merge candidate lists may be selected and used according to the shape of the current block.

Fig. 34

As shown in Figure 34 , it can be composed of an inter-area affine motion information list and an inter-area motion information list. When the decoded coding unit is in affine inter or afine merge mode, the first affine seed vector and the second affine seed vector may be stored in the inter-region afine motion information list HmvpAfCandList. Motion information in the inter-region affine merge candidate list is called an inter-decoding region affine merge candidate.

The merge candidates usable in the current coding unit may be configured as follows, and may have the same search order as the configuration order.

1. Spatial merge candidate (coding block adjacent merge candidate and coding block non-adjacent merge candidate)

2. Temporal merge candidate (merge candidate derived from previous reference picture)

3. Inter-decoding domain merge candidate

4. Inter decoding domain affine merge candidate

5. Zero motion merge candidate

First, the merge candidate list mergeCandList may be composed of a spatial merge candidate and a temporal merge candidate. The number of available spatial merge candidates and temporal merge candidates is defined as the number of available merge candidates (NumMergeCand). When the number of available merge candidates is less than the maximum allowable merge number, an inter-decoding region merge candidate may be added to the merge candidate list mergeCandList.

When adding the inter-region motion information list HmvpCandList to the merge candidate list mergeCandList, it may be checked whether motion information of the inter-decoding region merge candidate in the inter-region motion information list is the same as the motion information of the existing merge list mergeCandList. If the motion information is the same, it is not added to the merge list mergeCandList, and if the motion information is not the same, the inter-decoding region merge candidate may be added to the merge list mergeCandList.

Among the merge candidates of the inter-decoding area, a large index may be added to the merge candidate list mergeCandList, or a small index may be added to the merge candidate list mergeCandList.

The sub-block unit merge candidate can be derived as follows.

1. An initial shift vector (shVector) can be derived from a motion vector of a merge candidate block adjacent to the current block.

2. As shown in Equation (18), an initial shift vector can be added to the upper left sample (xSb,ySb) of the subblock in the coding unit to derive the shifted subblock having the upper left sample position (xColSb, yColSb).

(xColSb, yColSb) = (xSb + shVector[0]>> 4,ySb+shVector[1]>> 4) (18)

3. The motion vector of the collocated block corresponding to the center position of the sub-block containing (xColSb, yColSb) can be derived as the motion vector of the sub-block containing the upper left sample (xSb,ySb).

Fig. 35

Alternatively, when sub-block merge candidates are used in the currently decoded coding unit, motion information of a plurality of representative sub-blocks in the coding unit may be stored in the inter-region sub-block motion information list HSubMVPCandList. For example, a plurality of representative sub-blocks are the upper left sub-block of the coding unit (hereinafter, the first representative sub-block), the upper-right sub-block (hereinafter, the second representative sub-block), and the lower left sub-block (hereinafter, referred to as Figure 35 ). , May be configured as a third representative sub-block). The number of inter-region sub-block motion information lists may be set to a predefined value, and may generally be set less than the number of inter-region motion information lists. The motion vector of the first representative sub-block is referred to as the first representative sub-block motion vector, the motion vector of the second representative sub-block is referred to as the second representative sub-block motion vector, and the motion vector of the third representative sub-block is referred to as the third representative. It is called a sub-block motion vector.

A sub-block motion vector in the inter-region sub-block motion information list HSubMVPList may be used as an affine seed vector. Specifically, for example, the first representative sub-block motion vector of the sub-block motion candidate in the inter-region sub-block motion information list HSubMVPList is the first affine seed vector, and the second representative sub-block motion vector is the second affine seed vector. , And the third representative sub-block motion vector may be set as a third affine seed vector. As another example, the first representative sub-block motion vector in the inter-region sub-block motion information list HSubMVPList may be used as a fourth affine seed vector, and the second representative sub-block motion vector may be used as a fifth affine seed vector.

The motion vector in the inter-region motion information list HMvpCandList may be used as one of the affine seed vectors. For example, the motion vector in HMvpCandList may be used as a value of any one of a first affine seed vector to a fifth affine seed vector.

The inter decoding region motion candidate can be used as a motion vector predictor (MVP) candidate of the current coding unit, and this method is defined as an inter decoding region motion information prediction method.

An inter-decoding region affine motion candidate can be used as a motion vector predictor (MVP) candidate of the current coding unit, and this method is defined as an inter-decoding region motion information affine prediction method.

Motion information predictor candidates usable in the current coding unit may be configured as follows, and may have the same search order as the configuration order.

1. Spatial motion predictor candidate (same as the coding block adjacent merge candidate and the coding block non-adjacent merge candidate)

2. Temporal merge candidate (motion predictor candidate derived from previous reference picture)

3. Inter-decoding domain merge candidate

4. Inter decoding domain affine merge candidate

5. Zero motion motion predictor candidate

2.5 Symmetric MVD (Symmetric MVD) coding method

Fig. 36

As shown in Figure 36 , the symmetric MVD of the motion vector difference (MVD) of the motion vector used in L0 (the MVD with the same vector size and opposite direction) can be set as the MVD of L1, which is called the symmetric MVD encoding method. When using symmetric MVD coding, the MV can be derived as shown in the following equations (19) to (20).

(mvx0, mvy0) (mvp_x0 + mvd_x0, mvpy0 + mvdy0) (19)

(mvx0, mvy0) (mvp_x0-mvd_x0, mvpy0-mvdy0) (20)

That is, the motion prediction information of LIST1 is mv

A flag sym_mvd_flag indicating whether symmetric MVD is used for each coding unit may be signaled through a bitstream. If the value of sym_mvd_flag is 1, it indicates that the coding unit uses the symmetric MVD coding method.

When using symmetric MVD encoding, the first unidirectional reference picture has a smaller POC (Picture Order Count, picture output order) than the current picture in the first unidirectional reference picture list (RefPicList0), and the picture closest to the current picture (hereinafter, The left minimum distance picture) can be set, and the first unidirectional reference picture index RefPicList0 can be set as the reference picture index of the left minimum distance picture. Here, the nearest picture means a reference picture having the smallest difference between the POC of the current picture and the reference picture.

Alternatively, the first unidirectional reference picture may be set to RefPicList0[0].

In the case of using symmetric MVD coding, the second unidirectional reference picture can be set as a picture having a larger POC than the current picture in the second unidirectional reference picture list (RefPicList1), and the closest picture to the current picture (hereinafter, the right minimum distance picture). , The second unidirectional reference picture index RefPicList1 may be set as a reference picture index of the right minimum distance picture.

Alternatively, the second unidirectional reference picture may be set to RefPicList1[1].

When there is no picture with a smaller POC than the current picture in the first unidirectional reference picture list, a picture having a larger POC than the current picture in the first unidirectional reference picture list and closest to the current picture may be set as the first unidirectional reference picture. /

When there is no picture with a larger POC than the current picture in the second unidirectional reference picture list, a picture with a smaller POC than the current picture in the second unidirectional reference picture list and the closest to the current picture may be set as the second unidirectional reference picture.

When the POC of the first unidirectional reference picture and the POC of the second unidirectional reference picture are smaller than the POC of the current picture, and the POC of the first unidirectional reference picture and the POC of the second unidirectional reference picture are the same, symmetric mvd encoding is not used. May be. Specifically, a syntax table as shown in Table 5 may be used.

Table 5

When the first unidirectional reference picture and the second unidirectional reference picture are both on the left or on the right based on the current picture, the MVD of L1 is scaled and used as shown in the following equations (21) to (22) instead of symmetric MVD. May be.

(mvx0, mvy0) (mvp_x0 + mvd_x0, mvpy0 + mvdy0) (21)

(mvx0, mvy0) (mvp_x0 + scale*mvd_x0, mvpy0-mvdy0) (22)

In the following equations (21) to (22), the scale value can be defined as in equations (23) to (24).

scale = abs(POC_Curr-POC_L0) / abs(POC_Curr-POC_L1) (23)

scale = abs(POC_Curr-POC_L1) / abs(POC_Curr-POC_L0) (24)

In Equations (23) to (24), POC_Curr represents the POC value of the current picture, POC_L0 represents the POC value of the first unidirectional reference picture, and POC_L1 represents the POC value of the second unidirectional reference picture.

If there is a reference picture having a greater POC value than the current picture in the second unidirectional reference picture list, and at least one picture having a smaller POC value than the current picture in the first unidirectional reference picture list exists, the symmetric MVD encoding method is not used. May not.

Alternatively, when there is no reference picture having a lower POC value than the current picture in the first unidirectional reference picture list, and at least one reference picture having a larger POC value than the current picture in the second unidirectional reference picture list exists, the symmetric MVD encoding method May not be used.

3 Transform and quantization

An image obtained by subtracting the predicted image from the original image is called a residual image.

The residual image can be decomposed into binary frequency components through a two-dimensional transform such as a discrete cosine transform (DCT). Even if high-frequency components are removed from an image, there is a characteristic that significant distortion does not occur visually. If the value corresponding to the high frequency is decreased or set to 0, the compression efficiency can be increased while the visual distortion is not large.

Discrete sine transform (DST) may be used according to the size of the prediction block or the prediction mode. Specifically, for example, when the prediction block/coding block is in an intra prediction mode and the size of the prediction block/coding block is smaller than NxN, the DST transform may be used, and other prediction blocks/coding blocks may be set to use DCT.

DCT is a process that decomposes (transforms) an image into 2D frequency components using a cosine transform, and the frequency components at that time are expressed as a base image. For example, if DCT transformation is performed in an NxN block, N ² basic pattern components can be obtained. Performing DCT transformation is to obtain the size of each of the basic pattern components included in the original pixel block. The size of each basic pattern component is defined as a DCT coefficient.

In general, a discrete cosine transform (DCT) is mainly used for an image in which many non-zero components are distributed in a low frequency, and a Discrete Sine Transform (DST) may be used for an image in which a large number of high-frequency components are distributed.

DST represents a process of decomposing (converting) an image into 2D frequency components using sin transformation. A 2D image can be decomposed (transformed) into 2D frequency components using a transformation method other than DCT or DST transformation, and this is defined as 2D image transformation.

Among the residual images, a 2D image transformation may not be performed in a specific block, and this is defined as transform skip. Quantization can be applied after the transform skip. Whether to allow the transform skip may be determined based on at least one of the size or shape of the coding unit. As an example, the transform skip may be limited to be used only in coding units of a specific size or less. For example, it is possible to set the transform skip to be used only in blocks smaller than 32x32. Alternatively, it is also possible to restrict the use of transform skip to only a square block. Specifically, for example, transformation skip may be applied in units of 32x32, 16x16, 8x8, and 4x4 blocks.

Alternatively, when the sub-partition intra coding method is used, transformation skip may be selectively applied in units of sub-partitions. Specifically, as shown in Figure 37 , transform skip may be used in the upper sub-partition, DST7 in the horizontal direction and DCT8 (tu_mts_idx 3) in the vertical direction may be performed in the lower sub-partition.

Fig. 37

DCT, DST, or 2D image transformation may be applied to an arbitrary block in the 2D image, and the transformation used in this case is defined as a first transformation. After performing the first transformation, transformation may be performed again in a partial region of the transformation block, and this is defined as a second transformation.

The first transformation may use one of a plurality of transformation cores. Specifically, for example, one of DCT2, DCT8, or DST7 may be selected and used in the transform block. Alternatively, the transform cores of the horizontal direction transformation and the vertical direction transformation of the transformation block may be individually determined. Accordingly, the horizontal direction transformation and the vertical direction transformation may be the same or different. Different transform cores may be used in horizontal and vertical transforms, and this is defined as a multiple transform core encoding method (Multiple Transform Selection, MTS).

Block units for performing the first transformation and the second transformation may be set differently. Specifically, for example, after performing the first transform in an 8x8 block of the residual image, the second transform may be performed for each 4x4 sub-block. For another example, after performing the first transformation in each 4x4 block, the second transformation may be performed in each 8x8 size block.

The residual image to which the first transform is applied is defined as a first transform residual image.

DCT, DST, or 2D image transformation may be applied to the first transform residual image, and the transform used in this case is defined as a second transform. The 2D image to which the second transformation is applied is defined as a second transformation residual image.

A sample value in the block after performing the first transformation and/or the second transformation or the sample value in the block after performing the transformation skip is defined as a transform coefficient. Quantization refers to the process of dividing a transform coefficient by a predefined value to reduce the energy of a block. A value defined to apply quantization to a transform coefficient is defined as a quantization parameter.

A predefined quantization parameter may be applied in units of sequences or blocks. Typically, a quantization parameter can be defined with a value between 1 and 63.

After performing transformation and quantization, inverse quantization and inverse transformation may be performed to generate a residual reconstructed image. A first reconstructed image may be generated by adding a prediction image to the residual reconstructed image.

Index information'tu_mts_idx' indicating the transformation type of the current block may be signaled through a bitstream. tu_mts_idx may refer to one of a plurality of transformation type candidates representing an example of a combination of a vertical direction transformation type and a horizontal direction transformation type. On the basis of the transformation type candidate specified by tu_mts_idx, whether to skip the vertical direction transformation and the horizontal direction transformation, and the vertical direction transformation core and the horizontal direction transformation core may be determined. The conversion core may include at least one of DCT2, DCT8, and DST7. As shown in Table 5, a transform type candidate may be set for each tu_mts_idx. Specifically, as shown in Table 6, if tu_mts_idx is 0, DCT-II can be applied to both horizontal and vertical direction conversion, and if tu_mts_idx is 2, DCT-VIII is applied to horizontal direction conversion, and vertical direction conversion DST-VII can be applied.

Table 6

In the case of using the sub-partition encoding method, a different set of transform cores may be used for each sub-partition, or the same set of transform cores may be used for each coding unit. In this case, only the first sub-partition may signal tu_mts_idx through the bitstream.

Specifically, for example, as shown in the left figure of Figure 38 , DST7 may be used as the horizontal direction transform core and DCT8 may be used as the vertical direction transform core in all sub-partitions in the coding unit. For another example, as shown in the right figure of Figure 38 , DCT8 may be used as the horizontal direction transform core and DST8 may be used as the vertical direction transform core in all sub-partitions in the coding unit.

Fig. 38

5 In-loop filtering

In-loop filtering is a technique that adaptively performs filtering on a decoded image in order to reduce the loss of information that occurs during quantization and encoding. A deblocking filter, a sample adaptive offset filter (SAO), and an adaptive loop filter (ALF) are examples of in-loop filtering.

A second reconstructed image may be generated by performing at least one of a deblocking filter, a sample adaptive offset (SAO), or an adaptive loop filter (ALF) on the first reconstructed image.

After applying the deblocking filter to the reconstructed image, SAO and ALF may be applied.

In the video encoding process, transformation and quantization are performed in units of blocks. A loss occurs during the quantization process, and a discontinuity occurs at the boundary of the reconstructed image. Discontinuous images appearing at the boundary of a block are defined as blocking artifacts.

The deblocking filter is a method of mitigating block quality deterioration (blocking artifact) occurring at a block boundary of a first image and enhancing encoding performance.

Fig. 39

Block quality deterioration can be alleviated by performing filtering at the block boundary, and whether the block is coded in intra prediction mode as shown in Figure 39 , or whether the difference between the absolute values of motion vectors of neighboring blocks is greater than a predefined threshold. A block filter strength (BS) value may be determined based on at least one of whether or not reference pictures of neighboring blocks are identical to each other. When the BS value is 0, filtering is not performed, and when the BS value is 1 or 2, filtering may be performed at a block boundary.

Since quantization is performed on a transform coefficient, a ringing artifact occurs at the edge of an object, or a pixel value increases or decreases by a certain value compared to the original because quantization is performed in the frequency domain. The SAO can effectively reduce the ringing phenomenon by adding or subtracting a specific offset in units of blocks in consideration of the pattern of the first reconstructed image. SAO is composed of an edge offset (hereinafter referred to as EO) and a band offset (BO) according to characteristics of a reconstructed image. The edge offset is a method of adding an offset to the current sample differently according to the surrounding pixel sample pattern. Band offset is to reduce coding errors by adding a certain value to a set of pixels with similar pixel brightness values in a region. By dividing the pixel brightness into 32 uniform bands, pixels having similar brightness values can be made into one set. For example, four adjacent bands can be grouped into one category. One category can be set to use the same offset value.

ALF (Adaptive Loop Filter) is a method of generating a second reconstructed image using any one of predefined filters for a first reconstructed image or a reconstructed image that has subjected deblocking filtering to the first reconstructed image as shown in Equation (25). .

(25)

(25)

Figure 40

In this case, the filter can be selected in units of pictures or units of CTU.

In the Luma component, you can select any of the 5x5, 7x7 or 9x9 diamond shapes as shown in Figure 40 below. In the chroma component, it may be limited to use only the 5x5 diamond shape.

5 Video coding structure

Fig. 41

Figure 41 is a diagram showing the video coding structure. The input video signal can be divided into a predetermined coding unit with the smallest rate distortion value. A prediction image may be generated by performing at least one of inter prediction, intra prediction, or complex prediction for each coding unit. In order to perform inter prediction in the encoding step, motion estimation may be performed. A residual image may be generated by differentiating the predicted image from the original image.

A transform coefficient may be generated by performing a first transform and/or a second transform on the residual image, or a transform skip coefficient may be generated by performing a transform skip. Quantization may be performed on a transform coefficient or a transform skip coefficient to generate a quantized transform coefficient or a quantized transform skip coefficient.

A reconstructed image may be generated by performing inverse quantization and inverse transformation on the quantized transform coefficient or the quantized transform skip coefficient. In-loop filtering may be performed on the reconstructed image to generate a filtered reconstructed image. This filtered reconstructed image becomes the final output image in the encoding step. The filtered reconstructed image is stored as a reference picture buffer and can be used as a reference picture in another picture.

Claims (1)

Video coding and decoding method using affine inter prediction coding