KR20200082981A - Video signal encoding method and apparatus and video decoding method and apparatus - Google Patents

Abstract

Provided are a video encoding method and apparatus, which use a motion prediction candidate in units of sub-blocks. The video signal processing method and apparatus according to the present invention may improve video signal coding efficiency through motion prediction candidate coding in units of sub-blocks.

Description

Video signal processing method and apparatus {Video signal encoding method and apparatus and video decoding method and apparatus}

The present invention relates to a video signal processing method and apparatus.

The video image is compression-encoded by removing spatiotemporal redundancy and inter-view redundancy, which can be stored in a form suitable for a storage medium or transmitted through a communication line.

The present invention seeks to improve the coding efficiency of video signals.

In order to solve the above problems, the present invention provides a sub-block unit motion prediction candidate encoding method and apparatus.

The video signal processing method and apparatus according to the present invention can improve video signal coding efficiency through motion prediction candidate coding for each sub-block.

One Basic coding block structure

The picture is divided into a plurality of basic coding units (Coding Tree Units, hereinafter, CTU) as shown in Figure 1 .

Picture 1

It is configured not to overlap with other CTUs. For example, the CTU size may be set to 128x128 in the entire sequence, and any one of 128x128 to 256x256 may be selected and used as a picture unit.

The CTU may be hierarchically split to generate a coding block/coding unit (CU). Prediction and transformation can be performed in units of coding units, and are a basic unit for determining a prediction encoding mode. The predictive encoding mode represents a plurality of methods for generating a predictive image, and examples include intra prediction (hereinafter, intra prediction), inter prediction (hereinafter, inter prediction), or composite prediction. Can be lifted. Specifically, for example, a prediction block may be generated using at least one prediction coding mode of intra prediction, inter prediction, or complex prediction in units of coding units. When the reference picture points to the current picture in the inter-frame prediction mode, a prediction block may be generated in an area within the current picture that has already been decoded. Since a prediction block is generated using a reference picture index and a motion vector, it can be included in inter-frame prediction. Intra prediction is a method of generating a prediction block using information of a current picture, inter prediction is a method of generating a prediction block using information of another decoded picture, and complex prediction is a mixture of inter prediction and intra prediction How to use.

The CTU can be divided (partitioned) into a quad tree, binary tree, or triple tree, as shown in Figure 2 . The divided blocks may be further divided into quad trees, binary trees, or triple trees. The method of dividing the current block into four square partitions is called quad-tree partitioning, the method of dividing the current block into two non-square partitions, and the method of dividing the current block into three non-square partitions. This is called binary tree partitioning.

Binary partitioning in the vertical direction (SPLIT_BT_VER in Figure 2 ) is called vertical binary tree partitioning, and binary tree partitioning in the horizontal direction (SPLIT_BT_HOR in Figure 2 ) is called horizontal binary tree partitioning.

Triple partitioning in the vertical direction (SPLIT_TT_VER in Figure 2 ) is called vertical triple tree partitioning, and triple tree partitioning in the horizontal direction (SPLIT_TT_HOR in Figure 2 ) is called horizontal triple tree partitioning.

The number of additional divisions is called a partitioning depth, and the maximum number of partitioning depths can be set differently for each sequence or picture/tile set, and different partitioning depths are different according to the partitioning tree type (quad tree/binary tree/triple tree). It can be set to have, and can also signal the syntax to indicate this.

Picture 2

The divided coding block may be further divided into a coding unit by a method such as quad partitioning, binary partitioning, or multi-partitioning, or a coding unit may be configured without further division.

Picture 3

As shown in Figure 3 , a coding unit can be hierarchically set for one CTU, and a coding unit can be split using at least one of the coding units using binary tree partitioning, quad tree partitioning, and triple tree partitioning. This method is called multi-tree partitioning.

The coding unit generated by dividing any coding unit having a partitioning depth k is referred to as a lower coding unit, and the partitioning depth is k+1. A coding unit having a partitioning depth k that includes a lower coding unit having a partitioning depth k+1 is called an upper coding unit.

The partitioning type of the current coding unit may be limited according to the partitioning type of the higher coding unit and/or the partitioning type of the coding unit around the current coding unit.

Here, the partitioning type indicates an indicator indicating which partitioning is used among binary tree partitioning, quad tree partitioning, or triple tree partitioning.

2 Prediction image generation method

In video encoding, a predicted image can be generated by a plurality of methods, and a method of generating a predicted image is called a predictive encoding mode.

The prediction encoding mode may be configured as an intra prediction encoding mode, an inter prediction encoding mode, a current picture reference encoding mode, or a combined prediction mode.

The inter prediction encoding mode is called a prediction encoding mode that generates a prediction block (prediction image) of a current block using information of a previous picture, and the intra prediction encoding mode is a prediction that generates a prediction block using samples neighboring the current block. It is called an encoding mode. A prediction block may be generated using an already reconstructed image of the current picture, and this is called a current picture reference mode or an intra block copy mode.

A prediction block may be generated using at least two or more prediction encoding modes among an inter prediction encoding mode, an intra prediction encoding mode, or a current picture reference encoding mode, which is called a combined prediction mode.

2.1 Inter prediction coding method

A method of generating a prediction block (prediction image) of a block in a current picture using information of a previous picture is called an inter prediction encoding mode.

A prediction block may be generated from a specific block in a previous picture of the current block.

In the coding step, a block having the smallest reconstruction error among blocks in a previous picture may be searched and selected based on a collocated block.

In addition, an x-axis difference and a y-axis difference between the upper left sample of the current block and the upper left sample of the selected block may be defined as a motion vector, and transmitted to a bitstream for signaling. A block generated through interpolation or the like in a specific block of a reference picture specified by a motion vector is called a motion compensated predictor block.

The call block shows the block of the corresponding picture with the same position and size of the upper left sample and the current block as shown in Figure 4. The picture may be specified from the same syntax as the reference picture reference.

Picture 4

A residual block may be generated by differentiating a motion compensation prediction block from the original image of the current block.

As a motion vector, motion vectors having different pixel precisions may be used in sequence units, slice units, or block units. For example, the pixel precision of a motion vector in a specific block may be at least one of Octor-pel, Quarter-Pel, Half-Pel, Integer pel, and 4 Integer pel.

In the inter prediction mode, an inter prediction method using translaton motion and an affine inter prediction method using affine motion may be selectively used.

2.1.1 Merge mode coding method

Without encoding the motion information (motion vector, reference picture index, etc.) of the current coding unit, it can be derived from motion information of neighboring blocks. Motion information of any one of the neighboring blocks may be set as motion information of the current coding unit, and this is called a merge mode.

The neighboring blocks used in the merge mode may be blocks adjacent to the current coding unit (blocks that abut the boundary of the current coding unit) as shown in Figure 5 , merge candidate indexes 0 to 4, and adjacent as shown in Figure 5 merge candidate indexes 5 to 26. It may be a block that has not been done.

Picture 5

It can be set that the merge candidate is not available if the distance to the current block exceeds a predefined threshold.

Picture 6

Merge candidates may be derived in units of sub-blocks. As shown in Figure 6, the first available spatial peripheral merge candidate block in the search order (A1 --> B1 --> B0 --> A0) among the spatial peripheral merge candidate blocks A1, B1, B0, and A0 is the first spatial peripheral merge They are called candidates.

Picture 7

The reference picture indicated by the call picture reference index (collocated_ref_idx) is defined as a call picture (Collocated picture). The block in the call picture indicated by the motion vector (hereinafter, the initial shift vector) of the first spatial peripheral merge candidate is referred to as a call picture selection merge candidate block. Specifically, as illustrated in FIG. 7, a prediction block including sample/block A1' indicated by the motion vector of the initial spatial neighbor merge candidate A1 is referred to as a call picture selection merge candidate block.

The difference between the x-axis coordinates of the upper left sample of the current block and the x-axis coordinates of the first spatial peripheral merge candidate is defined as the coding block width offset, and the difference between the y-axis coordinates of the first spatial peripheral merge candidate and the y-axis coordinates of the upper left sample of the current block. Can be defined as the coding block height offset.

As shown in Figure 7, the x-coordinate of the call picture selection merge candidate block plus the coding block width offset is defined as the call-picture correspondence block x-coordinate, and the y-coordinate of the call picture selection merge candidate block plus the coding block height offset is added. It is defined as the y coordinate of the block corresponding to the call picture. In the call picture, the x-coordinate of the upper-left sample is the same as the x-coordinate of the call-picture-corresponding block, and the y-coordinate of the upper-left sample is the same as the x-coordinate of the call-picture-corresponding block. It is called a picture-adaptive block.

A motion vector of a sub-block of a call picture corresponding block may be set as a motion vector of a corresponding sub-block of the current block, which is called a sub-block merge candidate.

The sub-block merge candidate can also be used as a sub-block motion prediction candidate (MVP).

The merge candidate for each sub-block may be derived as in the following process.

1. An initial shift vector (shVector) may be derived from a motion vector of a neighbor candidate block of the current block.

2. As shown in Equation (1), an initial shift vector may be added to the upper left sample (xSb,ySb) of the sub block in the coding unit to derive a shift sub block having the position of the upper left sample (xColSb, yColSb).

(xColSb, yColSb) = (xSb + shVector[0]>> 4,ySb+shVector[1]>> 4) (1)

3. The motion vector of the collocated block corresponding to the center position of the sub-block containing (xColSb, yColSb) can be derived as the motion vector of the sub-block containing the upper left sample (xSb,ySb).

A motion vector may be derived from a reference picture other than the reference picture (hereinafter referred to as a call reference picture) used to specify a sub-block merge candidate to replace the unused predictive image.

Specifically, for example, a motion vector derived from LX (X is 0 or 1) in a specific sub-block in a sub-block merge candidate is not available (hereinafter, the first unidirectional sub-block), and the reference picture in the reference picture list When all of the POC values are equal to or less than the POC value of the current picture (hereinafter, forward prediction), a motion vector may be derived by using a reference picture of L[1-X], and the motion vector of L1 is equal to the current picture and L1. It can be scaled to a value proportional to the distance between and L1 and L0.

In the case of forward prediction, when at least one of a motion vector derived from a reference picture of L[X] and a motion vector derived from a reference picture of L[1-X] is not available, a reference picture specified by collocated_ref_idx (hereinafter, The motion vector may be derived from a call reference picture), which is called a call TMVP. Here, collocated_ref_idx represents syntax used when specifying a reference picture when generating a temporal motion vector predictor, and may be signaled through a bitstream.

If the syntax collocated_from_l0_flag value is 1, it indicates that the call reference picture can be derived from L0 in the reference picture list. For example, if the value of collocated_from_l0_flag is 1, RefPicList0[collocated_ref_idx] may be set as a call reference picture, and when value of collocated_from_l0_flag is 0, RefPicList1[collocated_ref_idx] may be set as a call reference picture.

If it is not forward prediction and the motion vector derived from LX (X is 0 or 1) is not available, the motion vector may be derived from the call reference picture.

When the motion vector derived from LX is not available in the first unidirectional sub-block (subblock non-available ATMVP), when the motion vector derived from LX of TMVP is available (hereinafter referred to as X-direction TMVP), the sub-block ratio The TMVP in the x-direction can be used as the motion vector of the ATMVP. Specifically, for example, when a motion vector derived from L0 is not available in the first unidirectional sub-block, and a motion vector derived from L0 of TMVP is available (hereinafter, 0-direction TMVP), the first unidirectional sub-block In L0, a motion vector of L0 may be used as a TMVP in the 0 direction.

A new temporal motion prediction candidate may be defined by weighted prediction of a sub-block unit merge candidate and a TMVP, which is called a weighted TMVP.

A plurality of sub-block unit motion predictor candidates may be configured as follows.

1. Sub-block unit merge candidate

2. Cole TMVP

3. Weighted TMVP

2.1.2 Inter decoding region merge method

Motion information (motion vector and reference picture index) of a coding unit that has already been encoded with inter prediction in a current picture may be stored in a list of a predefined size, which is called an inter-region motion information list.

The motion information (motion vector and reference picture index) in the inter-domain motion information list is referred to as an inter-decoding domain merge candidate.

The inter-decoding area merge candidate can be used as a merge candidate of the current coding unit, and this method is called an inter-decoding area merge method.

When the tile group is initialized, the inter-region motion information list is empty, and when a region of a picture is encoded/decoded, it can be added to the inter-region motion information list. The initial inter decoding region merge candidate of the inter region motion information list may be signaled through the tile group header.

If the coding unit is to inter-prediction encoding / decoding may update the motion information of the coding unit in the inter-domain motion information list, as shown in Figure 6. When the number of inter-decoding region merge candidates in the inter-region motion information list is the maximum value, the value having the smallest index of the inter-region motion information list (first, the inter-decoding region merge candidate in the inter-region motion information list) is removed. , The motion vector of the most recently coded/decoded inter-region may be added as an inter-decoding region merge candidate.

Figure 8

The motion vector mvCand of the decoded coding unit can be updated in the inter-domain motion information list HmvpCandList. At this time, if the motion information of the decoded coding unit is the same as any of the motion information in the inter-region motion information list (when both motion vectors and reference indices are the same), the inter-region motion information list is not updated, or Figure 10 and The motion vector mvCand of the decoded coding unit may be stored at the end of the inter-domain motion information list. At this time, if the index of HmvpCandList having motion information such as mvCand is hIdx, HMVPCandList [i] can be set to HVMPCandList[i-1] for all i greater than hIdx as shown in Figure 10 . When sub-block merge candidates are used in the currently decoded coding unit, motion information of a representative sub-block in the coding unit may be stored in the inter-region motion information list.

As an example, the representative serv block in the coding unit may be set as the upper left sub block in the coding unit as shown in Figure 9, or may be set as the middle sub block in the coding unit.

Picture 9

Fig. 10

A total of NumHmvp motion information (motion vector and reference picture index) can be stored in the inter-region motion information list, and NumHmvp is called an inter-region motion information list size.

A predefined value may be used for the size of the inter-region motion information list. The size of the inter-region motion information list may be signaled in the tile group header. For example, the size of the inter-region motion information list may be defined as 16 or 6.

A coding unit having an inter-prediction and affine motion vector may be limited not to have an inter-domain motion information list.

Alternatively, in the case of having inter-prediction and affine motion vectors, affine sub-block vectors may be added to the inter-domain motion information list. At this time, the position of the sub-block may be set to the upper left or upper right, or the central sub-block.

Alternatively, the average motion vector of each control point may be added to the inter-region merge candidate list.

If the motion vector MV0 derived by encoding/decoding a specific coding unit is the same as any of the inter decoding region merge candidates, MV0 may not be added to the inter region motion information list. Alternatively, an existing inter-decoding area merge candidate having the same motion vector as MV0 may be deleted, and MV0 may be newly included in the decoding area merge candidate to update an index assigned to MV0.

In addition to the inter-region motion information list, an inter-region motion information long term list HmvpLTList may be configured. The inter-region motion information long term list size may be set equal to the inter-region motion information list size or may be set to a different value.

The inter-region motion information long term list may be configured as an inter-decoding region merge candidate first added to the tile group start position. After the inter-region motion information long term list is configured with all available values, the inter-region motion information list may be configured or motion information in the inter-region motion information list may be set as motion information of the inter-region motion information long term list.

In this case, the long-term list of the inter-region motion information configured once may not be updated, or may be updated again when the decoded region of the tile group is more than half of the entire tile group, or may be updated every m CTU lines. The inter-region motion information list may be set to be updated every time it is decoded into the inter-region or to update in CTU line units.

Motion information and partition information or a form of a coding unit may be stored in the inter-region motion information list. It is also possible to perform the inter-decoding region merging method using only inter-region merge candidates having similar partition information and form to the current coding unit.

Alternatively, an inter-region merge candidate list may be individually configured according to a block type. In this case, one of a plurality of inter-region merge candidate lists may be selected and used according to the shape of the current block.

Picture 11

As shown in Figure 11 , it can be composed of an inter-domain affine motion information list and an inter-domain motion information list. When the decoded coding unit is in the affine inter or affine merge mode, the first affine seed vector and the second affine seed vector may be stored in the inter-domain affine motion information list HmvpAfCandList. The motion information in the inter-domain affine merge candidate list is called an inter-decoding region affine merge candidate.

The merge candidates usable in the current coding unit may be configured as follows, and may have the same search order as the configuration order.

1. Spatial merge candidate (coding block contiguous merge candidate and coding block non-contiguous merge candidate)

2. Temporary merge candidate (merged candidate derived from the previous reference picture)

3. Inter decoding region merge candidate

4. Inter-decoding area affine merge candidate

5. Zero motion merge candidate

First, the merge candidate list may include mergeCandList as a spatial merge candidate and a temporal merge candidate. The number of available spatial and temporal merge candidates is called the number of available merge candidates (NumMergeCand). When the number of available merge candidates is smaller than the maximum number of merges allowed, the inter-decoding area merge candidate can be added to the merge candidate list mergeCandList.

The inter decoding region motion candidate can be used as a motion vector predictor (MVP) candidate of the current coding unit, and this method is called an inter decoding region motion information prediction method.

The inter-decoding area affine motion candidate can be used as a motion vector predictor (MVP) candidate of the current coding unit, and this method is called an inter-decoding area motion information affine prediction method.

The motion information predictor candidates usable in the current coding unit may be configured as follows, and may have the same search order as the configuration order.

1. Spatial motion predictor candidate (same as coding block contiguous merge candidate and coding block non-contiguous merge candidate)

2. Temporary merge candidate (motion predictor candidate derived from the previous reference picture)

3. Inter decoding region merge candidate

4. Inter-decoding area affine merge candidate

5. Zero motion motion predictor candidate

2.2 Intra prediction coding method

Intra prediction is used to generate an intra prediction by using an already coded boundary sample around the current block, as shown in Figure 13 , which is called an intra reference sample.

Predict the average value of the intra reference samples, set the value of the entire sample of the prediction block (DC mode), or perform the horizontal direction weighted prediction to generate the horizontal direction prediction sample and the vertical direction reference sample weighted prediction to generate the vertical direction prediction sample. After generating, a prediction sample may be generated by weighted prediction of a horizontal prediction sample and a vertical prediction sample (Planar mode), or intra prediction may be performed using a directional intra prediction mode.

Picture 13

Using a 33 direction, as shown on the left in Figure 14 to (a total of 35 intra-prediction mode), and to perform intra prediction, using the 65 direction as shown in the right illustration in Figure 14 (total of 67 intra-prediction mode) It might be. When using directional intra prediction, an intra reference sample (reference reference sample) may be generated in consideration of the directionality of the intra prediction mode, and intra prediction may be performed therefrom.

The intra reference sample on the left side of the coding unit is called the left intra reference sample, and the intra reference sample on the upper side of the coding unit is called the upper intra reference sample.

When performing directional intra prediction, an intra direction parameter (intraPredAng), which is a parameter indicating a prediction direction (or prediction angle), may be set according to the intra prediction mode as shown in Table 3. Table 1 below is only an example based on the directional intra prediction mode having a value of 2 to 34 when using 35 intra prediction modes. Of course, the prediction direction (or prediction angle) of the directional intra prediction mode is further subdivided so that more than 33 directional intra prediction modes may be used.

Picture 14

Table 1

Picture 15

When intraPredAng is negative (for example, when the intra prediction mode index is between 11 and 25), the left intra reference sample and the upper intra reference sample are configured in 1D according to the angle of the intra prediction mode in the current block, as shown in Figure 15 . It can be reconstructed as a one-dimensional reference sample (Ref_1D).

Picture 16

When the intra prediction mode index is between 11 and 18, a one-dimensional reference sample can be generated counterclockwise from the intra reference sample located on the upper right side of the current block to the intra reference sample located on the lower left side of the current block, as shown in Figure 16 .

In other modes, a one-dimensional reference sample may be generated using only the upper side intra reference sample or the left side intra reference sample.

Picture 17

When the intra prediction mode index is between 19 and 25, a one-dimensional reference sample can be generated clockwise from the intra reference sample located at the lower left side of the current block to the intra reference sample located on the upper right side as shown in Figure 17 .

The weight-related parameter ifact applied to at least one reference sample determined based on the reference sample determination index iIdx and iIdx may be derived as in the following equations (2) to (3). iIdx and i _fact are variably determined according to the gradient of the directional intra prediction mode, and the reference sample specified by iIdx may correspond to an integer pel.

iIdx = (y+1) * P _ang /32 (2)

i _fact = [(y+1) * P _ang ] & 31 (3)

The prediction image may be derived by specifying at least one one-dimensional reference sample for each prediction sample. For example, the position of a one-dimensional reference sample that can be used for generating a prediction sample may be specified by considering the slope value of the directional intra prediction mode. It is also possible to have different directional intra prediction modes for each prediction sample. Multiple intra prediction modes may be used for one prediction block. The plurality of intra prediction modes may be represented by a combination of a plurality of non-directional intra prediction modes, or may be represented by a combination of one non-directional intra prediction mode and at least one directional intra prediction mode, or multiple directional intra prediction It can also be expressed as a combination of modes. Different intra prediction modes may be applied to each sample group in one prediction block. A given sample group may consist of at least one sample. The number of sample groups may be variably determined according to the size/number of samples of the current prediction block, or may be a fixed number preset in the encoder/decoder independently of the size/number of samples of the prediction block.

Specifically, for example, the location of the one-dimensional reference sample may be specified using the reference sample determination index iIdx.

According to the slope of the intra prediction mode, when it is not possible to express the slope of the intra prediction mode using only one one-dimensional reference sample, the first prediction image may be generated by interpolating adjacent one-dimensional reference samples as in Equation (4). When the angular line according to the slope/angle of the intra prediction mode does not pass the reference sample located in the integer pel, the first prediction image may be generated by interpolating the reference samples adjacent to the left/right or top/bottom of the angular line. . At this time, the filter coefficient of the interpolation filter used may be determined based on i _fact . For example, the filter coefficient of an interpolation filter can be derived based on the distance between a minority pel located on an angular line and a reference sample located on the integer pel.

P(x,y) = ((32-i _fact )/32)* Ref_1D(x+iIdx+1) + (ifact/32)* Ref_1D(x+iIdx+2) (4)

When the slope of the intra prediction mode can be expressed by only one one-dimensional reference sample (when the i _fact value is 0), the first prediction image can be generated as in the following equation (5).

P(x,y) = Ref_1D(x+iIdx+1) (5)

2.3 Wide angle intra prediction coding method

The prediction angle of the directional intra prediction mode may be set to a value between 45 and -135 as shown in Figure 18 .

Picture 18

When performing the intra prediction mode in the non-square coding unit, a disadvantage of predicting the current sample from the intra reference sample far from the current sample may occur instead of the intra reference sample close to the current sample due to a predefined prediction angle.

For example, in the coding unit having the width of the coding unit larger than the height of the coding unit (hereinafter, the horizontal coding unit) as shown in the left figure of FIG. 19 , intra prediction is performed at a distance L instead of the sample T having a short distance. Can be. As another example, in Figure 1 a large coding unit, the height of the coding unit as shown in the figure to the right of 9 than the coding unit width (the vertical direction coding unit) in from the sample T is a distance to the sample L instead of the distance near Intra prediction can be performed.

Picture 19

In the non-square coding unit, intra prediction may be performed at a prediction angle wider than a predetermined prediction angle, and this is called a wide angle intra prediction mode.

내지

의 예측 각도를 가질 수 있으며, 기존 인트라 예측 모드에서 사용된 각도를 벗어나는 예측 각도를 와이드 앵글 각도라고 부른다. Wide angle intra prediction mode

To

It can have a prediction angle of, and the prediction angle that deviates from the angle used in the existing intra prediction mode is called a wide angle angle.

In the left picture of Figure 19 , sample A in the horizontal coding unit can be predicted from the intra reference sample T using the wide angle intra prediction mode.

In the right figure of Figure 19 , sample A in the vertical coding unit can be predicted from the intra reference sample L using the wide angle intra prediction mode.

N+M intra prediction modes may be defined by adding M wide angle angles to N existing intra prediction modes. Specifically, for example, as shown in Table 2, a total of 95 intra prediction modes may be defined by adding 28 wide angle angles to 67 intra modes.

The intra prediction mode that can be used by the current block may be determined according to the shape of the current block. As an example, 65 directional intra prediction modes among 95 directional intra prediction modes may be selected based on at least one of the size, aspect ratio (eg, ratio of width and height) of the current block, and a reference line index.

Table 2

The intra prediction mode angle shown in Table 2 may be adaptively determined based on at least one of a shape of a current block and a reference line index. For example, intraPredAngle in Mode 15 may be set to have a larger value when the current block is square than when the current block is non-square. Alternatively, intraPredAngle in Mode 75 may be set to have a larger value when a non-contiguous reference line is selected than when the adjacent reference line is selected.

When using the wide angle intra prediction mode, the length of the upper intra reference sample can be set to 2W+1 and the length of the left intra reference sample can be set to 2H+1 as shown in Figure 20 .

Picture 20

When using the wide-angle intra-prediction, when encoding the intra-prediction mode of the wide-angle intra-prediction mode, the number of intra-prediction modes increases, so that the encoding efficiency may be lowered. The wide-angle intra-prediction mode can be encoded by replacing it with an existing intra-prediction mode that is not used in the wide-angle intra, and the replaced prediction mode is called a wide-angle alternative mode.

Specifically, for example, when 35 intra predictions are used as shown in FIG. 21 , wide angle intra prediction mode 35 may be encoded in intra prediction mode 2, which is a wide angle replacement mode, and wide angle intra prediction mode 36 may be replaced by a wide angle. The mode can be encoded in intra prediction mode 3.

Picture 21

Depending on the type of coding block or the ratio of the coding block height to the width, the number of modes and number of replacements can be set differently. Specifically, for example, as shown in Tables 3 to 4, the mode and number of replacements may be set differently according to the type of the coding block. Table 3 shows a case where 35 intra prediction modes are used, and Table 4 shows a case where 67 intra prediction modes are used.

Table 3

Table 4

2.4 Multi-line intra prediction coding method

Intra prediction may be performed on a plurality of intra reference lines as shown in the following figure 22 , which is called a multi-line intra prediction encoding method.

Fig. 22

Intra prediction may be performed by selecting any one of a plurality of intra reference lines composed of adjacent intra reference lines and non-contiguous intra reference lines, which is called a multi-line intra prediction method. The non-contiguous intra reference line includes a first non-contiguous intra reference line (non-contiguous reference line index 1), a second non-contiguous intra reference line (non-contiguous reference line index 2), and a third non-contiguous intra reference line (non-contiguous reference line) Index 3). It is also possible to use only some of the noncontiguous intra reference lines. For example, only the first non-contiguous intra reference line and the second non-contiguous intra reference line may be used, or only the first non-contiguous intra reference line and the third non-contiguous intra reference line may be used.

An intra reference line index (intra_luma_ref_idx), which is a syntax for specifying a reference line used for intra prediction, may be signaled in units of coding units.

Specifically, when using the adjacent intra reference line, the first non-contiguous intra reference line, and the third non-contiguous intra reference line, intra_luma_ref_idx may be defined as shown in Table 5 below.

Table 5

When a non-contiguous intra reference line is used, the non-directional intra prediction modules may be set not to be used. That is, when a non-contiguous intra reference line is used, DC mode or planar mode may not be used.

The number of samples belonging to the non-contiguous intra reference line can be set to be larger than the number of samples of the adjacent intra reference line. Also, the number of samples of the i+1 non-contiguous intra reference lines may be set to be larger than the number of samples of the i-noncontiguous intra reference lines. The difference between the number of upper samples of the i-th non-contiguous intra reference line and the number of upper samples of the i-th non-contiguous intra reference line may be indicated by the offset number of reference samples offsetX[i]. offsetX[1] represents the difference value between the number of upper samples of the first non-contiguous intra reference line and the number of upper samples of the adjacent intra reference line. The difference between the number of left samples of the i-th non-contiguous intra reference line and the number of left samples of the i-th non-contiguous intra reference line may be represented by offsetY[i]. offsetY[1] represents the difference value between the number of left samples of the first non-contiguous intra reference line and the number of left samples of the adjacent intra reference line.

The non-contiguous intra reference line with the intra reference line index i can be composed of the upper non-contiguous reference line refW + offsetX[i], the left non-contiguous reference line refH+ offsetY[i], and the upper left sample. The number of samples belonging to may be composed of refW + refH + offsetX[i] + offsetY[i] +1.

refW =2* nTbW (6)

refH =2*　*nTbH (7)

In equations (6) to (7), nTbW represents the width of the coding unit, nTbH represents the height of the coding unit, and whRatio can be defined as in the following equation (8).

whRatio = Min(log2(nTbW/nTbH),2) (8)

In the multi-line intra prediction encoding method, when a non-contiguous intra reference line is used, the wide-angle intra mode may be disabled. Alternatively, if the MPM mode of the current coding unit is a wide angle intra mode, a multi-line intra prediction coding method may be disabled. In this case, the non-contiguous intra reference line with the intra reference line index i can be composed of the upper non-contiguous reference line W + H + offsetX[i], the left non-contiguous reference line H + W + offsetY[i], and the upper left sample. , The number of samples belonging to the non-contiguous intra reference line may be composed of 2W + 2H + offsetX[i] + offsetY[i] +1, and offsetX[i] and offsetY[i] values may vary depending on the whRatio value. There is. For example, if the whRatio value is greater than 1, the offsetX[i] value can be set to 1, and the offsetY[i] value can be set to 0. If the whRatio value is less than 1, the offsetX[i] value is 0, and the offsetY[ i] The value may be set to 1.

3 Transformation and quantization

An image obtained by subtracting a predicted image from an original image is called a residual image.

The residual image may be decomposed into binary frequency components through a two-dimensional transform such as a DCT (Discrete cosine transform). There is a characteristic that visual distortion does not occur significantly even when a high frequency component is removed from an image. When the value corresponding to the high frequency is set to a small value or 0, compression efficiency can be increased without significant visual distortion.

Discrete sine transform (DST) may be used according to the size of a prediction block or a prediction mode. Specifically, for example, in the intra prediction mode, when the size of the prediction block/coding block is smaller than NxN, DST transformation may be set to be used, and DCT may be set to be used in other prediction blocks/coding blocks.

DCT is a process of decomposing (converting) an image into two-dimensional frequency components using cos transform, and the frequency components at that time are expressed as a base image. For example, if DCT transformation is performed on an NxN block, N ² basic pattern components can be obtained. Performing DCT conversion is to obtain the size of each of the basic pattern components included in the original pixel block. The size of each basic pattern component is called the DCT coefficient.

In general, a discrete cosine transform (DCT) is mainly used in an image in which a lot of non-zero components are distributed in a low frequency, and a discrete Sine Transform (DST) may be used in an image in which a large amount of high-frequency components are distributed.

DST denotes a process of decomposing (converting) an image into two-dimensional frequency components using a sin transform. A 2D image may be decomposed (converted) into 2D frequency components using a conversion method other than DCT or DST conversion, and this is called 2D image conversion.

A 2D image transformation may not be performed in a specific block among residual images, and this is called transform skip. Quantization can be applied after the transform skip.

DCT or DST or 2D image transformation may be applied to any block in the 2D image, and the transformation used at this time is called a first transformation. After performing the first transform, the transform may be performed again in a partial region of the transform block, which is called a second transform.

The first transformation may use one of a plurality of transformation cores. Specifically, for example, any one of DCT2, DCT8, or DST7 can be selected and used in the conversion block. Alternatively, different transform cores may be used in the horizontal and vertical transforms of the transform block.

Block units for performing the first transformation and the second transformation may be set differently. Specifically, for example, after performing the first transform on the 8x8 block of the residual image, the second transform may be performed for each 4x4 sub-block. As another example, after performing the first transform in each 4x4 block, the second transform may be performed in the 8x8 block.

The residual image to which the first transform is applied is called a first transform residual image.

DCT or DST or 2D image transformation may be applied to the first transform residual image, and the transform used at this time is referred to as a second transform. The 2D image to which the second transform is applied is called a second transform residual image.

The sample value in the block after performing the first transform and/or the second transform is called a transform coefficient. Quantization refers to the process of dividing the transform coefficients into predefined values to reduce the energy of the block. The value defined to apply quantization to the transform coefficient is called a quantization parameter.

Predefined quantization parameters may be applied in sequence units or block units. Typically, a quantization parameter may be defined as a value between 1 and 64.

After performing the transform and quantization, an inverse quantization and an inverse transform may be performed to generate a residual reconstructed image. A first reconstructed image may be generated by adding a predictive image to the residual reconstructed image.

4 In-loop filtering

In-loop filtering is a technique that adaptively filters the decoded image to reduce loss of information generated during quantization and encoding. A deblocking filter, a sample adaptive offset filter (SAO), and an adaptive loop filter (ALF) are examples of in-loop filtering.

A second reconstructed image may be generated by performing at least one of a deblocking filter, a sample adaptive offset (SAO), or an adaptive loop filter (ALF) on the first reconstructed image.

After applying a deblocking filter to the reconstructed image, SAO and ALF can be applied.

In the video encoding process, transformation and quantization are performed on a block basis. Loss occurs during the quantization process, and discontinuity occurs at the boundary of the reconstructed image. Discontinuous images appearing on the block boundary are called blocking artifacts.

The deblocking filter is a method for alleviating blocking artifacts occurring at a block boundary of the first image and improving encoding performance.

Picture 32

Filtering can be performed at the block boundary to alleviate the deterioration of the block quality, and whether the block is encoded in intra prediction mode as shown in Figure 32 , or whether the difference between the absolute values of the motion vectors of neighboring blocks is greater than a predetermined threshold. Whether or not the reference pictures of neighboring blocks are identical to each other may determine a block filter strength (BS) value based on at least one. When the BS value is 0, filtering is not performed, and when the BS value is 1 or 2, filtering may be performed at a block boundary.

Since the quantization is performed on the transform coefficient, since the quantization is performed in the frequency domain, a ring occurs at the edge of the object, or the pixel value is increased or decreased by a certain value compared to the original. The SAO can effectively reduce the ringing phenomenon by adding or subtracting a specific offset in block units in consideration of the pattern of the first reconstructed image. SAO is composed of an edge offset (hereinafter referred to as EO) and a band offset (BO) according to the characteristics of the reconstructed image. Edge offset is a method of differently adding an offset to a current sample according to a sample pattern of surrounding pixels. The band offset is to reduce coding errors by adding a constant value to a set of pixels having similar pixel brightness values in the region. By dividing the pixel brightness into 32 uniform bands, pixels having similar brightness values can be set as one set. For example, four adjacent bands can be grouped into one category. One category can be set to use the same offset value.

ALF (Adaptive Loop Filter) is a method of generating a second reconstructed image by using any one of filters defined in the reconstructed image obtained by performing deblocking filtering on the first reconstructed image or the first reconstructed image as shown in Equation (9). .

(9)

(9)

Fig. 33

At this time, the filter may be selected on a picture-by-picture or CTU basis.

In the Luma component, you can choose either 5x5, 7x7 or 9x9 diamond shape as shown in Figure 33 below. It is also possible to limit the use of 5x5 diamonds in the Chroma component.

5 Video encoding structure

Fig. 34

Figure 34 is a diagram showing the video encoding structure. The input video signal can be divided into predetermined coding units with the smallest rate distortion values. A prediction image may be generated by performing at least one of inter prediction, intra prediction, or complex prediction for each coding unit. In order to perform inter prediction in the encoding step, motion estimation may be performed. A residual image may be generated by differentiating a predicted image from the original image.

A transform coefficient may be generated by performing a first transform and/or a second transform on the residual image. The quantized transform coefficient may be generated by performing quantization on the transform coefficient.

The reconstructed image may be generated by performing inverse quantization and inverse transformation on the quantized transform coefficients. Filtering reconstructed images may be generated by performing in-loop filtering on the reconstructed images. This filtered reconstruction image becomes the final output image in the encoding step. The filtered reconstructed image is stored as a reference picture buffer and can be used as a reference picture in other pictures.

Claims (1)

Sub-block motion prediction candidate encoding and decoding method