KR20240024755A - Method for encoding/decoding a video signal and recording medium storing a bitsteram generated based on the method - Google Patents

Abstract

An image decoding method according to the present disclosure includes obtaining a first prediction block based on first prediction information for a first prediction unit in a current block; Obtaining a second prediction block based on second prediction information for a second prediction unit in the current block; and obtaining a final prediction block for the current block based on the first prediction block and the second prediction block. At this time, in the blending area formed around the boundary between the first prediction unit and the second prediction unit in the current block, the first prediction samples in the first prediction block and the second prediction samples in the second prediction block The final prediction sample can be derived by blending the prediction samples.

Description

Video signal encoding/decoding method and recording medium for storing bitstream generated based on the same {METHOD FOR ENCODING/DECODING A VIDEO SIGNAL AND RECORDING MEDIUM STORING A BITSTERAM GENERATED BASED ON THE METHOD}

The present disclosure relates to a video signal encoding/decoding method and a recording medium for storing a bitstream generated based on the same.

As display panels become increasingly larger, video services of higher quality are increasingly required. The biggest problem with high-definition video services is a significant increase in data volume, and to solve this problem, research is actively being conducted to improve video compression rates. As a representative example, in 2009, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union-Telecommunication (ITU-T) formed the Joint Collaborative Team on Video Coding (JCT-VC). JCT-VC proposed HEVC (High Efficiency Video Coding), a video compression standard with about twice the compression performance of H.264/AVC, and was approved as a standard on January 25, 2013. With the rapid development of high-definition video services, the performance of HEVC is gradually revealing its limitations.

The present disclosure provides a method for predicting blocks based on merge mode involving geometric partitioning when encoding/decoding video signals.

The present disclosure provides a method for modifying a motion vector using an offset vector when a merge mode involving geometric segmentation is applied when encoding/decoding a video signal.

The present disclosure provides a method of setting a different prediction mode for each prediction unit within a block when encoding/decoding a video signal.

The present disclosure provides a method of variably determining the size of a blending area for a block to which geometric division is applied when encoding/decoding a video signal.

The technical problems to be achieved by this disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by those skilled in the art from the description below. You will be able to.

An image decoding method according to the present disclosure includes obtaining a first prediction block based on first prediction information for a first prediction unit in a current block; Obtaining a second prediction block based on second prediction information for a second prediction unit in the current block; and obtaining a final prediction block for the current block based on the first prediction block and the second prediction block. At this time, in the blending area formed around the boundary between the first prediction unit and the second prediction unit in the current block, the first prediction samples in the first prediction block and the second prediction samples in the second prediction block The final prediction sample can be derived by blending the prediction samples.

An image encoding method according to the present disclosure includes obtaining a first prediction block based on first prediction information for a first prediction unit in a current block; Obtaining a second prediction block based on second prediction information for a second prediction unit in the current block; and obtaining a final prediction block for the current block based on the first prediction block and the second prediction block. At this time, in the blending area formed around the boundary between the first prediction unit and the second prediction unit in the current block, the first prediction samples in the first prediction block and the second prediction samples in the second prediction block The final prediction sample can be derived by blending the prediction samples.

In the video signal encoding/decoding method according to the present disclosure, the size of the blending area may be determined based on one of a plurality of blending area size candidates.

In the video signal encoding/decoding method according to the present disclosure, selecting one of the plurality of blending area size candidates may be performed based on index information decoded from a bitstream.

In the video signal encoding/decoding method according to the present disclosure, selecting one of the plurality of blending area size candidates is based on at least one of motion information of the first prediction unit and motion information of the second prediction unit. It can be done.

In the video signal encoding/decoding method according to the present disclosure, the first weight applied to the first prediction samples and the second weight applied to the second prediction samples in the blending region are adaptively adjusted according to location. can be decided.

In the video signal encoding/decoding method according to the present disclosure, a slope indicating the degree of change in the first weight within the blending area may be inversely proportional to the size of the blending area.

In the video signal encoding/decoding method according to the present disclosure, the plurality of blending area size candidates are asymmetric in that the size in the direction of the first prediction unit is different from the size in the direction of the second prediction unit with respect to the boundary. Can include blending area size candidates.

In the video signal encoding/decoding method according to the present disclosure, when prediction modes between the first prediction unit and the second prediction unit are different, the blending area may have an asymmetric shape with respect to the boundary.

In the video signal encoding/decoding method according to the present disclosure, when intra prediction is applied to the first prediction unit and inter prediction is applied to the second prediction unit, the direction of the first prediction unit is based on the boundary. The size of the blending area may be smaller than the size of the blending area in the second prediction unit direction.

In the video signal encoding/decoding method according to the present disclosure, the intra prediction mode of the first prediction unit is a planner mode, an intra prediction mode parallel to a dividing line dividing the current block into two, or an intra prediction mode perpendicular to the dividing line. It can be decided on one of the prediction modes.

The features briefly summarized above with respect to the present disclosure are merely exemplary aspects of the detailed description of the present disclosure described below, and do not limit the scope of the present disclosure.

According to the present disclosure, block prediction accuracy can be improved by using a merge mode involving geometric partitioning.

According to the present disclosure, when a merge mode involving geometric partitioning is applied, prediction accuracy of a block can be improved by modifying a motion vector using an offset vector.

According to the present disclosure, prediction accuracy can be improved by providing a method of setting a different prediction mode for each prediction unit in a block.

According to the present disclosure, prediction accuracy can be improved by providing a method for variably determining the size of an area where intra-block blending is performed.

The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

1 is a block diagram of a video encoder (encoder) according to an embodiment of the present disclosure.
Figure 2 is a block diagram of a video decoder (decoder) according to an embodiment of the present disclosure.
Figure 3 is a diagram illustrating a basic coding tree unit according to an embodiment of the present disclosure.
Figure 4 is a diagram showing various types of division of a coding block.
Figure 5 is a diagram illustrating a division aspect of a coding tree unit.
Figure 6 is a flowchart of an inter prediction method according to an embodiment of the present disclosure.
Figure 7 is a flowchart of the process of deriving motion information of the current block under merge mode.
Figure 8 is an example diagram illustrating merge candidate blocks.
Figure 9 is a diagram showing the types of dividing lines that divide coding blocks.
Figure 10 shows an example in which different prediction modes are applied to each of the first prediction unit and the second prediction unit.
11 shows that the weight applied to the first prediction sample and the weight applied to the second prediction sample are adaptively determined depending on the location where the weighted sum operation is performed, based on the boundary between the first prediction unit and the second prediction unit. This is a drawing showing an example.
Figure 12 is a diagram showing a plurality of blending area size candidates.
13 is a flowchart of an intra prediction method according to an embodiment of the present disclosure.
Figure 14 is a diagram showing intra prediction modes.
Figure 15 shows the positions of neighboring blocks used to derive the intra prediction mode of the current block.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

Encoding and decoding of images are performed on a block basis. For example, encoding/decoding processing such as transformation, quantization, prediction, in-loop filtering, or restoration may be performed on a coding block, transform block, or prediction block.

Hereinafter, the block subject to encoding/decoding will be referred to as the 'current block'. As an example, the current block may represent a coding block, a transform block, or a prediction block, depending on the current encoding/decoding processing step.

In addition, the term 'unit' used in this specification may be understood to represent a basic unit for performing a specific encoding/decoding process, and the 'block' may be understood to represent a sample array of a predetermined size. Unless otherwise stated, ‘block’ and ‘unit’ can be used with the same meaning. For example, in an embodiment described later, a coding block and a coding unit may be understood to have equivalent meaning.

1 is a block diagram of a video encoder (encoder) according to an embodiment of the present disclosure.

Referring to FIG. 1, the image encoding device 100 includes a picture segmentation unit 110, prediction units 120 and 125, a conversion unit 130, a quantization unit 135, a reordering unit 160, and an entropy encoding unit ( 165), an inverse quantization unit 140, an inverse transform unit 145, a filter unit 150, and a memory 155.

Each component shown in FIG. 1 is shown independently to represent different characteristic functions in the video encoding device, and does not mean that each component is comprised of separate hardware or a single software component. That is, each component is listed and included as a separate component for convenience of explanation, and at least two of each component can be combined to form one component, or one component can be divided into a plurality of components to perform a function, and each of these components can be divided into a plurality of components. Integrated embodiments and separate embodiments of the constituent parts are also included in the scope of the present disclosure as long as they do not deviate from the essence of the present disclosure.

Additionally, some components may not be essential components that perform essential functions in the present disclosure, but may simply be optional components to improve performance. The present disclosure can be implemented by including only essential components for implementing the essence of the present disclosure, excluding components used only to improve performance, and a structure that includes only essential components excluding optional components used only to improve performance. is also included in the scope of rights of this disclosure.

The picture division unit 110 may divide the input picture into at least one processing unit. At this time, the processing unit may be a prediction unit (PU), a transformation unit (TU), or a coding unit (CU). The picture division unit 110 divides one picture into a combination of a plurality of coding units, prediction units, and transformation units, and combines one coding unit, prediction unit, and transformation unit based on a predetermined standard (for example, a cost function). You can encode the picture by selecting .

For example, one picture may be divided into a plurality of coding units. To split the coding unit in a picture, a recursive tree structure such as the Quad Tree Structure can be used. Coding is split into other coding units with one image or the largest coding unit as the root. A unit can be divided into child nodes equal to the number of divided coding units. A coding unit that is no longer divided according to certain restrictions becomes a leaf node. That is, assuming that only square division is possible for one coding unit, one coding unit can be divided into up to four different coding units.

Hereinafter, in the embodiments of the present disclosure, the coding unit may be used to mean a unit that performs encoding or may be used to mean a unit that performs decoding.

A prediction unit may be divided into at least one square or rectangular shape of the same size within one coding unit, and any one of the prediction units divided within one coding unit may be a prediction unit of another prediction unit. It may be divided to have a different shape and/or size than the unit.

If the prediction unit for which intra prediction is performed based on the coding unit is not the minimum coding unit when generated, intra prediction can be performed without dividing the prediction unit into a plurality of prediction units NxN.

The prediction units 120 and 125 may include an inter prediction unit 120 that performs inter prediction and an intra prediction unit 125 that performs intra prediction. It is possible to determine whether to use inter prediction or intra prediction for a prediction unit, and determine specific information (eg, intra prediction mode, motion vector, reference picture, etc.) according to each prediction method. At this time, the processing unit in which the prediction is performed and the processing unit in which the prediction method and specific contents are determined may be different. For example, the prediction method and prediction mode are determined in prediction units, and prediction may be performed in transformation units. The residual value (residual block) between the generated prediction block and the original block may be input to the conversion unit 130. Additionally, prediction mode information, motion vector information, etc. used for prediction may be encoded in the entropy encoder 165 together with the residual value and transmitted to the decoder. When using a specific encoding mode, it is possible to encode the original block as is and transmit it to the decoder without generating a prediction block through the prediction units 120 and 125.

The inter prediction unit 120 may predict a prediction unit based on information on at least one picture among the pictures before or after the current picture, and in some cases, prediction based on information on a partially encoded region within the current picture. Units can also be predicted. The inter prediction unit 120 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

The reference picture interpolation unit may receive reference picture information from the memory 155 and generate pixel information of an integer number of pixels or less from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information of an integer pixel or less in 1/4 pixel units. In the case of color difference signals, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information of an integer pixel or less in 1/8 pixel units.

The motion prediction unit may perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm) can be used to calculate the motion vector. The motion vector may have a motion vector value in units of 1/2 or 1/4 pixels based on the interpolated pixels. The motion prediction unit can predict the current prediction unit by using a different motion prediction method. As a motion prediction method, various methods such as the skip method, the merge method, the Advanced Motion Vector Prediction (AMVP) method, and the intra block copy method can be used.

The intra prediction unit 125 may generate a prediction unit based on reference pixel information around the current block, which is pixel information in the current picture. If the neighboring block of the current prediction unit is a block on which inter prediction has been performed and the reference pixel is a pixel on which inter prediction has been performed, the reference pixel included in the block on which inter prediction has been performed is the reference pixel of the block on which intra prediction has been performed. It can be used in place of information. That is, when the reference pixel is not available, the unavailable reference pixel information can be replaced with at least one reference pixel among the available reference pixels.

In intra prediction, the prediction mode can include a directional prediction mode that uses reference pixel information according to the prediction direction and a non-directional mode that does not use directional information when performing prediction. The mode for predicting luminance information and the mode for predicting chrominance information may be different, and intra prediction mode information used to predict luminance information or predicted luminance signal information may be used to predict chrominance information.

When performing intra prediction, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit is made based on the pixel on the left, the pixel on the top left, and the pixel on the top of the prediction unit. can be performed. However, when performing intra prediction, if the size of the prediction unit and the size of the transformation unit are different, intra prediction can be performed using a reference pixel based on the transformation unit. Additionally, intra prediction using NxN partitioning can be used only for the minimum coding unit.

The intra prediction method can generate a prediction block after applying an Adaptive Intra Smoothing (AIS) filter to the reference pixel according to the prediction mode. The type of AIS filter applied to the reference pixel may be different. To perform the intra prediction method, the intra prediction mode of the current prediction unit can be predicted from the intra prediction mode of prediction units existing around the current prediction unit. When predicting the prediction mode of the current prediction unit using predicted mode information from neighboring prediction units, if the intra prediction mode of the current prediction unit and neighboring prediction units are the same, predetermined flag information is used to predict the current prediction unit and neighboring prediction units. Information that the prediction modes of are the same can be transmitted, and if the prediction modes of the current prediction unit and neighboring prediction units are different, entropy encoding can be performed to encode the prediction mode information of the current block.

Additionally, based on the prediction units generated by the prediction units 120 and 125, a residual block may be generated that includes residual information that is the difference between the prediction unit on which prediction was performed and the original block of the prediction unit. The generated residual block may be input to the conversion unit 130.

The transform unit 130 transforms the residual block including the original block and the residual value information of the prediction unit generated through the prediction units 120 and 125, such as DCT (Discrete Cosine Transform) or DST (Discrete Sine Transform). It can be converted using the method. Here, the DCT conversion core includes at least one of DCT2 or DCT8, and the DST conversion core includes DST7. Whether to apply DCT or DST to transform the residual block can be determined based on intra prediction mode information of the prediction unit used to generate the residual block. Transformation for residual blocks can also be skipped. A flag indicating whether to skip transformation for the residual block can be encoded. Transform skipping may be allowed for residual blocks whose size is below a threshold, luma component, or chroma component under 4:4:4 format.

The quantization unit 135 may quantize the values converted to the frequency domain by the conversion unit 130. The quantization coefficient may change depending on the block or the importance of the image. The value calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the realignment unit 160.

The rearrangement unit 160 may rearrange coefficient values for the quantized residual values.

The rearrangement unit 160 can change the coefficients in a two-dimensional block form into a one-dimensional vector form through a coefficient scanning method. For example, the realignment unit 160 can scan from DC coefficients to coefficients in the high frequency region using a zig-zag scan method and change it into a one-dimensional vector form. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional block-type coefficients in the column direction or a horizontal scan that scans the two-dimensional block-type coefficients in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, it can be determined which scan method among zig-zag scan, vertical scan, and horizontal scan will be used.

The entropy encoding unit 165 may perform entropy encoding based on the values calculated by the reordering unit 160. Entropy coding can use various coding methods, such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

The entropy encoding unit 165 receives the residual value coefficient information and block type information of the coding unit, prediction mode information, division unit information, prediction unit information and transmission unit information, and motion information from the reordering unit 160 and the prediction units 120 and 125. Various information such as vector information, reference frame information, block interpolation information, and filtering information can be encoded.

The entropy encoding unit 165 may entropy encode the coefficient value of the coding unit input from the reordering unit 160.

The inverse quantization unit 140 and the inverse transformation unit 145 inversely quantize the values quantized in the quantization unit 135 and inversely transform the values transformed in the transformation unit 130. The residual value generated in the inverse quantization unit 140 and the inverse transform unit 145 is restored by combining prediction units predicted through the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction units 120 and 125. You can create a block (Reconstructed Block).

The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

The deblocking filter can remove block distortion caused by boundaries between blocks in the restored picture. To determine whether to perform deblocking, it is possible to determine whether to apply a deblocking filter to the current block based on the pixels included in several columns or rows included in the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be applied depending on the required deblocking filtering strength. Additionally, when applying a deblocking filter, horizontal filtering and vertical filtering can be processed in parallel when vertical filtering and horizontal filtering are performed.

The offset correction unit may correct the offset of the deblocked image from the original image in pixel units. In order to perform offset correction for a specific picture, the pixels included in the image are divided into a certain number of areas, then the area to perform offset is determined and the offset is applied to that area, or the offset is performed by considering the edge information of each pixel. You can use the method of applying .

Adaptive Loop Filtering (ALF) can be performed based on a comparison between the filtered restored image and the original image. After dividing the pixels included in the image into predetermined groups, filtering can be performed differentially for each group by determining one filter to be applied to that group. Information related to whether to apply ALF may be transmitted for each coding unit (CU), and the shape and filter coefficients of the ALF filter to be applied may vary for each block. Additionally, an ALF filter of the same type (fixed type) may be applied regardless of the characteristics of the block to which it is applied.

The memory 155 may store a reconstructed block or picture calculated through the filter unit 150, and the stored reconstructed block or picture may be provided to the prediction units 120 and 125 when inter prediction is performed.

Figure 2 is a block diagram of a video decoder (decoder) according to an embodiment of the present disclosure.

Referring to FIG. 2, the image decoder 200 includes an entropy decoder 210, a reordering unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, 235, and a filter unit ( 240) and memory 245 may be included.

When a video bitstream is input from a video encoder, the input bitstream can be decoded in a procedure opposite to that of the video encoder.

The entropy decoding unit 210 may perform entropy decoding in a procedure opposite to the procedure in which entropy encoding is performed in the entropy encoding unit of the video encoder. For example, various methods such as Exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) can be applied in response to the method performed in the image encoder.

The entropy decoder 210 can decode information related to intra prediction and inter prediction performed by the encoder.

The reordering unit 215 may rearrange the bitstream entropy-decoded by the entropy decoding unit 210 based on the method in which the encoder rearranges the bitstream. Coefficients expressed in the form of a one-dimensional vector can be restored and rearranged as coefficients in the form of a two-dimensional block. The reordering unit 215 may receive information related to coefficient scanning performed by the encoder and perform reordering by reverse scanning based on the scanning order performed by the encoder.

The inverse quantization unit 220 may perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values of the rearranged blocks.

The inverse transform unit 225 may perform inverse transform, that is, inverse DCT or inverse DST, on the transform, that is, DCT or DST, performed by the transformer on the quantization result performed by the image encoder. Here, the DCT conversion core may include at least one of DCT2 or DCT8, and the DST conversion core may include DST7. Alternatively, if transformation is skipped in the video encoder, the inverse transformation unit 225 may not perform the inverse transformation. Inverse transformation can be performed based on the transmission unit determined by the video encoder. The inverse transform unit 225 of the video decoder may selectively perform a transformation technique (eg, DCT or DST) according to a plurality of information such as a prediction method, the size of the current block, and the prediction direction.

The prediction units 230 and 235 may generate a prediction block based on prediction block generation-related information provided by the entropy decoder 210 and previously decoded block or picture information provided by the memory 245.

As described above, when performing intra prediction in the same manner as in the operation of the video encoder, when the size of the prediction unit and the size of the transformation unit are the same, the pixel that exists on the left of the prediction unit, the pixel that exists in the upper left, and the pixel that exists on the top Intra prediction is performed for the prediction unit based on the pixel, but when performing intra prediction, if the size of the prediction unit and the size of the transformation unit are different, intra prediction is performed using a reference pixel based on the transformation unit. You can. Additionally, intra prediction using NxN partitioning only for the minimum coding unit can be used.

The prediction units 230 and 235 may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit discriminator receives various information such as prediction unit information input from the entropy decoder 210, prediction mode information of the intra prediction method, and motion prediction-related information of the inter prediction method, distinguishes the prediction unit from the current coding unit, and makes predictions. It is possible to determine whether a unit performs inter-prediction or intra-prediction. The inter prediction unit 230 uses the information required for inter prediction of the current prediction unit provided by the video encoder to make current prediction based on information included in at least one picture of the picture before or after the current picture including the current prediction unit. Inter prediction for units can be performed. Alternatively, inter prediction may be performed based on information on a pre-restored partial region within the current picture including the current prediction unit.

To perform inter prediction, based on the coding unit, the motion prediction methods of the prediction unit included in the coding unit are Skip Mode, Merge Mode, Motion Vector Prediction Mode (AMVP Mode), and Intra Block Copy. It is possible to determine which of the modes is used.

The intra prediction unit 235 may generate a prediction block based on pixel information in the current picture. If the prediction unit is a prediction unit that has performed intra prediction, intra prediction can be performed based on the intra prediction mode information of the prediction unit provided by the video encoder. The intra prediction unit 235 may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a part that performs filtering on the reference pixels of the current block, and can be applied by determining whether or not to apply the filter according to the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixel of the current block using the prediction mode and AIS filter information of the prediction unit provided by the video encoder. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

If the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on pixel values obtained by interpolating the reference pixel, the reference pixel interpolator may interpolate the reference pixel to generate a reference pixel in pixel units of an integer value or less. If the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is DC mode.

The restored block or picture may be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

Information on whether a deblocking filter has been applied to the corresponding block or picture can be received from the video encoder, and when a deblocking filter has been applied, information on whether a strong filter or a weak filter has been applied. In the deblocking filter of the video decoder, information related to the deblocking filter provided by the video encoder is provided and the video decoder can perform deblocking filtering on the corresponding block.

The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image during encoding and offset value information.

ALF can be applied to the coding unit based on ALF application availability information, ALF coefficient information, etc. provided from the encoder. This ALF information may be included and provided in a specific parameter set.

The memory 245 can store the restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to an output unit.

Figure 3 is a diagram illustrating a basic coding tree unit according to an embodiment of the present disclosure.

The coding block of the maximum size can be defined as a coding tree block. One picture is divided into a plurality of Coding Tree Units (CTU). The coding tree unit is the largest coding unit and may also be called LCU (Largest Coding Unit). Figure 3 shows an example in which one picture is divided into a plurality of coding tree units.

The size of the coding tree unit can be defined at the picture level or sequence level. For this purpose, information indicating the size of the coding tree unit may be signaled through a picture parameter set or sequence parameter set.

As an example, the size of the coding tree unit for all pictures in the sequence may be set to 128x128. Alternatively, either 128x128 or 256x256 can be determined as the size of the coding tree unit at the picture level. For example, in the first picture, the size of the coding tree unit may be set to 128x128, and in the second picture, the size of the coding tree unit may be set to 256x256.

By dividing the coding tree unit, a coding block can be created. A coding block represents a basic unit for encoding/decoding processing. For example, prediction or transformation may be performed for each coding block, or a prediction encoding mode may be determined for each coding block. Here, the prediction encoding mode represents a method of generating a prediction image. As an example, the predictive coding mode may be Intra Prediction (Intra Prediction), Inter Prediction (Inter Prediction), Current Picture Referencing (CPR), or Intra Block Copy (IBC). ) or may include Combined Prediction. A prediction block for a coding block can be generated using at least one prediction coding mode among intra prediction, inter prediction, current picture reference, or complex prediction.

Information indicating the prediction encoding mode of the current block may be signaled through a bitstream. As an example, the information may be a 1-bit flag indicating whether the predictive coding mode is intra mode or inter mode. Only when the prediction coding mode of the current block is determined to be inter mode, current picture reference or complex prediction may be used.

Current picture reference is used to set the current picture as a reference picture and obtain a prediction block of the current block from a region in the current picture that has already been encoded/decoded. Here, the current picture refers to a picture including the current block. Information indicating whether the current picture reference is applied to the current block may be signaled through the bitstream. As an example, the information may be a 1-bit flag. If the flag is true, the prediction coding mode of the current block may be determined as the current picture reference, and if the flag is false, the prediction mode of the current block may be determined as inter prediction.

Alternatively, the prediction coding mode of the current block may be determined based on the reference picture index. For example, when the reference picture index points to the current picture, the prediction coding mode of the current block may be determined based on the current picture reference. If the reference picture index points to a picture other than the current picture, the prediction coding mode of the current block may be determined as inter prediction. In other words, current picture reference is a prediction method using information from a region in the current picture where coding/decoding has been completed, and inter prediction is a prediction method using information from another picture where coding/decoding has been completed.

Complex prediction refers to a coding mode that combines two or more of intra prediction, inter prediction, and current picture reference. For example, when complex prediction is applied, a first prediction block may be generated based on one of intra prediction, inter prediction, or a current picture reference, and a second prediction block may be generated based on the other. When the first prediction block and the second prediction block are generated, the final prediction block may be generated through an average operation or a weighted sum operation of the first prediction block and the second prediction block. Information indicating whether complex prediction is applied may be signaled through the bitstream. The information may be a 1-bit flag.

Figure 4 is a diagram showing various types of division of a coding block.

A coding block may be divided into a plurality of coding blocks based on quad tree partitioning, binary tree partitioning, or triple tree partitioning. The divided coding block may also be divided into a plurality of coding blocks based on quad tree division, binary tree division, or triple tree division.

Quad tree partitioning refers to a partitioning technique that divides the current block into 4 blocks. As a result of quad tree partitioning, the current block can be divided into four square partitions (see 'SPLIT_QT' in (a) of Figure 4).

Binary tree splitting refers to a splitting technique that splits the current block into two blocks. Splitting the current block into two blocks along the vertical direction (i.e., using a vertical line across the current block) can be called vertical binary tree splitting, and dividing the current block into two blocks along the horizontal direction (i.e., using a vertical line across the current block) can be called vertical binary tree splitting. Splitting the current block into two blocks (using a horizontal line) can be called horizontal binary tree splitting. As a result of binary tree partitioning, the current block can be divided into two non-square partitions. 'SPLIT_BT_VER' in FIG. 4 shows the result of vertical binary tree division, and 'SPLIT_BT_HOR' in FIG. 4 (c) shows the result of horizontal binary tree division.

Triple tree partitioning refers to a partitioning technique that divides the current block into three blocks. Splitting the current block into three blocks along the vertical direction (i.e., using two vertical lines crossing the current block) can be called vertical triple tree division, and along the horizontal direction (i.e., using two vertical lines across the current block). Splitting the current block into three blocks (using two horizontal lines crossing) can be called horizontal triple tree division. As a result of triple tree partitioning, the current block can be divided into three non-square partitions. At this time, the width/height of the partition located in the center of the current block may be twice the width/height of other partitions. 'SPLIT_TT_VER' in FIG. 4 shows the result of vertical triple tree division, and 'SPLIT_TT_HOR' in FIG. 4 (e) shows the result of triple tree division in the horizontal direction.

The number of divisions of a coding tree unit can be defined as partitioning depth. The maximum division depth of a coding tree unit may be determined at the sequence or picture level. Accordingly, the maximum division depth of the coding tree unit may be different for each sequence or pilture.

Alternatively, the maximum segmentation depth for each segmentation technique can be determined individually. As an example, the maximum splitting depth allowed for quad tree splitting may be different from the maximum splitting depth allowed for binary tree splitting and/or triple tree splitting.

The encoder may signal information indicating at least one of the division type or division depth of the current block through a bitstream. The decoder may determine the division type and division depth of the coding tree unit based on the information parsed from the bitstream.

Figure 5 is a diagram illustrating a division aspect of a coding tree unit.

Partitioning a coding block using a partitioning technique such as quad tree partitioning, binary tree partitioning, and/or triple tree partitioning may be referred to as multi tree partitioning.

Coding blocks generated by applying multi-tree partitioning to a coding block may be called lower coding blocks. When the division depth of a coding block is k, the division depth of lower coding blocks is set to k+1.

Conversely, for coding blocks with a split depth of k+1, a coding block with a split depth of k may be called a higher coding block.

The partition type of the current coding block may be determined based on at least one of the partition type of the upper coding block or the partition type of the neighboring coding block. Here, the neighboring coding block is adjacent to the current coding block and may include at least one of a top neighboring block, a left neighboring block, or a neighboring block adjacent to the upper left corner of the current coding block. Here, the split type may include at least one of quad tree split, binary tree split, binary tree split direction, triple tree split, and triple tree split direction.

To determine the division type of the coding block, information indicating whether the coding block is divided may be signaled through the bitstream. The information is a 1-bit flag 'split_cu_flag', and if the flag is true, it indicates that the coding block is split by the head tree splitting technique.

If split_cu_flag is true, information indicating whether the coding block is quad-tree split may be signaled through the bitstream. The information is a 1-bit flag split_qt_flag. When the flag is true, the coding block can be divided into 4 blocks.

For example, in the example shown in FIG. 5, as the coding tree unit is divided into quad trees, four coding blocks with a division depth of 1 are generated. In addition, it is shown that quad-tree partitioning was applied again to the first and fourth coding blocks among the four coding blocks generated as a result of quad-tree partitioning. As a result, four coding blocks with a division depth of 2 can be generated.

Additionally, by applying quad tree division again to the coding block with a division depth of 2, a coding block with a division depth of 3 can be generated.

If quad tree partitioning is not applied to a coding block, binary tree partitioning is performed on the coding block, taking into account at least one of the size of the coding block, whether the coding block is located at a picture boundary, the maximum partition depth, or the partition type of the neighboring block. Alternatively, you can decide whether to perform triple tree partitioning. If it is determined that binary tree division or triple tree division is performed on the coding block, information indicating the division direction may be signaled through the bitstream. The information may be a 1-bit flag mtt_split_cu_vertical_flag. Based on the flag, it can be determined whether the division direction is vertical or horizontal. Additionally, information indicating whether binary tree partitioning or triple tree partitioning is applied to the coding block may be signaled through the bitstream. The information may be a 1-bit flag mtt_split_cu_binary_flag. Based on the flag, it can be determined whether binary tree partitioning or triple tree partitioning is applied to the coding block.

For example, in the example shown in FIG. 5, vertical binary tree division is applied to a coding block with a division depth of 1, and vertical triple tree division is applied to the left coding block among the coding blocks generated as a result of the division, It is shown that vertical binary tree partitioning is applied to the right coding block.

Inter prediction is a predictive coding mode that predicts the current block using information from the previous picture. As an example, a block at the same location as the current block in the previous picture (hereinafter referred to as collocated block) can be set as the prediction block of the current block. Hereinafter, a prediction block generated based on a block at the same location as the current block will be referred to as a collocated prediction block.

On the other hand, if an object that existed in the previous picture has moved to a different location in the current picture, the current block can be effectively predicted using the movement of the object. For example, if the movement direction and size of an object can be known by comparing the previous picture and the current picture, a prediction block (or prediction image) of the current block can be generated by considering the movement information of the object. Hereinafter, a prediction block generated using motion information may be referred to as a motion prediction block.

A residual block can be generated by differentiating the prediction block from the current block. At this time, if there is movement of the object, the energy of the residual block can be reduced by using a motion prediction block instead of a collocated prediction block, and thus the compression performance of the residual block can be improved.

As above, generating a prediction block using motion information can be called motion compensation prediction. In most inter predictions, prediction blocks can be generated based on motion compensation prediction.

The motion information may include at least one of a motion vector, reference picture index, prediction direction, or bidirectional weight index. A motion vector represents the movement direction and size of an object. The reference picture index specifies the reference picture of the current block among reference pictures included in the reference picture list. The prediction direction refers to either one-way L0 prediction, one-way L1 prediction, or two-way prediction (L0 prediction and L1 prediction). Depending on the prediction direction of the current block, at least one of motion information in the L0 direction or motion information in the L1 direction may be used. The bidirectional weight index specifies the weight applied to the L0 prediction block and the weight applied to the L1 prediction block.

Figure 6 is a flowchart of an inter prediction method according to an embodiment of the present disclosure.

Referring to FIG. 6, the inter prediction method includes determining the inter prediction mode of the current block (S601), acquiring motion information of the current block according to the determined inter prediction mode (S602), and predicting motion information based on the obtained motion information. Thus, it includes a step of performing motion compensation prediction for the current block (S603).

Here, the inter prediction mode represents various techniques for determining motion information of the current block and may include an inter prediction mode using translation motion information and an inter prediction mode using affine motion information. there is. As an example, the inter prediction mode using translational motion information may include a merge mode and a motion vector prediction mode, and the inter prediction mode using affine motion information may include an affine merge mode and an affine motion vector prediction mode. . The motion information of the current block may be determined based on information parsed from a bitstream or a neighboring block neighboring the current block, depending on the inter prediction mode.

The motion information may include at least one of a motion vector, a reference picture index, a prediction flag, and a bidirectional weight index. Additionally, motion information can be derived/set individually for each of the L0 and L1 directions.

Here, the motion vector represents the position difference between the object in the previous picture and the object in the current picture. The previous picture may be indicated by a reference picture index.

Additionally, information for determining the precision of the motion vector may be signaled through the bitstream. The information may be determined on a sequence basis, a slice basis, or a block basis. Motion vector precision can be set to octopel, quarterpel, halfpel, integerpel, 2 integerpel, or 4 integerpel. Alternatively, depending on the inter prediction mode, the motion vector precision of the current block may be determined, or the number/type of available motion vector precision candidates may be adaptively determined.

Motion information of the current block may be derived from motion information of other blocks in the current block. Here, the other block may be a block that has been encoded/decoded through inter prediction before the current block. Merge mode can be defined as setting the motion information of the current block to be the same as the motion information of other blocks. Additionally, setting the motion vector of another block as the predicted value of the motion vector of the current block can be defined as a motion vector prediction mode.

Figure 7 is a flowchart of the process of deriving motion information of the current block under merge mode.

A merge candidate for the current block can be derived (S701). The merge candidate for the current block may be derived from a block encoded/decoded by inter prediction prior to the current block.

At this time, the block used to derive the merge candidate may be called a ‘merge candidate block’. The merge candidate block may include at least one of a block adjacent to the current block (adjacent merge candidate block) or a block that is not adjacent to the current block (non-adjacent merge candidate block). Based on the motion information of the merge candidate block, motion information of the merge candidate may be set. As an example, the motion information of the merge candidate may be set to be the same as the motion information of the merge candidate block. Alternatively, either the L0 motion information or the L1 motion information of the merge candidate block can be set as the motion information of the merge candidate. Alternatively, the motion vector of the merge candidate block may be derived by scaling the motion vector of the merge candidate block.

Figure 8 is an example diagram illustrating merge candidate blocks.

In Figure 8, the positions of samples for determining the positions of merge candidate blocks are indicated. The position of the sample that determines the merge candidate blocks may be called a reference position sample.

The adjacent merge candidate block includes a reference position sample belonging to a row adjacent to the top of the current block or a column adjacent to the left of the current block. For example, in the example of FIG. 8, each of blocks including reference position samples marked with index 1 to index 5 may be set as an adjacent merge candidate block.

A non-adjacent merge candidate block includes a reference position sample belonging to a row that is not adjacent to the top of the current block or a column that is not adjacent to the left of the current blog. For example, in the example of FIG. 8, each of blocks including reference position samples specified with index 6 to index 23 may be set as a non-adjacent merge candidate block.

The positions of adjacent merge candidate blocks and/or non-adjacent merge candidate blocks may be predefined in the encoder and decoder. For example, as in the example shown in FIG. 8, reference position samples can be set at predetermined intervals between the boundary of the current block and the CTU boundary. As an example, a non-adjacent block in which ^at least one of the can be defined. Here, n may be a natural number such as 1, 2, or 3. For example, if the coordinates of the upper left sample of the current block are (0,0) and the adjacent merge candidate block includes a reference position sample at (-1, -1) position, (-1-4N, -1-4N ) Blocks containing the reference position sample of the position may be defined as non-adjacent merge candidate blocks. At this time, the maximum value of N may be determined by at least one of the distance between the current block and the maximum CTU boundary or a value predefined in the encoder and decoder.

As in the example of FIG. 8, a reference position sample can be set outside the boundary of the CTU to which the current block belongs (indexes 19 to 23 in FIG. 8). Accordingly, a block belonging to a CTU different from the current block may be set as a merge candidate block.

However, to prevent the line buffer from increasing indiscriminately, samples that are not adjacent to the CTU boundary may not be used as reference position samples. Accordingly, non-adjacent blocks adjacent to the CTU boundary can be used as merge candidate blocks, but non-adjacent blocks not adjacent to the CTU boundary cannot be used as merge candidate blocks.

A merge candidate list including merge candidates can be created (S702).

The indices of merge candidates in the merge candidate list may be assigned according to a predetermined order. For example, a merge candidate derived from a left neighboring block, a merge candidate derived from a top neighboring block, a merge candidate derived from an upper right neighboring block, a merge candidate derived from a lower left neighboring block, a merge candidate derived from an upper left neighboring block. And indexes can be assigned in the order of merge candidates derived from temporal neighboring blocks. Alternatively, the index may be assigned in the following order: merge candidates derived from adjacent merge candidate blocks, merge candidates derived from non-adjacent merge candidate blocks, and merge candidates derived from temporal neighboring blocks.

If the merge candidate list includes a plurality of merge candidates, at least one of the plurality of merge candidates can be selected (S703). Specifically, information for specifying one of a plurality of merge candidates may be signaled through a bitstream. As an example, information merge_idx indicating the index of one of the merge candidates included in the merge candidate list may be signaled through a bitstream.

A coding block may be divided into a plurality of prediction units, and prediction may be performed on each of the divided prediction units. Here, the prediction unit represents the basic unit for performing prediction. A method of performing prediction by dividing a coding block into a plurality of prediction units may be called geometric partition mode (GPM).

A coding block can be divided using at least one dividing line. For example, a coding block may be divided using at least one of a vertical line, a horizontal line, an oblique line, or a diagonal line.

Figure 9 is a diagram showing the types of dividing lines that divide coding blocks.

The number of prediction units generated by dividing the coding block may be 2, 3, 4, or more. For convenience of explanation, in the embodiments described later, it is assumed that the coding block is divided into two prediction units, and each of the two prediction units is referred to as a first prediction unit and a second prediction unit.

Information for determining the partition type of a coding block may be signaled through a bitstream. Specifically, based on the above information, at least one of the number of dividing lines dividing the coding block, the angle of the dividing lines, or the position of the dividing lines can be determined.

As an example, the syntax merge_gpm_partition_idx indicating one of the combination candidates of the variable angleIdx indicating the angle of the dividing line and the variable distanceIdx indicating the position of the dividing line may be explicitly signaled through the bitstream.

Here, the variable angleIdx may indicate one of a plurality of angle candidates, and the plurality of angle candidates may be defined as in the example shown in FIG. 9. The variable distanceIdx may represent one of a plurality of distance candidates indicating the distance from the center of the coding block to the dividing line. Depending on the variable distanceIdx, the coding block may be divided based on a dividing line that passes through the center of the coding block, or the coding block may be divided based on a dividing line that does not pass through the center of the coding block.

Table 1 shows an example in which different index values are assigned to each combination of the variable angleIdx and the variable distanceIdx.

merge_gpm_partition_idx 0 One 2 3 4 5 6 7 8 9 10 11 12 13 14 15 angleIdx 0 0 2 2 2 2 3 3 3 3 4 4 4 4 5 5 distanceIdx One 3 0 One 2 3 0 One 2 3 0 One 2 3 0 One merge_gpm_partition_idx 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 angleIdx 5 5 8 8 11 11 11 11 12 12 12 12 13 13 13 13 distanceIdx 2 3 One 3 0 One 2 3 0 One 2 3 0 One 2 3 merge_gpm_partition_idx 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 angleIdx 14 14 14 14 16 16 18 18 18 19 19 19 20 20 20 21 distanceIdx 0 One 2 3 One 3 One 2 3 One 2 3 One 2 3 One merge_gpm_partition_idx 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 angleIdx 21 21 24 24 27 27 27 28 28 28 29 29 29 30 30 30 distanceIdx 2 3 One 3 One 2 3 One 2 3 One 2 3 One 2 3

Unlike the above-described example, the syntax for determining the variable angleIdx indicating the angle of the dividing line and the syntax for determining the variable distanceIdx indicating the position of the dividing line may be signaled separately. Meanwhile, depending on the size and/or shape of the coding block, signaling of one of the two syntaxes may be omitted.

Depending on the size and/or shape of the coding block, the number of angle candidates that the variable angleIdx can indicate and/or the number of distance candidates that the variable distanceIdx can indicate may be adaptively determined.

After dividing the coding block into a first prediction unit and a second prediction unit, motion information for each of the first prediction unit and the second prediction unit can be derived. As an example, based on a merge candidate list derived based on the position and size of the coding block, merge candidates for each of the first prediction unit and the second prediction unit are selected, and movement of each of the first prediction unit and the second prediction unit is performed. Information can be derived.

To this end, a first merge index, e.g., gpm_merge_idx0, indicating a merge candidate of the first prediction unit, and a second merge index, e.g., gpm_merge_idx1, indicating a merge candidate of the second prediction unit may be signaled, respectively.

The merge candidate of the first prediction unit and the merge candidate of the second prediction unit may be different. Accordingly, when a merge candidate is selected based on the first merge index, a merge candidate for the second prediction unit can be selected from among the remaining merge candidates excluding the merge candidates. That is, after reallocating the indices of the remaining merge candidates excluding the merge candidate indicated by the first merge index, the second merge index can be encoded to indicate one of the merge candidates with the reallocated index. As an example, as a result of reallocation, an index of which 1 is reduced may be assigned to merge candidates whose index is larger than the merge candidate indicated by the first merge index.

In the decoding step, the values of the first merge index and the second merge index can be compared to select a merge candidate for the second prediction unit. For example, when the value of the second merge index gpm_merge_idx1 is smaller than the first merge index gpm_merge_idx0, a merge candidate having the value of the second merge index gpm_merge_idx1 as an index may be selected. On the other hand, if the value of the second merge index gpm_merge_idx1 is equal to or greater than the first merge index gpm_merge_idx0, a merge candidate whose index is the second merge index gpm_merge_idx1 plus 1 may be selected.

The motion information of the first merge candidate selected by the first merge index is set as the motion information of the first prediction unit, and the motion information of the second merge candidate selected by the second merge index is set as the motion information of the second prediction unit. You can.

Thereafter, the first prediction block may be obtained based on the motion information of the first prediction unit, and the second prediction block may be obtained based on the motion information of the second prediction unit. Afterwards, the final prediction block for the coding block can be obtained through a weighted sum operation of the first prediction block and the second prediction block. At this time, the weight applied to the prediction samples included in the first prediction block and the weight applied to the prediction samples included in the second prediction block may be adaptively determined according to the location of the final prediction sample.

Instead of using the merge candidate motion information as is, the motion information of the merge candidate can be modified and the modified motion information can be used. For example, when a merge candidate for the current block is selected, an offset vector added or subtracted from the motion vector of the selected merge candidate can be set as the motion vector of the current block. As above, modifying the motion vector of a merge candidate using an offset vector can be called Merge mode with Motion vector Difference (MMVD).

Information for determining the offset vector may be signaled through a bitstream. Specifically, the syntax mvd_direction_idx indicating the direction of the offset vector and the syntax mvd_distance_idx indicating the size of the offset vector may each be encoded and signaled.

The syntax mvd_direction_idx represents one of a plurality of direction candidates. As an example, Table 2 shows an example in which different index values are assigned to each of a plurality of direction candidates.

mmvd_distance_idx[ x0 ][ y0 ] MmvdDistance[ x0 ][ y0 ] ph_mmvd_fullpel_only_flag = = 0 ph_mmvd_fullpel_only_flag = = One 0 One 4 One 2 8 2 4 16 3 8 32 4 16 64 5 32 128 6 64 256 7 128 512

The syntax mvd_distance_idx represents one of a plurality of distance candidates. As an example, Table 3 shows an example in which different index values are assigned to each of a plurality of distance candidates.

mmvd_direction_idx[ x0 ][ y0 ] MmvdSign[ x0 ][ y0 ][ 0 ] MmvdSign[ x0 ][ y0 ][ One ] 0 +1 0 One -One 0 2 0 +1 3 0 -One

If a direction candidate and a distance candidate are selected using the above two syntaxes, an offset vector can be derived based on Equation 1 below.

In Equation 1, MmvdOffset[x0][y0][0] represents the horizontal component of the offset vector, and MmvdOffset[x0][y0][1] represents the vertical component of the offset vector. MmvdDistance represents the value of the distance candidate selected by mvd_distance_idx, and MmvdSign represents the value of the direction candidate selected by mvd_direction_idx.

GPM mode and MMVD can be used in combination. For example, when deriving at least one of motion information of the first prediction unit or motion information of the second prediction unit, MMVD may be applied. That is, the motion vector of the first prediction unit or the motion vector of the second prediction unit can be derived by adding or subtracting the offset vector from the motion vector of the merge candidate.

Information indicating whether MMVD is applied under GPM mode may be signaled through a bitstream. As an example, the information may be a 1-bit flag. Additionally, the information may be encoded and signaled for each prediction unit.

Table 4 shows an example in which a flag indicating whether to apply MMVD is signaled for each prediction unit.

merge_data(　x0,　y0,　cbWidth,　cbHeight,　chType　) { Descriptor ... gpm_mmvd_flag0 ae(v) gpm_mmvd_flag1 ae(v) if (gpm_mmvd_flag0){ gpm_merge_idx0 ae(v) gpm_mmvd_idx0 ae(v) Else gpm_merge_idx0 ae(v) } if (gpm_mmvd_flag1) gpm_mmvd_idx1 ae(v) if (!gpm_mmvd_flag0) gpm_merge_idx1 ae(v) else gpm_merge_idx1 ae(v) ... }

In Table 4, the syntax gpm_mmvd_flag0 indicates whether MMVD is applied to the first prediction unit. The value of syntax gpm_mmvd_flag0 is 1, indicating that MMVD is applied to the first prediction unit. In this case, the motion vector of the first prediction unit can be derived by adding or subtracting the offset vector from the motion vector of the merge candidate. On the other hand, the value of syntax gpm_mmvd_flag0 is 0, indicating that MMVD is not applied to the first prediction unit. In this case, the motion vector of the first prediction unit may be set to be the same as the motion vector of the merge candidate.

Additionally, the syntax gpm_mmvd_flag1 indicates whether MMVD is applied to the second prediction unit. The value of syntax gpm_mmvd_flag1 is 1, indicating that MMVD is applied to the second prediction unit. In this case, the motion vector of the second prediction unit can be derived by adding or subtracting the offset vector from the motion vector of the merge candidate. On the other hand, the value of syntax gpm_mmvd_flag1 is 0, indicating that MMVD is not applied to the second prediction unit. In this case, the motion vector of the second prediction unit may be set to be the same as the motion vector of the merge candidate.

Unlike the example in Table 4, a 1-bit flag indicating whether MMVD is applied may be signaled at the coding block level. As an example, at the coding block level, the syntax gpm_mmvd_flag indicating whether the MMVD mode is applied in combination with the GPM mode may be encoded and signaled.

When the value of the syntax gpm_mmvd_flag is 0, the motion vector of each of the first prediction unit and the second prediction unit may be set to be the same as the motion vector of their respective merge candidates.

The value of the syntax gpm_mmvd_flag is 1, indicating that MMVD is used in deriving the motion vector of at least one of the first prediction unit or the second prediction unit. In this case, information indicating whether MMVD is applied to each of the first prediction unit and the second prediction unit may be additionally encoded and signaled. As an example, the information may be an index indicating one of multiple cases to which MMVD can be applied. For example, when MMVD is applied only to the first prediction unit, when MMVD is applied only to the second prediction unit, and when MMVD is applied to both the first prediction unit and the second prediction unit, different indices are assigned to each, Information pointing to one of these can be encoded and signaled.

Alternatively, the information may include a first MMVD flag indicating whether MMVD is applied to the first prediction unit and a second MMVD flag indicating whether MMVD is applied to the second prediction unit. The first MMVD flag and the second MMVD flag may be sequentially encoded/decoded. At this time, when the first MMVD flag indicates that MMVD is not applied to the first prediction unit, encoding/decoding of the second MMVD flag may be omitted and it may be determined that MMVD is applied to the second prediction unit.

Alternatively, when it is determined that MMVD is applied at the coding block level, it may be set as a default that MMVD is applied to both the first prediction unit and the second prediction unit.

Or, when it is determined that MMVD is applied at the coding block level, based on the index of the merge candidate for the first prediction unit and the index of the merge candidate for the second prediction unit, respectively, the first prediction unit and the second prediction unit You can also decide whether MMVD applies to .

For example, after comparing the index of the merge candidate for the first prediction unit and the index of the merge candidate for the second prediction unit, MMVD can be applied only to the prediction unit that uses the merge candidate with a small index value.

Alternatively, it is possible to determine whether to apply MMVD to the prediction unit by comparing the index of the merge candidate and the threshold value. For example, after comparing the index of the merge candidate for the first prediction unit and the threshold, if the index is less than the threshold, MMVD can be applied to the first prediction unit. On the other hand, if the index is greater than or equal to the threshold, it may be determined that MMVD is not applied to the first prediction unit. Likewise, after comparing the index of the merge candidate for the second prediction unit and the threshold value, if the index is less than the threshold value, MMVD can be applied to the second prediction unit. On the other hand, if the index is greater than or equal to the threshold, it may be determined that MMVD is not applied to the second prediction unit.

In cases where MMVD is applied and cases where it is not, the maximum number of merge candidates that a prediction unit can select may be set differently.

For example, when MMVD is applied to the first prediction unit, one of N merge candidates included in the merge candidate list may be selected for the first prediction unit. On the other hand, when MMVD is applied, one of the two merge candidates with a small index among the N merge candidates included in the merge candidate list may be selected for the first prediction unit.

The above constraints can be equally applied to the second prediction unit.

When MMVD is applied to both the first prediction unit and the second prediction unit, encoding/decoding of the second merge index gpm_merge_idx1 for the second prediction unit may be omitted. As an example, in Table 4, when MMVD is applied to both the first prediction unit and the second prediction unit, encoding/decoding of the second merge index gpm_merge_idx1 is omitted. In this case, the value of the second merge index gpm_merge_idx1 may be inferred to have a value derived based on the value of the first merge index explicitly signaled through the bitstream. As an example, the value of the second merge index can be inferred according to Equation 2 below.

Alternatively, when MMVD is applied to only one of the first prediction unit and the second prediction unit, omit encoding/decoding of the merge index for the prediction unit to which MMVD is applied, and merge index only for prediction units to which MMVD is not applied. You can also encode/decode.

As an example, if MMVD is applied to the first prediction unit while MMVD is not applied to the second prediction unit, encoding/decoding of the first merge index merge_gpm_idx0 is omitted, while the second merge index merge_gpm_idx1 is transmitted through the bitstream. Can be signaled explicitly. In this case, a merge candidate whose index is a value obtained by adding or subtracting an offset from the default value or the second merge index can be used for the first prediction unit. Likewise, if MMVD is not applied to the first prediction unit, while MMVD is applied to the second prediction unit, the first merge index merge_gpm_idx0 is explicitly signaled through the bitstream, while the second merge index merge_gpm_idx1 /Decryption can be omitted. In this case, a merge candidate whose index is a default value or a value obtained by adding or subtracting an offset from the first merge index can be used for the second prediction unit. Here, the default value may be an integer such as 0 or 1, and the offset may be an integer such as 1 or 2.

For prediction units to which MMVD is applied, information for determining an offset vector may be additionally encoded/decoded. As an example, when MMVD is applied to the first prediction unit, the syntax gpm_mmvd_idx0 for determining the motion vector offset vector for the first prediction unit may be encoded/decoded. Likewise, when MMVD is applied to the second prediction unit, the syntax gpm_mmvd_idx1 for determining the motion vector offset vector for the second prediction unit may be encoded/decoded. Here, gpm_mmvd_idxX (X is 0 or 1) may indicate one of a plurality of offset vector candidates. Furthermore, each of the plurality of offset vector candidates may be different in at least one of direction or distance.

Alternatively, for a prediction unit to which MMVD is applied, information indicating one of a plurality of direction candidates, such as gpm_mmvd_direction_idxX, and information indicating one of a plurality of distance candidates, such as gpm_mmvd_distance_idxX, may each be encoded and signaled.

When MMVD is applied to both the first prediction unit and the second prediction unit, the first prediction unit and the second prediction unit may be set to use the same merge candidate. Table 5 is an example of a syntax structure when the first prediction unit and the second prediction unit use the same merge candidate.

merge_data(　x0,　y0,　cbWidth,　cbHeight,　chType　) { Descriptor ... gpm_mmvd_flag0 ae(v) gpm_mmvd_flag1 ae(v) if (gpm_mmvd_flag0 ==1 && gpm_mmmvd_flag1==1){ gpm_mmvd_idx0 ae(v) gpm_mmvd_idx1 ae(v) if(gpm_mmvd_idx0 != gpm_mmvd_idx1 ) { gpm_merge_idx0 ae(v) } else { gpm_merge_idx0 ae(v) gpm_merge_idx1 ae(v) } } else if (gpm_mmvd_flag0 ==1 && gpm_mmmvd_flag1==0){ gpm_mmvd_idx0 ae(v) gpm_merge_idx0 ae(v) } else { gpm_mmvd_idx1 ae(v) gpm_merge_idx1 ae(v) } } ... }

When MMVD is applied to both the first prediction unit and the second prediction unit, the merge index (i.e. merge_gpm_idx0) is signaled only for the first prediction unit, and the merge index (i.e. merge_gpm_idx1) is signaled for the second prediction unit. /May not be decrypted.

Furthermore, after determining the offset vector for the MMVD of each prediction unit, a merge candidate for each prediction unit may be selected. That is, as in the example of Table 5, after encoding/decoding the syntax gpm_mmvd_idxX indicating the offset vector of the prediction unit, the syntax gpm_merge_idxX indicating the merge candidate of the prediction unit can be encoded/decoded.

At this time, if the offset vector for the first prediction unit and the offset vector for the second prediction unit are the same, a merge candidate different from the merge candidate of the first prediction unit may be selected for the second prediction unit. For this, if the offset vector for the first prediction unit and the offset vector for the second prediction unit are the same (i.e., gpm_mmvd_idx0 and gpm_mmvd_idx1 are the same), then the first merge index gpm_merge_idx0 for the first prediction unit, as well as the second The second merge index gpm_merge_idx1 for the prediction unit may also be encoded and signaled.

Alternatively, if the offset vector for the first prediction unit and the offset vector for the second prediction unit are the same (i.e., gpm_mmvd_idx0 and gpm_mmvd_idx1 are the same), omit encoding/decoding of the second merge index gpm_merge_idx1 for the second prediction unit. And, according to Equation 2 described above, a merge candidate for the second prediction unit may be selected.

In the examples of Tables 4 and 5, after decoding the syntax gpm_mmvd_flagX indicating whether MMVD is applied to the prediction unit, the syntax gpm_merge_idxX indicating the merge candidate of the prediction unit is decoded. Unlike the example, after encoding/decoding the syntax gpm_merge_idxX indicating the merge candidate of the prediction unit, the syntax gpm_mmvd_flagX indicating whether MMVD is applied may be encoded/decoded.

At this time, information indicating whether MMVD is applied can be encoded and signaled only to prediction units that use a merge candidate whose index is smaller than the threshold. For example, if the index of the merge candidate selected for the first prediction unit is smaller than the threshold, the flag gpm_mmvd_flag0 indicating whether to apply MMVD to the first prediction unit may be encoded and signaled. On the other hand, if the index of the merge candidate selected for the first prediction unit is greater than or equal to the threshold, encoding/decoding the flag gpm_mmvd_flag0 for the first prediction unit may be omitted and MMVD may not be applied. Likewise, if the index of the merge candidate selected for the second prediction unit is smaller than the threshold, the flag gpm_mmvd_flag1 indicating whether to apply MMVD to the second prediction unit may be encoded and signaled. On the other hand, if the index of the merge candidate selected for the second prediction unit is greater than or equal to the threshold, encoding/decoding the flag gpm_mmvd_flag1 for the second prediction unit may be omitted and MMVD may not be applied.

Furthermore, unlike the example in Table 5, after determining the offset vector for the MMVD of each prediction unit, a merge candidate for each prediction unit may be selected. That is, after encoding/decoding the syntax gpm_mmvd_idxX indicating the offset vector of the prediction unit, the syntax gpm_merge_idxX indicating the merge candidate of the prediction unit can be encoded/decoded.

At this time, if the offset vector for the first prediction unit and the offset vector for the second prediction unit are the same, a merge candidate different from the merge candidate of the first prediction unit may be selected for the second prediction unit.

When GPM is applied, it is also possible to apply different prediction modes to the first prediction unit and the second prediction unit. Here, the prediction mode may refer to at least one of intra prediction, inter prediction, and intra block copy. For convenience of explanation, in the example described later, it is assumed that the prediction mode applied to each of the first prediction unit and the second prediction unit is one of intra prediction and inter prediction.

Figure 10 shows an example in which different prediction modes are applied to each of the first prediction unit and the second prediction unit.

As in the example shown in Figures 10 (a) and (b), intra prediction may be applied to the first prediction unit, while inter prediction may be applied to the second prediction unit. At this time, inter prediction is based on merge mode, as in the example shown in (a) of FIG. 10, or based on applying MMVD to merge mode, as in the example shown in (b) of FIG. 10. It may be done by.

A first prediction block may be obtained based on intra prediction information of the first prediction unit, and a second prediction block may be obtained based on inter prediction information of the second prediction unit. Afterwards, the final prediction block can be obtained based on a weighted sum operation of the first prediction block and the second prediction block.

As above, applying different prediction modes to each of the first and second prediction units within a block to which GPM is applied can be defined as GPM-CIIP (Combined Intra-Inter Prediction) mode.

Information indicating whether the GPM-CIIP mode is applied may be signaled through a bitstream. For example, when it is determined that GPM is applied to a coding block, a flag indicating whether CIIP is applied to the coding block may be additionally encoded and signaled.

Depending on the partition type of the coding block, the availability of GPM-CIIP mode may be determined. As an example, the GPM-CIIP mode is used only when the first prediction unit and the second prediction unit generated through division of the coding block have the same size, or when the dividing line dividing the coding block is a horizontal line, vertical line, or diagonal line. It can be determined that it is possible.

If it is determined that the GPM-CIIP mode is not available, the encoding/decoding of the above-described flag may be omitted. That is, the above-described flag can be encoded/decoded only when it is determined that the GPM-CIIP mode is available.

When the GPM-CIIP mode is applied, information for determining the prediction mode of each of the first prediction unit and the second prediction unit may be signaled.

As an example, a flag indicating whether intra prediction or inter prediction is applied to the first prediction unit may be encoded and signaled. If the flag indicates that intra prediction is applied to the first prediction unit, inter prediction may be applied to the second prediction unit. On the other hand, if the flag indicates that inter prediction is applied to the first prediction unit, intra prediction may be applied to the second prediction unit.

As another example, instead of encoding/decoding a flag indicating whether CIIP is applied, index information specifying one of the combinations of prediction modes for the first prediction unit and the second prediction unit may be encoded and signaled. For example, a combination of inter prediction and inter prediction, a combination of intra prediction and inter prediction, and a combination of inter prediction and intra prediction are each set as candidate combinations, and then index information indicating one of the candidate combinations is encoded. Signaling is possible.

The motion information of the prediction unit to which inter prediction is applied among the first prediction unit and the second prediction unit may be derived from the merge candidate. To this end, a merge index that specifies one of the merge candidates included in the merge candidate list may be encoded and signaled for the prediction unit to which inter prediction is applied.

Furthermore, MMVD may be applied to a prediction unit to which inter prediction is applied. In this case, a flag indicating whether MMVD is applied to the prediction unit to which inter prediction is applied may be encoded and signaled.

At this time, it may be determined whether MMVD is available by considering whether the index of the merge candidate selected for the prediction unit is lower than the threshold. That is, the flag can be signaled only when the index of the merge candidate is smaller than the threshold.

Alternatively, depending on whether MMVD is applied, the number of selectable merge candidates may be determined differently through the merge index. For example, when MMVD is not applied, one of N merge candidates included in the merge candidate list may be selected for the prediction unit to which inter prediction is applied. On the other hand, when MMVD is applied, one of the two merge candidates with a small index among the N merge candidates included in the merge candidate list may be selected for the prediction unit to which inter prediction is applied.

As another example, when the GPM-CIIP mode is applied, MMVD may be applied by default to the prediction unit to which inter prediction is applied. For prediction units to which inter prediction is applied, information for determining an offset vector may be permanently encoded and signaled.

The intra prediction mode of the prediction unit to which intra prediction is applied may be set to a predefined intra prediction mode. As an example, the predefined intra prediction mode may be Planar mode.

Alternatively, the predefined intra prediction mode may be an intra prediction mode parallel to the dividing line dividing the coding block or an intra prediction mode perpendicular to the dividing line.

As another example, after setting a plurality of predefined intra prediction modes as candidate modes, the intra prediction mode of the prediction unit may be determined based on an index indicating one of the plurality of candidate modes. As an example, the syntax gpm_intra_candidates_index indicating one of a plurality of candidate modes may be encoded and signaled through a bitstream. The decoder can set the candidate mode indicated by the syntax gpm_intra_candidates_index as the intra prediction mode of the prediction unit.

As described above, when GPM-based prediction is applied, such as GPM or GPM-CIIP, the first prediction block is obtained based on the prediction information of the first prediction unit and the first prediction block is obtained based on the prediction information of the second prediction unit. By weighting the second prediction blocks, the final prediction block for the coding block can be derived. Meanwhile, in an embodiment described later, the weight assigned to the prediction sample in the first prediction block will be referred to as a first weight, and the weight assigned to the prediction sample in the second prediction block will be referred to as the second weight. Additionally, it is assumed that the first prediction sample is one of the prediction samples included in the first prediction block, and the second prediction sample is one of the prediction samples included in the second prediction block.

The weight applied to the prediction samples included in the first prediction block and the weight applied to the prediction samples included in the second prediction block may be adaptively determined depending on the location. Specifically, based on the boundary between the first prediction unit and the second prediction unit, if the location where the weighted sum operation is performed is distant from the second prediction unit, the weight assigned to the first prediction sample increases, The weight assigned to the second prediction sample may be reduced. On the other hand, when the location where the weighted sum calculation is performed is far from the first prediction unit, the weight assigned to the first prediction sample may decrease, while the weight assigned to the second prediction sample may increase.

11 shows that the weight applied to the first prediction sample and the weight applied to the second prediction sample are adaptively determined depending on the location where the weighted sum operation is performed, based on the boundary between the first prediction unit and the second prediction unit. This is a drawing showing an example.

FIG. 11 shows an example in which the value of the weight W0 applied to the first prediction sample is adaptively determined according to the location of the first prediction sample. Specifically, the first weight W0 for samples included in an area within τ in the vertical direction from the boundary of the first prediction sample and the second prediction sample (GPM partitioning boundary in FIG. 1) is adapted according to the location. can be decided negatively.

Meanwhile, the sum of the first weight W0 applied to the first prediction sample and the second weight W1 applied to the second prediction sample may be a constant. For example, the sum of the first weight W0 and the second weight W1 may be a natural number expressed as a power of 2, such as 8, 16, or 32. In FIG. 11, it is illustrated that the sum of the first weight W0 and the second weight W1 is 8. Accordingly, the graph for the second weight W1 applied to the second prediction sample may be symmetrical with the graph for the first weight W0 based on the boundary between the first prediction unit and the second prediction unit.

As in the illustrated example, outside the blending region, one of the first weight W0 applied to the first prediction sample and the second weight W1 applied to the second prediction sample is set to 0. Accordingly, it can be understood that blending between the first prediction block and the second prediction block is performed only in the region between -τ and τ shown in FIG. 11. In this way, the first weight W0 applied to the first prediction sample and the second weight W1 applied to the second prediction sample are adaptively determined according to the sample position, so that blending between the first prediction block and the second prediction block is performed. The area where this is performed can be defined as a blending area. That is, in the example shown in FIG. 11, an area where the distance in the vertical direction of the boundary between the first prediction unit and the second prediction unit is between -τ and τ may be defined as a blending area.

The distance from the area outside the blending area, i.e. the boundary between the first prediction unit and the second prediction unit, is The first weight W0 and the second weight W1 for samples included in a larger area may have fixed values. For example, in the area included in the second prediction unit among the areas outside the blending area, the first weight W0 assigned to the first prediction sample may be set to 0. On the other hand, in the area included in the first prediction unit among the areas outside the blending area, the first weight W0 assigned to the first prediction sample may be set to 8.

In summary, the final prediction sample P _Diag (x, y) in the blending region can be obtained based on a weighted sum operation that applies the first weight W0 and the second weight W1 to the first prediction sample and the second prediction sample, respectively. there is. Specifically, when the sum of the first weight W0 and the second weight W1 is a natural number expressed as 2^N, the result of the weighted sum operation of the first prediction sample and the second prediction sample is shifted to the right by N, and the final Predicted samples P _Diag (x, y) can be obtained.

On the other hand, the final prediction sample P _Diag (x, y), which is included in the first prediction unit and is located outside the blending area, may be set to be the same as the first prediction sample at that location. Additionally, the final prediction sample P _Diag (x, y), which is included in the second prediction unit and is located outside the blending area, may be set to be the same as the second prediction sample at that location.

Equation 3 below shows an example in which the final prediction sample P _Diag (x,y) within a coding block is derived.

In Equation 3, P ₁ and P ₂ represent the first prediction sample and the second prediction sample, respectively. As in the example of Equation 3, the second weight w1 can be derived by differentiating the first weight w0 from the constant N. N may be a natural number expressed as a power of 2, such as 8, 16, or 32. N may be predefined in the encoder and decoder. Alternatively, N may be adaptively determined depending on the size of the blending area.

The size of the blending area for a block of arbitrary size may be fixed in the encoder and decoder. As another example, from a plurality of size candidates, a candidate corresponding to the size of the blending area for the coding block may be selected.

Figure 12 is a diagram showing a plurality of blending area size candidates.

In the example shown in Figure 12, (-τ/4, τ/4), (-τ/2, τ/2), (-τ, τ), (-2τ, 2τ) and (-4τ, 4τ) A total of 5 blending area size candidates are shown as shown.

One of the five blending area size candidates shown may be applied to the coding block. As an example, the blending area of a coding block may be determined by selecting one of a plurality of blending area size candidates.

Meanwhile, the first weight W0 and the second weight W1 within the blending area may change linearly. At this time, the slope or absolute value of the slope of the graph with respect to the first weight W0 may increase as the size of the blending area becomes smaller. That is, the smaller the size of the blending area, the greater the degree of change in the first weight W0 or the second weight W1 as the distance from the boundary between the first prediction unit and the second prediction unit increases.

The size of the blending area can be determined on a coding block basis. To this end, information for determining the size of the blending area can be explicitly encoded/decoded in units of coding blocks. As an example, the information may be index information, blending_area_idx, indicating one of a plurality of blending area size candidates. The encoder encodes the syntax blending_area_idx, which indicates the index of the blending size candidate applied to the coding block, among a plurality of blending size candidates, and the decoder encodes the blending area of the coding block based on the value of the syntax blending_area_idx. Table 6 below illustrates the mapping relationship between syntax blending_area_idx values and blending area size candidates.

blending_area_idx Blending area size candidates 0 ( One 2 3 4

Fewer or more blending region size candidates than those illustrated in Table 6 may be predefined in the encoder and decoder. When a blending area size candidate is selected, the first weight W0 and the second weight W1 may be determined according to the selected blending area size candidate.

Meanwhile, whether the size of the coding block, the division type of the coding block, the size of the prediction unit within the coding block, the shape of the prediction unit, and the prediction direction (e.g., L0 direction or L1 direction) of the first prediction unit and the second prediction unit are the same. Based on at least one of motion information of the first prediction unit and the second prediction unit, at least one of the number or type of blending area size candidates available for the coding block may be adaptively determined. Here, the motion information may be related to at least one of a motion vector, prediction direction, reference picture index, or POC of the reference picture.

For example, if at least one of the width or height of the first prediction unit or the second prediction unit is smaller than the first threshold, candidates with a size larger than the second threshold among the blending area candidates are set as unavailable for the coding block. It can be. As an example, when the width of the first prediction unit is smaller than 16 samples, (-τ/4, τ/4), (-τ/2, τ/2) and (-τ, τ) among the blending region candidates are While available for the coding block, (-2τ, 2τ) and (-4τ, 4τ) may be set as unavailable for the coding block.

or, depending on whether the difference between the motion vector derived from the merge candidate of the first prediction unit and the motion vector derived from the merge candidate of the second prediction unit is less than or equal to a first threshold, the number of available blending region size candidates and/ Alternatively, the type may be determined. The difference between the motion vector of the first prediction unit and the motion vector of the second prediction unit can be derived as in Equation 4 below.

In Equation 4, MVgpm1_x and MVgpm1_y represent the horizontal and vertical components of the motion vector of the first prediction unit, respectively. MVgpm2_x and MVgpm2_y represent the horizontal component and vertical component of the motion vector of the second prediction unit, respectively.

As an example, if the difference is less than or equal to the first threshold, only (-τ, τ) among the blending region size candidates may be available for the coding block. On the other hand, when the difference is greater than the first threshold, a plurality of blending area candidates may be available for the coding block.

The syntax blending_area_idx may indicate one of the available blending area size candidates for a coding block. The maximum bit length required to encode the syntax blending_area_idx can be adjusted according to the number of blending area size candidates available for the coding block. Meanwhile, when the number of blending area size candidates available for a coding block is 1, encoding/decoding of the syntax blending_area_idx can be omitted.

Omitting the encoding/decoding of index information, at least one of the size of the coding block, the type of the coding block, the size of the partition within the coding block, the partition type of the coding block, the size ratio of the prediction units within the coding block, or the motion information of each prediction unit Based on one, the blending area size for the coding block may be adaptively determined. As an example, the size of the blending area for the coding block may be determined based on the POC of the reference picture for the first prediction unit and the POC of the reference picture for the second prediction unit.

Alternatively, based on at least one of the size of the coding block, the type of the coding block, the size of the partition within the coding block, the partition type of the coding block, the size ratio of prediction units within the coding block, or motion information of each prediction unit, index information You can also decide whether to encode/decrypt. If it is determined not to encode/decode index information, a default blending area size candidate may be applied to the coding block. Here, the default blending area size candidate may be (-τ, τ).

Meanwhile, in FIG. 12, each of the blending area size candidates is illustrated as being symmetrical based on the boundary between the first prediction unit and the second prediction unit. Unlike the example shown, the blending area may be set in an asymmetric form. For example, based on the boundary between the first prediction unit and the second prediction unit, the size of the blending area in the direction of the first prediction unit is τ, while the size of the blending area in the direction of the second prediction unit is a value smaller than τ. (e.g., τ/2, τ/4, or τ/8).

Meanwhile, based on the boundary between the first prediction unit and the second prediction unit, the size of the blending area in the direction of the first prediction unit and the size of the blending area in the direction of the second prediction unit are different, and will be called an asymmetric blending area. You can.

Whether the blending area of the coding block is set asymmetrically depends on the size of the coding block, the division type of the coding block, the size of the prediction unit within the coding block, the shape of the prediction unit, and the prediction direction of the first prediction unit and the second prediction unit ( For example, the L0 direction or the L1 direction) may be determined based on at least one of the same or motion information of the first prediction unit and the second prediction unit.

As an example, if at least one of the width or height of the first prediction unit and the second prediction unit is smaller than the threshold, at least one asymmetric blending area size candidate may be set as available for the coding block. As an example, the size of the blending region for a coding block is, such as (-τ, τ/2), (-2τ, τ), (-τ, τ/4) or (-τ/2, τ/4). It can be set to one of the asymmetric blending area size candidates.

Alternatively, the availability of the asymmetric blending area size candidate may be determined based on whether the difference between the motion vector of the first prediction unit and the motion vector of the second prediction unit is less than or equal to a threshold. As an example, if the difference is less than a threshold, the asymmetric blending region size candidate may be determined to be unavailable for the coding block. On the other hand, if the difference is greater than the threshold, it may be determined that at least one asymmetric blending region size candidate is available for the coding block.

Meanwhile, the difference between the motion vector of the first prediction unit and the motion vector of the second prediction unit may be derived based on Equation 4 described above.

Alternatively, information indicating whether the blending area of the coding block is set asymmetrically may be encoded and signaled through a bitstream.

Meanwhile, the size of the asymmetric blending area may be determined by considering the prediction modes of the first prediction unit and the second prediction unit. As an example, as GPM-CIIP is applied, when intra prediction is applied to the first prediction unit and inter prediction is applied to the second prediction unit, the size of the blending area for the first prediction unit in which intra prediction is used is Inter prediction may be set to be smaller than the size of the blending area for the second prediction unit used. Or, conversely, the size of the blending area for the first prediction unit using intra prediction may be set to be larger than the size of the blending area for the second prediction unit using inter prediction.

Below, intra prediction will be described in more detail.

Intra prediction is predicting the current block using restored samples that have been encoded/decoded around the current block. At this time, the restored sample before the in-loop filter is applied can be used for intra prediction of the current block.

Intra prediction techniques include matrix-based intra prediction and general intra prediction that takes into account the directionality with surrounding reconstructed samples. Information indicating the intra prediction technique of the current block may be signaled through a bitstream. The information may be a 1-bit flag. Alternatively, the intra prediction technique of the current block may be determined based on at least one of the position, size, and shape of the current block or the intra prediction technique of a neighboring block. For example, if the current block spans a picture boundary, intra prediction based on the metric may be set not to be applied to the current block.

Matrix-based intra prediction is a method of obtaining a prediction block of the current block based on matrix multiplication between a matrix pre-stored in the encoder and decoder and restored samples around the current block. Information for specifying one of a plurality of pre-stored matrices may be signaled through a bitstream. The decoder may determine a matrix for intra prediction of the current block based on the information and the size of the current block.

General intra prediction is a method of obtaining a prediction block for the current block based on a non-directional intra prediction mode or a directional intra prediction mode. Hereinafter, with reference to the drawings, we will look at the intra prediction performance process based on general intra prediction in more detail.

13 is a flowchart of an intra prediction method according to an embodiment of the present disclosure.

The reference sample line of the current block can be determined (S1301). The reference sample line refers to a set of reference samples included in the kth line from the top and/or left of the current block. The reference sample may be derived from a restored sample for which encoding/decoding around the current block has been completed.

Index information identifying a reference sample line of the current block among a plurality of reference sample lines may be signaled through a bitstream. As an example, index information intra_luma_ref_idx for specifying the reference sample line of the current block may be signaled through a bitstream. The index information may be signaled on a coding block basis.

The plurality of reference sample lines may include at least one of the first line, second line, and third line from the top and/or left of the current block. Among the plurality of reference sample lines, a reference sample line consisting of a row adjacent to the top of the current block and a column adjacent to the left of the current block is called an adjacent reference sample line, and other reference sample lines are called non-adjacent reference sample lines. You can also call it.

Table 7 shows the indices assigned to each of the candidate reference sample lines.

index (intra_luma_ref_idx) Reference sample line 0 Adjacent reference sample line One First non-adjacent reference sample line 2 Second non-adjacent reference sample line

The reference sample line of the current block may be determined based on at least one of the position, size, and shape of the current block or the prediction coding mode of the neighboring block. For example, when the current block borders the boundary of a picture, tile, slice, or coding tree unit, an adjacent reference sample line may be determined as the reference sample line of the current block.

The reference sample line may include top reference samples located at the top of the current block and left reference samples located to the left of the current block. Top reference samples and left reference samples may be derived from reconstructed samples around the current block. The restored samples may be in a state before the in-loop filter is applied.

Next, the intra prediction mode of the current block can be determined (S1302). At least one of a non-directional intra prediction mode or a directional intra prediction mode may be determined as the intra prediction mode of the current block. The non-directional intra prediction mode includes planar and DC, and the directional intra prediction mode includes 33 or 65 modes from the lower left diagonal direction to the upper right diagonal direction.

Figure 14 is a diagram showing intra prediction modes.

Figure 14 (a) shows 35 intra prediction modes, and Figure 15 (b) shows 67 intra prediction modes.

More or fewer intra prediction modes than shown in FIG. 14 may be defined.

The intra prediction mode of the current block can be determined or predicted using information on neighboring blocks neighboring the current block. As an example, the intra prediction mode of the current block may be derived based on the edge direction and/or angle of the neighboring block. For example, when an edge exists in a neighboring block, an intra prediction mode with an angle that is the same as or most similar to the direction of the edge and/or the angle formed by the edge may be set as the prediction mode of the current block.

The neighboring block may include at least one of a left neighboring block or a top neighboring block of the current block. Accordingly, the intra prediction mode that is identical or most similar to the edge direction and/or edge angle of the left neighboring block or the intra prediction mode that is identical or most similar to the edge direction and/or edge angle of the top neighboring block is set as the prediction mode of the current block. You can.

As another example, the intra prediction mode of the neighboring block may be set to the intra prediction mode of the current block.

Figure 15 shows the positions of neighboring blocks used to derive the intra prediction mode of the current block.

As in the example shown in Figure 15, at least one of the left neighboring block (1), the top neighboring block (2), the upper right neighboring block (3), the lower left neighboring block (4), or the upper left neighboring block (5) is used. Thus, the intra prediction mode of the current block can be derived.

Setting the intra prediction mode of a neighboring block to the intra prediction mode of the current block can be defined as intra merge mode. In this case, the intra prediction mode of each neighboring block can be set as an intra merge candidate. When a plurality of intra merge candidates exist, index information identifying one of the plurality of intra merge candidates may be signaled through a bitstream. Alternatively, the intra prediction mode with the highest frequency among the plurality of intra merge candidates may be set as the intra prediction mode of the current block.

After selecting a plurality of intra merge candidates, each of the selected intra prediction modes can be assigned to a subblock. Accordingly, a different intra prediction mode may be assigned to each subblock within the current block. As an example, the intra prediction mode of the first subblock in the current block may be set to be the same as the first intra merge candidate, and the intra prediction mode of the second subblock in the current block may be set to be the same as the second intra merge candidate. You can.

When the intra prediction mode of the current block is determined, prediction samples for the current block can be obtained based on the determined intra prediction mode (S1303).

When DC mode is selected, prediction samples for the current block are generated based on the average value of reference samples. Specifically, the values of all samples within the prediction block may be generated based on the average value of the reference samples. The average value may be derived using at least one of top reference samples located at the top of the current block and left reference samples located to the left of the current block.

Depending on the type of the current block, the number or range of reference samples used to derive the average value may vary. For example, if the current block is a non-square block with a width greater than the height, the average value can be calculated using only the top reference samples. On the other hand, if the current block is a non-square block whose width is smaller than the height, the average value can be calculated using only the left reference samples. That is, if the width and height of the current block are different, the average value can be calculated using only reference samples adjacent to the longer side. Alternatively, based on the width and height ratio of the current block, it may be determined whether to calculate the average value using only the top reference samples or only the left reference samples.

When the planner mode is selected, prediction samples can be obtained using horizontal prediction samples and vertical prediction samples. Here, the horizontal prediction sample is obtained based on the left reference sample and the right reference sample located on the same horizontal line as the prediction sample, and the vertical prediction sample is the upper reference sample and the lower reference sample located on the same vertical line as the prediction sample. Obtained on the basis of a reference sample. Here, the right reference sample may be created by copying a reference sample adjacent to the upper right corner of the current block, and the lower reference sample may be created by copying a reference sample adjacent to the lower left corner of the current block. The horizontal prediction sample may be obtained based on a weighted sum operation of the left reference sample and the right reference sample, and the vertical prediction sample may be obtained based on the weighted sum operation of the upper reference sample and the lower reference sample. At this time, the weight given to each reference sample may be determined according to the location of the prediction sample. The prediction sample may be obtained based on an average operation or a weighted sum operation of the horizontal prediction sample and the vertical prediction sample. When a weighted sum operation is performed, weights given to the horizontal prediction sample and the vertical prediction sample may be determined based on the positions of the prediction samples.

When a directional prediction mode is selected, a parameter indicating the prediction direction (or prediction angle) of the selected directional prediction mode may be determined. Table 8 below shows the intra direction parameter intraPredAng for each intra prediction mode.

PredModeIntra One 2 3 4 5 6 7 IntraPredAng - 32 26 21 17 13 9 PredModeIntra 8 9 10 11 12 13 14 IntraPredAng 5 2 0 -2 -5 -9 -13 PredModeIntra 15 16 17 18 19 20 21 IntraPredAng -17 -21 -26 -32 -26 -21 -17 PredModeIntra 22 23 24 25 26 27 28 IntraPredAng -13 -9 -5 -2 0 2 5 PredModeIntra 29 30 31 32 33 34 IntraPredAng 9 13 17 21 26 32

Table 8 shows the intra direction parameters for each of the intra prediction modes with an index of any one of 2 to 34 when 35 intra prediction modes are defined. If more than 33 directional intra prediction modes are defined, Table 7 can be further subdivided to set the intra direction parameters for each directional intra prediction mode. The top reference samples and left reference samples of the current block are lined up. After arrangement, prediction samples can be obtained based on the value of the intra direction parameter. At this time, if the value of the intra direction parameter is a negative number, the left reference samples and the top reference samples can be arranged in a row.

Based on the intra direction parameters, reference sample determination parameters can be determined. The reference sample determination parameter may include a reference sample index for specifying the reference sample and a weight parameter for determining the weight applied to the reference sample.

The reference sample index iIdx and the weight parameter ifact can be obtained through the following equations 5 and 6, respectively.

In Equations 5 and 6, P _ang represents the intra direction parameter. The reference sample specified by the reference sample index iIdx corresponds to an integer pel.

To derive a prediction sample, at least one reference sample may be specified. Specifically, by considering the slope of the prediction mode, the location of the reference sample used to derive the prediction sample can be specified. As an example, the reference sample index iIdx can be used to specify the reference sample used to derive the prediction sample.

At this time, if the slope of the intra prediction mode is not expressed by one reference sample, a prediction sample can be generated by interpolating a plurality of reference samples. As an example, if the slope of the intra prediction mode is a value between the slope between the prediction sample and the first reference sample and the slope between the prediction sample and the second reference sample, the prediction sample is obtained by interpolating the first reference sample and the second reference sample. It can be obtained. That is, if the angular line following the intra prediction angle does not pass through a reference sample located at an integer pel, the prediction sample can be obtained by interpolating reference samples located adjacent to the left and right or above and below the location where the angular line passes. there is.

Equation 7 below shows an example of obtaining a prediction sample based on reference samples.

In Equation 7, P represents a prediction sample, and Ref_1D represents one of one-dimensionally arranged reference samples. At this time, the location of the reference sample can be determined by the location (x, y) of the prediction sample and the reference sample index iIdx.

If the gradient of the intra prediction mode can be expressed with one reference sample, the weight parameter i _fact is set to 0. Accordingly, Equation 7 can be simplified as Equation 8 below.

The derived residual image can be derived by subtracting the predicted image from the original image. At this time, when the residual image is changed to the frequency domain, even if high-frequency components among the frequency components are removed, the subjective image quality of the image does not decrease significantly. Accordingly, if the values of the high-frequency components are converted to small or the values of the high-frequency components are set to 0, compression efficiency can be increased without causing significant visual distortion. Reflecting the above characteristics, the current block can be transformed to decompose the residual image into two-dimensional frequency components. The transformation may be performed using a transformation technique such as Discrete Cosine Transform (DCT) or Discrete Sine Transform (DST).

DCT decomposes (or converts) the residual image into two-dimensional frequency components using cosine transform, and DST decomposes (or converts) the residual image into two-dimensional frequency components using sine transform. As a result of transforming the residual image, the frequency components can be expressed as a basis image. For example, when performing DCT transformation on a block of size NxN, N ² basic pattern components can be obtained. Through conversion, the size of each basic pattern component included in the NxN size block can be obtained. Depending on the conversion technique used, the size of the basic pattern component may be called a DCT coefficient or DST coefficient.

Transformation technique DCT is mainly used to transform images in which many non-zero low-frequency components are distributed. The transformation technique DST is mainly used for images with many high-frequency components distributed.

The residual image can also be transformed using a transformation technique other than DCT or DST.

Hereinafter, converting the residual image into two-dimensional frequency components will be referred to as two-dimensional image conversion. In addition, the sizes of basic pattern components obtained as a result of conversion are referred to as conversion coefficients. As an example, the transformation coefficient may mean a DCT coefficient or a DST coefficient. When both the first transformation and the second transformation, which will be described later, are applied, the transformation coefficient may mean the size of the basic pattern component generated as a result of the second transformation. Additionally, the residual sample to which transformation skip is applied will also be referred to as a transformation coefficient.

The conversion technique can be determined on a block basis. The transformation technique may be determined based on at least one of the prediction coding mode of the current block, the size of the current block, or the shape of the current block. For example, if the current block is encoded in intra prediction mode and the size of the current block is smaller than NxN, transformation may be performed using the transformation technique DST. On the other hand, if the above conditions are not satisfied, conversion may be performed using the conversion technique DCT.

2D image conversion may not be performed on some blocks of the residual image. Not performing 2D image transformation can be called transform skip. Transform skip indicates that the first transform and the second transform are not applied to the current block. When transform skip is applied, quantization may be applied to residual values for which transform is not performed.

Whether to allow transformation skipping for the current block may be determined based on at least one of the size or shape of the current block. For example, transform skip may be applied only when the size of the current block is smaller than the threshold. The threshold is related to at least one of the width, height, or number of samples of the current block and may be defined as 32x32, etc. Alternatively, transformation skipping can be allowed only for square blocks. As an example, skipping transformation may be allowed for square blocks of size 32x32, 16x16, 8x8, or 4x4. Alternatively, conversion skipping may be allowed only when the sub-partition intra coding method is not used.

After converting the current block using DCT or DST, the converted current block can be converted again. At this time, transformation based on DCT or DST can be defined as the first transformation, and re-transforming the block to which the first transformation has been applied can be defined as the second transformation.

The first transformation may be performed using one of a plurality of transformation core candidates. As an example, the first transformation may be performed using any one of DCT2, DCT8, or DST7.

Different conversion cores may be used for the horizontal and vertical directions. Information indicating the combination of the horizontal conversion core and the vertical conversion core may be signaled through the bitstream. As an example, tu_mts_idx may indicate one of a combination of a horizontal direction conversion core and a vertical direction conversion core.

The performance units of the first transformation and the second transformation may be different. As an example, a first transformation may be performed on an 8x8 block, and a second transformation may be performed on a 4x4 sub-block among the transformed 8x8 blocks. Alternatively, a second transform may be performed on transform coefficients belonging to three sub-blocks of 4x4 size. The three sub-blocks may include a sub-block located at the upper left of the current block, a sub-block neighboring to the right of the sub-block, and a sub-block neighboring to the bottom of the sub-block. Alternatively, the second transformation may be performed on an 8x8 block.

Transform coefficients of remaining areas where the second transform is not performed may be set to 0.

Alternatively, the first transformation may be performed on a 4x4 block, and the second transformation may be performed on an 8x8 area containing the transformed 4x4 block.

Information indicating the conversion type of the current block may be signaled through a bitstream. The information may be index information tu_mts_idx indicating one of combinations of a transformation type for the horizontal direction and a transformation type for the vertical direction.

It may be determined whether to explicitly determine the transformation type of the current block based on information signaled from the bitstream. The information may be signaled through a sequence, picture, or slice header. As an example, at the sequence level, information sps_explicit_intra_mts_flag indicating whether explicit transform type determination is allowed for blocks encoded with intra prediction and/or information indicating whether explicit transform type determination is allowed for blocks encoded with inter prediction. Information sps_explicit_inter_mts_flag may be signaled.

If explicit transformation type determination is allowed, the transformation type of the current block can be determined based on index information tu_mts_idx signaled from the bitstream.

On the other hand, if explicit transformation type determination is not allowed, the transformation type is determined based on at least one of the size of the current block, the shape, whether transformation in subblock units is allowed, or the location of the subblock containing a non-zero transformation coefficient. This can be decided. For example, the horizontal transformation type of the current block may be determined based on the width of the current block, and the vertical transformation type of the current block may be determined based on the height of the current block.

For example, if the width of the current block is less than 4 or greater than 16, the horizontal transformation type may be determined as DCT2. Otherwise, the transformation type in the horizontal direction may be determined as DST7.

If the height of the current block is less than 4 or greater than 16, the vertical transformation type may be determined as DCT2. Otherwise, the vertical transformation type may be determined as DST7.

Here, in order to determine the horizontal transformation type and the vertical transformation type, the threshold value compared with the width and height is predefined in the encoder and decoder, or is determined by the size, shape, or intra prediction mode of the current block. It may be adaptively determined based on at least one.

If the encoder performs transformation and quantization, the decoder can obtain a residual block through inverse quantization and inverse transformation. The decoder can obtain a restored block for the current block by adding the prediction block and the residual block.

When the restored block of the current block is obtained, information loss occurring during quantization and encoding can be reduced through in-loop filtering. The in-loop filter may include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF).

Although the above-described embodiments are explained based on a series of steps or flowcharts, this does not limit the chronological order of the invention, and may be performed simultaneously or in a different order as needed. Additionally, in the above-described embodiment, each of the components (e.g., units, modules, etc.) constituting the block diagram may be implemented as a hardware device or software, and a plurality of components may be combined to form a single hardware device or software. It may be implemented. The above-described embodiments may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., singly or in combination. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specifically configured to store and perform program instructions, such as ROM, RAM, flash memory, etc. The hardware devices may be configured to operate as one or more software modules to perform processing according to the present disclosure, and vice versa.

Claims (15)

Obtaining a first prediction block based on first prediction information for the first prediction unit in the current block;
Obtaining a second prediction block based on second prediction information for a second prediction unit in the current block; and
Comprising: obtaining a final prediction block for the current block based on the first prediction block and the second prediction block,
In a blending area formed around the boundary between the first prediction unit and the second prediction unit in the current block, first prediction samples in the first prediction block and second prediction samples in the second prediction block A video decoding method, characterized in that the final prediction sample is derived by blending the samples. According to claim 1,
The size of the blending area is determined based on one of a plurality of blending area size candidates. According to clause 2,
An image decoding method, wherein selecting one of the plurality of blending area size candidates is performed based on index information decoded from a bitstream. According to clause 2,
An image decoding method, wherein selecting one of the plurality of blending area size candidates is performed based on at least one of motion information of the first prediction unit and motion information of the second prediction unit. According to clause 2,
An image decoding method, wherein the first weight applied to the first prediction samples and the second weight applied to the second prediction samples in the blending region are adaptively determined depending on the location. According to clause 5,
A slope indicating the degree of change of the first weight within the blending area is inversely proportional to the size of the blending area. According to clause 2,
The plurality of blending region size candidates include asymmetric blending region size candidates whose sizes in the direction of the first prediction unit and in the direction of the second prediction unit are different with respect to the boundary. Decryption method. According to claim 1,
When prediction modes between the first prediction unit and the second prediction unit are different, the blending area has an asymmetric shape with respect to the boundary. According to clause 8,
When intra prediction is applied to the first prediction unit and inter prediction is applied to the second prediction unit, based on the boundary, the size of the blending area in the direction of the first prediction unit is equal to the size of the second prediction unit. An image decoding method, characterized in that it is smaller than the size of the blending area in the unit direction. According to clause 9,
The intra prediction mode of the first prediction unit is determined as one of a planner mode, an intra prediction mode parallel to a dividing line dividing the current block, or an intra prediction mode perpendicular to the dividing line. Image decoding method. Obtaining a first prediction block based on first prediction information for the first prediction unit in the current block;
Obtaining a second prediction block based on second prediction information for a second prediction unit in the current block; and
Comprising: obtaining a final prediction block for the current block based on the first prediction block and the second prediction block,
In a blending area formed around the boundary between the first prediction unit and the second prediction unit in the current block, first prediction samples in the first prediction block and second prediction samples in the second prediction block An image encoding method, characterized in that the final prediction sample is derived by blending the samples. According to claim 11,
An image encoding method, wherein the size of the blending area is determined based on one of a plurality of blending area size candidates. According to claim 12,
An image decoding method, wherein index information specifying one of the plurality of blending area size candidates is explicitly encoded in a bitstream. According to claim 12,
An image encoding method, wherein selecting one of the plurality of blending area size candidates is performed based on at least one of motion information of the first prediction unit and motion information of the second prediction unit. Obtaining a first prediction block based on first prediction information for the first prediction unit in the current block;
Obtaining a second prediction block based on second prediction information for a second prediction unit in the current block; and
Comprising: obtaining a final prediction block for the current block based on the first prediction block and the second prediction block,
In a blending area formed around the boundary between the first prediction unit and the second prediction unit in the current block, first prediction samples in the first prediction block and second prediction samples in the second prediction block A computer-readable recording medium for storing a bitstream generated by an image encoding method, characterized in that the final prediction sample is derived by blending the bitstream.

Images