KR20210123950A - Video signal encoding method and apparatus and video decoding method and apparatus - Google Patents

Abstract

A method and apparatus for encoding and decoding tile and slice information are provided. An image encoding apparatus includes a picture division unit, a prediction unit, a transform unit, a quantization unit, and a rearrangement unit, an entropy encoding unit, an inverse quantization unit, an inverse transform unit, a filter unit, and a memory. An object of the present invention is to improve the coding efficiency of a video signal.

Description

Video signal processing method and apparatus {Video signal encoding method and apparatus and video decoding method and apparatus}

The present invention relates to a video signal processing method and apparatus.

As the display panel becomes larger and larger, a video service with higher image quality is required. The biggest problem of the high-definition video service is that the amount of data is greatly increased, and in order to solve this problem, research to improve the video compression rate is being actively conducted. As a representative example, in 2009, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under ITU-T (International Telecommunication Union-Telecommunication) formed the Joint Collaborative Team on Video Coding (JCT-VC). JCT-VC proposed HEVC (High Efficiency Video Coding), which is a video compression standard having about twice the compression performance of H.264/AVC, and was approved as a standard on January 25, 2013. With the rapid development of high-definition video services, the performance of HEVC is gradually revealing its limits.

An object of the present invention is to improve the coding efficiency of a video signal.

In order to solve the above problems, the present invention provides a method for encoding/decoding tile and slice information and an apparatus for performing the method.

A video signal processing method and apparatus according to the present invention can improve video signal coding efficiency through a tile and slice information encoding/decoding method.

One Video encoding and decoding methods

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

Encoding and decoding of an image is performed in units of blocks. As an example, encoding/decoding processing such as transform, quantization, prediction, in-loop filtering, or reconstruction may be performed on the coding block, the transform block, or the prediction block.

Hereinafter, a block to be encoded/decoded will be referred to as a 'current block'. As an example, the current block may represent a coding block, a transform block, or a prediction block according to a current encoding/decoding process step.

In addition, it may be understood that the term 'unit' used in this specification indicates a basic unit for performing a specific encoding/decoding process, and 'block' indicates a sample array of a predetermined size. Unless otherwise specified, the terms 'block' and 'unit' may be used interchangeably. For example, in the embodiments to be described later, it may be understood that a coding block and a coding unit have the same meaning.

Figure 1

1 is a block diagram of an image encoder (encoder) according to an embodiment of the present invention.

Referring to FIG. 1 , the image encoding apparatus 100 includes a picture division unit 110 , prediction units 120 and 125 , a transform unit 130 , a quantization unit 135 , a rearrangement unit 160 , and an entropy encoding unit ( 165 ), an inverse quantization unit 140 , an inverse transform unit 145 , a filter unit 150 , and a memory 155 .

Each of the constituent units shown in FIG. 1 is independently illustrated to represent different characteristic functions in the image encoding apparatus, and does not mean that each constituent unit is composed of separate hardware or one software constituent unit. That is, each component is listed as each component for convenience of description, and at least two components of each component are combined to form one component, or one component can be divided into a plurality of components to perform a function, and each Integrated embodiments and separate embodiments of components are also included in the scope of the present invention without departing from the essence of the present invention.

In addition, some of the components are not essential components for performing essential functions in the present invention, but may be optional components for merely improving performance. The present invention can be implemented by including only essential components to implement the essence of the present invention, except for components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement Also included in the scope of the present invention.

The picture divider 110 may divide the input picture into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture splitter 110 divides one picture into a combination of a plurality of coding units, prediction units, and transformation units, and combines one coding unit, prediction unit, and transformation unit based on a predetermined criterion (eg, a cost function). can be selected to encode the picture.

For example, one picture may be divided into a plurality of coding units. In order to split a coding unit in a picture, a recursive tree structure such as a quad tree structure can be used. A coding in which one image or a largest coding unit is used as a root and is divided into other coding units. A unit may be divided having as many child nodes as the number of divided coding units. A coding unit that is no longer split according to certain restrictions becomes a leaf node. That is, when it is assumed that only square splitting is possible for one coding unit, one coding unit may be split into up to four different coding units.

Hereinafter, in an embodiment of the present invention, a coding unit may be used as a unit for performing encoding or may be used as a meaning for a unit for performing decoding.

A prediction unit may be split in the form of at least one square or rectangle of the same size within one coding unit, and one prediction unit among the split prediction units within one coding unit is a prediction of another. It may be divided to have a shape and/or size different from that of the unit.

When a prediction unit for performing intra prediction based on a coding unit is generated, if it is not the smallest coding unit, intra prediction may be performed without dividing the prediction unit into a plurality of prediction units NxN.

The prediction units 120 and 125 may include an inter prediction unit 120 performing inter prediction and an intra prediction unit 125 performing intra prediction. Whether to use inter prediction or to perform intra prediction for a prediction unit may be determined, and specific information (eg, intra prediction mode, motion vector, reference picture, etc.) according to each prediction method may be determined. In this case, a processing unit in which prediction is performed and a processing unit in which a prediction method and specific content are determined may be different. For example, a prediction method and a prediction mode may be determined in a prediction unit, and prediction may be performed in a transformation unit. A residual value (residual block) between the generated prediction block and the original block may be input to the transform unit 130 . In addition, prediction mode information, motion vector information, etc. used for prediction may be encoded in the entropy encoder 165 together with a residual value and transmitted to a decoder. When a specific encoding mode is used, it is also possible to encode the original block as it is without generating the prediction block through the prediction units 120 and 125 and transmit it to the decoder.

The inter prediction unit 120 may predict a prediction unit based on information on at least one of a picture before or after a picture of the current picture, and in some cases, prediction based on information of a partial region in the current picture that has been encoded Units can also be predicted. The inter prediction unit 120 may include a reference picture interpolator, a motion prediction unit, and a motion compensator.

The reference picture interpolator may receive reference picture information from the memory 155 and generate pixel information of integer pixels or less in the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter (DCT-based Interpolation Filter) with different filter coefficients may be used to generate pixel information of integer pixels or less in units of 1/4 pixels. In the case of the color difference signal, a DCT-based 4-tap interpolation filter in which filter coefficients are different to generate pixel information of integer pixels or less in units of 1/8 pixels may be used.

The motion prediction unit may perform motion prediction based on the reference picture interpolated by the reference picture interpolator. As a method for calculating the motion vector, various methods such as Full search-based Block Matching Algorithm (FBMA), Three Step Search (TSS), and New Three-Step Search Algorithm (NTS) may be used. The motion vector may have a motion vector value of 1/2 or 1/4 pixel unit based on the interpolated pixel. The motion prediction unit may predict the current prediction unit by using a different motion prediction method. Various methods, such as a skip method, a merge method, an AMVP (Advanced Motion Vector Prediction) method, an intra block copy method, etc., may be used as the motion prediction method.

The intra prediction unit 125 may generate a prediction unit based on reference pixel information around the current block, which is pixel information in the current picture. When a neighboring block of the current prediction unit is a block on which inter prediction is performed, and thus a reference pixel is a pixel on which inter prediction is performed, a reference pixel included in the block on which inter prediction is performed is a reference pixel of the block on which intra prediction has been performed. information can be used instead. That is, when the reference pixel is not available, the unavailable reference pixel information may be replaced with at least one reference pixel among the available reference pixels.

In intra prediction, the prediction mode may have a directional prediction mode in which reference pixel information is used according to a prediction direction and a non-directional mode in which directional information is not used when prediction is performed. A mode for predicting luminance information and a mode for predicting chrominance information may be different, and intra prediction mode information used for predicting luminance information or predicted luminance signal information may be utilized to predict chrominance information.

When intra prediction is performed, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit based on the pixel present on the left side, the pixel present on the upper left side, and the pixel present on the upper side of the prediction unit can be performed. However, when the size of the prediction unit is different from the size of the transformation unit when performing intra prediction, intra prediction may be performed using a reference pixel based on the transformation unit. In addition, intra prediction using NxN splitting may be used only for the smallest coding unit.

The intra prediction method may generate a prediction block after applying an adaptive intra smoothing (AIS) filter to a reference pixel according to a prediction mode. The type of AIS filter applied to the reference pixel may be different. In order to perform the intra prediction method, the intra prediction mode of the current prediction unit may be predicted from the intra prediction mode of the prediction unit existing around the current prediction unit. When the prediction mode of the current prediction unit is predicted using the mode information predicted from the neighboring prediction unit, if the intra prediction mode of the current prediction unit and the neighboring prediction unit are the same, the current prediction unit and the neighboring prediction unit are used using predetermined flag information It is possible to transmit information that the prediction modes of . , and if the prediction modes of the current prediction unit and the neighboring prediction units are different from each other, entropy encoding may be performed to encode prediction mode information of the current block.

In addition, a residual block including residual information, which is a difference value between a prediction unit and an original block of the prediction unit, in which prediction is performed based on the prediction unit generated by the prediction units 120 and 125 may be generated. The generated residual block may be input to the transform unit 130 .

The transform unit 130 transforms the residual block including the original block and the residual information of the prediction unit generated by the prediction units 120 and 125, such as DCT (Discrete Cosine Transform) or DST (Discrete Sine Transform). method can be used to transform Here, the DCT conversion core includes at least one of DCT2 or DCT8, and the DST conversion core includes DST7. Whether to apply DCT or DST to transform the residual block may be determined based on intra prediction mode information of a prediction unit used to generate the residual block. The transform for the residual block may be skipped. A flag indicating whether to skip transform for the residual block may be encoded. Transform skip may be allowed for residual blocks whose size is below a threshold, luma components, or chroma components under 4:4:4 format.

The quantizer 135 may quantize the values transformed by the transform unit 130 into the frequency domain. The quantization coefficient may change according to blocks or the importance of an image. The value calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160 .

The rearrangement unit 160 may rearrange the coefficient values on the quantized residual values.

The reordering unit 160 may change the two-dimensional block form coefficient into a one-dimensional vector form through a coefficient scanning method. For example, the rearranging unit 160 may use a Zig-Zag Scan method to scan from DC coefficients to coefficients in a high-frequency region and change them into a one-dimensional vector form. A vertical scan that scans a two-dimensional block shape coefficient in a column direction and a horizontal scan that scans a two-dimensional block shape coefficient in a row direction may be used instead of a zig-zag scan according to a size of a transform unit and an intra prediction mode. That is, it may be determined whether any of the zig-zag scan, the vertical scan, and the horizontal scan is used according to the size of the transform unit and the intra prediction mode.

The entropy encoding unit 165 may perform entropy encoding based on the values calculated by the reordering unit 160 . For entropy encoding, various encoding methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be used.

The entropy encoding unit 165 receives the residual value coefficient information and block type information, prediction mode information, division unit information, prediction unit information and transmission unit information, motion of the coding unit from the reordering unit 160 and the prediction units 120 and 125 . Various information such as vector information, reference frame information, interpolation information of a block, and filtering information may be encoded.

The entropy encoder 165 may entropy-encode the coefficient values of the coding units input from the reordering unit 160 .

The inverse quantizer 140 and the inverse transform unit 145 inversely quantize the values quantized by the quantizer 135 and inversely transform the values transformed by the transform unit 130 . The residual values generated by the inverse quantizer 140 and the inverse transform unit 145 are combined with the prediction units predicted through the motion estimation unit, the motion compensator, and the intra prediction unit included in the prediction units 120 and 125 and restored. You can create a Reconstructed Block.

The filter unit 150 may include at least one of a deblocking filter, an offset correcting unit, and an adaptive loop filter (ALF).

The deblocking filter may remove block distortion caused by the boundary between blocks in the reconstructed picture. In order to determine whether to perform deblocking, it may be determined whether to apply the deblocking filter to the current block based on pixels included in several columns or rows included in the block. When a deblocking filter is applied to a block, a strong filter or a weak filter can be applied according to the required deblocking filtering strength. In addition, in applying the deblocking filter, horizontal filtering and vertical filtering may be concurrently processed when performing vertical filtering and horizontal filtering.

The offset correcting unit may correct an offset from the original image in units of pixels with respect to the image on which the deblocking has been performed. In order to perform offset correction on a specific picture, a method of dividing pixels included in an image into a certain number of regions, determining the region to be offset and applying the offset to the region, or taking edge information of each pixel into consideration can be used to apply

Adaptive loop filtering (ALF) may be performed based on a value obtained by comparing the filtered reconstructed image and the original image. After dividing the pixels included in the image into a predetermined group, one filter to be applied to the corresponding group is determined, and filtering can be performed differentially for each group. As for information on whether to apply ALF, the luminance signal may be transmitted for each coding unit (CU), and the shape and filter coefficients of the ALF filter to be applied may vary according to each block. In addition, the ALF filter of the same type (fixed type) may be applied regardless of the characteristics of the target block.

The memory 155 may store the reconstructed block or picture calculated through the filter unit 150 , and the stored reconstructed block or picture may be provided to the predictors 120 and 125 when inter prediction is performed.

Figure 2

2 is a block diagram of an image decoder (decoder) according to an embodiment of the present invention.

Referring to FIG. 2 , the image decoder 200 includes an entropy decoding unit 210, a reordering unit 215, an inverse quantization unit 220, an inverse transform unit 225, prediction units 230 and 235, and a filter unit ( 240) and a memory 245 may be included.

When an image bitstream is input from an image encoder, the input bitstream may be decoded by a procedure opposite to that of the image encoder.

The entropy decoding unit 210 may perform entropy decoding in a procedure opposite to that performed by the entropy encoding unit of the image encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be applied corresponding to the method performed by the image encoder.

The entropy decoder 210 may decode information related to intra prediction and inter prediction performed by the encoder.

The reordering unit 215 may perform reordering based on a method of rearranging the entropy-decoded bitstream by the entropy decoding unit 210 by the encoder. Coefficients expressed in the form of a one-dimensional vector may be restored and rearranged as coefficients in the form of a two-dimensional block. The reordering unit 215 may receive information related to coefficient scanning performed by the encoder and perform reordering by performing a reverse scanning method based on the scanning order performed by the corresponding encoder.

The inverse quantizer 220 may perform inverse quantization based on the quantization parameter provided by the encoder and the reordered coefficient values of the blocks.

The inverse transform unit 225 may perform inverse transform, ie, inverse DCT or inverse DST, on the transform performed by the transform unit, ie, DCT or DST, on the quantization result performed by the image encoder. Here, the DCT conversion core may include at least one of DCT2 or DCT8, and the DST conversion core may include DST7. Alternatively, when the image encoder skips the transform, the inverse transform unit 225 may not perform the inverse transform either. Inverse transform may be performed based on a transmission unit determined by the image encoder. In the inverse transform unit 225 of the image decoder, a transformation technique (eg, DCT or DST) may be selectively performed according to a plurality of pieces of information such as a prediction method, a size of a current block, and a prediction direction.

The prediction units 230 and 235 may generate a prediction block based on the prediction block generation related information provided from the entropy decoding unit 210 and previously decoded block or picture information provided from the memory 245 .

As described above, when intra prediction is performed in the same manner as in the operation of the image encoder, when the size of the prediction unit and the size of the transformation unit are the same, the pixel present at the left side of the prediction unit, the pixel present at the upper left side, and the upper side exist Intra prediction is performed on the prediction unit based on the pixel can Also, intra prediction using NxN splitting may be used only for the smallest coding unit.

The prediction units 230 and 235 may include a prediction unit determiner, an inter prediction unit, and an intra prediction unit. The prediction unit determining unit receives various information such as prediction unit information input from the entropy decoder 210, prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, and divides the prediction unit from the current coding unit, and predicts It may be determined whether the unit performs inter prediction or intra prediction. The inter prediction unit 230 uses information necessary for inter prediction of the current prediction unit provided from the image encoder to predict the current based on information included in at least one of a picture before or after the current picture including the current prediction unit. Inter prediction may be performed on a unit. Alternatively, inter prediction may be performed based on information of a pre-restored partial region in the current picture including the current prediction unit.

In order to perform inter prediction, a method of predicting motion of a prediction unit included in a corresponding coding unit based on a coding unit is a skip mode, a merge mode, a motion vector prediction mode (AMVP mode), and an intra block copy. It is possible to determine whether the method is any of the modes.

The intra prediction unit 235 may generate a prediction block based on pixel information in the current picture. When the prediction unit is a prediction unit on which intra prediction is performed, intra prediction may be performed based on intra prediction mode information of the prediction unit provided by the image encoder. The intra prediction unit 235 may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolator, and a DC filter. The AIS filter is a part that performs filtering on the reference pixel of the current block, and may be applied by determining whether to apply the filter according to the prediction mode of the current prediction unit. AIS filtering may be performed on the reference pixel of the current block by using the prediction mode and AIS filter information of the prediction unit provided by the image encoder. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter may not be applied.

When the prediction mode of the prediction unit is a prediction unit in which intra prediction is performed based on a pixel value obtained by interpolating the reference pixel, the reference pixel interpolator may interpolate the reference pixel to generate a reference pixel of a pixel unit having an integer value or less. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter may generate the prediction block through filtering when the prediction mode of the current block is the DC mode.

The reconstructed block or picture may be provided to the filter unit 240 . The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

Information on whether a deblocking filter is applied to the corresponding block or picture and information on whether a strong filter or a weak filter is applied when the deblocking filter is applied may be provided from the video encoder. The deblocking filter of the image decoder may receive deblocking filter-related information provided from the image encoder, and the image decoder may perform deblocking filtering on the corresponding block.

The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image during encoding, information on the offset value, and the like.

ALF may be applied to a coding unit based on information on whether ALF is applied, ALF coefficient information, etc. provided from the encoder. Such ALF information may be provided by being included in a specific parameter set.

The memory 245 may store the reconstructed picture or block to be used as a reference picture or reference block, and may also provide the reconstructed picture to an output unit.

Fig. 3

3 is a diagram illustrating a basic coding tree unit according to an embodiment of the present invention.

A coding block of the maximum size may be defined as a coding tree block. One picture is divided into a plurality of Coding Tree Units (CTUs). The coding tree unit is a coding unit of the largest size, and may be referred to as a Largest Coding Unit (LCU). 3 shows an example in which one picture is divided into a plurality of coding tree units.

The size of the coding tree unit may be defined at a picture level or a sequence level. To this end, information indicating the size of the coding tree unit may be signaled through a picture parameter set or a sequence parameter set.

As an example, the size of the coding tree unit for all pictures in the sequence may be set to 128x128. Alternatively, at the picture level, either 128x128 or 256x256 may be determined as the size of the coding tree unit. For example, in the first picture, the size of the coding tree unit may be set to 128x128, and in the second picture, the size of the coding tree unit may be set to 256x256.

A coding block may be generated by dividing the coding tree unit. A coding block represents a basic unit for encoding/decoding processing. For example, prediction or transformation may be performed for each coding block, or a prediction coding mode may be determined for each coding block. Here, the prediction encoding mode indicates a method of generating a prediction image. For example, the prediction encoding mode includes intra prediction (Intra Prediction), inter prediction (Inter Prediction), Current Picture Referencing (CPR, or Intra Block Copy, IBC). ) or combined prediction. With respect to the coding block, the prediction block for the coding block may be generated by using at least one prediction coding mode of intra prediction, inter prediction, current picture reference, or complex prediction.

Information indicating the prediction encoding mode of the current block may be signaled through a bitstream. For example, the information may be a 1-bit flag indicating whether the prediction encoding mode is an intra mode or an inter mode. Only when the prediction encoding mode of the current block is determined to be the inter mode, the current picture reference or complex prediction may be available.

The current picture reference is to set the current picture as a reference picture and obtain a prediction block of the current block from an already encoded/decoded region in the current picture. Here, the current picture means a picture including the current block. Information indicating whether the current picture reference is applied to the current block may be signaled through the bitstream. For example, the information may be a 1-bit flag. When the flag is true, the prediction encoding mode of the current block may be determined as the current picture reference, and when the flag is false, the prediction mode of the current block may be determined as inter prediction.

Alternatively, the prediction encoding mode of the current block may be determined based on the reference picture index. For example, when the reference picture index indicates the current picture, the prediction encoding mode of the current block may be determined as the current picture reference. When the reference picture index indicates a picture other than the current picture, the prediction encoding mode of the current block may be determined as inter prediction. That is, the current picture reference is a prediction method using information on an encoded/decoded region within the current picture, and the inter prediction is a prediction method using information of another picture that has been encoded/decoded.

Complex prediction indicates a coding mode in which two or more of intra prediction, inter prediction, and current picture reference are combined. For example, when complex prediction is applied, a first prediction block may be generated based on any one of intra prediction, inter prediction, or a current picture reference, and a second prediction block may be generated based on the other one. When the first prediction block and the second prediction block are generated, the final prediction block may be generated through an average operation or a weighted sum operation of the first prediction block and the second prediction block. Information indicating whether complex prediction is applied may be signaled through a bitstream. The information may be a 1-bit flag.

Fig. 4

4 is a diagram illustrating various division forms of a coding block.

A coding block may be divided into a plurality of coding blocks based on quad tree division, binary tree division, or triple tree division. The divided coding block may be further divided into a plurality of coding blocks based on quad tree division, binary tree division, or triple tree division.

Quad tree splitting refers to a splitting technique in which a current block is split into four blocks. As a result of quad-tree splitting, the current block may be split into four square partitions (refer to 'SPLIT_QT' in FIG. 4(a)).

Binary tree splitting refers to a splitting technique in which a current block is split into two blocks. Splitting the current block into two blocks along the vertical direction (i.e., using a vertical line crossing the current block) can be called vertical binary tree splitting, and along the horizontal direction (i.e., using a vertical line crossing the current block) Splitting the current block into two blocks may be referred to as horizontal binary tree splitting. As a result of binary tree partitioning, the current block may be partitioned into two non-square partitions. 'SPLIT_BT_VER' of FIG. 4(b) indicates a result of vertical binary tree division, and 'SPLIT_BT_HOR' of FIG. 4(c) indicates a horizontal binary tree division result.

Triple tree splitting refers to a splitting technique in which a current block is split into three blocks. Splitting the current block into three blocks along the vertical direction (i.e., using two vertical lines intersecting the current block) can be referred to as vertical triple tree splitting, and along the horizontal direction (i.e., using two vertical lines crossing the current block) Splitting the current block into three blocks may be referred to as horizontal triple tree splitting. As a result of the triple tree division, the current block may be divided into three non-square partitions. In this case, the width/height of the partition located in the center of the current block may be twice as large as the width/height of other partitions. 'SPLIT_TT_VER' of FIG. 4 (d) shows a result of vertical triple-tree splitting, and 'SPLIT_TT_HOR' of FIG. 4 (e) shows a horizontal triple-tree splitting result.

The number of divisions of the coding tree unit may be defined as a partitioning depth. The maximum division depth of a coding tree unit may be determined at the sequence or picture level. Accordingly, the maximum division depth of the coding tree unit may be different for each sequence or feature.

Alternatively, the maximum splitting depth for each of the splitting techniques may be individually determined. As an example, the maximum division depth allowed for quad tree division may be different from the maximum division depth allowed for binary tree division and/or triple tree division.

The encoder may signal information indicating at least one of a splitting form or a splitting depth of the current block through a bitstream. The decoder may determine the splitting form and the splitting depth of the coding tree unit based on the information parsed from the bitstream.

5 is a diagram illustrating a splitting aspect of a coding tree unit.

Partitioning a coding block using a partitioning technique such as quad tree partitioning, binary tree partitioning, and/or triple tree partitioning may be referred to as multi-tree partitioning.

Coding blocks generated by applying multi-tree partitioning to the coding block may be referred to as lower coding blocks. When the division depth of the coding block is k, the division depth of the lower coding blocks is set to k+1.

Conversely, with respect to coding blocks having a split depth of k+1, a coding block having a split depth of k may be referred to as a higher coding block.

The division type of the current coding block may be determined based on at least one of the division type of the upper coding block or the division type of the neighboring coding block. Here, the neighboring coding block is adjacent to the current coding block, and may include at least one of an upper neighboring block, a left neighboring block, or a neighboring block adjacent to an upper left corner of the current coding block. Here, the splitting type may include at least one of whether to split a quad tree, whether to split a binary tree, a binary tree splitting direction, whether to split a triple tree, or a triple tree splitting direction.

In order to determine the division form of the coding block, information indicating whether the coding block is divided may be signaled through a bitstream. The information is a 1-bit flag 'split_cu_flag'. When the flag is true, it indicates that the coding block is split by the head tree splitting technique.

When split_cu_flag is true, information indicating whether a coding block is divided into a quad tree may be signaled through a bitstream. The information is a 1-bit flag split_qt_flag. When the flag is true, the coding block may be divided into 4 blocks.

Fig. 5

As an example, in the example shown in FIG. 5 , as the coding tree unit is divided into a quad tree, four coding blocks having a division depth of 1 are generated. In addition, it is shown that quad-tree splitting is applied to the first and fourth coding blocks among the four coding blocks generated as a result of quad-tree splitting. As a result, four coding blocks having a split depth of 2 can be generated.

In addition, by applying quad-tree splitting again to a coding block having a split depth of 2, a coding block having a split depth of 3 may be generated.

When quad tree division is not applied to a coding block, binary tree division is performed on the coding block by considering at least one of the size of the coding block, whether the coding block is located at a picture boundary, the maximum division depth, or the division form of a neighboring block. Alternatively, it may be determined whether to perform triple tree splitting. When it is determined that binary tree splitting or triple tree splitting is performed on the coding block, information indicating a splitting direction may be signaled through a bitstream. The information may be a 1-bit flag mtt_split_cu_vertical_flag. Based on the flag, it may be determined whether the division direction is a vertical direction or a horizontal direction. Additionally, information indicating which of binary tree splitting or triple tree splitting is applied to the coding block may be signaled through the bitstream. The information may be a 1-bit flag mtt_split_cu_binary_flag. Based on the flag, it may be determined whether binary tree splitting or triple tree splitting is applied to the coding block.

For example, in the example shown in FIG. 5, vertical binary tree splitting is applied to a coding block having a splitting depth of 1, and vertical triple tree splitting is applied to a left coding block among the coding blocks generated as a result of the splitting, It is shown that vertical binary tree splitting is applied to the right coding block.

Inter prediction is a prediction encoding mode for predicting a current block by using information on a previous picture. As an example, a block (hereinafter, collocated block, collocated block) at the same position as the current block in the previous picture may be set as the prediction block of the current block. Hereinafter, a prediction block generated based on a block at the same position as the current block will be referred to as a collocated prediction block.

On the other hand, if the object existing in the previous picture moves to a different position in the current picture, the current block can be effectively predicted using the motion of the object. For example, if the movement direction and size of the object can be known by comparing the previous picture with the current picture, the prediction block (or prediction image) of the current block may be generated in consideration of the motion information of the object. Hereinafter, a prediction block generated using motion information may be referred to as a motion prediction block.

A residual block may be generated by differentiating the prediction block from the current block. In this case, if there is motion of the object, by using the motion prediction block instead of the collocated prediction block, the energy of the residual block can be reduced, and thus the compression performance of the residual block can be improved.

As described above, generating a prediction block using motion information may be referred to as motion-compensated prediction. In most inter prediction, a prediction block can be generated based on motion compensation prediction.

The motion information may include at least one of a motion vector, a reference picture index, a prediction direction, or a bidirectional weight index. The motion vector indicates the movement direction and size of the object. The reference picture index specifies a reference picture of the current block among reference pictures included in the reference picture list. The prediction direction indicates any one of unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 prediction and L1 prediction). According to the prediction direction of the current block, at least one of movement information in the L0 direction and movement information in the L1 direction may be used. The bidirectional weight index specifies a weight applied to the L0 prediction block and a weight applied to the L1 prediction block.

Inter prediction includes the steps of determining an inter prediction mode of the current block, obtaining motion information of the current block according to the determined inter prediction mode, and performing motion compensation prediction on the current block based on the obtained motion information may include.

Here, the inter prediction mode represents various techniques for determining motion information of the current block, and may include an inter prediction mode using translation motion information and an inter prediction mode using affine motion information. have. As an example, the inter prediction mode using translational motion information may include a merge mode and a motion vector prediction mode, and the inter prediction mode using the affine motion information may include an affine merge mode and an affine motion vector prediction mode. . The motion information of the current block may be determined based on information parsed from a neighboring block or a bitstream adjacent to the current block according to the inter prediction mode.

The motion information of the current block may be derived from motion information of other blocks in the current block. Here, the other block may be a block encoded/decoded by inter prediction prior to the current block. Setting the motion information of the current block to be the same as the motion information of other blocks may be defined as a merge mode. Also, setting a motion vector of another block as a prediction value of the motion vector of the current block may be defined as a motion vector prediction mode.

A motion vector represents a position difference between an object in a previous picture and an object in the current picture. Information for determining the precision of a motion vector may be signaled through a bitstream. The information may be determined in units of sequences, slices, or blocks. The motion vector precision can be set to octopel, quarter pel, half pel, integer pel, or 4 integer pels.

Next, an inter prediction method using translational motion information will be described in detail.

The motion information of the current block may be derived from motion information of other blocks in the current block. Here, the other block may be a block encoded/decoded by inter prediction prior to the current block. Setting the motion information of the current block to be the same as the motion information of other blocks may be defined as a merge mode. Also, setting a motion vector of another block as a prediction value of the motion vector of the current block may be defined as a motion vector prediction mode.

Fig. 6

6 is a flowchart of a process of deriving motion information of a current block under a merge mode.

A merge candidate of the current block may be derived (S601). The merge candidate of the current block may be derived from a block encoded/decoded by inter prediction prior to the current block.

The motion information of the merge candidate may be set to be the same as the motion information of the candidate block. As an example, at least one of a motion vector, a reference picture index, a prediction direction, and a bidirectional weight index of the candidate block may be set as the motion information of the merge candidate.

A merge candidate list including merge candidates may be generated ( S602 ). The merge candidate may be divided into a neighboring merge candidate derived from a neighboring block adjacent to the current block and a non-adjacent merge candidate derived from a non-neighboring block.

Indices of merge candidates in the merge candidate list may be allocated according to a predetermined order.

When a plurality of merge candidates are included in the merge candidate, at least one of the plurality of merge candidates may be selected ( S603 ).

When the number of merge candidates included in the merge candidate list is smaller than the threshold, the merge candidates included in the prediction region motion information table may be added to the merge candidate list. Here, the threshold value may be a value obtained by subtracting an offset from the maximum number of merge candidates or the maximum number of merge candidates that the merge candidate list may include. The offset may be a natural number such as 1 or 2. The inte-region motion information table may include a merge candidate derived based on a block encoded/decoded before the current block.

The prediction region motion information table includes merge candidates derived from blocks encoded/decoded based on inter prediction within the current picture. As an example, motion information of a merge candidate included in the prediction region motion information table may be set to be the same as motion information of a block encoded/decoded based on inter prediction. Here, the motion information may include at least one of a motion vector, a reference picture index, a prediction direction, and a bidirectional weight index.

For convenience of description, a merge candidate included in the prediction region motion information table will be referred to as a prediction region merge candidate.

The maximum number of merge candidates that the prediction region motion information table can include may be predefined in the encoder and the decoder. As an example, the maximum number of merge candidates that the prediction region motion information table may include may be 1, 2, 3, 4, 5, 6, 7, 8 or more (eg, 16).

Alternatively, information indicating the maximum number of merge candidates that the prediction region motion information table may include may be signaled through a bitstream. The information may be signaled at a sequence, picture, or slice level. The information may indicate the maximum number of merge candidates that the prediction region motion information table may include. Alternatively, the information may indicate a difference between the maximum number of merge candidates that the prediction region motion information table may include and the maximum number of merge candidates that the merge candidate list may include.

Alternatively, the maximum number of merge candidates of the prediction region motion information table may be determined according to the size of the picture, the size of the slice, or the size of the coding tree unit.

Fig. 7

7 is a diagram for explaining an update aspect of a prediction region motion information table.

When inter prediction is performed on the current block, a prediction region merge candidate may be derived based on the current block. The motion information of the prediction region merge candidate may be set to be the same as the motion information of the current block.

When the prediction region motion information table is in an empty state, a prediction region merge candidate derived based on the current block may be added to the prediction region motion information table.

When the prediction region motion information table already includes the prediction region merge candidate, the redundancy check may be performed on the motion information of the current block (or the prediction region merge candidate derived based thereon). The redundancy check is for determining whether motion information of a prediction region merge candidate previously stored in the prediction region motion information table and motion information of the current block are the same. The redundancy check may be performed on all prediction region merge candidates previously stored in the prediction region motion information table. Alternatively, the redundancy check may be performed on prediction region merge candidates whose index is greater than or equal to a threshold value or less than or equal to a threshold value among prediction region merge candidates previously stored in the prediction region motion information table.

When an inter prediction merge candidate having the same motion information as the motion information of the current block is not included, a prediction region merge candidate derived based on the current block may be added to the prediction region motion information table. Whether the inter prediction merge candidates are the same may be determined based on whether motion information (eg, a motion vector and/or a reference picture index, etc.) of the inter prediction merge candidates is the same.

At this time, if the maximum number of prediction region merge candidates is already stored in the prediction region motion information table, the oldest prediction region merge candidate is deleted, and the prediction region merge candidate derived based on the current block is added to the prediction region motion information table. can be added Here, the oldest prediction region merge candidate may be a prediction region merge candidate having the largest index or a prediction region merge candidate having the smallest index.

When a coding block is divided into a plurality of prediction units, intra prediction or inter prediction may be performed on each of the divided prediction units.

Fig. 8

8 is a diagram illustrating an example of dividing a coding block into a plurality of prediction units using a diagonal line.

As in the example shown in (a) and (b) of FIG. 8 , a coding block may be divided into two triangular shape prediction units using a diagonal line.

In (a) and (b) of FIG. 8, it is illustrated that the coding block is divided into two prediction units using a diagonal line connecting two vertices of the coding block. However, the coding block may be divided into two prediction units by using a diagonal line in which at least one end of the line does not pass through the vertex of the coding block.

Fig. 9

9 is a diagram illustrating an example of dividing a coding block into two prediction units.

As in the example shown in (a) and (b) of FIG. 9 , the coding block may be divided into two prediction units by using a diagonal line at both ends touching the upper boundary and the lower boundary of the coding block, respectively. .

Alternatively, as in the example shown in (c) and (d) of FIG. 9 , the coding block is divided into two prediction units using diagonal lines at both ends touching the left and right boundaries of the coding block, respectively. can

Alternatively, the coding block may be divided into two prediction blocks having different sizes. As an example, the coding block may be divided into two prediction units having different sizes by setting the diagonal line dividing the coding block to be in contact with two boundary planes forming one vertex.

Fig. 10

10 shows examples of dividing a coding block into a plurality of prediction blocks having different sizes.

As in the example shown in (a) and (b) of FIG. 10 , the diagonal line connecting the upper left and lower right ends of the coding block does not pass through the upper left corner or the lower right corner of the coding block. Instead of passing through the upper left corner or the lower right corner, the left boundary, the right boundary, the upper boundary Alternatively, by setting it to cross the lower boundary, the coding block may be divided into two prediction units having different sizes.

Or, as in the example shown in (c) and (d) of Figure 10, the diagonal line connecting the upper right and lower left of the coding block passes through the upper left corner or the lower right corner of the coding block, instead of the left boundary, the right boundary, By setting it to cross the upper boundary or the lower boundary, it is possible to divide the coding block into two prediction units having different sizes.

The prediction method of FIGS. 8 to 10 is defined as a diagonal prediction method. In the diagonal prediction method, different prediction encoding/decoding methods may be used. Here, the predictive encoding/decoding method indicates an intra prediction method, an inter prediction method (including a merge prediction encoding method), an intra block copy encoding method, and the like.

As an example, intra prediction may be used in one prediction block and a merge prediction encoding method may be used in another prediction block, which may be defined as a diagonal joint prediction method.

Fig. 11

When the size of the coding block or the height-to-width ratio (hereinafter, whratio) of the coding block is equal to or greater than a threshold value, it may be set to allow only a predefined partition among the diagonal prediction methods. Here, whratio can be defined as in Equation 1.

For example, when the size of the coding block is Nx4 or Nx8, only the partition as shown in the right figure of FIG. 11 can be allowed in the diagonal prediction method, and when the size of the coding block is 4xN and 8xN, the diagonal prediction method of FIG. 11 Only partitions like the picture on the right can be allowed.

As another example, when the absolute value of whratio is N or more, it may be set to allow only one predefined partition in the diagonal prediction method.

The method described in FIG. 11 or the method is defined as a thin diagonal prediction method.

In the thin diagonal prediction method, different prediction encoding/decoding methods may be applied to each partition. For example, intra prediction is used in the first partition (left or upper partition), the merge prediction method is used in the second partition (right or lower partition), or the merge prediction method is used in the first partition, and the second partition may use an intra prediction method.

Intra prediction is to predict a current block by using a reconstructed sample that has been encoded/decoded around the current block. In this case, the reconstructed sample before the in-loop filter is applied may be used for intra prediction of the current block.

The intra prediction technique includes a matrix-based intra prediction and a general intra prediction in consideration of a direction with respect to a neighboring reconstructed sample. Information indicating the intra prediction technique of the current block may be signaled through a bitstream. The information may be a 1-bit flag. Alternatively, the intra prediction technique of the current block may be determined based on at least one of the position, size, and shape of the current block or the intra prediction technique of a neighboring block. For example, when the current block spans a picture boundary, it may be set so that intra prediction based on a metric is not applied to the current block.

Matrix-based intra prediction is a method of obtaining a prediction block of a current block based on a matrix product between a matrix pre-stored in the encoder and decoder and a reconstructed sample around the current block. Information for specifying any one of a plurality of pre-stored matrices may be signaled through a bitstream. The decoder may determine a matrix for intra prediction of the current block based on the information and the size of the current block.

General intra prediction is a method of obtaining a prediction block for a current block based on a non-directional intra prediction mode or a directional intra prediction mode. Hereinafter, a process for performing intra prediction based on general intra prediction will be described in more detail with reference to the drawings.

Fig. 12

12 is a flowchart of an intra prediction method according to an embodiment of the present invention.

A reference sample line of the current block may be determined. The reference sample line refers to a set of reference samples included in a line k-th distant from the top and/or left of the current block. The reference sample may be derived from a reconstructed sample that has been encoded/decoded around the current block.

Index information for identifying a reference sample line of a current block among a plurality of reference sample lines may be signaled through a bitstream. As an example, index information intra_luma_ref_idx for specifying a reference sample line of the current block may be signaled through a bitstream. The index information may be signaled in units of coding blocks. The plurality of reference sample lines may include at least one of a first line, a second line, a third line, or a fourth line from the top and/or left of the current block. Among the plurality of reference sample lines, a reference sample line composed of a row adjacent to the top of the current block and a column adjacent to the left of the current block is called an adjacent reference sample line, and other reference sample lines are called non-adjacent reference sample lines. may be called

Fig. 13

13 is a diagram illustrating candidate reference sample lines according to an embodiment of the present invention.

Only some of the plurality of reference sample lines may be selected as reference sample lines of the current block. As an example, the remaining reference sample lines excluding the third non-adjacent reference sample line among the plurality of reference sample lines may be set as candidate reference sample lines. Table 3 shows indices allocated to each of the candidate reference sample lines.

index (intra_luma_ref_idx) Reference sample line 0 Adjacent reference sample line One first non-adjacent reference sample line 2 second non-adjacent reference sample line

A larger number of candidate reference sample lines may be set than described, or a smaller number of candidate reference sample lines may be set. In addition, the number or positions of non-adjacent reference sample lines set as candidate reference sample lines are not limited to the described example. As an example, the first non-adjacent reference sample line and the third non-adjacent reference sample line are set as candidate reference sample lines, or the second non-adjacent reference sample line and the third non-adjacent reference sample line are set as the candidate reference sample lines. You can also set Alternatively, the first non-adjacent reference sample line, the second non-adjacent reference sample line, and the third non-adjacent reference sample line may all be set as candidate reference sample lines.

The reference sample line may include upper reference samples positioned at the top of the current block and left reference samples positioned to the left of the current block. The upper reference samples and the left reference samples may be derived from reconstructed samples around the current block. The reconstructed samples may be in a state before the in-loop filter is applied.

The number of reference samples included in the reference sample line may be determined based on a distance between the reference sample line and the reference sample line. For example, the number of reference samples included in a reference sample line having a distance of i from the current block may have a greater value than the number of reference samples included in a reference sample line having a distance of i−1 from the current block. . Accordingly, the number of reference samples included in the non-adjacent reference sample line may have a greater value than the number of reference samples included in the adjacent reference sample line.

A difference between the number of reference samples included in a non-adjacent reference sample line having a distance of i from the current block and the number of reference samples included in an adjacent reference sample line may be defined as a reference sample number offset. In this case, a difference in the number of upper reference samples positioned at the top of the current block may be defined as offsetX[i], and a difference in number of left reference samples positioned at the left of the current block may be defined as offsetY[i]. offsetX and offsetY may be determined based on a distance between the current block and a non-adjacent reference sample line. For example, offsetX and offsetY may be set to an integer multiple of i. As an example, offsetX[i] and offset[i] may be 2i.

Alternatively, the reference sample number offset may be determined based on the width and height ratio of the current block. Equation 1 shows an example of digitizing the width and height ratio of the current block.

It is also possible to digitize the width and height ratio of the current block in a different way than that expressed in Equation 1.

The values of offsetX and offsetY may be determined based on the width and height ratio of the current block. For example, when the value of whRatio is greater than 1, the value of offsetX may be set to be greater than the value of offsetY. For example, the value of offsetX may be set to 1, and the value of offsetY may be set to 0. On the other hand, when the value of whRatio is less than 1, the value of offsetY may be set larger than the value of offsetX. For example, the value of offsetX may be set to 0, and the value of offsetY may be set to 1.

Except for the top left reference sample with the same x-axis and y-axis coordinates, a non-adjacent reference sample line with a distance of i from the current block is (refW + offsetX[i]) top reference samples and (refH + offsetY[i]) It may be composed of left reference samples. Here, refW and refH represent the length of the adjacent reference sample line, and may be set as in Equations 2 and 3, respectively.

In Equations 2 and 3, nTbW represents a width of a coding block or transform block on which intra prediction is performed, and nTbH represents a height of a coding block or transform block on which intra prediction is performed.

As a result, a reference sample line having a distance of i from the current block may be composed of (refW + refH + offsetX[i] + offsetY[i] + 1) reference samples.

Next, the intra prediction mode of the current block may be determined (S1202). At least one of a non-directional intra prediction mode and a directional intra prediction mode may be determined as the intra prediction mode of the current block as the intra prediction mode of the current block. The non-directional intra prediction mode includes a planar and DC, and the directional intra prediction mode includes 33 or 65 modes from the lower left diagonal direction to the upper right diagonal direction.

Fig. 14

14 is a diagram illustrating intra prediction modes.

FIG. 14A shows 35 intra prediction modes, and FIG. 14B shows 67 intra prediction modes.

A larger or smaller number of intra prediction modes may be defined than shown in FIG. 14 .

When the reference sample line of the current block is a non-adjacent reference sample line, the non-directional intra prediction mode may be set not to be used. For example, when the reference sample line of the current block is the first non-adjacent reference sample line, the second non-adjacent reference sample line, or the third non-adjacent reference sample line, DC or the planner may be set to the intra prediction mode of the current block. have.

Alternatively, when the reference sample line of the current block is a non-adjacent reference sample line, at least one of the non-directional intra prediction mode and the intra prediction mode of a specific direction may be set not to be used. The non-directional intra prediction mode includes at least one of DC or planar, and the intra prediction mode in a specific direction includes a horizontal intra prediction mode (eg, INTRA_MODE18 shown in FIG. 14B), a vertical intra prediction mode (eg, , INTRA_MODE50 shown in FIG. 14B) or a diagonal intra prediction mode (eg, INTRA_MODE2, INTRA_34, or INTRA_MODE66 shown in FIG. 14B).

For the current block, information indicating whether the same MPM as the intra prediction mode of the current block is included in the MPM list may be signaled. The information is a 1-bit flag and may be referred to as an MPM flag. When the MPM flag indicates that the same MPM as the current block is included in the MPM list, the intra prediction mode of the current block may be derived based on at least one of a default mode and a Most Probable Mode (MPM).

The default mode may include at least one of DC, planar, vertical intra prediction mode, horizontal intra prediction mode, and diagonal intra prediction mode. Information indicating whether the intra prediction mode of the current block is the same as the default mode may be signaled through the bitstream. The information may be a 1-bit flag.

When the flag indicates that the intra prediction mode of the current block is the default mode, and a plurality of intra prediction modes are set as default modes, index information indicating any one of the default modes may be further signaled. The intra prediction mode of the current block may be set as a default mode indicated by the index information.

When the intra prediction mode of the current block is not the same as the default mode, the intra prediction mode of the current block may be derived using the MPM list.

Hereinafter, the MPM induction method will be described in detail.

The MPM may be derived based on the intra prediction mode of a neighboring block adjacent to the current block. Here, the neighboring block is a left neighboring block adjacent to the left of the current block, an upper neighboring block neighboring to the upper end of the current block, an upper left neighboring block neighboring to the upper left corner of the current block, and an upper right neighboring block adjacent to the upper right corner of the current block It may include at least one of an upper-right neighboring block or a lower-left neighboring block adjacent to a lower-left corner of the current block. When the coordinates of the upper left sample of the current block are (0, 0), the left neighboring block is (-1, 0), (-1, H-1) or (-1, (H-1)/2) A sample of the location may be included. Here, H represents the height of the current block. The top neighboring block may contain samples at positions (0, -1), (W-1, -1) or ((W-1)/2, -1). Here, W represents the width of the current block. The upper-left neighboring block may include a sample at the (-1, -1) position. The upper right neighboring block may include a sample at the (W, -1) position. The lower left neighboring block may include a sample at the (-1, H) position.

Based on the reference sample line of the current block, the number or position of the neighboring blocks used to derive the MPM may be determined differently. As an example, when the neighboring reference sample line is determined as the reference sample line of the current block, the upper neighboring block and the left neighboring block may be used to derive the MPM. On the other hand, when the non-adjacent reference sample line is determined as the reference sample line of the current block, the MPM is derived using at least one of an upper-left neighboring block, an upper-right neighboring block, or a lower-left neighboring block, in addition to the upper neighboring block and the left neighboring block. can do.

When a flag indicating whether the intra prediction mode of the current block is the same as the default mode is signaled, the same intra prediction mode as the default mode may be set not to be set as the MPM. As an example, when the default mode flag indicates whether the intra prediction mode of the current block is the planner, the intra prediction mode of the current block may be derived by using 5 MPMs excluding the MPM corresponding to the planner.

When the intra prediction mode of the current block is determined, prediction samples for the current block may be obtained based on the determined intra prediction mode ( S1203 ).

When the DC mode is selected, prediction samples for the current block are generated based on the average value of the reference samples. Specifically, a value of all samples in the prediction block may be generated based on an average value of reference samples. The average value may be derived using at least one of upper reference samples positioned at the top of the current block and left reference samples positioned at the left of the current block.

Depending on the shape of the current block, the number or range of reference samples used to derive the average value may vary. As an example, when the current block is a non-square block having a width greater than a height, an average value may be calculated using only upper reference samples. On the other hand, when the current block is a non-square block having a width smaller than a height, an average value may be calculated using only the left reference samples. That is, when the width and height of the current block are different, the average value may be calculated using only reference samples adjacent to the longer length. Alternatively, based on the width and height ratio of the current block, it may be determined whether to calculate the average value using only the upper reference samples or whether to calculate the average value using only the left reference samples.

When the planner mode is selected, a prediction sample may be obtained using the horizontal direction prediction sample and the vertical direction prediction sample. Here, the horizontal prediction sample is obtained based on a left reference sample and a right reference sample located on the same horizontal line as the prediction sample, and the vertical direction prediction sample includes an upper reference sample and a lower portion located on the same vertical line as the prediction sample. It is obtained based on a reference sample. Here, the right reference sample may be generated by copying the reference sample adjacent to the upper right corner of the current block, and the lower reference sample may be generated by copying the reference sample adjacent to the lower left corner of the current block. A horizontal direction prediction sample may be obtained based on a weighted sum operation of a left reference sample and a right reference sample, and a vertical direction prediction sample may be obtained based on a weighted sum operation of an upper reference sample and a lower reference sample. In this case, the weight given to each reference sample may be determined according to the location of the prediction sample. The prediction sample may be obtained based on an average operation or a weighted sum operation of the horizontal direction prediction sample and the vertical direction prediction sample. When the weighted sum operation is performed, weights given to the horizontal direction prediction sample and the vertical direction prediction sample may be determined based on the positions of the prediction samples.

When a directional prediction mode is selected, a parameter indicating a prediction direction (or prediction angle) of the selected directional prediction mode may be determined. Table 4 below shows the intra direction parameter intraPredAng for each intra prediction mode.

PredModeIntra
IntraPredAng One
- 2
32 3
26 4
21 5
17 6
13 7
9 PredModeIntraIntraPredAng 8
5 9
2 10
0 11
-2 12
-5 13
-9 14
-13 PredModeIntraIntraPredAng 15
-17 16
-21 17
-26 18
-32 19
-26 20
-21 21
-17 PredModeIntraIntraPredAng 22
-13 23
-9 24
-5 25
-2 26
0 27
2 28
5 PredModeIntraIntraPredAng 29
9 30
13 31
17 32
21 33
26 34
32

Table 4 shows the intra-direction parameters of each of the intra-prediction modes whose index is any one of 2 to 34 when 35 intra-prediction modes are defined. When more than 33 directional intra prediction modes are defined, Table 4 can be further subdivided to set intra-direction parameters of each directional intra prediction mode. After arranging, a prediction sample may be obtained based on the value of the intra direction parameter. In this case, when the value of the intra direction parameter is negative, the left reference samples and the upper reference samples may be arranged in a line.

Fig. 15

Fig. 16

15 and 16 are diagrams illustrating examples of a one-dimensional array in which reference samples are arranged in a line.

15 shows an example of a vertical one-dimensional array in which reference samples are arranged in a vertical direction, and FIG. 16 shows an example of a horizontal one-dimensional array in which reference samples are arranged in a horizontal direction. Assuming that 35 intra prediction modes are defined, the embodiments of FIGS. 15 and 16 will be described.

When the intra prediction mode index is any one of 11 to 18, a horizontal one-dimensional array in which the upper reference samples are rotated counterclockwise is applied, and when the intra prediction mode index is any one of 19 to 25, the left reference samples are A vertical one-dimensional array rotated in a clockwise direction can be applied. In arranging the reference samples in a line, the intra prediction mode angle may be considered.

Based on the intra direction parameter, a reference sample determination parameter may be determined. The reference sample determination parameter may include a reference sample index for specifying a reference sample and a weight parameter for determining a weight applied to the reference sample.

The reference sample index iIdx and the weight parameter ifact may be obtained through Equations 4 and 5 below.

In Equations 4 and 5, P _ang represents an intra direction parameter. The reference sample specified by the reference sample index iIdx corresponds to an integer pel.

In order to derive a prediction sample, at least one or more reference samples may be specified. Specifically, in consideration of the slope of the prediction mode, the position of the reference sample used to derive the prediction sample may be specified. As an example, a reference sample used to derive a prediction sample may be specified using the reference sample index iIdx.

In this case, when the slope of the intra prediction mode is not expressed by one reference sample, a prediction sample may be generated by interpolating a plurality of reference samples. For example, when the slope of the intra prediction mode is a value between the slope between the prediction sample and the first reference sample and the slope between the prediction sample and the second reference sample, the prediction sample is obtained by interpolating the first reference sample and the second reference sample. can be obtained That is, when the angular line following the intra prediction angle does not pass the reference sample located in the integer pel, the prediction sample can be obtained by interpolating the reference samples located adjacent to the left, right, top and bottom of the position where the angular line passes. have.

Equation 6 below shows an example of obtaining a prediction sample based on reference samples.

In Equation 6, P denotes a prediction sample, and Ref_1D denotes any one of the one-dimensionally arranged reference samples. In this case, the position of the reference sample may be determined by the position (x, y) of the prediction sample and the reference sample index iIdx.

When the gradient of the intra prediction mode can be expressed by one reference sample, the weight parameter i _fact is set to 0. Accordingly, Equation 6 can be simplified as Equation 7 below.

Intra prediction may be performed on the current block based on a plurality of intra prediction modes. As an example, an intra prediction mode may be derived for each prediction sample, and a prediction sample may be derived based on the intra prediction mode allocated to each prediction sample.

Fig. 17

17 is a diagram illustrating angles formed by directional intra prediction modes with a straight line parallel to the x-axis.

As in the example shown in FIG. 17 , the directional prediction modes may exist between the lower left diagonal direction and the upper right diagonal direction. When described as an angle formed by the x-axis and the directional prediction mode, the directional prediction modes may exist between 45 degrees (lower left diagonal direction) and -135 degrees (upper right diagonal direction).

When the current block has a non-square shape, according to the intra prediction mode of the current block, a reference sample farther from the prediction sample is used instead of the reference sample closer to the prediction sample among reference samples located on the angular line along the intra prediction angle Thus, a case of deriving a prediction sample may occur.

Fig. 18

18 is a diagram illustrating an aspect in which a prediction sample is obtained when a current block has a non-square shape.

For example, as in the example shown in (a) of FIG. 18 , the current block has a non-square width greater than the height, and the intra prediction mode of the current block is a directional intra prediction mode having an angle between 0 degrees and 45 degrees. is assumed to be In the above case, when deriving the prediction sample A near the right column of the current block, the left reference sample far from the prediction sample instead of the top reference sample T close to the prediction sample among the reference samples located on the angular mode along the angle L may be used.

As another example, as in the example shown in (b) of FIG. 18 , the current block is a square with a height greater than the width, and the intra prediction mode of the current block is a directional intra prediction mode between -90 and -135 degrees. assume that In the above case, when deriving the prediction sample A near the bottom row of the current block, the top reference sample far from the prediction sample instead of the left reference sample L close to the prediction sample among the reference samples located on the angular mode along the angle There may be cases where T is used.

Fig. 19

19 is It is a diagram illustrating wide-angle intra prediction modes.

In the example shown in FIG. 19 , intra prediction modes having an index of -1 to -14 and intra prediction modes having an index of 67 to 80 represent wide-angle intra prediction modes.

19 exemplifies 14 wide-angle intra prediction modes (-1 to -14) with an angle greater than 45 degrees and 14 wide-angle intra prediction modes (67 to 80) with an angle smaller than -135 degrees, but more A greater or lesser number of wide angle intra prediction modes may be defined.

When the wide-angle intra prediction mode is used, the length of the upper reference samples may be set to 2W+1, and the length of the left reference samples may be set to 2H+1.

By using the wide-angle intra prediction mode, the sample A shown in FIG. 18A is predicted using the reference sample T, and the sample A shown in FIG. 18B is predicted using the reference sample L can be

By adding the existing intra prediction modes and N wide-angle intra prediction modes, a total of 67 + N intra prediction modes may be used. As an example, Table 5 shows intra direction parameters of intra prediction modes when 20 wide-angle intra prediction modes are defined.

PredModeIntra -10 -9 -8 -7 -6 -5 -4 -3 -2 intraPredAngle 114 93 79 68 60 54 49 45 39 PredModeIntra -One 2 3 4 5 6 7 8 9 intraPredAngle 35 32 29 26 23 21 19 17 15 PredModeIntra 10 11 12 13 14 15 16 17 18 intraPredAngle 13 11 9 7 5 3 2 One 0 PredModeIntra 19 20 21 22 23 24 25 26 27 intraPredAngle -One -2 -3 -5 -7 -9 -11 -13 -15 PredModeIntra 28 29 30 31 32 33 34 35 36 intraPredAngle -17 -19 -21 -23 -26 -29 -32 -29 -26 PredModeIntra 37 38 39 40 41 42 43 44 45 intraPredAngle -23 -21 -19 -17 -15 -13 -11 -9 -7 PredModeIntra 46 47 48 49 50 51 52 53 54 intraPredAngle -5 -3 -2 -One 0 One 2 3 5 PredModeIntra 55 56 57 58 59 60 61 62 63 intraPredAngle 7 9 11 13 15 17 19 21 23 PredModeIntra 64 65 66 67 68 69 70 71 72 intraPredAngle 26 29 32 35 39 45 49 54 60 PredModeIntra 73 74 75 76 intraPredAngle 68 79 93 114

When the current block is non-square and the intra prediction mode of the current block obtained in step S602 falls within the transform range, the intra prediction mode of the current block may be converted into the wide-angle intra prediction mode. The transformation range may be determined based on at least one of a size, a shape, and a ratio of the current block. Here, the ratio may indicate a ratio between the width and the height of the current block. When the current block has a non-square shape having a width greater than a height, the transformation range starts from the upper right-side diagonal intra prediction mode index (eg, 66). (Index of the intra prediction mode in the upper right diagonal direction - N). Here, N may be determined based on the ratio of the current block. When the intra prediction mode of the current block belongs to a transformation range, the intra prediction mode may be converted into a wide-angle intra prediction mode. The transformation may be to subtract a value predefined for the intra prediction mode, and the predefined value may be the total number of intra prediction modes excluding wide-angle intra prediction modes (eg, 67).

According to the above embodiment, the intra prediction modes of Nos. 66 to 53 may be converted into Wide-angle intra prediction modes of Nos. -1 to -14, respectively.

When the current block has a non-square shape whose height is greater than the width, the transform range may be set from the intra prediction mode index (eg, 2) in the lower left diagonal direction (index of the intra prediction mode in the lower left diagonal direction + M). . Here, M may be determined based on the ratio of the current block. When the intra prediction mode of the current block belongs to a transformation range, the intra prediction mode may be converted into a wide-angle intra prediction mode. The transformation may be to add a predefined value to the intra prediction mode, and the predefined value may be the total number of directional intra prediction modes (eg, 65) excluding wide-angle intra prediction modes.

According to the above embodiment, each of the intra prediction modes of Nos. 2 to 15 may be converted into Nos. 67 to No. 80 wide-angle intra prediction modes.

Hereinafter, intra prediction modes belonging to the transform range will be referred to as wide-angle alternative intra prediction modes.

The transformation range may be determined based on the ratio of the current block. As an example, Tables 6 and 7 show the transformation ranges when 35 intra prediction modes are defined and 67 intra prediction modes are defined except for the wide-angle intra prediction mode, respectively.

Condition Replaced Intra Prediction Modes W/H = 2 Modes 2, 3, 4 W/H > 2 Modes 2, 3, 4, 5, 6 W/H = 1 None H/W = 1/2 Modes 32, 33, 34 H/W < 1/2 Modes 30, 31, 32, 33, 34

Condition Replaced Intra Prediction Modes W/H = 2 Modes 2, 3, 4, 5, 6, 7 W/H > 2 Modes 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 W/H = 1 None H/W = 1/2 Modes 61, 62, 63, 64, 65, 66 H/W < 1/2 Modes 57, 58, 59, 60, 61, 62, 63, 64, 65, 66

As in the examples shown in Tables 6 and 7, the number of wide-angle alternative intra prediction modes included in the transform range may be different according to the ratio of the current block. You can also set the conversion range as

Condition Replaced Intra Prediction Modes W/H = 16 Modes 12, 13, 14, 15 W/H = 8 Modes 12, 13 W/H = 4 Modes 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 H/W = 2 Modes 2, 3, 4, 5, 6, 7 H/W = 1 None W/H = 1/2 Modes 61, 62, 63, 64, 65, 66 W/H = 1/4 Modes 57, 58, 59, 60, 61, 62, 63, 64, 65, 66 W/H = 1/8 Modes 55, 56 H/W = 1/16 Modes 53, 54, 55, 56

When the non-adjacent reference sample line is determined as the reference sample line of the current block, or when a multi-line intra prediction encoding method that selects any one of a plurality of reference sample lines is used, the wide-angle intra prediction mode is used It can be set not to. That is, even when the current block is non-square and the intra-prediction mode of the current block belongs to the transformation range, the intra-prediction mode of the current block may not be converted to the wide-angle intra-prediction mode. An example in which the wide-angle intra prediction mode has different angles according to sample lines is shown.

When the adjacent reference sample line is determined as the reference sample line of the current block, the maximum angle of the wide-angle intra prediction mode may be set to α. On the other hand, when the non-adjacent reference sample line is determined as the reference sample line of the current block, the maximum angle of the wide-angle intra prediction mode may be set to β.

Depending on the index of the non-adjacent reference sample line, the angle of the wide-angle intra prediction mode may be different. In this case, as the reference sample line index of the current block increases, the angle of the wide-angle intra prediction mode may increase or decrease.

As wide-angle intra-prediction modes are used in addition to the existing intra-prediction modes, resources required for encoding the wide-angle intra-prediction modes increase, and encoding efficiency may decrease. Accordingly, instead of encoding the wide-angle intra-prediction modes as they are, the encoding efficiency can be improved by encoding alternative intra-prediction modes for the wide-angle intra prediction modes.

As an example, when the current block is encoded using the wide-angle intra prediction mode of No. 67, No. 2, which is the wide-angle alternative intra prediction mode of No. 67, may be encoded as the intra prediction mode of the current block. In addition, when the current block is encoded with the wide-angle intra prediction mode of -1, the wide-angle alternative intra prediction mode of -1, No. 66, may be encoded as the intra prediction mode of the current block.

The decoder may decode the intra prediction mode of the current block and determine whether the decoded intra prediction mode is included in a transform range. When the decoded intra prediction mode is the wide-angle replacement intra prediction mode, the intra prediction mode may be converted into the wide-angle intra prediction mode.

Alternatively, when the current block is encoded in the wide-angle intra prediction mode, the wide-angle intra prediction mode may be encoded as it is.

Encoding of the intra prediction mode may be performed based on the above-described MPM list. Specifically, when the neighboring block is encoded in the wide-angle intra prediction mode, the MPM may be set based on the wide-angle alternative intra prediction mode corresponding to the wide-angle intra prediction mode.

A residual image derived by differentiating the prediction image from the original image may be derived. In this case, when the residual image is changed to the frequency domain, the subjective image quality of the image is not significantly deteriorated even if the high-frequency components among the frequency components are removed. Accordingly, if the values of the high-frequency components are reduced or the values of the high-frequency components are set to 0, there is an effect of increasing the compression efficiency without significantly causing visual distortion. By reflecting the above characteristics, the current block may be transformed in order to decompose the residual image into 2D frequency components. The transformation may be performed using a transformation technique such as Discrete Cosine Transform (DCT) or Discrete Sine Transform (DST).

DCT decomposes (or transforms) the residual image into 2D frequency components by using cosine transform, and DST uses sine transform to decompose (or transforms) the residual image into 2D frequency components. As a result of transforming the residual image, frequency components may be expressed as a base image. For example, when DCT transformation is performed on a block of size NxN, N ² basic pattern components may be obtained. The size of each of the basic pattern components included in the NxN size block may be obtained through transformation. According to the used transformation technique, the size of the basic pattern component may be referred to as a DCT coefficient or a DST coefficient.

Transformation technique DCT is mainly used to transform an image in which a large number of non-zero low-frequency components are distributed. The transformation technique DST is mainly used for images in which high-frequency components are widely distributed.

The residual image may be transformed using a transform technique other than DCT or DST.

Hereinafter, transforming the residual image into two-dimensional frequency components will be referred to as a two-dimensional image transformation. In addition, the magnitude of the basic pattern components obtained as a result of the transformation will be referred to as a transformation coefficient. As an example, the transform coefficient may mean a DCT coefficient or a DST coefficient. When both a first transform and a second transform, which will be described later, are applied, the transform coefficient may mean a size of a basic pattern component generated as a result of the second transform.

The transformation scheme may be determined in units of blocks. The transformation technique may be determined based on at least one of a prediction encoding mode of the current block, the size of the current block, and the size of the current block. As an example, when the current block is encoded in the intra prediction mode and the size of the current block is smaller than NxN, transformation may be performed using the transform technique DST. On the other hand, if the above condition is not satisfied, the transformation may be performed using the transformation technique DCT.

2D image transformation may not be performed on some blocks of the residual image. A process in which 2D image transformation is not performed may be referred to as a transform skip. When transform skip is applied, quantization may be applied to residual values to which transform is not performed.

After transforming the current block using DCT or DST, the transformed current block may be transformed again. In this case, a transform based on DCT or DST may be defined as a first transform, and re-transformation of a block to which the first transform is applied may be defined as a second transform.

The first transform may be performed using any one of a plurality of transform core candidates. For example, the first conversion may be performed using any one of DCT2, DCT8, and DCT7.

Different transformation cores may be used for horizontal and vertical directions. Information indicating the combination of the horizontal direction conversion core and the vertical direction conversion core may be signaled through the bitstream.

The units of performing the first transformation and the second transformation may be different. As an example, a first transform may be performed on an 8x8 block, and a second transform may be performed on a subblock having a size of 4x4 among the transformed 8x8 blocks. In this case, the transform coefficients of the residual regions in which the second transform is not performed may be set to 0.

Alternatively, a first transform may be performed on a 4x4 block and a second transform may be performed on an 8x8 region including the transformed 4x4 block.

Information indicating whether the second transformation is performed may be signaled through a bitstream.

Information indicating the transformation type of the current block may be signaled through a bitstream. The information may be index information tu_mts_idx indicating one of combinations of a transform type for a horizontal direction and a transform type for a vertical direction.

Based on the transform type candidates specified by the index information tu_mts_idx, a transform core for a vertical direction and a transform core for a horizontal direction may be determined. Table 11 and Table 12 show transform type combinations according to tu_mts_idx.

tu_mts_idx transform type horizontal vertical 0 SKIP SKIP One DCT-II DCT-II 2 DST-VII DST-VII 3 DCT-VIII DST-VII 4 DST-VII DCT-VIII 5 DCT-VIII DCT-VIII

tu_mts_idx transform type horizontal vertical 0 DCT-II DCT-II One SKIP SKIP 2 DST-VII DST-VII 3 DCT-VIII DST-VII 4 DST-VII DCT-VIII 5 DCT-VIII DCT-VIII

The transform type may be determined as any one of DCT2, DST7, DCT8, and transform skip. Alternatively, a transform type combination candidate may be constructed using only transform cores, excluding transform skip. When Table 11 is used, if tu_mts_idx is 0, transform skip may be applied in the horizontal direction and the vertical direction. If tu_mts_idx is 1, DCT2 may be applied to the horizontal and vertical directions. If tu_mts_idx is 3, DCT8 may be applied to the horizontal direction and DCT7 may be applied to the vertical direction.

When Table 12 is used, if tu_mts_idx is 0, DCT2 may be applied to the horizontal direction and the vertical direction. When tu_mts_idx is 1, transform skip may be applied in the horizontal direction and the vertical direction. If tu_mts_idx is 3, DCT8 may be applied to the horizontal direction and DCT7 may be applied to the vertical direction.

Whether to encode the index information may be determined based on at least one of the size, shape, and number of non-zero coefficients of the current block. For example, when the number of non-zero coefficients is equal to or smaller than the threshold, the default transform type may be applied to the current block without signaling index information. Here, the default conversion type may be DST7. Alternatively, the default mode may be different according to the size, shape, or intra prediction mode of the current block.

The threshold value may be determined based on the size or shape of the current block. As an example, when the size of the current block is less than or equal to 32x32, the threshold is set to 2, and when the current block is larger than 32x32 (eg, when the current block is a coding block having a size of 32x64 or 64x32), the threshold You can set the value to 4.

Information indicating whether to explicitly determine the transformation type of the current block based on information signaled from the bitstream may be signaled through the bitstream. As an example, at the sequence level, information sps_explicit_intra_mts_flag indicating whether explicit transformation type determination is allowed for a block encoded by intra prediction and/or information indicating whether explicit transformation type determination is allowed for a block encoded by inter prediction Information sps_explicit_inter_mts_flag may be signaled.

When explicit transform type determination is allowed, the transform type of the current block may be determined based on index information tu_mts_idx signaled from the bitstream. On the other hand, when explicit transformation type determination is not allowed, the transformation type is based on at least one of the size and shape of the current block, whether transformation in sub-block units is allowed, or the position of a sub-block including a non-zero transformation coefficient. This can be determined. For example, the horizontal transformation type of the current block may be determined based on the width of the current block, and the vertical transformation type of the current block may be determined based on the height of the current block. For example, when the width of the current block is less than 4 or greater than 16, the horizontal transformation type may be determined to be DCT2. Otherwise, the transform type in the horizontal direction may be determined to be DST7. When the height of the current block is less than 4 or greater than 16, the transformation type in the vertical direction may be determined to be DCT2. Otherwise, the transformation type in the vertical direction may be determined as DST7. Here, in order to determine the transform type in the horizontal direction and the transform type in the vertical direction, a threshold value compared to a width and a height may be determined based on at least one of a size, a shape, and an intra prediction mode of the current block.

Alternatively, when the current block is a square with the same height and width, the horizontal transformation type and the vertical transformation type are set to be the same, while when the current block is a non-square type with different heights and widths, the horizontal transformation type and the vertical direction The conversion type can be set differently. For example, when the width of the current block is greater than the height, the horizontal transformation type may be determined as DST7, and the vertical transformation type may be determined as DCT2. When the height of the current block is greater than the width, the transformation type in the vertical direction may be determined as DST7 and the transformation type in the horizontal direction may be determined as DCT2.

The number and/or type of transform type candidates or the number and/or type of transform type combination candidates may be different depending on whether explicit transform type determination is allowed. As an example, when explicit transform type determination is allowed, DCT2, DST7, and DCT8 may be used as transform type candidates. Accordingly, each of the horizontal direction transformation type and the vertical direction transformation type may be set to DCT2, DST8, or DCT8. When explicit transform type determination is not allowed, only DCT2 and DST7 may be used as transform type candidates. Accordingly, each of the horizontal direction transformation type and the vertical direction transformation type may be determined as DCT2 or DST7.

If the encoder performs transform and quantization, the decoder may obtain a residual block through inverse quantization and inverse transform. The decoder may obtain a reconstructed block for the current block by adding the prediction block and the residual block.

When the reconstructed block of the current block is obtained, loss of information occurring in the quantization and encoding process can be reduced through in-loop filtering. The in-loop filter may include at least one of a deblocking filter, a sample adaptive offset filter (SAO), and an adaptive loop filter (ALF).

Based on at least one of whether the block is encoded in the intra prediction mode, whether the difference between the absolute motion vector values of the neighboring blocks is greater than a predefined threshold, and whether the reference pictures of the neighboring blocks are the same Thus, a block filter strength (hereinafter referred to as BS) value may be determined. If the BS value is 0, it indicates that filtering is not performed.

When the diagonal prediction encoding method is used, the BS value may be set to 1.

When the diagonal prediction encoding method is used, it may be set to use a plurality of BS values in a block.

For example, in the diagonal prediction encoding method, the BS value may be set to 2 at the boundary of a partition in which the intra mode is used, and the BS value may be set to 0 or 1 in the boundary of the partition in which the inter prediction mode is used.

Fig. 20

Specifically, for example, when the left triangular prediction block is inter prediction and the right triangular prediction block is intra prediction as shown in FIG. 20, the BS value is set to 2 at the upper boundary and the right boundary of the coding block belonging to the right triangular prediction block. may be set, and the BS value may be set to 0 or 1 at the left boundary and the lower boundary of the coding block belonging to the left triangular prediction block.

When the thin diagonal prediction encoding method is used, it may be set to use a plurality of BS values in a block. For example, in the thin diagonal prediction encoding method, the BS value may be set to 2 at the boundary of a partition in which the intra mode is used, and the BS value may be set to 0 or 1 in the boundary of the partition in which the inter prediction mode is used.

It is included in the scope of the present invention to apply the decoding process or the embodiments described based on the encoding process to the encoding process or the decoding process. It is also within the scope of the present invention to change the embodiments described in a certain order in an order different from that described.

Although the above-described embodiment has been described based on a series of steps or a flowchart, this does not limit the time-series order of the invention, and may be performed simultaneously or in a different order, if necessary. In addition, each of the components (eg, unit, module, etc.) constituting the block diagram in the above-described embodiment may be implemented as a hardware device or software, or a plurality of components may be combined to form one hardware device or software. may be implemented. The above-described embodiment may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Examples of the computer-readable recording medium include a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as a CD-ROM, a DVD, and a magneto-optical medium such as a floppy disk. media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

2 Video encoding method using tiles and bricks

High-resolution images are used in immersive media applications such as panoramic video or 360-degree video or 4K/8K Ultra High Definition (UHD). Parallelization is required for real-time or low-delay encoding of high-resolution images. A predetermined region of a picture may be independently encoded and/or decoded using a tile.

A block in a tile may be set not to encode/decode an image by referencing data in another tile or using it as a reference sample. For example, do not use data in other tiles as reference samples in intra prediction, do not use data in other tiles as candidates (eg, merge candidates, AMVP candidates, or HVMP candidates) for motion vector derivation, or other It is possible to disable the data in the tile from being used in calculating the context of the symbol.

The probability table of the CABAC (Context Adaptive Binary Arithmetric Coding) context can be initialized for each tile. It is possible to disable the loop filter at the tile boundary (the part of the boundary between two neighboring tiles). Alternatively, information indicating whether a loop filter is used in a tile boundary may be encoded.

Fig. 21

A picture may consist of one or at least one or more slices. The slice may be a raster-scan slice or a rectangular slice such as a raster-scan slice. A raster scan slice is defined as a group of one or more contiguous tile(s) when following a raster scan order. The left figure of FIG. 21 shows a raster scan slice. The raster scan slice may have a non-rectangular shape. A rectangular slice is defined as a group of tiles having a rectangular shape. The right figure of FIG. 19 shows a rectangular slice.

A boundary of a slice may coincide with a picture boundary and/or a tile boundary. For example, the boundary of the slice may be set such that the left boundary and the upper boundary are located at the picture boundary, or the boundary of the slice may be set so that the boundary of the already decoded tile is located.

A slice may consist of one or a plurality of tiles.

The tile may have a rectangular shape. Alternatively, non-rectangular tiles may be accepted. Information indicating whether a non-rectangular tile exists may be encoded. The boundary of the tile may coincide with the picture boundary and/or the CTU boundary.

Information indicating whether a plurality of tiles exist in a picture may be signaled through a bitstream. When a plurality of tiles exist in a picture, as shown in FIG. 22, tiles adjacent to the left or right may have the same height (hereinafter referred to as a horizontal tile set/tile row), and the widths of the tiles adjacent to the top or the bottom may be It may be configured to be the same (hereinafter referred to as a vertical tile set/tile column). Information indicating whether the height is the same as the previous tile or information indicating whether the width is the same as the previous tile may be signaled through the bitstream.

Alternatively, all tiles except tiles adjacent to the picture boundary in the picture may be restricted to have the same size.

Fig. 22

The width of a tile column can be signaled in all tile columns or some tile columns. As shown in Table 13, the number of tile columns that signal the width of the tile column (hereinafter, the number of exp tile columns) can be signaled.

num_exp_tile_coulmns_minus1+1 indicates the number of exp tile columns.

That is, the syntax tile_column_width_minus1[ i ] indicating the width of the tile column as much as the number of exp tile columns can be signaled.

The width of the i-th tile column can be expressed as tile_column_width_minus1[ i ] + 1, and the default grid can be set as CTB.

The width of the k-th tile column (k> num_exp_tile_columns_minus1) may be set to a smaller value between tile_column_width_minus1[num_exp_tile_columns_minus1] + 1 and remainingWidthInCtbY.

Here, it can be defined as PicWidthInCtbsY = Ceil(pic_width_in_luma_samples / CtbSizeY), and can be defined as the number of CTB columns in the picture.

The sum of the widths of the tile columns before the k-th tile column is defined as the cumulative tile column width.

remainingWidthInCtbY represents a difference value of the accumulated tile column width in PicWidthInCtbsY, and can be derived as follows.

The height of the tile row can be signaled in all tile rows or in some tile rows. As shown in Table 1, the number of tile rows that signal the height of the tile row (hereinafter, the number of exp tile rows) can be signaled.

num_exp_tile_rows_minus1+1 indicates the number of exp tile rows.

That is, the syntax tile_row_height_minus1[i] indicating the width of the tile row as much as the number of exp tile rows can be signaled.

The width of the i-th tile row can be expressed as tile_row_height_minus1 [i] + 1, and the default grid can be set as CTB.

The height of the k-th tile row (k> num_exp_tile_rows_minus1) may be set to a smaller value between tile_row_height_minus1[ num_exp_tile_rowss_minus1 ] + 1 and remainingHeightInCtbY.

Here, it can be defined as PicHeightInCtbsY = Ceil(pic_height_in_luma_samples / CtbSizeY), and it can be defined as the number of CTB rows in the picture.

The sum of the widths of the tile rows before the k-th tile row is defined as the cumulative tile row height.

remainingHeightInCtbY represents a difference value of the cumulative tile row height in PicHeightInCtbsY, and can be derived as follows.

If the num_exp_tile_columns_minus1 and num_exp_tile_rows_minus1 values are 0, if the tile_column_width_minus1[ i ] value is PicWidthInCtbsY　-　1, the tile_row_height_minus1[ i ] value can be set so that it does not become PicHeightIn-CtbsY1.

loop_filter_across_tiles_enabled_flag is a flag indicating whether in-loop filtering (deblocking, ALF, SAO, etc.) is performed at a tile boundary of a picture referring to the corresponding PPS. When the loop_filter_across_tiles_enabled_flag value is 1, it indicates that in-loop filtering is performed at the tile boundary of the picture referring to the corresponding PPS. .

loop_filter_across_slices_enabled_flag is a flag indicating whether in-loop filtering (deblocking, ALF, SAO, etc.) is performed at a slice boundary of a picture referring to the corresponding PPS. If the loop_filter_across_slices_enabled_flag value is 1, it indicates that in-loop filtering is performed at the tile boundary of the picture referring to the corresponding PPS. .

In the case of a rectangular slice, a syntax num_slices_in_pic_minus1 indicating the number of slices in a picture may be signaled.

Fig. 23

A slice may consist of a plurality of tiles. In this case, as shown in Table 13, the syntax slice_height_in_tiles_minus1[ i ] indicating the height of the i-th slice and the syntax slice_width_in_tiles_minus1[ i ] indicating the width of the i-th slice may be signaled.

slice_height_in_tiles_minus1[ i ]+1 indicates the height of the i-th slice (SliceHeight[i]), and the basic unit indicating the height of the slice is a tile row.

For example, the height of slice 4 (the fourth slice) of FIG. 23 may be 2, and slice_height_in_tiles_minus1[ 4 ] may be set to 1.

slice_width_in_tiles_minus1[ i ]+1 indicates the width of the i-th slice, and the basic unit indicating the width of the slice is the tile column.

slice_width_in_tiles_minus1[ i ] When the value is 0, it indicates that the tile is one slice or a plurality of slices.

When the value of slice_width_in_tiles_minus1[i] is 0, a syntax num_slices_in_tile_minus1[i] indicating the number of slices in a tile may be signaled.

In the case of having a plurality of slices in a tile, a syntax slice_height_in_ctu_minus1[i][j] indicating the height of the slice may be signaled. slice_height_in_ctu_minus1[i][j]+1 indicates the height of the j-th slice in the i-th tile, and the basic unit indicating the slice height may be set to a CTU row.

A syntax tile_idx_delta[ i ] indicating a difference between the tile index (upper-left tile index) of the i+1-th slice and the tile index (upper-left tile index) of the i-th slice (hereinafter, slice difference) may be signaled.

As shown in Table 14, a flag slice_width_in_tiles_minus1[i] and slice_height_in_tiles_minus1[i] indicating whether both values are 0 or at least one of which is not 0 may be signaled slice_width_height_present_flag[i].

If the slice_width_height_present_flag[i] value is 0, it indicates that both slice_width_in_tiles_minus1[i] and slice_height_in_tiles_minus1[i] values are 0, and indicates that slice_width_in_tiles_minus1[i] and slice_height_in_tiles_minus1[i] are not signaled.

If the value of slice_width_height_present_flag[i] is 1, it indicates that at least one of the values of slice_width_in_tiles_minus1[i] and slice_height_in_tiles_minus1[i] is not 0.

As shown in Table 15, the syntax uniform_slice_width_height_flag indicating whether the syntax slice_width_in_tiles_minus1[i] indicating the width of the rectangular slice in the picture and the syntax slice_height_in_tiles_minus1[i] indicating the height are the same may be signaled.

If the uniform_slice_width_height_flag value is 1, the value of the syntax slice_width_in_tiles_minus1[i] indicating the width of the slice is the same as the value of slice_width_in_tiles_minus1[0], and the value of the syntax slice_height_in_tiles_minus1[i] indicating the height of the slice is the same as the value of slice_height_in_in_tiles.

If the uniform_slice_width_height_flag value is 0, there is a slice whose syntax slice_width_in_tiles_minus1[i] value is different from the slice_width_in_tiles_minus1[0] value, or the syntax slice_height_in_tiles_minus1[i] value that is different from the slice height syntax slice_height_in_tiles_minus1[i] indicates the existence of

If it is in the rightmost tile column (last tile column) of the i-th slice, the syntax slice_width_in_tiles_minus1[i] does not signal, and its value can be derived to 0.

As shown in Table 16, if the i-th slice is in the lowest tile row (last tile row), the syntax slice_height_in_tiles_minus1[i] does not signal, and the value can be derived to 0.

If the i-th slice is in the Last tile column and the Last tile row at the same time (ie, the last tile in the picture), slice_width_in_tiles_minus1[i] and slice_width_in_tiles_minus1[i] are not signaled, and their values can be derived to 0, respectively.

If the remaining value is NumTileColumns -1 when the upper-left tile index of the i-th slice is divided by the number of tile columns, it can be determined that the i-th slice is in the last tile column.

If the quotient is NumTileRows -1 when the upper-left tile index of the i-th slice is divided by the number of tile columns, it can be determined that the i-th slice is in the last tile row.

Fig. 24

When the number of slices in the picture is two (num_slices_in_pic_minus1 value is 1), the picture is horizontally divided as shown in the left figure of FIG. As shown in the figure, by dividing the picture vertically, two rectangular slices (hereafter, vertical bisection slices) can be composed.

In the case of a horizontal bipartite slice, since the tileIdx of slice 1 (second slice) can be derived using slice_width_in_tiles_minus1[0] and slice_height_in_tiles_minus1[0], tile_idx_delta[i] can be omitted.

In the case of a vertically bipartite slice, since tileIdx of slice 1 (second slice) can be derived using slice_width_in_tiles_minus1[0], tile_idx_delta[i] can be omitted.

That is, a horizontally divided slice and/or a vertically divided slice may derive tileIdx without signaling tile_idx_delta[i].

As shown in Table 17, when the num_slices_in_pic_minus1 value is 0 or 1, the tile_idx_delta_present_flag value may not be signaled and the value may be set to 0.

Fig. 25

When the number of tile columns in a picture (hereinafter, NumTileColumns) is 1, a tile_idx_delta or tileIdx value may be derived from information indicating the width of the slice and/or information indicating the height of the slice.

For example, when the NumTileColumn value is 1 as shown in FIG. 25 , the tileIdx of the upper left tile of the i-th slice can be derived as follows.

tileIdx += 1+ slice_height_in_tiles_minus1[ i ]

As an example, when NumTileColumns is 1, a tile_idx_delta value or tileIdx can be derived from the syntax slice_width_in_tiles_minus1 and/or slice_height_in_tiles_minus1 value.

Fig. 26

When the number of tile rows in a picture (hereinafter, NumTileRows) is 1, a tile_idx_delta value or tileIdx may be derived from information indicating the width of a slice and/or information indicating a slice height.

For example, as shown in FIG. 26 , when the NumTileRows value is 1, the tileIdx of the upper left tile of the i-th slice may be derived as follows.

tileIdx += 1+ slice_width_in_tiles_minus1[ i ]

For example, when NumTileRows is 1, a tile_idx_delta value or tileIdx can be derived from the syntax slice_width_in_tile_minus1 and/or slice_height_in_tiles_minus1 value.

As described above, when the NumTileColumn value is greater than 1 or the NumTileRows value is greater than 1, the tile_idx_delta_present_flag value may be signaled.

When a tile consists of one or more slices (hereafter, multi-slice tile), the height of each slice is set to be the same, or only the height of the last slice (hereafter, lastSliceInTile) in the multi-slice tile is different, and all other slice heights are different. You can also set them to be the same. In this case, as shown in Table 19, the syntax num_slices_in_tile_minus1[i] indicating the number of slices in the multi-slice tile and the syntax slice_height_in_ctus_minus1[i] indicating the height of the first slice (hereinafter, firstSliceInTile) in the multi-slice tile may be signaled.

Among the slices in the multi-slice tile, if it is not firstSliceInTile or lastSliceInTile, it may be defined as middleSliceInTile.

The height of middleSliceInTile may be derived from the height of firstSliceInTile. The sum of the height of firstSliceInTile and the height of all middleSliceInTiles can be defined as the cumulative slice height.

The height of lastSliceInTile can be derived as a differential value from the height of the tile row to the height of the tile row as shown in the following derivation equation.

A tile in a picture may signal a syntax single_slice_per_tile_flag indicating that it is composed of one slice.

When the value of single_slice_per_tile_flag is 1, it indicates that the tile in the picture consists of one slice. That is, it indicates that one tile is composed of one slice.

When the single_slice_per_tile_flag value is 0, it indicates that the tile in the picture is not composed of one slice. That is, it is a multi-slice tile or indicates that a slice may be composed of several tiles.

When the single_slice_per_subpic_flag value is 0, the syntax single_slice_per_tile_flag may be signaled.

When the single_slice_per_tile_flag value is 1, the syntax num_slices_in_pic_minus1, tile_idx_delta_present_flag, slice_width_in_tiles_minus1 [i] that specifies a location in the form of slice size and the slices, slice_height_in_tiles_minus1 [i], num_ slices_in_tile_minus1 signal a [i], exp_slice_height_in_ctus_minus1 [j], tile_idx_delta [i] It is also possible to derive its value without ringing.

If the tile_idx_delta_present_flag value is 1 and the number of rectnagular slices in the picture is 2 (num_slice_in_pic_minus1 value is 1), as shown in Table 21, the syntax indicating the width of the i-th rectangular slice slice_width_in_tiles_minus1[ i ] and/or the height of the i-th rectangular slice The syntax slice_height_in_tiles_minus1[i] indicating the may not be signaled. If the value of tile_idx_delta[i] is smaller than NumTileColumns, the value of slice_width_in_tiles_minus1[ i ] may be set to tile_idx_delta[i]-1, and the value of slice_height_in_tiles_minus1[ i ] may be set to NumTileRows-1.

If the value of tile_idx_delta[i] is equal to or greater than NumTileColumns, the value of slice_width_in_tiles_in_minus1[i] can be set to NumTileColumn-1, and the value of slice_height_in_tiles_minus1[i] can be set to tile_idx_delta[i]/NumTileColumn.

That is, when the tile_idx_delta_present_flag value is 0 or the num_slices_in_pic_minus1 value is not 1, slice_width_in_tiles_minus1[ i ] and/or slice_height_in_tiles_minus1[ i ] may be signaled.

If the number of rectnagular slices in the picture is 2, the tile cannot be divided into a plurality of rectnagular slices. In this case, if a tile is divided into a plurality of rectnagular slices, it must consist of one rectnagular slice smaller than the tile and a rectnagular slice that spans two tiles. Such partitioning is not allowed.

When the number of rectangular slices in a picture is 2 and the number of tiles in a picture is greater than 1, as shown in Table 22, the syntax exp_slice_height_in_ctus_minus1[ j ] indicating the height of the slice in the tile and the syntax num_exp_slices_in_tile[ i ] indicating the number of slices in the tile may not be signaled.

When the number of tiles in a picture (NumTilesInPic) is 2 or less, the syntax tile_idx_delta[i] may not be signaled and its value may be set to 0 or 1.

That is, as shown in Table 23, when the number of tiles in a picture is two or less, the tile_idx_delta_present_flag may not be signaled and its value may be set to 0.

When a plurality of slices exist in a tile and the tile row height of the tile including the i-th rectangular slice is 2, the syntax num_exp_slice_in_tile[i] may not be signaled, and the value may be set to 1.

As shown in Table 24, the syntax num_exp_slice_in_tile[i] can be signaled only when the variable RowHeight[ SliceTopLeftTileIdx[ i ] / NumTileColumns ] representing the tile row height of the tile including the i-th rectangular slice is 2 or more.

If a plurality of slices exist in a tile and the tile row height of the tile including the i-th rectangular slice is 2, the syntax exp_slice_height_in_ctus_minus1[ j ] indicating the height of the slice is not signaled, and its value can be set to 1. have.

As shown in Table 25, the syntax exp_slice_height_in_ctus_minus1[ j ] can be signaled only when the variable RowHeight[ SliceTopLeftTileIdx[ i ] / NumTileColumns ] representing the tile row height of the tile including the i-th rectangular slice is 2 or more.

The syntax exp_slice_height_in_ctus_minus1[ i ][ j ] can be set to a value between 0 and RowHeight[ SliceTopLeftTileIdx[ i ] / NumTileColumns ] - 2.

As shown in Table 26, the syntax uniform_slice_width _flag indicating whether all values of the syntax slice_width_in_tiles_minus1[i] indicating the width of the rectangular slice in the picture are the same may be signaled.

If the uniform_slice_width_flag value is 1, it indicates that the value of the syntax slice_width_in_tiles_minus1[i] indicating the width of the slice is the same as the value of slice_width_in_tiles_minus1[0].

When the uniform_slice_width_flag value is 0, it indicates that there may be a case different from the slice_height_in_tiles_minus1[0] value among the syntax slice_height_in_tiles_minus1[i] values indicating the width of the slice.

A syntax uniform_slice_height_flag indicating whether the syntax slice_height_in_tiles_minus1[i] indicating the height of the rectangular slice within the picture is the same may be signaled.

If the uniform_slice_height_flag value is 1, it indicates that the value of the syntax slice_width_in_tiles_minus1[i] indicating the height of the slice is the same as the value of slice_width_in_tiles_minus1[0].

When the uniform_slice_height_flag value is 0, it indicates that there may be a case different from the slice_height_in_tiles_minus1[0] value among the syntax slice_height_in_tiles_minus1[i] values indicating the height of the slice.

Claims (1)

A method for encoding and decoding tile and slice information.