KR102511546B1 - Method and apparatus for processing a video signal - Google Patents

Abstract

The present invention relates to a video signal processing method and apparatus. An image decoding method according to the present invention includes: identifying a directional intra prediction mode of a current coding block; determining an interpolation filter type for directional intra prediction sample interpolation applied to the directional intra prediction mode according to a shape of the current coding block; and generating intra prediction samples by applying the determined interpolation filter. As intra prediction samples can be efficiently interpolated according to the present invention, encoding/decoding efficiency of a video signal can be increased.

Description

Video signal processing method and apparatus {METHOD AND APPARATUS FOR PROCESSING A VIDEO SIGNAL}

The present invention relates to a video signal processing method and apparatus.

Recently, demand for high-resolution and high-quality images such as high definition (HD) images and ultra high definition (UHD) images is increasing in various application fields. As image data becomes higher resolution and higher quality, the amount of data increases relatively compared to existing image data. Therefore, when image data is transmitted using a medium such as an existing wired/wireless broadband line or stored using an existing storage medium, transmission cost and Storage costs increase. High-efficiency video compression technologies can be used to solve these problems that occur as video data becomes high-resolution and high-quality.

An inter-prediction technique for predicting pixel values included in the current picture from pictures before or after the current picture as an image compression technique, an intra-prediction technique for predicting pixel values included in the current picture using pixel information within the current picture, There are various technologies such as entropy coding technology that assigns short codes to values with high frequency of occurrence and long codes to values with low frequency of occurrence. Such image compression technology can be used to effectively compress and transmit or store image data.

On the other hand, along with the increase in demand for high-resolution images, the demand for stereoscopic image contents as a new image service is also increasing. A video compression technique for effectively providing high-resolution and ultra-high-resolution 3D video contents is being discussed.

An object of the present invention is to provide a multi-tree partitioning method and apparatus capable of effectively dividing an encoding/decoding target block in encoding/decoding a video signal.

An object of the present invention is to provide a multi-tree partitioning method and apparatus for dividing an encoding/decoding target block into symmetrical or asymmetrical blocks when encoding/decoding a video signal.

An object of the present invention is to provide a method and apparatus for generating interpolated intra prediction samples corresponding to coding blocks partitioned by multi-tree partitioning.

An object of the present invention is to provide a recording medium including a video signal bitstream encoded by the above encoding method.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.

An image decoding method according to the present invention comprises: identifying a directional intra prediction mode of a current coding block; determining an interpolation filter type for directional intra prediction sample interpolation applied to the directional intra prediction mode according to a shape of a current coding block; and generating intra prediction samples by applying the determined interpolation filter.

In addition, interpolation filters having different filter taps are applied according to whether the shape of the current coding block is non-square or square.

In addition, when the shape of the current coding block is non-square, an interpolation filter having a smaller filter tap than the case of a square shape is applied.

Also, interpolation filters having different filter taps are applied according to the width or height of the current coding block. Here, if at least one of the width or height of the current coding block is smaller than the reference value, an interpolation filter having a small filter tap may be applied.

In addition, when the shape of the current coding block is non-square and at least one of width or height is smaller than the reference size, an interpolation filter having a filter tap smaller than that of the case where the current coding block is larger than the reference size is applied.

In addition, when the width-to-height ratio (w/h) of the current coding block is smaller than the threshold value, an interpolation filter having a smaller filter tap than the case where the width-to-height ratio (w/h) is larger than the threshold value is applied.

In addition, interpolation filters having different filter taps are applied according to the directional intra prediction mode of the current coding block.

In addition, an interpolation filter having a different filter tap is applied depending on whether the intra prediction mode is a horizontal mode or a vertical mode.

Further, when the type of the interpolation filter is determined, an interpolation filtering step of selectively performing any one of a vertical direction, a horizontal direction, and a vertical/horizontal direction is further included.

Also, depending on the shape of the current coding block, an interpolation filter having the same number of taps but different filter coefficients may be differently applied.

Also, depending on the shape of the current coding block, interpolation filters having the same number of taps but different filter strengths may be differently applied.

An image encoding method according to the present invention comprises: identifying a directional intra prediction mode of a current coding block; determining an interpolation filter type for directional intra prediction sample interpolation applied to the directional intra prediction mode according to a shape of a current coding block; and generating intra prediction samples by applying the determined interpolation filter.

An image decoding apparatus according to the present invention identifies a directional intra prediction mode of a current coding block, determines an interpolation filter type for directional intra prediction sample interpolation applied to the directional intra prediction mode according to the shape of the current coding block, and , and a decoder generating intra prediction samples by applying the determined interpolation filter.

According to the present invention, in a recording medium including a video signal bitstream, the video signal bitstream included in the recording medium includes the step of checking a directional intra prediction mode of a current coding block, according to the shape of the current coding block, Determining an interpolation filter type for directional intra prediction sample interpolation applied to the directional intra prediction mode, and generating intra prediction samples by applying the determined interpolation filter Characterized in that the encoding is performed by an image encoding method. do.

The features briefly summarized above with respect to the present invention are only exemplary aspects of the detailed description of the present invention that follows, and do not limit the scope of the present invention.

According to the present invention, encoding/decoding efficiency of a video signal can be increased by efficiently dividing encoding/decoding target blocks.

According to the present invention, encoding/decoding efficiency of a video signal can be increased by dividing a block to be encoded/decoded into symmetrical or asymmetrical blocks.

According to the present invention, encoding/decoding efficiency of a video signal can be increased by applying an intra prediction sample interpolation filter according to the shape and/or size of a current block.

The effects obtainable in the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

1 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present invention.
2 is a block diagram illustrating a video decoding apparatus according to an embodiment of the present invention.
3 is a diagram illustrating a partition mode applicable to a coding block when the coding block is coded by inter-prediction.
4 is a diagram illustrating a partition type in which quad tree and binary tree partitioning is allowed as an embodiment to which the present invention is applied.
5 illustrates an example of hierarchically splitting a coding block based on quad tree and binary tree splits as an embodiment to which the present invention is applied.
6 illustrates an example of hierarchically partitioning a coding block based on quad tree and symmetric binary tree partitioning as an embodiment to which the present invention is applied.
7 is a diagram illustrating a partition type in which asymmetric binary tree partitioning is allowed as an embodiment to which the present invention is applied.
8 illustrates a division form of a coding block based on quad tree and symmetric/asymmetric binary tree division as an embodiment to which the present invention is applied.
9 is a flowchart of a coding block partitioning method based on quad tree and binary tree partitioning as an embodiment to which the present invention is applied.
10 illustrates, for example, syntax elements included in a network abstraction layer (NAL) to which quad tree and binary tree partitioning are applied as an embodiment to which the present invention is applied.
11 is another embodiment to which the present invention is applied, and is a diagram showing a partition type in which asymmetric quad tree partitioning is allowed.
12 is a flowchart of a coding block partitioning method based on asymmetric quad tree partitioning as another embodiment to which the present invention is applied.
13 illustrates, for example, syntax elements included in a network abstraction layer (NAL) to which asymmetric quad tree partitioning is applied as another embodiment to which the present invention is applied.
14 is a diagram showing a partition type in which quad-tree and triple-tree partitions are allowed as another embodiment to which the present invention is applied.
15 is a flowchart of a coding block partitioning method based on quad-tree and triple-tree partitioning as another embodiment to which the present invention is applied.
FIG. 16 illustrates, for example, syntax elements included in a network abstraction layer (NAL) to which quad-tree and triple-tree partitioning are applied as another embodiment to which the present invention is applied.
17 is another embodiment to which the present invention is applied, and is a diagram illustrating a basic partition type in which multi-tree partitioning is allowed.
FIG. 18 is another embodiment to which the present invention is applied, and is a diagram showing an extended partition type in which multi-tree partitioning is allowed.
19 is a flowchart of a coding block partitioning method based on multi-tree partitioning as another embodiment to which the present invention is applied.
20 illustrates types of pre-defined intra prediction modes in an image encoder/decoder as an embodiment to which the present invention is applied.
21 illustrates types of intra prediction modes extended to an image encoder/decoder as an embodiment to which the present invention is applied.
22 is a flowchart schematically illustrating an intra prediction method as an embodiment to which the present invention is applied.
23 illustrates a method of correcting a predicted sample of a current block based on difference information of neighboring samples as an embodiment to which the present invention is applied.
24 and 25 show a method of correcting a predicted sample based on a predetermined correction filter as an embodiment to which the present invention is applied.
FIG. 26 is a table showing intra direction parameters (intraPredAng) from Mode 2 to Mode 34, which are directional intra prediction modes shown in FIG. 20 .
27 and 28 are diagrams illustrating a one-dimensional reference sample group in which reference samples are rearranged in a row according to the present invention.
29 is an embodiment to which the present invention is applied, and is a flowchart illustrating applying different interpolation filters according to types of coding units.
30 to 32 are exemplary flowcharts for applying different interpolation filters according to types of coding units as embodiments to which the present invention is applied.
FIG. 33 illustrates a syntax element included in a network abstraction layer (NAL) applied to intra prediction sample interpolation as another embodiment to which the present invention is applied, as an example.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. Like reference numerals have been used for like elements throughout the description of each figure.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

Terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

In addition, “unit” used in this application may be replaced with “block”, and therefore, in this specification, “coding tree unit” and “coding tree block”, “coding unit” and “coding block” ”, “prediction unit” and “prediction block”, “transformation unit” and “transformation block” can each be interpreted as the same meaning.

Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention will be described in more detail. Hereinafter, the same reference numerals are used for the same components in the drawings, and redundant descriptions of the same components are omitted.

1 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present invention.

Referring to FIG. 1 , an image encoding apparatus 100 includes a picture division unit 110, prediction units 120 and 125, a transform unit 130, a quantization unit 135, a rearrangement unit 160, an entropy encoding unit ( 165), an inverse quantization unit 140, an inverse transform unit 145, a filter unit 150, and a memory 155.

Each component shown in FIG. 1 is shown independently to represent different characteristic functions in the video encoding device, and does not mean that each component is made of separate hardware or a single software component. That is, each component is listed and included as each component for convenience of explanation, and at least two components of each component can be combined to form one component, or one component can be divided into a plurality of components to perform a function, and each of these components can be divided into a plurality of components. Integrated embodiments and separated embodiments of components are also included in the scope of the present invention as long as they do not depart from the essence of the present invention.

In addition, some of the components may be optional components for improving performance rather than essential components that perform essential functions in the present invention. The present invention can be implemented by including only components essential to implement the essence of the present invention, excluding components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement. Also included in the scope of the present invention.

The picture division unit 110 may divide an input picture into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture divider 110 divides one picture into a plurality of combinations of coding units, prediction units, and transformation units, and combines one coding unit, prediction unit, and transformation unit according to a predetermined criterion (eg, a cost function). You can encode a picture by selecting .

For example, one picture may be divided into a plurality of coding units. In order to divide coding units in a picture, a recursive tree structure such as a quad tree structure can be used. Coding that is divided into different coding units with one image or largest coding unit as the root A unit may be divided with as many child nodes as the number of divided coding units. A coding unit that is not further divided according to a certain limit becomes a leaf node. That is, when it is assumed that only square division is possible for one coding unit, one coding unit can be divided into up to four different coding units.

Hereinafter, in an embodiment of the present invention, a coding unit may be used as a unit for performing encoding or a unit for performing decoding.

The prediction unit may be divided into at least one square or rectangular shape having the same size within one coding unit, and one of the prediction units divided within one coding unit predicts another prediction unit. It may be divided to have a shape and/or size different from the unit.

When generating a prediction unit for performing intra prediction based on a coding unit, if the coding unit is not a minimum coding unit, intra prediction may be performed without dividing into a plurality of prediction units NxN.

The prediction units 120 and 125 may include the inter prediction unit 120 performing inter prediction and the intra prediction unit 125 performing intra prediction. It is possible to determine whether to use inter prediction or intra prediction for a prediction unit, and determine specific information (eg, intra prediction mode, motion vector, reference picture, etc.) according to each prediction method. In this case, a processing unit in which prediction is performed and a processing unit in which a prediction method and specific details are determined may be different. For example, a prediction method and a prediction mode may be determined in a prediction unit, and prediction may be performed in a transformation unit. A residual value (residual block) between the generated prediction block and the original block may be input to the transform unit 130 . In addition, prediction mode information and motion vector information used for prediction may be encoded in the entropy encoder 165 together with residual values and transmitted to the decoder. When a specific encoding mode is used, it is also possible to encode an original block as it is and transmit it to a decoder without generating a prediction block through the prediction units 120 and 125 .

The inter-prediction unit 120 may predict a prediction unit based on information on at least one picture among pictures before or after the current picture, and in some cases, prediction based on information on a partially coded region within the current picture. Units can also be predicted. The inter prediction unit 120 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

The reference picture interpolator may receive reference picture information from the memory 155 and generate pixel information of an integer pixel or less in the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients may be used to generate pixel information of an integer pixel or less in units of 1/4 pixels. In the case of a color difference signal, a DCT-based 4-tap interpolation filter with different filter coefficients may be used to generate pixel information of an integer pixel or less in units of 1/8 pixels.

The motion predictor may perform motion prediction based on the reference picture interpolated by the reference picture interpolator. As a method for calculating the motion vector, various methods such as Full search-based Block Matching Algorithm (FBMA), Three Step Search (TSS), and New Three-Step Search Algorithm (NTS) may be used. The motion vector may have a motion vector value in units of 1/2 or 1/4 pixels based on interpolated pixels. The motion prediction unit may predict the current prediction unit by using a different motion prediction method. Various methods such as a skip method, a merge method, an advanced motion vector prediction (AMVP) method, and an intra block copy method may be used as motion prediction methods.

The intra prediction unit 125 may generate a prediction unit based on reference pixel information around the current block, which is pixel information in the current picture. If a block adjacent to the current prediction unit is a block on which inter prediction is performed, and the reference pixel is a pixel on which inter prediction is performed, the reference pixel of the block on which intra prediction is performed surrounding the reference pixel included in the block on which inter prediction is performed information can be used instead. That is, when a reference pixel is unavailable, information on the unavailable reference pixel may be replaced with at least one reference pixel among available reference pixels.

In intra prediction, a prediction mode may include a directional prediction mode in which reference pixel information is used according to a prediction direction and a non-directional prediction mode in which directional information is not used during prediction. A mode for predicting luminance information and a mode for predicting chrominance information may be different, and intra prediction mode information or predicted luminance signal information used to predict luminance information may be used to predict chrominance information.

When intra prediction is performed, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction of the prediction unit is based on the pixel on the left, the top left, and the top of the prediction unit. can be performed. However, when performing intra prediction, when the size of a prediction unit and the size of a transformation unit are different, intra prediction may be performed using a reference pixel based on the transformation unit. In addition, intra prediction using NxN division may be used only for the smallest coding unit.

In the intra prediction method, a prediction block may be generated after applying an adaptive intra smoothing (AIS) filter to a reference pixel according to a prediction mode. The type of AIS filter applied to the reference pixel may be different. In order to perform the intra prediction method, the intra prediction mode of the current prediction unit may be predicted from the intra prediction modes of prediction units existing around the current prediction unit. When predicting the prediction mode of the current prediction unit using the mode information predicted from the neighboring prediction unit, if the intra-prediction mode of the current prediction unit and the neighboring prediction unit are identical, the current prediction unit and the neighboring prediction unit use predetermined flag information It is possible to transmit information that the prediction modes of are the same, and if the prediction modes of the current prediction unit and the neighboring prediction unit are different, entropy encoding may be performed to encode the prediction mode information of the current block.

In addition, a residual block may be generated that includes residual information that is a difference between a prediction unit performed prediction based on the prediction unit generated by the prediction units 120 and 125 and an original block of the prediction unit. The generated residual block may be input to the transform unit 130 .

In the transform unit 130, the residual block including the original block and the residual information of the prediction unit generated through the prediction units 120 and 125 is converted into DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT and It can be converted using the same conversion method. Whether to apply DCT, DST, or KLT to transform the residual block may be determined based on intra prediction mode information of a prediction unit used to generate the residual block.

The quantization unit 135 may quantize the values converted to the frequency domain by the transform unit 130 . A quantization coefficient may change according to a block or an importance of an image. The value calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160 .

The rearrangement unit 160 may rearrange the coefficient values for the quantized residual values.

The reordering unit 160 may change a 2D block-type coefficient into a 1-D vector form through a coefficient scanning method. For example, the reordering unit 160 may scan DC coefficients to high-frequency coefficients using a zig-zag scan method and change them into a one-dimensional vector form. Depending on the size of the transform unit and the intra prediction mode, instead of zig-zag scan, a vertical scan for scanning 2D block-type coefficients in a column direction and a horizontal scan for scanning 2-dimensional block-type coefficients in a row direction may be used. That is, it is possible to determine which scan method among zig-zag scan, vertical scan, and horizontal scan is used according to the size of the transform unit and the intra prediction mode.

The entropy encoding unit 165 may perform entropy encoding based on the values calculated by the reordering unit 160 . Entropy encoding may use various encoding methods such as, for example, exponential Golomb, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC).

The entropy encoding unit 165 receives residual value coefficient information and block type information of a coding unit from the reordering unit 160 and the prediction units 120 and 125, prediction mode information, division unit information, prediction unit information and transmission unit information, motion Various information such as vector information, reference frame information, block interpolation information, and filtering information can be encoded.

The entropy encoding unit 165 may entropy-encode the coefficient value of the coding unit input from the reordering unit 160 .

The inverse quantization unit 140 and the inverse transform unit 145 inversely quantize the values quantized by the quantization unit 135 and inverse transform the values transformed by the transform unit 130 . The residual generated by the inverse quantization unit 140 and the inverse transform unit 145 is combined with the prediction unit predicted through the motion estimation unit, the motion compensation unit, and the intra prediction unit included in the prediction units 120 and 125 and restored. You can create a Reconstructed Block.

The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

The deblocking filter can remove block distortion caused by a boundary between blocks in a reconstructed picture. In order to determine whether to perform deblocking, it may be determined whether to apply the deblocking filter to the current block based on pixels included in several columns or rows included in the block. When a deblocking filter is applied to a block, a strong filter or a weak filter may be applied according to the required deblocking filtering strength. In addition, in applying the deblocking filter, when vertical filtering and horizontal filtering are performed, horizontal filtering and vertical filtering may be processed in parallel.

The offset correction unit may correct an offset of the deblocked image from the original image in units of pixels. In order to perform offset correction for a specific picture, pixels included in the image are divided into a certain number of areas, then the area to be offset is determined and the offset is applied to the area, or the offset is performed considering the edge information of each pixel method can be used.

Adaptive Loop Filtering (ALF) may be performed based on a value obtained by comparing the filtered reconstructed image with the original image. After dividing the pixels included in the image into predetermined groups, filtering may be performed differentially for each group by determining one filter to be applied to the corresponding group. Information related to whether or not to apply ALF may be transmitted for each coding unit (CU) of a luminance signal, and the shape and filter coefficients of an ALF filter to be applied may vary according to each block. In addition, the ALF filter of the same form (fixed form) may be applied regardless of the characteristics of the block to be applied.

The memory 155 may store a reconstructed block or picture calculated through the filter unit 150, and the stored reconstructed block or picture may be provided to the predictors 120 and 125 when inter prediction is performed.

2 is a block diagram illustrating a video decoding apparatus according to an embodiment of the present invention.

Referring to FIG. 2 , the image decoder 200 includes an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230 and 235, a filter unit ( 240), memory 245 may be included.

When a video bitstream is input from the video encoder, the input bitstream may be decoded by a procedure opposite to that of the video encoder.

The entropy decoding unit 210 may perform entropy decoding by a procedure opposite to that performed by the entropy encoding unit of the image encoder. For example, various methods such as exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) may be applied corresponding to the method performed by the image encoder.

The entropy decoding unit 210 may decode information related to intra prediction and inter prediction performed by the encoder.

The rearrangement unit 215 may perform rearrangement based on a method in which the encoding unit rearranges the entropy-decoded bitstream in the entropy decoding unit 210 . Coefficients expressed in the form of one-dimensional vectors may be reconstructed into coefficients in the form of two-dimensional blocks and rearranged. The rearrangement unit 215 may perform rearrangement through a method of receiving information related to the coefficient scanning performed by the encoder and performing reverse scanning based on the scanning order performed by the corresponding encoder.

The inverse quantization unit 220 may perform inverse quantization based on the quantization parameter provided by the encoder and the rearranged coefficient value of the block.

The inverse transform unit 225 may perform inverse transforms, that is, inverse DCT, inverse DST, and inverse KLT, on the transforms performed by the transform unit, that is, DCT, DST, and KLT, on the quantization result performed by the video encoder. Inverse transformation may be performed based on the transmission unit determined by the video encoder. The inverse transform unit 225 of the video decoder may selectively perform transformation techniques (eg, DCT, DST, KLT) according to a plurality of pieces of information such as a prediction method, a size of a current block, and a prediction direction.

The prediction units 230 and 235 may generate a prediction block based on information related to prediction block generation provided from the entropy decoding unit 210 and previously decoded block or picture information provided from the memory 245 .

As described above, when intra prediction is performed in the same way as the operation in the video encoder, when the size of the prediction unit and the size of the transformation unit are the same, the pixel existing on the left, the pixel existing on the upper left, and the pixel existing on the upper side of the prediction unit are the same. Intra prediction is performed on a prediction unit based on a pixel to perform intra prediction, but when performing intra prediction, when the size of the prediction unit and the size of the transformation unit are different, intra prediction is performed using a reference pixel based on the transformation unit. can In addition, intra prediction using NxN division may be used only for the smallest coding unit.

The prediction units 230 and 235 may include a prediction unit determining unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit receives various information such as prediction unit information input from the entropy decoding unit 210, prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, classifies the prediction unit from the current coding unit, and predicts It is possible to determine whether a unit performs inter prediction or intra prediction. The inter-prediction unit 230 uses information necessary for inter-prediction of the current prediction unit provided from the video encoder to make current prediction based on information included in at least one picture among pictures before or after the current picture including the current picture. Inter prediction can be performed on units. Alternatively, inter prediction may be performed based on information of a pre-reconstructed partial region in the current picture including the current prediction unit.

In order to perform inter prediction, the motion prediction method of the prediction unit included in the corresponding coding unit based on the coding unit is skip mode, merge mode, AMVP mode, and intra block copy mode. You can judge which way it is.

The intra prediction unit 235 may generate a prediction block based on pixel information in the current picture. When the prediction unit is a prediction unit that has undergone intra prediction, intra prediction may be performed based on intra prediction mode information of the prediction unit provided by the video encoder. The intra predictor 235 may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolator, and a DC filter. The AIS filter is a part that performs filtering on reference pixels of the current block, and can be applied by determining whether to apply the filter according to the prediction mode of the current prediction unit. AIS filtering may be performed on reference pixels of the current block using the prediction mode of the prediction unit and AIS filter information provided by the image encoder. When the prediction mode of the current block is a mode in which AIS filtering is not performed, AIS filter may not be applied.

When the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on a pixel value obtained by interpolating the reference pixel, the reference pixel interpolator may interpolate the reference pixel to generate a reference pixel in pixel units having an integer value or less. When the prediction mode of the current prediction unit is a prediction mode for generating a prediction block without interpolating reference pixels, the reference pixels may not be interpolated. The DC filter may generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

The reconstructed block or picture may be provided to the filter unit 240 . The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

Information on whether a deblocking filter is applied to a corresponding block or picture and, when a deblocking filter is applied, information on whether a strong filter or a weak filter is applied may be provided from the video coder. The deblocking filter of the video decoder receives information related to the deblocking filter provided by the video encoder, and the video decoder may perform deblocking filtering on the corresponding block.

The offset correction unit may perform offset correction on the reconstructed image based on the type and offset value information of the offset correction applied to the image during encoding.

ALF may be applied to a coding unit based on ALF application information and ALF coefficient information provided from an encoder. Such ALF information may be included in a specific parameter set and provided.

The memory 245 may store a reconstructed picture or block so that it can be used as a reference picture or reference block, and may also provide the reconstructed picture to an output unit.

As described above, in the following embodiments of the present invention, for convenience of description, a coding unit is used as a term for a coding unit, but it may be a unit that performs not only encoding but also decoding.

In addition, the current block indicates a block to be encoded/decoded, and according to encoding/decoding steps, a coding tree block (or coding tree unit), a coding block (or coding unit), a transform block (or transform unit), or a prediction block (or prediction unit) and the like. In this specification, 'unit' represents a basic unit for performing a specific encoding/decoding process, and 'block' may represent a sample array of a predetermined size. Unless otherwise specified, 'block' and 'unit' can be used interchangeably. For example, in an embodiment described later, a coding block (coding block) and a coding unit (coding unit) may be understood as equivalent to each other.

One picture may be encoded/decoded after being divided into square or non-square basic blocks. In this case, the basic block may be referred to as a coding tree unit. A coding tree unit may be defined as the largest coding unit allowed by a sequence or slice. Information related to whether the coding tree unit is square or non-square or the size of the coding tree unit may be signaled through a sequence parameter set, a picture parameter set, or a slice header. A coding tree unit can be split into smaller sized partitions. In this case, when a partition created by dividing the coding tree unit is referred to as depth 1, a partition created by dividing the depth 1 partition may be defined as depth 2. That is, a partition generated by dividing a partition having a depth k in the coding tree unit may be defined as having a depth k+1.

3 is a diagram illustrating a partition mode applicable to a coding block when the coding block is coded by intra prediction or inter prediction. A partition of an arbitrary size generated by dividing a coding tree unit is defined as a coding unit. can do. For example, FIG. 3 (a) shows that a coding unit has a size of 2N×2N. A coding unit may be recursively divided or divided into basic units for performing prediction, quantization, transformation, or in-loop filtering. For example, a partition of an arbitrary size generated as a coding unit is divided is defined as a coding unit, or a transform unit (TU) or prediction unit that is a basic unit for performing prediction, quantization, transformation, or in-loop filtering. (PU: Prediction Unit).

Alternatively, when a coding block is determined, a prediction block having the same size as the coding block or a smaller size than the coding block may be determined through prediction division of the coding block. Prediction division of a coding block may be performed by a partition mode (Part_mode) indicating a division type of a coding block. The size or shape of a prediction block may be determined according to a partition mode of a coding block. A partition type of a coding block may be determined through information specifying one of partition candidates. In this case, partition candidates usable by the coding block may include an asymmetric partition type (eg, nLx2N, nRx2N, 2NxnU, 2NxnD) according to the size, shape or coding mode of the coding block. For example, a partition candidate usable by a coding block may be determined according to the coding mode of the current block. For example, when a coding block is encoded by inter-prediction, any one of eight partition modes may be applied to the coding block, as in the example shown in FIG. 3(b). On the other hand, when a coding block is coded by intra prediction, PART_2Nx2N or PART_NxN among 8 partition modes of FIG. 3 (b) may be applied to the coding block.

PART_NxN may be applied when a coding block has a minimum size. Here, the minimum size of a coding block may be predefined in an encoder and a decoder. Alternatively, information on the minimum size of a coding block may be signaled through a bitstream. For example, the minimum size of a coding block is signaled through a slice header, and accordingly, the minimum size of a coding block may be defined for each slice.

As another example, a partition candidate usable by a coding block may be determined differently according to at least one of a size or a shape of a coding block. For example, the number or type of partition candidates usable by a coding block may be determined differently according to at least one of a size or a shape of a coding block.

Alternatively, the type or number of asymmetric partition candidates among partition candidates usable by a coding block may be limited according to the size or shape of the coding block. For example, the number or type of asymmetric partition candidates usable by a coding block may be determined differently according to at least one of the size or shape of the coding block.

In general, the size of a prediction block may have a size of 64x64 to 4x4. However, when a coding block is encoded by inter-prediction, the prediction block may not have a 4x4 size in order to reduce memory bandwidth when motion compensation is performed.

Using the partition mode, it is also possible to partition a coding block recursively. That is, a coding block may be divided according to a partition mode indicated by a partition index, and each partition generated as the coding block is divided may be defined as a coding block.

Hereinafter, a method of recursively dividing a coding unit will be described in more detail. For convenience of description, it is hereinafter assumed that a coding tree unit is also included in the category of a coding unit. That is, in an embodiment described below, a coding unit may indicate a coding tree unit or may mean a coding unit generated as a coding tree unit is divided. Also, when a coding block is recursively partitioned, a 'partition' generated as the coding block is partitioned may be understood to mean a 'coding block'.

A coding unit may be divided by at least one line. In this case, a line dividing a coding unit may have a predetermined angle. Here, the predetermined angle may be a value within a range of 0 degrees to 360 degrees. For example, a 0 degree line may mean a horizontal line, a 90 degree line may mean a vertical line, and a 45 degree or 135 degree line may mean a diagonal line.

When a coding unit is divided by a plurality of lines, all of the plurality of lines may have the same angle. Alternatively, at least one of the plurality of lines may have an angle different from that of the other lines. Alternatively, a coding tree unit or a plurality of lines dividing a coding unit may be set to have a predefined angular difference (eg, 90 degrees).

Information about a coding tree unit or a line dividing a coding unit may be defined and encoded as a partition mode. Alternatively, information about the number of lines, direction, angle, position of a line in a block, and the like may be encoded.

For convenience of explanation, it is assumed that a coding tree unit or a coding unit is divided into a plurality of coding units using at least one of a vertical line and a horizontal line in an embodiment described below.

Assuming that partitioning of coding units is performed based on at least one of a vertical line and a horizontal line, the number of vertical lines or horizontal lines partitioning a coding unit may be at least one. For example, a coding tree unit or coding unit may be divided into two partitions using one vertical line or one horizontal line, or a coding unit may be divided into three partitions using two vertical lines or two horizontal lines. . Alternatively, the coding unit may be divided into 4 partitions having a length and a width of 1/2 by using one vertical line and one horizontal line.

When a coding tree unit or a coding unit is divided into a plurality of partitions using at least one vertical line or at least one horizontal line, the partitions may have a uniform size. Alternatively, one partition may have a different size than the other partitions, or each partition may have a different size.

In embodiments to be described later, it is assumed that a coding unit is divided into four partitions as a quad tree-based division, and that a coding unit is divided into two partitions as a binary tree-based division. In addition, it is assumed that the coding unit is divided into three partitions as a triple tree-based division. In addition, it is assumed that partitioning by applying at least two or more partitioning methods is multi-tree-based partitioning.

In the drawings to be described later, it will be shown that a predetermined number of vertical lines or a predetermined number of horizontal lines are used to divide the coding unit, but by using a larger number of vertical lines or horizontal lines than shown, the coding unit It will be said that partitioning into a greater number of partitions than shown or a smaller number of partitions than shown is also included in the scope of the present invention.

4 is a diagram illustrating a partition type in which quad tree and binary tree partitioning is allowed as an embodiment to which the present invention is applied.

An input video signal is decoded in units of predetermined blocks, and a basic unit for decoding the input video signal is called a coding block. A coding block may be a unit for performing intra/inter prediction, transformation, and quantization. In addition, a prediction mode (eg, an intra prediction mode or an inter prediction mode) is determined for each coding block, and prediction blocks included in the coding block may share the determined prediction mode. A coding block can be a square or non-square block with any size ranging from 8x8 to 64x64, or a square or non-square block with a size of 128x128, 256x256 or more.

Specifically, the coding block may be hierarchically partitioned based on at least one of a quad tree and a binary tree. Here, quad tree-based splitting is a method in which a 2Nx2N coding block is split into four NxN coding blocks (FIG. 4(a)), and binary tree-based splitting is a method in which one coding block is split into two coding blocks. each can mean Even if binary tree-based partitioning is performed, a square coding block may exist in a lower depth.

Binary tree-based partitioning may be performed symmetrically or asymmetrically. Also, a coding block divided based on a binary tree may be a square block or a non-square block such as a rectangle. As an example, the partition type in which binary tree-based partitioning is allowed is symmetric 2NxN (horizontal non-square coding unit) or Nx2N (vertical non-square coding unit), as shown in the example shown in FIG. 4 (b). ) can be Also, as an example, the partition type in which binary tree-based partitioning is allowed may include at least one of asymmetric nLx2N, nRx2N, 2NxnU, or 2NxnD, as in the example shown in FIG. 4 (c). .

Binary tree-based partitioning may restrictively allow either symmetrical or asymmetrical partitions. In this case, configuring the coding tree unit with square blocks may correspond to quad tree CU partitioning, and configuring the coding tree unit with symmetric non-square blocks may correspond to binary tree CU partitioning. Constructing a coding tree unit with square blocks and symmetric non-square blocks may correspond to quad and binary tree CU partitioning.

Hereinafter, the partitioning method based on the quad tree and binary tree is referred to as QTBT (Quad-Tree & Binary-Tree) partitioning.

As a result of splitting based on the quad tree and the binary tree, a coding block that is not further split may be used as a prediction block or a transform block. That is, in the QTBT (Quad-Tree & Binary-Tree) partitioning method based on a quad tree and a binary tree, a coding block may become a prediction block, and the prediction block may become a transform block. For example, when the QTBT partitioning method is used, a prediction image may be generated in units of coding blocks, and a residual signal, which is a difference between an original image and a prediction image, may be converted in units of coding blocks. Here, generating a prediction image in units of coding blocks may mean determining motion information based on coding blocks or determining one intra prediction mode based on coding blocks. Accordingly, the coding block may be coded using at least one of a skip mode, intra-prediction, and inter-prediction.

As another example, it is also possible to divide a coding block and use a prediction block or a transform block having a smaller size than the coding block.

In the QTBT division method, BT can be set such that only symmetric division is allowed. However, even when the object and the background are divided at the block boundary, if only symmetric binary division is allowed, encoding efficiency may be lowered. Accordingly, in the present invention, a method of asymmetrically partitioning a coding block in order to increase coding efficiency will be described later in another embodiment. Asymmetric Binary Tree Partitioning refers to splitting a coding block into two smaller coding blocks. As a result of asymmetric binary tree partitioning, a coding block can be split into two asymmetric types of coding blocks.

Binary-tree-based splitting may be performed on a coding block for which quad-tree-based splitting is no longer performed. Quad-tree-based splitting may no longer be performed on a coding block split based on a binary tree.

In addition, the division of the lower depth may be determined depending on the division form of the upper depth. For example, when binary tree-based splitting is allowed in two or more depths, only binary tree-based splitting in the same form as the binary tree splitting type of the upper depth may be allowed in lower depths. For example, if binary tree-based partitioning in the form of 2NxN is performed at the upper depth, binary tree-based partitioning in the form of 2NxN may be performed also in the lower depth. Alternatively, when binary tree-based partitioning in the Nx2N format is performed in the upper depth, binary tree-based partitioning in the Nx2N format may also be allowed in the lower depth.

Conversely, in a lower depth, it is also possible to allow only a binary tree-based division in a form different from that of an upper depth binary tree division.

For sequences, slices, coding tree units, or coding units, only certain types of binary tree-based partitioning may be restricted to be used. For example, it may be restricted to allow only 2NxN or Nx2N binary tree-based partitioning of a coding tree unit. The allowed partition type may be previously defined in the encoder or decoder, and information on the allowed partition type or the disallowed partition type may be encoded and signaled through a bitstream.

5 illustrates an example of hierarchically splitting a coding block based on quad tree and binary tree splits as an embodiment to which the present invention is applied.

As shown in FIG. 5 , the first coding block 300 having a split depth of k may be split into a plurality of second coding blocks based on a quad tree. For example, the second coding blocks 310 to 340 are square blocks having half the width and height of the first coding block, and the division depth of the second coding block may be increased to k+1.

The second coding block 310 having a division depth of k+1 may be divided into a plurality of third coding blocks having a division depth of k+2. The division of the second coding block 310 may be selectively performed using either a quart tree or a binary tree according to a division method. Here, the splitting method may be determined based on at least one of information indicating quad-tree-based splitting or binary-tree-based splitting information.

When the second coding block 310 is divided based on the quart tree, the second coding block 310 is divided into four third coding blocks 310a having half the width and height of the second coding block, and the third coding block 310a The segmentation depth can be increased to k+2. On the other hand, when the second coding block 310 is divided based on a binary tree, the second coding block 310 may be divided into two third coding blocks. In this case, each of the two third coding blocks is a non-square block having one half of the width and height of the second coding block, and the division depth may be increased to k+2. The second coding block may be determined as a horizontal or vertical non-square block according to a splitting direction, and the splitting direction may be determined based on information about whether binary tree-based splitting is in a vertical or horizontal direction.

Meanwhile, the second coding block 310 may be determined as an end coding block that is not further split based on a quad tree or a binary tree, and in this case, the corresponding coding block may be used as a prediction block or a transform block.

Like the division of the second coding block 310, the third coding block 310a may be determined as an end coding block or additionally divided based on a quad tree or a binary tree.

Meanwhile, the third coding block 310b divided based on the binary tree may be additionally divided into a vertical coding block 310b-2 or a horizontal coding block 310b-3 based on the binary tree, and the corresponding coding The division depth of a block can be increased to k+3. Alternatively, the third coding block 310b may be determined as an end coding block 310b-1 that is not further split based on a binary tree, and in this case, the corresponding coding block 310b-1 is used as a prediction block or a transformation block. can However, in the above-described division process, information on the size/depth of coding blocks for which quad tree-based division is allowed, information on the size/depth of coding blocks for which binary tree-based division is allowed, or binary tree-based division is allowed It may be limitedly performed based on at least one of information about the size/depth of a coding block that is not specified.

The size of a coding block may be limited to a predetermined number, or the size of a coding block within a predetermined unit may have a fixed value. For example, the size of a coding block in a sequence or a coding block in a picture may be limited to 256x256, 128x128, or 32x32. Information indicating the size of a coding block in a sequence or picture may be signaled through a sequence header or a picture header.

As a result of partitioning based on quad trees and binary trees, coding units may have a square shape or a rectangular shape of any size.

6 illustrates an example of hierarchically partitioning a coding block based on quad tree and symmetric binary tree partitioning as an embodiment to which the present invention is applied.

6 is a diagram showing an example in which only partitioning based on a specific type, eg, a symmetric binary tree, is permitted. (a) of FIG. 6 shows a limited example in which only binary tree-based partitioning in the form of Nx2N is allowed. For example, the depth 1 coding block 601 can be divided into two Nx2N blocks 601a and 601b in depth 2, and the depth 2 coding block 602 can be divided into two Nx2N blocks 602a and 602b in depth 3. .

6(b) shows a restricted example in which only binary tree-based partitioning in the form of 2NxN is allowed. For example, the depth 1 coding block 603 can be divided into two 2NxN blocks 603a and 603b in depth 2, and the depth 2 coding block 604 can be divided into two 2NxN blocks 604a and 604b in depth 3. .

6(c) shows an example of dividing a block divided into a symmetric binary tree into a symmetric binary tree again. For example, the depth 1 coding block 605 is divided into two Nx2N blocks (605a, 605b) at depth 2, and the depth 2 coding block 605a generated after the division is divided into two Nx2N blocks (605a1, 605a1, 605b) at depth 3. 605a2). The splitting method is equally applicable to 2N×N coding blocks generated by symmetric binary tree splitting.

Information indicating quad-tree-based partitioning to implement the quad-tree or binary-tree-based adaptive partitioning, information about the size/depth of coding blocks for which quad-tree-based partitioning is allowed, and binary tree-based partitioning instructions information, information on the size/depth of a coding block where binary tree-based splitting is allowed, information on the size/depth of a coding block where binary tree-based splitting is not allowed, or whether binary tree-based splitting is in the vertical direction, or Information on whether or not it is in a horizontal direction may be used. For example, quad_split_flag may indicate whether a coding block is split into 4 coding blocks, and binary_split_flag may indicate whether a coding block is split into 2 coding blocks. When a coding block is split into two coding blocks, is_hor_split_flag indicating whether the coding block is split in a vertical direction or a horizontal direction may be signaled.

Also, for a coding tree unit or a predetermined coding unit, the number of allowed binary tree splits, the allowed depth for binary tree splits, or the number of allowed depths for binary tree splits may be obtained. The information may be encoded in units of coding tree units or coding units and transmitted to a decoder through a bitstream.

For example, syntax 'max_binary_depth_idx_minus1' indicating a maximum depth at which binary tree splitting is allowed may be encoded/decoded through a bitstream. In this case, max_binary_depth_idx_minus1+1 may indicate the maximum depth at which binary tree splitting is allowed.

In addition, referring to the above-described example of FIG. 6 (c), the result of binary tree splitting is shown for coding units with a depth of 2 (eg, 605a and 605b) and coding units with a depth of 3 (eg, 605a1 and 605a2). . Accordingly, information indicating the number of binary tree splits in the coding tree unit (eg, twice), information indicating the maximum depth allowed for binary tree splits in the coding tree unit (eg, depth 3), or in the coding tree unit At least one of information indicating the number of depths (eg, two, depth 2 and depth 3) for which binary tree splitting is permitted may be encoded/decoded through a bitstream.

As another example, at least one of the number of binary tree splitting allowed, the depth at which binary tree splitting is allowed, or the number of depths at which binary tree splitting is allowed may be obtained for each sequence or slice. For example, the information may be encoded in units of sequences, pictures, or slices and transmitted through a bitstream. Accordingly, at least one of the number of binary tree splits, the maximum depth allowed for binary tree splits, and the number of depths allowed for binary tree splits may be different between the first slice and the second slice. For example, in the first slice, binary tree splitting is allowed at only one depth, whereas in the second slice, binary tree splitting is allowed at two depths.

As another example, at least one of the number of binary tree splitting allowed, the depth at which binary tree splitting is allowed, or the number of depths at which binary tree splitting is allowed may be set differently according to the temporal level identifier (Temporal_ID) of the slice or picture. there is. Here, the temporal level identifier (Temporal_ID) is used to identify each of a plurality of layers of an image having at least one scalability among view, spatial, temporal, or quality. will be.

In addition, it may be restricted not to use transform skip in a CU partitioned by binary partitioning. Alternatively, transformskip may be applied only in at least one of a horizontal direction and a vertical direction in a CU partitioned in a non-square pattern. Applying only the horizontal transform skip indicates that only scaling and quantization are performed without performing transform in the horizontal direction, and transform is performed by specifying at least one transform such as DCT or DST in the vertical direction.

Likewise, applying only the vertical transform skip indicates performing transformation by specifying at least one transform such as DCT or DST in the horizontal direction and performing only scaling and quantization without performing transform in the vertical direction. Syntax hor_transform_skip_flag indicating whether to apply transform skip in the horizontal direction and syntax ver_transform_skip_flag indicating whether to apply transform skip in the vertical direction may be signaled.

When transform skip is applied to at least one of the horizontal direction or the vertical direction, a signal may be signaled in which direction to apply transform skip according to the shape of the CU. Specifically, for example, in the case of a 2NxN type CU. Transform may be performed in the horizontal direction and transform skip may be applied in the vertical direction. In the case of an Nx2N type CU, transform skip may be applied in the horizontal direction and transform may be performed in the vertical direction. Here, the transform may be at least one of DCT and DST.

As another example, in the case of a 2NxN CU, transform may be performed in the vertical direction and transform skip may be applied in the horizontal direction. In the case of an Nx2N CU, transform skip may be applied in the vertical direction and transform in the horizontal direction can also be performed. Here, the transform may be at least one of DCT and DST.

7 is an embodiment to which the present invention is applied, and is a diagram showing a partition form in which asymmetric binary tree partitioning is allowed. A 2Nx2N coding block is two coding blocks with a width ratio of n:(1-n) or a height ratio of n. :(1-n) can be divided into two coding blocks. Here, n may represent a real number greater than 0 and less than 1.

In FIG. 7, for example, as asymmetric binary tree partitioning is applied to coding blocks, two coding blocks 701 and 702 having a width ratio of 1:3, or two coding blocks 703 and 704 having a width ratio of 3:1, Alternatively, two coding blocks 705 and 706 having a height ratio of 1:3 or two coding blocks 707 and 708 having a height ratio of 3:1 are generated.

Specifically, as a coding block having a size of WxH is divided in the vertical direction, a left partition having a width of 1/4W and a right partition having a width of 3/4W may be generated. As described above, a division type in which the width of the left partition is smaller than the width of the right partition may be referred to as an nLx2N binary partition.

As a coding block having a size of WxH is divided in the vertical direction, a left partition with a width of 3/4W and a right partition with a width of 1/4W may be created. As described above, a division type in which the width of the right partition is smaller than the width of the left partition may be referred to as an nRx2N binary partition.

As a coding block having a size of WxH is horizontally divided, an upper partition having a height of 1/4H and a lower partition having a height of 3/4H may be generated. As described above, a partition form in which the height of the upper partition is smaller than the height of the lower partition may be referred to as a 2NxnU binary partition.

As a coding block having a size of WxH is horizontally divided, an upper partition having a height of 3/4H and a lower partition having a height of 1/4H may be generated. As described above, a partition form in which the height of the lower partition is smaller than the height of the upper partition may be referred to as a 2NxnD binary partition.

7 illustrates a case where the width or height ratio between two coding blocks is 1:3 or 3:1, but the width ratio or height ratio between two coding blocks generated by asymmetric binary tree partitioning is not limited thereto. A coding block may be divided into two coding blocks having a different width ratio or a different height ratio than shown in FIG. 7 .

In the case of using asymmetric binary tree partitioning, an asymmetric binary partition type of a coding block may be determined based on information signaled through a bitstream. For example, a partition type of a coding block includes information indicating a division direction of the coding block and a coding block. It may be determined based on information indicating whether a first partition generated by the division has a size smaller than that of the second partition.

The information indicating the splitting direction of the coding block may be a 1-bit flag indicating whether the coding block is split vertically or horizontally. For example, hor_binary_flag may indicate whether a coding block is split in a horizontal direction. A value of hor_binary_flag of 1 may indicate that the coding block is split in the horizontal direction, and a value of hor_binary_flag of 0 may indicate that the coding block is split in the vertical direction. Alternatively, ver_binary_flag indicating whether the coding block is split in the vertical direction may be used.

Information indicating whether the first partition has a smaller size than the second partition may be a 1-bit flag. For example, is_left_above_small_part_flag may indicate whether the size of a left or upper partition generated as a coding block is divided is smaller than that of a right or lower partition. A value of is_left_above_small_part_flag of 1 means that the size of the left or top partition is smaller than that of the right or bottom partition, and a value of is_left_above_small_part_flag of 0 may mean that the size of the left or top partition is larger than that of the right or bottom partition. Alternatively, is_right_bottom_small_part_flag indicating whether the size of the right or bottom partition is smaller than that of the left or top partition may be used.

Alternatively, the size of the first partition and the second partition may be determined using information indicating a width ratio, a height ratio, or an area ratio between the first partition and the second partition.

A value of hor_binary_flag of 0 and a value of is_left_above_small_part_flag of 1 indicate an nLx2N binary partition, and a value of hor_binary_flag of 0 and a value of is_left_above_small_part_flag of 0 indicate an nRx2N binary partition. In addition, a value of hor_binary_flag of 1 and a value of is_left_above_small_part_flag of 1 indicates a 2NxnU binary partition, and a value of hor_binary_flag of 1 and a value of is_left_above_small_part_flag of 0 indicate a 2NxnD binary partition.

As another example, the asymmetric binary partition type of the coding block may be determined by index information indicating the partition type of the coding block. Here, the index information is information signaled through a bitstream, and may be coded with a fixed length (ie, a fixed number of bits) or with a variable length. As an example, Table 1 below shows partition indexes for each asymmetric binary partition.

Asymetric partition index Binarization nLx2N 0 0 nRx2N One 10 2NxnU 2 100 2NxnD 3 111

Asymmetric binary tree partitioning may be used depending on the QTBT partitioning method. For example, if quad tree partitioning or binary tree partitioning is no longer applied to a coding block, whether to apply asymmetric binary tree partitioning to the corresponding coding block. can be determined. Here, whether to apply asymmetric binary tree splitting to a coding block may be determined by information signaled through a bitstream. For example, the information may be a 1-bit flag 'asymmetric_binary_tree_flag', and based on the flag, whether or not asymmetric binary tree splitting is applied to the coding block may be determined. Alternatively, the coding block may be divided into two blocks. When determined, it may be determined whether the partitioning type is binary tree partitioning or asymmetric binary tree partitioning. Here, whether the division type of the coding block is binary tree division or asymmetric binary tree division may be determined by information signaled through a bitstream. For example, the information may be a 1-bit flag 'is_asymmetric_split_flag', and based on the flag, whether a coding block is split in a symmetrical or asymmetrical manner may be determined.

As another example, different indices may be assigned to the symmetric binary partition and the asymmetric binary partition, and whether the coding block is divided in a symmetrical or asymmetrical manner may be determined according to the index information. As an example, Table 2 shows an example in which different indexes are assigned to a symmetric binary partition and an asymmetric binary partition.

Binary partition index Binarization 2NxN (horizontal binary partition) 0 0 Nx2N (vertically oriented binary partition) One 10 nLx2N 2 110 nRx2N 3 1110 2NxnU 4 11110 2NxnD 5 11111

A coding tree block or coding block may be subdivided into a plurality of coding blocks through quad tree splitting, binary tree splitting, or asymmetric binary tree splitting. As an example, FIG. 8 is a diagram illustrating an example in which a coding block is divided into a plurality of coding blocks using QTBT and asymmetric binary tree partitioning. Referring to FIG. 9 , it can be seen that asymmetric binary tree partitioning is performed in depth 2 partitioning of the first grip, depth 3 partitioning of the second figure, and depth 3 partitioning of the third figure, respectively. Coding blocks divided through asymmetric binary tree partitioning may be constrained so that it is not further divided. For example, information related to a quad tree, a binary tree, or an asymmetric binary tree may not be encoded/decoded in a coding block generated through asymmetric binary tree partitioning. That is, for a coding block generated through asymmetric binary tree partitioning, a flag indicating whether to split a quad tree, a flag indicating whether to split a binary tree, a flag indicating whether to split asymmetric binary tree, and a binary tree or asymmetric binary tree split direction Encoding/decoding of syntax such as an indicating flag or index information indicating an asymmetric binary partition may be omitted.

As another example, whether to allow binary tree partitioning may be determined depending on whether QTBT is allowed. For example, asymmetric binary tree partitioning may be restricted so that it is not used in a picture or slice in which the QTBT-based partitioning method is not used.

Information indicating whether asymmetric binary tree partitioning is allowed may be coded and signaled in units of blocks, slices, or pictures. Here, information indicating whether asymmetric binary tree partitioning is allowed may be a 1-bit flag. For example, a value of is_used_asymmetric_QTBT_enabled_flag of 0 may indicate that asymmetric binary tree partitioning is not used. If binary tree partitioning is not used in units of pictures or units of slices, is_used_asymmetric_QTBT_enabled_flag may not be signaled and its value may be set to 0.

8 illustrates a division form of a coding block based on quad tree and symmetric/asymmetric binary tree division as an embodiment to which the present invention is applied.

(a) of FIG. 8 shows an example in which partitioning based on an asymmetric binary tree in the form of nLx2N is permitted. For example, the depth 1 coding block 801 is divided into two asymmetric nLx2N blocks (801a, 801b) in depth 2, and the depth 2 coding block 801b is divided into two symmetric Nx2N blocks (801b1, 801b2) in depth 3. A split example is shown.

8(b) shows an example in which partitioning based on an asymmetric binary tree in the form of nRx2N is allowed. For example, the depth 2 coding block 802 shows an example of being divided into two asymmetrical nRx2N blocks 802a and 802b in depth 3.

(c) of FIG. 8 shows an example in which partitioning based on an asymmetric binary tree in the form of 2NxnU is permitted. For example, the depth 2 coding block 803 shows an example of being divided into two asymmetric 2NxnU blocks 803a and 803b in depth 3.

A split type allowed for a coding block may be determined based on the size, shape, split depth or split type of the coding block. For example, at least one of a partition type, a partition type, or the number of partitions allowed between a coding block generated by quad tree partitioning and a coding block generated by binary tree partitioning may be different.

For example, when a coding block is generated by quad tree splitting, quad tree splitting, binary tree splitting, and asymmetric binary tree splitting may all be allowed in the corresponding coding block. That is, when a coding block is generated based on quad tree partitioning, all partition types shown in FIG. 10 can be applied to the coding block. represents a case where the coding block is quad-tree split, and Nx2N and 2NxN may represent a case where the coding block is binary tree split. Also, nLx2N, nRx2N, 2NxnU, and 2NxnD may indicate a case where a coding block is asymmetric binary tree split.

On the other hand, when a coding block is generated by binary tree splitting, asymmetric binary tree splitting may be restricted to the corresponding coding block. That is, when a coding block is generated based on binary tree partitioning, the application of an asymmetric partition type (nLx2N, nRx2N, 2NxnU, 2NxnD) among the partition types shown in FIG. 7 to the coding block may be restricted.

9 is a flowchart of a coding block partitioning method based on quad tree and binary tree partitioning as an embodiment to which the present invention is applied.

It is assumed that a depth k coding block is divided into depth k+1 coding blocks. First, it is determined whether quad tree partitioning is applied to the current block of depth k (S910). If quad tree partitioning is applied, the current block is divided into four square blocks (S920). On the other hand, if quad tree splitting is not applied, it is determined whether binary tree splitting is applied to the current block (S930). If binary tree splitting is also not applied, the current block becomes a depth k+1 coding block without splitting. As a result of the determination in S930, if binary tree partitioning is applied to the current block, it is checked whether symmetrical binary partitioning or asymmetrical binary partitioning is applied (S940). According to the determination result of S940, a partition type applied to the current block is determined (S950). For example, the partition shape applied to the step S950 may be one of the shapes of FIG. 4(b) in the case of a symmetrical type or one of the shapes of FIG. 4(c) in the case of an asymmetric type. Through the above S950, according to the determined partition type, the current block is divided into two depth k+1 coding blocks (S960).

10 illustrates, for example, syntax elements included in a network abstraction layer (NAL) to which quad tree and binary tree partitioning are applied as an embodiment to which the present invention is applied.

Compressed video to which the present invention is applied may be packetized in units of a Network Abstract Layer (hereinafter referred to as 'NAL') and transmitted through a transmission medium, for example. However, the present invention is not limited to NAL and can be applied to various data transmission methods to be developed in the future. A NAL unit to which the present invention is applied, for example, as shown in FIG. 10, a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), and at least one slice set (Slice) can include

For example, although syntax elements included in a sequence parameter set (SPS) are shown in FIG. 10 , it is also possible to include syntax elements in a picture parameter set (PPS) or a slice set (Slice). In addition, syntax elements commonly applied to sequence units or picture units for each syntax element may be included in a sequence parameter set (SPS) or a picture parameter set (PPS). On the other hand, a syntax element applied only to a corresponding slice is preferably included in a slice set (Slice). Therefore, it can be selected in consideration of encoding performance and efficiency.

In this regard, syntax elements to which quad tree and binary tree partitioning are applied are described as follows. It is possible to set all the syntax elements shown in FIG. 10 as essential elements, but it is also possible to selectively set dual syntax elements in consideration of encoding efficiency and performance.

For example, 'quad_split_flag' indicates whether a coding block is split into 4 coding blocks. 'binary_split_flag' may indicate whether a coding block is split into two coding blocks. When a coding block is split into two coding blocks, 'is_hor_split_flag' indicating whether the coding block is split in a vertical direction or a horizontal direction may be signaled. If “is_hor_split_flag = 1”, the horizontal direction can be defined as indicating the vertical direction if “is_hor_split_flag = 0”.

In addition, as another alternative, 'isUseBinaryTreeFlag' indicates whether binary tree partitioning is applied to the current block and, as a syntax element indicating the division direction of the coding block, 'hor_binary_flag' indicates whether the coding block is horizontally partitioned. can indicate For example, “hor_binary_flag = 1” indicates that the coding block is split in the horizontal direction, and “hor_binary_flag = 0” indicates that the coding block is split in the vertical direction. Alternatively, instead of 'hor_binary_flag', ver_binary_flag indicating whether the coding block is split in the vertical direction may be used to set in the same manner.

In addition, 'max_binary_depth_idx_minus1' may be defined as a syntax element representing the maximum depth at which binary tree splitting is allowed. For example, “max_binary_depth_idx_minus1 + 1” may indicate the maximum depth at which binary tree splitting is allowed.

In addition, 'hor_transform_skip_flag' as a syntax element indicating whether to apply horizontal transform skip and 'ver_transform_skip_flag' may be set as a syntax element indicating whether to apply vertical transform skip.

In addition, 'is_used_asymmetric_QTBT_enabled_flag' may be defined as a syntax element indicating whether asymmetric binary tree partitioning is allowed. For example, “is_used_asymmetric_QTBT_enabled_flag = 1” indicates that asymmetric binary tree partitioning is used, and “is_used_asymmetric_QTBT_enabled_flag = 0” indicates that asymmetric binary tree partitioning is not used. On the other hand, if binary tree partitioning is not used in units of pictures or units of slices, is_used_asymmetric_QTBT_enabled_flag may not be signaled and its value may be set to 0. Also, as another alternative, 'asymmetric_binary_tree_flag' may indicate whether asymmetric binary tree partitioning is applied to the current block.

Also, as a syntax element indicating asymmetric binary tree splitting, 'is_left_above_small_part_flag' may indicate whether the size of a left or upper partition generated as a coding block is split is smaller than that of a right or lower partition. For example, if “is_left_above_small_part_flag = 1”, it means that the size of the left or top partition is smaller than that of the right or bottom partition, and if “is_left_above_small_part_flag = 0”, the size of the left or top partition is larger than that of the right or bottom partition. that can mean Alternatively, instead of 'is_left_above_small_part_flag', 'is_right_bottom_small_part_flag' indicating whether the size of the right or bottom partition is smaller than that of the left or top partition may be used.

In this regard, it is possible to define an asymmetrical binary partition form of a coding block by combining the above syntax elements. For example, “hor_binary_flag = 0” and “is_left_above_small_part_flag = 1” indicate an nLx2N binary partition, and “hor_binary_flag = 0” and “is_left_above_small_part_flag = 0” indicate an nRx2N binary partition. In addition, “hor_binary_flag = 1” and “is_left_above_small_part_flag = 1” indicate a 2NxnU binary partition, and “hor_binary_flag = 1” and “is_left_above_small_part_flag = 0” indicate a 2NxnD binary partition. Similarly, the asymmetric binary partition type may be indicated by using a combination of 'ver_binary_flag' and 'is_right_bottom_small_part_flag'.

In addition, as an alternative, it is possible to define an asymmetric binary partition form of a coding block by indicating the index of the aforementioned Table 1 by 'Asymetric_partition_index' or by indicating the index of the aforementioned Table 2 by 'Binary_partition_index'. do.

As reviewed in the above example, a coding unit (or coding tree unit) may be recursively divided by at least one vertical line or horizontal line. For example, quad tree splitting is a method of splitting a coding block using horizontal and vertical lines, and binary tree splitting can be summarized as a method of splitting a coding block using horizontal or vertical lines. A partition type of a coding block subjected to quad tree partitioning and binary tree partitioning is not limited to the examples shown in FIGS. 4 to 8 , and extended partition types other than those shown may be used. That is, the coding block may be recursively divided in a form different from that shown in FIGS. 4 to 8 .

11 is another embodiment to which the present invention is applied, and is a diagram showing a partition type in which asymmetric quad tree partitioning is allowed.

When the current block is quad-tree split, at least one of horizontal lines and vertical lines may split the coding block in an asymmetrical form. Here, asymmetry may mean a case where the heights of blocks divided by horizontal lines are not the same or the widths of blocks divided by vertical lines are not the same. For example, a horizontal line may divide a coding block in an asymmetrical form, while a vertical line may divide a coding block in a symmetrical form, and a horizontal line may divide a coding block in a symmetrical form, whereas a vertical line may divide a coding block in an asymmetrical form. may be Alternatively, the coding block may be divided in an asymmetrical form by both horizontal and vertical lines.

11 (a) shows a symmetrical quad-tree splitting form of a coding block, and (b) to (k) are diagrams showing asymmetrical quad-tree splitting forms of a coding block. 11 (a) shows an example in which both horizontal and vertical lines are used for symmetric division. 11 (b) and (c) show examples in which horizontal lines are used for symmetrical segmentation, while vertical lines are used for asymmetrical segmentation. 11 (d) and (e) show examples in which vertical lines are used for symmetrical segmentation, while horizontal lines are used for asymmetrical segmentation.

In order to specify the division form of the coding block, information related to the division form of the coding block may be encoded. Here, the information may include a first indicator indicating whether a division type of a coding block is symmetrical or asymmetrical. The first indicator may be coded in units of blocks or may be coded in units of vertical lines or horizontal lines. For example, the first indicator may include information indicating whether vertical lines are used for symmetric division and information indicating whether horizontal lines are used for symmetric division.

Alternatively, the first indicator may be encoded only for at least one of a vertical line or a horizontal line, and another division type in which the first indicator is not encoded may be derived dependently by the first indicator. For example, another division type in which the first indicator is not encoded may have a value opposite to that of the first indicator. That is, when the first indicator indicates that the vertical line is used for asymmetric segmentation, the horizontal line may be set to be used for symmetric segmentation opposite to the first indicator.

When the first indicator indicates that the division is asymmetrical, a second indicator may be additionally coded for a vertical line or a horizontal line. Here, the second indicator may indicate at least one of a position of a vertical or horizontal line used for asymmetric division or a ratio between blocks divided by the vertical or horizontal line.

Quad tree splitting may be performed using a plurality of vertical lines or a plurality of horizontal lines. For example, it is also possible to divide a coding block into four blocks by combining at least one of one or more vertical lines or one or more horizontal lines.

11 (f) to (k) are diagrams illustrating examples of dividing a coding block asymmetrically by combining a plurality of vertical/horizontal lines and one horizontal/vertical line.

Referring to FIGS. 11 (f) to (k), quad tree partitioning divides a coding block into three blocks by two vertical lines or two horizontal lines, and divides any one of the three divided blocks into two blocks. This can be done by dividing At this time, as in the examples shown in FIGS. 11 (f) to (k), a block located in the middle among blocks divided by two vertical lines or two horizontal lines may be divided by one horizontal line or vertical line. In addition to the illustrated example, a block located at one boundary of a coding block may be divided by one horizontal or vertical line. Alternatively, information for specifying a partition to be divided among the three partitions (eg, a partition index) may be signaled through a bitstream.

At least one of horizontal lines or vertical lines may be used to divide a coding block in an asymmetrical form, and the other may be used to divide a coding block in a symmetrical form. For example, a plurality of vertical or horizontal lines may be used to symmetrically divide a coding block, or a single horizontal or vertical line may be used to symmetrically divide a coding block. Alternatively, both horizontal and vertical lines may be used to divide a coding block in a symmetrical or asymmetrical manner.

For example, FIG. 11(f) shows a partition form in which a coding block in the middle divided in an asymmetrical form by two vertical lines is divided into two symmetrical coding blocks by a horizontal line. In addition, FIG. 11(g) shows a partition form in which a coding block in the middle asymmetrically divided by two horizontal lines is divided into two symmetrical coding blocks by a vertical line.

On the other hand, FIGS. 11(h) and (i) show a partition form in which the middle coding block divided in an asymmetrical form by two vertical lines is further divided into two asymmetrical coding blocks by a horizontal line. 11(j) and (k) show a partition form in which a coding block in the middle divided in an asymmetrical form by two horizontal lines is further divided into two asymmetrical coding blocks by a vertical line.

In the case of combining multiple vertical/horizontal lines and one horizontal/vertical line, a coding block is divided into four partitions (ie, four coding blocks) composed of at least two different sizes. Dividing a coding block into four partitions having at least two different sizes may be referred to as triple type asymmetric quad-treeCU partitioning.

Information about triadic asymmetric quad tree partitioning may be coded based on at least one of the aforementioned first indicator or second indicator. For example, the first indicator may indicate whether a division type of a coding block is symmetrical or asymmetrical. The first indicator may be coded in units of blocks or may be coded in units of vertical lines or horizontal lines. For example, the first indicator may include information indicating whether one or more vertical lines are used for symmetric division and information indicating whether one or more horizontal lines are used for symmetric division.

Alternatively, the first indicator may be encoded only for at least one of a vertical line or a horizontal line, and another division type in which the first indicator is not encoded may be derived dependently by the first indicator.

When the first indicator indicates asymmetric division, a second indicator may be additionally coded for a vertical line or a horizontal line. Here, the second indicator may indicate at least one of a position of a vertical or horizontal line used for asymmetric division or a ratio between blocks divided by the vertical or horizontal line.

12 is a flowchart of a coding block partitioning method based on asymmetric quad tree partitioning as another embodiment to which the present invention is applied.

It is assumed that a depth k coding block is divided into depth k+1 coding blocks. First, it is determined whether quad tree partitioning is applied to the current block of depth k (S1210). As a result of the determination in step S1210, if quad tree splitting is not applied, the current block becomes a depth k+1 coding block without splitting. If, as a result of the determination in step S1210, quad tree partitioning is applied, it is determined whether asymmetric quad tree partitioning is applied to the current block (S1220). If asymmetric quad tree splitting is not applied and symmetric quad tree splitting is applied, the current block is divided into four square blocks (S1230).

On the other hand, if asymmetric quad-tree splitting is applied, it is determined whether or not three-type asymmetric quad-tree splitting is applied to the current block (S1240). If 3-type asymmetric quad tree splitting is not applied, the current block is divided into 4 2-type asymmetric blocks (S1250). At this time, it may be divided into any one of the partition types of FIGS. 11 (b) to (e) according to the partition information.

On the other hand, if 3-type asymmetric quad tree splitting is applied, the current block is divided into 4 3-type asymmetric blocks (S1260). At this time, it may be divided into any one of the partition types of FIGS. 11 (f) to (k) according to the partition information.

13 illustrates, for example, a syntax element included in a network abstraction layer (NAL) to which asymmetric quad tree partitioning is applied as another embodiment to which the present invention is applied. A NAL unit to which the present invention is applied may include, for example, a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), and at least one slice set (Slice).

For example, although syntax elements included in a sequence parameter set (SPS) are illustrated in FIG. 13 , it is also possible to include syntax elements in a picture parameter set (PPS) or a slice set (Slice). In addition, syntax elements commonly applied to sequence units or picture units for each syntax element may be included in a sequence parameter set (SPS) or a picture parameter set (PPS). On the other hand, a syntax element applied only to a corresponding slice is preferably included in a slice set (Slice). Therefore, it can be selected in consideration of encoding performance and efficiency.

A syntax element 'Is_used_asymmertic_quad_tree_flag' indicates whether quad tree splitting is performed asymmetrically. In addition, 'Is_used_triple_asymmertic_quad_tree_flag' indicates whether quad tree splitting is performed in a three-type asymmetric manner. Therefore, if “Is_used_asymmertic_quad_tree_flag = 0” means splitting a symmetric quad tree, 'Is_used_triple_asymmertic_quad_tree_flag' is not signaled. On the other hand, “Is_used_asymmertic_quad_tree_flag = 1” and “Is_used_triple_asymmertic_quad_tree_flag = 1” mean a 3-type asymmetric quad tree split. In addition, if “Is_used_asymmertic_quad_tree_flag = 1” and “Is_used_triple_asymmertic_quad_tree_flag = 0”, it means a two-type asymmetric quad tree split.

The syntax element 'hor_asymmetric_flag' indicates the direction of asymmetric quad tree splitting. That is, in the case of “Is_used_asymmertic_quad_tree_flag = 1”, it may indicate whether to asymmetrically split in the horizontal or vertical direction. For example, “hor_asymmetric_flag = 1” indicates asymmetry in the horizontal direction, and “hor_asymmetric_flag = 0” indicates asymmetry in the vertical direction. Also, as another alternative, it is possible to use 'ver_asymmetric_flag'.

The syntax element 'width_left_asymmetric_flag' indicates another direction of asymmetric quad tree splitting. That is, in the case of “Is_used_asymmertic_quad_tree_flag = 1”, it may indicate whether or not asymmetric partitioning is performed in the left or right direction in the width direction. For example, “width_left_asymmetric_flag' = 1” indicates asymmetry in the left direction of the width, and “width_left_asymmetric_flag = 0” indicates asymmetry in the right direction of width.

Also, the syntax element 'height_top_asymmetric_flag' indicates another direction of asymmetric quad tree splitting. That is, in the case of “Is_used_asymmertic_quad_tree_flag = 1”, it may indicate whether or not asymmetric partitioning is performed in the upper or lower direction in the height direction. For example, “height_top_asymmetric_flag' = 1” indicates asymmetry in the height-up direction, and “height_top_asymmetric_flag = 0” indicates asymmetry in the height-down direction.

In addition, the syntax element 'is_used_symmetric_line_flag' indicates whether a middle block is a symmetric block in the case of a 3-type asymmetric quad tree split. That is, in the case of “Is_used_asymmertic_quad_tree_flag = 1” and “Is_used_triple_asymmertic_quad_tree_flag = 1”, it indicates whether the middle block is divided symmetrically.

Accordingly, it is possible to express the partition types shown in FIGS. 11 (a) to (k) through a combination of the syntax elements. For example, if “Is_used_asymmertic_quad_tree_flag = 0”, it means that it is divided into 4 symmetrical blocks as in the partition type of FIG. 11(a).

In addition, if “Is_used_asymmertic_quad_tree_flag = 1” and “Is_used_triple_asymmertic_quad_tree_flag = 0”, it corresponds to any one of the partition types in FIGS. 11 (b) to (e). In this case, “hor_asymmetric_flag = 1” and “width_left_asymmetric_flag' = 1” mean the partition type of FIG. 11 (b). In addition, “hor_asymmetric_flag = 1” and “width_left_asymmetric_flag' = 0” mean the partition type of FIG. 11 (c). In addition, “hor_asymmetric_flag = 0” and “height_top_asymmetric_flag' = 1” mean the partition type of FIG. 11 (d). In addition, if “hor_asymmetric_flag = 0” and “height_top_asymmetric_flag' = 0” mean the partition type of FIG. 11 (e).

In addition, if “Is_used_asymmertic_quad_tree_flag = 1” and “Is_used_triple_asymmertic_quad_tree_flag = 1”, it corresponds to any one of the partition types in FIGS. 11(f) to (k). In this case, if “is_used_symmetric_line_flag = 1”, it corresponds to one of the partition types in FIGS. 11 (f) and (g), and if “is_used_symmetric_line_flag = 0”, it corresponds to one of the partition types in FIGS. 11 (h) to (k). applicable In addition, if the above “is_used_symmetric_line_flag = 1” and “hor_asymmetric_flag = 1”, it can be defined as the partition type of FIG. 11 (f), and if it is “hor_asymmetric_flag 0”, it can be defined as the partition type of FIG. 11 (g).

In addition, in the case of “Is_used_asymmertic_quad_tree_flag = 1”, “Is_used_triple_asymmertic_quad_tree_flag = 1” and “is_used_symmetric_line_flag = 0”, the partition type can be defined by “hor_asymmetric_flag”, “width_left_asymmetric_flag” and “height_top_asymmetric_flag”. For example, “hor_asymmetric_flag = 1” and “height_top_asymmetric_flag = 0” mean the partition type of FIG. 11 (h). In addition, if “hor_asymmetric_flag = 1” and “height_top_asymmetric_flag = 1”, it means the partition type of FIG. 11 (i). In addition, if “hor_asymmetric_flag = 0” and “width_left_asymmetric_flag' = 0”, it means the partition type of FIG. 11 (j). In addition, “hor_asymmetric_flag = 0” and “width_left_asymmetric_flag' = 1” mean the partition type of FIG. 11 (k).

Also, as another alternative, it is possible to indicate the partition types of FIGS. 11(a) to (k) as indexes by 'asymmetric_quadtree_partition_index'.

14 is a diagram showing a partition type in which quad-tree and triple-tree partitions are allowed as another embodiment to which the present invention is applied.

A coding block may be hierarchically partitioned based on at least one of a quad tree and a triple tree. Here, quad tree-based splitting is a method in which a 2Nx2N coding block is split into 4 NxN coding blocks (FIG. 14(a)), and triple tree-based splitting is a method in which one coding block is split into 3 coding blocks. each can mean Even if splitting based on a triple tree is performed, a coding block having a square shape may exist in a lower depth.

Splitting based on a triple tree may be performed symmetrically (FIG. 14(b)) or asymmetrically (FIG. 14(c)). Also, a coding block divided based on a triple tree may be a square block or a non-square block such as a rectangle. As an example, the partition type in which triple tree-based partitioning is allowed is symmetric 2Nx(2N/3) (horizontal direction non-square coding unit) having the same width or height, as in the example shown in FIG. 14 (b). ) or (2N/3)x2N (vertical non-square coding units). Also, as an example, a partition type in which triple tree-based partitioning is allowed may be an asymmetric partition type including coding blocks having at least different widths or heights, as in the example shown in FIG. 14 (c). there is. For example, in the asymmetrical triple tree partition form of FIG. 14 (c), at least two coding blocks 1401 and 1403 are defined to have the same width (or height) size and have a k value, and are located on both sides, and the remaining One block 1402 has a width (or height) value of 2k and can be defined to be positioned between the blocks 1401 and 1403 of the same size.

In this regard, a method of dividing a CTU or CU into three non-square sub-partitions as shown in FIG. 14 is called triple tree CU partitioning. CUs divided by triple tree partitioning may be restricted from performing additional partitioning.

15 is a flowchart of a coding block partitioning method based on quad-tree and triple-tree partitioning as another embodiment to which the present invention is applied.

It is assumed that a depth k coding block is divided into depth k+1 coding blocks. First, it is determined whether quad tree partitioning is applied to the current block of depth k (S1510). If quad tree partitioning is applied, the current block is divided into four square blocks (S1520). On the other hand, if quad tree partitioning is not applied, it is determined whether triple tree partitioning is applied to the current block (S1530). If triple tree splitting is also not applied, the current block becomes a depth k+1 coding block without splitting.

As a result of the determination in S1530, if triple tree partitioning is applied to the current block, it is checked whether symmetric triple partitioning or asymmetric triple partitioning is applied (S1540). According to the determination result of S1540, a partition type applied to the current block is determined (S1550). For example, the partition shape applied in the step S1550 may be one of the shapes of FIG. 14(b) in the case of a symmetrical type, and one of the shapes of FIG. 14(c) in the case of an asymmetric type. Through the above step S1550, according to the determined partition type, the current block is divided into three depth k+1 coding blocks (S1560).

FIG. 16 illustrates, for example, syntax elements included in a network abstraction layer (NAL) to which quad-tree and triple-tree partitioning are applied as another embodiment to which the present invention is applied. A NAL unit to which the present invention is applied may include, for example, a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), and at least one slice set (Slice).

For example, although syntax elements included in a sequence parameter set (SPS) are illustrated in FIG. 16 , it is also possible to include syntax elements in a picture parameter set (PPS) or a slice set (Slice). In addition, syntax elements commonly applied to sequence units or picture units for each syntax element may be included in a sequence parameter set (SPS) or a picture parameter set (PPS). On the other hand, a syntax element applied only to a corresponding slice is preferably included in a slice set (Slice). Therefore, it can be selected in consideration of encoding performance and efficiency.

The syntax element 'quad_split_flag' indicates whether a coding block is split into 4 coding blocks. 'triple_split_flag' may indicate whether a coding block is split into three coding blocks. When a coding block is split into three coding blocks, 'is_hor_split_flag' indicating whether the coding block is divided in a vertical direction or a horizontal direction may be signaled. If “is_hor_split_flag = 1”, the horizontal direction can be defined as representing the vertical direction if “is_hor_split_flag = 0”.

In addition, as another alternative, 'isUseTripleTreeFlag' indicates whether triple tree partitioning is applied to the current block, and also, as a syntax element indicating the splitting direction of the coding block, 'hor_triple_flag' indicates whether the coding block is split in the horizontal direction. can indicate For example, “hor_triple_flag = 1” indicates that the coding block is split in the horizontal direction, and “hor_triple_flag = 0” indicates that the coding block is split in the vertical direction. Alternatively, instead of 'hor_triple_flag', ver_triple_flag indicating whether the coding block is split in the vertical direction may be used to set in the same manner.

In addition, 'asymmetric_triple_tree_flag' may be defined as a syntax element indicating whether asymmetric triple tree partitioning is allowed. For example, “asymmetric_triple_tree_flag = 1” indicates that asymmetric triple tree partitioning is used, and “asymmetric_triple_tree_flag = 0” indicates that asymmetric triple tree partitioning is not used. On the other hand, when triple tree partitioning is not used in units of pictures or slices, 'asymmetric_triple_tree_flag' may not be signaled and its value may be set to 0.

Therefore, it is possible to express the partition types shown in FIGS. 14 (a) to (c) through a combination of the syntax elements. For example, if “isUseTripleTreeFlag = 0”, it means that it is divided into 4 symmetrical blocks as in the partition form of FIG. 14(a).

In addition, if “isUseTripleTreeFlag = 1” and “asymmetric_triple_tree_flag = 0”, it corresponds to one of the partition types in FIG. 14 (b). At this time, if “hor_triple_flag = 1”, it is defined as meaning the (2N/3)x2N partition type in FIG. 14 (b). If “hor_triple_flag = 0”, it can be defined as meaning a 2Nx (2N/3) partition type in FIG. 14 (b).

In addition, if “isUseTripleTreeFlag = 1” and “asymmetric_triple_tree_flag = 1”, it corresponds to one of the partition types in FIG. 14 (c). At this time, if “hor_triple_flag = 1”, it is defined as meaning the left partition type of FIG. 14 (c). If “hor_triple_flag = 0”, it can be defined as meaning the right partition type in FIG. 14 (c).

Also, as another alternative, it is possible to indicate the partition types of FIGS. 14(a) to (c) as indexes by 'asymmetric_tripletree_partition_index'.

17 is another embodiment to which the present invention is applied, and is a diagram illustrating a partition type in which multi-tree partitioning is allowed.

A method of partitioning a CTU or CU using at least one of the aforementioned quad-tree partitioning, binary partitioning, or triple-tree partitioning is called multi-tree CU partitioning. A CTU or CU may be partitioned using any N partitions among the above-described examples. Specifically, for example, CTU or CU may be partitioned using 9 partitioning as shown in FIG. 17 .

Partitioning may be performed using quad tree partitioning, binary tree partitioning, or triple tree partitioning in units of sequences or pictures, or CTUs or CUs may be partitioned using any one or both of them.

Quad tree partitioning is used by default, and binary tree partitioning and triple tree partitioning can be optionally used. In this case, whether binary tree partitioning and/or triple tree partitioning is used may be signaled in a sequence header (sequence parameter set) or a picture header (picture parameter set).

Alternatively, quad-tree partitioning and triple-tree partitioning can be used as default, and binary tree partitioning can be optionally used. For example, a syntax isUseBinaryTreeFlag indicating whether binary tree partitioning is used may be signaled in a sequence header. If the isUseBinaryTreeFlag value is 1, the CTU or CU can be partitioned using binary tree partitioning in the current sequence. A syntax isUseTripleTreeFlag indicating whether triple tree partitioning is used may be signaled in the sequence header. If the isUseTripleTreeFlag value is 1, the CTU or CU can be partitioned using triple tree partitioning in the current sequence header.

The partition type divided by multi-tree partitioning can be limited to, for example, 9 basic partitions shown in FIGS. 17 (a) to (i). 17 (a) shows a quad tree partition type, (b) to (c) shows a symmetric binary tree partition type, (d) to (e) shows an asymmetric triple tree partition type, and (f) to (i) shows the form of an asymmetrical binary tree partition. In relation to each partition type shown in FIG. 17, since it is the same as described above, a detailed description thereof will be omitted.

In addition, as another alternative, it can be extended to further include 12 partitions shown in FIGS. 18 (j) to (u) as a partition type divided by multi-tree partitioning. 18 (j) to (m) show asymmetric quad tree partition types, (n) to (s) show three types of asymmetric quad tree partition types, and (t) to (u) show symmetric triple tree partition types. Express. In relation to each partition type shown in FIG. 18, since it is the same as described above, a detailed description thereof will be omitted.

19 is a flowchart of a coding block partitioning method based on multi-tree partitioning as another embodiment to which the present invention is applied.

It is assumed that a depth k coding block is divided into depth k+1 coding blocks. First, it is determined whether quad tree partitioning is applied to the current block of depth k (S1910). If quad tree splitting is not applied, it is determined whether binary tree splitting is applied to the current block (S1950). Also, if binary tree splitting is not applied, it is determined whether triple tree splitting is applied to the current block (S1990). If, as a result of the determination in step S1950, triple tree splitting is not applied, the current block becomes a depth k+1 coding block without splitting.

Here, as a result of the determination in step S1910, if quad tree splitting is applied, whether symmetric or asymmetric quad tree splitting is checked (S1920). Thereafter, the block partition type of the current block is determined by checking the partition information (S1930), and the current block is divided into four blocks according to the determined partition type (S1940). For example, when a symmetrical quad tree is applied, it is partitioned in the form of a partition in FIG. 17 (a). In addition, when an asymmetrical quad tree is applied, it is divided into any one of the partition types of FIGS. 18 (j) to (m). Alternatively, when a 3-type asymmetric quad tree is applied, it is divided into any one of the partition types of FIGS. 18 (n) to (s). However, as described above, if only the basic partition type of FIG. 17 is applied to the multi-tree partition type, only the symmetric square block of FIG. 17 (a) can be applied without determining whether the quad tree is asymmetric.

In addition, as a result of the determination in step S1950, if binary tree splitting is applied, it is checked whether symmetric or asymmetric binary tree splitting is performed (S1960). Thereafter, the block partition type of the current block is determined by checking the partition information (S1970), and the current block is divided into two blocks according to the determined partition type (S1980). For example, when a symmetrical binary tree is applied, it is divided into any one of the partitions of FIGS. 17 (b) and (c). In addition, when an asymmetric binary tree is applied, it is divided into any one of the partition types of FIGS. 17 (f) to (i).

In addition, as a result of the determination in step S1990, if triple tree splitting is applied, whether symmetric or asymmetric triple tree splitting is checked (S1960). Thereafter, the block partition type of the current block is determined by checking the partition information (S1970), and the current block is divided into three blocks according to the determined partition type (S1980). For example, when an asymmetrical triple tree is applied, partitioning is performed in one of the partitions of FIGS. 17 (d) and (e). In addition, when a symmetric binary tree is applied, it is divided into any one of the partition types of FIGS. 18 (t) to (u). However, as described above, if only the basic partition type of FIG. 17 is applied to the multi-tree partition type, whether or not the triple tree is asymmetric is not determined, and the predefined asymmetric triple block of 17 (d) and (e) can only be applied.

. As a syntax element representing multi-tree partitioning, 'is_used_Multitree_flag' indicating whether multi-tree partitioning is enabled can be defined. In addition, it is possible to use the syntax elements shown and described in FIGS. 10, 13, and 16 as information for determining a multi-tree partition type.

20 illustrates types of pre-defined intra prediction modes in an image encoder/decoder as an embodiment to which the present invention is applied.

The image encoder/decoder may perform intra prediction using any one of pre-defined intra prediction modes. The pre-defined intra prediction mode for intra prediction may include a non-directional prediction mode (eg, planar mode, DC mode) and 33 directional prediction modes.

Alternatively, a greater number of directional prediction modes than 33 directional prediction modes may be used to increase the accuracy of intra prediction. That is, M extended directional prediction modes may be defined by further subdividing angles of directional prediction modes (M>33), and a predetermined angle may be used by using at least one of 33 pre-defined directional prediction modes. It is also possible to derive and use a directional prediction mode with .

More intra prediction modes than the 35 intra prediction modes shown in FIG. 20 may be used. For example, by further subdividing the angle of the directional prediction mode or by decoding a directional prediction mode having a predetermined angle using at least one of a predetermined number of predefined directional modes, more than 35 intra prediction modes can be obtained. Intra prediction mode can be used. In this case, using more intra prediction modes than 35 intra prediction modes may be referred to as an extended intra prediction mode.

21 is an example of an extended intra prediction mode, and the extended intra prediction mode may consist of two non-directional prediction modes and 65 extended directional prediction modes. The same may be used for each component, or different numbers of intra prediction modes may be used for each component. For example, 67 extended intra prediction modes may be used in the luminance component, and 35 intra prediction modes may be used in the chrominance component.

Alternatively, intra prediction may be performed using different numbers of intra prediction modes according to color difference formats. For example, in the case of the 4:2:0 format, intra prediction is performed using 67 intra prediction modes in the luminance component, and 35 intra prediction modes can be used in the chrominance component, and in the case of the 4:4:4 format , intra prediction may be used by using 67 intra prediction modes in both the luminance component and the chrominance component.

Alternatively, intra prediction may be performed using different numbers of intra prediction modes according to the size and/or shape of a block. That is, intra prediction may be performed using 35 intra prediction modes or 67 intra prediction modes according to the size and/or shape of the PU or CU. For example, if the size of the CU or PU is smaller than 64x64 or an asymmetric partition, intra prediction can be performed using 35 intra prediction modes, and the size of the CU or PU is equal to or greater than 64x64 In this case, intra prediction may be performed using 67 intra prediction modes. 65 directional intra prediction modes may be allowed in Intra_2Nx2N, and only 35 directional intra prediction modes may be allowed in Intra_NxN.

For each sequence, picture or slice, the size of a block to which the extended intra prediction mode is applied may be set differently. For example, in the first slice, the extended intra prediction mode is set to be applied to blocks larger than 64x64 (eg, CU or PU), and in the second slice, the extended intra prediction mode is set to be applied to blocks larger than 32x32. can Information indicating the size of a block to which the extended intra prediction mode is applied may be signaled per sequence, picture, or slice unit. For example, information indicating the size of a block to which the extended intra prediction mode is applied may be defined as 'log2_extended_intra_mode_size_minus4' obtained by subtracting the integer 4 after taking a log value of the size of the block. For example, a value of log2_extended_intra_mode_size_minus4 of 0 indicates that the extended intra prediction mode can be applied to a block having a size of 16x16 or greater or a block having a size of greater than 16x16, and a value of log2_extended_intra_mode_size_minus4 of 1 indicates that a block having a size of 32x32 or greater It may indicate that the extended intra prediction mode can be applied to a block having , or a block having a size larger than 32x32.

As described above, the number of intra prediction modes may be determined in consideration of at least one of a chrominance component, a chrominance format, and a size or shape of a block. In addition to the described example, intra prediction mode candidates (eg, the number of MPMs) used to determine the intra prediction mode of a block to be encoded/decoded are also dependent on at least one of a chrominance component, a chrominance format, and a size or shape of a block. may be determined accordingly. A method of determining an intra prediction mode of a block to be encoded/decoded and a method of performing intra prediction using the determined intra prediction mode will be described with reference to drawings to be described later.

22 is a flowchart schematically illustrating an intra prediction method as an embodiment to which the present invention is applied.

Referring to FIG. 22 , an intra prediction mode of a current block may be determined (S2200).

Specifically, the intra prediction mode of the current block may be derived based on the candidate list and index. Here, the candidate list includes a plurality of candidates, and the plurality of candidates may be determined based on intra prediction modes of neighboring blocks adjacent to the current block. Neighboring blocks may include at least one of blocks located on the top, bottom, left, right, or corner of the current block. The index may specify any one of a plurality of candidates belonging to the candidate list. A candidate specified by the index may be set as an intra prediction mode of the current block.

An intra prediction mode used by neighboring blocks for intra prediction may be set as a candidate. Also, an intra prediction mode having a direction similar to that of an intra prediction mode of a neighboring block may be set as a candidate. Here, an intra prediction mode having a similar direction may be determined as a value obtained by adding or subtracting a predetermined constant value from the intra prediction mode of a neighboring block. The predetermined constant value may be an integer of 1, 2 or more.

The candidate list may further include a default mode. The default mode may include at least one of a planar mode, a DC mode, a vertical mode, and a horizontal mode. The default mode may be adaptively added in consideration of the maximum number of candidates that can be included in the candidate list of the current block.

The maximum number of candidates that can be included in the candidate list may be 3, 4, 5, 6 or more. The maximum number of candidates that can be included in the candidate list may be a fixed value pre-set in the video encoder/decoder or may be variably determined based on attributes of the current block. Attributes may mean the location/size/shape of a block, the number/type of intra prediction modes available for a block, chrominance properties, chrominance format, and the like. Alternatively, information representing the maximum number of candidates that can be included in the candidate list may be signaled separately, and the maximum number of candidates that can be included in the candidate list may be variably determined using this information. Information representing the maximum number of candidates may be signaled at at least one of a sequence level, a picture level, a slice level, or a block level.

When the extended intra prediction mode and the pre-defined 35 intra prediction modes are selectively used, the intra prediction modes of neighboring blocks are converted into indexes corresponding to the extended intra prediction modes, or corresponding to the 35 intra prediction modes. It is possible to derive a candidate by converting to an index that A pre-defined table may be used for index conversion, or a scaling operation based on a predetermined value may be used. Here, the pre-defined table may define a mapping relationship between different intra prediction mode groups (eg, an extended intra prediction mode and 35 intra prediction modes).

For example, if the left neighboring block uses 35 intra prediction modes and the intra prediction mode of the left neighboring block is 10 (horizontal mode), it is converted from the extended intra prediction mode to index 16 corresponding to the horizontal mode. can

Alternatively, if the upper neighboring block uses the extended intra prediction mode and the intra prediction mode index of the upper neighboring block is 50 (vertical mode), it can be converted from 35 intra prediction modes to index 26 corresponding to the vertical mode. there is.

Based on the intra prediction mode determination method described above, the intra prediction mode may be independently derived for each of the luminance component and the chrominance component, or the chrominance component may be derived depending on the intra prediction mode of the luminance component.

Specifically, the intra prediction mode of the chrominance component may be determined based on the intra prediction mode of the luminance component as shown in Table 1 below.

Intra_chroma_pred_mode [xCb][yCb] IntraPredModeY[xCb][yCb] 0 26 10 One X(0<=X<=34) 0 34 0 0 0 0 One 26 34 26 26 26 2 10 10 34 10 10 3 One One One 34 One 4 0 26 10 One X

In Table 3, intra_chroma_pred_mode means information signaled to specify an intra prediction mode of a chroma component, and IntraPredModeY indicates an intra prediction mode of a luminance component. Referring to FIG. 22, a reference sample for intra prediction of a current block may be derived (S2210).

Specifically, reference samples for intra prediction may be derived based on neighboring samples of the current block. The neighboring sample may refer to a reconstructed sample of the above-described neighboring block, which may be a reconstructed sample before the in-loop filter is applied or a reconstructed sample after the in-loop filter is applied.

A neighboring sample reconstructed before the current block may be used as a reference sample, or a neighboring sample filtered based on a predetermined intra filter may be used as a reference sample. Filtering neighboring samples using an intra filter may be referred to as reference sample smoothing. The intra filter may include at least one of a first intra filter applied to a plurality of neighboring samples located on the same horizontal line and a second intra filter applied to a plurality of neighboring samples located on the same vertical line. Either the first intra filter or the second intra filter may be selectively applied according to the position of the neighboring sample, or the two intra filters may be overlapped. In this case, the filter coefficient of at least one of the first intra filter and the second intra filter may be (1,2,1), but is not limited thereto.

The filtering may be adaptively performed based on at least one of an intra prediction mode of the current block and a size of a transform block of the current block. For example, filtering may not be performed when the intra prediction mode of the current block is a DC mode, a vertical mode, or a horizontal mode. When the size of the transform block is NxM, filtering may not be performed. Here, N and M may be the same or different values, and may be any one of 4, 8, 16 or more values. For example, when the size of a transform block is 4x4, filtering may not be performed. Alternatively, filtering may be selectively performed based on a comparison result between the difference between the intra prediction mode and the vertical mode (or horizontal mode) of the current block and a pre-defined threshold. For example, filtering may be performed only when the difference between the intra prediction mode and the vertical mode of the current block is greater than a threshold value. As shown in Table 4, the threshold may be defined for each transform block size.

8x8 transform 16x16 transform 32x32 transform Threshold 7 One 0

The intra filter may be determined as one of a plurality of intra filter candidates pre-defined in an image encoder/decoder. To this end, a separate index specifying an intra filter of the current block among a plurality of intra filter candidates may be signaled. Alternatively, the intra filter may be determined based on at least one of the size/shape of the current block, the size/shape of the transform block, filter strength information, or variation of neighboring samples.

Referring to FIG. 22 , intra prediction may be performed using an intra prediction mode of a current block and a reference sample (S2220).

That is, the prediction sample of the current block may be obtained using the intra prediction mode determined in S2200 and the reference sample derived in S2210. However, in the case of intra prediction, since boundary samples of neighboring blocks are used, the quality of the predicted image may deteriorate. Therefore, a correction process for the predicted sample generated through the above-described prediction process may be further performed, and will be examined in detail with reference to FIGS. 23 and 24 below. However, the correction process to be described later is not limited to being applied only to intra-prediction samples, and may be applied to inter-prediction samples or reconstructed samples as well.

23 illustrates a method of correcting a predicted sample of a current block based on difference information of neighboring samples as an embodiment to which the present invention is applied.

A prediction sample of the current block may be corrected based on difference information of a plurality of neighboring samples of the current block. The correction may be performed on all prediction samples belonging to the current block or only on prediction samples belonging to a predetermined partial area. The partial area may be one row/column or a plurality of rows/columns, and may be a pre-set area for correction in an image encoder/decoder. For example, correction may be performed on one row/column located at the boundary of the current block or a plurality of rows/columns from the boundary of the current block. Alternatively, the partial region may be variably determined based on at least one of the size/shape of the current block and the intra prediction mode.

The neighboring samples may belong to at least one of the neighboring blocks located at the upper, left, and upper left corners of the current block. The number of neighboring samples used for correction may be 2, 3, 4 or more. Positions of neighboring samples may be variably determined according to positions of prediction samples that are to be corrected in the current block. Alternatively, some of the neighboring samples may have fixed positions regardless of the position of the prediction sample to be corrected, and the others may have variable positions according to the position of the prediction sample to be corrected.

The difference information of neighboring samples may mean a difference sample between neighboring samples, or may mean a value obtained by scaling the difference sample by a predetermined constant value (eg, 1, 2, 3, etc.). Here, the predetermined constant value may be determined in consideration of the position of the prediction sample to be corrected, the position of the column or row to which the prediction sample to be corrected belongs, and the position of the prediction sample within the column or row.

For example, when the intra prediction mode of the current block is a vertical mode, using a difference sample between the neighboring sample p(-1,y) adjacent to the left boundary of the current block and the upper-left neighboring sample p(-1,-1) A final prediction sample may be obtained as shown in Equation 1 below.

For example, when the intra prediction mode of the current block is a horizontal mode, using a difference sample between the neighboring sample p(x,-1) adjacent to the upper boundary of the current block and the upper left neighboring sample p(-1,-1) A final prediction sample may be obtained as shown in Equation 2 below.

For example, when the intra prediction mode of the current block is a vertical mode, using a difference sample between the neighboring sample p(-1,y) adjacent to the left boundary of the current block and the upper-left neighboring sample p(-1,-1) A final prediction sample may be obtained. In this case, the differential sample may be added to the predicted sample, or the differential sample may be scaled by a predetermined constant value and then added to the predicted sample. A predetermined constant value used for scaling may be determined differently according to columns and/or rows. For example, prediction samples may be corrected as shown in Equations 3 and 4 below.

For example, when the intra prediction mode of the current block is a horizontal mode, using a difference sample between the neighboring sample p(x,-1) adjacent to the upper boundary of the current block and the upper left neighboring sample p(-1,-1) A final prediction sample can be obtained, as described above in the vertical mode. For example, prediction samples may be corrected as shown in Equations 5 and 6 below.

24 and 25 show a method of correcting a prediction sample based on a predetermined correction filter as an embodiment to which the present invention is applied.

The prediction sample may be corrected based on the neighboring samples of the prediction sample to be corrected and a predetermined correction filter. In this case, the neighboring samples may be specified by an angular line of the directional prediction mode of the current block, and may be one or more samples located on the same angular line as the prediction sample to be corrected. Also, the neighboring samples may be prediction samples belonging to the current block or reconstructed samples belonging to neighboring blocks reconstructed before the current block.

At least one of the number of taps, strength, or filter coefficient of the correction filter is the location of the prediction sample to be corrected, whether the prediction sample to be corrected is located on the boundary of the current block, the intra prediction mode of the current block, the angle of the directional prediction mode, and the surroundings. It may be determined based on at least one of the prediction mode (inter or intra mode) of the block or the size/shape of the current block.

Referring to FIG. 24, when the index is 2 or 34 in the directional prediction mode, as shown in FIG. can be obtained Here, the lower left prediction/reconstruction sample 2401 may belong to a line preceding the line to which the prediction sample 2402 to be corrected belongs, may belong to the same block as the current sample, or may belong to a neighboring block adjacent to the current block. may belong to

Filtering of the prediction sample 2402 may be performed only on a line located at a block boundary or may be performed on a plurality of lines. Correction filters in which at least one of the number of filter taps or filter coefficients is different for each line may be used. For example, a (1/2,1/2) filter can be used for the first left line 2402 closest to the block boundary, and a (12/16, 4/16) filter can be used for the second line 2403. For the third line 2404, a (14/16, 2/16) filter may be used, and for the fourth line 2405, a (15/16, 1/16) filter may be used.

Alternatively, in the directional prediction mode, when the index is between 3 and 6 or between 30 and 33, filtering can be performed at the block boundary as shown in FIG. 25, and prediction samples can be corrected using a 3-tap correction filter. there is. Filtering can be performed using a 3-tap correction filter that takes as input the lower left sample 2502 of the predicted sample 2501 to be corrected, the lower sample 2503 of the lower left sample, and the predicted sample 2501 to be corrected. there is. Positions of neighboring samples used in the correction filter may be determined differently based on the directional prediction mode. Filter coefficients of the correction filter may be determined differently according to the directional prediction mode.

Different correction filters may be applied depending on whether the neighboring block is in an inter mode or an intra mode. When neighboring blocks are coded in intra mode, a filtering method that gives more weight to predicted samples than when coded in inter mode can be used. For example, when the intra prediction mode is 34, a (1/2,1/2) filter is used when the neighboring block is coded in the inter mode, and (4/16) when the neighboring block is coded in the intra mode. , 12/16) filters can be used.

Depending on the size/shape of the current block (eg, coding block or prediction block), the number of filtered lines in the current block may be different. For example, if the size of the current block is smaller than or equal to 32x32, filtering is performed on only one line at the block boundary; otherwise, filtering is performed on multiple lines including one line at the block boundary. may be

In this regard, FIGS. 24 and 25 are described based on the case of using the 35 intra prediction modes mentioned in FIG. 20 , but may be equally/similarly applied to the case of using the extended intra prediction mode of FIG. 21 .

When the intra prediction mode of the current block is the directional prediction mode, the intra prediction of the current block may be performed based on the directionality of the directional prediction mode. As an example, FIG. 26 illustrates intra direction parameters (intraPredAng) from Mode 2 to Mode 34, which are the directional intra prediction modes shown in FIG. 20 .

In FIG. 26, 33 directional intra prediction modes have been exemplarily described, but more or fewer directional intra prediction modes may be defined.

An intra-direction parameter for the current block may be determined based on a lookup table defining a mapping relationship between a directional intra-prediction mode and an intra-direction parameter. Alternatively, an intra-direction parameter for the current block may be determined based on information signaled through a bitstream.

Intra prediction of the current block may be performed using at least one of a left reference sample and an upper reference sample according to the directionality of the directional intra prediction mode. Here, the top reference sample is a reference sample having a y-axis coordinate smaller than the prediction target sample (x, 0) included in the top row in the current block (e.g., (-1, -1) to (2W-1, -1) )), and the left reference sample is reference samples having x-axis coordinates smaller than the prediction target sample (0, y) included in the leftmost column in the current block (eg, (-1, -1) to (- 1, 2H-1)).

Reference samples of the current block may be arranged in one dimension according to the directionality of the intra prediction mode. Specifically, when it is necessary to use both the top reference sample and the left reference sample during intra prediction of the current block, it is assumed that they are arranged in a row along the vertical or horizontal direction, and the reference sample of each prediction target sample can be selected. .

For example, when the intra direction parameter is a negative number (eg, in the case of intra prediction modes corresponding to Mode 11 to Mode 25 in FIG. 26 ), the top reference samples and the left reference samples are rearranged along the horizontal or vertical direction to achieve one-dimensional A reference sample group (P_ref_1D) may be configured.

27 and 28 are diagrams illustrating a one-dimensional reference sample group in which reference samples are rearranged in a row.

Whether to rearrange the reference samples vertically or horizontally may be determined according to the directionality of the intra prediction mode. When the intra direction parameter of the current block is a negative number, for example, when the intra prediction mode index is between 11 and 25, a one-dimensional wrapperance sample group obtained by rearranging the top reference samples and the left reference samples along the horizontal or vertical direction can utilize

For example, when the intra prediction mode index is between 11 and 18, as in the example shown in FIG. 27, the top reference samples of the current block are rotated counterclockwise so that the left reference samples and the top reference samples are in the vertical direction. It is possible to create a one-dimensional reference sample group arranged in

On the other hand, when the intra prediction mode index is between 19 and 25, as in the example shown in FIG. 28, the left reference samples of the current block are rotated clockwise so that the left reference samples and the top reference samples are A one-dimensional reference sample group arranged in a horizontal direction may be created.

When the intra direction parameter of the current block is not a negative number, intra prediction of the current block may be performed using only left reference samples or top reference samples. Accordingly, for intra prediction modes in which the intra direction parameter is not a negative number, a one-dimensional reference sample group may be generated using only the left reference sample or the upper reference sample.

Based on the intra-direction parameter, a reference sample determination index iIdx for specifying at least one reference sample used to predict a prediction target sample may be derived. In addition, a weight-related parameter i _fact used to determine a weight applied to each reference sample may be derived based on the intra-direction parameter. For example, Equations 7 and 8 below show examples of deriving parameters related to a reference sample determination index and a weight.

Based on the reference sample determination index, at least one reference sample may be specified for each prediction target sample. For example, based on the reference sample determination index, a location of a reference sample within a one-dimensional reference sample group for predicting a sample to be predicted within a current block may be specified. Based on the reference sample at the specified location, a prediction image (ie, prediction sample) for the prediction target sample may be generated.

A plurality of intra prediction modes may be used to perform intra prediction on a current block. For example, a different intra prediction mode or a different directional intra prediction mode may be applied for each prediction target sample in the current block. Alternatively, a different intra prediction mode or a different directional intra prediction mode may be applied to each sample group within the current block. Here, the predetermined sample group may represent a subblock having a predetermined size/shape, a block including a predetermined number of prediction target samples, or a predetermined region. The number of sample groups may be variably determined according to the size/shape of the current block, the number of samples to be predicted included in the current block, the intra prediction mode of the current block, etc. may be Alternatively, it is also possible to signal the number of sample groups included in the current block through a bitstream.

A plurality of intra prediction modes for the current block may be expressed as a plurality of intra prediction mode combinations. For example, the plurality of intra prediction modes may be expressed as a combination of a plurality of non-directional intra prediction modes, a combination of a directional prediction mode and a non-directional intra prediction mode, or a combination of a plurality of directional intra prediction modes. Alternatively, the intra prediction mode may be encoded/decoded for each unit to which a different intra prediction mode is applied.

Considering the intra prediction mode of the current block, when it is determined that the prediction target sample is not predicted with only one reference sample, prediction of the prediction target sample may be performed using a plurality of reference samples. Specifically, according to the intra prediction mode of the current block, prediction of a prediction target sample may be performed by interpolating a reference sample at a predetermined position and a neighboring reference sample adjacent to the reference sample at a predetermined position.

For example, if a virtual angular line according to the angle of the intra prediction mode or the slope of the intra prediction mode does not pass through an integer pel (ie, a reference sample at an integer position) in the one-dimensional reference sample group , A prediction image for a prediction target sample may be generated by interpolating a reference sample lying on a corresponding angle line and a reference sample adjacent to the left/right or above/below the reference sample. As an example, Equation 9 below shows an example of generating a prediction sample P(x, y) for a prediction target sample by interpolating two or more reference samples.

Coefficients of the interpolation filter may be determined based on the weight related parameter i _fact . For example, coefficients of the interpolation filter may be determined based on a distance between a fractional pel and an integer pel (ie, integer positions of respective reference samples) located on an angular line.

Considering the intra prediction mode of the current block, if the prediction target sample can be predicted with only one reference sample, a prediction image for the prediction target sample is generated based on the reference sample specified by the intra prediction mode of the current block. can

For example, when a virtual angular line according to the angle of the intra prediction mode or the slope of the intra prediction mode passes through an integer pel (ie, a reference sample at an integer position) in a one-dimensional reference sample group, an integer A prediction image for a prediction target sample may be generated by copying a reference sample at a pel position or considering a position between a reference sample at an integer pel position and a prediction target sample. For example, Equation 10 below copies the reference sample P_ref_1D(x+iIdx+1) in the one-dimensional reference sample group specified by the intra prediction mode of the current block to obtain a predicted image P(x, y) for the prediction target sample It shows an example of creating .

In the case of using a 4-tap interpolation filter, a prediction image may be generated as shown in Equation 11 below.

In Equation 11, when P_ref_1D(x+idx-1) is a sample outside the coding unit boundary, it can be replaced with P_ref_1D(x+idx). In the same way, when P_ref_1D(x+idx+2) in Equation 11 is a sample outside the coding unit boundary, it can be replaced with P_ref_1D(x+idx+1). The method of Equations 9 to 11 is called a directional intra prediction sample interpolation method.

However, applying the same interpolation filter to the various types of coding units described above may rather cause a decrease in encoding and decoding efficiency. For example, if the CU size is non-square or a prediction sample is generated using a 4 tap filter in a minimally asymmetric coding unit such as 2x1 or 16x2, the prediction image may be over-smoothed. Therefore, the present invention further proposes a method of applying different interpolation filters according to the type of coding unit.

29 is an embodiment to which the present invention is applied, and is a flowchart illustrating applying different interpolation filters according to types of coding units.

Referring to FIG. 29, first, an intra prediction mode is determined (S2900). For example, intra prediction mode determination may include the directional intra prediction mode according to FIGS. 20 and 21 described above.

Thereafter, the shape and/or size of the coding unit (CU) is checked (S2910). For example, the shape of the coding unit is divided into a square or non-square shape, a symmetrical or asymmetrical shape, and a minimally asymmetrical shape, as described in detail with reference to FIGS. 3 to 19 above. In addition, for example, the size of the coding unit, as described in detail in FIG. There are various sizes such as 8x4, 8x16, 8x32, 16x2, 16x4, 16x8, and 16x32.

Next, an interpolation filter type applied to directional intra prediction sample interpolation is determined according to the shape and/or size of the identified coding unit (S2920). For example, the interpolation filter may use a directional intra prediction sample interpolation method using different tap filters based on the width or height of the CU. Different tap filters may mean that at least one of the number of taps, filter coefficients, filter strength (strong/weak), and filtering direction (vertical/horizontal) is different. Also, the number of taps and filter coefficients may be differently determined according to filter strength. Also, as an interpolation filtering method, any one of only vertical, only horizontal, and both vertical and horizontal may be selectively performed. In addition, the filtering direction may be selected differently on a per-line (column/row) or sample-by-sample basis within the CU. Specifically, for example, if any one of the width or height of the coding unit is smaller than the reference value N, a 2-tap filter is used instead of a 4-tap filter to perform the directional intra prediction sample interpolation method, and in other CUs A directional intra prediction sample interpolation method may be performed using a 4-tap filter.

30 to 32 are exemplary flowcharts for applying different interpolation filters according to types of coding units as embodiments to which the present invention is applied.

Relatedly, FIG. 30 shows an example in which different interpolation filters are applied depending on whether a coding unit is square or non-square, and FIG. 31 shows an example in which different interpolation filters are applied according to width or height when a coding unit is non-square. 32 illustrates an example in which different interpolation filters are applied according to inter prediction modes.

Referring to FIG. 30 , it is first determined whether the coding unit is non-square (S3000). A non-square CU means a shape in which the width and height constituting the CU are different from each other.

If it is a square CU. Directional intra prediction sample interpolation is applied with an n-tap interpolation filter (S3010). On the other hand, in the case of non-square, directional intra prediction sample interpolation is applied with an m-tap interpolation filter (S3020). Here, n and m are constants greater than 0. n is a value greater than or equal to m. If n-tap and m-tap have the same number of taps, the n-tap filter and the m-tap filter may have different filter coefficients or may be classified as having different filter strengths (eg, strong/weak).

Specifically, for example, when the width and height of the CU are non-square with different values, a 2-tap filter is used instead of a 4-tap filter to perform the directional intra prediction sample interpolation method, and in other CUs, the 4-tap filter A directional intra prediction sample interpolation method may be performed using

Referring to FIG. 31 , first, it is determined whether a coding unit is non-square (S3100). If the coding unit is a non-square CU, it is determined whether at least one of the width or height of the coding unit is smaller than the reference size (S3110). For example, in the case of a 2x8 coding unit, the width size may be determined as 2, and in the case of an 8x2 coding unit, the height size may be determined as 2. Also, for example, the reference size may be set to 4, but is not limited thereto.

If, as a result of the determination in step S3110, at least one of the width or height of the CU is greater than the reference size (eg, 4), directional intra prediction sample interpolation is applied with an n-tap interpolation filter (S3120). On the other hand, if at least one of the width or height of the CU is smaller than the reference size (eg, 4), directional intra prediction sample interpolation is applied with an m-tap interpolation filter (S3130). Here, n and m are constants greater than 0. n is a value greater than or equal to m. If n-tap and m-tap have the same number of taps, the n-tap filter and the m-tap filter may have different filter coefficients or may be classified as having different filter strengths (eg, strong/weak). Alternatively, for example, an n=4 tap filter and an m=2 tap filter may be applied.

In addition, as a result of the determination in step S3100, if the coding unit is a square CU, directional intra prediction sample interpolation is applied to the n-tap interpolation filter (S3120). As another example, if the ratio of the width or height of the CU (i.e., w/h or h/w) is smaller than a specific threshold, a 2-tap filter is used instead of a 4-tap filter to perform the directional intra prediction sample interpolation method. And, in other CUs, a directional intra prediction sample interpolation method may be performed using a 4-tap filter.

Referring to FIG. 32, it is determined whether an intra prediction mode of a current coding unit is a horizontal mode or a vertical mode (S3210). An intra prediction mode having a direction similar to the horizontal intra prediction mode is called a horizontal intra prediction mode (horizontal mode), and an intra prediction mode having a direction similar to the vertical intra prediction mode is called a vertical intra prediction mode (vertical mode).

For example, when 67 intra prediction modes are used as shown in FIG. 21, intra prediction modes between MODE 11 and MODE 18 can be regarded as horizontal modes, and intra prediction modes between MODE 19 and MODE 27 are vertical modes. can be regarded as As another example, when 33 intra prediction modes are used as shown in FIG. 20, intra prediction modes between MODE 7 and MODE 13 may be regarded as horizontal modes, and intra prediction modes between MODE 23 and MODE 29 are considered as horizontal modes. It can be considered as a vertical mode.

If the coding unit corresponds to the intra-prediction horizontal mode, directional intra-prediction sample interpolation is applied with an n-tap interpolation filter (S3220). On the other hand, if the coding unit corresponds to the intra-prediction vertical mode, directional intra-prediction sample interpolation is applied with an m-tap interpolation filter (S3230). Here, n and m are constants greater than 0. n may be larger or smaller than m. Alternatively, n and m may be the same, and in this case, the n-tap filter and the m-tap filter may have different filter coefficients or different filter strengths.

In this regard, as another embodiment, in step S3210 of determining whether the intra prediction mode is a horizontal mode or a vertical mode, it is possible to determine whether to apply it according to the type of coding unit. That is, for example, step S3210 may be applied only when the coding unit is a non-square coding unit. Alternatively, step S3210 may be applied only when the coding unit is an asymmetric coding unit. Alternatively, step S3210 may be applied only when the coding unit is a minimally asymmetric coding unit. The minimally asymmetric coding unit is a type of asymmetric coding unit, and refers to a form in which the width or height of one coding unit is very short compared to the other. For example, a coding unit having a size of 2x16, 16x2, 4x32, or 32x4 may correspond to this.

Specifically, for example, in a 2x16 asymmetric coding unit, when the intra prediction mode is a horizontal intra prediction mode (horizontal mode), an n tap filter may be used, and when a vertical intra prediction mode (vertical mode) is used, On the other hand, as another example, in a 16x2 asymmetric coding unit, when the intra prediction mode is the horizontal intra prediction mode (horizontal mode), the m tap filter can be used, and the vertical intra prediction mode (vertical mode), an n-tap filter can be used. Here, n and m are constants greater than 0. n may be greater than m, or n and m may be equal, and in this case, the n-tap filter and the m-tap filter may have different filter coefficients or different filter strengths.

FIG. 33 illustrates a syntax element included in a network abstraction layer (NAL) applied to intra prediction sample interpolation as another embodiment to which the present invention is applied, as an example. A NAL unit to which the present invention is applied may include, for example, a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), and at least one slice set (Slice).

For example, although syntax elements included in a picture parameter set (PPS) are illustrated in FIG. 33 , corresponding syntax elements may be included in a sequence parameter set (SPS) or a slice set (Slice). In addition, syntax elements commonly applied to sequence units or picture units for each syntax element may be included in a sequence parameter set (SPS) or a picture parameter set (PPS). On the other hand, a syntax element applied only to a corresponding slice is preferably included in a slice set (Slice). Therefore, it can be selected in consideration of encoding performance and efficiency.

The syntax element 'PreSample_filter_flag' is information indicating whether an interpolation filter is applied to an intra prediction sample. For example, if the PreSample_filter_flag value is '1', it means that intra prediction sample interpolation is applied to the current coding block, and if the value of PreSample_filter_flag is '0', it means that intra prediction sample interpolation is not applied to the current coding block.

Also, the syntax element 'idx_PreSample_filte' is information indexing and displaying the type of interpolation filter applied to the intra prediction sample. For example, an interpolation filter type may be determined by a combination of the aforementioned 4-tap filter, 2-tap filter, filter coefficient, and filter strength, and may be used as indexing information.

Although the foregoing embodiments have been described based on a series of steps or flowcharts, this does not limit the chronological order of the invention, and may be performed simultaneously or in a different order as needed. In addition, each of the components (eg, units, modules, etc.) constituting the block diagram in the above-described embodiment may be implemented as a hardware device or software, or a plurality of components may be combined to form a single hardware device or software. may be implemented. The above-described embodiment may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. The hardware device may be configured to act as one or more software modules to perform processing according to the present invention.

100: encoder 110: picture division unit
120, 230: inter predictor 125, 235: intra predictor
130: conversion unit 135: quantization unit
200: decoder 210: entropy decoding unit
215: reordering unit 220: inverse quantization unit
225: inverse transform unit

Claims (15)

decoding residual coefficients of the current block;
inverse quantizing the residual coefficient;
determining whether to apply an inverse transform to the inverse quantized residual coefficient; and
Obtaining a residual sample from the inverse quantized residual coefficient by skipping or performing the inverse transform according to the determination,
When the current block is one of the partitions generated by dividing the coding block into two parts, skipping the inverse transform is not allowed for the current block. According to claim 1,
determining whether to divide the coding block into two parts; and
Further comprising determining whether to divide the coding block in a symmetrical form or in an asymmetrical form based on a first flag when it is determined to divide the coding block into two parts;
When it is determined to divide the coding block in a symmetrical form, the coding block is divided into two partitions of 1/2 size,
When it is determined to divide the coding block in an asymmetrical form, the coding block is divided into a 1/4 partition and a 3/4 partition. According to claim 2,
When the coding block is divided in an asymmetrical form, whether a first partition in the coding block has a larger size than a second partition in the coding block is determined based on a second flag. According to claim 1,
When the current block is one of the partitions generated by dividing the coding block into two parts, skipping the inverse transform in both vertical and horizontal directions of the current block is not allowed. obtaining a residual sample of the current block;
obtaining a transform coefficient by applying or skipping a transform to the residual sample;
quantizing the transform coefficient to obtain a quantized transform coefficient; and
Encoding the quantized transform coefficient,
When the current block is one of the partitions generated by dividing the coding block into two parts, skipping the transform is not allowed for the current block. According to claim 5,
determining whether to divide the coding block into two parts; and
Further comprising: encoding a first flag indicating whether the coding block is symmetrically divided or asymmetrically divided when it is determined to divide the coding block into halves;
When it is determined to divide the coding block in a symmetrical form, the coding block is divided into two partitions of 1/2 size,
When it is determined to divide the coding block in an asymmetrical form, the coding block is divided into a 1/4 size partition and a 3/4 size partition. According to claim 6,
When the coding block is divided in an asymmetrical form, a second flag indicating whether a first partition in the coding block is larger than a second partition in the coding block is encoded. According to claim 5,
When the current block is one of the partitions generated by dividing the coding block into two parts, skipping the transform in both vertical and horizontal directions of the current block is not allowed. A computer-readable recording medium containing a data stream generated by encoding an image,
The data stream includes information about a residual coefficient of a current block,
Inverse quantizing the residual coefficient to obtain an inverse quantized residual coefficient;
Applying or skipping an inverse transform to the inverse quantized residual coefficient to obtain a residual sample of the current block;
When the current block is one of the partitions generated by dividing the coding block into two parts, skipping the inverse transform is not allowed for the current block. delete delete delete delete delete delete

Images