RU2801823C2 - Method and device for image encoding/decoding using intra prediction - Google Patents

Abstract

FIELD: image coding and decoding.

SUBSTANCE: increased efficiency of image coding/decoding is achieved by determining the intra prediction mode of the current block, determining an initial sample for intra prediction of the current block, determining a matrix from a plurality of previously stored matrix groups based on the intra prediction mode, and predicting the current block based on the original sample and matrix.

EFFECT: increasing the efficiency of image coding/decoding.

14 cl, 10 dwg, 3 tbl

Description

FIELD OF TECHNOLOGY TO WHICH THE INVENTION RELATES

The present invention relates to a method and apparatus for encoding/decoding images.

BACKGROUND OF THE INVENTION

With the growing popularity of the Internet and portable terminals, as well as the development of information and communication technology, the use of multimedia data is growing rapidly. Therefore, in order to perform various services or tasks through image prediction in various systems, the need to improve the performance and efficiency of the image processing system is greatly increased. However, in this situation, the results of research and development are insufficient.

Thus, in the prior art method and apparatus for image encoding/decoding, image processing needs to be improved, especially image encoding or image decoding performance needs to be improved.

SUMMARY OF THE INVENTION

TECHNICAL PROBLEM

The purpose of the present invention is to improve the efficiency of coding/decoding through adaptive block division.

The purpose of the present invention is to improve the coding/decoding efficiency by matrix-based intra prediction.

The present invention provides a method and apparatus for determining an initial sample and a matrix for matrix-based intra prediction.

The present invention provides a method and apparatus for downsampling and upsampling matrix-based intra prediction.

TECHNICAL SOLUTION

In accordance with the method and device for processing video signals according to the present invention, the intra prediction mode of the current block is determined, the initial sample used for intra prediction of the current block is determined, a predetermined matrix is determined based on the intra prediction mode, and the current block is predicted based on the original sample and the matrix.

In the video signal processing method and apparatus according to the present invention, determining the initial sample may include the following operations. A neighboring region of the current block is determined and downsampled in the determined neighboring region.

In the video signal processing method and apparatus of the present invention, a neighboring area is divided into a plurality of sample groups. A sample group contains one or more samples. The representative value of the group of samples is determined as the original sample. The representative value may be one of the following: average, minimum, maximum value, mode value, and intermediate value.

In the video signal processing method and apparatus according to the present invention, the matrix is determined by additional consideration of the encoding information of the current block. The encoding information contains the size and shape of the current block and the angle or directionality of the intra prediction mode.

In the video signal processing method and apparatus according to the present invention, the current block prediction operation may include the following operation. A prediction block is generated by applying a matrix to the original sample.

In the video signal processing method and apparatus according to the present invention, prediction of the current block may further include the following operation. All or part of the prediction samples of the generated prediction block are permuted.

In the video signal processing method and apparatus according to the present invention, prediction of the current block may further include the following operation. Interpolation is performed with respect to the current block based on at least one of the following: a prediction block or reconstructed samples adjacent to the current block.

BENEFITS EFFECTS

According to the present invention, encoding/decoding efficiency can be increased by segmenting a block with a tree structure.

According to the present invention, encoding/decoding efficiency can be increased by matrix-based intra prediction.

According to the present invention, coding/decoding efficiency can be increased by downsampling or upsampling used for matrix-based intra prediction.

DESCRIPTION OF GRAPHICS

In FIG. 1 is a block diagram of an encoding apparatus according to an embodiment of the present invention.

In FIG. 2 is a block diagram of a decoding apparatus according to an embodiment of the present invention.

In FIG. 3 shows types of block separation according to an embodiment of the present invention.

In FIG. 4 shows a block segmentation method based on a tree structure according to an embodiment of the present invention.

In FIG. 5 shows a process for performing intra prediction on a current block based on a matrix, according to an embodiment of the present invention.

In FIG. 6 shows a method for determining an initial sample by downsampling a neighboring area according to an embodiment of the present invention.

In FIG. 7 illustrates a weighted average downsampling method according to an embodiment of the present invention.

In FIG. 8 shows a division of the first prediction sample and a method for interpolating the remaining areas according to an embodiment of the present invention.

In FIG. 9 shows a weighted value for a distance assigned in an interpolation operation according to an embodiment of the present invention.

In FIG. 10 shows a sequence of interpolation operations according to an embodiment of the present invention.

DETAILED DESCRIPTION

In accordance with the method and device for processing video signals according to the present invention, the intra prediction mode of the current block is determined, the initial sample used for intra prediction of the current block is determined, a predetermined matrix is determined based on the intra prediction mode, and the current block is predicted based on the original sample and the matrix.

In the video signal processing method and apparatus according to the present invention, determining the initial sample may include the following operations. A neighboring region of the current block is determined and downsampled in the determined neighboring region.

In the video signal processing method and apparatus of the present invention, a neighboring area is divided into a plurality of sample groups. A sample group contains one or more samples. The representative value of the group of samples is determined as the original sample. The representative value may be one of the following: average, minimum, maximum value, mode value, and intermediate value.

In the video signal processing method and apparatus according to the present invention, the matrix is determined by additional consideration of the encoding information of the current block. The encoding information contains the size and shape of the current block and the angle or directionality of the intra prediction mode.

In the video signal processing method and apparatus according to the present invention, prediction of the current block may include the following operation. A prediction block is generated by applying a matrix to the original sample.

In the video signal processing method and apparatus according to the present invention, prediction of the current block may further include the following operation. All or part of the prediction samples of the generated prediction sample are permuted.

In the video signal processing method and apparatus according to the present invention, prediction of the current block may further include the following operation. Interpolation is performed with respect to the current block based on at least one of the following: a prediction block or reconstructed samples adjacent to the current block.

The present invention may be changed and modified in various ways, and the present invention may have various embodiments. Specific embodiments are shown in the drawings and described in detail. However, it should be understood that these specific embodiments are not intended to limit the present invention to specific implementation methods, but are construed as including all modifications, equivalents, or substitutions that fall within the spirit and technical scope of the present invention. In the description of each graphic, like reference numerals refer to like constituent elements.

Although the terms "first", "second", etc. may be used to describe constituent elements, these constituent elements should not be limited to these terms. These terms are only used to distinguish one constituent element from another constituent element. For example, the first constituent element may be referred to as the second constituent element, and similarly the second element may be referred to as the first element without departing from the teachings of the present invention. The term "and/or" refers to combinations of a plurality of related listed objects or any of a plurality of related listed objects.

It should be understood that when a certain constituent element is referred to as "connected" or "associated" with another constituent element, the constituent element may be directly connected to or associated with another element. Other constituent elements may also be present between this constituent element and another constituent element. In contrast, when a certain constituent element is referred to as "directly connected" or "directly connected" to another constituent element, there are no other constituent elements between that constituent element and the other constituent element.

The terminology used in this document is for the purpose of describing particular embodiments only and is not intended to limit the present invention. Unless other meanings are explicitly expressed in the text, the singular forms include the plural forms. In the present description, it should also be understood that the terms "comprise" or "have" indicate the presence of the listed features, positive integers, steps, operations, constituent elements, components, or combinations thereof, which are described in the description, but do not exclude the presence or addition of one or more other features, positive integers, steps, operations, elements, components, and/or groups thereof.

Unless otherwise defined, all terms, including technical or scientific terms, have the same meaning as that which is commonly understood by those skilled in the art. Terms defined in widely used dictionaries should be interpreted as corresponding to the meanings in the relevant technical documents. If they are not clearly defined in the present description, their meanings should not be interpreted as ideal or overly formal.

Hereinafter, preferred exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Like reference numerals in the drawings refer to like constituent elements throughout this document, and unnecessary descriptions of such constituent elements will be omitted herein.

In FIG. 1 is a block diagram of an encoding apparatus according to an embodiment of the present invention.

As shown in FIG. 1, the encoding apparatus 100 includes an image separation part 110, prediction parts 120 and 125, a transform part 130, a quantization part 135, a permutation part 160, an entropy coding part 165, an inverse quantization part 140, a inverse transformation, a filtering part 150, and a storage device 155.

Each of the components shown in Fig. 1 is shown separately to represent different characteristic functions in an image encoding apparatus, and it may show that each component is represented by separate hardware. However, for ease of description, each component is listed as a separate component and included. At least two constituent parts in all constituent parts may constitute one constituent part, or one constituent part may be divided into several constituent parts to perform functions. Such exhaustive embodiments and individual embodiments of each constituent part, provided that they do not depart from the spirit of the present invention, are also included within the scope of the present invention.

Some constituent elements may be optional constituent elements only to improve performance and may not be mandatory constituent elements for performing important functions in the present invention. The present invention can be implemented by including only components that are important to an embodiment of the present invention, with the exception of components used only to improve performance. A structure containing only essential components, with the exception of visible components used only to improve performance, is within the scope of the present invention.

Part 110 for separating images may divide the input image into at least one block. At this time, the block may be a coding node (CU), a prediction node (PU), or a transform node (TU). The division may be based on at least one of the following: a quaternary tree, a binary tree, and a ternary tree. Quaternary tree is a method of dividing the top block into bottom blocks that have half the width and height of the top block using the quaternary splitting method. Binary tree is a method of dividing a top block into bottom blocks having at least one of the width or height of half of the top block by a binary splitting method. In binary tree division, a block can be not only a square shape, but also a shape other than a square by division based on the above binary tree, and have a height of half of the top block.

Further, in embodiments of the present invention, the encoding node may be a node for performing encoding, and may also be a node for performing decoding.

The prediction portions 120 and 125 may comprise an inter-picture prediction portion 120 for performing inter-picture prediction and an intra prediction portion 125 for performing intra prediction. It is determined which of the inter-picture prediction and intra prediction is performed at the prediction node, and specific information (e.g., intra prediction mode, motion vector, reference picture, etc.) can be determined according to each prediction method. Herein, the processing unit for performing prediction may be different from the processing unit for determining the prediction method and specific content. For example, a prediction method, a prediction mode, and the like can be determined by the prediction node, and prediction can also be performed by the transform node. The residual value (residual block) between the generated prediction block and the original block may be input to the transform portion 130. In addition, in the entropy encoding portion 165, prediction mode information, motion vector information, and the like, which are used for prediction, can be encoded along with the residual value and transmitted to the device for decoding. When a particular coding mode is used, the original block can also be coded in the original way and transmitted to the decoding part without the need to generate a prediction block by the prediction parts 120 and 125.

The inter-picture prediction portion 120 may predict a prediction node based on information about at least one of an image preceding the current image or an image following the current image, and may also predict the prediction node based on information about some coding areas in current image according to the situation. The inter-picture prediction part 120 may include a reference picture interpolation part, a motion prediction part, and a motion compensation part.

In the part for interpolating the reference image, information about the reference image is provided by the storage device 155, and information about pixels that are less than or equal to an integer pixel can be generated for the reference image. For the luma pixel, an 8-tap DCT-based interpolation filter with different filter coefficients can be used to generate information about pixels that are less than or equal to an integer pixel in a 1/4 pixel element. For chrominance signals, a 4-tap DCT-based interpolation filter with different filter coefficients can be used to generate information about pixels that are less than or equal to an integer pixel in a 1/8 pixel element.

The motion prediction part may perform motion prediction based on the reference image interpolated by the reference image interpolation part. Various methods such as full search block matching algorithm (FBMA), three-step search (TSS), new three-step search (NTS) algorithm, and the like can be used to calculate the motion vector. The motion vector may have a motion vector value in 1/2 or 1/4 pixel units based on the interpolated pixel. The motion prediction part can predict the current prediction node by various motion prediction methods. Various methods, such as a skip method, a merge method, an advanced motion vector prediction (AMVP) method, and the like, can be used as a motion prediction method.

The intra prediction part 125 may generate a prediction node based on information about the reference pixels adjacent to the current block, where the information about the reference pixels adjacent to the current block is information about pixels in the current image. Since the neighboring block of the current prediction node is a block on which inter-picture prediction is performed, when the reference pixels are pixels on which inter-picture prediction is performed, information about the reference pixels contained in the block on which prediction is performed is between images, can be replaced by information about the reference pixels of the adjacent block for which intra-prediction is performed. That is, when the reference pixels are unavailable, the unavailable reference pixel information can be replaced with at least one reference pixel among the available reference pixels.

The prediction mode for intra prediction may include an angle information based prediction mode in which reference pixel information is used according to the prediction direction, and an angle information non-information based prediction mode in which angle information is not used in performing prediction. The mode used to predict the luminance component and the mode used to predict the chrominance component may be different. The chrominance component may be predicted using an intra prediction mode for luminance component prediction, or using a predicted/recovered luminance component.

By the intra prediction method, it is possible to generate a prediction block after applying an adaptive intra-frame dithering (AIS) filter to the reference pixels according to the intra prediction mode. The types of AIS filter applied to the reference pixels can be different. To implement the intra prediction method, the intra prediction mode for the current prediction node may be predicted from the intra prediction mode of the prediction node adjacent to the current prediction node. When the prediction mode for the current prediction node is predicted using the mode information predicted from the neighbor prediction node, if the intra prediction mode for the current prediction node is the same as the mode for the neighbor prediction node, the predetermined flag information can be used to transmit information indicating that the intra prediction mode of the current prediction node is the same as that of the neighboring prediction node. If the intra prediction mode of the current prediction node is different from the mode of the neighboring prediction node, entropy encoding may be performed to encode information about the intra prediction mode of the current block.

In addition, prediction portions 120 and 125 may generate residual value information including a difference value between the generated prediction node and the original block. The generated residual block may be input to the transform node 130 .

The transform part 130 may use a transform such as DCT, DST, and the like to transform the residual block including the residual data. Herein, the transform type may be determined based on the intra prediction mode of the prediction node used to generate the residual block.

The quantization part 135 may perform quantization on the value converted to the frequency domain by the transform part 130 . The quantization factor may vary depending on the block or image importance. The value calculated from the quantization part 135 may be provided to the inverse quantization part 140 and the permutation part 160.

The permutation part 160 may perform permutation with respect to the coefficient value for the quantized residual block. The permutation part 160 may change the coefficients of the two-dimensional block form to a one-dimensional vector form by means of a coefficient scanning method. For example, the permutation portion 160 may use a predetermined scan type to scan from the DC coefficient to the high frequency coefficient to change it to a one-dimensional vector form.

The entropy encoding part 165 may perform entropy encoding based on the value calculated by the permutation part 160. Various coding methods such as exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), etc., can be used for entropy coding.

The entropy encoding part 165 can encode various information such as residual coefficient information and encoding node block type information, prediction mode information, split node information, prediction node information and transform node information, motion vector information, reference picture information, block interpolation information, filtering information, and the like, from the permutation part 160 and the prediction parts 120 and 125.

The entropy encoding part 165 may perform entropy encoding on the coefficient value of the encoding node input from the permutation part 160.

The inverse quantization part 140 and the inverse transform part 145 perform inverse quantization on the value quantized by the quantization part 135, and perform inverse transformation on the value converted by the transform part 130. The residual value generated by the inverse quantization part 140 and the inverse transform part 145 can be combined with the predicted prediction node by the motion analysis part, the motion compensation part, and the intra prediction part included in the prediction parts 120 and 125 to generate restored block.

The filtering portion 150 may comprise at least one of a deblocking filter, a bias correction portion, and an adaptive loop filter (ALF).

The deblocking filter can remove block distortion caused by the border between blocks in the reconstructed image. The determination of whether deblocking should be performed, whether a deblocking filter is applied to the current block, is made based on the pixels contained in multiple columns or rows contained in the block. When a deblocking filter is applied to a block, a strong filter or a weak filter can be applied according to the required degree of filtering to remove blocking. In addition, when applying a deblocking filter and performing vertical filtering and horizontal filtering, vertical filtering and horizontal filtering can be performed simultaneously.

The offset correction part may correct the offset of the original image in the pixel element of the image from which the deblocking has been removed. To perform offset correction for a specific image, a method is used in which, after dividing the pixels contained in the image into a certain number of areas, an area to be shifted is determined and an offset is applied to the area, or a method is applied in which an appropriate offset is used in accordance with the edge information of each pixel.

Adaptive loop filtering (ALF) may be performed based on a comparison value between the filtered reconstructed image and the original image. After dividing the pixels contained in the image into predetermined groups, a filter to be applied to that group is determined, and filtering can be performed differently for each group. To obtain information about whether to apply ALF, each coding unit (CU) may transmit luma signals, and the shape and coefficient of the ALF filter to be applied may be changed in accordance with each block. In addition, an ALF filter of the same shape (fixed shape) can be applied regardless of the characteristics of the applied block.

The storage device 155 may store the reconstructed blocks or images calculated by the filtering portion 150, and the stored reconstructed blocks or images may be provided to the prediction portions 120 and 125 when performing inter-picture prediction.

In FIG. 2 is a block diagram of a decoding apparatus according to an embodiment of the present invention.

As shown in FIG. 2, the decoding apparatus 200 may comprise an entropy decoding portion 210, a permutation portion 215, an inverse quantization portion 220, an inverse transform portion 225, prediction portions 230 and 235, a filtering portion 240, and a memory 245.

Each of the components shown in Fig. 2 is shown separately to represent different characteristic functions in the decoding apparatus, and it may show that each component is represented by separate hardware. However, for ease of description, each component is listed as a separate component and included. At least two constituent parts in all constituent parts may constitute one constituent part, or one constituent part may be divided into several constituent parts to perform functions. Such exhaustive embodiments and individual embodiments of each constituent part, provided that they do not depart from the spirit of the present invention, are also included within the scope of the present invention.

Part 210 for entropy decoding may perform entropy decoding on the input bitstream. For example, various techniques such as exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like, can be used to perform entropy decoding.

The entropy decoding portion 210 may perform decoding on information related to intra prediction and prediction performed between pictures performed by the encoding apparatus.

The permutation part 215 may perform permutation on the bitstream decoded in an entropy manner by the entropy decoding part 210. The coefficient represented by the one-dimensional vector form can be restored to the coefficient of the two-dimensional block form to perform the permutation. The permutation part 215 may receive information regarding the coefficient scan performed by the encoding apparatus, and may perform permutation by performing inverse scanning based on the scan order performed by the encoding apparatus.

The inverse quantization part 220 may perform inverse quantization based on the quantization parameter and the rearranged block coefficient value.

The inverse transform part 225 may perform an inverse transform on the inverse quantization transform coefficient according to a predetermined transform type. Herein, a transform type can be determined based on information about at least one of the following: prediction mode (inter-picture prediction/intra prediction), block size/shape, intra prediction mode, component type (luma/chrominance component) , separation type (QT, BT, TT, etc.), etc.

The prediction parts 230 and 235 may generate related information based on the prediction block provided in the entropy decoding part 210, and generate a prediction block based on the previously decoded block or image information provided by the memory 245.

The prediction portions 230 and 235 may comprise a prediction node determination portion, an inter-picture prediction portion, and an intra prediction portion. The prediction node determination part may receive various information such as prediction node information inputted by the entropy decoding part 210, information related to the intra prediction mode of the intra prediction method, information related to motion prediction of the inter-picture prediction method, and so on. The current coding node (CU) allocates a prediction node and determines which of the inter-picture prediction and intra-picture prediction is performed on the prediction node. The inter-picture prediction part 230 may use the information required for the inter-picture prediction of the current prediction node provided by the encoding device, and perform the inter-picture prediction for the current prediction node based on the information contained in at least at least one of the image preceding the current image or the image following the current image, including the current prediction node. Or, inter-picture prediction may also be performed on the current picture containing the current prediction node based on information about some of the reconstructed areas. For this purpose, some of the reconstructed areas may be added to the reference picture list.

To perform inter-picture prediction, it can be determined based on the encoding node whether the motion prediction method for the prediction node contained in the encoding node is a skip mode, a merge mode, an AMVP mode, and a source mode for the current picture.

Part 235 for intra prediction may generate a prediction block based on information about pixels in the current image. When the prediction node is a prediction node for which intra prediction has been performed, intra prediction may be performed based on the intra prediction mode information of the prediction node provided by the encoding apparatus. The intra prediction portion 235 may comprise an adaptive intra-frame smoothing (AIS) filter, a reference pixel interpolation portion, and a DC filter. As a component for filtering the reference pixels of the current block, the AIS filter may determine whether to apply the filter according to the prediction mode of the current prediction node. The prediction mode of the prediction node provided by the encoding device and the AIS filter information may be used to perform AIS filtering on the reference pixels of the current block. When the prediction mode of the current block is a mode not performing AIS filtering, the AIS filter may not be applied.

When the prediction node prediction mode is a prediction mode in which intra prediction is performed based on a pixel value obtained by performing interpolation on the reference pixels, the reference pixel interpolation portion may interpolate the reference pixels to generate the reference pixels in an element smaller than an integer value. When the prediction mode of the current prediction node is a prediction mode in which a prediction block is generated without interpolation of the reference pixels, the interpolation may not be performed on the reference pixels. When the prediction mode of the current block is the DC mode, the DC filter may generate a prediction block by filtering.

The reconstructed block or image may be provided to part 240 for filtering. The filtering part 240 may include a deblocking filter, a bias correction part, and an ALF.

Information relating to whether to apply a deblocking filter to a block or an image may be received from an encoding device. When a deblocking filter is applied, information regarding whether to apply a strong filter or a weak filter is received. Deblocking filter related information provided by the encoder may be received by the deblocker filter of the decoder, and deblocking filtering may be performed on the block by the decoder.

When encoding is performed, the offset correction portion may perform offset correction on the reconstructed image based on information about the type of offset correction and the offset value applied to the image.

ALF may be applied to the encoding node based on whether ALF has been applied, ALF coefficient information, and the like provided by the encoder. Information about this type of ALF may be provided by inclusion in a set of specific parameters.

The storage device 245 may store the reconstructed images or blocks as reference images or reference blocks, and may provide the reconstructed images to an output portion.

In FIG. 3 shows types of block separation according to an embodiment of the present invention.

As shown in FIG. 3, blocks a to s can be obtained according to the division setting and division method, and additional block shapes which are not shown can also be obtained.

As an example of (1), asymmetric partitioning may be allowed for tree partitioning. For example, for a binary tree, this could be a block like b or c, or it could also be a block like b~g. When the flag enabling asymmetric division is not explicitly or implicitly activated according to the encoding/decoding setting, the available candidate block may be b or c. When the flag enabling asymmetric split is enabled, the available candidate block may be b, d, e (horizontal split in this example) or c, f, g (vertical split in this example).

In the example, it is assumed that the ratio of the lengths of the left block and the right block or the top block and the bottom block in an asymmetric split is 1:3 or 3:1, which is not limited to this. A group of candidates with other ratios (eg 1:2, 1:4, 2:3, 2:5, 3:5, etc.) may also exist according to the encoding setting.

The following are various examples of split information generated by splitting based on a binary tree (a group of candidates with a ratio of 1:1, 1:3, and 3:1 in this example).

For example, in addition to a flag indicating whether splitting should be performed and a flag indicating a splitting direction, a flag indicating a splitting type may also be generated. In this document, the partition type may indicate a symmetrical partition or an asymmetric partition. When an asymmetric partition is specified as a partition type, a flag indicating a partition ratio may be generated, and an index may be assigned according to a preset. If a split ratio of 1:3 or 3:1 is supported as a candidate group, the split ratio may be selected by a 1-bit flag.

Alternatively, in addition to a flag indicating whether or not to perform a division and a flag indicating a division direction, a flag indicating a division ratio may also be generated. In an example, a candidate with a 1:1 symmetrical ratio may be included as a group of candidates for a split ratio.

The present invention assumes (when the flag enabling asymmetric partitioning is activated) that the binary tree partitioning has the structure mentioned in the previous example. Unless otherwise noted, binary tree means symmetrical binary tree.

As an example (2), additional tree-based partitioning may be allowed for tree partitioning. For example, division based on a ternary tree, a quaternary tree, an octal tree, and the like can be performed, whereby N division blocks can be obtained (3, 4, 8, and n are integers in this example). For a ternary tree, the supported block (when split into multiple blocks in this example) could be h~m. For a quad tree, the supported block may be n~p. For an octal tree, the supported block may be q. It may be implicitly determined according to the encoding/decoding setting whether to support tree division or explicitly generate associated information. In addition, according to the encoding/decoding setting, it may be used alone, or may be used in combination with binary tree, quaternary tree, or the like partitioning.

For example, blocks such as b and c can be used for a binary tree. When a binary tree and a ternary tree are used together (the example assumes that the application range of the binary tree and the application range of the ternary tree overlap), blocks such as b, c, i, or l can be used. When the flag allowing additional partitioning other than the current tree is explicitly or implicitly disabled according to the encoding/decoding setting, the available candidate block may be b or c. When the flag is enabled, the available candidate blocks may be b and i or b, h, i and j (horizontal split in this example) or the available candidate blocks may be c and l or c, k, l and m (vertical split in this example).

In the example, although it is described assuming that the ratio of the lengths of the left block, middle block and right block or top block, middle block and bottom block is 2:1:1 or 1:2:1 or 1:1:2 when partitioned based on ternary tree, the present invention is not limited to this, other ratios can also be set according to the coding.

The following is an example of split information generated from splitting based on a ternary tree (a candidate with a ratio of 1:2:1 in this example).

For example, in addition to a flag indicating whether splitting should be performed and a flag indicating a splitting direction, a flag indicating a splitting type may also be generated. In this document, the partition type may be a binary tree partition or a ternary tree partition.

In the present invention, the adaptive encoding/decoding setting according to the separation method can be applied.

For example, the division method may be determined according to the block type. For example, a quad-tree partitioning technique may be used for a coding block and a transformation block, and quad-tree and binary tree (or ternary tree, etc.) methods may be used for a prediction block.

For example, the partitioning method may be determined according to the block size. For example, a quaternary tree partitioning method may be used for some ranges between the maximum value and the minimum block value (eg, a×b~c×d when the latter is larger). In some ranges (for example, e×f~g×h), a binary tree (or ternary tree, etc.) split may be performed. Here, the range information according to the partitioning method may be explicitly generated or may be implicitly defined, and it may also be used when these ranges overlap.

For example, the division method may be determined according to the shape of the block (or the block before the division is performed). For example, when the block has a square shape, separation based on a quaternary tree and a binary tree (or a ternary tree, etc.) can be performed. Alternatively, when the block is rectangular, a binary tree (or ternary tree, etc.) split can be performed.

For example, the split setting may be determined according to the block type. For example, in tree partitioning, the coding block and the prediction block may use quaternary tree-based partitioning, and the transform block may use binary tree-based partitioning. Alternatively, the allowable split depth for a coding block may be set to m, the allowable split depth for a prediction block may be set to n, and the allowable split depth for a transform block may be set to o, and m, n, and o may be the same or different.

For example, the split setting may be determined according to the block size. For example, quaternary tree partitioning may be used in some ranges of a block (eg, a×b~c×d), binary tree partitioning may be used in some ranges (eg, e×f~g×h; it is assumed that c ×d is greater than g×h in this example), and ternary tree partitioning can be used in some ranges (e.g., i×j~k×l; in this example, g×h is assumed to be greater than or equal to k× l). In this document, ranges may include all ranges from the maximum block value to the minimum block value, and the ranges may have settings that do not overlap or settings that overlap. For example, the minimum value of some ranges may be the same as the maximum value of some ranges, or the minimum value of some ranges may be less than the maximum value of some ranges. If the ranges overlap, the split method with the larger maximum value may be in order of precedence, or information about which split method to use may be explicitly generated. That is, among the separation methods with order of priority, it can be determined whether the separation method with lower priority should be executed according to the separation result, or the separation method to be used can be determined according to the information for selecting the separation method.

For example, the split setting may be determined according to the shape of the block. For example, when the block is square, a quaternary tree partition may be used. Alternatively, when the block is rectangular, a binary tree or a ternary tree may be used for partitioning.

For example, the separation setting may be determined according to encoding/decoding information (eg, slice type, color component, encoding mode, etc.). For example, if the slice type is I, a quaternary tree (or binary tree, ternary tree) split may be used in some ranges (eg, a×b~c×d). When the slice type is P, it can be used in some ranges (eg, e×f~g×h). When the slice type is B, it can be used in some ranges (eg, i×j~k×l). In addition, when the slice type is I, the allowable split depth for quaternary tree split (or binary tree split or ternary tree split) may be set to m, when the slice type is P, the allowable split depth may be set to n, and when the slice type is B, the allowable splitting depth may be set to o, and m, n, and o may be the same or different. Some slice types may have the same setting as other slices (such as P and B slices).

As other examples, when the color component is a luminance component, the allowable splitting depth of the quaternary tree (or binary tree, ternary tree) may be set to m. When the color component is a chrominance component, it may be set to n, and m and n may be the same or different. In addition, the separation range (eg, a×b~c×d) of the quad tree (or binary tree, ternary tree) in the case where the color component is the luminance component may be the same as or different from the separation range (eg, e ×f~g×h) of a quaternary tree (or binary tree, ternary tree) in the case where the color component is a chromaticity component.

As other examples, if the coding mode is intra, the allowable split depth of the quad tree (or binary tree, ternary tree) may be m. When the coding mode is interframe, it may be n (it is assumed that n is greater than m in this example), and m and n may be the same or different. In addition, the division range based on the quad tree (or binary tree, ternary tree) in case the coding mode is intra may be the same as or different from the division range based on the quad tree (or binary tree, ternary tree) in the case where the encoding mode is interframe.

In this example, information about whether to support an adaptive split candidate group structure that depends on encoding/decoding information may be explicitly generated or implicitly determined.

The case of determining the separation method and setting the separation according to the encoding/decoding setting is described by way of examples. The examples show that variations may be present in other situations based on some instances of various factors. In addition, the separation method and separation setting may also be determined based on a combination of several factors. For example, the division method and division setting may be determined according to the block type, size and shape, encoding/decoding information, and the like.

In addition, factors related to the split method, split setting, and the like in this example may be implicitly determined, or information may be explicitly generated to determine whether adaptive situations are allowed, as mentioned in the above examples.

The split depth in the split setting indicates the number of spatial splits performed based on the original block (the split depth of the original block in this example is 0). The greater the depth of separation, the smaller blocks can be obtained as a result of separation. Depth-related settings may be set differently according to the separation method. For example, in a method for performing tree splitting, the split depth of a binary tree may be the same as the split depth of a ternary tree. The split depth of a quaternary tree may be different from the split depth of a binary tree. The appropriate depth can be used according to the type of tree.

In this example, when the corresponding splitting depth is used according to the tree type, the splitting depth for the start position (the block before splitting in this example) of the splitting of the tree can be set to 0. The splitting depth can be calculated by taking as the center the position where initially the split is performed, not based on the split range (the maximum value in this example) of each tree.

In FIG. 4 shows a block segmentation method based on a tree structure according to an embodiment of the present invention.

In this figure, the bold solid line shows the basic coding block, the thick dotted line shows the quaternary tree split boundaries, the double solid line shows the symmetric binary tree split boundaries, the solid line shows the ternary tree split boundaries, and the thin solid line shows the asymmetric binary tree split boundaries. The lines, with the exception of the thick solid line, show the separation boundaries for various separation methods. The partition settings described below (eg, partition type, partition information, partition information configuration order, etc.) are not limited to the case of this example, and there may be various modified examples.

For ease of description, the description is made with the assumption that the top left block, top right block, bottom left block, and bottom right block (N×N; 64×64), which are based on the base coding block (2N×2N; 128×128), have their own block division settings. First, it is assumed that 4 sub-blocks have already been obtained due to the split operation (split depth 0->1; that is, the split depth is increased by 1) for the original block, and to adjust the split of the quaternary tree in the case where the maximum coding block is 128× 128, the minimum coding block is 8×8 and the maximum division depth is 4, this is the usual setting for each block.

(first; upper left blocks; A1~A6)

In this example, a single tree splitting method is supported (quaternary tree in this example), the size and shape of the resulting block can be determined by setting the block splitting such as maximum coding block, minimum coding block, and splitting depth. In this example, the number of blocks obtained by division is one (perform division into two parts in the horizontal and vertical directions), division information necessary for one-time division (based on the 4M×4N block before division, the division depth is increased by 1), is a flag indicating whether splitting should be performed (in this example, if equal to 0, splitting is not performed; if equal to 1, splitting is performed), the obtained candidates may be 4M×4N and 2M×2N.

(second; upper right blocks; A7~A11)

This example supports splitting method based on multiple trees (quaternary trees and binary trees in this example), the size and shape of the resulting block can be determined through multi-block splitting settings. In this example, the maximum coding block for a binary tree is assumed to be 64x64, the minimum coding block length is 4, and the maximum split depth is 4.

In this example, the number of blocks obtained by splitting is 2 or more (in this example, 2 or 4), the split information needed for a one-time split (the split depth of the quad tree is increased by 1) is a flag indicating whether to perform a split, a flag indicating the type of split, and a flag indicating the direction of the split. Available candidates may be 4Mx4N, 4Mx2N, 2Mx4N, 4MxN/4Mx3N, 4Mx3N/4MxN, Mx4N/3Mx4N and 3Mx4N/Mx4N.

If the quad tree split range overlaps with the binary tree split range (i.e., the ranges under which quad tree split and binary tree split can be performed in the current stage) and the current block (the state before the split was performed) is block obtained by quad-tree division (blocks obtained by quad-tree division in the parent block < division depth is less than the current division depth by 1>), division information can be distinguished and configured in the following cases. That is, when blocks supported according to different partition settings can be obtained by multiple partition methods, classification can be performed as follows to generate partition information.

(1) The case where the quaternary tree partition overlaps with the binary tree partition.

Table 1

a b c d e QT 1 No split 0 0 SBT hor 0 1 0 0 ABT hor 1/4 0 1 0 1 0 ABT hor 3/4 0 1 0 1 1 SBT ver 0 1 1 0 ABT ver 1/4 0 1 1 1 0 ABT ver 3/4 0 1 1 1 1

In the table, "a" is a flag indicating whether or not to perform a quad tree split, and 1 indicates to perform a quad tree split (QT). If the flag is 0, a flag b is determined indicating whether to perform binary tree splitting. If b is 0, splitting is no longer performed on the block (no splitting). If b is 1, a binary tree split is performed.

"c" is a flag indicating the direction of the split. If c is 0, it is indicated that a horizontal split (hor) is performed. If c is 1, it is indicated that a vertical split (ver) is performed. "d" is a flag indicating the type of split. If d is 0, a symmetric binary tree (SBT) is indicated to be performed. If d is 1, an asymmetric binary tree (ABT) is indicated to be performed. Only when d is 1, the details of the split ratio (1/4 or 3/4) for the asymmetric binary tree are determined. If d is 0, in the left/right block or top/bottom block, the ratio of the left block to the top block is 1/4, and the ratio of the right block to the bottom block is 3/4. If d is equal to 1, the situation is the opposite of the above.

(2) The case where the division can be performed based on the binary tree only.

In a table, partitioning information may be represented by flags other than a, such as b–e.

For block A7 in FIG. 4, since quad tree division can be performed on blocks (A7~A11) before division is performed (that is, although quad tree division can be performed, binary tree division is performed instead of quaternary tree division), this corresponds to the case in which the separation information is generated in item (1).

On the contrary, for A8-A11, since the split based on binary tree instead of split based on quad tree has already been performed on blocks (A8~A11) before splitting (that is, in the case where blocks <A8~A11> are no longer quaternary tree division can be performed), this corresponds to the case in which division information is generated in item (2).

(third; lower left blocks; A12~A15)

This example supports splitting method based on multiple trees (quaternary tree, binary tree and ternary tree in this example), the size and shape of the resulting block can be determined through multi-block splitting settings. In this example, the maximum coding block for binary tree/ternary tree is assumed to be 64x64, the length of the minimum coding block is 4, and the maximum split depth is 4.

In this example, the number of blocks obtained by splitting is 2 or more (in this example, 2, 3, or 4), the split information needed for a one-time split is a flag indicating whether to perform split, a flag indicating the type of split , and a flag indicating the direction of the split. The available candidate may be 4Mx4N, 4Mx2N, 2Mx4N, 4MxN/4Mx2N/4MxN and Mx4N/2Mx4N/Mx4N.

If the quad tree division range overlaps with the binary tree/ternary tree division range and the current block is a block obtained by quaternary tree division, the division information can be distinguished and configured in the following cases.

(1) The case where the quad tree partition overlaps with the binary tree/quad tree partition.

table 2

a b c d QT 1 No split 0 0 B.T.hor 0 1 0 0 TT hor 0 1 0 1 B.T.hor 0 1 1 0 TT ver 0 1 1 1

In the table, "a" is a flag indicating whether to perform splitting based on a quaternary tree. If a is 1, this indicates that a quaternary tree split has been performed. If the flag is 0, a flag b indicating whether to perform binary tree splitting or ternary tree based splitting is determined. If b is 0, division is no longer performed on the block. If b is 1, a binary tree split or a ternary tree split is performed.

"c" is a flag indicating the split direction, where if c is 0, it indicates that a horizontal split is being performed. If c is 1, it is indicated that a vertical split is performed. "d" is a flag indicating the type of split. If d is 0, it is indicated that a binary tree (BT) split is performed. If d is 1, it is indicated that a ternary tree split (TT) is performed.

(2) The case where the division can be performed based on the binary tree only.

In a table, partitioning information may be represented by flags other than a, such as b-d.

For blocks A12 and A15 in FIG. 4, since quaternary tree partitioning can be performed on blocks A12~A15 before partitioning is performed, this applies to the case where partitioning information is generated in item (1).

On the contrary, for the blocks A13 and A14, since the division based on the ternary tree instead of the division based on the quaternary tree has already been performed on the blocks (A13 and A14) before the division is performed, this refers to the case where the division information is generated in step (2) .

(fourth; lower left blocks; A16~A20)

This example supports splitting method based on multiple trees (quaternary tree, binary tree and ternary tree in this example), the size and shape of the resulting block can be determined through multi-block splitting settings. In this example, the maximum coding block for binary tree/ternary tree is assumed to be 64x64, the length of the minimum coding block is 4, and the maximum split depth is 4.

In this example, the number of blocks obtained by splitting is 2 or more (in this example, 2, 3, or 4), the split information needed for a one-time split is a flag indicating whether to perform split, a flag indicating the type of split , and a flag indicating the direction of the split. The available candidate may be 4M×4N, 4M×2N, 2M×4N, 4M×N/4M×3N, 4M×3N/4M×N, M×4N/3M×4N, 3M×4N/M×4N, 4M× N/4M×2N/4M×N and M×4N/2M×4N/M×4N.

If the quad tree division range overlaps with the binary tree/ternary tree division range and the current block is a block obtained by quaternary tree division, the division information can be distinguished and configured in the following cases.

(1) The case where the quaternary tree partition overlaps with the binary tree/ternary tree partition.

Table 3

a b c d e f QT 1 No split 0 0 TT hor 0 1 0 0 SBT hor 0 1 0 1 0 ABT hor 1/4 0 1 0 1 1 0 ABT hor 3/4 0 1 0 1 1 1 TT ver 0 1 1 0 SBT ver 0 1 1 1 0 ABT ver 1/4 0 1 1 1 1 0 ABT ver 3/4 0 1 1 1 1 1

In the table, "a" is a flag indicating whether to perform splitting based on a quaternary tree. If a is 1, this indicates that a quaternary tree (QT) split has been performed. If the flag is 0, a flag b is determined indicating whether to perform binary tree splitting. If b is 0, division is no longer performed on the block. If b is 1, a binary tree split or a ternary tree split is performed.

"c" is a flag indicating the direction of the split. If c is 0, it is indicated that a horizontal split is being performed. If c is 1, it is indicated that a vertical split is performed. "d" is a flag indicating the type of split. If d is 0, it is indicated that a ternary tree split is performed. If d is equal to 1, it is indicated that a split based on a binary tree is performed. If d is equal to 1, a flag e is specified indicating the split type. Binary tree based symmetrical splitting is performed when e is 0. Binary tree based asymmetric splitting is performed when e is 1. If e is 1, information on the detailed splitting ratio of asymmetric binary tree based splitting is determined, which is the same as as in the previous example.

(2) The case where the division can be performed based on the binary tree only.

In a table, partitioning information may be represented by flags other than a, such as b-f.

Since block A20 in FIG. 4 refers to a case where quad-tree division can be performed on blocks (A16~A19) before division is performed, this refers to a case where division information is generated in item (1).

On the contrary, for blocks A16-A19, since the division based on binary tree instead of division based on quaternary tree has already been performed on the blocks (A16-A19) before the division is performed, this applies to the case where division information is generated in step (2).

In FIG. 5 shows a process for performing intra prediction on a current block based on a matrix, according to an embodiment of the present invention.

As shown in FIG. 5 determine the intra prediction mode used for intra prediction of the current block.

When intra prediction is performed on the current block, the encoder/decoder may determine an intra prediction mode. The current block may be a coding unit (CU), a prediction unit (PU), a transform unit (TU), or any of these sub-blocks.

(Embodiment 1) An intra prediction mode may be determined based on the transmitted signal information. The information may denote any of the N intra prediction modes preset in the encoder/decoder. The predefined intra prediction mode indicates all intra prediction modes of the current block that can be used. N may be a natural number less than or equal to 67 and greater than or equal to 11 (eg, 67, 35, and 11). In addition, the value of N may be determined based on the size of the current block. For example, when the current block is less than 8×8, N is determined to be 35. Otherwise, N may be determined to be 19 or 11.

(Embodiment 2) The intra prediction mode may also be determined by a default mode or an index predetermined in the encoding/decoding apparatus. The default mode may be at least one of planar mode (index 0), DC mode (index 1), horizontal mode (index 18), vertical mode (index 50), and diagonal mode (index 2, 34, 66). The index corresponds to the case where there are 67 preset intra prediction modes, and each mode may be assigned a different index according to the value of N.

(Embodiment 3) The intra prediction mode may be determined variably based on the coding information. The encoding information may contain not only information that is encoded and used for signal transmission to the encoding device, but also information obtained based on the information for signal transmission to the decoding device. The encoding information may be information associated with at least one of the current block or a neighboring block. The neighbor block contains the spatial and/or temporal neighbor block of the current block, and the spatial neighbor block may represent at least one of the left, top, top left, bottom left, or top right adjacent to the current block.

The encoding information may include the following: block size/shape, block availability, split type, number of splits, component type, prediction mode, intra prediction mode information, interframe mode, motion information, transform type, transform skip mode, non-zero information. residual coefficient, scan order, color format, filter information in the loop, etc. Block size can be represented by width or height, minimum/maximum value of width and height, sum of width and height, number of samples related to the block, etc. The availability of a block may be determined based on the position of the block, the range of the parallel processing section, the decoding order, and the like. The prediction mode may indicate information for indicating an intra mode or an inter frame mode. The intra prediction mode information may include whether the intra prediction mode is a prediction mode not based on angle information, whether the intra prediction mode is a vertical/horizontal mode, the directivity of the intra prediction mode, and information related to the number of predefined intra prediction modes in device for encoding/decoding. The interframe mode may indicate information for indicating a merge/skip mode, an AMVP mode, or an original mode of the current picture. The source mode of the current image indicates how the current block is predicted using the reconstructed regions of the current image. The current image may be the image to which the current block belongs. The current picture may be added to the reference picture list to perform inter-picture prediction, and the current picture may be adjacent to the short-term reference picture or the long-term reference picture in the reference picture list. The motion information may include the following: prediction direction flag, motion vector, reference picture index, etc.

(Embodiment 4) The intra prediction mode may also be obtained based on the MPM list and the MPM index. The MPM list contains multiple MPMs, and the MPM may be determined based on the intra prediction mode of the spatial and/or temporal neighboring block of the current block. The number of MPMs is x, and x may be 3, 4, 5, 6 or more integer.

For example, the MPM list may contain at least one of intra neighbor prediction mode, A mode, (A-n mode), (A+n mode), or default mode. The value of n can be 1, 2, 3, 4 or greater integer. Adjacent boxes may represent boxes adjacent to the left and/or top side current box. The default mode may be at least one of planar mode, DC mode, or predetermined directivity mode. The predetermined directivity mode may include at least one of horizontal mode (V mode), vertical mode (H mode), (V-k mode), (V+k mode), (H-k mode), or (H+k mode).

The MPM index may indicate an MPM that is the same as the intra prediction mode of the current block in the MPM list. That is, the MPM indicated by the MPM index can be set as the intra prediction mode of the current block.

Optionally, the intra prediction mode of the current block may be determined using any of Embodiments 1-4 above, and the intra prediction mode of the current block may be determined based on combinations of at least two of Embodiments 1-4. A predefined flag can be used for selection. Herein, a flag may be encoded by an encoder and used for signal transmission.

As shown in FIG. 5, an initial sample used for intra prediction of the current block is determined (S510).

The initial sample may be obtained from an adjacent region of the current block. The adjacent region of the current block may comprise at least one of a left region, a right region, a top region, a lower left region, an upper left region, a lower right region, or an upper right region adjacent to the current block.

An adjacent region may contain one or more sample lines. In particular, the number of sample lines related to the neighboring area is k, whereby k may be 1, 2, 3, 4, or may be a natural number greater than these numbers. The value of k may be a fixed value predetermined in the encoder/decoder, or may be variably determined based on the above encoding information. For example, when the current block has a first size (eg, 4×4, 4×8, 8×4), the adjacent area may be configured to 1 sample line. When the current block has a second size (for example, 8x8, 16x16, etc.), the adjacent area can be configured into 2 sample lines. The sampling line may be defined in the vertical direction or the horizontal direction according to the position of the adjacent area. In addition, the sample line may be in contact with the current block or may be removed from the current block by a predetermined distance in the vertical and/or horizontal direction.

Several sample lines exist continuously in the vertical and/or horizontal direction with the current block as the parent element, or may be separated from each other by a predetermined distance. As an option, if there are two sample lines in the upper region adjacent to the current block, they are appropriately called the first sample line and the second sample line in the direction from the bottom sample line of the two lines to the top. Herein, the first sample line and the second sample line may be in contact with each other or may be separated from each other by a predetermined distance. The predetermined distance can be represented by line length values i (i.e. width or height). i can be 0, 1, 2, 3, or a larger natural number. As an embodiment, if there are three sample lines in the upper portion adjacent to the current block, they are respectively called the first sample line, the second sample line, and the third sample line in the direction from the bottom sample line of the plurality of sample lines to the top. Herein, the first sample line may be in contact with the second sample line, and the second sample line may be in contact with the third sample line. Alternatively, the first sample line may also be spaced from the third sample line by the predetermined distance described above. Herein, the distance (d1) between the first sample line and the second sample line may be the same as the distance (d2) between the second sample line and the third sample line. Alternatively, d1 can be set larger than d2, and conversely, d1 can also be set smaller than d2. As an option, if there are more than 4 sample lines in the upper portion adjacent to the current block, 4 sample lines can be defined in the same manner as in the case of 3 sample lines. In addition, the embodiment can be applied not only to the sample line located in the upper portion, but also to the sample line located on the left side, and a detailed description is omitted here.

The initial sample can be obtained using all or part of the samples related to the neighboring area.

(Embodiment 1) Some samples in the adjacent area may be samples at positions predetermined in the encoder/decoder. The predetermined positions may include at least one of the leftmost sample, the rightmost sample, or the middle sample of the top line of samples. The predefined positions may include at least one of the topmost sample, the bottommost sample, or the middle sample of the left line of samples. Alternatively, the predetermined positions may include at least one of the odd-numbered samples of the top sample line and/or left sample line, or at least one of the odd-numbered samples. Alternatively, the predetermined positions may also include samples whose x-coordinates are a multiple of j among the samples of the upper sample line, or samples whose y-coordinates are a multiple of j among the samples of the left sample line. In this case, j can be 2, 3, 4 or be a larger natural number.

(Embodiment 2) Some neighbor area samples are determined based on the encoding information. The encoding information is the information described above, and detailed descriptions are omitted here.

Optionally, some samples may be designated by using any of the above Embodiment 1 or Embodiment 2, or based on a combination of Embodiment 1 and Embodiment 2. In this document, as described above, the distances between some of the samples may be set to be the same, but are not limited to this, and the distances between some of the samples can also be set to be different.

The number of some samples may be 1, 2, 3, 4 or more, which is preset in the device for encoding/decoding. In addition, the numbers of samples may be defined differently for the left portion of the neighboring region and the upper portion of the neighboring region of the current block. For example, when the width of the current box is greater than the height of the current box, the number of samples (numSamA) related to the top portion of the adjacent area may be greater than the number of samples (numSamL) related to the left portion of the adjacent area. Conversely, when the width of the current block is less than the height, numSamA may be less than numSamL. Alternatively, the number of some samples may also be variably determined based on the above encoding information.

The samples in the neighboring region may be prediction samples or reconstructed samples. Prediction samples may be obtained by intra-picture prediction or inter-picture prediction. The recovered samples may be the recovered samples before the loop filter was applied, or may be the recovered samples after the loop filter was applied.

On the other hand, the original samples may be obtained directly from the adjacent area samples (case 1) or may be obtained by downsampling the neighbor area samples (case 2). Optionally, any of Case 1 and Case 2 may be used. The selection may be made based on the above encoding information. For example, when the size of the current block is less than a predetermined threshold, the initial sample may be obtained based on case 1, otherwise, the initial sample may be obtained based on case 2. Here, the size may be represented by any of width, height, maximum/minimum the width and height values, the width-to-height ratio, or the product of the width times the height of the current block. For example, when the current block is less than 8x8, the original sample may be obtained from the neighbor area samples; otherwise, the original sample may be obtained by down-sampling the neighbor area samples. The downsampling method can be better understood with reference to FIG. 6 and FIG. 7.

As shown in FIG. 5, a matrix used for intra prediction of the current block may be determined (S520).

The matrix may be determined based on at least one of the intra prediction mode or the current block size determined in step S500. Alternatively, there may be a limitation that the matrix may be determined considering only the intra prediction mode of the current block. Or the matrix can be defined considering only the size of the current block. The size can be represented by any of the width or height, the minimum/maximum value of the width and height, the sum of the width and height, the number of samples related to the current block, and so on. However, it is not limited to this, and the matrix can be further determined considering the information about encoding the current block. The encoding information is the information described above, and detailed descriptions are omitted here.

In particular, the matrix predetermined in the encoding/decoding apparatus may be divided into a plurality of matrix groups. The plurality of matrix groups may be configured by the first matrix group, the second matrix group, ..., the m-th matrix group. In this case, m can be 2, 3, 4, 5 or be a larger natural number. Optionally, the current block may use any of a plurality of matrix groups based on the size of the current block. For example, the first matrix group may be used when the current block size is 4x4, the second matrix group may be used when the current block size is 8x4, 4x8, and 8x8, and the third matrix group may otherwise be used. The matrix group, selected based on the size of the current block, may contain one or more candidate matrices. Any of the plurality of candidate matrices may be determined by the matrix of the current block. The determination may be made based on coding information (eg, intra prediction mode) of the current block.

The number of predetermined matrices may be the same as the number of the above predetermined intra prediction modes. In addition, the number of predefined matrices may be less than the number of predefined intra prediction modes. Herein, one matrix may be associated with a plurality of intra prediction modes. For example, one matrix may be associated with 2 intra prediction modes. Herein, the number of predefined matrices may have a value that is 1/2 of the number of predefined intra prediction modes. However, it is not limited to this, and the number of intra prediction modes associated with one matrix may be 3, 4, 5, 6 or more.

As an embodiment, the correlation may be determined in terms of directivity and/or symmetry of the intra prediction mode.

The predetermined intra prediction mode may include a directivity mode having a predetermined angle. Directional modes can be divided into a first group of horizontal directive modes and a second group of vertical directive modes. The number of modes based on angle information is considered to be 65, the first group of modes may be configured as modes with an index of 2 to 34, and the second group of modes may be configured as modes with an index of 34 to 66.

The encoder/decoder only determines the matrix for the first mode group, and the second mode group may also use the matrix defined for the first mode group. In contrast, the encoder/decoder only determines the matrix for the second mode group, and the first mode group may also use the matrix defined for the second mode group. Herein, the number of defined matrices may have a value that is 1/2 of the number of predefined intra prediction modes. As an embodiment, if the number of mode groups having symmetry is x, then the number of predetermined matrices may be 1/x of the number of predetermined intra prediction modes. In this case, x can be 3, 4 or more.

The symmetry may include, given the -45° intra prediction mode as a starting point, the prediction angle symmetry between the vertical mode and the horizontal mode. Directional intra prediction mode has a prediction angle (PredAngle) that depends on different directivity. The vertical directionality mode may include, taking into account the -45° intra prediction mode as the source element, this mode and the mode with -45°<(PredAngle)≤45° along the x-direction from the mode. The horizontal directionality mode may include, taking into account the intra prediction mode with an angle of -45° as a source, modes other than this mode and having an angle of -45°<(PredAngle)≤45° along the y-direction from the mode.

As shown in FIG. 5, based on the original sample and the matrix, the current block can be predicted (S530).

When the initial sample is determined in step S510 and the matrix is determined in step S520, the encoder/decoder may predict the current block based on the original sample and the matrix.

The current block prediction step may include the following operation. The matrix is applied to the original sample to obtain a prediction sample (hereinafter referred to as the first prediction sample) of the DS block. The DS block may represent the current block or the current downsampled block. That is, the DS block may have the same size as the current block, and the DS block may also have a size that is 1/2, 1/4, 1/8, or 1/16 of the size (at least one of the width or height ) of the current block. For example, when the current block is a 4x4, 4x8, or 8x4 block, the DS block may be a 4x4 block. Or, when the current block is an 8x8, 8x16, or 16x8 block, the DS block may be a 4x4 or 8x8 block. Or, when the current block is greater than or equal to 16x16, the DS block may be an 8x8 or 16x16 block. However, the DS block is not limited to a square block shape, and may also be a block other than a square shape. Alternatively, the DS block may be limited to a square block shape. Applying a matrix may include multiplying the original sample by a weighted value obtained from the matrix.

The operation of obtaining the first prediction samples may include at least one of an operation of adding an offset value or a filtering operation.

The operation of obtaining the first prediction samples may further include the following operation. A permutation is performed on the first prediction samples. The permutation can only be performed when a plurality of intra prediction modes are associated with one matrix.

Alternatively, the permutation may be performed when the intra prediction mode of the current block belongs to the first group of horizontal directivity modes. For example, when the intra prediction mode of the current block belongs to the first horizontal directivity mode group, permutation is performed for the first prediction samples of the DS block. And, when the intra prediction mode of the current block belongs to the second group of vertical directivity modes, the permutation may not be performed for the first prediction samples of the DS block.

On the contrary, the permutation can be performed when the intra prediction mode of the current block belongs to the first vertical mode group. For example, when the intra prediction mode of the current block belongs to the first group of horizontal directivity modes, permutation may not be performed for the first prediction samples of the DS block. When the intra prediction mode of the current block belongs to the second vertical mode group, permutation may be performed for the first prediction samples of the DS block.

The permutation can be done by Equation 1 below. In an equation, x can represent a coordinate value on the x-axis, and y can represent a coordinate value on the y-axis. That is, the permutation may represent the process of assigning the first prediction sample with coordinates (x, y) to coordinates (y, x).

Equation 1

First_prediction_sample[x][y] = First_prediction_sample[y][x]

Alternatively, the permutation according to the present invention may also represent a process in which the DS block consisting of the first prediction samples is rotated at a predetermined angle. The predetermined angle may indicate 90 degrees or 180 degrees in a clockwise direction, or may indicate 90 degrees or 180 degrees in a counterclockwise direction.

The step of predicting the current block may further include the following operation. Upsampling is performed on the current block based on at least one of the neighboring recovered samples or the first prediction samples to obtain a second prediction sample.

In the upsampling process, at least one of whether to perform upsampling or how to perform upsampling may be determined based on the coding information of the current block. For example, at least one of whether to perform upsampling or how to perform upsampling may be determined based on the size of the DS block consisting of the first prediction samples and the size of the current block. The block size can be represented by the width or height, the minimum/maximum value of the width and height, the sum of the width and height, the number of samples related to the block, and so on.

It may be determined whether to perform upsampling only when the size of the DS block consisting of the first prediction samples is less than the size of the current block.

The method for performing upsampling may include the following operations. The ratio of the size of the DS block consisting of the first prediction samples to the size of the current block is used. The first prediction samples are distributed at predetermined positions in the current block. Perform interpolation with respect to the remaining areas in the current block. The remaining regions may represent regions in the current block except for the regions split for the first prediction samples. The division for the first prediction samples and the interpolation method for the remaining areas will be described in detail with reference to FIG. 8–10.

In FIG. 6 shows a method for determining an initial sample by downsampling a neighboring area according to an embodiment of the present invention.

As shown in FIG. 6, in the image (a) of FIG. 6 shows the case where the adjacent area used for intra prediction is located on the left side and the top portion adjacent to the current block. In addition, as an embodiment, the sample line located on the left side of the current block is in contact with the current block and consists of one sample line in the vertical direction. The sample line located on the top portion of the current block is in contact with the current block and consists of one sample line in the horizontal direction.

The initial sample may comprise a downsampling area formed by downsampling an adjacent area of the current block.

The downsampling area may be derived from the average, maximum, minimum, mode value, or filtered value of all or some of the samples belonging to the adjacent area.

When derived from the mean, a downsampling area may be formed by a method of assigning an average of N different samples to the samples in the downsampling area.

The N different samples may be consecutive samples or may be spaced apart. The defined interval may be an interval of one or more sample sizes. When there are multiple bins, the bin multiple may be uniform or non-uniform (where N is greater than 2 and less than the total number of samples belonging to the adjacent area). In addition, a combination of N different samples is called a sample group. Herein, the first group of samples may overlap with the second group of samples or may not overlap with the second group of samples.

For example, in FIG. 6 shows that N is 2, the two groups of samples do not overlap with each other, and the average of the two samples belonging to each group of samples is assigned to one sample in the downsampling area to perform downsampling.

Alternatively, 3 consecutive samples (S1, S2 and S3) may constitute the first group of samples, and the average of the three samples belonging to the first group of samples may be assigned to the samples (DS1) in the downsampling region. Three consecutive samples (S2, S3, and S4) may constitute the second group of samples, and the average of the three samples belonging to the second group of samples may be assigned to the samples (DS2) in the downsampling region.

Alternatively, after determining the minimum or maximum value of the two samples (S1, S2) belonging to the first group of samples, it can be assigned to the sample (DS1) in the downsampling area. Similarly, after determining the minimum or maximum value of the two samples (S3, S4) belonging to the second group of samples, it can be assigned to the sample (DS2) in the downsampling area. This can be done in a similar way when the first/second group of samples consists of three samples.

Alternatively, at the top of the adjacent sample area, samples at predetermined positions among the multiple samples belonging to the first group of samples may be assigned to a sample (DS1) in the downsampling area, and samples at predetermined positions among the multiple samples belonging to the second group of samples may be assigned to a sample (DS2) in the downsampling area. The preset positions may represent fixed positions predetermined in the encoder/decoder. For example, it can be any of the leftmost, rightmost, or middle positions. On the left side of the neighboring area, the samples at predetermined positions among the multiple samples belonging to each group of samples may also be assigned to the corresponding sample in the downsampling area. In this document, the preset positions may be any of the highest, lowest, or middle positions.

In FIG. 7 illustrates a weighted average downsampling method according to an embodiment of the present invention.

In an embodiment, the average value may be calculated by the following equation (hereinafter referred to as the first average calculation equation):

sum of samples related to the group of samples / number of samples;

or the average may be calculated by the following equation (hereinafter referred to as the second average calculation equation):

sum (weighted value × sample related to the group of samples) / number of samples.

In FIG. 7(a) shows the case where the above sample group consists of three samples. In this document, the weighted values applied for the 3 samples can be defined as a ratio of 1:2:1. As shown in FIG. 7(b), when the sample group consists of 5 samples, the weighted values can be determined as a ratio of 1:1:4:1:1. As shown in FIG. 7(c), when the sample group consists of 6 samples, the weighted value can be defined as a ratio of 1:2:1:2:2:1 or 1:2:2:1:2:1 in the Z direction with the upper left part as a starting point. In addition, although in FIG. 7(a) and FIG. 7(c) shows the weighted values applied to the top of the neighboring area, they can also be applied to the left side of the neighboring area.

The average may also include a result value obtained by applying a predetermined operation to the set of averages calculated by the first average calculation equation or the second average calculation equation. The predetermined operation may be the above first average calculation equation or the second average calculation equation. For example, if 3 samples (i.e., from the first sample to the third sample) belong to a group of samples, the mean (first value) of the first sample and the second sample and the mean (second value) of the second sample and the third sample can be calculated accordingly . The average value can be obtained from the average of the computed first value and second value.

The above downsampling method can only be applied to the top of the adjacent region. On the other hand, it can also only be applied to the left side of the adjacent area. Alternatively, the downsampling method (hereinafter referred to as the first method) of FIG. 6 can be applied to either the top or left side of the adjacent area. The downsampling method (hereinafter referred to as the second method) of FIG. 7 can be applied to another from the top or left side of the adjacent area.

In addition, considering the size/shape of the current block, at least one of the first method or the second method can optionally be used. For example, when the width of the current block is greater than a predetermined threshold, the first method may be applied to the top of an adjacent area of the current block. Otherwise, the second method can be applied. Downsampling can also be performed on the height of the current block in the same way. Or, when the current block has a shape other than a square, the first method can be applied to any of the top or left of the adjacent area, and the second method can be applied to another area. Herein, when the width of the current block is greater than the height, the first method may be applied to the top of the adjacent area, and the second method may be applied to the left side of the adjacent area. On the other hand, when the width of the current block is smaller than the height, the second method may be applied to the top of the adjacent area, and the first method may be applied to the left side of the adjacent area. When the current block is square, the same downsampling method for the top and left of the adjacent area can be used, and the downsampling method can be limited to the first method.

In FIG. 8 shows a division of the first prediction sample and a method for interpolating the remaining areas according to an embodiment of the present invention.

As shown in FIG. 8(a), prediction samples of a DS block may be assigned to prediction samples at predetermined positions in the current block. The predetermined positions may be determined considering the ratio of the current block size to the DS block size. For example, the corresponding relationship of prediction samples between the DS block and the current block can be determined by the following Equation 2.

Equation 2

First Prediction_Sample_curBLK[(x+1)×r-1][(y+1)×r-1]=First_Prediction_Sample_dsBLK[x][y]

Here, r represents the ratio of the current block size to the DS block size, and x and y represent the x and y coordinates of the first prediction sample in each DS block. First Prediction_Sample_curBLK may indicate the position of the first prediction sample in the current block, and First_Prediction_Sample_dsBLK may indicate the position of the first prediction sample in the DS block.

With regard to interpolation, as shown in FIG. 8(b), at least one element (hereinafter referred to as the interpolated original sample) of the first prediction samples split for the current block or reconstructed samples adjacent to the current block can be used to obtain samples that are not split for the first prediction samples. in the current block (hereinafter referred to as the interpolated target samples). In addition, the interpolated source sample may also contain a prediction sample generated by interpolation to the current interpolated target sample (i.e., the previous interpolated target sample).

The position and range of the reconstructed sample adjacent to the current block is the same as for the above original sample, and therefore detailed descriptions are omitted here.

According to the position of the interpolated target sample, the interpolated source sample may consist of a plurality of first prediction samples, or may consist of at least one first prediction sample and at least one reconstructed adjacent sample. Optionally, the reconstructed adjacent samples may use any of the samples having the same x-coordinate or the same y-coordinate as the interpolated target sample, or may use samples where at least one of the x-coordinate or y-coordinate is the same as for the interpolated target sample. The selection may be made based on the position of the interpolated target sample. For example, if the interpolated target sample has the same x-coordinate as the first prediction sample, the reconstructed adjacent samples may only include samples with the same x-coordinate as the interpolated target sample. In contrast, if the interpolated target sample has the same y-coordinate as the first prediction sample, the reconstructed adjacent samples may only include samples with the same y-coordinate as the interpolated target sample. Alternatively, the reconstructed adjacent samples may also include multiple samples located on the same horizontal and vertical lines as the interpolated target samples.

The interpolated target sample may be derived from a representative value of the plurality of interpolated source samples, wherein the representative value may include one of the following: average, minimum, maximum, mode value, and intermediate value.

The average value can be calculated by the following equation (hereinafter referred to as the first average calculation equation):

sum of difference initial samples / number of difference initial samples

Or, the average value can be calculated by the following equation (hereinafter referred to as the second average calculation equation):

sum (weighted value × difference original sample) / number of difference original samples

The weighted value according to the second averaging equation can be determined based on the relative/absolute distance between the interpolated target sample and the interpolated source sample, which is described in detail with reference to FIG. 9.

In FIG. 9 shows a weighted value for a distance assigned in an interpolation operation according to an embodiment of the present invention.

The weighted value according to the present invention may include a weighted value determined based on the distance between the interpolated target sample and the interpolated source sample. As an embodiment, as shown in FIG. 9, if interpolation is performed on the first interpolated target sample 910, since the ratio of the distance between the first interpolated target sample 910 and the first interpolated source sample 911 to the distance between the first interpolated target sample 910 and the second interpolated source sample 912 is 3:1, the ratio of the ratios of the weighted the values applied to the first interpolated source sample 911 and the second interpolated source sample 912 may be 1:3. If interpolation is performed on the second interpolated target sample 920, since the ratio of the distance between the second interpolated target sample 920 and the second interpolated source sample 921 to the distance between the second interpolated target sample 920 and the second interpolated source sample 922 is 1:1, the ratio of weighted values applied to the first interpolated source sample 921 and the second interpolated source sample 922 may be 1:1.

In addition, the interpolation filter according to the present invention may be directional. Orientation can be vertical, horizontal, z-shaped, diagonal, etc.

The interpolation may be performed based on a predetermined priority order. The priority order may be either a case of performing interpolation in the horizontal direction after performing interpolation in the vertical direction (first order) or a case of performing interpolation in the vertical direction after performing interpolation in the horizontal direction (second order). Or interpolation can be performed in the vertical direction and the horizontal direction at the same time (third order).

Only one of the above first order to third order orders may be used to perform the interpolation, or combinations of at least two of the first order to third order orders may be used to perform the interpolation. See the interpolation sequence in Fig. 10.

In FIG. 10 shows a sequence of interpolation operations according to an embodiment of the present invention.

In FIG. 10(a) shows the first order of FIG. 9. Specifically, interpolation may be first performed on a vertical line belonging to the first prediction samples, and then interpolation may be performed on a horizontal line based on the interpolated line and the interpolated original sample on the left side of the current block.

In FIG. 10(b) shows the second order of FIG. 9. Specifically, interpolation may be first performed on a horizontal line belonging to the first prediction samples, and then interpolation may be performed on a vertical line based on the interpolated line and the interpolated original sample at the top of the current block.

In FIG. 10(c) shows the third order of FIG. 9. First, interpolation may be performed on a vertical line and a horizontal line belonging to the first prediction samples. Interpolation can then be performed on the remaining samples that have not been interpolated. Here, interpolation may be performed on only the vertical line or only the horizontal line, or interpolation may be performed on the vertical line and the horizontal line at the same time. If the interpolation is performed on the vertical line and the horizontal line at the same time, one interpolated target sample may have the first interpolated value on the vertical line and the second interpolated value on the horizontal line at the same time. Herein, a representative value for the first interpolated value and the second interpolated value may be assigned to the interpolated target sample. Here, a representative value can be obtained from an average value, a minimum value, a maximum value, a mode value, or an intermediate value.

The interpolation order may be an order predetermined in the device for encoding/decoding, or it may optionally be determined based on the encoding information of the current block. The encoding information is the information described above and, therefore, detailed descriptions are omitted.

The order may be determined based on the block size. The block size can be represented by the width or height, the minimum/maximum value of the width and height, the sum of the width and height, the number of samples related to the block, and so on.

For example, when the size of the current block is larger than a predetermined threshold value, a first interpolation may be performed. Otherwise, a second interpolation may be performed. Conversely, when the size of the current block is less than a predetermined threshold, a second interpolation may be performed. Otherwise, the first interpolation may be performed. The threshold value can be 8, 16, 32, or a larger natural number.

Claims (46)

1. A video signal processing method performed by a decoding device, including: determining an intra prediction mode of the current block; determining an initial sample used for intra-prediction of the current block; determining a matrix from the plurality of previously stored matrix groups based on the intra prediction mode; And prediction of the current block based on the original sample and matrix, wherein the prediction of the current block based on the original sample and matrix includes: generating a prediction block by applying the matrix to the original sample. 2. The method according to p. 1, characterized in that the definition of the initial sample includes: determining the adjacent area of the current block, and performing downsampling on a certain neighboring region, wherein the neighboring region is divided into a plurality of sample groups, the sample group contains one or more samples, the representative value of the sample group is defined as the original sample, and the representative value is one of the following: average, minimum, maximum value, mode value, and intermediate value. 3. The method according to claim 1, characterized in that the matrix is additionally determined taking into account information about the encoding of the current block and encoding information contains the size and shape of the current block. 4. The method according to claim 1, characterized in that the prediction of the current block based on the original sample and the matrix further includes: performing a permutation of all or part of the prediction samples of the generated prediction block. 5. The method of claim. 1, characterized in that the prediction of the current block based on the original sample and the matrix further includes: performing interpolation with respect to the current block based on at least one of the prediction block or reconstructed samples adjacent to the current block. 6. Video signal processing method performed by the encoding device, including: determining an intra prediction mode of the current block; determining an initial sample used for intra-prediction of the current block; determining a matrix from the plurality of previously stored matrix groups based on the intra prediction mode; And prediction of the current block based on the original sample and matrix, wherein the prediction of the current block based on the original sample and matrix includes: generating a prediction block by applying the matrix to the original sample. 7. The method according to claim 6, characterized in that the definition of the initial sample includes: determining the adjacent area of the current block, and performing downsampling on a certain neighboring region, wherein the neighboring region is divided into a plurality of sample groups, the sample group contains one or more samples, the representative value of the sample group is defined as the original sample, and the representative value is one of the following: average, minimum, maximum value, mode value and intermediate value. 8. The method according to claim 6, characterized in that the matrix is additionally determined taking into account information about the encoding of the current block and encoding information contains the size and shape of the current block. 9. The method of claim 6, wherein predicting the current block based on the original sample and the matrix further comprises: performing a permutation of all or part of the prediction samples of the generated prediction block. 10. The method according to claim 6, characterized in that the prediction of the current block based on the original sample and the matrix further includes: performing interpolation with respect to the current block based on at least one of the prediction block or reconstructed samples adjacent to the current block. 11. A device for decoding for processing a video signal, comprising: a first determination node, configured to determine the intra prediction mode of the current block; a second determining node, configured to determine the original sample used for intra prediction of the current block; a third determination node, configured to determine a matrix from the plurality of previously stored matrix groups based on the intra prediction mode; And a generating node configured to generate a prediction block by applying a matrix to the original sample. 12. Decoder, containing: processor and a storage device, wherein the storage device is configured to store computer programs capable of running on the processor, and when the computer programs are run by the processor, the processor is configured to implement the method of any one of claims. 1–5. 13. An encoding device for processing a video signal, comprising: a first determination node, configured to determine the intra prediction mode of the current block; a second determining node, configured to determine the original sample used for intra prediction of the current block; a third determination node, configured to determine a matrix from the plurality of previously stored matrix groups based on the intra prediction mode; And a generating node configured to generate a prediction block by applying a matrix to the original sample. 14. An encoder containing: processor and a storage device, wherein the storage device is configured to store computer programs capable of running on the processor, and when the computer programs are run by the processor, the processor is configured to implement the method of any one of claims. 6–10.

Images