RU2812857C2 - Method and device for encoding/decoding image signals - Google Patents

Abstract

FIELD: video encoding.

SUBSTANCE: list of merge candidates for the current block is generated. One of the merger candidates included in the list of merger candidates is selected. The first affine source vector and the second affine source vector of the current block are obtained based on the first affine source vector and the second affine source vector of the specified merge candidate. An affine vector is obtained for a subblock in the current block by using the first affine source vector and the second affine source vector of the current block, where the subblock is a region with a size that is smaller than the size of the current block. The first affine source vector and the second affine merge candidate source vectors are obtained based on the motion information of a neighboring block adjacent to the current block. When a neighboring block is included in a code tree element different from the current block's code tree element, the first affine source vector and the second affine merge candidate source vectors are obtained based on the motion vectors of the bottom left subblock and bottom right subblock of the neighboring block.

EFFECT: increased efficiency of video coding.

12 cl, 35 dwg, 18 tbl

Description

BACKGROUND OF THE INVENTION

Field of technology to which the invention relates

The present invention relates to a method and apparatus for encoding and decoding a video signal.

Prior Art

As display panels become larger, more and higher quality video services are required. The biggest challenge of high-definition video services is the significant increase in data volume, and to address this issue, research is being actively conducted to improve video compression rates. As an illustrative example, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) within the telecommunications standards sector of the International Telecommunication Union (ITU-T) formed the Joint Video Coding Team (JCT-VC) in 2009. JCT-VC proposed High Efficiency Video Coding (HEVC), which is a video compression standard that has a compression efficiency approximately twice that of H.264/AVC, and it was approved as a standard on January 25, 2013. With the rapid development of high-definition video services The effectiveness of HEVC is gradually revealing its limitations.

SUMMARY OF THE INVENTION

One object of the present invention is to provide a method and apparatus for inter-image prediction using an affine model in video encoding/decoding.

Another object of the present invention is to provide a method for obtaining an affine source vector using a translational vector of a subblock and an apparatus for executing the method when encoding/decoding a video signal.

Another object of the present invention is to provide a method for obtaining an affine source vector by transforming the distance between a neighboring block and a current block as a power series of 2, and an apparatus for executing the method when encoding/decoding a video signal.

The technical problems to be solved by the present invention are not limited to the technical problems mentioned above, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

The video signal decoding and encoding method according to the present invention includes the following steps: generating a merge candidate list for the current block; identification of one of the plurality of merger candidates included in the list of merger candidates; obtaining a first affine source vector and a second affine source vector of the current block based on the first affine source vector and the second affine source vector of the specified merge candidate; obtaining an affine vector for a subblock in the current block by using the first affine source vector and the second affine source vector of the current block; and performing motion compensated prediction for the sub-block based on the affine vector. In this case, a subblock is an area with a size that is smaller than the size of the current block. In addition, the first affine source vector and the second affine source vector of the merge candidate may be obtained based on the motion information of a neighboring block adjacent to the current block.

In the video signal decoding and encoding method according to the present invention, when a neighboring block is included in a code tree element different from the code tree element of the current block, the first affine source vector and the second affine source vector of the merge candidate can be obtained based on the motion vectors of the lower left sub-block and the lower right subblock of the adjacent block.

In a method for decoding and encoding a video signal according to the present invention, a lower left sub-block may include a lower left reference sample located in a lower left corner of an adjacent block, and a lower right sub-block may include a lower right reference sample located in a lower left corner of an adjacent block.

In the video signal decoding and encoding method according to the present invention, the first affine source vector and the second affine source vector of the merge candidate can be obtained based on the value obtained based on the shift operation performed on the difference value between the motion vectors of the lower left sub-block and the lower right sub-block with using a scaling factor, and the scaling factor can be obtained based on the value obtained by adding the horizontal distance between the lower left reference sample and the lower right reference sample and the offset.

In the video signal decoding and encoding method according to the present invention, the first affine source vector and the second affine source vector of the merge candidate can be obtained based on the value obtained based on the shift operation performed on the difference value between the motion vectors of the lower left sub-block and the lower right sub-block with using a scaling factor, and the scaling factor can be obtained based on the distance between a neighboring sample adjacent to the right side of the lower right reference sample and the lower left reference sample.

In the video signal decoding and encoding method according to the present invention, the merge candidate list contains a first merge candidate obtained based on an upper neighbor block, defined as the first available block of the upper adjacent blocks located above the current block, and a second merge candidate obtained based on left neighbor block, defined as the first available block of the left neighbor blocks located to the left of the current block.

In the video signal decoding and encoding method according to the present invention, when a neighboring block is included in a code tree element the same as the code tree element of the current block, the first affine source vector and the second affine source vector of the merge candidate can be obtained based on the first affine source vector and the second affine source vector of the neighboring block.

The features briefly set forth above in relation to the present invention are merely illustrative aspects of the detailed description of the present invention, which will be described below, and do not limit the scope of the present invention.

According to the present invention, there is an effect of improving the prediction efficiency by performing an inter-image prediction method using an affine model.

According to the present invention, there is an effect of improving coding efficiency by obtaining an affine source vector using the translational motion vector of a sub-block.

According to the present invention, there is an effect of improving coding efficiency by obtaining an affine source vector by transforming the distance between a neighboring block and the current block as a power series of 2.

The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.

BRIEF DESCRIPTION OF GRAPHIC MATERIALS

In fig. 1 is a block diagram showing a video encoder according to one embodiment of the present invention.

In fig. 2 is a block diagram showing a video decoder according to one embodiment of the present invention.

In fig. 3 is a view showing a code tree base element according to one embodiment of the present invention.

In fig. 4 is a view showing various types of partitioning of a coding block.

In fig. 5 is a view showing a code tree element partitioning diagram.

In fig. 6 is a flowchart illustrating an inter-image prediction method according to one embodiment of the present invention.

In fig. 7 is a view showing the nonlinear motion of an object.

In fig. 8 is a flowchart illustrating a method for inter-image prediction based on affine motion according to one embodiment of the present invention.

In fig. 9 is a view showing an example of affine source vectors of each affine motion model.

In fig. 10 is a view showing an example of subblock affine vectors in a 4-parameter motion model.

In fig. 11 is a view showing a neighbor block that can be used to obtain a merge candidate.

In fig. 12 is a view showing an example of obtaining an affine source vector of a current block based on an affine source vector of an affine neighboring block.

In fig. 13 is a view showing an example of setting a motion vector of a sub-block as an affine source vector of an affine neighboring block.

In fig. 14-16 are views showing the positions of the reference samples.

In fig. 17 is a view showing an application example of the modified method for producing an affinity fusion vector.

In fig. 18 is a view showing an example of obtaining an affine source vector of a current block based on a non-affine neighboring block.

In fig. 19 is a view showing an example of replacing a neighboring block with another neighboring block.

In fig. 20 and 21 are views showing the flow of a motion compensation prediction method in which a plurality of fusion candidates are used.

In fig. 22 is a flowchart illustrating an intra prediction method according to one embodiment of the present invention.

In fig. 23 is a view showing reference samples included in each reference sample line.

In fig. 24 is a view showing intra prediction modes.

In fig. 25 and 26 are views showing an example of a one-dimensional array arranging reference samples in a line.

In fig. 27 is a view showing an example of angles formed between intra prediction modes based on angle information and a straight line parallel to the x-axis.

In fig. 28 is a view showing an embodiment of obtaining prediction samples when the current block has a non-square shape.

In fig. 29 is a view showing wide-angle intra prediction modes.

In fig. 30 is a flowchart illustrating the process of determining the quality indicator of a block.

In fig. 31 is a view showing preset candidate filters.

In fig. 32 is a view showing an example of dividing an image into a plurality of tiles.

In fig. 33 is a view showing an image division scheme according to the flexible tile production method.

In fig. 34 is a view showing an example of assigning a tile ID to each element of the code tree.

In fig. 35 is a view showing an example of selectively determining whether to apply a filter in a loop to each tile.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

An embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

Video encoding and decoding is performed through a block element. For example, an encoding/decoding process such as transform, quantization, prediction, in-loop filtering, reconstruction, and the like may be performed on an encoding block, a transform block, or a prediction block.

In the following, the block to be encoded/decoded will be referred to as the “current block”. For example, the current block may represent an encoding block, a transform block, or a prediction block according to the current stage of the encoding/decoding process.

In addition, it should be understood that the term "element" as used herein denotes a basic element for performing a particular encoding/decoding process, and the term "block" denotes an array of samples of a predetermined size. Unless otherwise specified, "block" and "element" may be used with the same meaning. For example, in the embodiment described below, it should be understood that a coding block and a coding element have the same meaning.

In fig. 1 is a block diagram showing a video encoder according to one embodiment of the present invention.

As shown in FIG. 1, the video encoding apparatus 100 may include an image partitioning portion 110, prediction portions 120 and 125, transform portion 130, quantization portion 135, permutation portion 160, entropy encoding portion 165, inverse quantization portion 140, 145 for deconversion, part 150 for filtering and storage device 155.

Each of the components shown in FIG. 1 is shown independently to represent characteristic functions different from each other in a video encoding apparatus, but this does not mean that each component is constituted by a configuration item of a separate hardware or a single software. That is, each component is listed as a component for convenience of explanation, and at least two of the components may be combined to form a single component, or one component may be divided into multiple components to perform a function. Combination embodiments and component separation embodiments are also included within the scope of the present invention as long as they do not depart from the spirit of the present invention.

In addition, some of the components are not essential components that perform essential functions in the present invention, but may be optional components only to enhance efficiency. The present invention can be implemented by including only the components necessary to implement the essence of the present invention, excluding components used to improve efficiency, and a structure containing only the necessary components, excluding optional components used to increase efficiency, is also included within the scope of the present invention .

The image partitioning portion 110 may partition the input image into at least one processing element. In this case, the processing element may be a prediction element (PU), a transformation element (TU), or a coding element (CU). The image partitioning portion 110 may partition an image into a plurality of encoding elements, prediction elements, and transform elements, and encode the image by selecting a plurality of encoding element, prediction element, and transform element based on a predetermined criterion (eg, a cost function).

For example, a single image may be split into multiple encoding elements. A recursive tree structure, such as a quaternary tree structure, may be used to partition the encoding elements in an image. An encoding element, split into different encoding elements using video or the largest encoding element as the root, can be split to have as many child nodes as there are split encoding elements. An encoding element that is no longer split according to a predefined constraint becomes a leaf node. That is, if it is assumed that only a square partition is possible for one encoding element, one encoding element can be split into at most four different encoding elements.

Further, in an embodiment of the present invention, an encoding element may be used in the value of an element performing encoding or in the value of an element performing decoding.

The prediction element may be an element divided into the shape of at least one square, rectangle, or the like. the same size within a single encoding element, or it may be any prediction element of the prediction elements partitioned within a single encoding element that is partitioned to have a different shape and/or size from those of another prediction element.

If the coding element is not the smallest coding element, when a prediction element that performs intra prediction based on the coding element is generated, intra prediction can be performed without dividing the image into a plurality of N×N prediction elements.

The prediction portions 120 and 125 may include an inter-picture prediction portion 120 that performs inter-picture prediction and an intra prediction portion 125 that performs intra prediction. It can be determined whether to use inter-picture prediction or perform intra prediction on a prediction element, and determine specific information (eg, intra prediction mode, motion vector, reference image, etc.) according to each prediction method. In this case, the processing element for performing the prediction may be different from the processing element for determining the prediction method and the specific content. For example, a prediction method and a prediction mode may be determined in a prediction element, and prediction may be performed in a transform element. A residual coefficient (residual block) between the generated prediction block and the original block may be input to the transformation portion 130. In addition, prediction mode information, motion vector information, and the like used for prediction can be encoded by the entropy encoding portion 165 together with a residual coefficient and transmitted to the decoder. When a particular encoding mode is used, the original block can be encoded as is and transmitted to the decoder without generating a prediction block by the prediction portions 120 and 125.

The inter-picture prediction portion 120 may predict a prediction element based on information about at least one image of the pictures before or after the current image, and in some cases, it may predict a prediction element based on information about a partial region that has been encoded. in the current image. The inter-image prediction portion 120 may include a reference image interpolation portion, a motion prediction portion, and a motion compensation portion.

The reference image interpolation portion may receive reference image information from the memory 155 and generate pixel information for an integer number of pixels or less from the reference image. In the case of a luminance pixel, an 8-tap DCT-based interpolation filter with a variable filter coefficient can be used to generate pixel information of an integer number of pixels or less through a 1/4 pixel element. In the case of a color difference signal, a 4-tap DCT-based interpolation filter with a variable filter coefficient can be used to generate pixel information of an integer number of pixels or less through a 1/8 pixel element.

The motion prediction portion may perform motion prediction based on a reference image interpolated by the reference image interpolation portion. Various methods such as Full Search Based Block Matching Algorithm (FBMA), Three Stage Search (TSS), New Three Stage Search (NTS) algorithm, and the like can be used as the motion vector calculation method. The motion vector may have a motion vector value as a 1/2 or 1/4 pixel element based on the interpolated pixels. The motion prediction portion may predict the current prediction element by changing the motion prediction method. Various methods such as a skip method, a merging method, an advanced motion vector prediction (AMVP) method, an intra-block copy method, etc. can be used as the motion prediction method.

The intra prediction portion 125 may generate a prediction element based on information about reference pixels located around the current block, which is information about a pixel in the current image. When a block adjacent to the current prediction element is a block on which inter-picture prediction has been performed, and thus the reference pixel is a pixel on which inter-picture prediction has been performed, the reference pixel is included in a block that has been subject to inter-picture prediction may be used in place of the reference pixel information of a nearby block that has been intra-predicted. That is, when a reference pixel is not available, at least one reference pixel among the available reference pixels may be used instead of the unavailable reference pixel information.

In intra prediction, a prediction mode may include a prediction mode based on angle information, which uses information about a reference pixel according to the prediction direction, and a prediction mode not based on angle information, which does not use direction information when making prediction. The mode for predicting the luminance information may be different from the mode for predicting the color difference information, and the intra prediction mode information used to predict the luminance information or the predicted luminance signal information may be used to predict the color difference information.

If the size of the prediction element is the same as the size of the transform element when intra prediction is performed, intra prediction may be performed on the prediction element based on the pixel on the left side, the pixel on the top left side, and the pixel on top of the prediction element. However, if the size of the prediction element is different from the size of the transformation element when intra prediction is performed, intra prediction may be performed using a reference pixel based on the transformation element. In addition, intra prediction using N×N partitioning can only be used for the smallest coding element.

By the intra prediction method, a prediction block can be generated after applying an adaptive intra smoothing (AIS) filter to a reference pixel according to the prediction mode. The type of AIS filter applied to the reference pixel can vary. To perform the intra prediction method, the intra prediction mode of the current prediction element can be predicted from the intra prediction mode of the prediction element existing in the vicinity of the current prediction element. When the prediction mode of the current prediction element is predicted using mode information predicted from a neighboring prediction element, if the intra prediction modes of the current prediction element are the same as the nearby prediction element, then information indicating that the prediction modes of the current prediction element are the same as the element nearby prediction element can be transmitted using the predetermined flag information, and if the prediction modes of the current prediction element and the nearby prediction element are different from each other, the prediction mode information of the current block can be encoded by performing entropy encoding.

In addition, a residual block may be generated containing a prediction element that has performed prediction based on the prediction element generated by the prediction portions 120 and 125 and residual coefficient information that is a difference value of the prediction element from the original block. The generated residual block may be input to the conversion portion 130.

The transform portion 130 may transform the residual block containing the original block and the residual coefficient information of the prediction element generated by the prediction portions 120 and 125 using a transform method such as discrete cosine transform (DCT), discrete sine transform (DST), or skip conversion. The use of DCT, DST or KLT for transforming the residual block can be determined based on information about the intra prediction mode of the prediction element used to generate the residual block.

The quantization portion 135 may quantize the values converted to the frequency domain by the conversion portion 130. Quantization factors may vary depending on the block or importance of the video. The value calculated by the quantization portion 135 may be provided to the inverse quantization portion 140 and the permutation portion 160.

The permutation portion 160 may perform permutation of coefficient values for the quantized residual coefficients.

The permutation portion 160 may change the coefficients of a two-dimensional block shape to a one-dimensional vector shape through a coefficient scanning method. For example, the permutation portion 160 may scan from the DC coefficients to the high frequency domain coefficients using a zigzag scanning method and change the coefficients into a one-dimensional vector form. According to the size of the transform element and the intra prediction mode, instead of zigzag scanning, vertical scanning in the form of scanning of two-dimensional block shape coefficients in the column direction and horizontal scanning of two-dimensional block shape coefficients in the row direction can be used. That is, according to the transform element size and the intra prediction mode, the scanning method to be used from zigzag scanning, vertical direction scanning, and horizontal direction scanning can be determined.

The entropy encoding portion 165 may perform entropy encoding based on the values calculated by the permutation portion 160. For entropy coding, various encoding methods can be used, such as exponential Golomb coding, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), and the like.

The entropy encoding portion 165 may encode various information such as residual coefficient information and coding element type information, prediction mode information, partition element information, prediction element information and transmission element information, motion vector information, reference frame, block interpolation information, and filtering information that is input from the permutation portion 160 and the prediction portions 120 and 125.

The entropy encoding portion 165 may perform entropy encoding of the coefficient value of the encoding element input from the permutation portion 160.

The inverse quantization portion 140 and the inverse transform portion 145 perform inverse quantization of the values quantized by the quantization portion 135 and inverse transform of the values transformed by the transform portion 130. The residual coefficient generated by the inverse quantization portion 140 and the inverse transform portion 145 may be combined with a prediction element predicted by the motion estimation portion, the motion compensation portion, and the intra prediction portion included in the prediction portions 120 and 125 to generating a recovered block.

The filtering portion 150 may comprise at least one of a deblocking filter, an offset correction element, and an adaptive loop filter (ALF).

The deblocking filter can remove block distortion occurring at the boundary between blocks in the reconstructed image. Determining whether to deblock or apply a deblocking filter to the current block can be made based on the pixels contained in multiple columns or rows contained in the block. A strong filter or a weak filter can be applied according to the required degree of filtering to remove blocking when applying a deblocking filter to a block. In addition, when vertical direction filtering and horizontal direction filtering are performed when using a deblocking filter, vertical direction filtering and horizontal direction filtering can be performed in parallel.

The offset correction element may correct the offset of the original video by means of a pixel element for the video that has been deblocked. To perform offset correction for a particular image, you can use a method of dividing the pixels included in the video into a certain number of regions, defining an area to perform the offset, and applying the offset to the area, or a method of applying the offset that takes into account the edge information of each pixel.

Adaptive contour filtering (ALF) can be performed based on the value obtained by comparing the reconstructed and filtered video with the original video. After dividing the pixels included in the video into predefined groups, a single filter can be defined to be applied to the corresponding group, and filtering can be performed differently for each group. A luminance signal, which is information related to whether to apply ALF, may be transmitted for each coding unit (CU), and the shape and filter coefficient of the ALF filter to be applied may vary depending on each block. In addition, an ALF filter of the same type (fixed type) can be applied regardless of the characteristic of the block used.

The storage device 155 may store the reconstructed block or image calculated by the filtering portion 150, and the reconstructed and stored block or image may be provided to the prediction portions 120 and 125 when performing inter-image prediction.

In fig. 2 is a block diagram showing a video decoder according to one embodiment of the present invention.

As shown in FIG. 2, video decoder 200 may include an entropy decoding portion 210, a permutation portion 215, an inverse quantization portion 220, an inverse transform portion 225, prediction portions 230 and 235, a filtering portion 240, and a memory 245.

When a video bitstream is input from a video encoder, the input bitstream can be decoded according to a procedure opposite to that of the video encoder.

The entropy decoding portion 210 may perform entropy decoding according to a procedure opposite to the procedure of performing entropy encoding in the entropy decoding portion of the video encoder. For example, various methods corresponding to the method performed by the video encoder may be used, such as exponential Golomb coding, context adaptive variable length coding (CAVLC), and context adaptive binary arithmetic coding (CABAC).

The entropy decoding portion 210 can decode information related to intra prediction and inter-picture prediction performed by the encoder.

The permutation portion 215 may perform permutation on the bit stream subjected to entropy decoding by the entropy decoding portion 210 based on the permutation method performed by the encoder. Coefficients expressed in one-dimensional vector form can be recovered and rearranged as coefficients in two-dimensional block form. The permutation portion 215 may receive information related to coefficient scanning performed by the encoding portion and perform reconstruction by an inverse scan method based on the scanning order performed by the corresponding encoding portion.

The inverse quantization portion 220 may perform inverse quantization based on the quantization parameter provided by the encoder and the permuted block coefficient value.

The deconversion portion 225 may perform the deconversion, i.e. reverse DCT, reverse DST or reverse KLT, for conversion, i.e. DCT, DST or KLT performed by the conversion part with respect to the quantization result performed by the video encoder. The inverse conversion may be performed based on the transfer element determined by the video encoder. The video decoder deconversion portion 225 may selectively perform a transformation method (eg, DCT, DST, KLT) according to a plurality of pieces of information such as a prediction method, a current block size, a prediction direction, and the like.

The prediction parts 230 and 235 may generate a prediction block based on the prediction block generation-related information provided by the entropy decoder 210 and the previously decoded block or image information provided by the storage device 245.

As described above, if the size of the prediction element and the size of the transform element are the same, when intra prediction is performed in the same manner as the operation in a video encoder, intra prediction is performed on the prediction element based on the pixel located on the left side, the pixel on the upper left side and the pixel on top of the prediction element. However, if the size of the prediction element and the size of the transformation element are different from each other when intra prediction is performed, intra prediction may be performed using a reference pixel based on the transformation element. In addition, intra prediction using N×N partitioning can only be used for the smallest coding element.

The prediction portions 230 and 235 may include a prediction element determination portion, an inter-picture prediction portion, and an intra prediction portion. The prediction element determination portion may receive various information such as prediction element information inputted from the entropy decoding portion 210, prediction mode information for the intra prediction method, motion prediction related information for the inter-picture prediction method, etc. etc., identify a prediction element from the current encoding element and determine whether the prediction element performs inter-picture prediction or intra-prediction. The inter-picture prediction portion 230 may perform inter-picture prediction regarding the current prediction element based on information included in at least one image of the images before or after the current image, including the current prediction element, by using the information , necessary for inter-picture prediction of the current prediction element provided by the video encoder. Alternatively, the inter-image prediction portion 230 may perform inter-image prediction based on the partial region information previously reconstructed in the current image containing the current prediction element.

To perform prediction carried out between pictures, it can be determined based on the encoding element whether the motion prediction method of the prediction element included in the corresponding encoding element is a skip mode, a merging mode, a motion vector prediction mode (AMVP mode), or an intra-block copy mode.

The intra prediction portion 235 may generate a prediction block based on information about a pixel in the current image. When the prediction element is a prediction element that has performed intra prediction, intra prediction may be performed based on the intra prediction mode information of the prediction element provided by the video encoder. The intra prediction portion 235 may include an adaptive intra-smoothing (AIS) filter, a reference pixel interpolation portion, and a DC filter. The AIS filter is a part that performs filtering on a reference pixel of the current block, and can determine whether to apply a filter according to the prediction mode of the current prediction element, and apply the filter. AIS filtering may be performed on a reference pixel of the current block by using the prediction mode and AIS filter information of the prediction element provided by the video encoder. When the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

When the prediction element prediction mode is a prediction element that performs intra prediction based on a pixel value obtained by interpolating a reference pixel, the reference pixel interpolation part may generate a reference pixel of an element composed of pixels that has an integer value or less by interpolation reference pixel. When the prediction mode of the current prediction element is a prediction mode that generates a prediction block without interpolating a reference pixel, the reference pixel may not be interpolated. The DC filter may generate a prediction block by filtering when the prediction mode of the current block is a DC mode.

The reconstructed block or image may be provided to part 240 for filtering. The filtering portion 240 may include a deblocking filter, an offset correction element, and an ALF.

Information about whether to apply a deblocking filter to the corresponding block or image, and information about whether to apply a strong filter or a weak filter when the deblocking filter is applied may be provided by the video encoder. The video decoder deblocking filter may be provided with information associated with the deblocking filter provided by the video encoder, and the video decoder may perform deblocking filtering on the corresponding block.

The offset correction element may perform offset correction on the reconstructed video based on the offset correction type and offset value information applied to the video when encoding is performed.

ALF may be applied to an encoding element based on information about whether to apply ALF and information about ALF coefficients provided by the encoder. ALF information can be provided for inclusion in a set of specific parameters.

Storage device 245 may store the reconstructed image or block and use it as a reference image or reference block, and may provide the reconstructed image to an output element.

In fig. 3 is a view showing a code tree base element according to one embodiment of the present invention.

A maximum size coding block can be defined as a code tree block. The image is divided into many code tree units (CTUs). A code tree element is a coding element having a maximum size and may be called a large coding unit (LCU). In fig. Figure 3 shows an example in which an image is divided into many code tree elements.

The size of a code tree element can be determined at the image level or the sequence level. To do this, information indicating the size of the code tree element may be transmitted by an image parameter set or a sequence parameter set.

For example, the code tree element size for the entire image in the sequence can be set to 128×128. Alternatively, at the image level, any size between 128×128 and 256×256 may be defined as the code tree element size. For example, the code tree element size may be set to 128×128 in the first image, and the code tree element size may be set to 256×256 in the second image.

Coding blocks can be generated by splitting a code tree element. The encoding block specifies a base element for performing encoding/decoding. For example, prediction or transformation may be performed for each encoding block, or a predictive encoding mode may be determined for each encoding block. Here, the predictive encoding mode indicates a method for generating video prediction. For example, the predictive coding mode may include intra-screen prediction (intra-prediction), inter-screen prediction (inter-picture prediction), current picture reference (CPR), or intra-block copy (IBC), or combined prediction. For a coding block, a prediction block may be generated by using at least one predictive coding mode of intra prediction, inter-picture prediction, anchor to current picture, and combined prediction.

Information indicating the predictive encoding mode of the current block may be transmitted via a bitstream. For example, the information may be a 1-bit flag indicating whether the predictive coding mode is an intra-picture mode or an inter-picture mode. Only when the predictive encoding mode of the current block is determined to be an inter-picture mode can lock to the current picture or combined prediction be used.

The purpose of snapping to the current picture is to establish the current picture as a reference picture and obtain the prediction block of the current block from the region that has already been encoded/decoded in the current picture. In this case, the current image means the image containing the current block. Information indicating whether an anchor to the current image is applied to the current block may be transmitted via a bit stream. For example, the information may be a 1-bit flag. When the flag is true, the predictive encoding mode of the current block can be defined as anchor to the current picture, and when the flag is false, the predictive mode of the current block can be defined as inter-picture prediction.

Alternatively, the predictive coding mode of the current block may be determined based on the index of the reference picture. For example, when the reference picture index indicates the current picture, the predictive encoding mode of the current block may be determined to be anchored to the current picture. When the reference picture index indicates a picture different from the current picture, the predictive coding mode of the current block can be defined as prediction performed between pictures. That is, anchoring to the current image is a prediction method using information about the region in which encoding/decoding has been performed in the current image, and prediction performed between images is a prediction method using information about another image in which encoding/decoding has been performed. encoding/decoding.

Combined prediction represents a coding mode in which two or more of intra-prediction, inter-picture prediction, and current-picture anchoring are combined. For example, when applying combined prediction, a first prediction block may be generated based on one of intra prediction, inter-image prediction, and current image anchoring, and a second prediction block may be generated based on another of them. When generating the first prediction block and the second prediction block, the final prediction block may be generated by an averaging operation or a weighted addition operation of the first prediction block and the second prediction block. Information indicating whether combined prediction is applied may be transmitted via a bitstream. The information may be a 1-bit flag.

In fig. 4 is a view showing various types of partitioning of a coding block.

The encoding block may be partitioned into a plurality of encoding blocks based on a quaternary tree partition, a binary tree partition, or a ternary tree partition. The split encoding block may be split again into a plurality of encoding blocks based on a quaternary tree split, a binary tree split, or a ternary tree split.

Quaternary tree partitioning refers to a partitioning method that splits the current block into four blocks. As a result of quaternary tree partitioning, the current block can be split into four square-shaped partitions (see “SPLIT_QT” in Fig. 4(a)).

Binary tree partitioning refers to a partitioning method that splits the current block into two blocks. Splitting the current block into two blocks along the vertical direction (i.e. using a vertical line intersecting the current block) can be called binary tree splitting in the vertical direction, and splitting the current block into two blocks along the horizontal direction (i.e. with using a horizontal line intersecting the current block) can be called a binary tree partition in the horizontal direction. Binary tree partitioning can split the current block into two non-square partitions. "SPLIT_BT_VER" in Fig. 4(b) represents the result of binary tree splitting in the vertical direction, and “SPLIT_BT_HOR” in FIG. 4(c) presents the result of binary tree partitioning in the horizontal direction.

Ternary tree partitioning refers to a partitioning method that splits the current block into three blocks. Splitting the current block into three blocks along the vertical direction (i.e. using two vertical lines crossing the current block) can be called ternary tree splitting in the vertical direction, and splitting the current block into three blocks along the horizontal direction (i.e. using two horizontal lines crossing the current block) can be called ternary tree partitioning in the horizontal direction. As a result of ternary tree partitioning, the current block can be partitioned into three non-square partitions. In this case, the width/height of the section located in the center of the current block can be twice the width/height of other sections. "SPLIT_TT_VER" in Fig. 4(d) represents the result of ternary tree partitioning in the vertical direction, and “SPLIT_TT_HOR” in FIG. 4(e) presents the result of ternary tree partitioning in the horizontal direction.

The number of splits of a code tree element can be defined as the split depth. The maximum split depth of a code tree element can be determined at the sequence level or the image level. Accordingly, the maximum depth of a code tree element may differ for each sequence or image.

Alternatively, the maximum splitting depth for each splitting method can be determined separately. For example, the maximum partition depth allowed for a quaternary tree partition may be different from the maximum partition depth allowed for a binary tree partition and/or ternary tree partition.

The encoder may transmit information indicating at least one of a split type and a split depth of the current block via a bit stream. The decoder may determine the split type and split depth of a code tree element based on information parsed from the bit stream.

In fig. 5 is a view showing a code tree element partitioning diagram.

Partitioning a coding block using a partitioning method such as quaternary tree partitioning, binary tree partitioning, and/or ternary tree partitioning may be referred to as multi-tree partitioning.

Coding blocks generated by applying a multi-tree partition to a coding block may be called lower coding blocks. When the partition depth of the encoding block is k, the partition depth of the lower encoding blocks is set to k+1.

Conversely, for encoding blocks having partition depth k+1, the encoding block having partition depth k may be called an upper encoding block.

The partition type of the current encoding block may be determined based on at least one of the partition type of the upper encoding block and the partition type of the adjacent encoding block. Here, a neighboring encoding block is an encoding block adjacent to the current encoding block, and may include at least one of an upper adjacent block and a left adjacent block of the current encoding block, and an adjacent block adjacent to the upper left corner. In this case, the type of splitting may include at least one of the following: whether a quaternary tree splitting was used, whether a binary tree splitting was used, a binary tree splitting direction, whether a ternary tree splitting was used, and a ternary splitting direction tree.

To determine the type of splitting of the encoding block, information indicating whether the coding block can be split may be transmitted by a bitstream. The information is a 1-bit flag "split_cu_flag", and when the flag is true, it indicates that the encoding block is split using a quadtree (→ quaternary tree) splitting method.

When split_cu_flag is true, information indicating whether a quaternary tree coding block is split can be transmitted via the bitstream. The information is a 1-bit flag in the form split_qt_flag, and when the flag is true, the encoding block can be split into four blocks.

For example, in the example shown in FIG. 5, when a code tree element is split based on the quaternary tree, four coding blocks having a split depth of 1 are generated. In addition, it is shown that the splitting based on the quaternary tree is applied again to the first and fourth coding blocks of the four coding blocks generated by the splitting based on a quaternary tree. As a result, four coding blocks having a partition depth of 2 can be generated.

In addition, encoding blocks having a partition depth of 3 can be generated by repeatedly applying the quaternary tree partition to a encoding block having a partition depth of 2.

When a quaternary tree partition is not applied to an encoding block, it can be determined whether a binary tree partition or a ternary tree partition is performed on the encoding block, taking into account at least one of the size of the encoding block, whether the block is located encoding at the image boundary, the maximum partition depth and the type of partition of the adjacent block. When it is determined that binary tree partitioning or ternary tree partitioning should be performed on an encoding block, information indicating the direction of partitioning may be transmitted by a bitstream. The information may be a 1-bit flag in the form mtt_split_cu_vertical_flag. Based on the flag, it can be determined whether the dividing direction is the vertical direction or the horizontal direction. Additionally, information indicating whether a binary tree partition or a ternary tree partition is applied to the encoding block may be transmitted by the bitstream. The information can be a 1-bit flag in the form mtt_split_cu_binary_flag. Based on the flag, it can be determined whether a binary tree partition or a ternary tree partition is applied to the encoding block.

For example, in the example shown in FIG. 5, it is shown that a binary tree partition in the vertical direction is applied to a coding block having a partition depth of 1, a ternary tree partition in the vertical direction is applied to the left encoding block of the encoding blocks generated by the partition, and a partition into the binary tree base in the vertical direction is applied to the right encoding block.

Inter-picture prediction is a predictive coding mode that predicts the current block by using information about the previous picture. For example, a block at the same position as the current block in the previous image (hereinafter referred to as a compatible block) can be set as a prediction block of the current block. Hereinafter, a prediction block generated based on a block at the same position as the current block will be called a compatible prediction block.

On the other hand, when an object located in the previous image has moved to a different position in the current image, the current block can be efficiently predicted by using the movement of the object. For example, when the motion direction and size of an object can be learned by comparing a previous image and a current image, a prediction block (or video prediction) of the current block can be generated taking into account the motion information of the object. Hereinafter, the prediction block generated by using the motion information may be called a motion prediction block.

A residual block can be generated by subtracting a prediction block from the current block. In this case, when there is motion of the object, the energy of the residual block can be reduced by using a motion prediction block instead of a compatible prediction block, and thus the compression efficiency of the residual block can be improved.

As described above, generating a prediction block by using motion information may be called motion-compensated prediction. In most inter-image predictions, a prediction block may be generated based on motion compensated prediction.

The motion information may comprise at least one of a motion vector, a reference image index, a prediction direction, and a bidirectional weight index. The motion vector represents the direction of motion and the size of the object. The reference picture index indicates the reference picture of the current block of the reference pictures included in the reference picture list. The prediction direction indicates any of unidirectional L0 prediction, unidirectional L1 prediction, and bidirectional prediction (L0 prediction and L1 prediction). According to the prediction direction of the current block, at least one of the movement information in the L0 direction and the movement information in the L1 direction may be used. The bidirectional weight index indicates the weighting value applied to the L0 prediction block and the weighting value applied to the L1 prediction block.

In fig. 6 is a flowchart illustrating an inter-image prediction method according to one embodiment of the present invention.

As shown in FIG. 6, the inter-picture prediction method includes the steps of determining (S601) an inter-picture prediction mode of the current block, obtaining (S602) information about the movement of the current block according to the determined inter-picture prediction mode, and performing (S603) the inter-picture prediction motion compensation for the current block based on received motion information.

Here, the inter-picture prediction mode represents various methods for determining motion information of the current block, and may include an inter-picture prediction mode that uses translational motion information and an inter-picture prediction mode that uses affine motion information. . For example, an inter-image prediction mode that uses translational motion information may include a fusion mode and a motion vector prediction mode, and an inter-image prediction mode that uses affine motion information may include an affine fusion mode and a vector prediction mode. affine movement. The motion information of the current block may be determined based on a neighboring block adjacent to the current block or information analyzed from the bitstream according to an inter-picture prediction mode.

Next, an inter-image prediction method that uses affine motion information will be described in detail.

In fig. 7 is a view showing the nonlinear motion of an object.

Nonlinear object motion can be generated in video. For example, as shown in the example of FIG. 7, such nonlinear movement of an object can occur as approaching, moving away, rotating, affine transformation, etc. cameras. When nonlinear motion of an object occurs, the motion of the object cannot be effectively expressed by a translational motion vector. Accordingly, encoding efficiency can be improved by using affine motion instead of translational motion in the region where nonlinear object motion occurs.

In fig. 8 is a flowchart illustrating a method for inter-image prediction based on affine motion according to one embodiment of the present invention.

Whether the inter-image prediction method based on affine motion is applied to the current block can be determined based on the information parsed from the bitstream. In particular, whether an inter-image prediction method based on affine motion is applied to the current block can be determined based on at least one of a flag indicating whether the affine fusion mode is applied to the current block and a flag indicating whether whether the mode of predicting the affine motion vector to the current block.

When the inter-image prediction method based on affine motion is applied to the current block, an affine motion model of the current block can be determined (S801). The affine motion model may be defined as at least one of a six-parameter affine motion model and a four-parameter affine motion model. A six-parameter affine motion model expresses affine motion using six parameters, and a four-parameter affine motion model expresses affine motion using four parameters.

Equation 1 expresses affine motion using six parameters. Affine motion represents translational motion for a predefined region defined by affine source vectors.

[Equation 1]

When affine motion is expressed using six parameters, complex motion can be expressed. However, as the number of bits required to encode each parameter increases, the encoding efficiency may decrease. Accordingly, affine motion can be expressed using four parameters. Equation 2 expresses the affine motion using four parameters.

[Equation 2]

Information for determining the affine motion model of the current block can be encoded and transmitted via a bitstream. For example, the information could be a 1-bit flag in the form "affine_type_flag". When the flag value is 0, it may indicate that a 4-parameter affine motion model is applied, and when the flag value is 1, it may indicate that a 6-parameter affine motion model is applied. The flag may be encoded through a slice, tile, or block element (eg, through an encoding block or encoding tree element). When a flag is passed at the slice level, the affine motion model defined at the slice level can be applied to all blocks belonging to the slice.

Alternatively, the affine motion model of the current block may be determined based on the inter-image affine prediction mode of the current block. For example, when the affine merging mode is applied, the affine motion model of the current block can be defined as a 4-parameter motion model. On the other hand, when the affine motion vector prediction mode is applied, information for determining the affine motion model of the current block can be encoded and transmitted by a bitstream. For example, when the affine motion vector prediction mode is applied to the current block, the affine motion model of the current block can be determined based on a 1-bit flag in the form “affine_type_flag”.

Then, the affine source vector of the current block can be obtained (S802). When a 4-parameter affine motion model is selected, motion vectors at two control points of the current block can be obtained. On the other hand, when the 6-parameter affine motion model is selected, the motion vectors at the three control points of the current block can be obtained. The motion vector at a control point can be called an affine source vector. The reference point may include at least one of the top left corner, top right corner, and bottom left corner of the current block.

In fig. 9 is a view showing an example of affine source vectors of each affine motion model.

In the 4-parameter affine motion model, affine source vectors can be obtained for two of the upper left corner, upper right corner and lower left corner. For example, as shown in the example of FIG. 9(a), when the 4-parameter affine motion model is selected, the affine vector can be obtained by using the affine source vector sv ₀ for the top left corner of the current block (for example, the top left sample (x1, y1)) and the affine source vector sv ₁ for the top right corner of the current block (for example, top right count (x1, y1)). You can also use an affine source vector for the bottom left corner instead of an affine source vector for the top left corner, or use an affine source vector for the bottom left corner instead of an affine source vector for the top right corner.

In the 6-parameter affine motion model, affine source vectors can be obtained for the upper left corner, upper right corner and lower left corner. For example, as shown in the example of FIG. 9(b), when the 6-parameter affine motion model is selected, the affine vector can be obtained by using the affine source vector sv ₀ for the top left corner of the current block (for example, the top left sample (x1, y1)), the affine source vector sv ₁ for the top right corner of the current block (for example, top right sample (x1, y1)) and the affine source vector sv ₂ for the top left corner of the current block (for example, top left sample (x2, y2)).

In the embodiment described below, in the 4-parameter affine motion model, the affine source vectors of the top left control point and the top right control point will be called the first affine source vector and the second affine source vector, respectively. In embodiments that use the first affine seed vector and the second affine seed vector described below, at least one of the first affine seed vector and the second affine seed vector may be replaced by the lower left reference point affine seed vector (third affine seed vector) or the affine source vector of the lower right control point (the fourth affine source vector).

In addition, in the 6-parameter affine motion model, the affine source vectors of the top left control point, top right control point and bottom left control point will be called the first affine source vector, the second affine source vector and the third affine source vector, respectively. In embodiments that use a first affine seed vector, a second affine seed vector, and a third affine seed vector described below, at least one of the first affine seed vector, the second affine seed vector, and the third affine seed vector may be replaced by an affine seed vector bottom right control point (the fourth affine source vector).

An affine vector can be obtained (S803) for each sub-block by using affine source vectors. In this case, the affine vector represents the translational motion vector obtained from the affine source vectors. An affine subblock vector may be called an affine subblock motion vector or a subblock motion vector.

In fig. 10 is a view showing an example of subblock affine vectors in a 4-parameter motion model.

The affine sub-block vector can be obtained based on the position of the reference point, the position of the sub-block and the affine source vector. For example, Equation 3 shows an example of obtaining an affine subblock vector.

[Equation 3]

In Equation 3, (x, y) denotes the position of the sub-block. Here, the position of the sub-block indicates the position of the reference sample included in the sub-block. The reference sample may be a sample located in the upper left corner of the subblock, or a sample for which at least one of the x-axis and y-axis coordinates represents a center point. (x ₀ , y ₀ ) denotes the position of the first control point and (sv _0x , sv _0y ) denotes the first affine source vector. In addition, (x ₁ , y ₁ ) denotes the position of the second control point and (sv _1x , sv _1y ) denotes the second affine source vector.

When the first control point and the second control point correspond to the top left corner and top right corner of the current block, respectively, x ₁ -x ₀ can be set to a value equal to the width of the current block.

Thereafter, motion compensation prediction can be performed (S804) for each sub-block using the affine vector of each sub-block. By performing motion-compensated prediction, a prediction block for each sub-block can be generated. Sub-block prediction blocks can be set as prediction blocks of the current block.

The affine source vector of the current block can be obtained based on the affine source vector of a neighboring block adjacent to the current block. When the inter-image prediction mode of the current block is an affine fusion mode, an affine source vector of a fusion candidate included in the fusion candidate list can be defined as an affine source vector of the current block. In addition, when the inter-picture prediction mode of the current block is an affine fusion mode, motion information including at least one of a reference image index of the current block, a specific direction prediction flag, and a bidirectional weighting value can be set to the same. as merger candidate information.

A merge candidate can be obtained based on the adjacent block of the current block. The neighbor block may include at least one of a spatial neighbor block spatially adjacent to the current block and a temporary neighbor block included in an image different from the current image.

In fig. 11 is a view showing a neighbor block that can be used to obtain a merge candidate.

The adjacent block of the current block may include at least one of an adjacent block (A) adjacent to the left side of the current block, an adjacent block (B) adjacent to the top side of the current block, an adjacent block (C) adjacent to the top right corner of the current block , an adjacent block (D) adjacent to the lower left corner of the current block, and an adjacent block (E) adjacent to the upper left corner of the current block. When the coordinates of the top left sample of the current block are (x ₀ , y ₀ ), the left neighbor block A contains a sample at position (x ₀ -1, y ₀ +H-1) and the top neighbor block B contains a sample at position (x ₀ +W-1, y ₀ -1). In this case, W and H represent the width and height of the current block, respectively. The upper right adjacent block C contains a sample at position (x ₀ +W, y ₀ -1) and the lower left adjacent block D contains a sample at position (x ₀ -1, y ₀ +H). The top left adjacent block E contains a sample at position (x ₀ -1, y ₀ -1).

When a neighboring block is encoded in an affine inter-picture prediction mode, the affine source vector of a merge candidate can be obtained based on the affine source vector of the neighboring block. In the following, a neighboring block encoded in the affine inter-image prediction mode will be called an affine neighboring block.

A merge candidate for the current block can be generated by searching neighboring blocks in a predefined scan order. The scanning order can be preset in the encoder and decoder. For example, neighbor blocks can be searched in the following order: A, B, C, D, and E. In addition, merge candidates can be sequentially obtained from the found affine neighbor blocks. Alternatively, the scan order may be adaptively determined based on at least one of the size, shape, and affine motion model of the current block. That is, the scanning orders of blocks that differ in at least one of size, shape, and affine motion pattern may differ from each other.

Alternatively, blocks located on top of the current block are searched sequentially and a merge candidate is obtained from the affine neighbor block found first, and blocks located on the left side of the current block are searched sequentially and a merge candidate is obtained from the affine neighbor block found first. first. Here, neighboring blocks located on top of the current block include at least one of adjacent block E, adjacent block B and adjacent block C, and blocks located on the left side of the current block may include at least one of block A and block D In this case, the neighboring block E can also be classified as the block located on the left side of the current block.

Although not shown, a merge candidate may be obtained from a temporary neighbor of the current block. In this case, the temporary neighboring block may include a block at the same position as the position of the current block in the fusion image or a block adjacent to it. In particular, when the temporal neighbor of the current block is encoded in an affine inter-picture prediction mode, a merge candidate can be obtained based on the affine source vector of the temporal merge candidate.

A merge candidate list containing the merge candidates may be generated, and an affine source vector of one of the merge candidates included in the merge candidate list may be determined to be the affine source vector of the current block. To achieve this, index information for identifying any of the merge candidates may be encoded and transmitted via a bit stream.

As another example, neighboring blocks are searched according to scan order, and the affine source vector of the current block can be obtained from the affine source vector of the affine neighbor block found first.

As described above, in affine merge mode, the affine source vector of the current block can be obtained using the affine source vector of the adjacent block.

When the inter-picture prediction mode of the current block is an affine motion vector prediction mode, the affine source vector of the motion vector prediction candidate included in the motion vector prediction candidate list can be determined as the affine source vector prediction value of the current block. The affine source vector of the current block can be obtained by adding the difference value of the affine source vectors with the prediction value of the affine source vector.

An affine source vector prediction candidate can be obtained based on the adjacent block of the current block. Specifically, neighboring blocks above the current block are searched according to a predetermined scanning order, and the first candidate for affine seed vector prediction can be obtained from the affine neighboring block found first. In addition, neighboring blocks located on the left side of the current block are searched according to a predetermined scanning order, and the second candidate for affine seed vector prediction can be obtained from the affine neighboring block found first.

Information for determining the difference value of the affine source vectors may be encoded and transmitted via a bitstream. The information may include size information indicating the magnitude of the difference value of the affine source vectors, and code information indicating the sign of the difference value of the affine source vectors. The difference value of the affine source vectors for each control point can be set to the same. Alternatively, the difference value of the affine source vectors can be set differently for each reference point.

As described above, an affine source vector of a merge candidate or a prediction candidate can be obtained from an affine source vector of an affine neighboring block, and an affine source vector of a current block can be obtained using the resulting affine source vector of a merge candidate or a prediction candidate. affine source vector. Alternatively, after searching for affine neighbor blocks according to a predetermined scanning order, the affine source vector of the current block can be obtained from the affine source vector of the affine neighbor block found first.

Next, a method for obtaining an affine source vector of a current block, a merge candidate, or a prediction candidate affine source vector from an affine source vector of an affine neighboring block will be described in detail. In the embodiments described below, it should be understood that obtaining the affine source vector of the current block represents obtaining the affine source vector of the merge candidate or obtaining the affine source vector of the prediction candidate affine source vector.

In fig. 12 is a view showing an example of obtaining an affine source vector of a current block based on an affine source vector of an affine neighboring block.

When the first affine source vector nv ₀ for the top left control point and the second affine source vector nv ₁ for the top right control point are stored for the affine neighbor block, the third affine source vector nv ₂ for the bottom left control point of the affine neighbor block can be obtained based on the first an affine source vector and a second affine source vector. Equation 4 shows an example of obtaining a third affine seed vector.

[Equation 4]

In Equation 4, (nv _0x , nv _0y ) denotes the first affine source vector nv ₀ , (nv _1x , nv _1y ) denotes the second affine source vector nv _{1 ,} and (nv _2x , nv _2y ) denotes the third affine source vector nv ₂ . In addition, (x _n0 , x _n0 ) denotes the position of the first control point, (x _n1 , x _n1 ) denotes the position of the second control point, and (x _n2 , x _n2 ) denotes the position of the third control point.

Thereafter, the affine source vector of the current block can be obtained using the first affine source vector, the second affine source vector, and the third affine source vector. Equation 5 shows an example of obtaining the first affine source vector v ₀ of the current block, and Equation 6 shows an example of obtaining the second affine source vector v ₁ of the current block.

[Equation 5]

[Equation 6]

In Equations 5 and 6, (v _0x , v _0y ) denotes the first affine source vector sv ₀ of the current block and (v _1x , v _1y ) denotes the second affine source vector sv ₁ of the current block. In addition, (x ₀ , y ₀ ) denotes the position of the first control point and (x ₁ , y ₁ ) denotes the position of the second control point. For example, the first control point indicates the top left corner of the current block and the second control point indicates the top right corner of the current block.

In the example described above, it was described that the affine source vectors of the current block are obtained using three affine source vectors for the affine neighboring block. As another example, the affine source vectors of the current block can be obtained using only two of the affine source vectors of an affine neighboring block.

Alternatively, the current block's affine source vectors may be obtained using a fourth bottom-right affine source vector instead of the first top-left corner affine source vector, the second top-right affine source vector, or the third bottom-left affine source vector of an affine adjacent block.

Specifically, when the top border of the current block is in contact with the top border of a code tree element, and it is desirable to use an affine source vector for the top reference point (e.g., top left corner or top right corner) of an affine neighbor block adjacent to the top side of the current block ( (hereinafter referred to as the upper affine neighbor block), they are desirable to be stored in the storage device in advance, and the problem of increasing the number of linear buffers may arise. Accordingly, when the top boundary of the current block is in contact with the top boundary of a code tree element, it may be determined that it is necessary to use the affine source vector of the bottom reference point (e.g., bottom left corner or bottom right corner) instead of the affine source vector of the top reference point for the top affine neighboring block. For example, the affine source vectors of the current block can be obtained using a third affine source vector for the bottom left corner and a fourth affine source vector for the bottom right corner of the top affine adjacent block. In this case, the affine source vectors for the bottom corners can be copied from the affine source vectors for the top corners or derived from the affine source vectors for the top corners. For example, the first affine source vector, the second affine source vector, or the third affine source vector may be converted to or replace the fourth affine source vector for the lower right corner.

Equations 7 and 8 show an example of obtaining the first affine seed vector and the second affine seed vector of the current block using the third affine seed vector for the lower left control point and the fourth affine seed vector for the lower right control point of the adjacent affine vector.

[Equation 7]

[Equation 8]

In Equations 7 and 8, (x _n2 , y _n2 ) denotes the coordinates of the bottom left control point of the affine neighbor block and (x _n3 , y _n3 ) denotes the coordinates of the bottom right control point of the affine neighbor block. (x ₀ , y ₀ ) denotes the coordinates of the top left control point of the current block and (x ₁ , y ₁ ) denotes the coordinates of the top right control point of the current block. (nv _2x , nv _2y ) denotes the affine source vector of the bottom left control point of the affine neighbor block (i.e., the third affine source vector) and (nv _3x , nv _3y ) denotes the affine source vector of the bottom right control point of the affine neighbor block (i.e. e. the fourth affine source vector). (v _0x , v _0y ) denotes the affine source vector of the top left control point of the current block (i.e., the first affine source vector) and (v _1x , v _1y ) denotes the affine source vector of the top right control point of the current block (i.e. second affine source vector).

The division operation included in Equations 7 and 8 can be changed to a shift operation. The shift operation can be performed based on the value obtained from the width between the bottom left control point and the bottom right control point (ie (x _n3 -x _n2 )).

As shown in the example described above, the affine source vectors of the current block can be obtained from the affine source vectors of the affine neighboring block that have been encoded/decoded. To do this, the affine source vectors of the affine neighbor block that have been encoded/decoded must be stored in a storage device. However, when the affine source vectors of the affine neighbor block are stored in the storage device in addition to the translational vectors (ie, affine vectors) of the subblocks included in the affine neighbor block, the problem of increasing the storage device usage occurs. To solve this problem, the affine source vector of the current block can be obtained using the motion vector of the subblock adjacent to the reference point of the affine neighbor block, instead of the affine source vector of the affine neighbor block. That is, the motion vector of a subblock adjacent to the reference point of an affine neighbor block can be set as the affine source vector of the affine neighbor block. Here, a sub-block is a block having a size/shape preset in the encoder and decoder, and may be a block having a basic size/shape for storing a motion vector. For example, a subblock may be a 4x4 square block. Alternatively, the motion vector for a particular reference position can be set to the affine source vector of the affine neighbor block.

In fig. 13 is a view showing an example of setting a motion vector of a sub-block as an affine source vector of an affine neighboring block.

The motion vector of a subblock adjacent to a reference point can be set as the affine source vector of the reference point. For example, in the example shown in FIG. 13, the motion vector (nv _4x , nv _4y ) of the sub-block (lower left sub-block) adjacent to the lower left corner of the affine neighboring block can be set as the affine source vector (nv _2x , nv _2x ) of the lower left control point and the motion vector (nv _5x , nv _5y ) of the subblock (lower right subblock) adjacent to the lower right corner can be set as an affine source vector (nv _3x , nv _3y ) of the control point in the lower right corner. In this case, the bottom-left sub-block may mean a sub-block containing a sample (for example, a sample at position (x _n2 , y _n2-1 )) adjacent to the bottom-left sample point (x _n2 , y _n2 ) in the adjacent affine block, and a bottom-right a sub-block may mean a block containing a sample (for example, a sample at position (x _n3-1 , y _n3-1 )) adjacent to the lower right sample point (x _n3 , y _n3 ) in an adjacent affine block. When the affine source vectors of the current block are obtained based on Equations 7 and 8, the third affine source vector of the affine neighboring block can be replaced by the motion vector of the lower left sub-block, and the fourth affine source vector can be replaced by the motion vector of the lower right sub-block.

Hereinafter, in the embodiment described below, a sub-block used as an affine source vector of an affine neighbor block will be called an affine sub-block.

According to one embodiment of the present invention, an affine subblock can be determined based on a count at a particular position. For example, a subblock containing a sample at a particular position can be set as an affine subblock. In what follows, a sample at a specific position will be called an affine reference sample. In addition, the reference sample used to determine the affine subblock of the lower left reference point is called the lower left reference sample, and the reference sample used to determine the affine subblock of the lower right reference point is called the lower right reference sample.

The lower left reference sample and the lower right reference sample may be selected from samples included in the affine adjacent block. For example, at least one of the upper left sample, lower left sample, upper right sample, and lower left sample of the lower left subblock may be set as the lower left reference sample, and at least one of the upper left sample, lower left sample, upper right reference and the lower left sample of the lower right subblock can be set as the lower right reference sample. Accordingly, the motion vectors of the lower left sub-block containing the lower left reference sample and the lower right sub-block containing the lower right reference sample can be set as an affine source vector for the lower left reference point and an affine source vector for the lower right reference point, respectively.

As another example, at least one of the lower left reference sample and the lower right reference sample may be set to be a sample located outside the affine neighbor block. This will be described in detail with reference to FIG. 14-16.

In fig. 14-16 are views showing the positions of the reference samples.

As shown in the example in FIG. 14(a), the upper left sample of the lower left sub-block may be set as a reference sample (x _n4 , y _n4 ) for the lower left reference point. Accordingly, the lower left subblock containing the reference sample (x _n4 and y _n4 ) can be set as an affine subblock for the lower left reference point.

A sample located on the right side of the top right sample of the bottom right subblock can be set as a reference sample (x _n5 , y _n5 ) for the bottom right reference point. Accordingly, a subblock adjacent to the right side of the lower right subblock containing the reference sample (x _n5 and y _n5 ) can be set as an affine subblock for the lower right reference point.

Alternatively, as shown in the example of FIG. 14(b), a sample located on the left side of the upper left sample of the lower left sub-block can be set as a reference sample (x _n4 , y _n4 ) for the lower left reference point. Accordingly, a subblock adjacent to the left side of the lower left subblock containing the reference sample (x _n4 and y _n4 ) can be set as an affine subblock for the lower left reference point.

The top right sample of the bottom right subblock can be set as the reference sample (x _n5 , y _n5 ) for the bottom right reference point. Accordingly, the lower right subblock containing the reference sample (x _n5 and y _n5 ) can be set as an affine subblock for the lower right reference point.

Alternatively, as shown in the example of FIG. 15(a), the lower left sample of the lower left sub-block may be set as a reference sample (x _n4 , y _n4 ) for the lower left reference point. Accordingly, the lower left subblock containing the reference sample (x _n4 and y _n4 ) can be set as an affine subblock for the lower left reference point.

A sample located on the right side of the lower right sample of the lower right subblock can be set as a reference sample (x _n5 , y _n5 ) for the lower right reference point. Accordingly, a subblock adjacent to the right side of the lower right subblock containing the reference sample (x _n5 and y _n5 ) can be set as an affine subblock for the lower right reference point.

Alternatively, as shown in the example of FIG. 15(b), a sample located on the left side of the lower left sample of the lower left sub-block can be set as a reference sample (x _n4 , y _n4 ) for the lower left reference point. Accordingly, a subblock adjacent to the left side of the lower left subblock containing the reference sample (x _n4 and y _n4 ) can be set as an affine subblock for the lower left reference point.

The bottom right sample of the bottom right subblock can be set as the reference sample (x _n5 , y _n5 ) for the bottom right reference point. Accordingly, the lower right subblock containing the reference sample (x _n5 and y _n5 ) can be set as an affine subblock for the lower right reference point.

Alternatively, as shown in the example of FIG. 16(a), a sample located between the upper left sample and the lower left sample of the lower left sub-block (eg, the middle left sample) may be set as a reference sample (x _n4 , y _n4 ) for the lower left reference point. Accordingly, the lower left subblock containing the reference sample (x _n4 and y _n4 ) can be set as an affine subblock for the lower left reference point.

A sample located on the right side of a sample located between the top right sample and the bottom right sample of the bottom right subblock (eg, the middle right sample) can be set as the reference sample (x _n5 , y _n5 ) for the bottom right reference point. Accordingly, a subblock adjacent to the right side of the lower right subblock containing the reference sample (x _n5 and y _n5 ) can be set as an affine subblock for the lower right reference point.

Alternatively, as shown in the example of FIG. 16(b), a sample located on the left side of a sample located between the upper left sample and the lower left sample of the lower left sub-block can be set as a reference sample (x _n4 , y _n4 ) for the lower left reference point. Accordingly, a subblock adjacent to the left side of the lower left subblock containing the reference sample (x _n4 and y _n4 ) can be set as an affine subblock for the lower left reference point.

A sample located between the upper right sample and the lower right sample of the lower right subblock may be set as a reference sample (x _n5 , y _n5 ) for the lower right reference point. Accordingly, the lower right subblock containing the reference sample (x _n5 and y _n5 ) can be set as an affine subblock for the lower right reference point.

When the affine source vectors of the current block are obtained based on Equations 7 and 8, the third affine source vector of the affine neighboring block can be replaced by the motion vector of the affine sub-block for the lower left control point, and the fourth affine source vector can be replaced by the motion vector of the affine sub-block for the bottom right control point . In addition, the position of the lower left reference point may be replaced by the position of the lower left reference sample, and the position of the lower right reference point may be replaced by the position of the lower right reference sample.

Unlike the embodiments described with reference to FIGS. 14-16, a subblock containing a sample adjacent to a reference sample may be set as an affine subblock. Specifically, a sample located outside an affine neighbor sub-block may be set as a reference sample, and a sub-block contained in an affine neighbor block may be set as an affine sub-block. For example, in the example shown in FIG. 14(a), a sample located on the right side of the upper right sample of the lower right sub-block can be set as a reference sample (x _n5 , y _n5 ) and the lower right sub-block can be set as an affine sub-block for the lower right corner. Alternatively, in the example shown in FIG. 14(b), a sample located to the left of the upper left sample of the lower left sub-block can be set as a reference sample (x _n4 , y _n4 ) and the lower left sub-block can be set as an affine sub-block for the lower left corner.

The embodiments described above can be equally applied to FIGS. 15 and 16. That is, in the example shown in FIG. 15(a) or FIG. 16(a), a sample located on the right side of the lower right sample or middle right sample of the lower right sub-block can be set as a reference sample (x _n5 , y _n5 ) and the lower right sub-block can be set as an affine sub-block for the lower right corner. Alternatively, in the example shown in FIG. 15(b) or FIG. 16(b), a sample located on the left side of the lower left sample or the middle left sample of the lower left sub-block can be set as a reference sample (x _n4 , y _n4 ) and the lower left sub-block can be set as an affine sub-block for the lower left corner.

As shown in the examples described above, the affine source vector of the affine neighbor block can be obtained using the motion vector of the affine sub-block. To this end, in the case of a block that has been encoded/decoded, the motion vector can be stored by a subblock element.

As another example, after storing a minimum number of affine source vectors for an affine neighbor block, the motion vector of the affine sub-block can be obtained using the stored affine source vectors.

Equations 9 and 10 show an example of obtaining the motion vector of an affine subblock using the affine source vectors of an affine neighboring block.

[Equation 9]

[Equation 10]

In Equations 9 and 10, (nv _4x , nv _4y ) denotes the affine subblock motion vector for the bottom left control point and (nv _5x , nv _5y ) denotes the affine subblock motion vector for the bottom right control point. Since the motion vector of the affine subblock and the affine source vector of the control point are set equal, (nv _4x , nv _4y ) can be replaced by the affine source vector (nv _2x , nv _2y ) for the bottom left control point or (nv _5x , nv _5y ) can be replaced affine source vector (nv _3x , nv _3y ) for the lower right control point.

(x _n4 , y _n4 ) denotes the position of the reference sample for the lower left subblock. Alternatively, the center position of the bottom left subblock or the position of the bottom left control point may be used instead. (x _n5 , y _n5 ) denotes the position of the reference sample for the lower right subblock. Alternatively, the center position of the lower right subblock or the position of the lower right control point may be used instead.

Equations 9 and 10 can be applied when the current block is not in contact with a code tree element boundary. When the current block is in contact with the upper boundary of a code tree element, instead of using Equations 9 and 10, the translation vector of the affine subblock determined based on the lower left reference sample can be set as the third affine source vector, and the translation vector of the affine subblock. determined based on the lower right reference sample, can be set as the fourth affine source vector.

In Equations 7 and 8, (x _n3 -x _n2 ) denotes the width between the bottom left control point and the bottom right control point. As described above, x _n3 may be replaced by the lower right reference position x _n5 and x _n2 may be replaced by the lower left reference position x _n4 . Next, (x _n3 -x _n2 ) or the value obtained by replacing the positions in the above equation with the positions of the reference samples (for example, (x _n5 -x _n4 )) is defined as the variable W _seed , and the variable is called the width of the original subvector.

According to the positions of the reference samples, a case may arise when the width of the original subvector is not a power series of the number 2 (for example, 2 ⁿ ). For example, when the lower left sample of the lower left subblock is set as the lower left reference sample and the lower right sample of the lower right subblock is set as the lower right reference sample, the width of the original subvector is not a multiple of 2. As described above, when the width of the original subvector is not a power series of the number 2, the width of the original subvector may be converted to a power series of 2. The transformation may involve adding an offset to or subtracting an offset from the width of the original subvector, or using the position of a sample adjacent to the reference sample instead of the position of the reference sample. For example, the transformed width of the original subvector can be obtained by adding 1 to the width between the lower left reference sample and the lower right reference sample. Alternatively, the width between an adjacent reference sample adjacent to the right side of the lower right reference sample and the lower left reference sample may be set as the transformed width of the original subvector. The affine source vector of the current block can then be obtained by applying the transformed width of the source subvector to Equations 7 and 8.

The division operation included in Equations 7 and 8 can be changed to a shift operation. The shift operation can be performed based on the value obtained from the transformed width of the original subvector (ie, the value expressed as a power series of 2).

When the reference sample used to determine the affine subblock does not belong to the affine neighbor block, the affine source vector of the affine neighbor block can be obtained based on the sample adjacent to the reference sample from the samples included in the affine neighbor block. Specifically, the translation vector of a subblock containing a sample adjacent to a reference sample in an affine neighbor block (hereinafter referred to as a neighbor reference sample) can be set as the affine source vector of the affine neighbor block. Thus, a method for obtaining an affine source vector using a neighboring reference sample can be defined as a modified method for obtaining an affine fusion vector.

In fig. 17 is a view showing an application example of the modified method for producing an affinity fusion vector.

When the lower right reference sample (x _n5 , y _n5 ) of the affine neighbor block E does not belong to the affine neighbor block, the affine source vector can be obtained based on the sample (x _n5 -1, y _n5 ) adjacent to the left side of the lower right reference sample , from the samples included in the affine neighboring block. In particular, the translation vector of the subblock containing the adjacent reference sample (x _n5 -1, y _n5 ) can be set as the affine source vector of the lower right reference point.

In the example shown in FIG. 17, it is shown that the sample adjacent to the right side of the upper right sample of the lower right sub-block is the lower right reference sample. Even when a sample adjacent to the right side of the lower right sample of the lower right sub-block, or a sample adjacent to the right side of the middle right sample of the lower right sub-block, is set as the lower right reference sample, an affine source vector can be obtained based on the sample adjacent to the left side adjacent reference reading.

Moreover, even when the lower left reference sample does not belong to an affine neighbor block, according to the embodiment described above, an affine source vector can be obtained based on a sample adjacent to the right side of the lower left reference sample.

Since the position of the reference sample and the sub-block to obtain the affine source vector are set differently, the width of the source sub-vector can be set to a power series of 2.

A fusion candidate, an affine source vector prediction candidate, or an affine source vector of the current block can be obtained using a neighboring block that has not been encoded in an inter-image affine mode in the vicinity of the current block. In particular, blocks not encoded in the inter-image affine mode can be combined, and their combination can be set as a fusion candidate or an affine source vector prediction candidate. For example, a combination of at least one of the motion vector of any of the adjacent blocks adjacent to the top left corner of the current block, the motion vector of any of the adjacent blocks adjacent to the top right corner of the current block, and the motion vector of any of the adjacent blocks adjacent to the bottom left corner of the current block, can be set as a fusion candidate or an affine source vector prediction candidate. In this case, the motion vector of the adjacent block adjacent to the upper left corner, the motion vector of the adjacent block adjacent to the upper right corner, and the motion vector of the adjacent block adjacent to the lower left corner can be set as the first affine source vector for the upper left control point , the second affine source vector for the top right control point and the third affine source vector for the bottom left control point, respectively.

Alternatively, a fusion candidate, an affine source vector prediction candidate, or an affine source vector of the current block can be obtained using a neighboring block that is not encoded in the inter-image affine mode, in a modified method for obtaining the affine merge vector that is described above. In the following, a neighboring block that is not encoded in the affine mode carried out between images will be called a non-affine neighboring block.

In fig. 18 is a view showing an example of obtaining an affine source vector of a current block based on a non-affine neighboring block.

In the example shown in FIG. 18, all neighbor blocks adjacent to the current block are assumed to be non-affine neighbor blocks.

When it is desired to obtain an affine source vector of the current block from a non-affine neighbor block A from neighboring blocks adjacent to the current block, a lower left reference sample and a lower right reference sample can be set for block A. For example, a sample adjacent to the left side of the block's lower left sample A may be set as the lower left reference count, and the lower right sample of block A may be set as the lower right reference count. Since the lower left reference sample is outside of block A, the motion vector of the subblock containing the sample adjacent to the right side of the lower left reference sample can be set as the third affine source vector of block A. Additionally, the motion vector of the subblock containing the lower right reference sample , can be set as the fourth affine source vector of block A. Then, the first affine source vector and the second affine source vector for the current block can be obtained from block A based on Equations 9 and 10.

The method of obtaining an affine source vector from a non-affine neighbor block can only be used when performing motion-compensated prediction on a non-affine neighbor block by a sub-block element. Here, the prediction method for performing motion compensation prediction by a subblock element may include at least one of STMVP, ATMVP, Bidirectional Optical Flow (BIO), Overlapping Block Motion Compensation (OBMC), and Decoder Side Motion Vector Refinement (DMVR) .

In the embodiment described above, it has been described that when the upper boundary of the current block is in contact with the boundary of a code tree element, a merge candidate, an affine source vector prediction candidate, or an affine source vector of the current block is obtained using the third affine source vector for the bottom left control point and a fourth affine source vector for the bottom right control point of the affine neighbor block located on top of the current block.

As another example, when the top boundary of the current block is in contact with the boundary of a code tree element, and a neighboring block located on top of the current block belongs to a code tree element different from the code tree element of the current block, a merge candidate, an affine prediction candidate source vector or an affine source vector of the current block can be obtained using the neighbor block closest to the neighbor block of the blocks included in the code tree element to which the current block belongs, instead of the neighbor block.

In the example shown in FIG. 19, it is shown that the current block is in contact with the upper boundary of the code tree element, and blocks B, C and E located on top of the current block belong to a code tree element different from the code tree element of the current block. Accordingly, instead of using block E, the affine source vector of the current block can be obtained using block F adjacent to block E from the blocks included in the code tree element to which the current block belongs.

For motion-compensated prediction of the current block, affine source vectors of multiple blocks can be used. For example, you can select a plurality of merge candidates from a list of merge candidates and obtain an affine source vector or a sub-block vector for the current block based on the affine source vectors of the selected merge candidates. Encoding/decoding a current block using affine source vectors of multiple blocks may be called a multiple affine fusion encoding method.

Information indicating whether a multiple affine merge encoding method is applied to the current block may be encoded and transmitted via a bit stream. Alternatively, whether to apply a multiple affine merge encoding method to the current block may be determined based on at least one of a number of affine neighbor blocks from neighboring blocks adjacent to the current block, a number of merge candidates included in the merge candidate list, and models of affine movement of the current block.

In fig. 20 and 21 are views showing the flow of a motion compensation prediction method in which a plurality of fusion candidates are used.

In fig. 20 is a view showing an example of obtaining affine source vectors of a current block using the affine source vectors of a plurality of merge candidates. In fig. 21 is a view showing an example of obtaining the motion vector of each sub-block using affine source vectors of a plurality of merge candidates.

The affine source vectors of the current block can be generated based on the summation, difference, averaging, or weighted summation of the affine source vectors of two merge candidates.

Equations 11 and 12 show an example of obtaining the affine source vector of the current block by summing the affine source vectors of merge candidates.

[Equation 11]

[Equation 12]

In Equations 11 and 12, sv ₄ denotes the first affine source vector of the current block, sv ₀ denotes the first affine source vector of the first merge candidate, and sv ₂ denotes the first affine source vector of the second merge candidate. In addition, sv ₅ denotes the second affine source vector of the current block, sv ₁ denotes the second affine source vector of the first merge candidate, and sv ₃ denotes the second affine source vector of the second merge candidate.

In addition, Equations 13 and 14 show an example of obtaining the affine source vector of the current block through a weighted summation operation of the affine source vectors of merge candidates.

[Equation 13]

[Equation 14]

As another example, a first sub-block motion vector and a second sub-block motion vector may be generated for each sub-block in the current block based on the affine source vectors of the first merge candidate and the affine source vectors of the second merge candidate. Thereafter, the final subblock motion vector may be generated based on a summation, difference, averaging, or weighted summation operation of the first subblock motion vector and the second subblock motion vector.

Equation 15 shows an example of obtaining a final sub-block motion vector by summing the first sub-block motion vector and the second sub-block motion vector.

[Equation 15]

In Equation 15, V ₀ denotes the first motion vector of the sub-block, V ₁ denotes the second motion vector of the sub-block, and V ₂ denotes the final motion vector of the sub-block.

In addition, Equation 16 shows an example of obtaining a final sub-block motion vector through a weighted summation operation of the first sub-block motion vector and the second sub-block motion vector.

[Equation 16]

Intra prediction is designed to predict the current block using the reconstructed samples that have been encoded/decoded around the current block. In this case, the samples reconstructed before applying the filter in the loop can be used for intra-prediction of the current block.

The intra prediction method includes matrix-based intra prediction and general intra prediction taking into account directionality relative to neighboring reconstructed samples. Information indicating the intra prediction method of the current block may be transmitted via a bitstream. The information may be a 1-bit flag. Alternatively, the intra prediction method of the current block may be determined based on at least one of the location, size and shape of the current block or based on the intra prediction method of the neighboring block. For example, when the current block is on an image boundary, it may be determined that there is no need to apply matrix-based intra prediction to the current block.

Intra prediction, which is matrix-based intra prediction, is a method of obtaining a block prediction of a current block by an encoder and decoder based on a matrix product between a previously stored matrix and reconstructed samples around the current block. Information to indicate any of a plurality of previously stored matrices may be transmitted via a bit stream. The decoder may determine a matrix for intra prediction of the current block based on the information and size of the current block.

General intra prediction is a method of obtaining a prediction block for a current block based on an intra prediction mode not based on angle information or an intra prediction mode based on angle information. Next, the process of performing intra prediction based on the general intra prediction will be described in more detail with reference to drawings.

In fig. 22 is a flowchart illustrating an intra prediction method according to one embodiment of the present invention.

The reference sample line of the current block can be determined (S2201). A reference sample line means a set of reference samples included in the kth line from the top and/or left side of the current block. The reference samples can be obtained from the reconstructed samples that have been encoded/decoded around the current block.

Index information for identifying a reference sample line of a current block from a plurality of reference sample lines may be transmitted by a bit stream. The plurality of reference sample lines may comprise at least one of a first line, a second line, a third line, and a fourth line from above and/or to the left of the current block. Table 1 shows the indices assigned to the reference sample lines, respectively. Table 1 assumes that the first line, second line, and fourth line are used as reference sample line candidates.

[ Table 1 ] Index Reference lines 0 First reference line 1 Second reference line 2 Fourth reference line

The reference sample line of the current block may be determined based on at least one of the location, size and shape of the current block or based on the predictive encoding mode of a neighboring block. For example, when the current block is in contact with the boundary of an image, tile, slice, or code tree element, the first reference sample line may be defined as the current block's reference sample line.

The line of reference samples may include top reference samples located above the current block and left reference samples located to the left of the current block. The top reference samples and left reference samples can be obtained from the reconstructed samples around the current block. The reconstructed samples may be in the state before applying the filter in the loop.

In fig. 23 is a view showing reference samples included in each reference sample line.

According to the intra prediction mode of the current block, a prediction sample can be obtained by using at least one of the reference samples belonging to the reference sample line.

Next, the intra prediction mode of the current block can be determined (S2202). Regarding the intra prediction mode of the current block, at least one of the intra prediction mode not based on angle information and the intra prediction mode based on angle information may be defined as the intra prediction mode of the current block. The intra prediction mode not based on angle information includes planar mode and DC mode, and the intra prediction mode based on angle information includes 33 or 65 modes between the down-left diagonal direction and the up-right diagonal direction.

In fig. 24 is a view showing intra prediction modes.

In fig. 24(a) shows 35 intra prediction modes, and FIG. 24(b) shows 67 intra prediction modes.

More or fewer intra prediction modes may be defined than those shown in FIG. 24.

The most probable mode (MPM) can be set based on the intra prediction mode of a neighboring block adjacent to the current block. Here, the neighbor block may include a left neighbor block adjacent to the left side of the current block and an upper neighbor block adjacent to the top side of the current block. When the coordinates of the top left sample of the current block are (0, 0), the left adjacent block may contain a sample at position (-1, 0), (-1, H-1) or (-1, (H-1)/2 ). In this case, H denotes the height of the current block. The top adjacent block may contain a sample at position (0, -1), (W-1, -1) or ((W-1)/2, -1). In this case, W denotes the width of the current block.

When a neighboring block is encoded using normal intra prediction, the MPM can be obtained based on the intra prediction mode of the neighboring block. Specifically, the intra prediction mode of the left neighbor block can be set as the variable candIntraPredModeA, and the intra prediction mode of the top neighbor block can be set as the variable candIntraPredModeB.

In this case, when a neighboring block is unavailable (for example, when a neighboring block is not yet encoded/decoded or when the position of a neighboring block is outside the image boundary), when a neighboring block is encoded using matrix-based intra prediction, when a neighboring block is encoded using prediction between pictures, or when a neighboring block is included in a code tree element other than the current block, the variable candIntraPredModeX (in this case X represents A or B) derived from the intra prediction mode of the neighboring block can be set as the default mode . Here, the default mode may comprise at least one of a planar mode, a DC mode, a vertical direction mode, and a horizontal direction mode.

Alternatively, when a neighboring block is encoded using matrix-based intra prediction, the intra prediction mode corresponding to the index value for indicating any of the matrices may be set to candIntraPredModeX. To this end, a lookup table showing the relationship between index values for specifying the matrix and intra prediction modes may be previously stored in the encoder and decoder.

MPMs can be obtained based on the variable candIntraPredModeA and the variable candIntraPredModeB. The number of MPMs included in the MPM list can be preset in the encoder and decoder. The number of MPMs included in the MPM list can be set in the encoder and decoder. For example, the number of MPMs may be 3, 4, 5, or 6. Alternatively, information indicating the number of MPMs may be transmitted via a bit stream. Alternatively, the number of MPMs may be determined based on at least one of a neighboring block predictive coding mode and the size and shape of the current block.

In the embodiments described below, the number of MPMs is assumed to be three, and the three MPMs are called MPM[0], MPM[1], and MPM[2]. When the number of MPMs is more than three, the MPMs can be configured to include three MPMs described in the embodiments described below.

When candIntraPredA and candIntraPredB are the same, and candIntraPredA is planar mode or DC mode, MPM[0] and MPM[1] can be set to planar mode and DC mode, respectively. MPM[2] can be set as a vertical direction intra prediction mode, a horizontal direction intra prediction mode, or a diagonal direction intra prediction mode. The diagonal direction intra prediction mode may be a down-left diagonal intra prediction mode, an up-left intra prediction mode, or an up-right intra prediction mode.

When candIntraPredA and candIntraPredB are the same, and candIntraPredA is an intra prediction mode based on angle information, MPM[0] can be set to the same as candIntraPredA. MPM[1] and MPM[2] can be set as intra prediction mode similar to candIntraPredA, respectively. An intra prediction mode similar to candIntraPredA may be an intra prediction mode in which the difference between candIntraPredA and the index is ±1 or ±2. The modulo (%) and offset operation can be used to obtain an intra prediction mode similar to candIntraPredA.

When candIntraPredA and candIntraPredB are different, MPM[0] can be set to the same as candIntraPredA, and MPM[1] can be set to the same as candIntraPredB. In this case, when both candIntraPredA and candIntraPredB are intra prediction modes not based on angle information, MPM[2] can be set to be the same as the vertical direction intra prediction mode, the horizontal direction intra prediction mode, or the intra predictions in the diagonal direction. Alternatively, when at least one of candIntraPredA and candIntraPredB is an intra prediction mode based on angle information, MPM[2] may be set to be the same as the intra prediction mode obtained by adding or subtracting an offset to or from the planar mode, mode DC or one of candIntraPredA and candIntraPredB, whichever is greater. In this case, the offset can be 1 or 2.

An MPM list containing a plurality of MPMs is generated, and information indicating whether an MPM that is the same as the intra prediction mode of the current block is included in the MPM list can be transmitted by a bitstream. The information is a 1-bit flag and may be called an MPM flag. When the MPM flag indicates that an MPM that is the same as that of the current block is included in the MPM list, index information indicating one of the MPMs may be transmitted via a bitstream. The MPM indicated by the index information can be set as the intra prediction mode of the current block. When the MPM flag indicates that an MPM that is the same as that of the current block is not included in the MPM list, residual mode information indicating any of the intra prediction residual modes other than MPM may be transmitted by a bit stream. The residual mode information indicates an index value corresponding to the intra prediction mode of the current block when indexes are reassigned to the residual intra prediction modes except MPM. The decoder may arrange the MPMs in ascending order and determine the intra prediction mode of the current block by comparing the residual mode information with the MPM. For example, when the residual mode information is less than or equal to MPM, the intra prediction mode of the current block can be obtained by adding 1 to the residual mode information.

Instead of setting the default mode to MPM, information indicating whether the intra prediction mode of the current block is the default mode may be transmitted via a bit stream. The information is a 1-bit flag, and the flag may be called a default mode flag. The default mode flag can be sent only when the MPM flag indicates that an MPM that is the same as that of the current block is included in the MPM list. As described above, the default mode may comprise at least one of a planar mode, a DC mode, a vertical direction mode, and a horizontal direction mode. For example, when planar mode is set as the default mode, the default mode flag may indicate whether the intra prediction mode of the current block is planar mode. When the default mode flag indicates that the intra prediction mode of the current block is not the default mode, one of the MPMs indicated by the index information may be set as the intra prediction mode of the current block.

When multiple intra prediction modes are set as default modes, index information indicating any of the default modes can be passed on. The intra prediction mode of the current block can be set as the default mode indicated by the index information.

When the reference line index of the current block is not 0, it can be set to not use the default mode. Accordingly, when the index of the reference sample line is not 0, the default mode flag is not transmitted, and the value of the default mode flag may be set to the same as a preset value (ie, false).

When the intra prediction mode of the current block is determined, prediction samples can be obtained for the current block based on the determined intra prediction mode (S2203).

When DC mode is selected, prediction samples are generated for the current block based on the average of the reference samples. In particular, the values of all samples in the prediction block can be generated based on the average value of the reference samples. The average may be obtained using at least one of the top reference samples located on top of the current block and the left reference samples located to the left of the current block.

Depending on the shape of the current block, the number or range of reference samples used to obtain the average may vary. For example, when the current block is a non-square block that is wider than it is tall, the average can be calculated using only the top reference samples. On the other hand, when the current block is a non-square block whose width is less than its height, the average can be calculated using only the left reference samples. That is, when the width and height of the current block differ, the average can be calculated using only the reference samples adjacent to the longer one. Alternatively, based on the width/height ratio of the current block, you can determine whether to calculate the average using only the top reference samples or calculate the average using only the left reference samples.

When the planar mode is selected, a prediction sample can be obtained using a horizontal direction prediction sample and a vertical direction prediction sample. Here, the prediction count in the horizontal direction is obtained based on the left reference sample and the right reference sample located on the same horizontal line as the prediction count, and the prediction count in the vertical direction is obtained based on the upper reference sample and the lower reference sample located on the same vertical line as the prediction count. Here, the right reference sample may be generated by copying a reference sample adjacent to the upper right corner of the current block, and the lower reference sample may be generated by copying the reference sample adjacent to the lower left corner of the current block. The prediction count in the horizontal direction can be obtained based on the weighted addition operation of the left reference sample and the right reference sample, and the prediction count in the vertical direction can be obtained based on the weighted addition operation of the upper reference sample and the lower reference sample. In this case, the weighting value assigned to each reference sample can be determined according to the position of the prediction sample. The prediction sample may be obtained based on an averaging operation or a weighted summation operation of the prediction sample in the horizontal direction and the prediction sample in the vertical direction. When the weighted addition operation is performed, weighting values applied to the prediction sample in the horizontal direction and the prediction sample in the vertical direction can be determined based on the position of the prediction sample.

When the angle information-based prediction mode is selected, a parameter indicating the prediction direction (or prediction angle) of the selected angle information-based prediction mode can be determined. Table 2 shows the intraPredAng parameter of the intra direction of each intra prediction mode.

[ Table 2 ] PredModeIntraIntraPredAng 1- 232 326 421 517 613 79 PredModeIntraIntraPredAng 85 92 100 11-2 12-5 13-9 14-13 PredModeIntraIntraPredAng 15-17 16-21 17-26 18-32 19-26 20-21 21-17 PredModeIntraIntraPredAng 22-13 23-9 24-5 25-2 260 272 285 PredModeIntraIntraPredAng 299 3013 3117 3221 3326 3432

When 35 intra prediction modes are defined, Table 2 shows the internal direction parameter of each intra prediction mode having any index from 2 to 34. When more than 33 intra prediction modes based on angle information are defined, Table 2 is further divided to set the internal parameter directions of each intra prediction mode based on angle information.

After arranging the top reference samples and left reference samples of the current block in a line, a prediction sample can be obtained based on the value of the inner direction parameter. In this case, when the value of the inner direction parameter is a negative value, the left reference samples and the top reference samples can be arranged in a line.

In fig. 25 and 26 are views showing an example of a one-dimensional array arranging reference samples in a line.

In fig. 25 is a view showing an example of a one-dimensional array in the vertical direction arranging reference samples in the vertical direction, and FIG. 26 is a view showing an example of a one-dimensional array in the horizontal direction placing reference samples in the horizontal direction. The embodiments shown in FIGS. 25 and 26 will be described with the assumption that 35 intra prediction modes are defined.

When the intra prediction mode index is any of 11-18, a one-dimensional array in the horizontal direction in which the top reference samples are rotated counterclockwise can be applied, and when the intra prediction mode index is any of 19-25, a one-dimensional array in vertical direction in which the left reference readings rotate clockwise. When arranging the reference samples in a line, the angle of the internal prediction mode can be taken into account.

The reference determination parameter may be determined based on the internal direction parameter. The reference sample definition parameter may comprise a reference sample index to indicate the reference sample and a weighting value parameter to specify a weighting value to apply to the reference sample.

The index iIdx of the reference count and the parameter _ifact of the weighting value can be obtained using Equations 17 and 18 shown below, respectively.

[Equation 17]

[Equation 18]

In Equations 17 and 18, P _ang denotes the internal direction parameter. The reference sample, indicated by the reference sample index iIdx, corresponds to an integer pixel.

At least one or more reference samples may be specified to obtain the prediction sample. In particular, the position of the reference sample used to obtain the prediction sample may be specified taking into account the slope of the prediction mode. For example, the reference sample used to obtain the prediction sample may be specified using the reference sample index iIdx.

In this case, when the slope of the intra prediction mode is not expressed by a single reference sample, the prediction sample can be generated by interpolating a plurality of reference samples. For example, when the slope of the intra prediction mode is a value between the slope between the prediction sample and the first reference sample and the slope between the prediction sample and the second reference sample, the prediction sample can be obtained by interpolating the first reference sample and the second reference sample. That is, when the corner line passing at the intra prediction angle does not pass through the reference sample located in the whole pixel, the prediction sample can be obtained by interpolating the reference samples adjacent to the left and right or the upper or lower sides of the position at which the corner line passes through.

Equation 19 shows an example of obtaining a prediction sample based on reference samples.

[Equation 19]

In Equation 19, P denotes the prediction sample and Ref_1D denotes any of the one-dimensionally located reference samples. In this case, the position of the reference sample can be determined by the position (x, y) of the prediction sample and the index iIdx of the reference sample.

When the intra prediction mode slope can be expressed as a single reference sample, the weighting value parameter i _fact is set to 0. Accordingly, Equation 19 can be simplified as shown in Equation 20.

[Equation 20]

Intra prediction for the current block may be performed based on a variety of intra prediction modes. For example, an intra prediction mode may be obtained for each prediction sample, and a prediction sample may be obtained based on the intra prediction mode assigned to each prediction sample.

Alternatively, an intra prediction mode may be obtained for each region, and intra prediction for each region may be performed based on the intra prediction mode assigned to each region. In this case, the area may contain at least one sample. At least one of the size and shape of the patch may be adaptively determined based on at least one of the size, shape, and intra prediction mode of the current block. Alternatively, at least one of the size and shape of the patch may be preset in the encoder and decoder to be independent of the size or shape of the current block.

Alternatively, intra prediction may be performed based on each of the plurality of intra predictions, and the final prediction sample may be obtained based on an averaging operation or a weighted summation operation of the plurality of prediction samples obtained by the plurality of intra predictions. For example, a first prediction sample may be obtained by performing intra prediction based on a first intra prediction mode, and a second prediction sample may be obtained by performing intra prediction based on a second intra prediction mode. Thereafter, the final prediction sample can be obtained based on an averaging operation or a weighted summation operation of the first prediction sample and the second prediction sample. In this case, the weighting value assigned to each of the first prediction sample and the second prediction sample may be determined taking into account at least one of whether the first intra prediction mode is a prediction mode not based on angle information/a prediction mode based on the angle information, whether the second intra prediction mode is a prediction mode not based on angle information/a prediction mode based on angle information, and an intra prediction mode of the adjacent block.

The plurality of intra prediction modes may be a combination of an intra prediction mode not based on angle information and a prediction mode based on angle information, a combination of prediction modes based on angle information, or a combination of prediction modes not based on angle information .

In fig. 27 is a view showing an example of angles formed between intra prediction modes based on angle information and a straight line parallel to the x-axis.

As shown in the example in FIG. 27, prediction modes based on angle information may exist between the down-left diagonal direction and the up-right diagonal direction. When described in relation to the angle formed by the x-axis and the prediction mode based on angle information, the prediction modes based on angle information may exist between 45 degrees (diagonal direction down and left) and -135 degrees (diagonal direction up and right ).

When the current block has a non-square shape, according to the intra prediction mode of the current block, it may be the case that a prediction sample is obtained by using a reference sample further from the prediction sample instead of a reference sample even closer to the prediction sample from reference samples located on a corner line passing at the angle of the internal predictions.

In fig. 28 is a view showing an embodiment of obtaining prediction samples when the current block has a non-square shape.

For example, as shown in the example of FIG. 28(a), it is assumed that the current block has a non-square shape whose width is larger than the height, and the intra prediction mode of the current block is an intra prediction mode based on angle information having an angle from 0 to 45 degrees. In the above case, when a prediction sample A near the right column of the current block is obtained, it may be a case of using the left reference sample L distant from the prediction sample instead of the top reference sample T located near the prediction sample from the reference samples located in the angular mode passing at an angle.

As another example, as shown in the example of FIG. 28(b), it is assumed that the current block has a non-square shape whose height is greater than the width, and the intra prediction mode of the current block is an intra prediction mode based on angle information having an angle of -90 to -135 degrees. In the above case, when a prediction sample A near the bottom line of the current block is obtained, it may be a case of using the top reference sample T away from the prediction sample instead of the left reference sample L located near the prediction sample from the reference samples located in the angular mode passing at an angle.

To solve the problem described above, when the current block has a non-square shape, the intra prediction mode of the current block can be replaced by an intra prediction mode in the opposite direction. Accordingly, prediction modes based on angle information having an angle larger or smaller than those of the prediction modes based on angle information shown in FIG. 24 can be used for a non-square shaped block. Such an intra prediction mode based on angle information can be defined as a wide-angle intra prediction mode. The wide-angle intra prediction mode represents an intra prediction mode based on angle information that does not belong to the range of 45 to -135 degrees.

In fig. 29 is a view showing wide-angle intra prediction modes.

In the example shown in FIG. 29, intra prediction modes with an index of -1 to -14 and intra prediction modes with an index of 67 to 80 represent wide-angle intra prediction modes.

Although 14 wide-angle intra prediction modes (-1 to -14) having an angle greater than 45 degrees and 14 wide-angle intra prediction modes (67 to 80) having an angle less than -135 degrees are shown in FIG. 29, more or less wide-angle intra prediction modes may be defined.

When the wide-angle intra prediction mode is used, the length of the top reference samples can be set to 2W+1, and the length of the left reference samples can be set to 2H+1.

Since the wide-angle intra prediction mode is used, sample A shown in FIG. 28(a) can be predicted using the reference sample T, and the sample A shown in FIG. 28(b) can be predicted using the reference sample L.

By adding the existing intra prediction modes and the N wide-angle intra prediction modes, a total of 67+N intra prediction modes can be used. For example, Table 3 shows the inner direction parameters of the intra prediction modes when 20 wide-angle intra prediction modes are defined.

[ Table 3 ] PredModeIntra -10 -9 -8 -7 -6 -5 -4 -3 -2 intraPredAngle 114 93 79 68 60 54 49 45 39 PredModeIntra -1 2 3 4 5 6 7 8 9 intraPredAngle 35 32 29 26 23 21 19 17 15 PredModeIntra 10 eleven 12 13 14 15 16 17 18 intraPredAngle 13 eleven 9 7 5 3 2 1 0 PredModeIntra 19 20 21 22 23 24 25 26 27 intraPredAngle -1 -2 -3 -5 -7 -9 -eleven -13 -15 PredModeIntra 28 29 thirty 31 32 33 34 35 36 intraPredAngle -17 -19 -21 -23 -26 -29 -32 -29 -26 PredModeIntra 37 38 39 40 41 42 43 44 45 intraPredAngle -23 -21 -19 -17 -15 -13 -eleven -9 -7 PredModeIntra 46 47 48 49 50 51 52 53 54 intraPredAngle -5 -3 -2 -1 0 1 2 3 5 PredModeIntra 55 56 57 58 59 60 61 62 63 intraPredAngle 7 9 eleven 13 15 17 19 21 23 PredModeIntra 64 65 66 67 68 69 70 71 72 intraPredAngle 26 29 32 35 39 45 49 54 60 PredModeIntra 73 74 75 76 intraPredAngle 68 79 93 114

When the current block has a non-square shape and the intra prediction mode of the current block obtained in step S2202 belongs to the conversion range, the intra prediction mode of the current block can be converted to a wide-angle intra prediction mode. The transformation range may be determined based on at least one of the size, shape, and ratio of the current block. In this case, the ratio can represent the ratio of the width and height of the current block.

When the current block has a non-square shape whose width is greater than its height, the transformation range can be set from the intra prediction mode index (eg, 66) of the diagonal up and right direction (the intra prediction mode index in the diagonal up and right direction is N). In this case, N can be determined based on the ratio of the current block. When the intra prediction mode of the current block belongs to the conversion range, the intra prediction mode can be converted to a wide-angle intra prediction mode. The transformation may be the subtraction of a predetermined value from the intra prediction mode, and the predetermined value may be the total number (eg, 67) of intra prediction modes excluding wide-angle intra prediction modes.

According to the embodiment described above, intra prediction modes 66 to 53 can be converted to wide-angle intra prediction modes -1 to -14, respectively.

When the current block has a non-square shape whose height is larger than the width, the transformation range can be set from the intra prediction mode index (eg, 2) of the down-left diagonal direction (down-left intra prediction mode index + M). In this case, M can be determined based on the ratio of the current block. When the intra prediction mode of the current block belongs to the conversion range, the intra prediction mode can be converted to a wide-angle intra prediction mode. The transformation may be adding a predetermined value to the intra prediction mode, and the predetermined value may be the total number (eg, 65) of intra prediction modes based on the angle information, excluding wide-angle intra prediction modes.

According to the embodiment described above, intra prediction modes 2 to 15 can be converted to wide-angle intra prediction modes 67 to 80, respectively.

Hereinafter, intra prediction modes belonging to the conversion range will be referred to as wide-angle replacement intra prediction modes.

The conversion range can be determined based on the ratio of the current block. For example, Tables 4 and 5 show the conversion range when 35 intra prediction modes and 67 intra prediction modes excluding wide-angle intra prediction modes are defined, respectively.

[ Table 4 ] Condition Replaced intra prediction modes W/H = 2 Modes 2, 3, 4 W/H > 2 Modes 2, 3, 4, 5, 6 W/H = 1 No H/W = 1/2 Modes 32, 33, 34 H/W < 1/2 Modes 30, 31, 32, 33, 34

[ Table 5 ] Condition Replaced intra prediction modes W/H = 2 Modes 2, 3, 4, 5, 6, 7 W/H > 2 Modes 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 W/H = 1 No H/W = 1/2 Modes 61, 62, 63, 64, 65, 66 H/W < 1/2 Modes 57, 58, 59, 60, 61, 62, 63, 64, 65, 66

As shown in the examples of Tables 4 and 5, the number of wide-angle replacement intra prediction modes belonging to the conversion range may vary according to the ratio of the current block.

Since wide-angle intra prediction modes are used in addition to existing intra prediction modes, resources required for encoding wide-angle intra prediction modes increase, and thus coding efficiency may be reduced. Accordingly, coding efficiency can be improved by encoding replacement intra prediction modes for wide-angle intra-prediction modes instead of encoding the wide-angle intra-prediction modes themselves.

For example, when the current block is encoded using the 67th wide-angle intra prediction mode, the 2nd intra prediction mode, which is a replacement intra prediction mode of the 67th wide-angle intra prediction mode, can be encoded using the intra prediction mode of the current block. In addition, when the current block is encoded using the -1st wide-angle intra prediction mode, the 66th intra prediction mode, which is a replacement intra prediction mode of the -1st wide-angle intra prediction mode, can be encoded using the intra prediction mode of the current block.

The decoder may decode the intra prediction mode of the current block and determine whether the decoded intra prediction mode belongs to the transform range. When the decoded intra prediction mode is a wide-angle replacement intra-prediction mode, the intra-prediction mode can be converted to a wide-angle intra-prediction mode.

Alternatively, when the current block is encoded using the wide-angle intra prediction mode, the wide-angle intra prediction mode may be encoded as is.

Intra prediction mode coding can be performed based on the MPM list described above. The following will describe in detail how to configure the MPM list. In the embodiments described below, it is assumed that 10 wide-angle intra prediction modes (-1 to -10) with an angle greater than 45 degrees and 10 wide-angle intra prediction modes (67 to 76) with an angle less than -135 degrees are defined.

When a neighboring block is encoded using a wide-angle intra prediction mode, the MPM may be set based on a wide-angle replacement intra prediction mode corresponding to the wide-angle intra prediction mode. For example, when a neighboring block is encoded using the wide-angle intra prediction mode, the variable candIntraPredX (X is A or B) can be set equal to the wide-angle replacement intra prediction mode.

Alternatively, the method for obtaining the MPM may be determined according to the shape of the current block. For example, when the current block is a square shape whose width is equal to its height, candIntraPredX can be set to the wide-angle intra prediction replacement mode. On the other hand, when the current block is non-square, candIntraPredX can be set to the wide-angle intra prediction mode.

Alternatively, whether to set candIntraPredX equal to the wide-angle intra prediction mode can be determined based on whether the wide-angle intra prediction mode of a neighboring block can be applied to the current block. For example, when the current block has a non-square shape whose width is greater than its height, the wide-angle intra prediction mode whose index is greater than the index of the diagonal up-right intra prediction mode is set to candIntraPredX as is. However, the wide-angle replacement intra prediction mode corresponding to it is set as candIntraPredX for the wide-angle intra prediction mode whose index is less than the index of the down-left diagonal direction intra prediction mode. On the other hand, when the current block has a non-square shape whose height is greater than its width, the wide-angle intra prediction mode whose index is smaller than the index of the diagonal down-left intra prediction mode is set to candIntraPredX as is. However, the wide-angle replacement intra prediction mode corresponding to it is set as candIntraPredX for the wide-angle intra prediction mode whose index is greater than the index of the diagonal upward and right direction intra prediction mode.

That is, whether to obtain an MPM using the wide-angle intra prediction mode as is or whether to obtain an MPM using the wide-angle replacement intra prediction mode can be determined according to whether a neighboring block encoded in the wide-angle intra prediction mode has the same or similar shape as the current block.

Alternatively, the wide-angle neighbor intra prediction mode can be set to candIntraPredX regardless of the shape of the current block.

Finally, candIntraPredX can be set equal to the wide-angle intra prediction mode or the wide-angle substituting intra-prediction mode of an adjacent block.

The MPM can be derived from candIntraPredA and candIntraPredB. In this case, the MPM can be obtained in an intra prediction mode like candIntraPredA or candIntraPredB. An intra prediction mode like candIntraPredA or candIntraPredB can be obtained based on the modulo and offset operation. In this case, the constant and offset used for the modulo operation can be defined differently according to the shape of the current block.

Table 6 shows an example of obtaining MPM according to the shape of the current block.

[Table 6] i) if cuWidth is equal to cuHeight
candModeList[0] = candIntraPredModeA
candModeList[1] = 2 + ((candIntraPredModeA + 61) % 64)
candModeList[2] = 2 + ((candIntraPredModeA - 1) % 64)
ii) otherwise (i.e. cuWidth is not equal to cuHeight)
candModeList[0] = candIntraPredModeA
candModeList[1] = 2 + ((candIntraPredModeA + 81) % 84)
candModeList[2] = 2 + ((candIntraPredModeA - 1) % 84)

It is assumed that candIntraPredA and candIntraPredB are the same, and candIntraPredA is an intra prediction mode based on angle information. When the current block is square-shaped, an intra prediction mode like candIntraPredA can be obtained based on a modulo operation based on a value obtained by subtracting 1 from the total number of intra prediction modes based on angle information, except for wide-angle intra prediction modes. For example, when the number of intra prediction modes based on angle information other than wide-angle intra prediction modes is 65, the MPM can be obtained based on the value obtained based on candIntraPredA and modulo 64 operation. On the other hand, when the current block has non-square form, an intra prediction mode like candIntraPredA can be obtained based on a modulo operation based on a value obtained by subtracting 1 from the total number of intra prediction modes based on angle information, including wide-angle intra prediction modes. For example, when the number of wide-angle intra prediction modes is 20, the MPM can be obtained based on the value obtained from candIntraPredA and modulo 84 operation.

Since the constant used for the modulo operation is set differently according to the shape of the current block, it can be determined whether the wide-angle intra prediction mode can be set to the same as the intra prediction mode based on angle information like candIntraPredA. For example, the wide-angle intra prediction mode may not be set to an intra prediction mode based on angle information like candIntraPredA in a modulo 64 operation, while the wide-angle intra prediction mode may be set to an intra prediction mode based on angle information. corners, similar to candIntraPredA in a modulo operation using 84.

Alternatively, when candIntraPredA and candIntraPredB are the same, the MPM can be obtained taking into account the shape of the current block and whether candIntraPredA is a wide-angle intra prediction mode.

Table 7 shows an example of obtaining MPM according to the shape of the current block.

[Table 7] - If candIntraPredModeB is equal to candIntraPredModeA, the following applies:
-- If candIntraPredModeA is less than 2 (i.e. equal to INTRA_PLANAR or INTRA_DC), candModeList[x] for x = 0...2 is obtained as follows:
candModeList[0] = INTRA_PLANAR
candModeList[1] = INTRA_DC
candModeList[2] = INTRA_ANGULAR50
-- otherwise candModeList[x] for x = 0.. 2 is obtained as follows:
i) if cuWidth is equal to cuHeight
1. if candIntrapredA is less than 0 or candIntraPredModeA is greater than 66
candModeList[0] = INTRA_PLANAR
candModeList[1] = INTRA_DC
candModeList[2] = INTRA_ANGULAR50
2. otherwise, if candIntraPredA >= 0 and candIntraPredModeA < 67)
candModeList[0] = candIntraPredModeA
candModeList[1] = 2 + ((candIntraPredModeA + 61) % 64)
candModeList[2] = 2 + ((candIntraPredModeA - 1) % 64)
ii) else if cuWidth is greater than cuHeight
candModeList[0] = candIntraPredModeA
candModeList[1] = 2 + ((candIntraPredModeA + 81) % 84)
candModeList[2] = 2 + ((candIntraPredModeA - 1) % 84)
iii) otherwise (i.e. cuWidth is greater than cuHeight)
candModeList[0] = candIntraPredModeA
candModeList[1] = 2 + ((candIntraPredModeA + 81) % 84)
candModeList[2] = -8 + ((candIntraPredModeA - 1) % 84)
- Otherwise (candIntraPredModeB is not equal to candIntraPredModeA), the following applies:
-- candModeList[0] and candModeList[1] are obtained as follows:
candModeList[0] = candIntraPredModeA
candModeList[1] = candIntraPredModeB
-- If neither candModeList[0] and candModeList[1] are equal to INTRA_PLANAR, candModeList[2] is set to INTRA_PLANAR
-- Else, if neither candModeList[0] and candModeList[1] are equal to INTRA_DC, candModeList[2] is set to INTRA_DC,
-- Otherwise candModeList[2] is set to INTRA_ANGULAR 50.

CandIntraPredA and candIntraPredB are assumed to be the same.

When the current block is square and candIntraPredA is a wide-angle intra prediction mode, MPMs can be set to the default modes. For example, MPM[0], MPM[1], and MPM[2] may be set to planar mode, DC mode, and vertical intra prediction mode, respectively.

When the current block has a square shape and candIntraPredA is not a wide-angle intra prediction mode but an intra prediction mode based on angle information, the MPMs can be set equal to candIntraPredA and an intra prediction mode based on angle information like it. For example, MPM[0] can be set to candIntraPredA, and MPM[1] and MPM[2] can be set to candIntraPredA and an internal prediction mode based on angle information like it.

When the current block has a non-square shape and candIntraPredA is an intra prediction mode based on angle information, the MPMs can be set equal to candIntraPredA and an intra prediction mode based on angle information like it. For example, MPM[0] can be set to candIntraPredA, and MPM[1] and MPM[2] can be set to candIntraPredA and an internal prediction mode based on angle information like it.

An internal prediction mode based on angle information, similar to candIntraPredA, can be obtained using modulo and offset operations. In this case, the constant used for the modulo operation can vary according to the shape of the current block. In addition, the offset used to obtain an intra prediction mode based on angle information like candIntraPredA can be set differently according to the shape of the current block. For example, when the current block has a non-square shape whose width is greater than its height, an intra prediction mode based on angle information like candIntraPredA can be obtained using an offset of 2. On the other hand, when the current block has a non-square shape whose height larger than the width, an intra prediction mode based on angle information, similar to candIntraPredA, can be obtained using offsets 2 and -8.

Alternatively, the MPMs may be derived by considering whether candIntraPredX is the wide-angle intra prediction mode having the largest or smallest index.

Table 8 shows an example of obtaining MPM taking into account the wide-angle intra prediction mode index.

[Table 8] - If candIntraPredModeB is equal to candIntraPredModeA, the following applies:
-- If candIntraPredModeA is less than 2 (i.e. equal to INTRA_PLANAR or INTRA_DC), candModeList[x] for x = 0...2 is obtained as follows:
candModeList[0] = INTRA_PLANAR
candModeList[1] = INTRA_DC
candModeList[2] = INTRA_ANGULAR50
-- otherwise candModeList[x] for x = 0.. 2 is obtained as follows:
i) If (candIntraPredModeA < 0 and candIntraPredModeA != -10)
candModeList[0] = candIntraPredModeA
candModeList[1] = 2 + ((candIntraPredModeA - 3) % 84)
candModeList[2] = 2 + ((candIntraPredModeA - 1) % 84)
ii) else if (candIntraPredModeA = -10)
candModeList[0] = candIntraPredModeA
candModeList[1] = -1
candModeList[2] = -9
iii) else if (candIntraPredModeA > 67 and candIntraPredModeA < 76)
candModeList[0] = candIntraPredModeA
candModeList[1] = 2 + ((candIntraPredModeA + 81) % 84)
candModeList[2] = 2 + ((candIntraPredModeA - 1) % 84)
iv) otherwise candIntraPredModeA is 67
candModeList[0] = candIntraPredModeA
candModeList[1] = 66
candModeList[2] = 68
v) otherwise candIntraPredModeA is 77
candModeList[0] = candIntraPredModeA
candModeList[1] = 76
candModeList[2] = 67
vi) else if (candIntraPredModeA >= 0 and candIntraPredModeA < 67)
candModeList[0] = candIntraPredModeA
candModeList[1] = 2 + ((candIntraPredModeA + 61) % 64)
candModeList[2] = 2 + ((candIntraPredModeA - 1) % 64)
- Otherwise (candIntraPredModeB is not equal to candIntraPredModeA), the following applies:
-- candModeList[0] and candModeList[1] are obtained as follows:
candModeList[0] = candIntraPredModeA
candModeList[1] = candIntraPredModeB
-- If neither candModeList[0] and candModeList[1] are equal to INTRA_PLANAR, candModeList[2] is set to INTRA_PLANAR
-- Else, if neither candModeList[0] and candModeList[1] are equal to INTRA_DC, candModeList[2] is set to INTRA_DC,
-- Otherwise candModeList[2] is set to INTRA_ANGULAR 50.

CandIntraPredA and candIntraPredB are assumed to be the same. For convenience of explanation, wide-angle intra prediction modes having an index value less than that of the diagonal down and left intra prediction mode are called wide-angle intra prediction modes in the downward direction, and wide-angle intra prediction modes having an index value greater than that of the diagonal intra prediction mode up and right directions are called wide-angle right direction intra prediction modes.

When candIntraPredA is a wide-angle downward intra prediction mode, the MPMs may be set equal to candIntraPredA and an angle information-based intra prediction mode like it. In this case, when candIntraPredA is the wide-angle downward intra prediction mode having the smallest value, the MPM may be set equal to the wide-angle downward intra prediction mode having the predetermined index value. Here, the predetermined index may be an index having the largest value among the indexes of the wide-angle intra prediction modes in the downward direction. For example, when candIntraPredA is -10, MPM[0], MPM[1], and MPM[2] can be set to -10, -1, and -9, respectively.

When candIntraPredA is a wide-angle intra-prediction mode in the right direction, the MPMs may be set equal to candIntraPredA and an intra-prediction mode based on angle information like it. In this case, when candIntraPredA is the right-direction wide-angle intra prediction mode having the largest value, the MPM may be set equal to the right-direction wide-angle intra prediction mode having the predetermined index value. Here, the predetermined index may be an index having the smallest value among the indexes of the wide-angle intra prediction modes in the right direction. For example, when candIntraPredA is 77, MPM[0], MPM[1], and MPM[2] can be set to 77, 76, and 67, respectively.

Alternatively, when the index obtained by subtracting or adding 1 from or to the candIntraPredA index is less than the smallest value or greater than the largest value of the intra prediction mode indices, the MPM may be set to the default mode. Here, the default mode may comprise at least one of a planar mode, a DC mode, a vertical intra prediction mode, a horizontal intra prediction mode, and a diagonal intra prediction mode.

Alternatively, when the index obtained by subtracting or adding 1 from or to the candIntraPredA index is less than the smallest value or greater than the largest value of the intra prediction mode indices, MPM may be set equal to the intra prediction mode opposite candIntraPredA or the mode intra prediction, similar to the intra prediction mode, the opposite of candIntraPredA.

Alternatively, MPM candidates can be obtained taking into account the shape of the current block and the shape of the neighboring block. For example, the method for obtaining the MPM when both the current block and the adjacent block are non-square in shape may be different from the method for obtaining the MPM when the current block is square in shape and the adjacent block is in a non-square shape.

The MPMs in the MPM list may be permuted (or reordered) based on at least one of the size of the current block, the shape of the current block, the size of the adjacent block, and the shape of the adjacent block. In this case, the permutation indicates a reassignment of the indices assigned to each of the MPMs. For example, a smaller index may be assigned to the same MPM as the intra prediction mode of a neighboring block having the same size or shape as the current block.

It is assumed that MPM[0] and MPM[1] are set to the intra prediction mode candIntraPredA of the left neighbor block and the intra prediction mode candIntraPredB of the top neighbor block, respectively.

When the current block and the upper neighboring block have a non-square shape whose width is greater than the height, the MPMs may be rearranged such that the intra prediction mode candIntraPredB of the upper neighboring block may have a smaller index. That is, candIntraPredB can be swapped to MPM[0], and candIntraPredA can be swapped to MPM[1].

Alternatively, when the current block and the upper neighboring block have a non-square shape whose height is greater than the width, the MPMs can be rearranged such that the intra prediction mode candIntraPredB of the upper neighboring block has a smaller index. That is, candIntraPredB can be swapped to MPM[0], and candIntraPredA can be swapped to MPM[1].

Alternatively, when the current block and the upper neighboring block are square-shaped, the MPMs can be rearranged such that the intra prediction mode candIntraPredB of the upper neighboring block has a smaller index. That is, candIntraPredB can be swapped to MPM[0], and candIntraPredA can be swapped to MPM[1].

Instead of permuting the MPM, when candIntraPredX is initially assigned to the MPM, at least one of the size of the current block, the shape of the current block, the size of the adjacent block, and the shape of the adjacent block can be considered.

MPMs can be rearranged based on the size or shape of the current block. For example, when the current block has a non-square shape whose width is greater than its height, the MPMs may be rearranged in descending order. On the other hand, when the current block has a non-square shape whose height is greater than its width, the MPMs can be rearranged in ascending order.

The resulting residual video can be obtained by subtracting the prediction video from the original video. In this case, when the residual video is changed to the frequency domain, the subjective quality of the video is not significantly reduced although the high-frequency components are removed from the frequency components. Accordingly, when the high-frequency component values are converted to small values or the high-frequency component values are set to 0, the effect of increasing compression efficiency occurs without generating significant visual distortion. Reflecting this characteristic, the current block can be transformed to decompose the residual video into two-dimensional frequency components. The conversion can be done using a transform method such as discrete cosine transform (DCT) or discrete sine transform (DST).

DCT decomposes (or transforms) the residual video into two-dimensional frequency components using a cosine transform, and DST decomposes (or transforms) the residual video into two-dimensional frequency components using a sine transform. As a result of the residual video transformation, the frequency components can be expressed as the base video. For example, when DCT is performed on an N×N block, N ² base circuit components can be obtained. The size of each of the base circuit components included in an N×N block can be obtained through transformation. According to the conversion method used for this, the component size of the underlying circuit can be called the DCT coefficient or the DST coefficient.

The DCT conversion method is mainly used for video conversion in which many non-zero low-frequency components are distributed. The DST conversion method is mainly used for videos in which many high-frequency components are distributed.

The residual video may be converted using a conversion method other than DCT or DST.

In the following, the conversion of residual video into two-dimensional frequency components will be referred to as two-dimensional video conversion. In addition, the size of the base circuit components resulting from the transformation is called the transformation factor. For example, the conversion factor can mean a DCT factor or a DST factor. When both the first transform and the second transform described below are applied, the transform factor may indicate the size of the base circuit component generated as a result of the second transform.

The conversion method can be defined for each block. The transformation method may be determined based on at least one of a predictive coding mode of the current block, a size of the current block, and a size of the current block. For example, when the current block is encoded in the intra prediction mode and the size of the current block is smaller than N×N, the transformation can be performed using the DST transformation method. On the other hand, when the above conditions are not satisfied, the conversion can be done using the DCT conversion method.

2D video conversion may not be performed for some residual video blocks. Failure to perform 2D video conversion may be referred to as skip conversion. When skip transform is applied, quantization can be applied to the residual coefficients that were not transformed.

After converting the current block using DCT or DST, the converted current block can be converted again. In this case, the transformation based on the DCT or DST can be defined as the first transform, and the retransformation of the block to which the first transform is applied can be defined as the second transform.

The first transformation may be performed using any of a variety of candidate transformation bases. For example, the first conversion may be performed using any of DCT2, DCT8 or DCT7.

Different transformation bases can be used for the horizontal direction and the vertical direction. Information indicating a combination of a translation base in the horizontal direction and a transformation basis in the vertical direction may be transmitted by a bit stream.

The elements for performing the first transformation and the second transformation may be different. For example, the first transformation may be performed on an 8x8 block, and the second transformation may be performed on a 4x4 subblock of the transformed 8x8 block. In this case, the transformation coefficients of the residual areas for which the second transformation has not been performed can be set to 0.

Alternatively, the first transform may be performed on the 4x4 block and the second transform may be performed on the 8x8 portion containing the transformed 4x4 block.

Information indicating whether the second conversion has been performed may be transmitted via the bit stream.

The decoder may invert the second transform (second inverse transform) and may inverse the first transform (first inverse transform) with respect to the result of the inverse transform. By performing the second deconversion and the first deconversion, residual signals for the current block can be obtained.

Quantization is designed to reduce the energy of a block, and the quantization process involves the process of dividing the conversion coefficient into a specific constant value. A constant value can be obtained using a quantization parameter, and the quantization parameter can be defined as a value from 1 to 63.

When the encoder performs the transform and quantization, the decoder can obtain the residual block by inverse quantization and inverse transform. The decoder can obtain a reconstructed block for the current block by adding the prediction block and the residual block.

When the reconstructed block of the current block is obtained, the information loss occurring in the quantization and encoding process can be reduced by in-loop filtering. The filter in the loop may include at least one of a deblocking filter, an adaptive sample offset filter (SAO), and an adaptive loop filter (ALF). In the following, the reconstructed block before applying the filter in the loop is called the first reconstructed block, and the reconstructed block after applying the filter in the loop is called the second reconstructed block.

The second reconstructed block may be obtained by applying at least one of a deblocking filter, SAO, and ALF to the first reconstructed block. In this case, SAO or ALF can be applied after applying the deblocking filter.

The deblocking filter is designed to reduce the video quality degradation (block artifact) at the block boundary that occurs when performing quantization on individual blocks. To apply a deblocking filter, a block quality score (BS) between the first reconstructed block and the adjacent reconstructed block can be determined.

In fig. 30 is a flowchart illustrating the process of determining the quality indicator of a block.

In the example shown in FIG. 30, P denotes the first reconstructed block, and Q denotes the adjacent reconstructed block. In this case, the adjacent reconstructed block may be a block adjacent to the left side or top side of the current block.

In the example shown in FIG. 30, it is shown that the block quality is determined taking into account the P and Q predictive coding modes, whether a non-zero transform coefficient is included, whether inter-picture prediction is performed using the same reference picture, and whether the value the difference between motion vectors is greater than or equal to the threshold value.

Whether a deblock filter should be used can be determined based on the quality of the block. For example, when the block quality score is 0, filtering may not be performed.

SAO is designed to reduce the ringing phenomenon (ring artifact) that occurs when performing frequency domain quantization. SAO can be performed by adding or subtracting an offset determined given the design of the first recovered video. The method for determining the offset includes edge offset (EO) or stripe offset. EO denotes a method for determining the offset of the current sample according to the pattern of surrounding pixels. BO denotes a method of applying a common bias to a set of pixels having similar brightness values in an area. Specifically, the brightness of pixels can be divided into 32 equal sections, and pixels having similar brightness values can be set as one set. For example, four adjacent bands of 32 bands can be set as one group, and the same offset value can be applied to the samples belonging to the four bands.

ALF is a method of generating a second restored video by applying a filter having a predetermined size/shape to a first restored video or a restored video to which a deblocking filter has been applied. Equation 21 shows an example application of ALF.

[Equation 21]

Any of the predefined candidate filters may be selected as an image element, a code tree element, an encoding block, a prediction block, or a transform block. Each of the candidate filters may differ in either size or shape.

In fig. 31 is a view showing preset candidate filters.

As shown in the example in FIG. 31, at least one of the 5x5, 7x7, and 9x9 diamond shapes may be selected.

Only the 5x5 diamond shape can be used for the chroma component.

For real-time or low-latency encoding of high-definition video, such as panoramic videos, 360-degree videos, or 4K/8K UHD (ultra high-definition) video, a method for dividing the image into multiple sections and encoding/decoding multiple sections in parallel can be considered. . To do this, the image can be divided into tiles, i.e. basic element of parallel encoding/decoding, and tiles can be processed in parallel.

Tiles can be constrained to be rectangular in shape. When encoding/decoding a tile, data from other tiles is not used. The Context Adaptive Binary Arithmetic Coding (CABAC) context probability table may be initialized by the tile element, and may be set not to apply an in-loop filter at tile boundaries.

In fig. 32 is a view showing an example of dividing an image into a plurality of tiles.

The tile contains at least one code tree element, and the tile's boundary overlaps with the border of the code tree element.

As shown in the example in FIG. 32, the image may be divided into multiple sets of tiles. Information for dividing an image into multiple sets of tiles may be transmitted via a bitstream.

According to the type of image partitioning, tiles can have the same size in all areas except for the image borders.

Alternatively, the image may be sliced such that tiles adjacent in the horizontal direction may have the same height, or the image may be sliced such that tiles adjacent in the vertical direction may be the same width.

Since the image is divided using at least one of a vertical line and a horizontal line intersecting the image, each of the tiles belongs to its own column and/or row. In the embodiment described below, the column to which a tile belongs is called a tile column, and the row to which a tile belongs is called a tile row.

Information for determining the tile form of the image may be transmitted via a bitstream. The information may be encoded and transmitted using a picture parameter set or a sequence parameter set. The information is for determining the number of tiles in the image, and may include information indicating the number of rows of tiles and information indicating the number of columns of tiles. For example, the num_tile_columns_minus1 syntax element specifies the value obtained by subtracting 1 from the number of tile columns, and the num_tile_rows_minus1 syntax element specifies the value obtained by subtracting 1 from the number of tile rows.

In the example shown in FIG. 32, since the number of tile columns is 4 and the number of tile rows is 3, num_tile_columns_minus1 can be 3, num_tile_rows_minus1 can be 2.

When an image is divided into multiple tiles, information indicating the size of the tile may be transmitted via a bitstream. For example, when an image is divided into a plurality of tile columns, information indicating the width of each tile column is transmitted via a bitstream, and when an image is divided into a plurality of tile rows, information indicating the height of each tile row is transmitted via a bitstream. For example, a column_width_minus1 syntax element indicating the width of a column of tiles may be encoded and transmitted for each tile column, and a row_height_minus1 syntax element indicating the height of a row of tiles may be encoded and transmitted for each row of tiles.

Column_width_minus1 may indicate a value obtained by subtracting 1 from the width of a column of tiles, and row_height_minus1 may indicate a value obtained by subtracting 1 from the height of a row of tiles.

The column_width_minus1 encoding can be skipped for the last column of tiles, and the row_height_minus1 encoding can be skipped for the last row of tiles. The width of the last column of tiles and the height of the last row can be obtained taking into account the image size.

The decoder can determine the tile size based on column_width_minus1 and row_height_minus1.

Table 9 shows the syntax table for dividing an image into tiles.

As shown in Table 9, a num_tile_columns_minus1 syntax element indicating the number of tile columns and a num_tile_rows_minus1 syntax element indicating the number of rows may be passed.

Next, a uniform_spacing_flag syntax element can be passed, indicating whether the image is divided into tiles of equal size. When uniform_spacing_flag is true, tiles in the rest of the area excluding the borders of the image can be spaced equally.

When uniform_spacing_flag is false, the column_width_minus1 syntax element indicating the width of each column of tiles and the row_height_minus1 syntax element indicating the height of each row of tiles may be passed.

The syntax element loop_filter_across_tiles_enabled_flag specifies whether the edge filter is enabled on tile boundaries.

A column of tiles having the smallest width of the tile columns may be called a minimum-width tile, and a row of tiles having the smallest height of the tile rows may be called a minimum-height tile. Information indicating the width of a minimum-width tile and information indicating a height of a minimum-height tile may be transmitted by a bit stream. For example, the syntax element min_column_width_minus1 specifies the value obtained by subtracting 1 from the width of the minimum-width tile, and the min_row_height_minus1 syntax element specifies the value obtained by subtracting 1 from the height of the minimum-height tile.

Information indicating the difference value with the minimum tile width can be passed for each column of tiles. For example, the diff_column_width syntax element specifies the value of the difference in width between the current tile column and the minimum tile column. The value of the difference in width can be expressed as the value of the difference in the number of columns of code tree elements. The decoder can obtain the width of the current tile by adding the minimum width tile width obtained from min_column_width_minus1 and the difference width value obtained from diff_column_width.

In addition, information indicating the difference value with the minimum tile height may be transmitted for each row of tiles. For example, the diff_row_height syntax element specifies the height difference value between the current tile row and the minimum tile row. The value of the difference in height can be expressed as the value of the difference in the number of rows of code tree elements. The decoder can obtain the height of the current tile by adding the minimum height tile height obtained from min_row_height_minus1 and the height difference value obtained from diff_row_height.

Table 10 shows a syntax table containing size difference information.

The image can be broken up so that horizontally adjacent tiles can have different heights, or the image can be broken up so that vertically adjacent tiles can have different widths. Such an image dividing method may be called a flexible tiling method, and tiles divided by the flexible tiling method may be called flexible tiles.

In fig. 33 is a view showing an image division scheme according to the flexible tile production method.

The search order of tiles generated by image partitioning may follow a predefined scanning order. In addition, an index can be assigned to each of the tiles according to a predefined scan order.

The tile scanning order can be any of raster scanning, diagonal scanning, vertical scanning, and horizontal scanning. In fig. 33(a)-33(d) respectively show examples of assigning an index to each of the tiles according to raster scanning, diagonal scanning, vertical direction scanning and horizontal scanning.

The next scanning order can be determined according to the size or position of the current tile. For example, when the height of the current tile and the height of the tile adjacent to the right side of the current tile are different (for example, when the height of the tile adjacent to the right is greater than the height of the current tile), the tile located on the leftmost side of the tiles located on the same vertical line , that the tile adjacent to the bottom side of the current tile can be defined as a scan target after the current tile.

The order in which tiles are scanned can be determined by an image or sequence element.

Alternatively, the order in which the tiles are scanned may be determined based on the size of the first tile in the image. For example, when the width of the first tile is greater than the height, the scanning order of the tiles can be set to horizontal scanning. When the height of the first tile is greater than the width, the scanning order can be set to vertical scanning. When the width of the first tile is equal to the height, the scanning order of the tiles can be set as raster scanning or diagonal scanning.

Information indicating the total number of tiles may be transmitted via a bitstream. For example, when using the flexible tiles method, the syntax element number_of_tiles_in_picture_minus2, obtained by subtracting 2 from the total number of tiles in the picture, may be passed. The decoder can recognize the number of tiles contained in the current picture based on number_of_tiles_in_picture_minus2.

Table 11 shows a syntax table containing information about the number of tiles.

To reduce the number of bits required to encode the tile size, information indicating the size of the subtile may be encoded and transmitted. A subtile is the basic element that makes up a tile, and each tile can be configured to contain at least one subtile. A subtile may contain one or more code tree elements.

For example, the subtile_width_minus1 syntax element specifies the value obtained by subtracting 1 from the subtile's width. The subtile_height_minus1 syntax element specifies the value obtained by subtracting 1 from the subtile's height.

Information may be encoded and transmitted indicating whether other tiles other than the first tile are the same size as the previous tile. For example, the use_previous_tile_size_flag syntax element indicates whether the size of the current tile is the same as the size of the previous tile. When use_previous_tile_size_flag is true, it indicates that the size of the current tile is the same as the size of the previous tile. When use_previous_tile_size_flag is false, information indicating the size of the current tile can be encoded and transmitted. For the first tile, the use_previous_tile_size_flag encoding may be skipped and the flag may be set to false.

The tile size information may comprise a tile_width_minus1[i] syntax element indicating the width of the i-th tile, and a tile_height_minus1[i] syntax element indicating the height of the i-th tile.

Information indicating the size of the tile may indicate a difference value with the size of the subtile. Since subtile size information is used, encoding/decoding efficiency can be improved by reducing the number of bits required to encode the size of each tile. For example, the width of the i-th tile tileWidth can be obtained based on Equation 22 shown below, and the height of the i-th tile tileHeight can be obtained based on Equation 23 shown below.

[Equation 22]

[Equation 23]

Alternatively, encoding of the subtile size information may be skipped, and the size of the i-th tile may be encoded as is as subtile size information. The subtile size information may optionally be encoded. Information indicating whether the subtile size information is encoded may be conveyed by a video parameter set, a sequence parameter set, or an image parameter set.

The tile size-related information described above may be encoded and transmitted as indicating the number of code tree elements. For example, column_width_minus1, min_column_width_minus1, subtile_width_minus1, or tile_width_minus1 may indicate the number of columns of code tree elements contained in a tile. Additionally, diff_column_width can specify the difference value between the number of columns of code tree elements contained in the minimum width tile and the number of columns of code tree elements contained in the current tile.

Additionally, row_height_minus1, min_row_height_minus1, subtile_height_minus1, or tile_height_minus1 may indicate the number of rows of code tree elements contained in the tile. Additionally, diff_row_height can indicate the difference value between the number of code tree rows contained in the minimum height tile and the number of code tree rows contained in the current tile.

The decoder may determine the tile size based on the number of columns of code tree elements and/or the number of rows of code tree elements obtained based on the syntax elements and size of the code tree element. For example, the width of the i-th tile can be set to (tile_width_minus1[i] + 1)*(width of the code tree element), and the height of the i-th tile can be set to (tile_height_minus1[i] + 1)*(height of the code tree element tree).

At the same time, information indicating the size of a code tree element may be transmitted by a sequence parameter set or an image parameter set.

Table 11 describes that the use_previous_tile_size_flag syntax element is used to indicate whether the size of the current tile is the same as the size of the previous tile. As another example, information indicating whether the width of the current tile is the same as the width of the previous tile, or information indicating whether the height of the current tile is the same as the height of the previous tile may be encoded and transmitted.

Table 12 shows a syntax table containing information indicating whether the width of the current tile is the same as the width of the previous tile.

The use_previous_tile_width_flag syntax element specifies whether the width of the current tile is the same as the width of the previous tile. When use_previous_tile_width_flag is true, the width of the current tile can be set to the same as the width of the previous tile. In this case, the encoding of information indicating the width of the current tile can be skipped, and the width of the current tile can be obtained from the width of the previous tile.

When use_previous_tile_width_flag is false, information indicating the width of the current tile may be passed. For example, tile_width_minus1[i] could indicate a value obtained by subtracting 1 from the width of the i-th tile.

The use_previous_tile_width_flag syntax element can be encoded and passed only when it is determined that the size of the current tile is different from the size of the previous tile (for example, when the value of use_previous_tile_size_flag is 0).

Tile_width_minus1[i] can be the value obtained by subtracting 1 from the number of columns of code tree elements contained in the i-th tile. The decoder can obtain the number of columns of code tree elements belonging to the i-th tile by adding 1 to tile_width_minus1[i], and calculate the width of the tile by multiplying the resulting value by the width of the code tree element.

Table 13 shows a syntax table further containing information indicating whether the height of the current tile is the same as the height of the previous tile.

The use_previous_tile_height_flag syntax element specifies whether the height of the current tile is the same as the height of the previous tile. When use_previous_tile_height_flag is true, the height of the current tile can be set to the same as the height of the previous tile. In this case, the encoding of information indicating the height of the current tile can be skipped, and the height of the current tile can be obtained from the height of the previous tile.

When use_previous_tile_height_flag is false, information indicating the height of the current tile can be passed. For example, tile_height_minus1[i] could indicate the value obtained by subtracting 1 from the height of the i-th tile.

The use_previous_tile_height_flag syntax element can be encoded and passed only when it is determined that the size of the current tile is different from the size of the previous tile (for example, when the value of use_previous_tile_size_flag is 0). Additionally, the use_previous_tile_height_flag syntax element can only be passed when use_previous_tile_width_flag is false.

Table 12 shows an example use case for use_previous_tile_width_flag, and Table 13 shows an example use case for use_previous_tile_width_flag and use_previous_tile_height_flag. Although not shown in the tables above, the encoding of use_previous_tile_width_flag can be omitted and only use_previous_tile_height_flag can be used.

Which of use_previous_tile_height_flag and use_previous_tile_size_flag will be used can be determined based on at least one of the scanning order of the tiles, the width and height of the first tile, and the width and height of the previous tile. For example, when the tile scanning order is a vertical direction, use_previous_tile_height_flag can be used, whereas when the tile scanning order is a horizontal direction, use_previous_tile_width_flag can be used. Alternatively, when the first tile or previous tile has a non-square shape whose width is greater than its height, use_previous_tile_width_flag can be used, whereas when the first tile or previous tile has a non-square shape whose height is greater than its width, use_previous_tile_height_flag can be used.

While transmitting the number of tiles contained in an image, encoding of information related to the tile size may be skipped for the last tile.

Table 14 shows an example in which the encoding of tile size information is skipped for the last tile.

When the size of tiles other than the last tile is specified, the remaining area in the image can be set as the last tile.

Each code tree element may be assigned an identifier to identify the tile to which the code tree element belongs (hereinafter referred to as Tile ID).

In fig. 34 is a view showing an example of assigning a tile ID to each element of the code tree.

The same tile ID can be assigned to code tree elements belonging to the same tile. In particular, the Nth TileID can be assigned to code tree elements belonging to tile N.

Variables x and y indicating the position of the code tree element in the image can be defined to determine the tile ID assigned to each code tree element. Here, x denotes the value obtained by dividing the x-coordinate from the position (x0, y0) of the top left sample of the code tree element by the width of the code tree element, and y denotes the value obtained by dividing the y-coordinate from the position (x0, y0) of the top left reference of the code tree element by the height of the code tree element. Specifically, x and y can be obtained using equations 24 and 25 shown below.

[Equation 24]

[Equation 25]

Assigning a tile ID to each code tree element can be done using the process described below.

i) Tile ID initialization

The tile ID of each code tree element can be initialized to a value obtained by subtracting 1 from the number of tiles in the image.

ii) Obtaining tile ID

In the embodiment described above, it was described that a flag indicating whether an in-line filter is allowed to be applied at tile boundaries is passed through an image parameter set. However, when it is set not to use an in-line filter on all tile boundaries, the problem of reducing subjective image quality and reducing encoding efficiency may arise.

Therefore, information indicating whether to enable a filter in the loop for each tile can be encoded and transmitted.

In fig. 35 is a view showing an example of selectively determining whether to apply a filter in a loop to each tile.

As shown in the example in FIG. 35, whether to enable an in-loop filter (eg, deblock filter, SAO and/or ALF) on a horizontal or vertical boundary can be determined on a per-tile basis.

Table 17 shows an example in which information indicating whether to enable a filter in the loop is encoded for each tile.

In the example in Table 17, the syntax element loop_filter_across_tiles_flag[i] specifies whether the loop filter is allowed to be applied to the i-th tile. When the value of loop_filter_across_tiles_flag[i] is 1, it indicates that the loop filter can be used on the horizontal and vertical boundaries of the tile whose tile ID is i. When loop_filter_across_tiles_flag[i] is 0, it indicates that the loop filter cannot be used on the horizontal and vertical boundaries of a tile whose tile ID is i.

Information may be encoded indicating whether to enable a filter in the loop for each of the horizontal and vertical directions.

Table 18 shows an example in which information indicating whether to enable a filter in the loop is encoded separately for the horizontal direction and the vertical direction.

In the example in Table 18, the loop_filter_hor_across_tiles_flag[i] syntax element specifies whether a filter is allowed on a loop that crosses the i-th tile in the horizontal direction. The loop_filter_ver_across_tiles_flag[i] syntax element specifies whether a filter is allowed on a loop that crosses the i-th tile in the vertical direction.

When the value of loop_filter_hor_across_tiles_flag[i] is 1, it indicates that the loop filter can be used on the horizontal boundary of a tile whose tile ID is i. When loop_filter_hor_across_tiles_flag[i] is 0, it indicates that the loop filter cannot be used on the horizontal boundary of a tile whose tile ID is i.

When the value of loop_filter_ver_across_tiles_flag[i] is 1, it indicates that an in-loop filter can be used on the vertical boundary of a tile whose tile ID is i, and when the value of loop_filter_ver_across_tiles_flag[i] is 0, it indicates that an in-loop filter cannot be used. used on the vertical border of a tile whose tile ID is i.

Alternatively, information indicating whether a filter is enabled in a loop for a tile group containing a plurality of tiles may be encoded and transmitted. The information can be used to determine whether to enable a filter in the loop for a plurality of tiles contained in a group of tiles.

To determine a tile group, at least one of the number of tiles belonging to the tile group, the size of the tile group, and image partitioning information may be transmitted via a bit stream. Alternatively, a region having a size predetermined in the encoder and decoder may be set as a group of tiles.

Encoding of information indicating whether a filter is enabled in the loop may be omitted, and whether to enable a filter in the loop can be determined based on at least one of the number of code tree elements contained in the tile, the width of the tile, and the height of the tile. For example, when the tile width is smaller than the reference value, a filter in the loop for the horizontal direction may be enabled, and when the tile height is smaller than the reference value, a filter in the loop for the vertical direction may be enabled.

When an in-loop filter is used at a tile boundary, reconstructed off-tile data can be generated from the data contained in the tile. In this case, the reconstructed video outside the tile can be obtained by padding or interpolating the data contained in the tile. The in-loop filter can then be applied using the recovered data outside the tile.

The application of the embodiments described above, focused on a decoding process or an encoding process, to an encoding process or a decoding process is included within the scope of the present invention. Changing the embodiments described in a predetermined order in an order different from the described order is also included within the scope of the present invention.

Although the above embodiments have been described based on a number of steps or flowcharts, this does not limit the timing of the present invention, and they can be executed simultaneously or in a different order as necessary. In addition, each of the components (eg, elements, modules, etc.) constituting the block diagram in the embodiments described above may be implemented as a hardware device or software, or a plurality of components may be combined to be implemented as one hardware device or software. The embodiments described above may be implemented in the form of program instructions that can be executed by various computer components and recorded in a computer-readable storage medium. The computer-readable storage medium may contain program instructions, data files, data structures, and the like. independently or in combination. Computer readable storage media includes, for example, magnetic media such as hard disk, floppy disk and magnetic tape, optical storage media such as CD-ROM and DVD, magnetic-optical media such as floppy optical disk, and hardware devices specially designed with the ability to store and execute software commands, such as ROM, RAM, flash memory, etc. The hardware devices described above may be configured to operate using one or more software modules to perform a process according to the present invention, and vice versa.

The present invention can be applied to an electronic device that encodes and decodes video.

Claims (34)

1. A video decoding method including generating a list of merge candidates for the current block; identification of one of the plurality of merger candidates included in the list of merger candidates; obtaining a first affine source vector and a second affine source vector of the current block based on the first affine source vector and the second affine source vector of the specified merge candidate; obtaining an affine vector for a sub-block in the current block by using a first affine source vector and a second affine source vector of the current block, wherein the sub-block is a region with a size that is smaller than the size of the current block; wherein the first affine source vector and the second affine source vector of the merge candidate are obtained based on the motion information of a neighboring block adjacent to the current block; in case a neighboring block is included in a code tree element different from the code tree element of the current block, the first affine source vector and the second affine source vector of the merge candidate are obtained based on the motion vectors of the lower left sub-block and the lower right sub-block of the neighboring block. 2. The method of claim 1, wherein the lower left sub-block contains a lower left reference sample located in the lower left corner of the adjacent block, and the lower right sub-block contains a lower right reference sample located in the lower left corner of the adjacent block. 3. The method according to claim 2, characterized in that the first affine source vector and the second affine source vector of the merge candidate are obtained based on the value obtained based on the shift operation performed on the difference value between the motion vectors of the lower left sub-block and the lower right sub-block using a scaling factor, and the scaling factor is obtained based on the value obtained by adding the horizontal distance between the lower left reference sample and the lower right reference sample and the offset. 4. The method according to claim 1, characterized in that the first affine source vector and the second affine source vector of the current block are obtained using the third affine source vector for the lower left control point and the fourth affine source vector for the lower right control point of the adjacent affine vector, when this where (x _n2 , y _n2 ) denotes the coordinates of the bottom left control point of the affine neighbor block, and (x _n3 , y _n3 ) denotes the coordinates of the bottom right control point of the affine neighbor block; (x ₀ , y ₀ ) denotes the coordinates of the top left control point of the current block, and (x ₁ , y ₁ ) denotes the coordinates of the top right control point of the current block; (nv _2x , nv _2y ) denotes the affine source vector of the lower left control point of the third affine source vector, and (nv _3x , nv _3y ) denotes the affine source vector of the lower right control point of the fourth affine source vector; And (v _0x , v _0y ) denotes the top left control point affine source vector of the first affine source vector, and (v _1x , v _1y ) denotes the top right control point affine source vector of the second affine source vector. 5. The method according to claim 1, characterized in that the upper boundary of the current block is in contact with the upper boundary of the code tree element of the current block. 6. A video encoding method including generating a list of merge candidates for the current block; identification of one of the plurality of merger candidates included in the list of merger candidates; obtaining a first affine source vector and a second affine source vector of the current block based on the first affine source vector and the second affine source vector of the specified merge candidate; obtaining an affine vector for a sub-block in the current block by using a first affine source vector and a second affine source vector of the current block, wherein the sub-block is a region with a size that is smaller than the size of the current block; wherein the first affine source vector and the second affine source vector of the merge candidate are obtained based on the motion information of a neighboring block adjacent to the current block; And in case a neighboring block is included in a code tree element different from the code tree element of the current block, the first affine source vector and the second affine source vector of the merge candidate are obtained based on the motion vectors of the lower left sub-block and the lower right sub-block of the neighboring block. 7. The method of claim 6, wherein the lower left subblock includes a lower left reference sample located in the lower left corner of the adjacent block, and the lower right subblock includes a lower right reference sample located in the lower left corner of the adjacent block. 8. The method according to claim 7, characterized in that the first affine source vector and the second affine source vector of the merge candidate are obtained based on the value obtained based on the shift operation performed on the difference value between the motion vectors of the lower left sub-block and the lower right sub-block using a scaling factor, and the scaling factor is obtained based on the value obtained by adding the horizontal distance between the lower left reference sample and the lower right reference sample and the offset. 9. The method according to claim 6, characterized in that the first affine source vector and the second affine source vector of the current block are obtained using the third affine source vector for the lower left control point and the fourth affine source vector for the lower right control point of the adjacent affine vector, when this where (x _n2 , y _n2 ) denotes the coordinates of the bottom left control point of the affine neighbor block, and (x _n3 , y _n3 ) denotes the coordinates of the bottom right control point of the affine neighbor block; (x ₀ , y ₀ ) denotes the coordinates of the top left control point of the current block, and (x ₁ , y ₁ ) denotes the coordinates of the top right control point of the current block; (nv _2x , nv _2y ) denotes the affine source vector of the lower left control point of the third affine source vector, and (nv _3x , nv _3y ) denotes the affine source vector of the lower right control point of the fourth affine source vector; And (v _0x , v _0y ) denotes the top left control point affine source vector of the first affine source vector, and (v _1x , v _1y ) denotes the top right control point affine source vector of the second affine source vector. 10. The method according to claim 6, characterized in that the upper boundary of the current block is in contact with the upper boundary of the code tree element of the current block. 11. A video decoder configured to perform the method according to any one of claims. 1–5. 12. A video encoder configured to perform the method according to any one of claims. 6–10.