CN117201808A

CN117201808A - Image encoding/decoding method and apparatus, and recording medium storing bit stream

Info

Publication number: CN117201808A
Application number: CN202311311003.2A
Authority: CN
Inventors: 高玄硕; 姜晶媛; 李镇浩; 李河贤; 林成昶; 全东山; 赵承眩; 金晖容; 崔振秀
Original assignee: Electronics and Telecommunications Research Institute ETRI
Current assignee: Electronics and Telecommunications Research Institute ETRI
Priority date: 2016-08-01
Filing date: 2017-07-31
Publication date: 2023-12-08
Also published as: KR102435675B1; KR20180014674A; WO2018026148A1; CN117201809A; CN117201807A; CN109792515A; CN109792515B

Abstract

The present application relates to an image encoding/decoding method and apparatus and a recording medium storing a bitstream. An image decoding method for performing intra frame prediction with respect to a current block according to the present application may include the steps of: decoding first information indicating whether residual signal prediction for predicting a residual block of a current block is performed; the residual signal prediction is performed when the first information indicates a first value.

Description

Image encoding/decoding method and apparatus, and recording medium storing bit stream

The present application is a divisional application of the application patent application entitled "image encoding/decoding method and apparatus and recording medium storing bit stream", having application number "201780061056.X", on the date of application 2017, month 07, 31.

Technical Field

The present application relates to a method and apparatus for encoding/decoding an image. In particular, the present application relates to a method and apparatus for encoding/decoding an image using intra prediction and a recording medium storing a bitstream generated by the image encoding method/apparatus of the present application.

Background

Recently, demands for high resolution and high quality images such as High Definition (HD) images and Ultra High Definition (UHD) images have been growing in various application fields. However, the data amount of higher resolution and quality image data is increased as compared with conventional image data. Accordingly, when image data is transmitted by using a medium such as a conventional wired broadband network and a wireless broadband network, or when image data is stored by using a conventional storage medium, the cost of transmission and storage increases. In order to solve these problems that occur with the increase in resolution and quality of image data, efficient image encoding/decoding techniques are required for higher resolution and higher quality images.

Image compression techniques include various techniques including: inter prediction techniques that predict pixel values included in a current picture from a previous picture or a subsequent picture of the current picture; an intra prediction technique of predicting a pixel value included in a current picture by using pixel information in the current picture; transform and quantization techniques for compressing the energy of the residual signal; entropy coding techniques that assign short codes to high frequency of occurrence values and long codes to low frequency of occurrence values; etc. By using such an image compression technique, image data can be efficiently compressed and can be transmitted or stored.

Disclosure of Invention

Technical problem

An object of the present invention is to provide a method and apparatus for encoding and decoding an image to improve compression efficiency and a recording medium storing a bitstream generated by the image encoding method/apparatus of the present invention.

Another object of the present invention is to provide a method and apparatus for encoding and decoding an image using intra prediction to improve compression efficiency and a recording medium storing a bitstream generated by the image encoding method/apparatus of the present invention.

Another object of the present invention is to provide a method and apparatus for encoding and decoding an image by performing intra prediction using residual signal prediction, and a recording medium storing a bitstream generated by the image encoding method/apparatus of the present invention.

Solution scheme

An image decoding method for performing intra prediction with respect to a current block according to the present invention may include: decoding first information indicating whether residual signal prediction for predicting a residual block of a current block is performed; if the first information indicates a first value, the residual signal prediction is performed.

In the image decoding method of the present invention, the residual signal prediction may be performed based on a previously decoded reconstructed block.

The image decoding method of the present invention may further include: an intra-frame displacement vector (IDV) is decoded, wherein the previously decoded reconstructed block may be specified by the decoded IDV.

The image decoding method of the present invention may further include: decoding an intra prediction mode for the residual signal prediction; and generating a prediction block of the reconstructed block based on the decoded intra prediction mode.

The image decoding method of the present invention may further include: generating a residual block of the reconstructed block based on the reconstructed block and a prediction block of the reconstructed block, wherein the residual block of the reconstructed block may be a prediction block of a residual block of a current block.

The image decoding method of the present invention may further include: decoding a second order residual block of the current block; and generating a residual block of the current block based on the prediction block of the residual block of the current block and the second order residual block.

In the image decoding method of the present invention, if information about an IDV is not included in a bitstream, decoding of the IDV may be performed by selecting one of a plurality of IDVs included in a predetermined search range using a predetermined search method.

In the image decoding method of the present invention, the predetermined search method may be the same method as the search method used in the image encoding method.

An image decoding apparatus including an intra predictor for performing intra prediction with respect to a current block according to the present invention may include: an intra predictor configured to: decoding first information indicating whether residual signal prediction for predicting a residual block of a current block is performed; if the first information indicates a first value, the residual signal prediction is performed.

An image encoding method for performing intra prediction with respect to a current block according to the present invention may include: performing residual signal prediction for predicting a residual block of the current block; and encodes first information indicating whether the residual signal prediction is performed.

In the image encoding method of the present invention, the residual signal prediction may be performed based on a previously decoded reconstructed block.

The image encoding method of the present invention may further include: an Intra Displacement Vector (IDV) specifying the previously decoded reconstructed block is encoded.

The image encoding method of the present invention may further include: determining an intra prediction mode for the residual signal prediction; generating a prediction block of the reconstructed block based on the determined intra prediction mode; and encoding the determined intra prediction mode.

In the image encoding method of the present invention, the predetermined search method may include: the cost function value of the residual block of the current block is compared with the cost function value of the second order residual block of the current block.

An image encoding apparatus including an intra predictor for performing intra prediction with respect to a current block according to the present invention may include: an intra predictor configured to: a residual signal prediction for predicting a residual block of a current block is performed, and first information indicating whether the residual signal prediction is performed is encoded.

A recording medium storing a bitstream generated by an image encoding method for performing intra prediction with respect to a current block according to the present invention may store the bitstream generated by the image encoding method, wherein the image encoding method includes: performing residual signal prediction for predicting a residual block of the current block; and encodes first information indicating whether the residual signal prediction is performed.

Advantageous effects

According to the present invention, an image encoding/decoding method and apparatus for improving compression efficiency and a recording medium storing a bitstream generated by the image encoding method/apparatus of the present invention can be provided.

Also, according to the present invention, it is possible to provide an image encoding/decoding method and apparatus using intra prediction, which improves compression efficiency, and a recording medium storing a bitstream generated by the image encoding method/apparatus of the present invention.

Also, according to the present invention, an image encoding/decoding method and apparatus for performing intra prediction using residual signal prediction and a recording medium storing a bitstream generated by the image encoding method/apparatus of the present invention may be provided.

Drawings

Fig. 1 is a block diagram showing a configuration of an encoding apparatus according to an embodiment of the present invention.

Fig. 2 is a block diagram showing a configuration of a decoding apparatus according to an embodiment of the present invention.

Fig. 3 is a diagram schematically showing a partition structure of an image when the image is encoded and decoded.

Fig. 4 is a diagram illustrating a form of a Prediction Unit (PU) that may be included in a Coding Unit (CU).

Fig. 5 is a diagram illustrating a form of a Transform Unit (TU) that may be included in a Coding Unit (CU).

Fig. 6 is a diagram for explaining an embodiment of a process of intra prediction.

Fig. 7 is a diagram for explaining an embodiment of a process of inter prediction.

Fig. 8 is a diagram for explaining a transform set according to an intra prediction mode.

Fig. 9 is a diagram for explaining the processing of the transformation.

Fig. 10 is a diagram for explaining scanning of quantized transform coefficients.

Fig. 11 is a diagram for explaining block partitioning.

Fig. 12 is a diagram depicting a method for performing intra prediction on a current block according to an embodiment of the present invention.

Fig. 13 is a diagram describing a method for deriving an intra prediction mode of a current block from neighboring blocks.

Fig. 14 is an exemplary diagram depicting a current block, an upper layer block, and a neighboring block.

Fig. 15 is an exemplary diagram showing a luminance block and a chrominance block having a composition ratio of color components of 4:2:0.

Fig. 16 is a diagram depicting an embodiment of deriving an intra prediction mode of each of one or more sub-blocks into which a current block is divided.

Fig. 17 is a diagram depicting an embodiment in which a current block is divided into sub-blocks.

Fig. 18 is a diagram depicting another embodiment of dividing a current block into sub-blocks.

Fig. 19 is a diagram depicting another embodiment of dividing a current block into sub-blocks.

Fig. 20 is a diagram depicting another embodiment of dividing a current block into sub-blocks.

Fig. 21 is a diagram depicting an embodiment of deriving an intra prediction mode of a current block using a seed point intra prediction mode.

Fig. 22 is a diagram depicting an embodiment of making an spim list including two spims.

Fig. 23 is an exemplary diagram showing an embodiment of making an spim list including three spims.

Fig. 24 is an exemplary diagram showing an embodiment of making an spim list including four spims.

Fig. 25 is an exemplary diagram showing a sub-block size in the case where the size of the current block is 16×16.

Fig. 26 is a diagram depicting an embodiment of assigning intra prediction modes using determined IPDF.

Fig. 27 is an exemplary diagram illustrating a reconstructed block adjacent to a current block.

Fig. 28 is a diagram depicting an embodiment of deriving an intra prediction mode using neighboring reconstructed blocks.

Fig. 29 is a diagram depicting an embodiment of deriving intra prediction modes based on sub-blocks.

Fig. 30 is a diagram depicting another embodiment of deriving an intra prediction mode based on sub-blocks.

Fig. 31 is an exemplary diagram illustrating a syntax structure including intra prediction mode information.

Fig. 32 is an exemplary diagram depicting neighboring reconstructed sample lines that may be used for intra prediction of a current block.

Fig. 33 is a diagram depicting an embodiment of constructing reference samples for sub-blocks included in a current block.

Fig. 34 is a diagram depicting a method for replacing unavailable reconstructed samples with available reconstructed samples.

Fig. 35 is a diagram illustrating an exemplary threshold for determining whether to apply filtering for a block size.

Fig. 36 is an exemplary diagram illustrating whether filtering is performed according to a block size and/or an intra prediction mode.

Fig. 37 is an exemplary diagram illustrating intra prediction according to the shape of a current block.

Fig. 38 is a diagram depicting filtering for DC-mode intra prediction.

Fig. 39 is a diagram depicting Planar (Planar) mode intra prediction.

FIG. 40 is a diagram depicting a slave P _ref Generating a 1D reference sample array p _1,ref Is a diagram of an embodiment of (a).

FIG. 41 is a diagram depicting an embodiment using reference samples at different angles depending on the location of the samples in the prediction block.

Fig. 42 is a view showing the application of cuv=0.1, cw by applying to a current block of size 4×4 ₀ ＝1.0、cw ₁ ＝1.2、cw ₂ =1.4 and cw ₃ An example of prediction is performed in the direction from the upper right side to the lower left side=1.6.

FIG. 43 is a graph showing cuv and cw as application of FIG. 42 _i An example of the location of a reference pixel used by a predicted pixel within the current block.

Fig. 44 is a view showing that by applying cuv=0.1, cw to a current block of size 4×4 ₀ ＝1.0、cw ₁ ＝1.2、cw ₂ =1.4 and cw ₃ The diagram of an embodiment in which prediction is performed in the direction from the upper left edge to the lower right edge (type-1) =1.6.

FIG. 45 is a graph showing cuv and cw as application graph 44 _i An example of the location of a reference pixel used by a predicted pixel within the current block.

Fig. 46 is a view showing that by applying cuv=0.1, cw to a current block of size 4×4 ₀ ＝1.0、cw ₁ ＝1.2、cw ₂ =1.4 and cw ₃ An example of prediction is performed in the direction from the lower left side to the upper right side=1.6.

FIG. 47 is a graph showing cuv and cw as application graph 46 _i An example of the location of a reference pixel used by a predicted pixel within the current block.

Fig. 48 is a view showing that by applying cuv=0.1, cw to a current block of size 4×4 ₀ ＝1.0、cw ₁ ＝1.2、cw ₂ =1.4 and cw ₃ The diagram of an embodiment in which prediction is performed in the direction from the upper left edge to the lower right edge (type-2) =1.6.

FIG. 49 is a graph showing cuv and cw as application graph 48 _i An example of the location of a reference pixel used by a predicted pixel within the current block.

Fig. 50 is a view showing that by applying cuv=0.6, cw to a current block of size 4×4 ₀ ＝1.0、cw ₁ ＝1.4、cw ₂ =1.8 and cw ₃ =2.2, embodiments in which prediction is performed in the direction from the upper side to the lower left sideIs a diagram of (a).

FIG. 51 is a graph showing cuv and cw as application graph 50 _i An example of the location of a reference pixel used by a predicted pixel within the current block.

Fig. 52 is a view showing that by applying cuv=0.6, cw to a current block of size 4×4 ₀ ＝1.0、cw ₁ ＝1.4、cw ₂ =1.8 and cw ₃ An illustration of an embodiment in which prediction is performed in the direction from the upper side to the lower right side, =2.2.

FIG. 53 is a graph showing cuv and cw as application graph 52 _i An example of the location of a reference pixel used by a predicted pixel within the current block.

Fig. 54 is a view showing that by applying cuv=0.6, cw to a current block of size 4×4 ₀ ＝1.0、cw ₁ ＝1.4、cw ₂ =1.8 and cw ₃ An illustration of an embodiment in which prediction is performed in the direction from the left side to the upper right side, =2.2.

FIG. 55 is a graph showing cuv and cw as application graph 54 _i An example of the location of a reference pixel used by a predicted pixel within the current block.

Fig. 56 is a diagram showing the application of cuv=0.6, cw by applying to a current block of size 4×4 ₀ ＝1.0、cw ₁ ＝1.4、cw ₂ =1.8 and cw ₃ An illustration of an embodiment in which prediction is performed in the direction from the left side to the lower right side, =2.2.

FIG. 57 is a diagram showing cuv and cw as application diagram 56 _i An example of the location of a reference pixel used by a predicted pixel within the current block.

Fig. 58 is a diagram for explaining another embodiment of using different direction modes in units of samples in a target block.

Fig. 59 is a diagram depicting an embodiment of predicting a residual signal.

Fig. 60 is a diagram depicting prediction of a residual signal using a second order residual signal block.

Fig. 61 is a diagram depicting an embodiment of residual signal prediction.

Fig. 62a and 62b are diagrams depicting embodiments in which residual signal prediction is performed in an encoder.

Fig. 63 is a diagram depicting an embodiment in which residual signal prediction is performed in a decoder.

Detailed Description

Many modifications may be made to the present invention and there are various embodiments of the present invention, examples of which will now be provided with reference to the accompanying drawings and described in detail. However, the present invention is not limited thereto, and although the exemplary embodiments may be construed to include all modifications, equivalents, or alternatives falling within the technical spirit and scope of the present invention. Like reference numerals refer to the same or similar functionality in all respects. In the drawings, the shape and size of elements may be exaggerated for clarity. In the following detailed description of the invention, reference is made to the accompanying drawings that show, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure. It is to be understood that the various embodiments of the disclosure, although different, are not necessarily mutually exclusive. For example, the particular features, structures, and characteristics described herein in connection with one embodiment may be implemented in other embodiments without departing from the spirit and scope of the disclosure. Further, it is to be understood that the location or arrangement of individual elements within each disclosed embodiment may be modified without departing from the spirit and scope of the disclosure. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims, appropriately interpreted, along with the full range of equivalents to which the claims are entitled.

The terms "first," "second," and the like, as used in the specification, may be used to describe various components, but these components are not to be construed as limiting the terms. The term is used merely to distinguish one component from another. For example, a "first" component may be termed a "second" component, and a "second" component may be similarly termed a "first" component, without departing from the scope of the present invention. The term "and/or" includes a combination of items or any of a plurality of items.

It will be understood that in the present specification, when an element is referred to simply as being "connected" or "coupled" to another element, it can be "directly connected" or "directly coupled" to the other element or be connected or coupled to the other element with other elements interposed therebetween. In contrast, it will be understood that when an element is referred to as being "directly coupled" or "directly connected" to another element, there are no intervening elements present.

Further, constituent elements shown in the embodiments of the present invention are independently shown so as to exhibit characteristic functions different from each other. Therefore, this does not mean that each constituent element is composed in a separate constituent unit of hardware or software. In other words, for convenience, each component includes each of the enumerated components. Thus, at least two of each component may be combined to form one component, or one component may be divided into a plurality of components to perform each function. Embodiments in which each component is combined and embodiments in which one component is divided are also included in the scope of the present invention without departing from the essence of the present invention.

The terminology used in the description presented herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The use of the expression in the singular includes the plural unless it has a distinct meaning in context. In this specification, it will be understood that terms, such as "comprises," "comprising," "has," "having," "includes," etc., are intended to specify the presence of stated features, amounts, steps, acts, elements, components, or combinations thereof disclosed in the specification, but are not intended to preclude the possibility that one or more other features, amounts, steps, acts, elements, components, or combinations thereof may be present or added. In other words, when a specific element is referred to as being "included", elements other than the corresponding element are not excluded, and additional elements may be included in the embodiments of the present invention or be within the scope of the present invention.

Furthermore, some constituent elements may not be indispensable constituent elements performing the necessary functions of the present invention, but may be optional constituent elements merely improving the performance thereof. The present invention can be implemented by including only essential constituent elements for implementing the essence of the present invention, excluding constituent elements used in enhancing performance. Structures that include only the indispensable components and exclude optional components that are used in merely enhancing performance are also included in the scope of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In describing exemplary embodiments of the present invention, well-known functions or constructions will not be described in detail since they would unnecessarily obscure the present invention. The same constituent elements in the drawings are denoted by the same reference numerals, and repetitive description of the same elements will be omitted.

Further, hereinafter, an image may mean a picture constituting a video, or may mean a video itself. For example, "encoding or decoding or both" may mean "encoding or decoding or both" a video, and may mean "encoding or decoding or both one image among a plurality of images of a video. Here, the picture and the image may have the same meaning.

Description of the terms

An encoder: may mean a device that performs the encoding.

A decoder: it may mean a device performing decoding.

Analysis: the value of the syntax element may be defined by performing entropy decoding, or the entropy decoding itself may be defined.

And (3) block: it may mean the samples of an mxn matrix. Here, M and N are positive integers, and a block may mean a sample matrix in two dimensions.

Sampling points: is the basic unit of a block and may indicate a value ranging from 0 to 2Bd-1 depending on the bit depth (Bd). A sample may mean a pixel in the present invention.

A unit: it may mean a unit of encoding and decoding an image. In encoding and decoding an image, a unit may be an area generated by partitioning one image. Further, a unit may mean a sub-division unit when one image is partitioned into a plurality of sub-division units during encoding or decoding. When encoding and decoding an image, predetermined processing for each unit may be performed. A cell may be partitioned into sub-cells that are smaller in size than the cell. Units may mean blocks, macro blocks, coding tree units, coding tree blocks, coding units, coding blocks, prediction units, prediction blocks, transform units, transform blocks, etc. in terms of function. Further, to distinguish a cell from a block, the cell may include a luminance component block, a chrominance component block of the luminance component block, and a syntax element of each color component block. The cells may have various sizes and shapes, in particular, the shape of the cells may be a two-dimensional geometry, such as rectangular, square, trapezoidal, triangular, pentagonal, etc. Further, the unit information may include at least one of a unit type (indicating an encoding unit, a prediction unit, a transformation unit, etc.), a unit size, a unit depth, an order in which the units are encoded and decoded, and the like.

Reconstructing neighboring units: it may mean a reconstruction unit that is previously encoded or decoded spatially/temporally, and the reconstruction unit is adjacent to the encoding/decoding target unit. Here, reconstructing a neighboring unit may mean reconstructing a neighboring block.

Adjacent blocks: may mean a block adjacent to the encoding/decoding target block. The block adjacent to the encoding/decoding target block may mean a block having a boundary in contact with the encoding/decoding target block. Neighboring blocks may mean blocks located at neighboring vertices of the encoding/decoding target block. Neighboring blocks may mean reconstructing neighboring blocks.

Cell depth: the degree to which a cell is partitioned may be intended. In the tree structure, the root node may be the highest node and the leaf node may be the lowest node.

The symbols: it may mean that syntax elements of the target unit for encoding/decoding, encoding parameters, values of transform coefficients, and the like.

Parameter set: header information in the structure of the bitstream may be intended. The parameter sets may comprise at least one parameter set of a video parameter set, a sequence parameter set, a picture parameter set or an adaptive parameter set. Further, the parameter set may mean stripe header information, parallel block (tile) header information, and the like.

Bit stream: may mean a bit string including encoded image information.

Prediction unit: it may mean a basic unit when inter prediction or intra prediction is performed and compensation for the prediction. One prediction unit may be partitioned into a plurality of partitions. In this case, each partition of the plurality of partitions may be a basic unit in performing prediction and compensation, and each partition obtained from the prediction unit partition may be a prediction unit. Furthermore, one prediction unit may be partitioned into a plurality of small prediction units. The prediction unit may have various sizes and shapes, and in particular, the shape of the prediction unit may be a two-dimensional geometric figure such as a rectangle, a square, a trapezoid, a triangle, a pentagon, and the like.

Prediction unit partitioning: may mean the shape of the partitioned prediction unit.

Reference picture list: it may mean a list including at least one reference picture, wherein the at least one reference picture is used for inter prediction or motion compensation. The type of the reference picture List may be List Combined (LC), list 0 (L0), list 1 (L1), list 2 (L2), list 3 (L3), or the like. At least one reference picture list may be used for inter prediction.

Inter prediction indicator: it may mean one of the following: inter prediction direction (unidirectional prediction, bidirectional prediction, etc.) of an encoding/decoding target block in the case of inter prediction, the number of reference pictures used to generate a prediction block by the encoding/decoding target block, and the number of reference blocks used to perform inter prediction or motion compensation by the encoding/decoding target block.

Reference picture index: may mean an index of a particular reference picture in the reference picture list.

Reference picture: it may mean a picture that a particular unit refers to for inter prediction or motion compensation. The reference picture may be referred to as a reference picture.

Motion vector: is a two-dimensional vector for inter prediction or motion compensation, and may mean an offset between an encoding/decoding target picture and a reference picture. For example, (mvX, mvY) may indicate a motion vector, mvX may indicate a horizontal component, and mvY may indicate a vertical component.

Motion vector candidates: it may mean a unit that becomes a prediction candidate when predicting a motion vector, or may mean a motion vector of the unit.

Motion vector candidate list: it may mean a list configured by using motion vector candidates.

Motion vector candidate index: may mean an indicator indicating a motion vector candidate in the motion vector candidate list. The motion vector candidate index may be referred to as an index of a motion vector predictor.

Motion information: it may mean a motion vector, a reference picture index, and an inter prediction indicator, and information including at least one of reference picture list information, a reference picture, a motion vector candidate index, and the like.

Merging candidate list: it may mean a list configured by using merge candidates.

Combining candidates: spatial merge candidates, temporal merge candidates, combined bi-predictive merge candidates, zero merge candidates, etc. may be included. The merge candidates may include motion information such as prediction type information, reference picture indexes for each list, motion vectors, and the like.

Merging index: may mean information indicating a merge candidate in the merge candidate list. Further, the merge index may indicate a block of derived merge candidates among reconstructed blocks spatially/temporally adjacent to the current block. Further, the merge index may indicate at least one of a plurality of pieces of motion information of the merge candidate.

A conversion unit: it may mean a basic unit when encoding/decoding similar to transforming, inverse transforming, quantizing, inverse quantizing, and transform coefficient encoding/decoding is performed on the residual signal. One transform unit may be partitioned into multiple small transform units. The transform unit may have various sizes and shapes. In particular, the shape of the transformation unit may be a two-dimensional geometry, such as a rectangle, square, trapezoid, triangle, pentagon, etc.

Scaling: it may mean a process of multiplying a factor by the transform coefficient level, as a result of which the transform coefficient may be generated. Scaling may also be referred to as dequantization.

Quantization parameters: may mean values used in scaling the transform coefficient levels during quantization and dequantization. Here, the quantization parameter may be a value mapped to a quantized step size.

Delta (Delta) quantization parameter: it may mean a difference between a quantization parameter of the encoding/decoding target unit and a predicted quantization parameter.

Scanning: it may mean a method of ordering the order of coefficients within a block or matrix. For example, the operation of ordering a two-dimensional matrix into a one-dimensional matrix may be referred to as scanning, and the operation of ordering a one-dimensional matrix into a two-dimensional matrix may be referred to as scanning or inverse scanning.

Transform coefficients: may mean coefficient values generated after performing the transformation. In the present invention, the quantized transform coefficient level (i.e., the transform coefficient to which quantization is applied) may be referred to as a transform coefficient.

Non-zero transform coefficients: a transform coefficient having a value other than 0 may be intended, or a transform coefficient level having a value other than 0 may be intended.

Quantization matrix: it may mean a matrix used in quantization and inverse quantization in order to improve the subject quality (subject quality) or object quality (subject quality) of an image. The quantization matrix may be referred to as a scaling list.

Quantization matrix coefficients: each element of the quantization matrix may be implied. The quantized matrix coefficients may be referred to as matrix coefficients.

Default matrix: it may mean a predetermined quantization matrix that is predefined in the encoder and decoder.

Non-default matrix: it may mean a quantization matrix transmitted/received by a user without being predefined in the encoder and decoder.

Coding tree unit: may be formed of one luma component (Y) coding tree unit and associated two chroma component (Cb, cr) coding tree units. Each coding tree unit may be partitioned using at least one partitioning method (such as a quadtree, binary tree, etc.) to constitute sub-units such as coding units, prediction units, transform units, etc. The coding tree unit may be used as a term for indicating a block of pixels, i.e. a processing unit in a decoding/encoding process of an image, such as a partition of an input image.

Coding tree blocks: may be used as a term for indicating one of a Y coding tree unit, a Cb coding tree unit, and a Cr coding tree unit.

The encoding apparatus 100 may be a video encoding apparatus or an image encoding apparatus. The video may include one or more images. The encoding device 100 may encode one or more images of the video in temporal order.

Referring to fig. 1, the encoding apparatus 100 may include a motion prediction unit 111, a motion compensation unit 112, an intra prediction unit 120, a switcher 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190.

The encoding apparatus 100 may encode the input picture in an intra mode or an inter mode or both the intra mode and the inter mode. Further, the encoding apparatus 100 may generate a bitstream by encoding an input picture, and may output the generated bitstream. When the intra mode is used as the prediction mode, the switcher 115 can switch to intra. When the inter mode is used as the prediction mode, the switcher 115 may switch to the inter mode. Here, the intra mode may be referred to as an intra prediction mode, and the inter mode may be referred to as an inter prediction mode. The encoding apparatus 100 may generate a prediction block of an input picture. Further, after generating the prediction block, the encoding apparatus 100 may encode a residual between the input block and the prediction block. The input picture may be referred to as a current image that is a target of current encoding. The input block may be referred to as a current block or may be referred to as an encoding target block that is a target of current encoding.

When the prediction mode is an intra mode, the intra prediction unit 120 may use a pixel value of a previously encoded block adjacent to the current block as a reference pixel. The intra prediction unit 120 may perform spatial prediction by using the reference pixels, and may generate prediction samples of the input block by using the spatial prediction. Here, intra prediction may mean intra frame prediction.

When the prediction mode is an inter mode, the motion prediction unit 111 may search for a region optimally matching the input block from the reference picture in the motion prediction process, and may derive a motion vector by using the searched region. The reference picture may be stored in a reference picture buffer 190.

The motion compensation unit 112 may generate a prediction block by performing motion compensation using the motion vector. Here, the motion vector may be a two-dimensional vector for inter prediction. Further, the motion vector may indicate an offset between the current picture and the reference picture. Here, the inter prediction may mean inter frame prediction.

When the value of the motion vector is not an integer, the motion prediction unit 111 and the motion compensation unit 112 may generate a prediction block by applying an interpolation filter to a partial region in the reference picture. In order to perform inter prediction or motion compensation based on the coding unit, which method is used for a motion prediction and compensation method of a prediction unit in the coding unit may be determined among a skip mode, a merge mode, an AMVP mode, and a current picture reference mode. Inter prediction or motion compensation may be performed according to each mode. Here, the current picture reference mode may mean a prediction mode using a previously reconstructed region of the current picture having the encoding target block. In order to indicate the pre-reconstructed region, a motion vector for the current picture reference mode may be defined. Whether or not to encode the encoding target block in the current picture reference mode may be encoded by using the reference picture index of the encoding target block.

The subtractor 125 may generate a residual block by using a residual between the input block and the prediction block. The residual block may be referred to as a residual signal.

The transform unit 130 may generate transform coefficients by transforming the residual block, and may output the transform coefficients. Here, the transform coefficient may be a coefficient value generated by transforming the residual block. In the transform skip mode, the transform unit 130 may skip the transform of the residual block.

Quantized transform coefficient levels may be generated by applying quantization to transform coefficients. Hereinafter, in an embodiment of the present invention, the quantized transform coefficient level may be referred to as a transform coefficient.

The quantization unit 140 may generate quantized transform coefficient levels by quantizing the transform coefficients according to quantization parameters, and may output the quantized transform coefficient levels. Here, the quantization unit 140 may quantize the transform coefficient by using a quantization matrix.

The entropy encoding unit 150 may generate a bitstream by performing entropy encoding on the values calculated by the quantization unit 140 or on the encoding parameter values calculated in the encoding process, etc., according to the probability distribution, and may output the generated bitstream. The entropy encoding unit 150 may perform entropy encoding on information for decoding an image and entropy encoding on information of pixels of the image. For example, the information for decoding the image may include syntax elements or the like.

When entropy encoding is applied, symbols are represented by allocating a small number of bits to symbols having a high occurrence probability and allocating a large number of bits to symbols having a low occurrence probability, thereby reducing the size of a bitstream encoding a target symbol. Therefore, the compression performance of image coding can be improved by entropy coding. For entropy encoding, the entropy encoding unit 150 may use an encoding method such as exponential golomb, context Adaptive Variable Length Coding (CAVLC), and Context Adaptive Binary Arithmetic Coding (CABAC). For example, the entropy encoding unit 150 may perform entropy encoding by using a variable length coding/coding (VLC) table. Further, the entropy encoding unit 150 may derive a binarization method of the target symbol and a probability model of the target symbol/binary bit, and then may perform arithmetic encoding by using the derived binarization method or the derived probability model.

In order to encode the transform coefficient levels, the entropy encoding unit 150 may change coefficients in the form of a two-dimensional block into a one-dimensional vector form by using a transform coefficient scanning method. For example, coefficients in two-dimensional form may be changed to one-dimensional vector form by scanning the coefficients of the block with an upper right scan. Depending on the size of the transform unit and the intra prediction mode, a vertical direction scan for scanning coefficients in the form of two-dimensional blocks in the column direction and a horizontal direction scan for scanning coefficients in the form of two-dimensional blocks in the row direction may be used instead of using an upper right scan. That is, depending on the size of the transform unit and the intra prediction mode, it may be determined which scan method among the upper right scan, the vertical direction scan, and the horizontal direction scan is to be used.

The encoding parameters may include information such as syntax elements encoded by an encoder and transmitted to a decoder, and may include information that may be derived in an encoding or decoding process. The encoding parameter may mean information necessary to encode or decode an image. For example, the encoding parameters may include at least one value or combination of the following: block size, block depth, block partition information, unit size, unit depth, unit partition information, partition flags in the form of a quadtree, partition flags in the form of a binary tree, partition direction in the form of a binary tree, intra prediction mode, intra prediction direction, reference sample filtering method, prediction block boundary filtering method, filter tap, filter coefficient, inter prediction mode, motion information, motion vector, reference picture index, inter prediction direction, inter prediction indicator, reference picture list, motion vector predictor, motion vector candidate list, information on whether a motion merge mode is used, motion merge candidate list, information on whether a skip mode is used, interpolation filter type, motion vector size, motion vector index accuracy of motion vector representation, type of transform, size of transform, information on whether additional (secondary) transforms are used, information on whether residual signals are present, coding block patterns, coding block flags, quantization parameters, quantization matrices, filter information in loops, information on whether filters are applied in loops, filter coefficients in loops, binarization/anti-binarization methods, context models, context binary bits, bypass binary bits, transform coefficients, transform coefficient levels, transform coefficient level scanning methods, image display/output order, stripe identification information, stripe types, stripe partition information, parallel block identification information, parallel block types, parallel block partition information, picture types, bit depths, and information of luminance signals or chrominance signals.

The residual signal may mean the difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming a difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming and quantizing a difference between the original signal and the predicted signal. The residual block may be a residual signal of a block unit.

When the encoding apparatus 100 performs encoding by using inter prediction. The encoded current picture may be used as a reference picture for another image to be subsequently processed. Accordingly, the encoding apparatus 100 may decode the encoded current picture and may store the decoded image as a reference picture. To perform decoding, inverse quantization and inverse transformation may be performed on the encoded current picture.

The quantized coefficients may be dequantized by the dequantization unit 160 and may be inverse transformed by the inverse transformation unit 170. The inverse quantized and inverse transformed coefficients may be added to the prediction block by adder 175, whereby a reconstructed block may be generated.

The reconstructed block may pass through a filter unit 180. The filter unit 180 may apply at least one of a deblocking filter, a Sample Adaptive Offset (SAO), and an Adaptive Loop Filter (ALF) to the reconstructed block or the reconstructed picture. The filter unit 180 may be referred to as a loop filter.

The deblocking filter may remove block distortion occurring at boundaries between blocks. To determine whether the deblocking filter is operated, whether the deblocking filter is applied to the current block may be determined based on pixels included in several rows or columns in the block. When a deblocking filter is applied to a block, a strong filter or a weak filter may be applied depending on the required deblocking filter strength. Further, when the deblocking filter is applied, horizontal direction filtering and vertical direction filtering can be processed in parallel.

The sample adaptive offset may add an optimal offset value to the pixel value in order to compensate for the coding error. The sample adaptive offset may correct an offset between the deblocking filtered image and the original picture for each pixel. In order to perform offset correction on a specific screen, a method of applying offset in consideration of edge information of each pixel may be used, or the following method may be used: the pixels of the image are partitioned into a predetermined number of areas, an area to be subjected to offset correction is determined, and offset correction is applied to the determined area.

The adaptive loop filter may perform filtering based on a value obtained by comparing the reconstructed picture with the original picture. Pixels of the image may be partitioned into predetermined groups, one filter applied to each group is determined, and different filtering may be performed at each group. Information about whether the adaptive loop filter is applied to the luminance signal may be transmitted for each Coding Unit (CU). The shape and filter coefficients of the adaptive loop filter applied to each block may vary. Furthermore, an adaptive loop filter having the same form (fixed form) may be applied regardless of the characteristics of the target block.

The reconstructed block that passes through the filter unit 180 may be stored in a reference picture buffer 190.

The decoding apparatus 200 may be a video decoding apparatus or an image decoding apparatus.

Referring to fig. 2, the decoding apparatus 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra prediction unit 240, a motion compensation unit 250, an adder 255, a filter unit 260, and a reference picture buffer 270.

The decoding apparatus 200 may receive the bit stream output from the encoding apparatus 100. The decoding apparatus 200 may decode the bitstream in an intra mode or an inter mode. In addition, the decoding apparatus 100 may generate a reconstructed picture by performing decoding, and may output the reconstructed picture.

When the prediction mode used in decoding is an intra mode, the switcher may be switched into the intra mode. When the prediction mode used in decoding is an inter mode, the switcher may be switched to an inter mode.

The decoding apparatus 200 may obtain a reconstructed residual block from an input bitstream and may generate a prediction block. When the reconstructed residual block and the prediction block are obtained, the decoding apparatus 200 may generate a reconstructed block as a decoding target block by adding the reconstructed residual block and the prediction block. The decoding target block may be referred to as a current block.

The entropy decoding unit 210 may generate symbols by performing entropy decoding on the bitstream according to the probability distribution. The generated symbols may include symbols having quantized transform coefficient levels. Here, the method of entropy decoding may be similar to the method of entropy encoding described above. For example, the method of entropy decoding may be the inverse of the method of entropy encoding described above.

In order to decode the transform coefficient levels, the entropy decoding unit 210 may perform transform coefficient scanning, whereby coefficients in the form of one-dimensional vectors may be changed to the form of two-dimensional blocks. For example, coefficients in the form of one-dimensional vectors may be changed to the form of two-dimensional blocks by scanning the coefficients of the blocks with an upper right scan. Depending on the size of the transform unit and the intra prediction mode, vertical direction scanning and horizontal direction scanning may be used instead of upper right scanning. That is, depending on the size of the transform unit and the intra prediction mode, it is possible to determine which scanning method among the upper right scanning, the vertical direction scanning, and the horizontal direction scanning is used.

The quantized transform coefficient levels may be dequantized by the dequantization unit 220 and may be inverse transformed by the inverse transform unit 230. The quantized transform coefficient levels are dequantized and inverse transformed to produce a reconstructed residual block. Here, the inverse quantization unit 220 may apply a quantization matrix to the quantized transform coefficient level.

When the intra mode is used, the intra prediction unit 240 may generate a prediction block by performing spatial prediction using pixel values of a previously decoded block adjacent to the decoding target block.

When the inter mode is used, the motion compensation unit 250 may generate a prediction block by performing motion compensation using both the reference picture stored in the reference picture buffer 270 and the motion vector. When the value of the motion vector is not an integer, the motion compensation unit 250 may generate a prediction block by applying an interpolation filter to a partial region in the reference picture. In order to perform motion compensation, based on the coding unit, which method is used for a motion compensation method of a prediction unit in the coding unit may be determined among a skip mode, a merge mode, an AMVP mode, and a current picture reference mode. Furthermore, motion compensation may be performed according to the mode. Here, the current picture reference mode may mean a prediction mode using a previously reconstructed region within the current picture having the decoding target block. The previously reconstructed region may not be adjacent to the decoding target block. To indicate the previously reconstructed region, a fixed vector may be used for the current picture reference mode. In addition, a flag or index indicating whether the decoding target block is a block decoded according to the current picture reference mode may be signaled and may be derived by using the reference picture index of the decoding target block. The current picture for the current picture reference mode may exist at a fixed position (e.g., a position with reference picture index 0 or a last position) within the reference picture list for the decoding target block. Further, the current picture may be variably located within the reference picture list, for which a reference picture index indicating a position of the current picture may be signaled. Here, signaling a flag or index may mean that an encoder entropy encodes and includes the corresponding flag or index in a bitstream, and that a decoder entropy decodes the corresponding flag or index from the bitstream.

The reconstructed residual block may be added to the prediction block by adder 255. The block generated by adding the reconstructed residual block and the prediction block may pass through the filter unit 260. The filter unit 260 may apply at least one of a deblocking filter, a sample adaptive offset, and an adaptive loop filter to the reconstructed block or the reconstructed picture. The filter unit 260 may output the reconstructed picture. The reconstructed picture may be stored in a reference picture buffer 270 and may be used for inter prediction.

Fig. 3 is a diagram schematically showing a partition structure of an image when the image is encoded and decoded. Fig. 3 schematically shows an embodiment of partitioning a unit into a plurality of sub-units.

For efficient partitioning of images, coding Units (CUs) may be used in encoding and decoding. Here, the encoding unit may mean a unit for encoding, and the unit may be a combination of 1) a syntax element and 2) a block including image samples. For example, "partition of a unit" may mean "partition of a block associated with a unit". The block partition information may include information about the depth of the unit. The depth information may indicate the number of times a unit is partitioned or the degree to which the unit is partitioned, or both.

Referring to fig. 3, an image 300 is sequentially partitioned for each Largest Coding Unit (LCU), and a partition structure is determined for each LCU. Here, LCU and Coding Tree Unit (CTU) have the same meaning. One unit may have depth information based on a tree structure and may be hierarchically partitioned. Each partitioned subunit may have depth information. The depth information indicates the number of times the unit is partitioned or the degree to which the unit is partitioned, or both, and thus, the depth information may include information about the size of the sub-unit.

The partition structure may mean a distribution of Coding Units (CUs) in LCU 310. A CU may be a unit for efficiently encoding/decoding an image. The distribution may be determined based on whether a CU is to be partitioned multiple times (a positive integer equal to or greater than 2, including 2, 4, 8, 16, etc.). The width size and height size of the partitioned CU may be half the width size and half the height size of the original CU, respectively. Alternatively, the width and height sizes of the partitioned CUs may be smaller than those of the original CUs, respectively, according to the number of partitions. The partitioned CU may be recursively partitioned into a plurality of further partitioned CUs, wherein the further partitioned CUs have a width size and a height size smaller than a width size and a height size of the partitioned CUs according to the same partitioning method.

Here, the partitioning of the CU may be recursively performed up to a predetermined depth. The depth information may be information indicating the size of the CU, and may be stored in each CU. For example, the depth of the LCU may be 0, and the depth of the minimum coding unit (SCU) may be a predetermined maximum depth. Here, the LCU may be a coding unit having the above-described maximum size, and the SCU may be a coding unit having a minimum size.

Whenever LCU 310 starts to be partitioned and the width size and height size of the CU are reduced by the partitioning operation, the depth of the CU is increased by 1. In the case of CUs that cannot be partitioned, the CU may have a size of 2n×2n for each depth. In the case of CUs that can be partitioned, a CU having a size of 2n×2n may be partitioned into a plurality of n×n-sized CUs. The size of N is halved each time the depth increases by 1.

For example, when one coding unit is partitioned into four sub-coding units, the width size and the height size of one of the four sub-coding units may be a half width size and a half height size of an original coding unit, respectively. For example, when a coding unit of a 32×32 size is partitioned into four sub-coding units, each of the four sub-coding units may have a 16×16 size. When one coding unit is partitioned into four sub-coding units, the coding units may be partitioned in a quadtree form.

For example, when one coding unit is partitioned into two sub-coding units, the width size or the height size of one of the two sub-coding units may be a half width size or a half height size of the original coding unit, respectively. For example, when a coding unit of a 32×32 size is vertically partitioned into two sub-coding units, each of the two sub-coding units may have a 16×32 size. For example, when a coding unit of a 32×32 size is horizontally partitioned into two sub-coding units, each of the two sub-coding units may have a 32×16 size. When one coding unit is partitioned into two sub-coding units, the coding units may be partitioned in a binary tree form.

Referring to fig. 3, the size of the LCU having the minimum depth 0 may be 64×64 pixels, and the size of the SCU having the maximum depth 3 may be 8×8 pixels. Here, a CU having 64×64 pixels (i.e., LCU) may be represented by depth 0, a CU having 32×32 pixels may be represented by depth 1, a CU having 16×16 pixels may be represented by depth 2, and a CU having 8×8 pixels (i.e., SCU) may be represented by depth 3.

Further, information on whether a CU is to be partitioned may be represented by partition information of the CU. The partition information may be 1-bit information. Partition information may be included in all CUs except SCU. For example, a CU may not be partitioned when the value of partition information is 0, and a CU may be partitioned when the value of partition information is 1.

A CU that is no longer partitioned among a plurality of CUs partitioned from an LCU may be partitioned into at least one Prediction Unit (PU). This process may also be referred to as partitioning.

A PU may be a base unit for prediction. The PU may be encoded and decoded in any of a skip mode, an inter mode, and an intra mode. The PUs may be partitioned in various forms depending on the mode.

In addition, the encoding unit may not be partitioned into a plurality of prediction units, and the encoding unit and the prediction unit may have the same size.

As shown in fig. 4, in skip mode, a CU may not be partitioned. In the skip mode, a 2N×2N mode 410 having the same size as a non-partitioned CU may be supported.

In inter mode, 8 partition modes may be supported in a CU. For example, in the inter mode, a 2n×2n mode 410, a 2n×n mode 415, an n×2n mode 420, an n×n mode 425, a 2n×nu mode 430, a 2n×nd mode 435, an nl×2n mode 440, and an nr×2n mode 445 may be supported. In intra mode, 2n×2n mode 410 and n×n mode 425 may be supported.

One coding unit may be partitioned into one or more prediction units. One prediction unit may be partitioned into one or more sub-prediction units.

For example, when one prediction unit is partitioned into four sub prediction units, the width size and the height size of one of the four sub prediction units may be half the width size and half the height size of the original prediction unit. For example, when a prediction unit of 32×32 size is partitioned into four sub-prediction units, each of the four sub-prediction units may have a size of 16×16. When one prediction unit is partitioned into four sub-prediction units, the prediction units may be partitioned in a quadtree form.

For example, when one prediction unit is partitioned into two sub prediction units, the width size or height size of one of the two sub prediction units may be half the width size or half the height size of the original prediction unit. For example, when a prediction unit of a 32×32 size is vertically partitioned into two sub-prediction units, each of the two sub-prediction units may have a size of 16×32. For example, when a prediction unit of a 32×32 size is horizontally partitioned into two sub-prediction units, each of the two sub-prediction units may have a 32×16 size. When one prediction unit is partitioned into two sub-prediction units, the prediction units may be partitioned in a binary tree form.

A Transform Unit (TU) may be a basic unit for transformation, quantization, inverse transformation, and inverse quantization within a CU. The TU may have a square shape or a rectangular shape, etc. The TUs may be determined independently in terms of the size of the CU or the form of the CU, or both.

A CU that is no longer partitioned among CUs partitioned from an LCU may be partitioned into at least one TU. Here, the partition structure of the TUs may be a quadtree structure. For example, as shown in FIG. 5, one CU 510 may be partitioned one or more times according to a quadtree structure. The case where one CU is partitioned at least once may be referred to as recursive partitioning. By partitioning, one CU 510 may be formed of TUs having different sizes. Alternatively, a CU may be partitioned into at least one TU depending on the number of vertical lines partitioning the CU or the number of horizontal lines partitioning the CU, or both. A CU may be partitioned into TUs that are symmetric to each other, or may be partitioned into TUs that are asymmetric to each other. In order to partition a CU into TUs symmetric to each other, information of the size/shape of the TUs may be signaled and may be derived from the information of the size/shape of the CU.

Furthermore, the encoding unit may not be partitioned into transform units, and the encoding unit and the transform unit may have the same size.

One coding unit may be partitioned into at least one transform unit, and one transform unit may be partitioned into at least one sub-transform unit.

For example, when one transform unit is partitioned into four sub-transform units, the width size and the height size of one of the four sub-transform units may be half the width size and half the height size of the original transform unit, respectively. For example, when a 32×32 sized transform unit is partitioned into four sub-transform units, each of the four sub-transform units may have a 16×16 size. When one transform unit is partitioned into four sub-transform units, the transform units may be partitioned in a quadtree form.

For example, when one transform unit is partitioned into two sub-transform units, the width size or height size of one of the two sub-transform units may be half the width size or half the height size of the original transform unit, respectively. For example, when a transform unit of a 32×32 size is vertically partitioned into two sub-transform units, each of the two sub-transform units may have a 16×32 size. For example, when a transform unit of a 32×32 size is horizontally partitioned into two sub-transform units, each of the two sub-transform units may have a 32×16 size. When one transform unit is partitioned into two sub-transform units, the transform units may be partitioned in a binary tree form.

When the transformation is performed, the residual block may be transformed by using at least one of predetermined transformation methods. For example, the predetermined transform method may include Discrete Cosine Transform (DCT), discrete Sine Transform (DST), KLT, and the like. Which transform method is applied to transform the residual block may be determined by using at least one of: inter prediction mode information of the prediction unit, intra prediction mode information of the prediction unit, and the size/shape of the transform block. Information indicating the transformation method may be signaled.

The intra prediction mode may be a non-directional mode or a directional mode. The non-directional mode may be a DC mode or a planar mode. The direction mode may be a prediction mode having a specific direction or angle, and the number of direction modes may be M equal to or greater than 1. The direction mode may be indicated as at least one of a mode number, a mode value, and a mode angle.

The number of intra prediction modes may be N equal to or greater than 1, including a non-directional mode and a directional mode.

The number of intra prediction modes may vary depending on the size of the block. For example, when the size of a block is 4×4 or 8×8, the number of intra prediction modes may be 67, when the size of a block is 16×16, the number of intra prediction modes may be 35, when the size of a block is 32×32, the number of intra prediction modes may be 19, and when the size of a block is 64×64, the number of intra prediction modes may be 7.

The number of intra prediction modes may be fixed to N regardless of the block size. For example, the number of intra prediction modes may be fixed to at least one of 35 or 67 regardless of the block size.

The number of intra prediction modes may vary depending on the type of color component. For example, the number of prediction modes may vary depending on whether the color component is a luminance signal or a chrominance signal.

Intra-coding and/or decoding may be performed by using sample values or coding parameters included in reconstructed neighboring blocks.

In order to encode/decode a current block according to intra prediction, it may be identified whether a sample included in a reconstructed neighboring block is available as a reference sample of an encoding/decoding target block. When there is a sample that cannot be used as a reference sample of the encoding/decoding target block, the sample value is copied and/or interpolated to a sample that cannot be used as a reference sample by using at least one sample among samples included in the reconstructed neighboring block, whereby a sample that cannot be used as a reference sample can be used as a reference sample of the encoding/decoding target block.

In intra prediction, a filter may be applied to at least one of the reference samples or the prediction samples based on at least one of an intra prediction mode and a size of an encoding/decoding target block. Here, the encoding/decoding target block may mean a current block, and may mean at least one of an encoding block, a prediction block, and a transform block. The type of filter applied to the reference samples or the prediction samples may vary depending on at least one of the intra prediction mode or the size/shape of the current block. The type of filter may vary depending on at least one of the number of filter taps, the filter coefficient values, or the filter strength.

In the non-directional planar mode among the intra prediction modes, when a prediction block of an encoding/decoding target block is generated, a sample value in the prediction block may be generated according to a sample point by using a weighted sum of an upper reference sample point of a current sample point, a left reference sample point of the current sample point, an upper right reference sample point of the current block, and a lower left reference sample point of the current block.

In the non-directional DC mode among the intra prediction modes, when a prediction block of an encoding/decoding target block is generated, the prediction block may be generated by a mean value of an upper reference sample of the current block and a left reference sample of the current block. Further, filtering may be performed on one or more upper rows and one or more left columns adjacent to the reference sample in the encoding/decoding block by using the reference sample value.

In case of a plurality of direction modes (angle modes) among the intra prediction modes, a prediction block may be generated by using an upper right reference sample and/or a lower left reference sample, and the plurality of direction modes may have different directions. To generate the predicted sample values, interpolation of real units may be performed.

In order to perform the intra prediction method, an intra prediction mode of a current prediction block may be predicted from intra prediction modes of neighboring prediction blocks adjacent to the current prediction block. In case of predicting an intra prediction mode of a current prediction block by using mode information predicted from neighboring intra prediction modes, when the current prediction block and the neighboring prediction block have the same intra prediction mode, information that the current prediction block and the neighboring prediction block have the same intra prediction mode may be transmitted by using predetermined flag information. When the intra prediction mode of the current prediction block is different from the intra prediction modes of neighboring prediction blocks, the intra prediction mode information of the encoding/decoding target block may be encoded by performing entropy encoding.

The quadrangle shown in fig. 7 may indicate an image (or screen). Furthermore, the arrows of fig. 7 may indicate the prediction direction. That is, an image may be encoded or decoded or both encoded and decoded according to a prediction direction. Each picture can be classified into an I picture (intra picture), a P picture (unidirectional prediction picture), a B picture (bi-prediction picture), and the like according to the coding type. Each picture may be encoded and decoded according to an encoding type of each picture.

When the image that is the encoding target is an I picture, the picture itself can be intra-encoded without inter prediction. When the image as the encoding target is a P picture, the image may be encoded by inter prediction or motion compensation using a reference picture only in the forward direction. When the image as the encoding target is a B picture, the image can be encoded by inter prediction or motion compensation using reference pictures in both the forward and reverse directions. Alternatively, the image may be encoded by inter prediction or motion compensation using a reference picture in one of the forward and reverse directions. Here, when the inter prediction mode is used, the encoder may perform inter prediction or motion compensation, and the decoder may perform motion compensation in response to the encoder. The images of the P picture and the B picture, which are encoded or decoded using the reference picture or are encoded and decoded, may be regarded as images for inter prediction.

Hereinafter, the inter prediction according to the embodiment will be described in detail.

Inter prediction or motion compensation may be performed by using both reference pictures and motion information. In addition, inter prediction may use the skip mode described above.

The reference picture may be at least one of a previous picture and a subsequent picture of the current picture. Here, the inter prediction may predict a block of the current picture from the reference picture. Here, the reference picture may mean an image used in predicting a block. Here, the region within the reference picture may be indicated by using a reference picture index (refIdx) indicating the reference picture, a motion vector, or the like.

Inter prediction may select a reference picture and a reference block within the reference picture that is associated with the current block. The prediction block of the current block may be generated by using the selected reference block. The current block may be a block as a current encoding target or a current decoding target among blocks of the current picture.

The motion information may be derived from the processing of inter prediction by the encoding apparatus 100 and decoding apparatus 200. Furthermore, the derived motion information may be used when performing inter prediction. Here, the encoding apparatus 100 and the decoding apparatus 200 may improve encoding efficiency or decoding efficiency or both by using motion information of reconstructed neighboring blocks or motion information of co-located blocks (col blocks) or both. The col block may be a block within a previously reconstructed co-located picture (col picture) that is related to the spatial position of the encoding/decoding target block. The reconstructed neighboring block may be a block within the current picture, as well as a block previously reconstructed by encoding or decoding or both. Further, the reconstructed block may be a block adjacent to the encoding/decoding target block, or a block located at an outer corner of the encoding/decoding target block, or both. Here, the block located at the outer corner of the encoding/decoding target block may be a block vertically adjacent to an adjacent block horizontally adjacent to the encoding/decoding target block. Alternatively, the block located at the outer corner of the encoding/decoding target block may be a block horizontally adjacent to an adjacent block vertically adjacent to the encoding/decoding target block.

The encoding apparatus 100 and the decoding apparatus 200 may determine blocks existing at positions spatially related to the encoding/decoding target block within the col picture, respectively, and may determine the predefined relative positions based on the determined blocks. The predefined relative position may be an internal position or an external position of a block existing at a position spatially related to the encoding/decoding target block or both the internal position and the external position. Furthermore, the encoding device 100 and the decoding device 200 may derive col blocks, respectively, based on the determined predefined relative positions. Here, the col picture may be one picture of at least one reference picture included in the reference picture list.

The method of deriving motion information may vary according to the prediction mode of the encoding/decoding target block. For example, the prediction modes applied to the inter prediction may include Advanced Motion Vector Prediction (AMVP), merge mode, and the like. Here, the merge mode may be referred to as a motion merge mode.

For example, when AMVP is applied as a prediction mode, the encoding apparatus 100 and the decoding apparatus 200 may generate a motion vector candidate list by using a motion vector of a reconstructed neighboring block or a motion vector of a col block or both, respectively. Either the motion vector of the reconstructed neighboring block or the motion vector of the col block or both may be used as motion vector candidates. Here, the motion vector of the col block may be referred to as a temporal motion vector candidate, and the motion vector of the reconstructed neighboring block may be referred to as a spatial motion vector candidate.

The encoding apparatus 100 may generate a bitstream, which may include the motion vector candidate index. That is, the encoding apparatus 100 may generate a bitstream by entropy encoding the motion vector candidate index. The motion vector candidate index may indicate an optimal motion vector candidate selected from among motion vector candidates included in the motion vector candidate list. The motion vector candidate index may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream.

The decoding apparatus 200 may entropy-decode the motion vector candidate index from the bitstream, and may select a motion vector candidate of the decoding target block among the motion vector candidates included in the motion vector candidate list by using the entropy-decoded motion vector candidate index.

The encoding apparatus 100 may calculate a Motion Vector Difference (MVD) between a motion vector of a decoding target block and a motion vector candidate, and entropy-encode the MVD. The bitstream may include entropy encoded MVDs. The MVD may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. Here, the decoding apparatus 200 may entropy-decode MVDs received from the bitstream. The decoding apparatus 200 may derive a motion vector of the decoding target block from the sum of the decoded MVD and the motion vector candidates.

The bitstream may include a reference picture index indicating a reference picture, etc., and the reference picture index may be entropy encoded and transmitted from the encoding apparatus 100 to the decoding apparatus 200 through the bitstream. The decoding apparatus 200 may predict a motion vector of the decoding target block by using motion information of neighboring blocks and may derive the motion vector of the decoding target block by using the predicted motion vector and the motion vector difference. The decoding apparatus 200 may generate a prediction block of the decoding target block based on the derived motion vector and the reference picture index information.

As another method of deriving motion information, a merge mode is used. The merge mode may mean a merging of motions of a plurality of blocks. A merge mode may mean that motion information of one block is applied to another block. When the merge mode is applied, the encoding apparatus 100 and the decoding apparatus 200 may generate a merge candidate list by using motion information of a reconstructed neighboring block or motion information of a col block or both, respectively. The motion information may include at least one of: 1) motion vectors, 2) reference picture indices, and 3) inter prediction indicators. The prediction indicator may indicate unidirectional (L0 prediction, L1 prediction) or bidirectional.

Here, the merge mode may be applied to each CU or each PU. When the merge mode is performed at each CU or each PU, the encoding apparatus 100 may generate a bitstream by entropy decoding predefined information and may transmit the bitstream to the decoding apparatus 200. The bitstream may comprise said predefined information. The predefined information may include: 1) A merge flag as information indicating whether a merge mode is performed for each block partition, 2) a merge index as information indicating which block is merged among neighboring blocks adjacent to the encoding target block. For example, neighboring blocks adjacent to the encoding target block may include a left neighboring block of the encoding target block, an upper neighboring block of the encoding target block, a temporal neighboring block of the encoding target block, and the like.

The merge candidate list may indicate a list storing motion information. Further, a merge candidate list may be generated before the merge mode is performed. The motion information stored in the merge candidate list may be at least one of the following motion information: the motion information of a neighboring block adjacent to the encoding/decoding target block, the motion information of a co-located block in the reference picture related to the encoding/decoding target block, the motion information newly generated by pre-combining the motion information existing in the merge motion candidate list, and the zero merge candidate. Here, the motion information of neighboring blocks adjacent to the encoding/decoding target block may be referred to as spatial merging candidates. The motion information of the co-located block in the reference picture related to the encoding/decoding target block may be referred to as a temporal merging candidate.

The skip mode may be a mode in which mode information of neighboring blocks themselves is applied to the encoding/decoding target block. The skip mode may be one of modes for inter prediction. When the skip mode is used, the encoding apparatus 100 may entropy-encode information about which block motion information is used as the motion information of the encoding target block, and may transmit the information to the decoding apparatus 200 through a bitstream. The encoding apparatus 100 may not transmit other information (e.g., syntax element information) to the decoding apparatus 200. The syntax element information may include at least one of motion vector difference information, a coded block flag, and a transform coefficient level.

The residual signal generated after intra prediction or inter prediction may be transformed into the frequency domain through a transform process as part of a quantization process. Here, the first transformation may use DCT type 2 (DCT-II) as well as various DCT, DST kernels. These transform kernels may perform a separable transform on the residual signal for performing a 1D transform in the horizontal and/or vertical directions, or may perform a 2D non-separable transform on the residual signal.

For example, in the case of a 1D transform, DCT-II, DCT-V, DCT-VIII, DST-I, and DST-VII as shown in the following tables may be used for the DCT and DST types used in the transform. For example, as shown in tables 1 and 2, the DCT or DST type used in the transformation by synthesizing the transformation set can be derived.

TABLE 1

Transform set	Transformation
		0	DST_VII、DCT-VIII
1	DST-VII、DST-I
		2	DST-VII、DCT-V

TABLE 2

Transform set	Transformation
		0	DST_VII、DCT-VIII、DST-I
1	DST-VII、DST-I、DCT-VIII
		2	DST-VII、DCT-V、DST-I

For example, as shown in fig. 8, different transform sets are defined for the horizontal direction and the vertical direction according to the intra prediction mode. Next, the encoder/decoder may perform transformation and/or inverse transformation by transforming using the intra prediction mode of the current encoding/decoding target block and the related transformation set. In this case, entropy encoding/decoding is not performed on the transform set, and the encoder/decoder may define the transform set according to the same rule. In this case, entropy encoding/decoding indicating which transform is used among the transforms of the transform set may be performed. For example, when the block size is equal to or smaller than 64×64, three transform sets are synthesized according to the intra prediction mode as shown in table 2, and three transforms are used for each horizontal direction transform and vertical direction transform to combine and perform a total of nine transform methods. Next, the residual signal is encoded/decoded by using an optimal transform method, whereby encoding efficiency can be improved. Here, in order to perform entropy encoding/decoding on information on which transform method is used among three transforms in one transform set, truncated unary binarization may be used. Here, in order to perform at least one of the vertical transform and the horizontal transform, entropy encoding/decoding may be performed on information indicating which one of the transforms of the transform set is used.

After completing the above-described first transform, as shown in fig. 9, the encoder may perform a second transform for the transform coefficients to increase the energy concentration. The secondary transform may perform a separable transform for performing a 1D transform in a horizontal and/or vertical direction, or may perform a 2D non-separable transform. The transformation information used may be transmitted or may be derived by the encoder/decoder from the current encoding information and the neighboring encoding information. For example, as a 1D transform, a set of transforms for a secondary transform may be defined. Entropy encoding/decoding is not performed on the transform set, and the encoder/decoder may define the transform set according to the same rule. In this case, information indicating which transform among the transforms of the transform set is used may be transmitted, and the information may be applied to at least one residual signal through intra prediction or inter prediction.

At least one of the number or type of transform candidates is different for each set of transforms. At least one of the number or type of transform candidates may be determined differently based on at least one of: the location, size, partition form, and direction/non-direction of the block (CU, PU, TU, etc.) or prediction mode (intra/inter mode) or intra prediction mode.

The decoder may perform the secondary inverse transform depending on whether the secondary inverse transform is performed, and may perform the primary inverse transform depending on whether the primary inverse transform is performed from the result of the secondary inverse transform.

The above-described primary transform and secondary transform may be applied to at least one signal component among luminance/chrominance components, or may be applied according to the size/shape of an arbitrary coding block. Entropy encoding/decoding may be performed on indexes indicating whether the first transform/secondary transform is used or not and both the used first transform/secondary transform in any encoding block. Alternatively, the index may be derived by default by the encoder/decoder from at least one piece of current/neighbor coding information.

The residual signal generated after intra prediction or inter prediction is subjected to quantization processing after being subjected to primary transform and/or secondary transform, and the quantized transform coefficient is subjected to entropy encoding processing. Here, as shown in fig. 10, the quantized transform coefficients may be scanned in a diagonal direction, a vertical direction, and a horizontal direction based on at least one of an intra prediction mode or a size/shape of a minimum block.

Further, the quantized transform coefficients, on which entropy decoding is performed, may be arranged in a block form by being inversely scanned, and at least one of inverse quantization or inverse transformation may be performed on the relevant blocks. Here, as a method of the inverse scan, at least one of the diagonal direction scan, the horizontal direction scan, and the vertical direction scan may be performed.

For example, when the size of the current encoding block is 8×8, first transform, second transform, and quantization may be performed on the residual signal for the 8×8 block, and then, scan and entropy encoding may be performed on the quantized transform coefficients for each of the four 4×4 sub-blocks according to at least one of the three scan order methods shown in fig. 10. Further, inverse scanning may be performed on the quantized transform coefficients by performing entropy decoding. The quantized transform coefficients on which the inverse scan is performed become transform coefficients after being inverse quantized, and at least one of a secondary inverse transform or a primary inverse transform is performed, whereby a reconstructed residual signal can be generated.

In the video encoding process, one block may be partitioned as shown in fig. 11, and an indicator corresponding to partition information may be signaled. Here, the partition information may be at least one of: partition flag (split_flag), quadtree partition flag (qb_flag), quadtree partition flag (quadtree_flag), binary tree partition flag (binaryre_flag), and binary tree partition type flag (btype_flag). Here, split_flag is a flag indicating whether a block is partitioned or not, qb_flag is a flag indicating whether a block is partitioned in a quadtree form or a binary tree form, quadtree_flag is a flag indicating whether a block is partitioned in a quadtree form or not, binaryre_flag is a flag indicating whether a block is partitioned in a binary tree form or not, btype_flag is a flag indicating whether a block is vertically partitioned or horizontally partitioned in the case of partition in a binary tree form.

When the partition flag is 1, it may indicate that the partition is executed, and when the partition flag is 0, it may indicate that the partition is not executed. In the case of a quadtree/binary tree flag, 0 may indicate a quadtree partition and 1 may indicate a binary tree partition. Alternatively, 0 may indicate a binary tree partition and 1 may indicate a quadtree partition. In the case of a binary tree partition type flag, 0 may indicate a horizontal direction partition and 1 may indicate a vertical direction partition. Alternatively, 0 may indicate a vertical direction zone and 1 may indicate a horizontal direction zone.

For example, the partition information of fig. 11 may be derived by signaling at least one of quadtree_ flag, binarytree _flag and btype_flag as shown in table 3.

TABLE 3

For example, the partition information of fig. 11 may be derived by signaling at least one of split_flag, qb_flag, and btype_flag as shown in table 4.

TABLE 4

The partitioning method may be performed in a quadtree form only or a binary tree form only according to the size/shape of the block. In this case, the split_flag may mean a flag indicating whether the partitioning is performed in the quadtree form or the binary tree form. The size/shape of the block may be derived from the depth information of the block, and the depth information may be signaled.

When the size of the block is in a predetermined range, the partitioning may be performed only in a quadtree form. Here, the predetermined range may be defined as at least one of a size of a maximum block or a size of a minimum block that can be partitioned only in a quadtree form. Information indicating the size of the maximum block/minimum block of partitions allowed in the form of a quadtree may be signaled through a bitstream, and may be signaled in units of at least one of a sequence, a picture parameter, or a slice (segment). Alternatively, the size of the maximum block/minimum block may be a fixed size preset in the encoder/decoder. For example, when the size of a block ranges from 256×256 to 64×64, the partitioning may be performed only in a quadtree form. In this case, the split_flag may be a flag indicating whether or not the partitioning is performed in the quadtree form.

When the size of the block is in a predetermined range, the partitioning may be performed only in a binary tree form. Here, the predetermined range may be defined as at least one of a size of a maximum block or a size of a minimum block that can be partitioned only in a binary tree form. Information indicating the size of the largest/smallest block of partitions that are allowed in the form of a binary tree may be signaled through a bitstream, and may be signaled in units of at least one of a sequence, a picture parameter, or a stripe (segment). Alternatively, the size of the maximum block/minimum block may be a fixed size preset in the encoder/decoder. For example, when the size of the block ranges from 16x16 to 8x8, the partitioning may be performed only in a binary tree form. In this case, the split_flag may be a flag indicating whether or not the partitioning is performed in a binary tree form.

After partitioning a block in a binary tree form, partitioning may be performed only in a binary tree form when the partitioned block is further partitioned.

When the width size or the length size of the partitioned blocks cannot be further partitioned, at least one indicator may not be signaled.

In addition to the quadtree-based binary tree partitioning, the quadtree-based partitioning may be performed after the binary tree partitioning.

As shown in fig. 12, intra prediction may include: deriving intra prediction modes (S1210), constructing reference points (S1220), and/or performing intra prediction (S1230).

In the deriving of the intra prediction mode (S1210), the intra prediction modes of neighboring blocks may be used, the intra prediction mode of the current block may be decoded (e.g., entropy decoded) from the bitstream, and the intra prediction mode of the color component of the current block and/or the intra prediction mode of the current block may be derived using the intra prediction mode of the transform model. Alternatively, in the step of deriving the intra prediction mode (S1210), the intra prediction mode of the current block may be derived using the intra prediction mode of the neighboring block, a combination of one or more intra prediction modes of the neighboring block, and/or the intra prediction mode derived by the MPM.

In the step of constructing the reference sample (S1220), the reference sample may be constructed by performing reference sample selection and/or reference sample filtering.

In the step of performing intra prediction (S1230), intra prediction may be performed for the current block using non-directional prediction, prediction based on location information, prediction between color components, and/or residual signal prediction. In the step of performing intra prediction (S1230), filtering may be additionally performed on the prediction samples. If direction prediction is performed, different direction predictions may be performed according to one or more sample units. For example, the one or more sample units may be a single sample, a group of samples, a line, and/or a sub-block.

Hereinafter, the step of deriving the intra prediction mode (S1210) will be described in more detail.

As described above, in order to derive the intra prediction mode of the current block, at least one of the following methods may be used: methods for using intra prediction modes of one or more neighboring blocks, methods for decoding intra prediction modes of a current block from a bitstream, methods for using coding parameters of neighboring blocks, and methods for using intra prediction modes between color components. The neighboring blocks may be one or more blocks reconstructed prior to encoding/decoding of the current block.

If a neighboring block is located outside the boundary of at least one predetermined unit, such as a picture, a slice, a parallel block, and a Coding Tree Unit (CTU), or PCM mode or inter prediction has been applied to the neighboring block, it may be determined that the neighboring block is not available.

The size of the current block may be w×h, where W and H are positive integers and may be equal or different. W and/or H may be, for example, at least one of 2, 4, 8, 16, 32, 64, 128, 256, and 512.

Fig. 13 is a diagram depicting a method for deriving an intra prediction mode of a current block from neighboring blocks.

In fig. 13, a to k marked on the neighboring block may represent an intra prediction mode of the neighboring block or a number of the intra prediction mode. The location of the neighboring blocks used to derive the intra prediction mode of the current block may be a predefined fixed location. Alternatively, information about the locations of neighboring blocks may be derived by encoding/decoding. In the present disclosure, encoding/decoding may include entropy encoding and entropy decoding.

In case that the intra prediction mode of the neighboring block is used, the specific mode of the neighboring block may be derived as the intra prediction mode of the current block. For example, the intra prediction mode i of the neighboring block to which the left-side sample (-1, 0) neighboring to the sample (0, 0) of the current block belongs may be derived as the intra prediction mode of the current block. Alternatively, the intra prediction mode f of the neighboring block to which the upper sample (0, -1) adjacent to the sample (0, 0) of the current block belongs may be derived as the intra prediction mode of the current block. Alternatively, the intra prediction mode b of the neighboring block to which the upper left sample (-1, -1) adjacent to the sample (0, 0) of the current block belongs may be derived as the intra prediction mode of the current block. Alternatively, the intra prediction mode g of the neighboring block to which the upper sample (W-1, -1) adjacent to the sample (W-1, 0) of the current block belongs may be derived as the intra prediction mode of the current block. Alternatively, the intra prediction mode k of the neighboring block to which the upper right sample (W-1) adjacent to the sample (W-1, 0) of the current block belongs may be derived as the intra prediction mode of the current block. Alternatively, the intra prediction mode j of the neighboring block to which the left-side sample (1, h-1) neighboring the sample (0, h-1) of the current block belongs may be derived as the intra prediction mode of the current block. Alternatively, the intra prediction mode l of the neighboring block to which the lower left sample (-1, h) neighboring to the sample (0, h-1) of the current block belongs may be derived as the intra prediction mode of the current block. Alternatively, the intra prediction mode of a neighboring block located at a predetermined position among the neighboring blocks may be derived as the intra prediction mode of the current block. For example, the predetermined position may be encoded/decoded from a bitstream, or may be derived based on encoding parameters. For example, the predetermined position may be a block having an intra prediction mode e.

Alternatively, one or more of the neighboring blocks of the current block may be selected. The selection may be made based on information explicitly signaled by the bitstream. Alternatively, the selection may be made according to a predetermined criterion between the encoder and the decoder. The intra prediction mode of the current block may be derived from the intra prediction modes of the selected one or more neighboring blocks. For example, the intra prediction mode of the current block may be derived using statistics of intra prediction modes of selected neighboring blocks. For example, the statistics may include a minimum, a maximum, an average, a weighted average, a most frequent value, and/or a median.

For example, a mode having a smaller mode number or a larger mode number among intra prediction modes i and f of neighboring blocks to which a left side sample and an upper side sample adjacent to the sample (0, 0) of the current block belong may be derived as the intra prediction mode of the current block. For example, if the intra prediction modes of the selected neighboring blocks are b, f, g, i and j, a mode having the smallest value among the mode numbers of the intra prediction modes may be derived as the intra prediction mode of the current block. For example, if the intra prediction modes of the selected neighboring block are i, b, and f, a mode having a median value of mode numbers of the intra prediction modes may be derived as the intra prediction mode of the current block. For example, the most frequent one of the intra prediction modes of neighboring blocks adjacent to the current block may be derived as the intra prediction mode of the current block.

Alternatively, the intra prediction mode of the current block is derived by combining intra prediction modes of one or more neighboring blocks. The intra prediction mode may be represented as at least one of a mode number, a mode value, and a mode angle. For example, an average value of one or more intra prediction modes of neighboring blocks may be derived as the intra prediction mode of the current block. The average value of the two intra prediction modes may represent at least one of a median value between the two mode numbers, a median value of the two mode values, and an intermediate angle between the two mode angles.

For example, a mode corresponding to an average value of mode values of intra prediction modes i and f of neighboring blocks to which a left side sample and an upper side sample adjacent to the sample (0, 0) of the current block belong may be derived as the intra prediction mode of the current block. For example, the intra prediction mode pred_mode of the current block may be derived by at least one of the method (1), the method (2), and the method (3) described in [ equation 1 ].

[ equation 1]

Pred_mode＝(i+f)>>1 (1)

Pred_mode＝(i+f+1)>>1 (2)

Pred_mode＝(i+f)/2 (3)

Alternatively, if the intra prediction mode i of the neighboring block is a non-directional mode, the intra prediction mode of the current block may be derived as the intra prediction mode i. Alternatively, if the intra prediction mode f of the neighboring block is a directional mode, the intra prediction mode of the current block may be derived as the intra prediction mode f.

Alternatively, the intra prediction mode of the current block may be derived as a mode corresponding to an average value of at least one of the mode values of the intra prediction modes b, f, g, i and j of the neighboring blocks. For example, the intra prediction mode pred_mode of the current block may be derived by at least one of the method (1), the method (2), the method (3), and the method (4) described in [ equation 2 ].

[ equation 2]

Pred_mode＝(f+g+i+j+2)>>2 (1)

Pred_mode＝(b+f+g+i+j)/5 (2)

Pred_mode＝(i+f+k+l+2)>>2 (3)

Pred_mode＝(b+f+k+i+l)/5 (4)

Alternatively, a mode corresponding to an average value of available intra prediction modes of neighboring blocks in the vicinity may be derived as an intra prediction mode of the current block. For example, if a left neighboring block of the current block is located outside the boundary of a picture, a parallel block, a slice, and/or a CTU, or the left neighboring block of the current block corresponds to at least one of a PCM mode or an inter prediction mode and is thus unavailable, a mode corresponding to the statistics of intra prediction modes (e.g., f and g) of the upper neighboring block may be derived as the intra prediction mode of the current block.

For example, a weighted average or a weighted sum may be used as the statistics of intra prediction modes of neighboring blocks. Here, the weights may be allocated based on directionality of intra prediction modes of neighboring blocks. For example, patterns assigned relatively large weights may be predefined or signaled. For example, the mode to which the relatively large weight is assigned may be at least one of a vertical direction mode, a horizontal direction mode, and a non-direction mode. These modes may be assigned the same weight or different weights. For example, the weighted sum of intra prediction modes i and f may be derived as the intra prediction mode pred_mode of the current block by the following [ equation 3 ]. In the following [ equation 3], the pattern f may be a pattern (e.g., a vertical direction pattern) to which a relatively large weight is assigned.

[ equation 3]

Pred_mode＝(i+3*f+2)>>2

Alternatively, the weights to be used for the weighted sum may be determined based on the size of the neighboring blocks. For example, if the size of an upper block adjacent to the current block is greater than the size of a left block adjacent to the current block, a greater weight may be assigned to the intra prediction mode of the upper adjacent block. Alternatively, a larger weight may be assigned to the intra prediction mode of the smaller neighboring block.

Alternatively, if one or more intra prediction modes of neighboring blocks are non-directional modes, the non-directional modes may be derived as intra prediction modes of the current block. Alternatively, the intra prediction mode of the current block may be derived using intra prediction modes of neighboring blocks other than the non-directional mode. If the intra prediction modes of the neighboring blocks are all non-directional modes, the intra prediction mode of the current block may be derived as at least one of a DC mode or a planar mode.

Alternatively, the intra prediction mode of the current block may be derived using the Most Probable Mode (MPM) of intra prediction modes based on neighboring blocks. If the MPM is used, one or more pieces of information regarding the intra prediction mode of the current block may be encoded/decoded.

If MPM is used, an MPM list may be constructed. The MPM list may include intra-prediction modes derived based on intra-prediction modes of neighboring blocks. The MPM list may include N candidate patterns. N is a positive integer and may vary depending on the size and/or shape of the current block. Alternatively, the information about N may be signaled by a bitstream.

For example, an intra prediction mode of the current block, which is derived using one or more intra prediction modes of neighboring blocks, may be a candidate mode included in the MPM list.

In the example shown in fig. 13, intra prediction modes of neighboring blocks at the same positions of (-1, h-1), (W-1, -1), (W-1), (-1, h), and (-1, -1) adjacent to the current block may be used, and for example, MPM lists may be made in the order of j, g, plane, DC, l, k, and b. Alternatively, the MPM list may be made in the order of i, f, plane, DC, l, k, and b. The repetition pattern may be included once in the MPM list. If the MPM list is not filled due to the presence of a repeating pattern, additional candidate patterns may be included in the list based on the patterns included in the list. For example, a pattern corresponding to +N or-N (N is a positive integer, e.g., 1) of the patterns included in the list may be added to the list. Alternatively, at least one mode among a horizontal mode, a vertical mode, a 45 degree mode, a 135 degree mode, and a 225 degree mode, which is not included in the list, may be added to the list. Alternatively, the MPM list may be constructed using a combination and/or statistics of more than 1 intra prediction modes of neighboring blocks.

There may be a plurality of MPM lists, and methods for constructing each MPM list may be different from each other. For example, 3 MPM lists (MPM list 1, MPM list 2, and MPM list 3) may be constructed. In this case, the intra prediction mode included in each MPM list may not be repeated.

An indicator (e.g., prev _ intra _ luma _ pred _ flag) indicating whether the same mode as the intra prediction mode of the current block exists in the derived MPM list may be encoded in or decoded from the bitstream.

If the indicator indicates that there is the same mode as the intra prediction mode of the current block in the MPM list, index information (e.g., mpm_idx) indicating which mode among modes included in the MPM list is the mode may be encoded in or decoded from the bitstream. The intra prediction mode of the current block may be derived based on the decoded index information.

If the indicator indicates that the same mode as the intra prediction mode of the current block does not exist in the MPM list, information regarding the intra prediction mode of the current block may be encoded in or decoded from the bitstream. The intra prediction mode of the current block may be derived based on the decoded information on the intra prediction mode of the current block. Here, the intra prediction modes not included in the MPM list may be arranged in at least one of an ascending order or a descending order. Alternatively, one or more of the intra prediction modes not included in the MPM list may be divided into one or more groups. For example, modes corresponding to +n or-N (N is a positive integer, e.g., 1, 2, or 3) of intra prediction modes included in the MPM list may be grouped into one group. Here, the group may include a predetermined number (e.g., 8 or 16) of intra prediction modes, and the modes included in the group may not be included in the MPM list.

Alternatively, as described above, when there are a plurality of MPM lists, the indicator (e.g., prev_intra_luma_pred_flag) may indicate whether there is the same mode as the intra prediction mode of the current block in MPM list 1. In case that the indicator indicates that the same mode as the intra prediction mode of the current block does not exist in the MPM list 1, it may be determined whether the same mode as the intra prediction mode of the current block exists in the MPM list 2. In case that the same mode as the intra prediction mode of the current block exists in the MPM list 2, the corresponding mode may be derived as the intra prediction mode of the current block. In the case where the same mode as the intra prediction mode of the current block does not exist in the MPM list 2, the determination for the MPM list 3 may be performed. The determination for the plurality of MPM lists may be sequentially performed using the same method as above.

Alternatively, separate information indicating one of the plurality of MPM lists may be transmitted. In this case, the intra prediction mode of the current block may be derived by using the MPM list indicated by the separate information and the indicator (e.g., prev_intra_luma_pred_flag).

Alternatively, the predetermined candidates in the derived MPM list may be derived as the intra prediction mode of the current block. For example, a mode corresponding to list 0 (i.e., a first mode in the MPM list) may be derived as an intra prediction mode of the current block. Alternatively, an index corresponding to a predetermined mode in the list may be encoded/decoded, and the corresponding mode may be derived as an intra prediction mode of the current block.

Regarding the construction of the MPM list, one MPM list may be made for a block of a predetermined size. If the predetermined size block is divided into a plurality of sub-blocks, each of the plurality of sub-blocks may use the MPM list.

For example, if the current block corresponds to the predetermined size block, an MPM list may be made for the current block. If the current block is divided into one or more sub-blocks, an intra prediction mode may be derived for each sub-block using the MPM list. For example, if the current block is 8×8 in size and divided into four 4×4 sub-blocks, an MPM list may be made for the current block and may be used for each sub-block.

Regarding the construction of the MPM list, the MPM list may be made for each of a plurality of sub-blocks into which a block of a predetermined size is divided.

For example, if the current block corresponds to a block of the predetermined size, an MPM list may be made for each sub-block of the current block using intra prediction modes of neighboring blocks of the current block. For example, if the size of the current block is 8×8 and there are four 4×4 sub-blocks, an MPM list may be made for each of the four sub-blocks using intra prediction modes of neighboring blocks of the current block. Thus, the MPM list may be made for four sub-blocks simultaneously.

Alternatively, the intra prediction mode of the current block may be derived using at least one of the intra prediction mode of the current block and the intra prediction modes of neighboring blocks derived by the MPM.

For example, if the intra prediction mode of the current block, which is derived by the MPM, is pred_mpm, the intra prediction mode of the current block may be derived by changing pred_mpm to a specific mode using one or more intra prediction modes of neighboring blocks.

For example, pred_mpm may be increased by N or decreased by N by comparing the size of the intra prediction modes of pred_mpm and neighboring blocks. Here, N may be a predetermined integer such as +1, +2, +3, 0, -1, -2, or-3. For example, pred_mpm may be increased if pred_mpm is less than the intra-prediction mode of the neighboring block and/or the statistics of intra-prediction modes of one or more neighboring blocks. Alternatively, if pred_mpm is greater than the intra prediction modes of neighboring blocks, pred_mpm may be decreased. Alternatively, the intra prediction mode of the current block may be derived based on pred_mpm and/or a value compared to pred_mpm.

In the example shown in fig. 13, if pred_mpm is less than the mode value f, pred_mpm+1 may be derived as an intra prediction mode of the current block. Alternatively, if pred_mpm is less than the mode value g, pred_mpm+1 may be derived as an intra prediction mode of the current block. Alternatively, if pred_mpm is smaller than the mode value f, pred_mpm+2 may be derived as an intra prediction mode of the current block. Alternatively, if pred_mpm is greater than the mode value f, pred_mpm-1 may be derived as an intra prediction mode of the current block. Alternatively, if pred_mpm is less than the mode value i, pred_mpm+1 may be derived as the intra prediction mode of the current block. Alternatively, if pred_mpm is smaller than the average of the mode values f and i, pred_mpm+1 may be derived as the intra prediction mode of the current block. Alternatively, if pred_mpm is smaller than the average of the pattern values f and i, the difference between pred_mpm and the average may be increased by 1/2. For example, pred_mpm+ { ((f+i+1) > > 1-pred_mpm+1) > >1} may be derived as the intra prediction mode of the current block.

Alternatively, if one of the modes of the pred_mpm and the neighboring block is a non-directional mode and the other is a directional mode, the non-directional mode may be derived as an intra prediction mode of the current block or the directional mode may be derived as an intra prediction mode of the current block.

In case that the intra prediction mode of the current block is derived using a Most Probable Mode (MPM) list, at least one of the following MPM lists may be used, for example. Alternatively, the intra prediction mode of the current block may be entropy encoded/decoded.

-an MPM list for the current block.

-at least one of the MPM lists for upper layer blocks of the current block.

-at least one of the MPM lists for neighboring blocks of the current block.

Information required to make the MPM list, such as whether the MPM list of the current block is used, whether at least one of the MPM lists of the upper layer blocks of the current block is used, and whether at least one of the MPM lists of the neighboring blocks of the current block is used, may be entropy encoded/decoded in at least one of a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), an Adaptive Parameter Set (APS), a slice, a header, parallel blocks, CTU, CU, PU, and TUs.

The upper layer block may be a block having a depth value smaller than that of the current block. In addition, the upper layer block may represent at least one block among blocks having a smaller depth value, including the current block. Here, the depth value may represent a value increased by 1 each time a block is divided. For example, the depth value of an undivided CTU may be 0.

Further, the upper layer block may mean the following embodiment or a combination of at least one embodiment among the following embodiments. If the first block is an upper layer block and the second block is a current block, the first block may include the second block. Here, the first block may be a block having a shallower depth than the second block. Here, a block with a shallower depth may represent a block with a smaller depth value. Alternatively, the first block may represent a block of a larger size than the second block. Here, a larger block may represent a block of shallower depth.

Information indicating at least one of the size or depth of the first block may be signaled by the encoder. The information may be signaled at a level of at least one of VPS, SPS, picture, slice, parallel block, and block.

At least one of the size or depth of the first block may be derived based on at least one of the size or depth of the second block. For example, at least one of the size or depth of the first block may be derived based on a value obtained by adding a predetermined constant to the size (or depth) of the second block (current block) or subtracting the predetermined constant from the size (or depth) of the second block (current block). Alternatively, at least one of the size or depth of the first block may have a fixed value preset in the encoder/decoder.

The neighboring block may be at least one of a block spatially and/or temporally neighboring the current block. Neighboring blocks may have been encoded/decoded. In addition, the neighboring block may have a depth (or size) equal to or different from the depth (or size) of the current block. The neighboring block may represent a block located at a predetermined position with respect to the current block. Here, the predetermined position may be at least one of a left upper position, an upper position, a right upper position, a left side position, and a left lower position with respect to the current block.

Alternatively, the predetermined position may be a position within a picture different from a picture to which the current block belongs. The block at the predetermined position may represent at least one of a co-located block in a different picture and/or a block adjacent to the co-located block of the current block. Alternatively, the block at the predetermined position may be a block corresponding to the current block in a specific region of a different picture having the same prediction mode as that of the current block.

Further, the neighboring block may mean the following embodiment or a combination of at least one of the following embodiments. If the first block is a neighboring block and the second block is a current block, the first block is an encoded/decoded block neighboring the second block. The first block may be a block having a depth equal to a depth of the second block. The first block may be a block having a size equal to a size of the second block.

The first block and the second block may belong to the same coding block (CTU, CU, etc.) or different coding blocks. The depth and/or size of the first block may be different from the depth and/or size of the second block.

The MPM list of the upper layer block or the neighboring block may represent an MPM list fabricated based on the upper layer block or the neighboring block. Intra prediction modes of encoded/decoded blocks adjacent to an upper layer block or an adjacent block may be added to an MPM list of the upper layer block or the adjacent block.

For example, in fig. 14, a block U (a block marked with a thick solid line) may be an upper layer block of a block F, a block G, a block H, a block I, and a block J. At least one of the block F, the block G, the block H, the block I, and the block J may be a current block.

For example, in fig. 14, a block V (a block marked with a thick dotted line) may be an upper layer block of the block G, the block H, the block I, and the block J. At least one of the block G, the block H, the block I, and the block J may be a current block.

For example, in fig. 14, the block W (block marked with a dot pattern) may be an upper layer block of the blocks G, H, and I. At least one of the block G, the block H, and the block I may be a current block.

For example, in fig. 14, the block X (block marked with a diamond pattern) may be an upper layer block of the block H and the block I. At least one of the block H and the block I may be a current block.

For example, in fig. 14, the neighboring block of the current block D may be at least one of block B, block C, and block K.

For example, in fig. 14, the neighboring block of the current block L may be at least one of block C, block D, block E, block H, and block K.

For example, in fig. 14, the neighboring blocks of the current block P may be at least one of block E, block H, block I, block J, block L, block N, and block O.

For example, in fig. 14, the neighboring block of the current block S may be at least one of block I, block J, block P, block Q, and block R.

The intra prediction mode of the current block may be derived or entropy encoded/decoded using the N MPM lists. Here, N may be 0 or a positive integer. That is, the intra prediction mode of the current block may be derived or entropy encoded/decoded using a plurality of MPM lists. In addition, the plurality of MPM lists may represent a plurality of MPM lists or a plurality of lists. The N MPM lists for the current block may include at least one of an MPM list of the current block, an MPM list of an upper layer block, and an MPM list of a neighboring block.

Furthermore, the N MPM lists may be generated using at least one of the encoding parameters for the current block.

Further, at least one of the plurality of sub-blocks of the predetermined block may be a current block. In this case, an upper layer block of the plurality of sub-blocks may be the predetermined block. Here, the plurality of sub-blocks may be included in the predetermined block. Further, the plurality of sub-blocks may be blocks divided from the predetermined block. Further, at least one of the sub-blocks, which do not correspond to the current block, among the plurality of sub-blocks divided from the predetermined block may be a neighboring block of the current block. Here, the sub-block may mean a lower layer block opposite to an upper layer block in meaning.

For example, if the current block is block H in fig. 14, a plurality of MPM lists for the current block may be made in at least one of the following methods. The plurality of MPM lists may include at least one of an MPM list made based on a current block and an MPM list made based on an upper layer block of the current block.

For example, in fig. 14, an MPM list made based on an upper layer block X may be used for the current block H.

For example, in fig. 14, an MPM list made based on an upper layer block W may be used for the current block H.

For example, in fig. 14, an MPM list made based on an upper layer block V may be used for the current block H.

For example, in fig. 14, an MPM list made based on an upper layer block U may be used for a current block H.

For example, in fig. 14, at least one of MPM lists made based on at least one of the upper layer block X, the upper layer block W, the upper layer block V, and the upper layer block U may be used to derive an intra prediction mode of the current block H.

For example, if the current block is block H in fig. 14, a plurality of MPM lists for the current block may be made in at least one of the following methods. The plurality of MPM lists may include at least one of an MPM list made based on a current block and an MPM list made based on a neighboring block of the current block.

For example, in fig. 14, an MPM list made based on the neighboring block E may be used for the current block H.

For example, in fig. 14, an MPM list made based on the neighboring block G may be used for the current block H.

For example, in fig. 14, at least one of MPM lists made based on at least one of the neighboring block E and the neighboring block G may be used for the current block H.

The intra prediction mode of the current block may be derived using the prepared MPM list, or the intra prediction mode of the current block may be entropy encoded/decoded.

Using the MPM list of the upper layer block or the MPM list of the neighboring block for the current block may mean using the MPM list of the upper layer block or the MPM list of the neighboring block in an operation of deriving or encoding/decoding an intra prediction mode of the current block.

The plurality of MPM lists for the current block may include MPM lists for N upper layer blocks. Here, N is 0 or a positive integer.

Information such as the number, depth, and/or depth range of upper layer blocks and/or the difference between the depth of the current block and the depth of the upper layer block may be required in the preparation of the MPM list for the upper layer block. This information required to make the MPM list of the upper layer block may be entropy encoded/decoded in at least one of VPS, SPS, PPS, APS, a slice header, a parallel block header, CTU, CU, PU, and a TU.

For example, if the depth value of the current block is D, the MPM list of the upper layer block may be made in at least one of the following methods. As described above, the MPM list of the upper layer block manufactured may be used in an operation of deriving an intra prediction mode of the current block or encoding/decoding the intra prediction mode of the current block. Here, D is 0 or a positive integer.

For example, an MPM list of an upper layer block having a depth value of D-1 may be used for the current block.

For example, an MPM list of an upper layer block having a depth value of D-2 may be used for the current block.

For example, an MPM list of an upper layer block having a depth value of D-K may be used for the current block. Here, K is a positive integer equal to or smaller than D.

For example, an MPM list of an upper layer block having depth values ranging from D-1 to D-2 may be used for the current block.

For example, an MPM list of an upper layer block having depth values ranging from D-1 to D-3 may be used for the current block.

For example, an MPM list of an upper layer block having depth values ranging from D-1 to D-K may be used for the current block. Here, K is a positive integer equal to or smaller than D.

For example, an MPM list of an upper layer block having depth values ranging from D-K to D-l may be used for the current block. Here, K and l may be positive integers equal to or less than D. In addition, K may be less than l.

For example, if one upper layer block is used, an MPM list of an upper layer block having a depth value of D-1 or 0 may be used for the current block.

For example, if two upper layer blocks are used, an MPM list of at least two upper layer blocks among the upper layer blocks having depth values of 0, 1, D-1, or D-2, respectively, may be used for the current block.

For example, if K upper layer blocks are used, an MPM list of at least an upper layer block among the upper layer blocks having depth values ranging from D-l to D-K and the upper layer blocks having depth values ranging from 0 to K-1 may be used for the current block. K may be a positive integer less than or equal to D.

For example, if K upper layer blocks are used, an MPM list of K upper layer blocks with respect to the current block may be used for the current block. K may be a positive integer less than or equal to D.

If the plurality of MPM lists for the current block includes an MPM list of an upper layer block, information on a size and/or depth of the current block may be used to derive a number and/or depth value of the upper layer block used.

For example, if the current block is a w×h block with a depth value D, the size of the current block may be expressed as the number of pixels w×h. If the number of pixels in the current block is equal to or greater than a predetermined threshold, an MPM list of one or more of the upper layer blocks having depth values ranging from D-1 to D-K may be used for the current block. Here, K may be a positive integer less than D.

Further, for example, if the number of pixels in the current block is less than the predetermined threshold, an MPM list of one or more of the upper layer blocks having depth values ranging from D-1 to D-L may be used for the current block. Here, L may be a positive integer greater than K.

For example, if the depth value of the current block is D and D is less than or equal to the predetermined depth value T, an MPM list of one or more of the upper layer blocks having depth values ranging from D-1 to D-K may be used for the current block. K may be a positive integer less than D.

For example, if the depth value D of the current block is greater than T, an MPM list of one or more of the upper layer blocks having depth values ranging from D-1 to D-L may be used for the current block. L may be a positive integer greater than K.

The plurality of MPM lists for the current block may include MPM lists for N neighboring blocks. The N neighboring blocks may include neighboring blocks at predetermined positions. N may be 0 or a positive integer.

Information such as the number N, depth values, sizes, and/or locations of included neighboring blocks may be required to make an MPM list of neighboring blocks. Information required to make an MPM list of neighboring blocks may be entropy encoded/decoded in at least one of VPS, SPS, PPS, APS, a slice header, a parallel block header, CTU, CU, PU, and a TU. The number and/or location of neighboring blocks may be variably determined according to the size, shape, and/or location of the current block. If the depth value of the neighboring block is a predetermined value or falls within a predetermined range, an MPM list of the neighboring block may be made. The predetermined range may be defined by at least one of a minimum value or a maximum value. The information about at least one of the minimum value or the maximum value may be entropy encoded/decoded in the aforementioned predetermined unit.

For example, in fig. 14, if the current block is a block P and has a depth value D, a plurality of MPM lists for the current block may be fabricated in at least one of the following methods. Here, the plurality of MPM lists may include at least one of an MPM list made based on a current block and an MPM list made based on a neighboring block of the current block.

For example, an MPM list made based on the left neighboring block L or N of the current block P may be used for the current block P.

For example, an MPM list made based on the upper left neighboring block E of the current block P may be used for the current block P.

For example, an MPM list made based on a lower left neighboring block O of the current block P may be used for the current block P.

For example, an MPM list made based on an upper neighboring block H or I of the current block P may be used for the current block P.

For example, an MPM list made based on the upper right neighboring block J of the current block P may be used for the current block P.

For example, an MPM list of at least two neighboring blocks among an upper left neighboring block, an upper neighboring block, a right neighboring block, a left neighboring block, and a lower left neighboring block of the current block P may be used for the current block P.

For example, an MPM list of at least three neighboring blocks among an upper left neighboring block, an upper neighboring block, a right neighboring block, a left neighboring block, and a lower left neighboring block of the current block P may be used for the current block P.

For example, an MPM list of at least four neighboring blocks among an upper left neighboring block, an upper neighboring block, a right neighboring block, a left neighboring block, and a lower left neighboring block of the current block P may be used for the current block P.

For example, an MPM list of at least five neighboring blocks among an upper left neighboring block, an upper neighboring block, a right neighboring block, a left neighboring block, and a lower left neighboring block of the current block P may be used for the current block P.

An intra prediction mode derived based on at least one of the current block, the upper layer block, and the neighboring block may be included in one MPM list for the current block. That is, if not a plurality of MPM lists are used for the current block but a single MPM list is used for the current block, the MPM list may be composed of at least one of intra prediction modes derived based on at least one of a plurality of intra prediction modes based on at least one of the current block, an upper layer block, and a neighboring block.

If the N MPM lists for the current block include an MPM list for at least one of the upper layer block and the neighboring block, an order in which the N MPM lists are created may be determined. Here, N may be 0 or a positive integer.

The order of creating the MPM list may be preset in the encoder and the decoder. Alternatively, the order of making the MPM list may be determined based on the encoding parameters of each respective block. Alternatively, the order of making the MPM list may be determined based on the encoding parameters of the current block. Alternatively, information on the order in which the MPM list is made may be entropy encoded/decoded.

For example, in fig. 14, the current block may be a block H, and the MPM LIST for the current block may be referred to as mpm_list_cur.

The MPM LISTs for blocks X, W, V and U, which are made based on the upper layer block of the current block, may be referred to as mpm_list_ X, MPM _list_ W, MPM _list_v and mpm_list_u, respectively.

In addition, MPM LISTs for blocks L, E and G, which are made based on neighboring blocks of the current block, may be referred to as mpm_list_ L, MPM _list_e and mpm_list_g, respectively.

The order of making the N MPM lists for the current block may be determined in at least one of the methods described later. The number of upper layer blocks and/or neighboring blocks used in the method described later is only an example, and thus a different number of upper layer blocks and/or neighboring blocks may also be used.

For example, a plurality of MPM LISTs may be made for the current block H in the order mpm_list_cur- > mpm_list_x- > mpm_list_w- > mpm_list_v- > mpm_list_u.

For example, a plurality of MPM LISTs may be made for the current block H in the order mpm_list_u- > mpm_list_v- > mpm_list_w- > mpm_list_x- > mpm_list_cur.

For example, when mpm_list_cur is used as the first MPM LIST, a plurality of MPM LISTs may be made for the current block H using MPM LISTs for at least K upper-layer blocks arranged in ascending or descending order of depth values. Here, K may be 0 or a positive integer.

For example, a plurality of MPM LISTs may be made for the current block H in the order mpm_list_cur- > mpm_list_l- > mpm_list_g- > mpm_list_e.

For example, a plurality of MPM LISTs may be made for the current block H in the order mpm_list_l- > mpm_list_g- > mpm_list_e- > mpm_list_cur.

For example, when mpm_list_cur is used as the first MPM LIST, a plurality of MPM LISTs may be made for the current block H using MPM LISTs for at least one of the upper left neighboring block, the lower left neighboring block, the upper neighboring block, and the upper right neighboring block in a predetermined order.

For example, a plurality of MPM LISTs may be made for the current block H in the order mpm_list_cur- > mpm_list_x- > mpm_list_l.

For example, a plurality of MPM LISTs may be made for the current block H in the order mpm_list_cur- > mpm_list_l- > mpm_list_x.

For example, a plurality of MPM lists may be made for the current block H in the following order: mpm_list_cur- > MPM LIST for K upper layer blocks in a predetermined order- > MPM LIST for L neighboring blocks in a predetermined order. Here, each of K and L may be 0 or a positive integer.

For example, a plurality of MPM lists may be made for the current block H in the following order: mpm_list_cur- > MPM LIST for L neighboring blocks in a predetermined order- > MPM LIST for K upper layer blocks in a predetermined order. Here, each of K and L may be 0 or a positive integer.

For example, a plurality of MPM lists may be made for the current block H in the following order: mpm_list_cur- > MPM LIST for at least K blocks in an upper layer block or neighboring blocks in a predetermined order. Here, K may be a positive integer.

For example, a plurality of MPM LISTs may be made for the current block H using mpm_list_cur and MPM LISTs for at least K blocks of an upper layer block or neighboring blocks in a predetermined order. Here, K may be a positive integer.

The latter MPM list may not include the intra prediction mode in the former MPM list.

In addition, the variable length code for the indicator of the preceding MPM list may be shorter than the variable length code for the indicator of the following MPM list.

The preceding MPM list may include a fewer number of candidates than the following MPM list.

The indicators may be assigned to the MPM list in the order in which the MPM list was made.

The N MPM lists for the current block may include MPM lists for at least one of the upper layer block and the neighboring block according to at least one of a plurality of orders in which the MPM lists are made. Here, the plurality of MPM lists may be made to include the same intra prediction mode without redundancy. Here, N may be 0 or a positive integer.

The N MPM LISTs for the current block may be represented by mpm_list_1, mpm_list_2, … mpm_list_n. At least one of mpm_list_cur, mpm_list_ X, MPM _list_ W, MPM _list_ V, MPM _list_ U, MPM _list_ L, MPM _list_e, and mpm_list_g may correspond to at least one of mpm_list_1, mpm_list_2, … mpm_list_n.

The number of intra prediction modes that can be included in each MPM list may be expressed by C1, C2,..cn. N, C1, C2..cn may each be 0 or a positive integer. Some or all of C1 to CN may be equal or different. In addition, at least one of C1, C2, … CN may be predetermined in the encoder and decoder. In addition, at least one of C1, C2, … CN may be determined based on the encoding parameters of each respective block. In addition, at least one of C1, C2, … CN may be entropy encoded/decoded.

For example, the intra prediction mode included in mpm_list_1 may be represented by mpm_list_1_mode_1, mpm_list_1_mode_2, and.

In order to prevent the same intra prediction mode from being redundantly included in the MPM list, the intra prediction mode in the following MPM list may be used to check redundancy of the intra prediction mode included in the preceding MPM list. When there is a redundant intra prediction mode after the redundancy check, the intra prediction mode may be removed from the MPM list. After the redundancy pattern is removed, a predetermined intra prediction mode may be added to the MPM list.

The redundancy check of the patterns included in the MPM list may be performed in the step of manufacturing the plurality of MPM lists. Alternatively, the redundancy check may be performed after all of the plurality of MPM lists are fabricated. Alternatively, the redundancy check may be performed every time the intra prediction mode is included in the MPM list.

For example, if mpm_list_1 for the current block includes C1 intra prediction modes, the intra prediction modes in mpm_list_1 may be non-redundant mpm_list_1_mode_1, mpm_list_1_mode_2, mpm_list_1_mode_c1.

For example, if mpm_list_2 includes C2 non-redundant intra prediction modes, it may be checked whether each of the intra prediction modes in mpm_list_2 is identical to at least one of the intra prediction modes in mpm_list_1.

If the intra prediction mode mpm_list_2_mode_x included in mpm_list_2 is the same as the mode included in mpm_list_1, the redundant intra prediction mode mpm_list_2_mode_x may be removed from mpm_list_2. Here, the mpm_list_2_mode_x may be at least one of mpm_list_2_mode_1, mpm_list_2_mode_2, …, mpm_list_2_mode_c2.

For example, if at least one intra prediction mode is removed from mpm_list_2, at least one of the predetermined intra prediction modes may be included in mpm_list_2. The at least one intra prediction mode included in mpm_list_2 among the predetermined intra prediction modes may not be redundant with at least one of the intra prediction modes included in mpm_list_1. Alternatively, the at least one intra prediction mode included in mpm_list_2 among the predetermined intra prediction modes may not be redundant with any one of the intra prediction modes included in mpm_list_1.

The predetermined intra prediction modes added to replace redundant prediction modes may include at least one of intra prediction modes including, for example: intra_planar, intra_dc, horizontal mode, vertical mode, 45 degree mode, 135 degree mode, 225 degree mode, mpm_list_2_mode_x±delta, intra_dm, and intra_lm. Intra_dm may represent such INTRA prediction modes: in the intra prediction mode, the intra chroma prediction mode is determined to be the same as the intra luma prediction mode. In addition, intra_lm may represent such INTRA prediction modes: in the intra prediction mode, at least one of a chroma prediction block/a chroma residual block/a chroma reconstruction block may be generated based on at least one of a luma prediction block/a luma residual block/a luma reconstruction block. In addition, delta (delta) may be a positive integer.

When delta continuously increases by 1 from 1 each time until the number of intra prediction modes included in mpm_list_2 is C2, a predetermined intra prediction mode (mpm_list_2_mode_x±delta) may be included in mpm_list_2. Alternatively, the predetermined intra prediction modes may be arranged in a predetermined order, and at least one of the predetermined intra prediction modes may be included in mpm_list_2 in the order until the number of intra prediction modes included in mpm_list_2 is C2.

For example, if mpm_list_3 for the current block includes C3 non-redundant intra prediction modes, it may be checked whether each intra prediction mode included in mpm_list_3 is equal to at least one of the intra prediction modes included in mpm_list_1 and mpm_list_2.

If the intra prediction mode mpm_list_3_mode_x included in mpm_list_3 is the same as the mode included in mpm_list_1 or mpm_list_2, mpm_list_3_mode_x may be removed from mpm_list_3. Here, the mpm_list_3_mode_x may be at least one of mpm_list_3_mode_1, mpm_list_3_mode_2, and mpm_list_3_mode_c3.

For example, if at least one intra prediction mode is removed from mpm_list_3, at least one of the predetermined intra prediction modes may be included in mpm_list_3. The at least one intra prediction mode included in mpm_list_3 among the predetermined intra prediction modes may not be redundant with at least one of the intra prediction modes included in mpm_list_1 and mpm_list_2. Alternatively, the at least one intra prediction mode included in mpm_list_3 among the predetermined intra prediction modes may not be redundant with any one of the intra prediction modes included in mpm_list_1 and mpm_list_2.

For example, when delta continuously increases by 1 from 1 each time until the number of intra prediction modes included in mpm_list_3 is C3, a predetermined intra prediction mode (mpm_list_3_mode_x±delta) may be included in mpm_list_3. Alternatively, the predetermined intra prediction modes may be arranged in a predetermined order, and at least one of the predetermined intra prediction modes may be included in mpm_list_3 in the order until the number of intra prediction modes included in mpm_list_3 is C3.

For example, if mpm_list_k for the current block includes CK non-redundant intra prediction modes, it may be checked whether each intra prediction mode included in mpm_list_k is identical to at least one of the intra prediction modes included in mpm_list_1, mpm_list_2, … mpm_list_ (K-1). K may be a positive integer equal to or less than the maximum number N of MPM lists available for the current block.

If the intra prediction MODE mpm_list_k_mode_x included in mpm_list_k is the same as the MODEs included in mpm_list_1, mpm_list_2, … mpm_list_ (K-1), the redundant intra prediction MODE mpm_list_k_mode_x may be removed from mpm_list_k. Here, the mpm_list_k_mode_x may be at least one of mpm_list_k_mode_1, mpm_list_k_mode_2, …, mpm_list_k_mode_ck.

For example, if at least one intra prediction mode is removed from mpm_list_k, at least one of the predetermined intra prediction modes may be included in mpm_list_k. The at least one intra prediction mode included in mpm_list_k among the predetermined intra prediction modes may not be redundant with at least one of the intra prediction modes included in mpm_list_1, mpm_list_2, …, and mpm_list_ (K-1). Alternatively, the at least one intra prediction mode included in mpm_list_k among the predetermined intra prediction modes may not be redundant with any one of the intra prediction modes included in mpm_list_1, mpm_list_2, …, and mpm_list_ (K-1).

For example, in place of the intra prediction mode removed from the mpm_list_k, when a specific intra prediction mode that is not redundant with at least one or any one of the intra prediction modes included in the mpm_list_1, mpm_list_2, …, and mpm_list_ (K-1) exists among the predetermined intra prediction modes, the specific intra prediction mode may be added to the mpm_list_k.

For example, when delta is continuously increased from 1 each time by 1 until the number of intra prediction MODEs included in mpm_list_k is CK, a predetermined intra prediction MODE (mpm_list_k_mode_x±delta) may be included in mpm_list_k. Alternatively, the predetermined intra prediction modes may be arranged in a predetermined order, and at least one of the predetermined intra prediction modes may be included in the mpm_list_k in the order until the number of intra prediction modes included in the mpm_list_k is CK.

If the intra prediction mode of the current block is derived using the N MPM lists or is entropy encoded/decoded, an indicator (MPM flag) indicating whether the intra prediction mode of the current block is included in the intra prediction mode in each of the N MPM lists may be entropy encoded/decoded for each of the N MPM lists.

For example, if N MPM lists are used, up to N indicators may be encoded/decoded, where each indicator is for one MPM list, such as mpm_flag_1, mpm_flag_2, mpm_flag_n. Alternatively, a maximum of (N-1) indicators may be encoded/decoded, in which case an unencoded/decoded indicator for another MPM list may be derived based on some or all of the (N-1) indicators. For example, the indicator may not be encoded/decoded for any one of the N MPM lists (e.g., the last MPM list counted in a predetermined order). When there is an intra prediction mode of the current block among intra prediction modes of a specific MPM list, the indicator may have a first value for the specific MPM list. The indicator may have a second value when the intra prediction mode of the current block does not exist. Here, the first value may be 1 and the second value may be 0. That is, the indicator may be flag information.

In addition, if the indicator for a specific MPM list has a first value among the N indicators, the indicators for other MPM lists other than the indicator for the specific MPM list may have a second value.

In addition, if the indicator for the kth MPM list among the N indicators has the first value, the indicators for the (k+1) th MPM list to the nth MPM list may not be entropy encoded/decoded. Here, K may be a positive integer equal to or greater than 1 and equal to or less than N.

When there is the same intra prediction mode as the intra prediction mode of the current block among intra prediction modes included in a specific MPM list among the N MPM lists, index information (MPM index) indicating a position or number of the same intra prediction mode as the intra prediction mode of the current block in the specific MPM list may be entropy encoded. In addition, the same intra prediction mode as the intra prediction mode of the current block among the intra prediction modes included in the specific MPM list may be identified by entropy decoding the index information. The index information may be entropy-encoded into a fixed length code or a variable length code. In addition, the index information may be used to derive an intra prediction mode of the current block.

When there is no intra prediction mode identical to the intra prediction mode of the current block among the intra prediction modes included in the N MPM lists, the remaining intra prediction modes of the current block may be entropy encoded in the encoder. The remaining intra prediction modes may be used to identify intra prediction modes of the current block that are not included in at least one of the MPM lists. Alternatively, the remaining intra prediction modes may be used to identify intra prediction modes of the current block that are not included in any one of the MPM lists.

If the total number of intra prediction modes is Y and the sum of the numbers of intra prediction modes included in the N MPM lists for the current block is X, remaining intra prediction modes indicating the same intra prediction mode as the intra prediction mode of the current block among (Y-X) intra prediction modes may be entropy encoded. Here, all X intra prediction modes included in the N MPM lists may be arranged based on at least one of a size, an angle, an order, and an identification number of the intra prediction modes. The alignment may be performed in ascending or descending order. The aligned X intra prediction modes may be compared with the intra prediction mode of the current block. The predetermined value may be subtracted from the intra prediction mode of the current block if the comparison indicates that the intra prediction mode of the current block is greater than the aligned X intra prediction modes. The predetermined value may be 1.

Alternatively, for example, the intra prediction mode of the current block may be compared with a mode having the largest reference value (e.g., in terms of at least one of size, angle, order, and identification number) among the aligned X intra prediction modes. The predetermined value may be subtracted from the intra prediction mode of the current block if the comparison indicates that the intra prediction mode of the current block is greater than the mode having the maximum reference value.

In addition, for example, the subtracted intra prediction mode of the current block may be compared with a mode having a second largest reference value among the aligned X intra prediction modes. The predetermined value may be further subtracted from the subtracted intra prediction mode of the current block if the comparison indicates that the subtracted intra prediction mode is greater than the mode having the second largest reference value.

The subtracting operation based on the comparison may be repeated until a mode having the smallest reference value among the aligned X intra prediction modes. The final subtracted intra prediction mode of the current block may be entropy encoded as the remaining intra prediction mode.

The remaining intra prediction modes of the current block may be entropy decoded and used to identify the same intra prediction modes as the intra prediction modes of the current block among the intra prediction modes not included in the N MPM lists. If the total number of intra prediction modes is Y and the sum of the numbers of intra prediction modes included in the N MPM lists for the current block is X, the remaining intra prediction modes indicating the same intra prediction mode as the intra prediction mode of the current block among the (Y-X) intra prediction modes may be entropy decoded.

After the remaining intra prediction modes are entropy decoded, the X intra prediction modes may be arranged based on at least one of a size, an angle, an order, and an identification number of the intra prediction modes. The alignment may be performed in ascending or descending order. The remaining intra prediction modes that are entropy decoded may be compared with values of the X intra prediction modes. If the comparison indicates that the remaining intra prediction modes that are entropy-decoded are equal to or greater than the values of the X intra prediction modes, the values of the remaining intra prediction modes that are entropy-decoded may be increased by a predetermined value. The predetermined value may be 1.

Alternatively, for example, a mode having a minimum reference value (e.g., in terms of at least one of size, angle, order, and identification number of intra prediction modes) among the aligned X intra prediction modes may be compared with the remaining intra prediction modes that are entropy decoded. The predetermined value may be added to the value of the entropy-decoded residual intra prediction mode if the comparison indicates that the entropy-decoded residual intra prediction mode is equal to or greater than the value of the mode having the smallest reference value.

In addition, a mode having a second smallest reference value among the arranged X intra prediction modes may be compared with an added value of the remaining intra prediction modes that are entropy-decoded. The predetermined value may be further added to the added value of the entropy-decoded residual intra prediction mode if the comparison indicates that the added value of the entropy-decoded residual intra prediction mode is equal to or greater than the value of the mode having the second smallest reference value.

The comparison-based addition operation may be repeated until a mode having the largest reference value among the aligned X intra prediction modes. The final added residual intra prediction mode may be entropy decoded as the intra prediction mode of the current block.

The intra prediction modes of the current block for which the N MPM lists are used may be entropy encoded/decoded in the following manner of the embodiment.

An embodiment for the case where one MPM list is used for the current block will be described below.

When the same intra prediction mode as the intra prediction mode of the current block exists among the intra prediction modes included in mpm_list_1, the encoder may entropy encode an indicator mpm_flag_1 indicating whether the same intra prediction mode as the intra prediction mode of the current block exists in mpm_list_1 as a first value. If mpm_flag_1 is the first value, index information mpm_idx_1 regarding mpm_list_1 may be further entropy encoded.

When the intra prediction mode identical to the intra prediction mode of the current block does not exist among the intra prediction modes included in the mpm_list_1, the encoder may entropy encode an indicator mpm_flag_1 indicating whether the intra prediction mode identical to the intra prediction mode of the current block exists in the mpm_list_1 to a second value. If MPM_FLAG_1 is the second value, the residual intra prediction MODE REM_MODE may be further entropy encoded.

The decoder may entropy-decode an indicator mpm_flag_1 indicating whether there is the same intra prediction mode as that of the current block in mpm_list_1. If mpm_flag_1 is a first value, the decoder may derive an intra prediction mode of the current block by further entropy decoding index information mpm_idx_1 regarding mpm_list_1.

If the MPM_FLAG_1 is the second value, the decoder may derive the intra prediction MODE of the current block by further entropy decoding the remaining intra prediction MODEs REM_MODE.

An embodiment for the case where two MPM lists are used for the current block will be described below.

When there is the same intra prediction mode as the intra prediction mode of the current block among the intra prediction modes included in the mpm_list_1 and mpm_list_2, the encoder may entropy encode the indicators mpm_flag_1 and mpm_flag_2 indicating which LIST among the mpm_list_1 and mpm_list_2 includes the same intra prediction mode as the intra prediction mode of the current block. If there is an intra prediction mode of the current block in mpm_list_1, mpm_flag_1 may be a first value and mpm_flag_2 may be a second value. In this case, index information mpm_idx_1 regarding mpm_list_1 may be further entropy-encoded. Alternatively, if there is an intra prediction mode of the current block in mpm_list_2, mpm_flag_1 may be a second value and mpm_flag_2 may be a first value. In this case, index information mpm_idx_2 regarding mpm_list_2 may be further entropy-encoded.

When there is no intra prediction mode identical to the intra prediction mode of the current block among the intra prediction modes included in the mpm_list_1 and mpm_list_2, the encoder may entropy encode indicators mpm_flag_1 and mpm_flag_2 indicating which LIST of the mpm_list_1 and mpms_list_2 includes the intra prediction mode identical to the intra prediction mode of the current block to a second value. If mpm_flag_1 and mpm_flag_2 are the second values, the residual intra prediction MODE rem_mode may be further entropy encoded.

The decoder may entropy-decode indicators mpm_flag_1 and mpm_flag_2 indicating which LIST of mpm_list_1 and mpm_list_2 includes the same intra prediction mode as that of the current block. If mpm_flag_1 is a first value and mpm_flag_2 is a second value, an intra prediction mode of the current block exists in mpm_list_1, and the decoder may derive the intra prediction mode of the current block by further entropy-decoding index information mpm_idx_1 regarding mpm_list_1. Alternatively, if mpm_flag_1 is the second value and mpm_flag_2 is the first value, an intra prediction mode of the current block exists in mpm_list_2, and the decoder may derive the intra prediction mode of the current block by further entropy-decoding index information mpm_idx_2 about mpm_list_2. Alternatively, if mpm_flag_1 and mpm_flag_2 are the second values, the decoder may derive the intra prediction MODE of the current block by entropy decoding the remaining intra prediction MODEs rem_mode. Here, the case where both mpm_flag_1 and mpm_flag_2 are the first value does not occur.

Hereinafter, another embodiment for the case where two MPM lists are used for the current block will be described.

TABLE 5

As in the example of [ table 5], the encoder may entropy encode the intra prediction mode of the current block by sequentially checking whether there is the same intra prediction mode as the intra prediction mode of the current block among the intra prediction modes included in each of mpm_list_1 and mpm_list_2 according to at least one of a plurality of orders in which a plurality of MPM LISTs are made.

For example, if there is the same intra prediction mode as that of the current block in mpm_list_1, mpm_flag_1 may be entropy encoded to a first value. Here, the index information mpm_idx_1 regarding mpm_list_1 may be further entropy-encoded.

For example, when the same intra prediction mode as that of the current block does not exist in mpm_list_1, mpm_flag_1 may be entropy encoded to a second value. Here, when the same intra prediction mode as that of the current block exists in mpm_list_2 (in the case where mpm_flag_1 is a second value), mpm_flag_2 may be entropy-encoded to the first value. Here, the index information mpm_idx_2 regarding mpm_list_2 may be further entropy-encoded.

For example, when the same intra prediction mode as the intra prediction mode of the current block does not exist in mpm_list_1 and mpm_list_2, mpm_flag_1 and mpm_flag_2 may be entropy encoded to a second value. Here (if mpm_flag_1 and mpm_flag_2 are second values), the residual intra prediction MODE rem_mode may be further entropy encoded.

As in the example of [ table 5], the decoder may sequentially entropy-decode the indicators mpm_flag_1 and/or mpm_flag_2 indicating which LIST among mpm_list_1 and mpm_list_2 includes the same intra-prediction mode as that of the current block in at least one of a plurality of orders in which a plurality of MPM LISTs are made.

For example, if mpm_flag_1 is entropy-decoded to a first value, an intra prediction mode of the current block exists in mpm_list_1, and the intra prediction mode of the current block may be derived by further entropy-decoding index information mpm_idx_1 regarding mpm_list_1.

For example, if mpm_flag_1 is entropy-decoded to a second value, mpm_flag_2 may be entropy-decoded. If the decoded mpm_flag_2 is the first value, an intra prediction mode of the current block exists in mpm_list_2, and the intra prediction mode of the current block may be derived by further entropy-decoding index information mpm_idx_2 about mpm_list_2.

For example, if mpm_flag_1 and mpm_flag_2 are entropy-decoded to a second value, the intra prediction MODE of the current block may be derived by further entropy-decoding the remaining intra prediction MODEs rem_mode. Here, the steps of determining whether the mpm_flag_1 and the mpm_flag_2 are the first value or the second value may be sequentially performed.

Now, an embodiment will be described for the case where N MPM lists are used for the current block.

When the same intra prediction mode as the intra prediction mode of the current block exists among the intra prediction modes included in mpm_list_1, mpm_list_2, …, and mpm_list_n, the encoder may entropy encode indicators mpm_flag_1, mpm_flag_2, …, and mpm_flag_n indicating which LIST of mpm_list_1, mpm_list_2, …, and mpm_list_n includes the same intra prediction mode as the intra prediction mode of the current block. If there is an intra prediction mode of the current block in mpm_list_1, mpm_flag_1 may be a first value, and mpm_flag_2, …, and mpm_flag_n other than mpm_flag_1 may be a second value. In this case, the index information mpm_idx_1 regarding mpm_list_1 may be further entropy-encoded.

Alternatively, if there is an intra prediction mode of the current block in mpm_list_2, mpm_flag_2 may be a first value, and mpm_flag_1, …, and mpm_flag_n other than mpm_flag_2 may be a second value. In this case, the index information mpm_idx_2 regarding mpm_list_2 may be further entropy-encoded.

Alternatively, if there is an intra prediction mode of the current block in mpm_list_n, mpm_flag_n may be a first value, and mpm_flag_1, …, and mpm_flag_ (N-1) other than mpm_flag_n may be a second value. In this case, the index information mpm_idx_n regarding mpm_list_n may be further entropy-encoded.

When there is no intra prediction mode identical to the intra prediction mode of the current block among the intra prediction modes included in mpm_list_1, mpm_list_2, …, and mpm_list_n, the encoder entropy encodes mpm_flag_1, mpm_flag_2, …, and mpm_flag_n indicating which LIST of mpm_list_1, mpm_list_2, …, and mpm_list_n includes the intra prediction mode identical to the intra prediction mode of the current block as a second value. If mpm_flag_1, mpm_flag_2, …, and mpm_flag_n are second values, the residual intra prediction MODE rem_mode may be further entropy encoded.

The decoder may entropy-decode indicators mpm_flag_1, mpm_flag_2, …, and mpm_flag_n indicating which LIST of mpm_list_1, mpm_list_2, …, and mpm_list_n includes the same intra prediction mode as that of the current block. If mpm_flag_1 is a first value and mpm_flag_2, …, and mpm_flag_n other than mpm_flag_1 are second values, an intra prediction mode of the current block may exist in mpm_list_1. In this case, the intra prediction mode of the current block may be derived by further entropy-decoding index information mpm_idx_1 regarding mpm_list_1.

Alternatively, if mpm_flag_2 is a first value and mpm_flag_1, …, and mpm_flag_n other than mpm_flag_2 are second values, the intra prediction mode of the current block may exist in mpm_list_2. In this case, the intra prediction mode of the current block may be derived by further entropy-decoding index information mpm_idx_2 regarding mpm_list_2.

Alternatively, if mpm_flag_n is a first value and mpm_flag_1, …, and mpm_flag_ (N-1) other than mpm_flag_n are second values, an intra prediction mode of the current block may exist in mpm_list_n. In this case, the intra prediction mode of the current block may be derived by further entropy-decoding index information mpm_idx_n regarding mpm_list_n.

Alternatively, if mpm_flag_1, mpm_flag_2, …, and mpm_flag_n are second values, the intra prediction MODE of the current block may be derived by further entropy decoding the remaining intra prediction MODEs rem_mode. Here, the case where all of mpm_flag_1, mpm_flag_2, …, and mpm_flag_n are the first values does not occur.

Now, another embodiment for the case where N MPM lists are used for the current block will be described.

TABLE 6

As in the example of [ table 6], the encoder may entropy encode the intra prediction mode of the current block by sequentially checking whether the same intra prediction mode as the intra prediction mode of the current block exists among the intra prediction modes included in each of mpm_list_1, mpm_list_2, …, mpm_list_n according to at least one of a plurality of orders in which a plurality of MPM LISTs are made.

For example, when the same intra prediction mode as the intra prediction mode of the current block does not exist in mpm_list_1, mpm_list_2, …, mpm_list_ (N-1), mpm_flag_1, mpm_flag_2, …, mpm_flag_ (N-1) may be entropy encoded to a second value. Here, if there is the same intra prediction mode as that of the current block in mpm_list_n (if mpm_flag_1, mpm_flag_2, …, mpm_flag_ (N-1) are second values), index information mpm_idx_n about mpm_list_n may be further entropy-encoded.

For example, when the same intra prediction mode as the intra prediction mode of the current block does not exist in mpm_list_1, mpm_list_2, …, mpm_list_n, mpm_flag_1, mpm_flag_2, …, mpm_flag_n may be entropy encoded to a second value. Here, the residual intra prediction MODE rem_mode may be further entropy encoded.

As in the example of [ table 6], the decoder may sequentially entropy-decode at least one of the indicators mpm_flag_1, mpm_flag_2, …, mpm_flag_n indicating which LIST among mpm_list_1, mpm_list_2, …, mpm_list_n includes the same intra-prediction mode as that of the current block in at least one of the plurality of orders in which the plurality of MPM LISTs are made.

For example, if mpm_flag_1, mpm_flag_2, …, mpm_flag_ (N-1) are entropy decoded to a second value, mpm_flag_n may be entropy decoded. If the decoded mpm_flag_n is the first value, an intra prediction mode of the current block exists in the mpm_list_n, and the intra prediction mode of the current block may be derived by further entropy-decoding index information mpm_idx_n about the mpm_list_n.

For example, if mpm_flag_1, mpm_flag_2, …, mpm_flag_n are entropy-decoded to a second value, the intra prediction MODE of the current block may be derived by further entropy-decoding the remaining intra prediction MODEs rem_mode. Here, the step of determining whether mpm_flag_1, mpm_flag_2, …, mpm_flag_n is the first value or the second value may be sequentially performed.

A FLAG (such as mpm_flag_n) designating the MPM list or an index indicating one of the plurality of MPM lists may be encoded/decoded. Information indicating whether an MPM-based intra prediction derivation method is to be used for a current block (or a current slice, a current picture, a current sequence, etc.) may be encoded/decoded. The index may be encoded/decoded if the information indicates that an MPM-based intra prediction derivation method is used. At least one of the number or type of the plurality of MPM lists may be fixedly predefined in the encoder/decoder, or may be variably determined based on parameters related to the size, depth, shape, position, etc. of the current/neighboring block. For example, the number of MPM lists predefined in the encoder/decoder may be 1, 2, 3 or more. The maximum number of intra prediction modes included in each MPM list may be forced to be equal. The maximum number may be fixedly preset in the encoder/decoder or may be signaled in a predetermined unit (e.g., sequence, picture, slice, block, etc.). If the number of intra prediction modes included in a specific MPM list is less than the maximum number, a predetermined mode may be added to the specific MPM list. The added mode may be a preset default mode or an intra prediction mode included in another MPM list. Notably, modes other than intra prediction modes included in the particular MPM list may be added. Redundancy checks between MPM lists may be omitted. One of the MPM lists may share at least one identical intra prediction mode with another MPM list.

The encoder may make a plurality of MPM LISTs ranging from mpm_list_1 to mpm_list_n in at least one of a plurality of orders in which the plurality of MPM LISTs are made. The total number of candidate intra prediction modes in the N MPM lists may be K. N and K may be positive integers.

For example, mpm_list_combined may be composed of K or fewer candidate intra prediction modes among the candidate intra prediction modes of the N MPM LISTs.

For example, if the same intra prediction mode as the intra prediction mode of the current block exists in mpm_list_combined, an indicator mpm_flag_combined indicating whether the same intra prediction mode as the intra prediction mode of the current block exists in mpm_list_combined may be entropy-encoded as a first value. Here, the index information mpm_idx_combined about mpm_list_combined may be additionally entropy-encoded.

When the same intra prediction mode as the intra prediction mode of the current block does not exist in the mpm_list_combined, the mpm_flag_combined may be entropy encoded to a second value. If the mpm_flag_combined is the second value, the residual intra prediction MODE rem_mode may be additionally entropy encoded.

The decoder may entropy-decode an indicator mpm_flag_combined indicating whether there is an intra prediction mode identical to the intra prediction mode of the current block in the mpm_list_combined. If mpm_flag_combined is the first value, the intra prediction mode of the current block may be derived by further entropy decoding the index information mpm_idx_combined. If the mpm_flag_combined is the second value, the intra prediction MODE of the current block may be derived by additionally entropy decoding the remaining intra prediction MODE rem_mode.

According to the present invention, an intra prediction mode of a current block can be derived by performing encoding/decoding. Here, the intra prediction mode of the current block may be entropy encoded/decoded without using the intra prediction modes of neighboring blocks.

According to the present invention, intra prediction modes of different color components may be used to derive intra prediction modes of a current block. For example, if the current block is a chroma block, the intra prediction mode of one or more corresponding luma blocks corresponding to the target chroma block may be used to derive the intra prediction mode of the current block. The respective luma block may be determined based on at least one of a size, a shape, or an encoding parameter of the chroma block. Alternatively, the respective luminance block may be determined based on at least one of a size, shape, or encoding parameter of the luminance block.

In the example shown in fig. 15, the respective luminance blocks corresponding to the chrominance blocks may be at least one of block a, block B, block C, and block D.

For example, an intra prediction mode of a luminance block a corresponding to a sample position (0, 0) of a chrominance block may be derived as an intra prediction mode of the chrominance block.

For example, an inter prediction mode of the luminance block D corresponding to a sample position (nS/2 ) at the center of the chrominance block may be derived as an intra prediction mode of the chrominance block.

For example, the intra prediction mode of the chroma block may be derived using a combination of intra prediction modes of one or more blocks of a plurality of blocks within a luma block corresponding to the size of the chroma block. For example, a mode corresponding to an average value of an intra prediction mode of the luminance block a corresponding to the sample point position (0, 0) and an intra prediction mode of the luminance block D corresponding to the sample point position (nS/2 ) may be derived as an intra prediction mode of the chrominance block. Alternatively, a mode corresponding to an average value of intra prediction modes of block a, block B, block C, and block D within a luminance block corresponding to a size of a chroma block may be derived as an intra prediction mode of the chroma block. One or more of a variety of statistical values including maximum, minimum, most frequent, median, and weighted average may be used in place of the average.

For example, the intra prediction mode of the chroma block may be derived based on at least one of a size, a shape, or a depth of the luma block. For example, the size of the luminance block a corresponding to the sample position (0, 0) of the chroma block and the size of the luminance block D corresponding to the sample position (nS/2 ) of the chroma block may be compared, and the intra prediction mode of the luminance block D having a relatively large size may be derived as the intra prediction mode of the chroma block. Alternatively, if the size of the luminance block corresponding to the predetermined position of the chrominance block is equal to or greater than the size of the chrominance block, the intra prediction mode of the chrominance block may be derived using the intra prediction mode of the luminance block.

For example, the intra prediction mode of the chroma block may be derived based on at least one of a size, a shape, or a depth of the chroma block. For example, if the size of a chroma block falls within a predetermined range, an intra prediction mode of a luma block corresponding to a sample position (0, 0) of the chroma block may be derived as the intra prediction mode of the chroma block. Alternatively, if the size of the chroma block falls within a predetermined range, the size of the luma block corresponding to the sample position (0, 0) of the chroma block and the size of the luma block corresponding to the sample position (nS/2 ) of the chroma block may be compared, and the intra prediction mode of the luma block having a larger size may be derived as the intra prediction mode of the chroma block. Alternatively, if the chroma block is shaped as a rectangle, the intra prediction mode of the luma block corresponding to the center position of the chroma block may be derived as the intra prediction mode of the chroma block.

For example, if the current block is divided into lower layer blocks or sub-blocks, the intra prediction mode of each sub-block may be derived using at least one of the above-described methods for deriving the intra prediction mode of the current block.

The size of the current block and the size of the sub-block may be mxn. M and N may be the same positive integer or different positive integers. For example, the size of the current block or sub-block may be at least one of CTU, CU, SU (signaling unit), QTMax, QTMin, BTMax, BTMin, 4×4, 8×8, 16×16, 32×32, 64×64, 128×128, 256×256, 4×8, 8×16, 16×8, 32×64, 32×8, and 4×32. Here, QTMax and QTMin may represent a maximum size and a minimum size allowed to be divided into a quadtree, respectively, and BTMax and BTMin may represent a maximum size and a minimum size allowed to be divided into a binary tree, respectively. Hereinafter, the size of the sub-block may represent a division structure of the sub-block.

The sub-block size may vary according to the size of the current block. For example, 1/N of the vertical size and the horizontal size of the current block may be a sub-block size. N may be a positive integer and may be at least one of 2, 4, 8, 16, 32, and 64. For example, if the size of the current block is 32×32 and N in 1/N of the horizontal and vertical sizes of the current block is 4, the sub-block size may be 8×8.

Alternatively, the sub-block size may be a predetermined fixed size regardless of the size of the current block. For example, the sub-block size may be a minimum size regardless of the size of the current block, and the sub-block size may be, for example, 4×4.

Alternatively, the sub-block size may be determined based on the partition structure of neighboring blocks of the current block. For example, if a nearby neighboring block is divided, the sub-block size may be determined by dividing the current block.

The sub-block size may be determined based on intra prediction modes of neighboring blocks of the current block. For example, the sub-block size may be determined by dividing a block into sub-blocks based on a boundary according to which intra prediction modes of neighboring blocks become different.

The sub-block size may be determined based on coding parameters of neighboring blocks. For example, the sub-block size may be determined by dividing a block into sub-blocks based on whether neighboring blocks are intra-coded blocks or inter-coded blocks.

At least one of the current block size, the sub-block size, and N according to which the current block is equally divided by N may be fixed to a predetermined value.

For example, in case that the predetermined fixed size for the current block is 16×16, if the size of the current block is 16×16, the current block may be divided into sub-blocks, and the intra prediction mode for each sub-block may be derived.

For example, in case that the predetermined fixed size for the current block is CTU and N is 4, if the size of the current block is CTU, the intra prediction mode may be derived based on sub-blocks, each of which is generated by dividing the height and width of the CTU by 4.

One or more sub-blocks may be further divided into smaller blocks. For example, if the current block size is 32×32 and the sub-block size is 16×16, each of the one or more sub-blocks may be divided into smaller blocks of sizes 8×8, 4×4, 16×8, 4×16, etc., respectively.

At least one of the current block size, the sub-block size, and N according to which the current block is equally divided by N may be encoded/decoded.

The partition structure for the sub-blocks of the current block may be encoded/decoded. The size and/or shape of the sub-blocks into which the current block is divided may vary. Furthermore, an intra prediction mode may be derived for each sub-block.

An indicator (e.g., a flag) indicating that the intra prediction mode of the current block is derived using the intra prediction modes of neighboring blocks may be encoded/decoded. For example, the indicator may be ndip_flag (neighbor mode dependent intra prediction). The indicator may be encoded/decoded for at least one of the current block or each sub-block. The indicator may be encoded/decoded only when the current block size or sub-block size corresponds to a predetermined size or a predetermined size range. The predetermined size may be, for example, 64×64 or BTMax. As described above, the current block may be divided into a plurality of sub-blocks. The partition structure of the sub-blocks may be predefined or encoded/decoded.

If ndip_flag is 1 for the current block, intra prediction modes of neighboring blocks may be used to derive intra prediction modes of the current block or intra prediction modes of each sub-block of the current block. In this case, at least one of prev_intra_luma_pred_flag, mpm_idx, rem_intra_luma_pred_mode, intra_chroma_pred_mode, split_flag, qb_ flag, quadtree _ flag, binarytree _flag, and btype_flag for the current block and/or sub-block may not be encoded/decoded.

For example, if ndip_flag is 1 for a current block, an intra prediction mode of the current block may be decoded, and then the intra prediction mode of each sub-block may be derived using the decoded intra prediction mode and intra prediction modes of neighboring blocks. In this case, at least one of prev_intra_luma_pred_flag, mpm_idx, rem_intra_luma_pred_mode, intra_chroma_pred_mode, split_flag, qb_ flag, quadtree _ flag, binarytree _flag, and btype_flag for the sub-block may not be encoded/decoded.

If ndip_flag is 0 for the current block, information related to at least one of intra prediction mode of the current block or sub-block or partition information of the sub-block may be encoded/decoded.

Among the plurality of sub-blocks of the current block, the intra prediction mode of the first sub-block may be derived in a different manner from other sub-blocks. The first sub-block may be one of a plurality of sub-blocks in the current block. For example, the first sub-block may be the first sub-block in a zig-zag scanning order.

The intra prediction mode of the first sub-block may refer to an initial mode. For example, if the intra prediction mode of each sub-block is derived as an average value of intra prediction modes of blocks left and above the each sub-block, the initial mode may be derived in a different method. The different method for deriving the initial mode may be at least one method of a plurality of methods for deriving an intra prediction mode according to the present invention.

For example, an nth (e.g., first) pattern listed in the MPM list may be derived as the initial pattern. Alternatively, the most frequent one of the intra prediction modes of one or more neighboring blocks of the current block may be derived as the initial mode. Alternatively, an intra prediction mode for current block encoding/decoding may be derived as the initial mode. Alternatively, an intra prediction mode for the first sub-block encoding/decoding may be derived as the initial mode.

Regarding the derivation of intra prediction modes for sub-blocks in a current block, intra prediction modes for one or more sub-blocks may be derived in any order. The arbitrary order may be a scanning order, and the scanning may correspond to at least one of raster scanning, upper right scanning, vertical scanning, horizontal scanning, diagonal scanning, and zig-zag scanning. The number of sub-blocks from which the intra prediction mode is derived in the scan order may be 1 or more. The arbitrary order may be adaptively determined according to intra prediction modes of neighboring blocks.

First, it may be determined whether the size of the current block corresponds to a predetermined size (S1610). The predetermined size may be determined according to a horizontal length or a vertical length of the current block. For example, the determination operation may be performed depending on whether the horizontal length or the vertical length of the current block is a partitionable length for partitioning into sub-blocks at step S1610.

For example, in the case where the current block is a square having the same horizontal length and vertical length, if a length obtained by dividing each of the horizontal length and the vertical length by N is equal to or greater than an arbitrary length, the size of the current block may correspond to the predetermined size. For example, in the case where N is 4 and the arbitrary length is 4, if the size of the current block is one of 256×256, 128×128, 64×64, 32×32, and 16×16, the size of the current block may correspond to the predetermined size.

For example, in the case where the current block is a rectangle having different horizontal and vertical lengths, if a smaller one of a length obtained by dividing a larger one of the horizontal and vertical lengths by M and a length obtained by dividing a smaller one of the horizontal and vertical lengths by N is equal to or greater than an arbitrary length, the size of the current block may correspond to the predetermined size. For example, in the case where M is 4, n is 2, and the arbitrary length is 4, if the size of the current block is one of 128×64, 64×128, 128×32, 32×128, 128×16, 16×128, 128×8, 8×128, 64×32, 32×64, 64×16, 16×64, 64×8, 8×64, 32×16, 16×32, 32×8, 8×32, 16×8, and 8×16, the size of the current block may correspond to the predetermined size.

For example, if the current block is not further divided, the size of the current block may correspond to the predetermined size. For example, if the partition information, the quadtree partition information, and/or the binary tree partition information about the current block is 0 (indicating that the current block is not partitioned), and the horizontal length or the vertical length of the current block is greater than the minimum length, the size of the current block may correspond to the predetermined size. Here, the minimum length may be 4.

If the size of the current block does not correspond to the predetermined size (no in S1610), at least one of partition information of the current block, an intra prediction mode of the current block, or an intra prediction mode of a sub-block may be decoded (S1660). If the current block is not divided, an intra prediction mode of the current block may be decoded. If the current block is divided into sub-blocks, an intra prediction mode of each sub-block may be decoded.

If the size of the current block corresponds to the predetermined size (yes in S1610), the ndip_flag may be decoded (S1620). In the next step, the decoded ndip_flag value may be checked (S1630).

If ndip_flag is 0 (no in S1630), at least one of partition information regarding the current block, an intra prediction mode of the current block, and an intra prediction mode of a sub-block may be decoded (S1660).

If ndip_flag is 1 (yes in S1630), the current block may be divided into sub-blocks (S1640). Here, the partitioning may be performed such that each sub-block has a predetermined size and/or shape. Alternatively, the division may be performed based on the decoded division information.

In the next step, an intra prediction mode of a sub-block generated by dividing the current block may be derived (S1650). The intra-prediction mode of the block may be pushed based on the intra-prediction modes of neighboring blocks. Alternatively, the intra prediction mode of the current block may be decoded and used.

Intra prediction may be performed using an intra prediction mode derived for the current block or sub-block (S1670).

The order in which the intra prediction modes of the plurality of sub-blocks in the current block are derived may be based on a raster scan order of the current block. Alternatively, the order in which the intra prediction modes of the plurality of sub-blocks in the current block are derived may be a raster scan order based on a predetermined block size. For example, the intra prediction modes of the sub-blocks may be inferred in the order of C1, C2, C3, …, C16. Alternatively, the intra prediction modes of the sub-blocks may be inferred in the order of C1, C2, C5, C6, C3, C4, …, C12, C15, C16. Alternatively, the intra prediction mode of each sub-block may be derived in parallel. The intra prediction mode of each sub-block may be derived in at least one of a plurality of methods for deriving the intra prediction mode of the current block.

For example, if the current block is divided into sub-blocks and an intra prediction mode for each sub-block is derived, intra prediction modes of neighboring blocks may be used.

For example, statistics of intra prediction modes of blocks left and above a sample at a position (0, 0) of each sub-block may be derived as the intra prediction mode of said each sub-block. For example, if the statistical value is an average value, the intra prediction mode of each sub-block shown in fig. 17 may be derived by the following [ equation 4 ].

[ equation 4]

C1＝(j+d+1)>>1，

C2＝(C1+e+1)>>1，

C3＝(C2+g+1)>>1，

C4＝(C3+g+1)>>1，

C5＝(j+C1+1)>>1，

C6＝(C5+C2+1)>>1，

C7＝(C6+C3+1)>>1，

C8＝(C7+C4+1)>>1，

C9＝(k+C5+1)>>1，

C10＝(C9+C6+1)>>1，

C11＝(C10+C7+1)>>1，

C12＝(C11+C8+1)>>1，

C13＝(l+C9+1)>>1，

C14＝(C13+C10+1)>>1，

C15＝(C14+C11+1)>>1，

C16＝(C15+C12+1)>>1。

Alternatively, the sizes of blocks left and above the sample at the position (0, 0) of each sub-block may be compared with each other, and the intra prediction mode of the block having the larger size may be derived as the intra prediction mode of each sub-block. If the two blocks have the same size, the average value of the intra prediction modes of the left and upper blocks may be derived as the intra prediction mode of each sub-block.

Alternatively, the sizes of the intra prediction modes of the blocks at the left and upper sides of the sample point at the position (0, 0) of each sub-block may be compared with each other, and the intra prediction mode having a smaller value may be derived as the intra prediction mode of each sub-block. If the values of the two modes are equal, one of the two modes may be derived as an intra prediction mode of the each sub-block.

Alternatively, the intra prediction mode of each sub-block may be derived using intra prediction modes of neighboring blocks of the current block. Here, the intra prediction mode of one or more neighboring blocks of the current block to the left and/or above the sample position (0, 0) of each sub-block may be used. For example, the intra prediction mode of each sub-block shown in fig. 17 may be derived by the following [ equation 5 ].

[ equation 5]

C1＝(j+d+1)>>1，

C2＝e，

C3＝g，

C4＝g，

C5＝j，

C6＝(j+e+1)>>1，

C7＝g，

C8＝g，

C9＝k，

C10＝k，

C11＝(k+g+1)>>1，

C12＝g，

C13＝l，

C14＝l，

C15＝l，

C16＝(l+g+1)>>1。

Alternatively, if the intra prediction modes of neighboring blocks of the current block are all non-directional modes, the intra prediction modes of the sub-block may be derived as at least one of the non-directional modes (e.g., DC mode and planar mode).

As shown in fig. 18, the size and/or shape of the sub-blocks in the current block may vary. The partition structure and/or size of the current block and/or sub-block may be determined by encoding/decoding. The intra prediction mode of each sub-block may be derived according to at least one of the above-described methods for deriving the intra prediction mode of the current block or sub-block.

For example, statistics of intra prediction modes of blocks left and above a sample at a position (0, 0) of each sub-block may be derived as the intra prediction mode of said each sub-block. For example, if the statistical value is an average value, the intra prediction mode of the sub-block shown in fig. 18 may be derived by the following [ equation 6 ].

[ equation 6]

C1＝(j+d+1)>>1，

C2＝(C1+e+1)>>1，

C3＝(C2+g+1)>>1，

C4＝(k+C1+1)>>1，

C5＝(C4+C2+1)>>1，

C6＝(l+C4+1)>>1，

C7＝(C6+C5+1)>>1，

C8＝(C5+C3+1)>>1，

C9＝(C8+C3+1)>>1，

C10＝(C7+C8+1)>>1。

Alternatively, statistics of intra prediction modes of one or more neighboring blocks adjacent to each sub-block may be derived as intra prediction modes of said each sub-block. For example, if the statistical value is an average value, the intra prediction mode of each sub-block shown in fig. 18 may be derived by the following equation [7 ].

[ equation 7]

C1＝(b+j+d)/3，

C2＝(C1+d+e)/3，

C3＝(C2+e+g)/3，

C4＝(k+j+C1)/3，

C5＝(C4+C2+C1)/3，

C6＝(l+C4+k)/3，

C7＝(C6+C5+C4)/3，

C8＝(C5+C3+C2)/3，

C9＝(C8+C3)/2，

C10＝(C5+C8+C9+C7)/4。

In case that the current block is divided into sub-blocks and an intra prediction mode for each sub-block is derived, the intra prediction mode of the current block may be derived by the MPM, and then the intra prediction mode of each sub-block may be derived using the derived intra prediction mode of the current block and the intra prediction modes of neighboring blocks. Here, the intra prediction mode of the sub-block may be inferred according to at least one of a plurality of methods for deriving the intra prediction mode of the current block using at least one of the intra prediction mode of the current block and the intra prediction modes of neighboring blocks derived by the MPM. If the intra prediction mode of the current block, which is derived by the MPM, is pred_mpm, the intra prediction mode of the sub-block may be inferred in the following manner.

For example, the average value of intra prediction modes of blocks left and above the sample at the position (0, 0) of each sub-block may be compared with pred_mpm. Pred_mpm+1 may be derived as the intra prediction mode of the each sub-block if the average value is greater than Pred_mpm, and Pred_mpm-1 may be derived as the intra prediction mode of the each sub-block if the average value is less than Pred_mpm. Alternatively, the average value of intra prediction modes of blocks left and above a sample at the position (0, 0) of each sub-block, and pred_mpm may be derived as the intra prediction mode of each sub-block. Alternatively, the intra prediction mode of the sub-block may be inferred by comparing the intra prediction modes of the blocks left and above the sample at the position (0, 0) of each sub-block with pred_mpm, and adjusting pred_mpm. Here, at least one of the above-described statistical values may be used instead of the average value.

In fig. 19, the number marked on each block represents the number of the intra prediction mode of the block. In addition, cx denotes the x-th sub-block of the current block (x is 1, …, 16). In addition, the arrow indicates the direction or angle of the intra prediction mode of the block.

For example, statistics of intra prediction modes of blocks left and above a sample at a position (0, 0) of each sub-block may be derived as the intra prediction mode of said each sub-block. The statistical value may be, for example, an average value. Alternatively, if at least one of the intra prediction modes of the neighboring block is a non-directional mode, the intra prediction mode of the sub-block may be derived as a directional mode among the intra prediction modes of the neighboring block. The non-directional modes may include, for example, a planar mode (mode number 0) and a DC mode (mode number 1). For example, the intra prediction mode of each sub-block shown in fig. 19 may be derived by the following equation [8 ].

[ equation 8]

C1＝(30+50+1)>>1＝40

C2＝(40+50+1)>>1＝45

C3＝45

C4＝(45+58+1)>>1＝52

C5＝(18+40+1)>>1＝29

C6＝(29+45+1)>>1＝37

C7＝(37+45+1)>>1＝41

C8＝(41+52+1)>>1＝47

C9＝(16+29+1)>>1＝23

C10＝(23+37+1)>>1＝30

C11＝(30+41+1)>>1＝36

C12＝(36+47+1)>>1＝42

C13＝23

C14＝(23+30+1)>>1＝27

C15＝(27+36+1)>>1＝32

C16＝(32+42+1)>>1＝37

In fig. 20, the number marked on each sub-block represents the number of the intra prediction mode of the block. In addition, cx denotes the x-th sub-block of the current block (x is 1, …, 14). In addition, the arrow indicates a direction or angle for intra prediction of a block.

In the embodiment described with reference to fig. 20, at least one of the intra prediction mode of the current block and the partition information of the sub-block may be derived by decoding. The intra prediction mode of each sub-block in the current block may be derived using the derived intra prediction mode of the current block and an average of intra prediction modes of blocks left and above a sample point at a position (0, 0) of each sub-block. For example, if the derived intra prediction mode of the current block is greater than the average value, 1/2 of the average value may be subtracted from the derived intra prediction mode of the current block. If the derived intra prediction mode of the current block is equal to or less than the average value, the derived intra prediction mode of the current block may be added to 1/2 of the average value. Here, at least one of the above-described statistical values may be used instead of the average value.

Alternatively, if at least one of the intra prediction modes of the neighboring block is a non-directional mode, the intra prediction mode of the sub-block may be derived as a directional mode among the intra prediction modes of the neighboring block. The non-directional modes may include, for example, a planar mode (mode number 0) and a DC mode (mode number 1). For example, if the derived intra prediction mode of the current block is 52, the intra prediction mode of each sub-block shown in fig. 20 may be derived by the following [ equation 9 ].

[ equation 9]

C1:abs(52-((30+50+1)>>1)+1)>>1＝6,C1＝52-6＝46

C2:abs(52-((46+50+1)>>1)+1)>>1＝2,C2＝52-2＝50

C3:abs(52-((50+64+1)>>1)+1)>>1＝3,C3＝52+3＝55

C4:abs(52-((18+46+1)>>1)+1)>>1＝10,C4＝52-10＝42

C5:abs(52-((42+50+1)>>1)+1)>>1＝3,C5＝52-3＝49

C6:abs(52-((42+49+1)>>1)+1)>>1＝3,C6＝52-3＝49

C7:abs(52-((49+55+1)>>1)+1)>>1＝0,C7＝52-0＝52

C8:abs(52-((52+55+1)>>1)+1)>>1＝1,C8＝52+1＝53

C9:abs(52-((16+42+1)>>1)+1)>>1＝12,C9＝52-12＝40

C10:abs(52-((16+40+1)>>1)+1)>>1＝12,C10＝52-12＝40

C11:abs(52-40+1)>>1＝6,C11＝52-6＝46

C12:abs(52-((40+49+1)>>1)+1)>>1＝4,C12＝52-4＝48

C13:abs(52-((48+49+1)>>1)+1)>>1＝2,C13＝52-2＝50

C14:abs(52-((50+52+1)>>1)+1)>>1＝1,C14＝52-1＝51

In deriving the intra prediction mode, the current block may be divided into a plurality of sub-blocks smaller than the current block, and then the intra prediction mode of each sub-block may be derived. Here, the intra prediction mode may represent an intra prediction direction. The intra prediction modes may be included in a set of intra prediction modes predefined in the encoder and decoder.

For example, an Intra Prediction Direction Field (IPDF) for a current block may be generated using at least one of intra prediction modes of a block encoded/decoded by using intra prediction among reconstructed blocks adjacent to the current block, and the intra prediction mode of the current block. The predetermined transformation model may be used in the process of generating IPDF. After IPDF is generated, the IPDF may be used to determine an intra prediction mode for each sub-block of the current block.

In addition, similar to the above example, if the current block is divided from a block greater than the current block or smaller than the current block depth, the current block may be a sub-block of the block greater than the current block or smaller than the current block depth. In this case, the intra prediction mode of the current block may be derived using at least one of the intra prediction modes of the block encoded/decoded by intra prediction among reconstructed blocks adjacent to the block greater than the current block or smaller than the current block in depth. The intra prediction mode of the current block may be derived by generating IPDF.

At least one of a rigid transformation, a similarity transformation, an affine transformation, a homography transformation, a 3D transformation, and other transformations may be used as the predetermined transformation model. The homography transform may be a perspective transform.

Since the intra prediction mode of each sub-block divided from the current block can be derived using at least one of the intra prediction modes of the block encoded/decoded by using intra prediction among reconstructed blocks adjacent to the current block and the intra prediction mode of the current block, the number of bits required for entropy encoding/decoding the intra prediction mode of each sub-block can be reduced.

The size of the sub-block may be smaller than or equal to the size of the current block. For example, if the size of the current block is m×n (M and N are positive integers), the size of the sub-block may be M/k×n/L. K and M may be positive integers, and L may be a positive factor of N. In addition, M/K or N/L may be a positive integer.

In addition, there may be P sub-blocks in the current block. P may be 0 or a positive integer. For example, there may be 1, 2, 4, or 16 sub-blocks in the current block.

In addition, when the current block is divided into sub-blocks, information indicating whether the current block has been divided into sub-blocks may not be separately entropy encoded/decoded. Whether the current block is divided into sub-blocks may be determined based on information indicating whether an intra prediction mode of the current block has been derived based on the sub-blocks.

In addition, since the intra prediction mode of the sub-block is inferred using at least one of the intra prediction modes of the block encoded/decoded by using the intra prediction among the reconstructed blocks adjacent to the current block and the intra prediction mode of the current block, the intra prediction mode of the sub-block may not be entropy encoded/decoded. However, the intra prediction mode of the current block may be entropy encoded/decoded. In particular, if the current block is composed of one sub-block, the intra prediction mode of the current block may not be entropy encoded/decoded, and at least one of the intra prediction modes of the block encoded/decoded by intra prediction among reconstructed blocks adjacent to the current block may be used to derive the intra prediction mode of the current block.

A block encoded/decoded by using intra prediction among reconstructed blocks adjacent to the current block for IPDF generation may be referred to as a seed block. The location of the seed block may be referred to as a seed point. The intra prediction mode possessed by the seed block including the seed point may be referred to as a seed point intra prediction mode (spim).

Fig. 21 is a diagram depicting an embodiment of deriving an intra prediction mode of a current block using spim.

In the example shown in fig. 21, the size of the current block may be 16×16, and the size of each sub-block may be (16/4) × (16/4). Here, the seed block may be at least one of a plurality of neighboring blocks that have been encoded/decoded through intra prediction. The seed block or seed point may be located in a fixed position relative to the current block. Alternatively, at least one of the above, left, upper left, lower right, and upper right blocks or locations may be determined as a seed block or seed point.

For example, an intra prediction mode of at least one of the upper neighboring blocks c, d, e, f and g of the current block may be used as the spim.

Alternatively, the intra prediction mode of the upper right block h adjacent to the current block may be used as the spim.

Alternatively, an intra prediction mode of at least one of the upper left blocks a and b adjacent to the current block may be used as the spim.

Alternatively, the intra prediction mode of at least one of the left blocks i, j, k, and l adjacent to the current block may be used as the spim.

Alternatively, the intra prediction mode of the lower left block m adjacent to the current block may be used as the spim.

For example, the intra prediction mode of the current block may also be used as the spim.

IPDF may be generated using spim of one or more seed points.

For example, if the coordinates of a plurality of seed points for determining parameters of a predetermined transformation model are (x_sp, y_sp) (sp=1, 2,3, …), and the spim of a seed block including the seed points is mode_sp (sp=1, 2,3, …), the coordinates (x_sp ', y_sp') of the transformed seed points may be determined as (x_sp ', y_sp') = (x_sp+dx, y_sp+dy). Here, dx may represent a displacement in the x-axis direction, and dy may represent a displacement in the y-axis direction. In addition, dx=d_sub_cos Θ and dy=d_sub_sin Θ may be determined.

Here, Θ may be determined according to spimp. For example, if the intra prediction mode is a directional mode as shown in fig. 6, each spim may have unique directionality, and a positive angle with respect to the x-axis may be determined as Θ.

For example, for vertical intra prediction mode, Θ=270°.

For example, for horizontal intra prediction mode, Θ=0°.

For example, for the lower left diagonal intra prediction mode, Θ=225 °.

For example, for the upper right diagonal intra prediction mode, Θ=45°.

For example, for the lower right diagonal intra prediction mode, Θ=135°.

Here, for an intra prediction mode having non-directivity (such as a DC mode or a plane mode), Θ may be determined as a specific value. The specific value may be, for example, 0, 90, 180 or 270 degrees.

Spim may represent the directionality from the seed point, and dsub may represent the magnitude of the vector having that directionality. The size of d_sub may be determined according to the size and/or shape of the seed block to which the seed point belongs. In addition, d_sub may have a fixed value P for each intra prediction mode. Here, P may be 0 or an integer. For example, if the size of the current block is mxn (M and N are positive integers) and d_cur=s (S is a positive integer), d_sub (sub=1, 2,3, …) of the seed block of k×l (K and L are positive integers) may be determined as d_sub=s (k×l)/(m×n). For example, if the size of the current block is 16×16 in fig. 21, d_cur=s and d_sub of the seed block may be determined according to at least one of the following methods.

For example, for seed block D or e, d_sub=s (4×8)/(16×16) =s/8 may be determined.

For example, for seed block g or j, d_sub=s (8×8)/(16×16) =s/4 may be determined.

For example, for seed block k or l, d_sub=s (8×4)/(16×16) =s/8 may be determined.

For example, for seed block m or h, d_sub=s (16×16)/(16×16) =s may be determined.

For example, d_sub may be determined as S for each seed block.

Candidates may be configured by making a spim list to generate IPDF for the current block. The spim list may be generated using intra prediction modes of at least one of neighboring blocks of the current block. For example, the spim may be a set of one or more of the upper left (spim_tl), upper right (spim_tr), lower left (spim_bl) and lower right (spim_br) spims of the current block. For the current block of w×h, spim_tl may use at least one of the intra prediction modes of the upper neighboring block, the upper left neighboring block, and the left neighboring block with respect to the position (0, 0) of the current block as a candidate. The spimp_tr may use at least one of the intra prediction modes of the upper neighboring block and the upper right neighboring block with respect to the position (W-1, 0) of the current block as a candidate. The spimp_bl may use at least one of the intra prediction modes of the left neighboring block and the lower left neighboring block with respect to the position (0, h-1) of the current block as a candidate. The spimp_br may represent an intra prediction mode of a neighboring block of the current block. Alternatively, the spimp_br may be used to represent an intra prediction mode of the current block.

In the example shown in fig. 21, spimp_tl may have at least one of intra prediction modes adjacent to blocks d, b, and j. In addition, the spim_tr may have at least one of intra prediction modes adjacent to the blocks g and h. In addition, the spimp_bl may have at least one of intra prediction modes adjacent to the blocks i and m. In addition, the spimp_br may have at least one of intra prediction modes of the current block.

In addition, seed blocks or seed points may be searched in a predetermined order. For example, by searching for seed blocks or seed points in the order of left, up, down, up-left, up-right, and up, the spim list may be composed of intra prediction modes at the respective seed blocks or seed points.

In order to make the spim list, processing of removing intra prediction modes having directions different from directions of other modes based on the similarity between intra prediction modes may be performed for at least one candidate of a plurality of candidates for each of spim_tl, spim_tr, spim_bl, and spim_br. Here, intra Prediction Mode Difference (IPMD) may be used as a criterion for measuring the similarity. Non-directional MODEs (e.g., DC_MODE and PLANAR_MODE) may be removed when the SPIM list is made.

For example, if ipmd=abs (spim_tl_mode-spim_neighbor) > Th1 (Th 1 is a positive integer) in the comparison between each of the three candidate modes spim_tl_mode of spim_tl and the candidate mode spim_neighbor (neighbor=tr or BL or BR), the corresponding mode may be removed from the candidate set for spim_tl.

For example, if ipmd=abs (spim_tr_mode-spim_neighbor) > Th2 (Th 2 is a positive integer) in the comparison between each of the two candidate patterns spim_tr_mode of spim_tr and the candidate pattern spim_neighbor (neighbor=tl or BL or BR), the corresponding pattern may be removed from the candidate set for spim_tr.

For example, if ipmd=abs (spim_bl_mode-spim_neighbor) > Th3 (Th 3 is a positive integer) in the comparison between each of the two candidate patterns spim_bl_mode of spim_bl and the candidate pattern spim_neighbor (neighbor=tl or TR or BR), the corresponding pattern may be removed from the candidate set for spim_bl.

For example, if ipmd=abs (spim_br_mode-spim_neighbor) > Th4 (Th 4 is a positive integer) in the comparison between each of the candidate patterns spim_br_mode of spim_br and the candidate patterns spim_neighbor (neighbor=tl or TR or BL), the corresponding pattern may be removed from the candidate set for spim_br.

For example, if at least one of the plurality of candidates for each of spim_tl, spim_tr, spim_bl, and spim_br is a non-directional intra prediction mode (such as a DC mode or a planar mode), the corresponding mode may be removed from the candidate set.

After the candidate having the smallest similarity among the plurality of candidates for making the spim list is removed, the number of spims required for IPDF generation may be determined according to the specific 2D transform model used. For example, the 2D transformation model may include a rigid transformation, a similarity transformation, an affine transformation, and a homography transformation. In addition, the number of spims may be determined as variable values 1, 2, 3, 4, or N (N is a positive integer) according to the 2D transform model.

For example, if the transformation model for IPDF generation is a rigid transformation, at least two spims may be required.

As described in equation 10, the rigid transformation may have a degree of freedom of 3 (3-DoF). Here, (x, y) may be pre-transform (pre-transform) coordinates of the seed point, and (x ', y') may be post-transform (post-transform) coordinates of the seed point. Θ, tx, and ty are model parameters to be determined, where Θ, tx, and ty may be rotation angle, x-axis displacement, and y-axis displacement, respectively.

[ equation 10]

The pair of (x, y) - (x ', y') can be obtained using Θ determined from one spim, and two equations related to Θ, tx, and ty can be determined by substituting the pair of (x, y) - (x ', y') into equation 10. In addition, four equations related to Θ, tx, and ty may be determined from two spims, and three of the four equations may be used to determine the rigid transformation model.

The two spims may be determined by selecting at least two of spim_tl, spim_tr, spim_bl, and spim_br. The selected spim may be added to the spim list.

As shown in fig. 22, the spim list may be populated in ascending order of the IPMD sum of the two spim candidate patterns.

For example, one of the candidate patterns for spim_tl and one of the candidate patterns for spim_tr may be used as two spims.

For example, one of the candidate patterns for spim_tl and one of the candidate patterns for spim_bl may be used as two spims.

For example, one of the candidate patterns for spim_tl and one of the candidate patterns for spim_br may be used as two spims.

For example, one of the candidate patterns for spim_tr and one of the candidate patterns for spim_bl may be used as two spims.

For example, one of the candidate patterns for spim_tr and one of the candidate patterns for spim_br may be used as two spims.

For example, one of the candidate patterns for spim_bl and one of the candidate patterns for spim_br may be used as two spims.

If both spims are not filled, the spim list may be filled with the available spims. For example, the spim list may be populated with spim ± delta. Here, delta may be any positive integer, e.g., 1, 2, 3 ….

Once the two spims (spim 1, spim 2) are determined, four equations related to Θ, tx, and ty can be generated by [ equation 10 ]. Three of the four equations may be used to determine parameters for the rigid transformation model. The determined model may be used for IPDF generation.

For example, the rigid transformation may be determined using at least one of two equations calculated by SPIM1 and two equations calculated by SPIM 2.

For example, if a similar transformation is used as the transformation model for IPDF generation, at least two spims may be required.

The similarity transformation may have a 4-DoF as described in equation 11. Here, (x, y) may be pre-transformed coordinates of the seed point and (x ', y') may be post-transformed coordinates of the seed point. a. b, c and d may be model parameters to be determined.

[ equation 11]

/>

The pair of (x, y) - (x ', y') can be obtained using Θ determined from one spim, and two equations related to a, b, c, and d can be determined by substituting the pair of (x, y) - (x ', y') into equation 11. In addition, four equations related to a, b, c, and d may be determined from the two spims, and may be used to obtain a similar transformation model.

The two spims may be determined by selecting at least two of spim_tl, spim_tr, spim_bl, and spim_br. The selected spim may be added to the spim list. Here, as shown in equation 22, the spim list may be filled in ascending order of the sum of IPMD values of the two spim candidate patterns.

Once the two spims (spim 1, spim 2) are determined, the parameters for the similar transformation model can be determined by generating four equations related to Θ, tx, and ty by equation 11. The determined model may be used for IPDF generation.

For example, if affine transformation is used as transformation model for IPDF generation, at least three spims may be required.

The affine transformation may have a 6-DoF as described in equation 12. Here, (x, y) may be pre-transformed coordinates of the seed point and (x ', y') may be post-transformed coordinates of the seed point. a. b, c, d, e and f may be model parameters to be determined.

[ equation 12]

The pair (x, y) - (x ', y') can be obtained using Θ determined from one spim, and two equations related to a, b, c, d, e and f can be determined by substituting the pair (x, y) - (x ', y') into equation 12. In addition, six equations related to a, b, c, d, e and f can be determined from three spims, and can be used to obtain an affine transformation model.

Three spims may be determined by selecting at least three of spim_tl, spim_tr, spim_bl, and spim_br. The selected spim may be added to the spim list.

As shown in fig. 23, the spim list may be populated with spims in ascending order of the sum of IPMD values of the three spim candidate patterns.

For example, one of the candidate patterns for spim_tl, one of the candidate patterns for spim_tr, and one of the candidate patterns for spim_bl may be used as three spims.

For example, one of the candidate patterns for spim_tl, one of the candidate patterns for spim_tr, and one of the candidate patterns for spim_br may be used as three spims.

For example, one of the candidate patterns for spim_tl, one of the candidate patterns for spim_bl, and one of the candidate patterns for spim_br may be used as three spims.

For example, one of the candidate patterns for spim_tr, one of the candidate patterns for spim_bl, and one of the candidate patterns for spim_br may be used as three spims.

If three spims are not filled, then the spim list may be filled with the available spims. For example, the spim list may be populated with spim ± delta. Here, delta may be any positive integer, e.g., 1, 2, 3 ….

Once the three spims (spim 1, spim 2, spim 3) are determined, the parameters for the affine transformation model can be determined by generating six equations related to a, b, c, d, e and f by equation 12. The determined model may be used for IPDF generation.

For example, if a homography transform (or perspective transform) is used as the transform model for IPDF generation, at least four spims may be required.

The affine transformation may have an 8-DoF as described in equation 13. Here, (x, y) may be pre-transformed coordinates of the seed point and (x ', y') may be post-transformed coordinates of the seed point. h1, h2, h3, h4, h5, h6, h7 and h8 may be model parameters to be determined.

[ equation 13]

The pair of (x, y) - (x ', y') can be obtained using Θ determined from one spim, and two equations related to h1, h2, h3, h4, h5, h6, h7, and h8 can be determined by substituting the pair of (x, y) - (x ', y') into [ equation 13 ]. In addition, eight equations related to h1, h2, h3, h4, h5, h6, h7, and h8 may be determined from the four spims, and the homography transformation model may be obtained using the eight equations.

Four spims may be determined by selecting at least four of spim_tl, spim_tr, spim_bl, and spim_br. The selected spim may be added to the spim list.

As shown in fig. 24, the spim list may be sequentially filled with spims in an ascending order of the sum of IMPD values of the four spim candidate patterns.

For example, two of the candidate patterns for spim_tl and two of the candidate patterns for spim_tr may be used as four spims.

For example, two of the candidate patterns for spim_tl and two of the candidate patterns for spim_bl may be used as four spims.

For example, two candidate patterns among candidate patterns for spim_tl and two candidate patterns among candidate patterns for spim_br may be used as four spims.

For example, two candidate patterns among candidate patterns for spim_bl and two candidate patterns among candidate patterns for spim_br may be used as four spims.

For example, two candidate patterns among candidate patterns for spim_tl, one candidate pattern among candidate patterns for spim_tr, and one candidate pattern among candidate patterns for spim_bl may be used as four spims.

For example, two candidate patterns among candidate patterns for spim_tl, one candidate pattern among candidate patterns for spim_tr, and one candidate pattern among candidate patterns for spim_br may be used as four spims.

For example, two candidate patterns among candidate patterns for spim_tl, one candidate pattern among candidate patterns for spim_bl, and one candidate pattern among candidate patterns for spim_br may be used as four spims.

For example, two candidate patterns among candidate patterns for spim_tr, one candidate pattern among candidate patterns for spim_tl, and one candidate pattern among candidate patterns for spim_bl may be used as four spims.

For example, two candidate patterns among candidate patterns for spim_tr, one candidate pattern among candidate patterns for spim_tl, and one candidate pattern among candidate patterns for spim_br may be used as four spims.

For example, two candidate patterns among candidate patterns for spim_bl, one candidate pattern among candidate patterns for spim_tl, and one candidate pattern among candidate patterns for spim_tr may be used as four spims.

For example, two candidate patterns among candidate patterns for spim_bl, one candidate pattern among candidate patterns for spim_tl, and one candidate pattern among candidate patterns for spim_br may be used as four spims.

For example, two candidate patterns among candidate patterns for spim_bl, one candidate pattern among candidate patterns for spim_tr, and one candidate pattern among candidate patterns for spim_br may be used as four spims.

For example, two candidate patterns among candidate patterns for spim_br, one candidate pattern among candidate patterns for spim_tl, and one candidate pattern among candidate patterns for spim_tr may be used as four spims.

For example, two candidate patterns among candidate patterns for spim_br, one candidate pattern among candidate patterns for spim_tl, and one candidate pattern among candidate patterns for spim_bl may be used as four spims.

For example, two candidate patterns among candidate patterns for spim_br, one candidate pattern among candidate patterns for spim_tr, and one candidate pattern among candidate patterns for spim_bl may be used as four spims.

For example, one of candidate patterns for spim_br, one of candidate patterns for spim_tr, one of candidate patterns for spim_bl, and one of candidate patterns for spim_br may be used as four spims.

If four spims are not filled, the spim list may be filled with the available spims. For example, the spim list may be populated with spim ± delta. Here, delta may be any positive integer, e.g., 1, 2, 3 ….

Once the four spims (spim 1, spim 2, spim 3, spim 4) are determined, the parameters for the homography transformation model can be determined by generating eight equations related to h1, h2, h3, h4, h5, h6, h7, and h8 by equation 13. The determined model may be used for IPDF generation.

After generating the IPDF using the predetermined transform model, the generated IPDF may be used to allocate an intra prediction mode to a kxl sub-block within the current block of wxh. Here, the size kxl of each sub-block (K is a positive integer equal to or smaller than M, and L is a positive integer equal to or smaller than H) may be a fixed size equal to or smaller than the size of the current block. Alternatively, the sub-block size may be adaptively determined using the current block size and/or IPMD. Alternatively, the sub-block size may be equal to the size of the current block.

As shown in (a) of fig. 25, the size of the sub-block may be fixed to 8×8. Alternatively, as shown in (b) of fig. 25, the size of the sub-block may be fixed to 4×4. Alternatively, as shown in (c) of fig. 25, the size of the sub-block may be fixed to 2×2. Alternatively, as shown in (d) of fig. 25, the size of the sub-block may be fixed to 1×1. The fixed size 1×1 may be the size of the sample unit.

For example, the size of the sub-block may be determined based on the size of the current block.

For example, the size of the sub-block may be determined based on at least one of IPMD of the current block among spim_tl, spim_tr, spim_bl, and spim_br.

For example, the size of the sub-block may be determined based on at least one of IPMD of the current block among spim_tl, spim_tr, spim_bl and spim_br and the size of the current block.

Information about the size of the sub-block may be entropy encoded/decoded in the bitstream. Here, the information may be entropy encoded/decoded in at least one of VPS, SPS, PPS, APS, a slice header, a parallel block header, CTU, CU, PU, TU, a block, and a sub-block.

Information about the size of the sub-block may be adaptively derived according to the size of the current block and/or IPMD without transmitting the information.

In addition, the sub-block size may be determined based on at least one of the encoding parameters of the current block and the encoding parameters of neighboring blocks of the current block.

The intra prediction mode of the sub-block may be allocated using the determined IPDF. The intra prediction mode at a specific location in each sub-block may be implemented as a vector by substituting the coordinates of the specific location into the determined IPDF model. The specific position may be determined as a position of a pixel in the sub-block or a position adjacent to a boundary of the sub-block. For example, at least one of an upper left position, an upper right position, a lower left position, a lower right position, and an intermediate position of the sub-block may be determined as the specific position. If the coordinates of the specific location in the sub-block are (x, y) and the transformed coordinates of the location calculated by IPDF are (x ', y'), then θ can be used _SB ＝atan[(y’-y)/(x’-x)]To determine the direction theta of the vector _SB 。

As shown in fig. 26, θ _SB May be mapped to intra-prediction modes pointing in the most similar direction among a plurality of direction modes. θ can be performed using a look-up table (LUT) _SB Operation to intra prediction mode mapping.

In addition, when the intra prediction mode of the sub-block is allocated using IPDF, the IDF may be allocated as the intra prediction mode of the sub-block based on the nearest neighbor method.

In addition, when the intra prediction mode is allocated to the sub-block using IPDF, the intra prediction mode may be allocated to the sub-block by quantizing the IPDF to an integer.

Furthermore, when the intra prediction mode is allocated to the sub-block using IPDF, the intra prediction mode may be allocated to the sub-block by rounding the IPDF to the nearest integer.

For sub-block by sub-block intra prediction based on a transform model, the information to be additionally entropy encoded/decoded in the bitstream may include at least one of the following.

Information indicating whether sub-block-by-sub-block intra prediction based on a transform model has been performed for a current block: TBIP_flag

The tbip_flag may be information such that: the information indicates whether to derive an intra prediction mode of each sub-block using at least one of intra prediction modes of a block encoded/decoded by using intra prediction among reconstructed blocks adjacent to the current block and the intra prediction mode of the current block.

Index indicating the spim set used: SPIPM_idx

Now, another embodiment of deriving an intra prediction mode based on a sub-block using a transform model will be described. For example, when there is a block encoded/decoded by sub-block-by-sub-block intra prediction based on a transform model among neighboring reconstructed blocks, an intra prediction mode may be derived for each sub-block using an IPDF model of the neighboring block, instead of directly generating IPDF of the current block.

In order to determine whether there is any block encoded/decoded by sub-block-by-sub-block intra prediction based on the transform model among the neighboring reconstructed blocks of the current block, a predefined scan order may be used. The scanning order may be at least one of the following orders.

For example, in fig. 27, scanning may be performed in the order of a- > B- > C- > D- > E. Alternatively, the scanning may be performed in the order of a- > B- > D- > C- > E. Alternatively, the scanning may be performed in the order of B- > a- > D- > C- > E. Alternatively, the scanning may be performed in the order of E- > a- > B- > C- > D. Alternatively, the scanning may be performed in any other order.

Alternatively, a portion of blocks A, B, C, D and C may be excluded from scanning. Alternatively, blocks other than A, B, C, D and C may be scanned. The neighboring-reconstructed block to be scanned may be determined based on at least one of a size, shape, and encoding parameters of at least one of the neighboring-reconstructed block and the current block as described herein.

In the example shown in fig. 28, in the case where a block a, which is a neighboring block to the current block, is encoded/decoded by intra prediction of a sub-block unit using a transform model, IPDF of the block a may be generated using at least one of spim_a_tl, spim_a_tr, spim_a_bl, and spim_a_br.

At least one of the spipm_cur_tl, the spipm_cur_tr, the spipm_cur_bl, and the spipm_cur_br of the current block may be derived using the IPDF of the block a, and the IPDF of the current block may be generated using the derived at least one of the spipm_cur_tl, the spipm_cur_tr, the spipm_cur_bl, and the spipm_cur_br, thereby performing sub-block-by-sub-block intra prediction.

If at least one of the neighboring reconstructed blocks of the current block is a block encoded/decoded using IPDF, IPDF of the current block may be derived using IPDF of the corresponding neighboring reconstructed block. In addition, information of the tbip_flag may be entropy encoded/decoded.

Another embodiment of deriving an intra prediction mode based on sub-blocks using an isospatial model will be described.

In case an isospatial model is used, at least two spims may be required. For example, as shown in (a) of fig. 29, one of the candidate patterns for spim_tl, one of the candidate patterns for spim_tr, one of the candidate patterns for spim_bl, and one of the candidate patterns for spim_br, that is, four candidate patterns in total, may be selected. As shown in fig. 24, the four selected spim candidate patterns may be populated in the spim list in ascending order of the sum of IPMD values.

The intra prediction mode of the outermost sub-block of the current block may be first determined using spim_tl, spim_tr, spim_bl, and/or spim_br. Equally spatially determining the intra prediction mode may mean dividing values of the intra prediction mode at equal intervals using at least two intra prediction modes and allocating the divided intra prediction mode values to the sub-blocks.

For example, in the example shown in (a) of fig. 29, if spim_tl is 24 and spim_tr is 21, the intra prediction modes of the sub-blocks A, B, C and D may be spatially determined using the values of spim_tl and spim_tr, etc. For example, as shown in (b) of fig. 29, a=24, b=23, c=22, and d=21 can be determined.

For example, in the example shown in (a) of fig. 29, if spim_tl is 25 and spim_bl is 38, the intra prediction modes of the sub-blocks A, E, I and M may be spatially determined using the values of spim_tl and spim_bl, etc. For example, as shown in (b) of fig. 29, a=24, e=29, i=34, and m=38 can be determined.

For example, in the example shown in (a) of fig. 29, if spim_bl is 38 and spim_br is 35, the intra prediction modes of the sub-blocks M, N, O and P may be spatially determined using the values of spim_bl and spim_br, etc. For example, as shown in (b) of fig. 29, m=38, n=37, o=36, and p=35 can be determined.

For example, in the example shown in (a) of fig. 29, if spim_tr is 21 and spim_br is 35, the intra prediction modes of the sub-blocks D, H, I and P may be spatially determined using the values of spim_tr and spim_br and the like. For example, as shown in (b) of fig. 29, d=21, h=26, i=31, and p=35 can be determined.

After the intra prediction modes of the outermost sub-blocks of the current block are all determined, the intra prediction mode of the second outermost sub-block may be determined. In the example shown in fig. 29, the second outermost sub-blocks may be sub-blocks F, G, J and K. Here, the spim_tl may be reset to a value of a pattern (a pattern of the sub-block a in (a) of fig. 29) at an upper left position of an upper left sub-block (the sub-block F in (a) of fig. 29) among the second outermost sub-blocks. In addition, the spim_tr may be reset to a value of a pattern (a pattern of the sub-block D in (a) of fig. 29) at an upper right position of an upper right sub-block (the sub-block G in (a) of fig. 29) among the second outermost sub-blocks. In addition, the spimp_bl may be reset to a value of a mode (a mode of the sub-block M in (a) of fig. 29) at a lower left position of a lower left sub-block (a sub-block J in (a) of fig. 29) among the second outermost sub-blocks. In addition, the spim_br may be reset to a value of a pattern (a pattern of the sub-block P in (a) of fig. 29) at a lower right position of a lower right sub-block (a sub-block K in (a) of fig. 29) among the second outermost sub-blocks. This operation may be recursively repeated until the mode of all sub-blocks in the current block is determined.

For example, in the example shown in (b) of fig. 29, if spim_tl is 24 and spim_tr is 21, intra prediction modes of the sub-blocks F and G may be spatially determined using values of spim_tl and spim_tr, etc. For example, as shown in (c) of fig. 29, f=24 and g=21 may be determined.

For example, in the example shown in (b) of fig. 29, if spim_bl is 38 and spim_br is 35, intra prediction modes of the sub-blocks J and K may be spatially determined using values of spim_tl and spim_tr, etc. For example, as shown in (c) of fig. 29, j=38 and k=35 may be determined.

The information to be additionally entropy encoded/decoded for sub-block-wise intra prediction based on the isospatial model may be at least one of the following.

Information indicating whether sub-block-by-sub-block intra prediction based on an isospatial model has been performed for the current block: ES_flag

Index indicating the spim set used: SPIPM_idx

Now, another embodiment of deriving an intra prediction mode based on sub-blocks using a bilinear filter model will be described.

In order to determine an intra prediction mode based on sub-blocks using a bilinear filter model, at least two spims may be required. For example, as shown in (a) of fig. 30, a total of four candidate patterns may be selected by selecting one of the candidate patterns for spim_tl, one of the candidate patterns for spim_tr, one of the candidate patterns for spim_bl, and one of the candidate patterns for spim_br. As shown in fig. 24, the four selected spim candidate patterns may be sequentially filled in the spim list in ascending order of the sum of IPMD values.

The mode of the upper left sub-block (sub-block a in (a) of fig. 30) in the current block may be determined as spim_tl. In addition, the mode of the upper right sub-block (sub-block D in (a) of fig. 30) in the current block may be determined as spim_tr. In addition, the mode of the lower left sub-block (sub-block M in (a) of fig. 30) in the current block may be determined as spimp_bl. In addition, the mode of the lower right sub-block (sub-block P in (a) of fig. 30) in the current block may be determined as spim_br. As shown in (b) of fig. 30, intra prediction modes of the upper left Fang Zikuai, upper right sub-block, lower left Fang Zikuai, and lower right sub-block in the current block may be determined as spim_tl, spim_tr, spim_bl, and spim_br, respectively. However, the intra prediction modes of the upper left Fang Zikuai, upper right sub-block, lower left Fang Zikuai, and lower right sub-block in the current block are not limited to the spim_tl, spim_tr, spim_bl, and spim_br, and at least one of the intra prediction modes of the upper left Fang Zikuai, upper right sub-block, lower left Fang Zikuai, and lower right sub-block in the current block may be determined as at least one of the spim_tl, spim_tr, spim_bl, and spim_br.

A bilinear filter scheme may be used to determine intra prediction modes for other sub-blocks. For example, the following [ equation 14] may be used. In [ equation 14], the function () may be at least one of floor () or ceil () or round (). In the example shown in fig. 30, function () may be round (). In addition, in [ equation 14], #of SubBlk in wdt may represent the number of sub-blocks in the horizontal direction of the current block. Similarly, # of SubBlk in hgt may represent the number of sub-blocks in the vertical direction in the current block.

[ equation 14]

For example, as shown in (c) of fig. 30, intra prediction modes of the remaining sub-blocks may be determined using equation 14.

The information to be additionally entropy encoded/decoded for sub-block-wise intra prediction based on the bilinear filter model may be at least one of the following.

Information indicating whether sub-block-by-sub-block intra prediction based on a bilinear filter model is performed for the current block: BF_flag

Index indicating the spim set used: SPIPM_idx

An intra prediction mode may be derived based on the sub-block using at least one of intra prediction modes of a block encoded/decoded by intra prediction among reconstructed blocks adjacent to the current block and the intra prediction mode of the current block, and intra prediction may be performed based on the sub-block using the derived intra prediction mode. Here, samples included in a sub-block previously encoded/decoded based on the sub-block may be used as reference samples for the subsequent sub-block-by-sub-block intra prediction.

The encoder may generate transform coefficients by performing at least one of a first transform, a second transform, and quantization on a residual block generated after sub-block-by-sub-block intra prediction. The generated transform coefficients may be entropy encoded. The first transform, the second transform, and the quantization may be performed with respect to the current block, or may be performed based on the sub-block. For example, at least one of the first transform, the second transform, and the quantization may be performed for the entire current block, or at least one of the first transform, the second transform, and the quantization may be performed for each sub-block. Here, any one of the first transform, the second transform, and the quantization may not be performed on the current block or the sub-block.

The transform coefficients may be entropy decoded in a decoder. The reconstructed residual block may be generated by performing at least one of inverse quantization, a first inverse transform, and a second inverse transform on the entropy-decoded transform coefficients. The first transform, the second transform, and the quantization may be performed with respect to the current block, and the first transform, the second transform, and the quantization may be performed based on the sub-block. For example, at least one of the first transform, the second transform, and the quantization may be performed for the entire current block, and at least one of the first transform, the second transform, and the quantization may be performed for each sub-block. Here, any one of the first transform, the second transform, and the quantization may not be performed on the current block or the sub-block.

The intra prediction information may be entropy encoded/entropy decoded from the bitstream. Fig. 31 is an exemplary diagram illustrating a syntax structure including intra prediction mode information. As shown in fig. 31, the intra prediction information may include at least one of the following pieces of information. Here, the intra prediction information may be signaled in at least one of VPS, SPS, PPS, APS, a slice header, and a parallel block header.

A flag indicating whether the MPMs match: for example, prev_intra_luma_pred_flag

Index indicating position in MPM list: for example mpm_idx

Intra luminance prediction mode information: for example, rem_intra_luma_pred_mode

Intra chroma prediction mode information: for example, intra_chroma_pred_mode

Curvature parameters of curved intra prediction mode: for example, cuv

Weighting parameters for curved intra prediction modes: for example, cw1, cw2, …, cWNs-1

LUT for curved intra prediction

A flag indicating that intra prediction modes of the current block and the sub-block are derived using intra prediction modes of neighboring blocks: for example, NDIP_flag

A flag indicating whether residual signal prediction is to be performed: for example, SRP_flag

Indicator (MPM flag): for example, mpm_flag_1, mpm_flag_2, …, mpm_flag_n, wherein the indicator is for indicating whether an intra prediction mode identical to an intra prediction mode of a current block is included in an intra prediction mode of an MPM list for each of N MPM lists when the intra prediction mode of the current block is derived or entropy-encoded/decoded using the N MPM lists

Index information: for example, mpm_idx_1, mpm_idx_2, …, mpm_idx_n, wherein the index information is used to indicate a position or order of an intra prediction mode in an MPM list when the same intra prediction mode as the intra prediction mode of the current block is included in the intra prediction mode of a specific one of the N MPM lists.

Information indicating whether transform model-based intra prediction per sub-block is performed for the current block, and an index indicating the spim set used: for example, TBIP_flag and SPIPM_idx

Information indicating whether an isospatial model is used for the current block to derive an intra prediction mode based on the sub-block, and an index indicating the set of spims used: ES_flag and SPIPM_idx

Information indicating whether to use a bilinear filter model for the current block to derive an intra prediction mode based on the sub-block, and an index indicating the spim set used: BF_flag and SPIPM_idx

When the MPM (most probable mode) flag is 1, an intra prediction mode of a luminance component may be derived from candidate modes including intra modes of neighboring units that have been encoded/decoded by using the MPM index mpm_idx.

When the MPM (most probable mode) flag is 0, the intra prediction mode of the luminance component may be encoded/decoded by using intra prediction mode information rem_intra_luma_pred_mode regarding the luminance component.

The intra prediction mode of the chrominance component may be encoded/decoded by using intra prediction mode information intra_chroma_pred_mode and/or a corresponding intra prediction mode of the chrominance component block.

In the case of a warped intra prediction mode, the prediction sample value may be derived using reference samples of different angles according to the positions (x, y) of the samples inside the prediction block. Samples within the prediction block may be grouped into a plurality of groups, and the first group may use a directional intra prediction mode having a different angle from that of the second group. Each group may include one or more samples. Each group may be triangular in shape, rectangular in shape, or other geometric shape.

The curvature parameter cuv of the curved intra prediction mode may represent a curvature applied to the curved intra prediction mode. The warped intra prediction may be performed on the current block by using at least one cuv. The curvature parameter may be derived from at least one curvature parameter among curvature parameters of neighboring blocks.

Single or multiple weight parameters cw of the warped intra prediction mode may be applied to the current block. When a plurality of weight parameters cw are applied to a current block, different weight parameters may be applied to predetermined units of the current block, such as pixels, rows, columns, or sub-blocks. The weight parameter may be derived from at least one weight parameter among weight parameters of neighboring blocks.

The neighboring blocks used to derive the curvature parameters and/or weight parameters may be encoded/decoded blocks adjacent to the current block on the upper, left, and/or right sides.

Various forms of curved intra prediction may be performed by using at least one of cuv and cw.

When N cuvs and M cw are used, intra prediction of the current block may be performed by generating at least (n×m) ×4 prediction blocks.

For example, when a single cuv and a single cw are used, intra prediction of a current block may be performed by generating at least four prediction blocks.

For example, when two cuvs and a single cw are used, intra prediction of a current block may be performed by generating at least eight prediction blocks.

For example, when a single cuv and two cw are used, intra prediction of a current block may be performed by generating at least eight prediction blocks.

For example, when two cuvs and two cw are used, intra prediction of a current block may be performed by generating at least sixteen prediction blocks.

Information about at least two cuv and/or cw may be encoded/decoded by using default values and delta values. Here, the default value may represent a single cuw value and/or a single cw value, and the delta value may represent a constant value.

For example, when 2 cuvs are used in the current block, the two curvature parameters may be default_cuv and default_cuv+delta_cuv.

For example, when N cuvs are used in the current block, the N curvature parameters may be defaultjcuv, defaultjcuv+delta_cuv, defaultjcuv+2 x delta_cuv, …, and defaultjcuv+ (N-1) x delta_cuv. (where N is a positive integer greater than 2).

For example, when 2n+1 cuv are used in the current block, 2n+1 curvature parameters may be default_cuv, default_cuv+delta_cuv, default_cuv-delta_cuv, default_cuv+2 x delta_cuv, default_cuv-2 x delta_cuv, …, default_cuv+n x delta_cuv, and default_cuv-N x delta_cuv. (where N is a positive integer greater than 1).

For example, when two cw are used in the current block, the two weight parameters may be default_cw and default_cw+delta_cw. (here, default_cw+delta_cw is the sum of the element units of the vector).

For example, when M cws are used in the current block, the M weight parameters may be default_cw, default_cw+delta_cw, default_cw+2×delta_cw, …, and default_cw+ (M-1) ×delta_cw. (here, default_cw+delta_cw is the sum of the element units of the vector, and M is a positive integer greater than 2).

For example, when 2m+1 cws are used in the current block, 2m+1 curvature parameters may be default_cw, default_cw+delta_cw, default_cw-delta_cw, default_cw+2×delta_cw, default_cw-2×delta_cw, …, default_cw+m×delta_cw, and default_cw-m×delta_cw. (where M is a positive integer greater than 1).

The information about cuv and/or cw described above may be encoded as a bitstream or may be decoded from a bitstream. Alternatively, the encoder or decoder may share and store information about the number of cuv and/or cw or the values of cuv and/or cw in a format such as a look-up table.

When srp_flag is equal to 1, residual signal prediction may be performed using an intra prediction mode determined for a current block or sub-block.

The intra prediction information may be entropy encoded/decoded from the bitstream based on at least one of the encoding parameters. For example, ndip_flag may be encoded/decoded based on block partition information.

For example, if at least one of the split_ flag, quadtree _flag and the binaryrate_flag is "0" and thus the block is not further divided, the ndip_flag may be encoded/decoded.

For example, if the binarytre_flag is 1, the ndip_flag may not be encoded/decoded.

At least one of the pieces of intra prediction information described above may not be signaled based on at least one of the size and shape of the block.

For example, if the size of the current block is a predetermined size, at least one piece of intra prediction information regarding the current block may not be signaled, and at least one piece of information regarding intra prediction corresponding to the size of a previously encoded/decoded upper layer block may be used.

For example, if the current block is shaped as a rectangle, at least one piece of intra prediction information regarding the current block may not be signaled, and at least one piece of information regarding intra prediction corresponding to the size of a previously encoded/decoded upper layer block may be used.

If the srp_flag is 1, the residual signal may be predicted using an intra prediction mode determined for the current block or sub-block.

When at least one of the pieces of intra prediction information is entropy encoded/decoded, at least one of the following binarization methods may be used.

Truncated rice binarization method

K-order exponential Columbus binarization method

Finite K-order exponential Columbus binarization method

Fixed length binarization method

-unitary binarization method

Truncated unary binarization method

Now, the reference sample construction step S1220 will be described in detail.

In intra prediction of a current block or a sub-block having a smaller size and/or shape than the current block based on the derived intra prediction mode, a reference sample may be constructed for the prediction. The following description is given in the context of the current block, and the current block may mean a sub-block. The reference samples may be constructed using one or more reconstructed samples or combinations of samples that are adjacent to the current block. In addition, filtering may be applied when constructing the reference samples. Here, the reference sample point may be constructed using each reconstructed sample point on the plurality of reconstructed sample points as it is. Alternatively, the reference samples may be constructed after filtering between samples on the same reconstructed sample line. Alternatively, the reference samples may be constructed after filtering between samples on different reconstructed sample lines. The constructed reference samples can be represented by ref [ m, n ], and the reconstructed neighboring samples or samples obtained by filtering the reconstructed neighboring samples can be represented by rec [ m, n ]. Here, m or n may be a predetermined integer value. In the case where the size of the current block is W (horizontal) ×h (vertical), if the upper-left-most sample point of the current block is (0, 0), the relative position of the upper-left-most reference sample point closest to the sample point may be set to (-1, -1).

As shown in fig. 32, one or more reconstructed sample lines adjacent to the current block may be used to construct a reference sample.

For example, one of the plurality of reconstructed sample lines shown in fig. 32 may be selected, and the reference sample point may be constructed using the selected reconstructed sample line. A predetermined one of the plurality of reconstructed sample lines may be fixedly selected as the selected reconstructed sample line. Alternatively, a specific one of the plurality of reconstructed sample lines may be adaptively selected as the selected reconstructed sample line. In this case, an indicator for the selected reconstructed sample line may be signaled.

For example, one or more of the plurality of reconstructed sample lines shown in fig. 32 may be used in combination to construct a reference sample. For example, the reference sample may be constructed as a weighted sum (or weighted average) of one or more reconstructed samples. Weights for weighted averaging may be assigned based on distance from the current block. Here, a shorter distance from the current block may be assigned a larger weight. For example, the following [ equation 15] may be used.

[ equation 15]

ref[-1,-1]＝(rec[-2,-1]+2*rec[-1,-1]+rec[-1,-2]+2)>>2

ref[x,-1]＝(rec[x,-2]+3*rec[x,-1]+2)>>2,(x＝0～W+H-1)

ref[-1,y]＝(rec[-2,y]+3*rec[-1,y]+2)>>2,(y＝0～W+H-1)

Alternatively, the reference sample may be constructed using at least one of an average value, a maximum value, a minimum value, a median value, and a most frequent value of the plurality of reconstructed samples based on at least one of a distance from the current block or an intra prediction mode.

Alternatively, the reference samples may be constructed based on changes (variations) in the values of a plurality of successive reconstructed samples. For example, the reference sample may be constructed based on at least one of whether a difference between values of two consecutive reconstructed samples is equal to or greater than a threshold value, whether the values of the two consecutive reconstructed samples change continuously or discontinuously, and so on. For example, if the difference between rec < -1 > and rec < -2 > -1 > is equal to or greater than a threshold value, ref < -1 > may be determined as rec < -1 >, or a value obtained by applying a weighted average using a predetermined weight assigned to rec < -1 >. For example, if the values of a plurality of consecutive reconstructed samples are changed by n each time as the plurality of consecutive reconstructed samples are closer to the current block, the reference samples ref < -1 > -1] may be determined as rec < -1 > -n.

At least one of the number and the position of the reconstructed sample lines and the construction method for constructing the reference sample points may be differently determined according to whether an upper boundary or a left side boundary of the current block corresponds to a boundary of at least one of a picture, a slice, a parallel block, and a Coding Tree Block (CTB).

For example, when the reconstructed sample lines 1 and 2 are used to construct the reference sample points, when the upper boundary of the current block corresponds to the CTB boundary, the reconstructed sample line 1 may be used for the upper side and the reconstructed sample lines 1 and 2 may be used for the left side.

For example, when the reconstructed sample lines 1 to 4 are used to construct the reference sample points, when the upper boundary of the current block corresponds to the CTB boundary, the reconstructed sample lines 1 and 2 may be used for the upper side and the reconstructed sample lines 1 to 4 may be used for the left side.

For example, when the reconstructed sample line 2 is used to construct a reference sample point, when the upper boundary of the current block corresponds to the CTB boundary, the reconstructed sample line 1 may be used for the upper side and the reconstructed sample line 2 may be used for the left side.

One or more reference sample lines may be constructed by the above process.

The reference sample construction method of the upper side of the current block may be different from the reference sample construction method of the left side of the current block.

Information indicating that a reference sample has been constructed using at least one of the above methods may be encoded/decoded. For example, information indicating whether or not a plurality of reconstructed sample lines are used may be encoded/decoded.

If the current block is divided into a plurality of sub-blocks, and each sub-block has an independent intra prediction mode, a reference sample may be constructed for each sub-block.

As shown in fig. 33, if the size of the current block is 16×16 and 16 4×4 sub-blocks have independent intra prediction modes, a reference sample for each sub-block may be constructed according to at least one of the following methods according to a scanning scheme for predicting the sub-block.

For example, a reference sample point may be constructed for each sub-block using N reconstructed reference sample lines adjacent to the current block. In the example shown in fig. 33, N is 1.

For example, in the case where a plurality of sub-blocks are predicted in raster scan order of 1- >2- >3- > … - >15- >16, a reference sample for the kth sub-block may be constructed using a sample of at least one of the left sub-block, upper Fang Zikuai, upper right sub-block, and lower left sub-block that has been encoded/decoded.

For example, in the case where a plurality of sub-blocks are predicted in a zigzag scanning order of 1- >2- >5- >6- >3- >4- >7- > … - >15- >16, a sample of at least one of the left sub-block, the upper Fang Zikuai, the upper right sub-block, and the lower left sub-block, which have been encoded/decoded, may be used to construct a reference sample for the kth sub-block.

For example, in the case of predicting a plurality of sub-blocks in a zig-zag scan order of 1- >2- >5- >9- >6- >3- >4- > … - >15- >16, a sample point for at least one of the left sub-block, the upper Fang Zikuai, the upper right sub-block, and the lower left sub-block, which have been encoded/decoded, may be used to construct a reference sample point for the kth sub-block.

For example, in the case where a plurality of sub-blocks are predicted in the vertical scan order of 1- >5- >9- >13- >2- >6- > … - >12- >16, a sample of at least one of the left sub-block, the upper Fang Zikuai, the upper right sub-block, and the lower left sub-block, which have been encoded/decoded, may be used to construct a reference sample for the kth sub-block.

In case that a plurality of sub-blocks are predicted in a scan order other than the above scan order, a reference sample for a kth sub-block may be constructed using a sample of at least one of the left sub-block, the upper Fang Zikuai, the upper right sub-block, and the lower left sub-block that have been encoded/decoded.

In selecting the reference sample point, a decision and/or a filling regarding availability of the blocks comprising the reference sample point may be performed. For example, if a block including a reference sample is available, the reference sample may be used. Meanwhile, if a block including a reference sample is not available, the unavailable reference sample may be replaced with one or more available neighboring reference samples by padding.

If the reference sample exists outside at least one of the picture boundary, the parallel block boundary, the slice boundary, the CTB boundary, and the predetermined boundary, it may be determined that the reference sample is not available.

In the case where the current block is encoded by CIP (constrained intra prediction), if a block including a reference sample is encoded/decoded according to an inter prediction mode, it may be determined that the reference sample is not available.

If it is determined that the neighboring reconstructed sample point is not available, the available neighboring reconstructed sample point may be used to replace the unavailable sample point. For example, as shown in fig. 34, in the case where there are available and unavailable spots, one or more available spots may be used to replace the unavailable spot.

The sample values of the unavailable samples may be replaced with the sample values of the available samples in a predetermined order. Available samples adjacent to an unavailable sample may be used to replace the unavailable sample. In the case where there are no adjacent available samples, the first occurring available sample or the nearest available sample may be used. The replacement order of unavailable samples may be the order bottom left-most to top right-most. Alternatively, the order of replacement of unusable samples may be the order of top right to bottom left. Alternatively, the replacement order of unavailable samples may be an order of top left-most to top right-most and/or bottom left-most. Alternatively, the replacement order of unavailable samples may be an order of top right and/or bottom left to top left.

As shown in fig. 34, the unusable samples may be replaced in order from the leftmost lower sample position 0 to the rightmost upper sample. In this case, the values of the first four unavailable samples may be replaced with the value of the first occurring or closest available sample a. The value of the last available sample b can be used to replace the value of the next 13 unavailable samples.

Alternatively, combinations of available samples may be used to replace unavailable samples. For example, the unavailable sample may be replaced with an average of available samples adjacent to both ends of the unavailable sample. For example, in fig. 34, the first four unavailable samples may be replaced with the value of available sample a, and the next 13 unavailable samples may be replaced with the average of available sample b and available sample c. Alternatively, the 13 unavailable samples may be filled with any value between the value of available sample b and the value of available sample c. In this case, the unavailable samples may be replaced with different values. For example, since the unavailable sample point is closer to the available sample point a, the value of the unavailable sample point may be replaced with a value close to the value of the available sample point a. Similarly, since the unavailable sample point is closer to the available sample point b, the value of the unavailable sample point may be replaced with a value close to the value of the available sample point b. That is, the value of the unavailable spot may be determined based on the distance from the unavailable spot to the available spots a and/or b.

To replace the unavailable spot, one or more of a plurality of methods including the above method may be selectively applied. The method for replacing the unavailable samples may be signaled by information included in the bitstream, or a method predetermined by an encoder and a decoder may be used. Alternatively, the method for replacing the unavailable spot may be deduced through a predetermined scheme. For example, the method for replacing unavailable samples may be selected based on the difference between the values of available samples a and b and/or the number of unavailable samples. For example, a method for replacing an unavailable sample may be selected based on a comparison between a difference between values of two available samples and a threshold and/or a comparison between a number of unavailable samples and a threshold. For example, if the difference between the values of two available samples is greater than a threshold and/or if the number of unavailable samples is greater than a threshold, the value of the unavailable sample may be replaced with a different value.

For the constructed one or more reference samples, it may be determined whether to apply filtering according to at least one of an intra prediction mode, a size, and a shape of the current block. If filtering is applied, different filter types may be used according to at least one of intra prediction mode, size, and shape of the current block.

For example, for each of the plurality of reference sample lines, it may be determined differently whether to apply filtering and/or filter type. For example, filtering may be applied to a first adjacent line, while filtering may not be applied to a second adjacent line.

For example, both filtered and unfiltered values may be used for the reference samples.

For example, if the intra prediction mode intropredmode of the target decoded block is a direction prediction mode, the smaller of the difference between the direction prediction mode and the vertical direction mode (given 33 direction modes, of which the index is 26) and the difference between the direction prediction mode and the horizontal direction mode (given 33 direction modes, of which the index is 10) [ mindistverhor=min (Abs (intropredmode-26), abs (intropredmode-10)) ] may be calculated.

For example, the threshold may be 10 for a 4×4 block, 7 for an 8×8 block, 1 for a 16×16 block, 0 for a 32×32 block, and 10 for a 64×64 block.

If minDistVerHor is greater than a threshold assigned to the corresponding block size (minDistVerHor > interhorverdistthresh), filtering may be performed, and if minDistVerHor is equal to or less than the threshold, filtering may not be performed.

Fig. 36 is an exemplary diagram illustrating whether filtering is performed according to a block size and/or an inter prediction mode. In fig. 36, X may represent no filtering and O may represent filtering.

For example, as shown in fig. 36, whether to apply filtering to a block having 8 may be determined according to a block size and/or a direction prediction mode<＝N _S <Target encoding/decoding block=32.

In another embodiment where filtering is applied, bilinear interpolation filtering may be performed on target encoding/decoding blocks having a size. For example, N may be used for block size _S Is a target encoding/decoding block of (1), a second order differential value is calculated in the vertical direction and the horizontal direction. If the second order differential value is less than a predetermined threshold (e.g., |p _ref (-1,-1)-2×p _ref (N _S /2,-1)+p _ref (N _S ,-1)|<2 ^Bd-Ca And/or |p _ref (-1,-1)-2×p _ref (-1,N _S /2)+p _ref (-1,N _S )|<2 ^Bd-C (B _d Is as satisfying 1 <＝C<＝B _d Positive integer bit depth of) of (a), then for y=0, …, N _S -1, the reference pixel p can be derived _ref (-1,y)＝(N _s -1-y)/N _S ×p _ref (-1,-1)+(y+1)/N _S ×p _ref (-1,N _S -1), or for x=0, …, N _S -1, p can be deduced _ref (x,-1)＝(N _s -1-x)/N _S ×p _ref (-1,-1)+(x+1)/N _S ×p _ref (N _S -1,-1)。

For example, at least one different filter among a 3-tap filter, a 5-tap filter, a 7-tap filter, and an N-tap filter (N is a positive integer) may be selected and applied according to at least one of an intra prediction mode, a size, and a shape of the current block.

Hereinafter, the step of performing intra prediction (S1230) will be described in detail.

Intra prediction may be performed for a current block or sub-block based on the derived intra prediction mode and the reference sample. In the following description, a current block may mean a sub-block.

For example, non-directional intra prediction may be performed. The non-directional intra prediction mode may be at least one of a DC mode and a planar mode.

Intra prediction in DC mode may be performed using an average of one or more of the constructed reference samples. Filtering may be applied to one or more prediction samples on the boundary of the current block. DC mode intra prediction may be adaptively performed according to at least one of a size and a shape of a current block.

For example, as shown in (a) of fig. 37, if the current block is shaped as a square, the current block may be predicted using an average value of reference samples above and to the left of the current block.

For example, as shown in (b) of fig. 37, if the current block is shaped as a rectangle, the current block may be predicted using an average value of reference samples adjacent to the longer one of the width and the length of the current block.

For example, if the size of the current block falls within a predetermined range, a predetermined sample is selected from among upper or left reference samples of the current block, and prediction may be performed using an average value of the selected samples.

For example, DC mode intra prediction may be performed using an average of one or more of the constructed reference samples. For example, the following [ equation 16] may be used.

[ equation 16]

Filtering may be applied to one or more prediction samples located on the boundary of the current block. For example, filtering may be performed for N columns and/or rows to the left and/or above the current block. N may be a positive integer greater than 1.

Fig. 38 is a diagram depicting filtering for DC-mode intra prediction.

For example, if n=1, filtering may be performed for one upper row and/or one left column of the target block, as shown in fig. 38. The filtering may be performed by the following equation 17.

[ equation 17]

Planar mode intra prediction may be performed by calculating a weighted sum considering distances from one or more constructed reference samples according to the location of a target intra prediction sample of the current block.

For example, depending on the location (x, y) of the target prediction sample, the prediction block may be calculated as a weighted sum of N reference samples. N may be a positive integer, for example, 4.

Fig. 39 is a diagram depicting planar mode intra prediction.

For example, as shown in fig. 39, if each of the width and length of the target encoding/decoding block is 4 (N _S =4), the predicted pixel value at each pixel position (x, y) can be derived as a weighted sum of the upper reference pixel c, the left reference pixel b, the upper right pixel d of the target encoding/decoding block, and the lower left pixel a of the target encoding/decoding block. For example, one can go through [ equation 18]The weighted sum is derived.

[ equation 18]

For example, directional intra prediction may be performed. The direction prediction mode may be at least one of a horizontal mode, a vertical mode, and a mode having a predetermined angle.

Horizontal/vertical mode intra prediction may be performed using one or more reference samples on horizontal/vertical lines at the location of the target intra prediction sample.

Intra prediction in a mode having a predetermined angle may be performed using one or more reference samples on a line at a predetermined angle with respect to the position of the target intra prediction sample. Here, N reference samples may be used. N may be a positive integer such as 2, 3, 4, 5, or 6. Further, prediction may be performed by applying an N-tap filter (such as a 2-tap, 3-tap, 4-tap, 5-tap, or 6-tap filter), for example.

For example, intra prediction may be performed based on the location information. The location information may be encoded/decoded and the reconstructed sample block at the location may be derived as an intra-prediction block for the current block. Alternatively, a block similar to the current block detected by the decoder may be derived as an intra prediction block for the current block.

For example, intra color component prediction may be performed. For example, intra prediction may be performed for a chroma component using a reconstructed luma component of a current block. Alternatively, intra prediction may be performed for one reconstructed chroma component Cb of the current block with respect to another chroma component Cr.

Intra prediction may be performed by using one or more of the above-described various intra prediction methods in combination. For example, an intra-prediction block may be constructed for a current block by a weighted sum of a block predicted using a predetermined non-directional intra-prediction mode and a block predicted using a predetermined directional intra-prediction mode. Here, different weights may be applied according to at least one of an intra prediction mode, a block size, a shape, and/or a sample position of the current block.

Alternatively, regarding the combined use of one or more intra prediction modes, a prediction block may be constructed using a weighted sum of values predicted by using the intra prediction mode of the current block and values predicted by using a predetermined mode included in the MPM list.

Alternatively, intra prediction may be performed using one or more sets of reference samples. For example, intra prediction may be performed for the current block using a weighted sum of the following blocks: blocks predicted intra-frame using reference samples that are not obtained by applying filtering to the constructed reference samples, and blocks predicted intra-frame using reference samples that are obtained by applying filtering to the constructed reference samples.

In intra prediction, a filtering operation may be performed using neighboring reconstructed samples. Here, the filtering operation may or may not be performed according to at least one of an intra prediction mode, a block size, a shape, and/or a sample position of the current block. The filtering operation may be included in the intra prediction process and thus performed as one step. At least one of a filter tap, coefficients, the number of lines to be applied, and the number of samples to be applied may be adaptively determined based on at least one of an intra prediction mode, a block size, and a shape of a current block when filtering is performed.

When intra prediction is performed by dividing a current block into sub-blocks and deriving an intra prediction mode of each sub-block using intra prediction modes of neighboring blocks, filtering may be applied to each sub-block of the current block. For example, a low pass filter may be applied to the entire current block. Alternatively, a filter may be applied to samples located on the boundary of each sub-block. Alternatively, a filter may be applied to the prediction block or reconstruction block of each sub-block, and one or more samples of the sub-block to which the filter is applied may be used in intra prediction of a subsequent sub-block.

When the current block is divided into sub-blocks and intra prediction is performed for each sub-block, each sub-block may refer to at least one of an encoding/decoding block, a prediction block, and a transform block. For example, if the size of the current block is 64×64 and the size of the sub-block is 16×16, an intra prediction mode, which is a prediction block of each sub-block, may be derived and/or intra prediction may be performed for the prediction block. If each of the one or more sub-blocks is further divided into 8×8 or 4×4 blocks, each of the 8×8 or 4×4 blocks may be a transform block, and intra prediction may be performed for a block obtained by the further division using an intra prediction mode of the 16×16 block.

In directional intra prediction, a current block may be encoded/decoded using at least one directional mode of N directional modes. Here, N may be a positive integer such as 33 or 65. In this case, each directional intra prediction mode may have a predetermined angle value.

In the directional intra prediction, a current block may be encoded/decoded using a directional mode in units of M samples. M may be a positive integer. In this case, the direction mode in units of samples may mean a mode in which one or more prediction target sample units in the current block are predicted using one or more direction intra prediction modes.

In directional intra prediction, a constructed reference sample may be reconstructed from a directional prediction mode. For example, if the direction prediction mode is a mode in which all left and upper reference samples are used, a unidirectional array may be constructed using the left or upper reference samples.

For example, as shown in fig. 40, one or more left side reference samples may be used to construct a 1D array of upper reference samples. The upper reference sample may be constructed using different ones of the left samples according to the direction pattern. The upper reference sample may be constructed by moving the left reference sample, or the upper reference sample may be constructed by using a weighted sum of one or more left reference samples.

In directional intra prediction, real-based interpolation prediction may be performed. For example, as described below, an offset (iIdx) and/or a weight (iFact) for predicting the interpolation of a sample may be determined from the position of the sample in the current block based on an angle parameter (inprepagle) corresponding to each direction prediction mode.

For example, assuming interpolation is performed in units of 1/32pel, the offset and weight for the direction pattern having the vertical direction can be determined by the following [ equation 19 ].

[ equation 19]

iIdx＝((y+1)*intraPredAngle)>>5

iFact＝((y+1)*intraPredAngle)&31

Can be according to [ equation 19]]The value of iFact in (c) to determine the different predicted sample values. For example, if iFact is not 0, reference point P _1,ref Not based on the position of integers (integer points), but based on the position of real numbers. Thus, the following [ equation 20] can be passed]A plurality of reference samples (e.g., two left and right adjacent reference samples) adjacent to the real position are used to generate a predicted sample value at the target sample position (x, y). Here, the adjacent plurality of reference samples may be 4 or 6 left and right adjacent reference samples.

[ equation 20]

predSamples[x]ly]＝((32-iFact)*p _1，ref [x+iIdx+1]+iFact*p _1，ref [x+iIdx+2]+16)>>5

For example, if iFact is 0, it can be obtained by the following [ equation 21 ]]To generate predicted sample values. Alternatively, reference point P may be used _1,ref And left and right reference samples to apply 3 taps [1/4:2/4:1/4 ]]A filter.

[ equation 21]

predSamples[x][y]＝p _1,ref [x+iIdx+1]

In case of at least one of the horizontal mode and/or the vertical mode in the direction prediction mode, filtering may not be performed with respect to the reference samples. In addition, the reference sample may not require interpolation prediction. Furthermore, since prediction can be performed only with the upper or left reference sample, a process of constructing a 1D array for the reference sample may not be required.

As shown in fig. 41, in the directional intra prediction, a unit to which a directional mode is applied may be changed. That is, prediction may be performed using one or more direction modes based on at least one of a sample unit, a sample group unit, and a line unit in the target block.

For example, prediction may be performed based on the current block using a directional mode. Alternatively, prediction may be performed using a direction mode for each target sample line used for prediction in the current block. That is, prediction may be performed using a different directional pattern for each of at least one line of a horizontal line or a vertical line of the current block. Alternatively, prediction may be performed using a direction pattern for each predetermined target sample group used for prediction in the current block. That is, prediction may be performed using a different direction mode for each group including N samples in the current block. For example, prediction may be performed using a direction mode for each target sample in the current block for prediction. That is, prediction may be performed using a different direction pattern for each target sample for prediction in the current block.

When one direction prediction is performed in units of a current block, if the encoding target block includes a curve having a plurality of image characteristics, encoding efficiency may be reduced. In order to improve coding efficiency, prediction may be performed using one or more direction modes in units of at least one of a sample, a sample group, and a line inside a target block.

Fig. 41 (a) shows an exemplary case in which a different direction pattern is used for each sample in the target block. In the example shown in (a) of fig. 41, a predicted value may be generated for each sample using a reference sample located at an angle of each direction pattern.

Fig. 41 (b) shows an exemplary case in which a different direction pattern is used for each horizontal line in the target block. In the example shown in (b) of fig. 41, a predicted value may be generated for each horizontal line using a reference sample point located at an angle of each direction pattern.

Fig. 41 (c) shows an exemplary case in which a different direction pattern is used for each vertical line in the target block. In the example shown in (c) of fig. 41, a predicted value may be generated for each vertical line using a reference sample point located at an angle of each direction pattern.

Fig. 41 (d) shows an exemplary case where a different direction pattern is applied to each sample group in the diagonal direction in the target block. In the example shown in (d) of fig. 41, a predicted value may be generated for each diagonal sample group in the diagonal direction using a reference sample located at an angle of each direction pattern.

Fig. 41 (e) shows an exemplary case in which a different direction pattern is used for each L-shaped line in the target block. In the example shown in (e) of fig. 41, a predicted value may be generated for each right angle line using a reference sample point located at an angle of each direction pattern.

In addition to the example of fig. 41, various methods for grouping one or more samples may be used. For example, a different directional pattern may be applied to each block generated by dividing the height and/or width by an arbitrary number.

As an example of using different direction modes in units of samples in the target block, a curvature parameter cuv and a weight parameter cwi (i=0, 1, …, N _S -1，N _S : block size) to determine the location of the reference pixel used to generate the predicted value for any location (x, y) within the predicted block.

For example, in the case of intra prediction of "from upper right side to lower left side", the position of a reference pixel for generating a predicted value of an arbitrary position (x, y) within a predicted block may be determined in line units as in the following equation 22.

[ equation 22]

Line y→p (pos, -1), where pos=x+1/2 (x+y) +cw _y *cuv*(x+1)，

Wherein y=0, 1, …, N _s -1

In the case of intra prediction using equation 22 above, the magnitude of curvature may be adjusted by adjusting cuv. cuv may be a real number equal to or greater than zero. For example, the larger the cuv value becomes, the larger the curvature becomes, and the position of the reference pixel can be moved rightward. Alternatively, the smaller the cuv value becomes, the smaller the curvature becomes, and the position of the reference pixel can be moved leftward (up to the position of x).

When the target block is NxM, cw _i There may be a parameter comprising N weights, wherein the parameter refers to the height of the block or the number of rows. Each weight may be a real number equal to or greater than zero. Can be achieved by adjusting cw _i To adjust the positions of reference pixels used by the predicted pixels included in the corresponding rows. For example, cw _i The larger the value becomes, the position of the reference pixel used by the predicted pixel of the i-th row may be shifted rightward. Alternatively, cw _i The smaller the value becomes, the position of the reference pixel can be moved to the left (up to the position of x).

Thus, the curvature parameter cuv and/or the weight row parameter cw can be calculated by _i Combinations are made to perform various forms of intra prediction.

To generate the predicted pixel values, the described interpolation prediction in 1/N pel units may be performed. N may be a positive integer.

For example, when the position of the reference pixel of the prediction target pixel is a positive integer unit, the prediction target pixel predSamples [ x ] [ y ] may be derived from the reference pixel p (pos, -1). pos may refer to the location of the reference pixel.

For example, when the position of the reference pixel of the prediction target pixel is a real inter unit, predSamples [ x ] [ y ] may be derived as an interpolation prediction number obtained by performing interpolation prediction in units of 1/N pel on p (floor (pos), -1) and p (ceil (pos), -1). floor (pos) may be an integer value equal to or less than pos, and may refer to a maximum value. ceil (pos) may be an integer equal to or greater than pos, and may refer to a minimum value.

As described above, for computational convenience, p may be added prior to generating the predicted sample value _ref Conversion to p _1,ref 。

When the positions pos of the reference pixels exceed the maximum range of available reference pixels (P (7, -1) is exceeded in fig. 42), the calculated positions of all the reference pixels can be used after being converted into respective normalized values by matching with the maximum range of available reference pixels.

For example, in the case of intra prediction of "from the upper left side to the lower right side (type-1)", the position of the reference pixel for generating the prediction value of an arbitrary position (x, y) within the prediction block may be determined in line units as in the following equation 23.

[ equation 23]

Line y→p (pos, -1), where pos=x-1/2 (N _s -1-x+y)-cw _y *cuv*(N _s -x)，

Wherein y=0, 1, …, N _s -1

In the case of intra prediction using equation 23 above, the magnitude of curvature may be adjusted by adjusting cuv. cuv may be a real number equal to or greater than zero. For example, the larger the cuv value becomes, the larger the curvature becomes, and the position of the reference pixel can be moved leftward. Alternatively, the smaller the cuv value becomes, the smaller the curvature becomes, and the position of the reference pixel may be moved rightward (up to the position of x).

When the target block is NxM, cw _i May be a bagParameters including N weights, where the parameters refer to the height of a block or the number of rows. Each weight may be a real number equal to or greater than zero. Can be achieved by adjusting cw _i To adjust the positions of reference pixels used by the predicted pixels included in the corresponding rows. For example, cw _i The larger the value becomes, the position of the reference pixel used by the predicted pixel of the i-th row may be shifted leftward. Alternatively, cw _i The smaller the value becomes, the position of the reference pixel can be moved rightward (up to the position of x).

Thus, the curvature parameter cuv and/or the weight row parameter cw can be calculated by _i Combinations are made to perform various forms of warped intra prediction.

For example, when the position of the reference pixel of the prediction target pixel is a positive integer unit, the prediction target pixel predSamples [ x ] [ y ] may be derived as the reference pixel p (pos, -1).

For example, when the position of the reference pixel of the prediction target pixel is a real unit, predSamples [ x ] [ y ] may be derived as an interpolation prediction value obtained by performing interpolation prediction in units of 1/N pel on p (floor (pos), -1) and p (ceil (pos), -1).

When the positions pos of the reference pixels exceed the maximum range of available reference pixels (in fig. 29, P (-7, -1)) the calculated positions of all reference pixels can be used after being converted into respective normalized values by matching with the maximum range of available reference pixels.

For example, in the case of intra prediction from the lower left side to the upper right side, the position of the reference pixel for generating the prediction value of an arbitrary position (x, y) within the prediction block may be determined according to the column unit as in the following equation 24.

[ equation 24]

Column x→p (-1, pos), where pos=y+1/2 (x+y) +cw _x *cuv*(y+1)，

Wherein x=0, 1, …, N _s -1

In the case of intra prediction using equation 24 above, the magnitude of curvature may be adjusted by adjusting cuv. cuv may be a real number equal to or greater than zero. For example, the larger the cuv value becomes, the larger the curvature becomes, and the position of the reference pixel may be moved downward. Alternatively, the smaller the cuv value becomes, the smaller the curvature becomes, and the position of the reference pixel may be moved upward (up to the position of y).

When the target block is NxM, cw _i There may be parameters including M weights, where the parameters refer to the width of the block or the number of columns. Each weight may be a real number equal to or greater than zero. Can be achieved by adjusting cw _i To adjust the position of the reference pixel used by the predicted pixel included in the corresponding column. For example, cw _i The greater the value becomes, the position of the reference pixel used by the predicted pixel of the i-th column may be moved downward. Alternatively, cw _i The smaller the value becomes, the position of the reference pixel can be moved upward (up to the position of y).

Thus, the curvature parameter cuv and/or the weight train parameter cw can be calculated by _i Combinations are made to perform various forms of intra prediction.

For example, when the position of the reference pixel of the prediction target pixel is an integer unit, the prediction target pixel predSamples [ x ] [ y ] may be derived as the reference pixel p (-1, pos).

For example, when the position of the reference pixel of the prediction target pixel is a real unit, predSamples [ x ] [ y ] may be derived as an interpolation prediction value obtained by performing interpolation prediction in units of 1/N pel on p (-1, floor (pos)) and p (-1, ceil (pos)).

When the position pos of the reference pixel exceeds the maximum range of available reference pixels (P (-1, 7) is exceeded in fig. 46), the calculated positions of all the reference pixels can be used after being converted into the respective normalized values by matching with the maximum range of available reference pixels.

For example, in the case of intra prediction of "from the upper left side to the lower right side (type-2)", the position of the reference pixel for generating the prediction value of any position (x, y) within the prediction block may be determined in accordance with the column unit as in the following equation 25.

[ equation 25]

Column x→p (-1, pos), where pos=y-1/2 (N _s -1-y+x)-cw _x *cuv*(N _s -y)，

Wherein x=0, 1, …, N _s -1

In the case of intra prediction using the above equation 25, the magnitude of curvature may be adjusted by adjusting cuv. cuv may be a real number equal to or greater than zero. For example, the larger the cuv value becomes, the larger the curvature becomes, and the position of the reference pixel may be moved upward. Alternatively, the smaller the cuv value becomes, the smaller the curvature becomes, and the position of the reference pixel may be moved downward (up to the position of y).

When the target block is NxM, cw _i There may be parameters including M weights, where the parameters refer to the width of the block or the number of columns. Each weight may be a real number equal to or greater than zero. Can be achieved by adjusting cw _i To adjust the position of the reference pixel used by the predicted pixel included in the corresponding column. For example, cw _i The greater the value becomes, the position of the reference pixel used by the predicted pixel of the i-th column may be moved upward. Alternatively, cw _i The smaller the value becomes, the position of the reference pixel can be moved downward (up to the position of y).

When the positions pos of the reference pixels exceed the maximum range of available reference pixels (in fig. 48, P (-1, -7)) the calculated positions of all reference pixels can be used after being converted into respective normalized values by matching with the maximum range of available reference pixels.

For example, in the case of intra prediction of "upper side to lower left side", the position of a reference pixel for generating a predicted value of an arbitrary position (x, y) within a predicted block may be determined in line units as in the following equation 26.

[ equation 26]

Line y→p (pos, -1), where pos=x+cw _y * cuv x y, where y=0, 1, …, N _s -1 in case of intra prediction using equation 26 above, the magnitude of curvature can be adjusted by adjusting cuv. cuv may be a real number equal to or greater than zero. For example, the larger the cuv value becomes, the larger the curvature becomes, and the position of the reference pixel can be moved rightward. Alternatively, the smaller the cuv value becomes, the smaller the curvature becomes, and the position of the reference pixel can be moved leftward (up to the position of x).

Fig. 50 is a view showing that by applying cuv=0.6, cw to a current block of size 4×4 ₀ ＝1.0、cw ₁ ＝1.4、cw ₂ =1.8 and cw ₃ An illustration of an embodiment in which prediction is performed in the direction from the upper side to the lower left side, =2.2.

FIG. 51 is a graph showing cuv and cw as application graph 50 _i Results of (3)Is a diagram of an embodiment of the location of reference pixels used by predicted pixels within a current block.

For example, when the position of the reference pixel of the prediction target pixel is an integer unit, the prediction target pixel predSamples [ x ] [ y ] may be derived as the reference pixel p (pos, -1).

When the positions pos of the reference pixels exceed the maximum range of available reference pixels (P (7, -1) is exceeded in fig. 50), the calculated positions of all the reference pixels can be used after being converted into respective normalized values by matching with the maximum range of available reference pixels.

For example, in the case of intra prediction of "upper side to lower right side", the position of a reference pixel for generating a predicted value of an arbitrary position (x, y) within a predicted block may be determined in line units as in the following equation 27.

[ equation 27]

Line y→p (pos, -1), where pos=x-cw _y * cuv x y, where y=0, 1, …, N _s -1

In the case of intra prediction using equation 27 above, the magnitude of curvature may be adjusted by adjusting cuv. cuv may be a real number equal to or greater than zero. For example, the larger the cuv value becomes, the larger the curvature becomes, and the position of the reference pixel can be moved leftward. Alternatively, the smaller the cuv value becomes, the smaller the curvature becomes, and the position of the reference pixel may be moved rightward (up to the position of x).

When the target block is NxM, cw _i May be a set of N weightsWherein the parameter refers to the height of the block or the number of rows. Each weight may be a real number equal to or greater than zero. Can be achieved by adjusting cw _i To adjust the positions of reference pixels used by the predicted pixels included in the corresponding rows. For example, cw _i The larger the value becomes, the position of the reference pixel used by the predicted pixel of the i-th row may be shifted leftward. Alternatively, cw _i The smaller the value becomes, the position of the reference pixel can be moved rightward (up to the position of x).

For example, when the position of the reference pixel of the prediction target pixel is an integer unit, the prediction target pixel predSamples [ x ] [ y ] may be derived as p (pos, -1).

When the positions pos of the reference pixels exceed the maximum range of available reference pixels (P (-7, -1) is exceeded in fig. 52), the calculated positions of all the reference pixels can be used after being converted into respective normalized values by matching with the maximum range of available reference pixels.

For example, in the case of intra prediction of "from left side to upper right side", the position of a reference pixel for generating a predicted value of an arbitrary position (x, y) within a predicted block may be determined according to a column unit as in equation 28 below.

[ equation 28]

Column x→p (pos, -1), where pos=y+cw _x * cuv x, where x=0, 1, …, N _s -1 in case of intra prediction using equation 28 above, the magnitude of curvature can be adjusted by adjusting cuv. cuv may be a real number equal to or greater than zero. For example, the larger the cuv value becomes, the larger the curvature becomes, and the position of the reference pixel may be moved downward. Alternatively, the smaller the cuv value becomes, the smaller the curvature becomes, and the position of the reference pixel may be moved upward (up to the position of y).

Thus, the curvature parameter cuv and/or the weight train parameter cw can be calculated by _i Combinations are made to perform various forms of warped intra prediction.

When the position pos of the reference pixel exceeds the maximum range of available reference pixels (P (-1, 7) is exceeded in fig. 54), the calculated positions of all the reference pixels can be used after being converted into respective normalized values by matching with the maximum range of available reference pixels.

For example, in the case of intra prediction of "from left side to right lower side", the position of a reference pixel for generating a predicted value of an arbitrary position (x, y) of a predicted block may be determined according to a column unit as in equation 29 below.

[ equation 29]

Column x→p (pos, -1), where pos=y-cw _x * cuv x, where x=0, 1, …, N _s -1

In the case of intra prediction using equation 29 above, the magnitude of curvature may be adjusted by adjusting cuv. cuv may be a real number equal to or greater than zero. For example, the larger the cuv value becomes, the larger the curvature becomes, and the position of the reference pixel may be moved upward. Alternatively, the smaller the cuv value becomes, the smaller the curvature becomes, and the position of the reference pixel may be moved downward (up to the position of y).

When the target block is NxM, cw _i There may be parameters including M weights, where the parameters refer to the width of the block or the number of columns. Each weight may be a real number equal to or greater than zero. Can be used forBy adjusting cw _i To adjust the position of the reference pixel used by the predicted pixel included in the corresponding column. For example, cw _i The greater the value becomes, the position of the reference pixel used by the predicted pixel of the i-th column may be moved upward. Alternatively, cw _i The smaller the value becomes, the position of the reference pixel can be moved downward (up to the position of y).

To generate the predicted pixel values, the described interpolation prediction may be performed. N may be a positive integer.

When the positions pos of the reference pixels exceed the maximum range of available reference pixels (in fig. 56, P (-1, -7)) the calculated positions of all reference pixels can be used after being converted into respective normalized values by matching with the maximum range of reference pixels.

In the embodiment described with reference to fig. 42 to 57, a single curvature parameterThe number cuv is applied to the current block and a single weight parameter cw is applied to a row or column of the current block. However, it is not limited thereto. In other words, at least one curvature parameter cuv _i And/or at least one weight parameter cw _i May be applied to the current block. For example, as described with reference to fig. 41, different curvature parameters cuv _i And different weight parameters cw _i Can be applied to a pixel unit, a horizontal line unit, a vertical line unit, a diagonal line unit, a right angle line unit, a sub-block unit, and/or an arbitrary pixel group unit of the current block.

As shown in fig. 58, based on the intra prediction mode selected in units of blocks, intra prediction in units of samples may be performed. In this case, intra prediction in units of samples may be additionally performed.

For example, when the selected intra prediction MODE is a non-directional MODE (PLANAR_MODE or DC_MODE), prediction in units of samples may be performed more than just.

In (a) of fig. 58, when the intra prediction mode of the selected block is included in the range a, at least one of prediction in units of samples from the lower left side to the upper right side and from the left side to the upper right side may be performed.

Alternatively, in (a) of fig. 58, when the intra prediction mode of the selected block is included in the range B, at least one of prediction in units of samples from the upper left side to the lower right side (type 2) and from the left side to the lower right side may be performed.

Alternatively, in (a) of fig. 58, when the intra prediction mode of the selected block is included in the range C, at least one of prediction in units of samples from the upper left side to the lower right side (type 1) and from the upper side to the lower right side may be performed.

Alternatively, in (a) of fig. 58, when the intra prediction mode of the selected block is included in the range D, at least one of prediction in units of samples from the direction of the upper right side to the lower left side and from the direction of the upper side to the lower left side may be performed.

Alternatively, in (b) of fig. 58, when the intra prediction mode of the selected block is included in the range a, at least one of prediction in units of samples from the left side to the lower right side direction and from the left side to the upper right side direction may be performed.

Alternatively, in (B) of fig. 58, when the intra prediction mode of the selected block is included in the range B, at least one of prediction in units of samples from the upper side→the lower left side direction and the upper side→the lower right side direction may be performed.

Fig. 59 is a diagram depicting an embodiment of predicting a residual signal.

Residual signal prediction may be additionally performed with respect to a residual signal constructed from intra prediction.

Intra prediction is performed for the current block using samples of neighboring blocks that have been encoded/decoded. Therefore, when the samples of the current block are farther from the reference samples, the residual signal is larger, thereby reducing coding efficiency.

To further reduce the residual signal, residual signal prediction may be additionally performed for the current block or each sub-block.

The reconstructed neighboring block that has been encoded/decoded before the current block may be set as a search range. Different search ranges may be set according to the size, shape, partition depth, and/or intra prediction mode of the current block.

For example, all blocks reconstructed before the current block may be set as the search range. If a left block, an upper block, and/or an upper right block adjacent to the current block are included in the search range, the partial region of the reconstructed block may include the current block or a block after the current block that has not yet been started to be encoded. In this case, the available samples of the reconstructed block may be used to fill the corresponding sample values, or a block after the current block that has not yet been started to be encoded may be removed from the search range.

For example, N blocks among blocks reconstructed before the current block may be set as the search range. N may be a positive integer equal to or greater than 1.

For example, as shown in fig. 59, if the width and length of the current block cur_blk are W and H, respectively, and the position of the upper left pixel of the current block is defined as (0, 0), 10 reconstructed blocks including pixels at positions (-2*W, -2*H), (-W, -2*H), (0, -2*H), (W, -2*H), (-2*W, -H), (-W, -H), (0, -H), (W, -H), (-2×w, 0), and (-W, 0) may be set as the search range.

For example, if the current block is a non-square block having unequal widths W and lengths H, the search range may be set in accordance with W and/or H. For example, if the current block is a w×h non-square block, the search range may be set to a size (k×w) × (l×h) in the area where the neighboring block is reconstructed. Here, each of K and L may be a positive integer equal to or greater than 1.

The prediction block of the current block may be constructed using an intra prediction mode predModeIntra of the current block determined through intra prediction among the above-described various intra prediction modes.

For example, in fig. 59, a prediction block constructed using predModeIntra may be represented by prd_blk_best.

In addition to the prediction block prd_blk_best constructed by predModeIntra, one or more prediction blocks of the current block may be constructed.

For example, at least one additional prediction block may be constructed using at least one of all directional/non-directional intra prediction modes.

For example, N intra prediction modes adjacent to predModeIntra may be used to construct an additional prediction block. Here, N may be a positive integer equal to or greater than 1. For example, as shown in fig. 59, if n=2, two additional prediction blocks prd_blk_plus_one and prd_blk_minus_one may be constructed. For example, if predModeIntra is a directional mode, an additional prediction block prd_blk_plus_one may be constructed using predmodeintra+1, and other additional prediction blocks may be constructed using predModeIntra-1. A positive integer equal to or greater than 1 may be added to predModeIntra, or a positive integer equal to or greater than 1 may be subtracted from predModeIntra.

For example, an additional prediction block may be constructed using N neighboring intra prediction modes with respect to the angle to predModeIntra. Here, N may be a positive integer equal to or greater than 1. If the angle is θ, an additional prediction block may be constructed using intra prediction modes falling within an angle range from- θ to θ.

For example, M additional prediction blocks may be constructed by combining N additional prediction blocks constructed using N intra prediction modes. Here, M and N may be positive integers. For example, if predModeIntra is a non-directional MODE PLANAR_MODE, an additional prediction block PRD_BLK_plus_one may be constructed using DC_MODE, and another additional prediction block PRD_BLK_minus_one may be constructed using a weighted sum of PRD_BLK_best and PRD_BLK_plus_one. Alternatively, if predModeIntra is a non-directional MODE dc_mode, an additional prediction block prd_blk_plus_one may be constructed using the planar_mode, and another additional prediction block prd_blk_minus_one may be constructed using a weighted sum of prd_blk_best and prd_blk_plus_one. Alternatively, if predModeIntra is a direction MODE anglar_mode, an additional prediction block prd_blk_plus_one may be constructed using anglar_mode, and another additional prediction block prd_blk_minus_one may be constructed using a weighted sum of prd_blk_best and prd_blk_plus_one.

The plurality of difference blocks may be obtained by using the current block and at least one constructed prediction block of the plurality of constructed prediction blocks. For example, a difference block between the current block and a prediction block constructed using all directional/non-directional modes may be constructed. For example, a difference block between a current block and n+1 prediction blocks may be constructed, wherein the n+1 prediction blocks are constructed using predModeIntra and N intra prediction modes adjacent to predModeIntra in angle or intra prediction mode. Here, N is a positive integer equal to or greater than 1. For example, as shown in fig. 59, if n=2, three difference blocks may be constructed as follows.

For example, a difference block res_blk_best between the current block and prd_blk_best may be obtained. For example, a difference block res_blk_plus between the current block and the prd_blk_plus_one may be obtained. For example, a difference block res_blk_minusone between the current block and the prd_blk_minus_one may be obtained.

For example, N additional prediction blocks may be constructed using N intra prediction modes, M additional prediction blocks (M and N are positive integers) may be constructed by combining the constructed N prediction blocks, and then a difference block between the three prediction blocks and the current block may be constructed.

An operation of calculating an optimal Intra Displacement Vector (IDV) for each sample within the search range may be performed. As shown in fig. 59, the IDV may be defined as a vector (x, y) from the upper left sample position of the current block to the sample position within the search range.

For each IDV, the sample position indicated by the IDV may be used as (0, 0) to construct a reconstructed block rec_blk having the same size as the current block.

One or more residual signal blocks res_blk_idv may be constructed for the reconstructed block rec_blk constructed at the position indicated by the IDV using the prd_blk_best and at least one of the plurality of additionally constructed prediction blocks.

For example, if N additional prediction blocks are used, n+1 residual signal blocks may be constructed by [ equation 30] for the reconstructed block rec_blk constructed at the position indicated by the IDV.

[ equation 30]

RES_BLK_IDV_best＝REC_BLK-PRD_BLK_best

RES_BLK_IDV_1＝REC_BLK-PRD_BLK_1

RES_BLK_IDV_2＝REC_BLK-PRD_BLK_2

…

RES_BLK_IDV_N＝REC_BLK-PRD_BLK_N

For example, if two additional prediction blocks are used as shown in fig. 59, three residual signal blocks may be constructed for the reconstructed block rec_blk constructed at the position indicated by the IDV by [ equation 31 ].

[ equation 31]

RES_BLK_IDV_best＝REC_BLK-PRD_BLK_best

RES_BLK_IDV_plusone＝REC_BLK-PRD_BLK_plus_one

RES_BLK_IDV_minusone＝REC_BLK-PRD_BLK_minus_one

As many second order residual blocks as the number of prediction blocks applied to the reconstructed block at the location indicated by each IDV within the search range can be constructed.

As shown in fig. 60, the difference between res_blk_ 'MODE' and res_blk_ idv_ 'MODE' may be used to obtain a second order residual signal block res_blk_ SEC. Here, "MODE" may be a predetermined intra prediction MODE.

For example, if N prediction blocks are applied to a reconstructed block, N second order residual signal blocks may be constructed for the reconstructed block res_blk by [ equation 32 ].

[ equation 32]

RES_BLK_SEC_1＝RES_BLK_1-RES_BLK_IDV_1

RES_BLK_SEC_2＝RES_BLK_2-RES_BLK_IDV_2

…

RES_BLK_SEC_N＝RES_BLK_N-RES_BLK_IDV_N

For example, if a prediction block generated by using predModeIntra and two additional prediction blocks are used, as shown in fig. 60, three second order residual signal blocks may be constructed for the reconstructed block rec_blk by [ equation 33 ].

[ equation 33]

RES_BLK_SEC_best＝RES_BLK_best-RES_BLK_IDV_best

RES_BLK_SEC_plusone＝RES_BLK_plusone-RES_BLK_IDV_plusone

RES_BLK_SEC_minusone＝RES_BLK_minusone-RES_BLK_IDV_minusone

Whether or not to perform residual signal prediction for a reconstructed block at a position indicated by each IDV within a search range may be determined by comparing a cost function value of res_blk_best with a cost function value of a residual signal block, which is minimum among a plurality of second order residual signal blocks res_blk_sec according to rate distortion optimization. Whether residual signal prediction is performed may be indicated with, for example, an indicator srp_flag (second order residual prediction). For example, if residual signal prediction is performed, a first value may be allocated to srp_flag, and if residual signal prediction is not performed, a second value may be allocated to srp_flag. The first value may be 1 and the second value may be 0.

For example, if the minimum value among the cost function values of the second order residual signal block is smaller than the cost function value generated from res_blk_best, the first value may be allocated to srp_flag. In addition, at least one of the srp_flag, the IDV, the intra prediction mode used to generate the second order residual signal block having the smallest cost function value, and the second order residual signal of the res_blk_sec may be encoded in or decoded from the bitstream.

For example, if the minimum value of the cost function values of the second order residual signal block is equal to or greater than the cost function value generated from res_blk_best, the second value may be allocated to srp_flag. In addition, at least one of the residual signal of res_blk_best, srp_flag, and predModeIntra may be encoded in or decoded from the bitstream.

Since the above operations are repeatedly performed for all IDVs indicating all samples within the search range, all IDVs indicating second order residual signal blocks having a cost function value smaller than the cost function value of res_blk_best can be determined. The IDV having the smallest cost function value may be finally selected from all the determined IDVs.

Fig. 61 is a diagram depicting an embodiment of residual signal prediction.

Curblk represents a current block to be encoded/decoded. Res_blk may be a first order residual signal representing a residual signal block between a prediction block of the current block and cur_blk. The prediction block may be one of a plurality of prediction blocks constructed for the current block in the encoder/decoder.

Idv_rec_blk represents a reconstructed block constructed from IDV. Res_blk_idv represents a residual signal block between a prediction block of the current block and idv_rec_blk. The prediction block may be one of a plurality of prediction blocks constructed for the current block in the encoder/decoder.

Res_blk_sec represents a second order residual signal block between res_blk and res_blk_idv.

In fig. 61, if the cost function value of res_blk is smaller than the cost function value of res_blk_sec, a first value may be allocated to srp_flag. In addition, at least one of the srp_flag, the IDV, the intra prediction mode for generating the second order residual signal block having the smallest cost function value, and the second order residual signal of the res_blk_sec may be encoded in or decoded from the bitstream. The first value may be 0 or 1.

In fig. 61, if the cost function value of res_blk is equal to or greater than the cost function value of res_blk_sec, the second value may be allocated to the srp_flag. In addition, at least one of the first order residual signal of res_blk, srp_flag, and predModeIntra may be encoded in or decoded from the bitstream. The second value may be 0 or 1.

The residual signal of the current block may be derived using at least one of the first residual signal and the second residual signal. The neighboring block idv_rec_blk may be a block indicated by the above-described IDV, and may be neighboring to the current block (e.g., a block reconstructed before the current block, such as a block to the left of the current block and a block above the current block). The IDV may be encoded as it is, or may be encoded by encoding a difference between the IDV and the IDV of another block in the current picture. Res_blk_idv may be derived as a value obtained by subtracting a predetermined prediction block from a reconstructed neighboring block. Here, the intra prediction mode used to generate the prediction block may be an intra prediction mode of a neighboring block idv_rec_blk or a current block, or may be a preset mode in an encoder/decoder. Alternatively, the predetermined prediction block may be set to be the same as the prediction block for the current block. Alternatively, an upper layer block including the current block may be defined, and a residual signal shared by blocks belonging to the upper layer block may be defined as the second residual signal. The residual prediction technique based on the above-described second residual signal may be selectively performed based on a predetermined flag srp_flag. The flag may be signaled by at least one of a sequence, a picture, a slice, and a block. Alternatively, if the flag is constant, the flag may be derived as a fixed value preset in the encoder/decoder according to the size, shape and/or depth of the current block.

An indicator (flag) indicating whether residual signal prediction is performed for the current block may be encoded/decoded. For example, the indicator may be an srp_flag, and may be encoded/decoded according to at least one of the current block or each sub-block.

At least one of the IDV, the intra prediction mode (mode_x in fig. 61) selected for residual signal prediction of the current block, and other coding parameter information as information required for residual signal prediction may be encoded in or decoded from the bitstream.

The IDV information of the current block may be predictive encoded/decoded using the IDVs of neighboring blocks. For example, the difference between the IDV of the current block and the IDV of the neighboring block may be encoded/decoded, such as the minimum, maximum, median, average, most frequent value, or weighted sum.

Alternatively, if at least one of the IDVs of the neighboring blocks is identical to the IDV of the current block, the encoder may make and use the IDV list using the IDVs of the neighboring blocks according to a predefined method. The encoder may encode information (e.g., a flag) indicating that at least one of the IDVs of the neighboring block is identical to the IDV of the current block, and/or information (e.g., an index) indicating the identical IDV in the IDV list as the IDV information of the current block. If the flag information indicates that at least one of the IDVs of the neighboring block is identical to the IDV of the current block, the decoder may make an IDV list according to a predefined method. The decoder may derive the IDV of the current block using the generated IDV list and the index information.

Intra prediction may be performed for the current block (S6201), and a best intra prediction mode predModeIntra may be determined (S6202). In step S6202, a prediction block prd_blk_best may be generated using the best intra prediction mode. A residual block res_blk_bset between the prediction block prd_blk_best generated using predModeIntra and the current block may be obtained (S6203).

N additional intra prediction modes may be determined based on predModeIntra determined at step S6202 (S6204). In step S6205, n+1 prediction blocks prd_blk_ 'mode' may be generated from n+1 intra prediction modes including predModeIntra and N additional intra prediction modes. In step S6206, n+1 residual blocks res_blk_ 'mode' may be obtained using the current block and n+1 prediction blocks prd_blk_ 'mode'.

In step S6207, the IDV may be moved and determined within the search range using a pixel-by-pixel search or a predetermined search method. The reconstructed block rec_blk_idv may be obtained at the determined IDV position (S6208). In step S6209, n+1 IDV residual blocks res_blk_idv_ 'mode' may be obtained using rec_blk_idv and n+1 prediction blocks prd_blk_ 'mode'.

In step S6210, n+1 second order residual blocks res_blk_sec_mode 'may be obtained using res_blk_mode' and res_blk_idv_mode 'corresponding to res_blk_mode'.

In step S6211, the RD cost of each block RES_BLK_SEC_mode' may be compared to the RD cost of RES_BLK_best. If the RD cost of at least one block RES_BLK_SEC_ 'mode' is less than the RD cost of RES_BLK_best (YES in S6211), the process may proceed to step S6212. Otherwise (no in step S6211), the process may proceed to step S6213.

In step S6212, a first value (e.g., 1) may be assigned to the srp_flag, and the best cost may be determined as the RD cost of the at least one block res_blk_sec_mode'.

In step S6213, it may be determined whether the current IDV is the last IDV within the search range. If the current IDV is not the last IDV (NO in step S6213), the process proceeds to step S6207, and the above operation may be repeated for the next IDV. If the current IDV is the last IDV (YES in step S6213), the process may proceed to step S6214.

In step S6214, it may be determined whether the srp_flag is a first value (e.g., 1). If SRP_flag is the first value (YES in S6214), the process may proceed to step S6215. If SRP_flag is not the first value (NO in S6214), the process may proceed to step S6216.

At least one of srp_flag, IDV, intra prediction mode (mode ' used in pred_blk_mode ') and res_blk_sec_mode ' for generating a second order residual signal block having the smallest cost function value, and the second order residual signal may be determined as an encoding target at step S6215 and encoded at step S6217.

At least one of the first order residual signal of res_blk (e.g., res_blk_best), srp_flag, and predModeIntra may be determined as an encoding target at step S6216 and encoded at step S6217.

If the IDV is not signaled, the decoder may derive the IDV by performing the search process of the encoder in the same manner.

In step S6301, the related information included in the bit stream may be decoded.

If the SRP_flag is not the first value (e.g., 1) (NO in S6302), step S6303 may be performed.

In step S6303, a prediction block prd_blk_best corresponding to predModeIntra may be generated. In addition, the residual block res_blk_best may be decoded.

At step S6304, a reconstructed block may be generated using prd_blk_best and res_blk_best.

If the SRP_flag is a first value (e.g., 1) (YES in S6302), steps S6305, S6306, and S6307 may be performed.

In step S6305, the second order residual block res_blk_sec_mode' may be decoded.

In step S6306, a reconstructed block rec_blk_idv corresponding to the decoded IDV may be generated.

In step S6307, a prediction block prd_blk_ 'mode' corresponding to the decoded 'mode' may be generated.

In step S6308, the generated rec_blk_idv and prd_blk_ 'mode' may be used to generate an IDV residual block res_blk_idv_ 'mode'.

At step S6309, the residual block res_blk may be generated using the generated res_blk_sec_mode 'and res_blk_idv_mode'.

In step S6310, the reconstructed block may be generated using the generated res_blk and prd_blk_ 'mode'.

As described above, if the IDV is not signaled, the decoder may derive the IDV by performing a search operation of the encoder in the same manner.

The intra encoding/decoding process may be performed for each of the luminance signal and the chrominance signal. For example, in the intra encoding/decoding process, at least one method of deriving an intra prediction mode, dividing a block, constructing reference samples, and performing intra prediction may be differently applied for a luminance signal and a chrominance signal.

The intra encoding/decoding process may be equally performed for the luminance signal and the chrominance signal. For example, when an intra encoding/decoding process is applied for a luminance signal, at least one of deriving an intra prediction mode, dividing a block, constructing a reference sample, and performing intra prediction may be equally applied to a chrominance signal.

The method may be performed in the encoder and decoder in the same manner. For example, in the intra encoding/decoding process, at least one method of deriving an intra prediction mode, dividing a block, constructing reference samples, and performing intra prediction may be equally applied in an encoder and a decoder. In addition, the order of application methods may be different in the encoder and decoder. For example, in performing intra encoding/decoding with respect to a current block, an encoder may encode an intra prediction mode determined by performing at least one intra prediction after constructing a reference sample.

Embodiments of the present invention may be applied according to the size of at least one of the encoded block, the prediction block, the block, and the unit. Here, the size may be defined as a minimum size and/or a maximum size in order to apply the embodiment, and may be defined as a fixed size to which the embodiment is applied. Further, the first embodiment may be applied in a first size and the second embodiment may be applied in a second size. That is, embodiments may be applied multiple times depending on the size. Further, the embodiments of the present invention may be applied only when the size is equal to or larger than the minimum size and equal to or smaller than the maximum size. That is, the embodiment may be applied only when the block size is in a predetermined range.

For example, the embodiment can be applied only when the size of the encoding/decoding target block is equal to or larger than 8×8. For example, the embodiment can be applied only when the size of the encoding/decoding target block is equal to or larger than 16×16. For example, the embodiment can be applied only when the size of the encoding/decoding target block is equal to or larger than 32×32. For example, the embodiment can be applied only when the size of the encoding/decoding target block is equal to or larger than 64×64. For example, the embodiment may be applied only when the size of the encoding/decoding target block is equal to or larger than 128×128. For example, the embodiment can be applied only when the encoding/decoding target block size is 4×4. For example, the embodiment can be applied only when the size of the encoding/decoding target block is equal to or smaller than 8×8. For example, the embodiment can be applied only when the size of the encoding/decoding target block is equal to or larger than 16×16. For example, the embodiment may be applied only when the size of the encoding/decoding target block is equal to or greater than 8×8 and equal to or less than 16×16. For example, the embodiment may be applied only when the size of the encoding/decoding target block is equal to or greater than 16×16 and equal to or less than 64×64.

Embodiments of the present invention may be applied according to temporal layers. An identifier for identifying a temporal layer to which an embodiment may be applied may be signaled and an embodiment may be applied for the temporal layer indicated by the indicator. Here, the identifier may be defined to indicate a minimum layer and/or a maximum layer to which an embodiment may be applied, and may be defined to indicate a specific layer to which an embodiment may be applied.

For example, the embodiment may be applied only when the temporal layer of the current picture is the lowest layer. For example, the embodiment may be applied only when the temporal layer identifier of the current picture is 0. For example, the embodiment may be applied only when the temporal layer identifier of the current picture is equal to or greater than 1. For example, the embodiment may be applied only when the temporal layer of the current picture is the highest layer.

As described in the embodiments of the present invention, the reference picture set used in the process of reference picture list construction and reference picture list modification may use at least one of reference picture lists L0, L1, L2, and L3.

According to an embodiment of the present invention, at least one up to N motion vectors of the encoding/decoding target block may be used when the deblocking filter calculates the boundary strength. Here, N indicates a positive integer equal to or greater than 1, such as 2, 3, 4, and the like.

In motion vector prediction, an embodiment of the present invention may be applied when a motion vector has at least one of the following units: a 16-pixel (16-pel) unit, an 8-pixel (8-pel) unit, a 4-pixel (4-pel) unit, an integer-pixel (integer-pel) unit, a 1/2-pixel (1/2-pel) unit, a 1/4-pixel (1/4-pel) unit, a 1/8-pixel (1/8-pel) unit, a 1/16-pixel (1/16-pel) unit, a 1/32-pixel (1/32-pel) unit, and a 1/64-pixel (1/64-pel) unit. Further, in performing motion vector prediction, a motion vector may be optionally used for each pixel unit.

The type of the strip to which the embodiments of the present invention are applied may be defined and the embodiments of the present invention may be applied according to the type of the strip.

For example, when the slice type is T (three-way prediction) -slice, the prediction block may be generated by using at least three motion vectors, and may be used as a final prediction block of the encoding/decoding target block by calculating a weighted sum of the at least three prediction blocks. For example, when the slice type is Q (four-way prediction) -slice, the prediction block may be generated by using at least four motion vectors, and may be used as a final prediction block of the encoding/decoding target block by calculating a weighted sum of at least four prediction blocks.

The embodiments of the present invention can be applied to inter prediction and motion compensation methods using motion vector prediction and inter prediction and motion compensation methods using skip mode, merge mode, and the like.

The shape of the block to which the embodiments of the present invention are applied may have a square or a non-square shape.

In the above-described embodiment, the method is described based on a flowchart having a series of steps or units, but the present invention is not limited to the order of the steps, and some steps may be performed simultaneously with other steps or may be performed in a different order from other steps. Furthermore, it will be understood by those of ordinary skill in the art that steps in the flowcharts are not mutually exclusive, and other steps may be added to the flowcharts, or some steps may be deleted from the flowcharts without affecting the scope of the present invention.

Embodiments include various aspects of the examples. All possible combinations for the various aspects may not be described, but one of ordinary skill in the art will be able to recognize different combinations. Accordingly, the present invention is intended to embrace all such alternatives, modifications and variances which fall within the scope of the appended claims.

The embodiments of the present invention can be implemented in the form of program instructions executable by various computer components and recorded on a computer-readable recording medium. The computer readable recording medium may include individual program instructions, data files, data structures, etc., or a combination of program instructions, data files, data structures, etc. The program instructions recorded in the computer-readable recording medium may be specially designed and constructed for the present invention, or they may be known to those skilled in the art of computer software technology. Examples of the computer-readable recording medium include: magnetic recording media (such as hard disks, floppy disks, and magnetic tape); an optical data storage medium (such as a CD-ROM or DVD-ROM); magneto-optical media (such as floppy disks); and hardware devices that are specially constructed for storing and carrying out the program instructions (e.g., read-only memory (ROM), random Access Memory (RAM), flash memory, etc.). Examples of program instructions include not only machine language code, which is formed by a compiler, but also high-level language code that may be implemented by a computer using an interpreter. The hardware devices may be configured to be operated by one or more software modules to perform the processes according to the invention, and vice versa.

Although the present invention has been described in terms of specific terms (such as detailed elements) and limited embodiments and figures, they are merely provided to facilitate a more general understanding of the present invention, and the present invention is not limited to the above-described embodiments. Those skilled in the art to which the invention pertains will appreciate that various modifications and changes can be made from the above description.

The spirit of the invention should, therefore, not be limited to the above-described embodiments, and the full scope of the appended claims and equivalents thereof should be accorded the full scope of the invention.

Industrial applicability

The present invention can be used when encoding/decoding an image.

Claims

1. An image decoding method, comprising:

determining a reference sample line of the current block;

deriving an intra prediction mode of the current block;

deriving a reference sample point of the current block based on the reference sample point line of the current block;

generating a prediction block by performing intra prediction for a current block based on the intra prediction mode and the reference sample; and is also provided with

A reconstructed block is obtained based on the prediction block,

wherein the step of deriving the intra prediction mode of the current block comprises:

determining whether to derive the intra prediction mode of the current block from a most probable mode MPM,

wherein when determining to derive the intra-prediction mode of the current block from the MPM, a single flag is decoded from a bitstream to derive the intra-prediction mode of the current block from the first MPM list or the second MPM list,

Wherein based on the single flag, it is determined whether to derive the intra prediction mode of the current block from a first MPM list or a second MPM list,

wherein, in response to the single flag indicating that the intra-prediction mode of the current block is derived from a second MPM list, further decoding an MPM index from the bitstream to determine the intra-prediction mode of the current block, wherein the MPM index specifies one of the MPMs included in the second MPM list,

wherein in response to determining that the intra-prediction mode of the current block is not derived from the MPM, the intra-prediction mode of the current block is derived based on information specifying one of the remaining intra-prediction modes other than the MPM.

2. The method of claim 1, wherein, when the current block is one of sub-blocks generated by partitioning the encoded block, the MPM is derived based on a position of the encoded block instead of the position of the current block.

3. The method according to claim 1,

wherein, in response to the intra prediction mode of the current block being a directional mode, a prediction sample is modified by a second reference sample,

wherein the prediction sample is obtained based on a first reference sample, wherein the first reference sample is located in the direction of the intra prediction mode from the prediction sample,

Wherein a second reference sample is located in a direction different from the direction of the intra prediction mode from the prediction sample.

4. The method of claim 1, wherein the reference sample line is selected from a plurality of reference sample line candidates based on information decoded from a bitstream,

wherein, in response to the current block being one of the sub-blocks generated by partitioning the encoded block, intra prediction for the current block is invoked by utilizing only a first reference sample line candidate of the plurality of reference sample line candidates.

5. The method of claim 4, wherein the number of the plurality of reference sample line candidates available for the current block is determined based on whether an upper boundary of the current block corresponds to a boundary of a coding tree block.

6. The method of claim 1, wherein, in the case where the current block is rectangular and the intra prediction mode is a DC mode, intra prediction of the current block is performed using an average value of reference samples adjacent to a longer side among a width and a height of the current block.

7. An image encoding method, comprising:

determining a reference sample line of the current block;

determining an intra prediction mode of the current block;

Determining a reference sample point of the current block based on the reference sample point line of the current block;

A residual block of the current block is obtained by subtracting the prediction block from the original block,

wherein the method further comprises:

it is determined whether the intra prediction mode of the current block is identical to one of the most probable modes MPM,

wherein when it is determined that the intra prediction mode of the current block is the same as one of the MPMs, a flag for selecting one of the first MPM list and the second MPM list is encoded into the bitstream,

wherein, in response to the flag being encoded to indicate the second MPM list, an MPM index specifying one of the MPMs included in the second MPM list is also encoded into the bitstream,

wherein in response to determining that the intra prediction mode of the current block is not identical to any one of the MPMs, information specifying one of the remaining intra prediction modes other than the MPM that is identical to the intra prediction mode of the current block is encoded into the bitstream.

8. A bit stream transmitting method includes

Generating a bitstream by encoding video based on an image encoding method; and

A bit stream is transmitted and,

the image coding method comprises the following steps:

determining a reference sample line of the current block;

determining an intra prediction mode of the current block;

generating a prediction block by performing intra prediction for a current block based on the intra prediction mode and the reference sample; and

wherein the method further comprises: