CN112399179A

CN112399179A - Method of decoding and encoding image and computer readable medium

Info

Publication number: CN112399179A
Application number: CN202011356439.XA
Authority: CN
Inventors: 李培根
Original assignee: KT Corp
Current assignee: KT Corp
Priority date: 2017-09-08
Filing date: 2018-09-06
Publication date: 2021-02-23
Also published as: US20200145648A1; US11388395B2; CN116600127A; CN116634154A; WO2019050291A1; CN116582670A; KR20190028324A; CN110710216B; US20210084292A1; US11089299B2; US11743455B2; CN110710216A; US20220311999A1

Abstract

The present invention relates to a method of decoding an image using a decoding apparatus, a method of encoding an image using an encoding apparatus, and a computer readable medium, the method of decoding an image including: deciding an intra prediction mode of a current block in an image from intra prediction modes predefined in a decoding apparatus, the intra prediction modes including a planar mode, a DC mode, and a bi-directional intra prediction mode; deciding a reference sample for intra prediction of the current block based on whether the intra prediction mode is a bidirectional intra prediction mode; and generating a prediction block of the current block by performing intra prediction on a current sample in the current block based on reference samples, wherein the reference samples include an upper end reference sample adjacent to the current block, a left side reference sample adjacent to the current block, and a plurality of right side reference samples adjacent to the current block.

Description

Method of decoding and encoding image and computer readable medium

The application is a divisional application of a Chinese patent application with the international application number of PCT/KR2018/010413, the invention name of which is 'video signal processing method and device', and the application number of 201880035977.3 entering the Chinese national stage, which is submitted on 6.9.2018.

Technical Field

The invention relates to a video signal processing method and device.

Background

In recent years, there has been an increasing demand for High-resolution and High-quality images such as hd (High definition) images and uhd (ultra High definition) images in various application fields. Since the higher the resolution and quality of image data, the more the data amount increases with respect to the conventional image data, the higher the transfer cost and storage cost increases if the image data is transferred by a medium such as a conventional wireless broadband circuit or stored by a conventional storage medium. In order to solve these problems accompanying high resolution and high quality of image data, an efficient image compression technique can be used.

As an image compression technique, there are various techniques such as an inter-picture prediction technique for predicting a pixel value included in a current picture from a picture previous to or next to the current picture, an intra-picture prediction technique for predicting a pixel value included in the current picture using pixel information within the current picture, an entropy coding technique for assigning a short code to a value having a high frequency of occurrence and a long code to a value having a low frequency of occurrence, and the like, which can efficiently compress image data using such an image compression technique and then perform transmission or storage.

On the other hand, as the demand for high-resolution images increases, the demand for stereoscopic image content also increases as a new image service. Video compression techniques for efficiently providing high resolution as well as ultra-high resolution stereoscopic image content are being discussed.

Disclosure of Invention

An object of the present invention is to provide a method and apparatus for efficiently performing intra prediction on a block to be encoded/decoded when encoding/decoding a video signal.

An object of the present invention is to provide a method and apparatus for performing intra prediction using a plurality of reference samples that are not adjacent to each other when encoding/decoding a video signal.

An object of the present invention is to provide a method and apparatus for performing intra prediction using right and lower reference samples when encoding/decoding a video signal.

The technical problem to be solved by the present invention is not limited to the technical problem mentioned above, and other technical problems not mentioned above can be clearly understood by those skilled in the art to which the present invention pertains from the following description.

The method for decoding an image by the decoding device according to the present invention may be: determining an intra prediction mode for a current block in a picture from intra prediction modes predefined in the decoding device, the intra prediction modes including a planar mode, a DC mode, and a bi-directional intra prediction mode; determining reference samples for intra prediction of the current block based on the intra prediction mode being a bi-directional intra prediction mode; and generating a prediction block of the current block by performing intra prediction on a current sample in the current block based on the reference samples, wherein the reference samples include an upper end reference sample adjacent to the current block, a left side reference sample adjacent to the current block, and a plurality of right side reference samples adjacent to the current block.

The method for encoding an image using an encoding device according to the present invention may be: acquiring a prediction block of a current block in an image; acquiring a residual block of the current block based on the original block of the current block and the prediction block; and encoding the residual block to generate a bitstream, wherein an intra prediction mode of the current block is determined from intra prediction modes including a planar mode, a DC mode, and a bi-directional intra prediction mode, wherein prediction samples of the prediction block are obtained by using reference samples corresponding to the intra prediction mode of the current block, wherein the reference samples include an upper end reference sample adjacent to the current block, a left side reference sample adjacent to the current block, and a plurality of right side reference samples adjacent to the current block.

The present invention relates to a computer readable medium having stored thereon instructions which, when executed by a processor, perform a method, which may comprise: determining an intra prediction mode of a current block in an image from intra prediction modes predefined in a decoding apparatus, the intra prediction modes including a planar mode, a DC mode, and a bi-directional intra prediction mode; determining reference samples for intra prediction of the current block based on the intra prediction mode being a bi-directional intra prediction mode; and generating a prediction block of the current block by performing intra prediction on a current sample in the current block based on the reference samples, wherein the reference samples include an upper end reference sample adjacent to the current block, a left side reference sample adjacent to the current block, and a plurality of right side reference samples adjacent to the current block.

The video signal decoding method and apparatus according to the present invention may be: the method includes determining whether bidirectional intra prediction is applied to a current block, deriving a reference sample of the current block, and acquiring a prediction sample of the current block based on a first reference sample and a second reference sample specified by an intra prediction mode of the current block when bidirectional intra prediction is applied to the current block.

The video signal encoding method and apparatus according to the present invention may be: the method includes determining whether bidirectional intra prediction is applied to a current block, deriving a reference sample of the current block, and acquiring a prediction sample of the current block based on a first reference sample and a second reference sample specified by an intra prediction mode of the current block when bidirectional intra prediction is applied to the current block.

The video signal encoding/decoding method and apparatus according to the present invention may be: the first reference sample is a left side or upper end reference sample of the current block, and the second reference sample is a right side or lower end reference sample of the current block.

The video signal encoding/decoding method and apparatus according to the present invention may be: the right-side reference sample is generated by interpolating a right-side lower-end reference sample and a right-side upper-end reference sample, and the lower-end reference sample is generated by interpolating the right-side lower-end reference sample and the left-side lower-end reference sample.

The video signal encoding/decoding method and apparatus according to the present invention may be: the first reference sample is selected by applying an intra prediction mode of the current block in a forward direction, and the second reference sample is selected by applying an intra prediction mode of the current block in a reverse direction.

The video signal encoding/decoding method and apparatus according to the present invention may be: the prediction sample is obtained by a weighting operation of a first prediction sample obtained based on the first reference sample and a second prediction sample obtained based on the second reference sample.

The video signal encoding/decoding method and apparatus according to the present invention may be: the weight applied to both the first prediction sample and the second prediction sample is determined based on at least one of a position of a sample to be predicted, a size, a shape, or an intra prediction mode of the current block.

The video signal encoding/decoding method and apparatus according to the present invention may be: the bi-directional intra prediction is applied only to a predetermined area within the current block.

The video signal encoding/decoding method and apparatus according to the present invention may be: the step of determining whether to apply bidirectional intra prediction to the current block includes a step of determining whether an intra prediction mode of the current block is a directional intra prediction mode, and a step of decoding information indicating whether the directional intra prediction mode is used as the bidirectional intra prediction mode.

The video signal encoding/decoding method and apparatus according to the present invention may be: whether or not bidirectional intra prediction is applied to the current block is determined based on whether or not bidirectional intra prediction is applied to a peripheral block adjacent to the current block.

The video signal encoding/decoding method and apparatus according to the present invention may be: when the intra prediction mode of the current block is a directional intra prediction mode within a predetermined range, it is determined that bidirectional intra prediction is applied to the current block.

The features of the present invention briefly summarized above are merely exemplary embodiments of the present invention described in detail below, and do not limit the scope of the present invention.

According to the present invention, intra prediction can be efficiently performed on a block to be encoded/decoded.

According to the present invention, there is an advantage in that efficiency of intra prediction can be improved by performing intra prediction using a plurality of reference samples that are not adjacent.

According to the present invention, there is an advantage in that the efficiency of intra prediction can be improved by using right and lower end reference samples.

The effects obtainable by the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art to which the present invention pertains from the following descriptions.

Drawings

Fig. 1 is a block diagram showing an image encoding device according to an embodiment of the present invention.

Fig. 2 is a block diagram showing an image decoding device according to an embodiment of the present invention.

Fig. 3 is a diagram showing an example of hierarchically dividing a coding block based on a tree structure (tree structure) to which an embodiment of the present invention is applied.

FIG. 4 is a diagram showing a partition modality that allows binary tree based partitioning, to which an embodiment of the present invention is applicable.

Fig. 5 is a diagram showing an example of binary tree based segmentation that allows only a specific modality, to which an embodiment of the present invention is applicable.

Fig. 6 is a diagram showing an example for explaining encoding/decoding of information on the allowable number of times of binary tree division to which an embodiment of the present invention is applied.

Fig. 7 is a diagram showing partition modes that can be applied to a coding block to which an embodiment of the present invention is applied.

Fig. 8 is a diagram showing the types of intra prediction modes defined in advance in an image encoder/decoder, to which an embodiment of the present invention is applied.

Fig. 9 is a diagram showing the type of extended intra prediction mode to which an embodiment of the present invention is applied.

Fig. 10 is a sequence diagram schematically illustrating an intra prediction method according to an embodiment to which the present invention is applied.

FIG. 11 is a diagram illustrating a method for correcting a prediction sample of a current block based on difference information of neighboring samples according to an embodiment of the present invention.

Fig. 12 and 13 are diagrams showing one-dimensional reference sample groups obtained by rearranging reference samples in a row.

Fig. 14 is a diagram showing an example of deriving a right-side reference sample or a lower-side reference sample using a plurality of reference samples.

Fig. 15 and 16 are diagrams for explaining determination of a right-side reference sample and a lower-side reference sample for a non-square block according to an embodiment of the present invention.

Fig. 17 is a diagram for explaining an example of deriving a second reference sample using a first reference sample.

Fig. 18 is a diagram showing reference samples constituting a one-dimensional reference sample group.

Fig. 19 is a diagram showing an area to which bidirectional intra prediction is applied.

Fig. 20 is an example of identifying a directional prediction mode indicating that bidirectional intra prediction is allowed.

FIG. 21 is a flowchart of a method for intra prediction of a current block based on a bi-directional intra prediction mode according to the present invention.

Detailed Description

While the present invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. However, the present invention is not limited to the specific embodiments, and it should be understood that all changes, equivalents, and substitutions included in the spirit and technical scope of the present invention are included therein. In the description of the drawings, like reference numerals are used to refer to like elements.

The terms such as first and second may be used in the description of various components, but the components are not limited to the terms. The above terms are used only for the purpose of distinguishing one component from another component. For example, a first component may be termed a second component, and similarly, a second component may be termed a first component, without departing from the scope of the present invention. The term "and/or" includes a combination of a plurality of related description items or any one of a plurality of related description items.

When a certain component is referred to as being "connected to" or "connected to" another component, it is to be understood that other components may be interposed therebetween, although the component may be directly connected to or connected to the other component. Conversely, when a component is referred to as being "directly connected to" or "directly coupled to" another component, it is understood that no other component is present therebetween.

The terminology used in the description presented herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular references include plural references unless a context clearly dictates otherwise. In the present application, the terms "including" or "having" or the like should be understood to specify the presence of the features, numbers, steps, actions, structural elements, components, or combinations thereof described in the specification, but do not preclude the presence or addition of one or more other features, numbers, steps, actions, structural elements, components, or combinations thereof.

Preferred embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Hereinafter, the same reference numerals are used for the same components in the drawings, and redundant description of the same components is omitted.

Referring to fig. 1, the image encoding apparatus 100 may include a picture dividing part 110,

prediction parts

120 and 125, a conversion part 130, a quantization part 135, a rearrangement part 160, an entropy encoding part 165, an inverse quantization part 140, an inverse conversion part 145, a filtering part 150, and a storage part 155.

The components shown in fig. 1 are shown separately for the purpose of showing different specific functions in the image coding apparatus, and do not mean that each component is constituted by separate hardware or one software component unit. That is, each of the components is included in the form of a component for convenience of description, at least two components of each of the components may be combined together to form one component, or one component may be divided into a plurality of components to perform functions, and embodiments in which the components are combined together and embodiments in which the components are separated are included in the scope of the claims of the present invention as long as the essence of the present invention is not deviated.

In addition, some of the structural elements may not be essential structural elements for performing essential functions in the present invention, but may be selective structural elements for improving performance. The present invention may be realized by including only components necessary for realizing the essence of the present invention, other than components used for improving performance, and a configuration including only necessary components, other than optional components used for improving performance, is also included in the scope of the present invention.

The picture dividing unit 110 may divide the input picture into at least one processing unit. In this case, the processing Unit may be a Prediction Unit (PU), a Transform Unit (TU), or a Coding Unit (CU). The picture dividing unit 110 may divide a picture into a plurality of combinations of coding units, prediction units, and conversion units, and select a combination of a coding unit, a prediction unit, and a conversion unit according to a predetermined criterion (for example, a cost function) to encode the picture.

For example, one picture may be divided into a plurality of coding units. In order to divide a coding unit from a picture, a recursive Tree Structure such as a Quad Tree Structure (Quad Tree Structure) may be used, and a coding unit divided into other coding units based on one image or a maximum size coding unit (large coding unit) may be divided into sub nodes having the same number of divided coding units. A coding unit that cannot be divided again due to a certain restriction becomes a leaf node. That is, when it is assumed that only one coding unit can be divided into squares, one coding unit can be divided into four other coding units at most.

Hereinafter, in the embodiment of the present invention, the coding unit may be used in the meaning of a unit in which coding is performed or in the meaning of a unit in which decoding is performed.

The prediction unit may be a unit divided from within one coding unit so as to have at least one square or rectangle of the same size, or may be a unit divided from within one coding unit so that any one of the prediction units divided from within one coding unit has a different shape and/or size from the other prediction unit.

When a prediction unit for performing intra prediction is generated based on a coding unit, intra prediction may be performed without being divided into a plurality of prediction units NxN if the prediction unit is not the minimum coding unit.

The

prediction sections

120, 125 may include an inter prediction section 120 that performs inter prediction, and an intra prediction section 125 that performs intra prediction. Specific information (e.g., an intra prediction mode, a motion vector, a reference picture, etc.) may be decided according to each prediction method after deciding whether to use inter prediction or perform intra prediction for a prediction unit. At this time, the processing unit for performing prediction and the processing unit for deciding the prediction method and the specific content may be different. For example, the prediction method, prediction mode, and the like may be determined by a prediction unit, and the prediction may be performed by a conversion unit. A residual value (residual block) between the generated prediction block and the original block may be input to the conversion part 130. The prediction mode information, the motion vector information, and the like used for prediction may be encoded by the entropy encoding unit 165 together with the residual value and transmitted to the decoder. When a specific coding mode is used, the prediction block may not be generated by the

prediction units

120 and 125, but the original block may be directly encoded and transmitted to the decoding unit.

The inter prediction unit 120 may perform prediction of a prediction unit based on information of at least one of a picture previous to or next to the current picture, or may perform prediction of a prediction unit based on information of a partial region already coded in the current picture in some cases. The inter prediction unit 120 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

The reference picture interpolation unit may receive reference picture information from the storage unit 155 and generate pixel information of integer pixels or less in the reference picture. In the case of a luminance pixel, a DCT-based 8-tap Interpolation Filter (DCT-based Interpolation Filter) having different Filter coefficients in order to generate pixel information of integer pixels or less in 1/4 pixel units may be used. In the case of color difference signals, a DCT-based 4-tap Interpolation Filter (DCT-based Interpolation Filter) having different Filter coefficients in order to generate pixel information of integer pixels or less in 1/8 pixel units may be used.

The motion prediction section may perform motion prediction based on the reference picture interpolated by the reference picture interpolation section. As a method for calculating a motion vector, various methods such as FBMA (Full Search-based Block Matching Algorithm), TSS (Three Step Search), NTS (New Three-Step Search Algorithm) and the like can be used. The motion vector may have a motion vector value of 1/2 or 1/4 pixel units based on the interpolated pixels. The motion prediction unit may predict the current prediction unit by using a different motion prediction method. As the Motion prediction method, various methods such as a Skip (Skip) method, a Merge (Merge) method, an amvp (advanced Motion Vector prediction) method, and an Intra Block Copy (Intra Block Copy) method can be used.

The intra prediction unit 125 may generate a prediction unit based on pixel information in the current picture, that is, reference pixel information around the current block. Since the peripheral block of the current prediction unit is a block on which inter prediction is performed, when the reference pixel is a pixel on which inter prediction is performed, the reference pixel included in the block on which inter prediction is performed may be used instead of the reference pixel information of the peripheral block on which intra prediction is performed. That is, if the reference pixel is unavailable, the unavailable reference pixel information may be used in place of at least one of the available reference pixels.

In the intra prediction, the prediction mode may have a directional prediction mode in which reference pixel information is used along a prediction direction, and a non-directional mode in which the directional information is not used when performing prediction. The mode for predicting luminance information and the mode for predicting color difference information may be different, and intra prediction mode information used for predicting luminance information or predicted luminance signal information may be used for predicting color difference information.

When performing intra prediction, if the size of the prediction unit and the size of the conversion unit are the same, intra prediction on the prediction unit may be performed based on pixels existing on the left side of the prediction unit, pixels existing on the upper left side, and pixels existing on the upper side. However, when performing intra prediction, if the size of the prediction unit and the size of the conversion unit are different, intra prediction may be performed using reference pixels based on the conversion unit. Also, intra prediction using NxN partition may be used only for the minimum coding unit.

In the Intra prediction method, the prediction block may be generated after applying an ais (adaptive Intra smoothing) filter to the reference pixel according to the prediction mode. The type of AIS filter that is applied to the reference pixel may vary. In order to perform the intra prediction method, the intra prediction mode of the current prediction unit may be predicted from intra prediction modes of prediction units existing at the periphery of the current prediction unit. In the case of predicting a prediction mode of a current prediction unit using mode information predicted from a peripheral prediction unit, if intra prediction modes of the current prediction unit and the peripheral prediction unit are the same, information that the prediction modes of the current prediction unit and the peripheral prediction unit are the same may be transmitted using predetermined flag (flag) information, and if the prediction modes of the current prediction unit and the peripheral prediction unit are different, entropy encoding may be performed to encode prediction mode information of a current block.

Further, a Residual block including Residual value (Residual) information that is a difference between the prediction unit in which the prediction is performed and the original block of the prediction unit may be generated based on the prediction unit generated by the

prediction units

120 and 125. The generated residual block may be input to the conversion part 130.

The conversion unit 130 may convert the residual block including the original block and the residual value (residual) information of the prediction unit generated by the

prediction units

120 and 125, by using a conversion method such as dct (discrete Cosine transform), dst (discrete Sine transform), or KLT. Whether DCT, DST, or KLT is applied to convert the residual block is determined based on intra prediction mode information of a prediction unit used to generate the residual block.

The quantization part 135 may quantize the values converted into the frequency region by the conversion part 130. The quantization coefficients may be changed according to blocks or according to the importance of the image. The value calculated by the quantization unit 135 may be supplied to the inverse quantization unit 140 and the rearrangement unit 160.

The rearranging section 160 may perform rearrangement of the coefficient values on the quantized residual values.

The rearranging section 160 may convert the two-dimensional block shape coefficients into a one-dimensional vector shape by a Coefficient Scanning (coeffient Scanning) method. For example, the rearrangement unit 160 may Scan the DC coefficients to the high frequency domain coefficients by a zigzag Scan (Zig-Zag Scan) method, and change the configuration to a one-dimensional vector configuration. Depending on the size of the conversion unit and the intra prediction mode, a vertical scan in which the two-dimensional block state coefficients are scanned in the column direction, a horizontal scan in which the two-dimensional block state coefficients are scanned in the row direction may be used instead of the zigzag scan. That is, which scanning method is used for zigzag scanning, vertical scanning, and horizontal scanning can be determined according to the size of the conversion unit and the intra prediction mode.

The entropy encoding section 165 may perform entropy encoding based on the value calculated by the rearranging section 160. The entropy Coding may use various Coding methods such as Exponential Golomb (explicit Golomb), CAVLC (Context-Adaptive Variable Length Coding), CABAC (Context-Adaptive Binary Arithmetic Coding), and the like, for example.

The entropy encoding unit 165 can encode, from the rearranging unit 160 and the

prediction units

120 and 125, a plurality of types of information such as residual value coefficient information and block type information of a coding unit, prediction mode information, partition unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, interpolation information of a block, and filter information.

The entropy encoding unit 165 may entropy encode the coefficient value of the coding unit input from the rearrangement unit 160.

In the inverse quantization unit 140 and the inverse conversion unit 145, inverse quantization is performed on the value quantized by the quantization unit 135, and inverse conversion is performed on the value converted by the conversion unit 130. The Residual value (Residual) generated by the inverse quantization unit 140 and the inverse transform unit 145 may be combined with the prediction units predicted by the motion estimation unit, the motion compensation unit, and the intra prediction unit included in the

prediction units

120 and 125 to generate a Reconstructed Block (Reconstructed Block).

The filtering part 150 may include at least one of a deblocking filter, an offset correction part, and an alf (adaptive Loop filter).

The deblocking filter may remove block distortion generated in the reconstructed picture due to boundaries between blocks. In order to determine whether to perform deblocking, it may be determined whether to apply a deblocking filter to the current block based on pixels included in several columns or rows included in the block. In the case of applying the deblocking Filter to the block, a Strong Filter (Strong Filter) or a Weak Filter (Weak Filter) may be applied according to the required deblocking filtering strength. Further, when the deblocking filter is applied, horizontal direction filtering and vertical direction filtering can be processed in parallel when vertical filtering and horizontal filtering are performed.

The offset correction unit may correct an offset between the image subjected to deblocking and the original image on a pixel-by-pixel basis. In order to correct the offset of a specific picture, a method of dividing pixels included in an image into a certain number of regions, determining a region to be subjected to offset, and applying the offset to the corresponding region, or a method of applying the offset in consideration of edge information of each pixel may be used.

Alf (adaptive Loop filtering) may be performed based on a value obtained by comparing the filtered reconstructed image with the original image. The pixels included in the image may be divided into predetermined groups, and one filter applied to each group may be determined, so that the filtering may be performed distinctively for each group. The information on whether or not the ALF is applied may be transmitted in a Coding Unit (CU) for a luminance signal, and may be different in the pattern of the ALF filter to be applied and the filter coefficient for each block. Furthermore, the ALF filter of the same form (fixed form) can be applied regardless of the characteristics of the application target block.

The storage unit 155 may store the reconstructed block or picture calculated by the filtering unit 150, and the stored reconstructed block or picture may be supplied to the

prediction units

120 and 125 when performing inter prediction.

Referring to fig. 2, the image decoder 200 may include an entropy decoding part 210, a rearranging part 215, an inverse quantization part 220, an inverse transformation part 225, prediction parts 230 and 235, a filtering part 240, and a storage part 245.

In the case of inputting an image bitstream in an image encoder, the input bitstream may be decoded in a reverse procedure to the image encoder.

The entropy decoding part 210 may perform entropy decoding in a reverse step of the entropy encoding performed in the entropy encoding part of the image encoder. For example, in correspondence with a method executed in an image encoder, various methods such as Exponential Golomb (explicit Golomb), CAVLC (Context-Adaptive Variable Length Coding), CABAC (Context-Adaptive Binary Arithmetic Coding) and the like can be applied.

The entropy decoding unit 210 can decode information related to intra prediction and inter prediction performed by the encoder.

The rearranging section 215 may perform rearrangement of the bitstream entropy decoded in the entropy decoding section 210 based on the method rearranged in the encoding section. The coefficients expressed in the form of one-dimensional vectors may be rearranged into coefficients in the form of two-dimensional blocks. In the rearranging section 215, rearranging may be performed by a method of receiving information on coefficient scanning performed in the encoding sections and scanning in the reverse direction based on the scanning order performed in the respective encoding sections.

The inverse quantization part 220 may perform inverse quantization based on a quantization parameter provided in the encoder and a coefficient value of the rearranged block.

The inverse conversion section 225 may perform inverse DCT, inverse DST, and inverse KLT, which are inverse conversions to the DCT, DST, and KLT, which are conversions performed in the conversion section, on the quantization result performed in the image encoder. The inverse conversion may be performed based on the transmission unit decided in the image encoder. In the inverse transform section 225 of the image decoder, a transform method (for example, DCT, DST, KLT) can be selectively executed according to a plurality of information such as a prediction method, the size of the current block, and a prediction direction.

The prediction parts 230, 235 may generate a prediction block based on the prediction block generation-related information provided in the entropy decoding part 210 and previously decoded block or picture information provided in the storage part 245.

As described above, in the same manner as the operation in the image encoder, when the size of the prediction unit and the size of the conversion unit are the same at the time of performing intra prediction, intra prediction on the prediction unit is performed based on the pixels existing on the left side of the prediction unit, the pixels existing on the upper left side, and the pixels existing on the upper end, but when the size of the prediction unit and the size of the conversion unit are different at the time of performing intra prediction, intra prediction is performed using the reference pixels based on the conversion unit. Also, intra prediction using NxN partition may be used only for the minimum coding unit.

The prediction units 230 and 235 may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit may receive input of a plurality of information such as prediction unit information, prediction mode information of an intra prediction method, and motion prediction related information of an inter prediction method, which are input from the entropy decoding unit 210, divide a prediction unit from a current coding unit, and determine whether the prediction unit performs inter prediction or intra prediction. The inter prediction unit 230 may perform inter prediction on the current prediction unit based on information included in at least one of a picture immediately preceding or a picture immediately following the current picture including the current prediction unit, using information required for inter prediction of the current prediction unit provided in the image encoder. Alternatively, inter prediction is performed based on information of a partial region reconstructed within a current picture including a current prediction unit.

In order to perform inter prediction, it is possible to determine, based on a coding unit, which of a Skip Mode (Skip Mode), a Merge Mode (Merge Mode), an AMVP Mode (AMVP Mode), and an intra block copy Mode is a motion prediction method for a prediction unit included in the corresponding coding unit.

The intra prediction part 235 may generate a prediction block based on pixel information within the current picture. In the case where the prediction unit is a prediction unit in which intra prediction is performed, intra prediction may be performed based on intra prediction mode information of the prediction unit provided in the image encoder. The Intra prediction unit 235 may include an ais (adaptive Intra smoothing) filter, a reference picture interpolation unit, and a DC filter. The AIS filter is a part that performs filtering on reference pixels of the current block, and whether to apply the filter may be decided according to a prediction mode of the current prediction unit. The AIS filtering may be performed on the reference pixels of the current block using the prediction mode of the prediction unit provided in the image encoder and the AIS filter information. In the case where the prediction mode of the current block is a mode in which the AIS filtering is not performed, the AIS filter may not be applicable.

The reference picture interpolation unit may interpolate a reference pixel to generate a reference pixel in pixel units of integer values or less when the prediction mode of the prediction unit is a prediction unit in which intra prediction is performed based on a pixel value obtained by interpolating the reference pixel. In the case where the prediction mode of the current prediction unit is a prediction mode in which a prediction block is generated without interpolating reference pixels, the reference pixels may not be interpolated. The DC filter may generate a prediction block through filtering in case that the prediction mode of the current block is the DC mode.

The reconstructed block or picture may be provided to the filtering part 240. The filtering unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

Information on whether a deblocking filter is applied to a corresponding block or picture, and information on whether a strong filter or a weak filter is applied if the deblocking filter is applied may be received from an image encoder. In the deblocking filter of the image decoder, deblocking filter-related information provided in the image encoder may be received, and deblocking filtering may be performed on a corresponding block in the image decoder.

The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image at the time of encoding, offset value information, and the like.

The ALF may be applied to the coding unit based on ALF application or non-application information, ALF coefficient information, and the like, which are provided from the encoder. Such ALF information may be provided by including in a specific set of parameters.

The storage 245 may store the reconstructed picture or block so as to be applicable as a reference picture or a reference block, and may also supply the reconstructed picture to the output.

As described above, in the following, in the embodiment of the present invention, for convenience of description, a Coding Unit (Coding Unit) is used as a term of a Coding Unit, but may be a Unit that performs decoding in addition to Coding.

Also, the current block represents an encoding/decoding object block, and may represent a coding tree block (or a coding tree unit), an encoding block (or an encoding unit), a transform block (or a transform unit), or a prediction block (or a prediction unit), etc. according to the encoding/decoding steps.

A picture can be divided into square or non-square basic blocks to be encoded/decoded. At this time, the basic block may be referred to as a Coding Tree Unit (Coding Tree Unit). A coding tree unit may be defined as a coding unit of the largest size allowed in a Sequence (Sequence) or slice (slice). Information about whether a coding tree unit is square or non-square, or the size of a coding tree unit, may be signaled (Signaling) by a sequence parameter set, a picture parameter set, or a slice header (SliceHeader), etc. The coding tree unit may be partitioned into smaller sized partitions. At this time, in the case where a partition generated by dividing the coding tree unit is set to depth 1(depth), a partition generated by dividing the partition set to depth 1 may be defined as depth 2. That is, a partition generated by dividing a partition set to a depth k within the coding tree unit may be defined to have a depth k + 1.

Partitions of an arbitrary size generated as the coding tree unit is divided may be defined as the coding unit. The coding unit may be recursively divided, or divided into basic units for performing prediction, quantization, conversion, or loop filtering (in-loop filter), etc. As an example, a partition of an arbitrary size generated as the coding unit is divided may be defined as a coding unit, or as a conversion unit or a prediction unit, which is a basic unit for performing prediction, quantization, conversion, loop filtering, or the like.

The coding tree unit or the partition of the coding unit may be performed based on at least one of a Vertical Line (Vertical Line) or a Horizontal Line (Horizontal Line). In addition, the number of vertical or horizontal lines partitioning the coding tree unit or the coding unit may be at least one. As an example, the coding tree unit or the coding unit may be divided into two partitions using one vertical line or one horizontal line, or may be divided into three partitions using two vertical lines or two horizontal lines. Alternatively, the coding tree unit or coding unit may be divided into four partitions of length and width 1/2 using one vertical line and one horizontal line.

In the case of dividing the coding tree unit or the coding unit into a plurality of partitions using at least one vertical line or at least one horizontal line, the partitions may have a uniform size or different sizes. Alternatively, any of the partitions may have a different size than the remaining partitions.

In the embodiments described later, it is assumed that the coding tree unit or the coding unit is divided into a quad tree, a tri tree, or a binary tree structure. However, the coding tree unit or the coding unit may be divided by a greater number of vertical lines or a greater number of horizontal lines.

Fig. 3 shows an example of hierarchical division of a coding block based on a tree structure (tree structure), to which an embodiment of the present invention is applied.

The input image signal is decoded into a predetermined block unit, and thus a basic unit for decoding the input image signal is called an encoding block. The coding block may become a unit for performing intra/inter prediction, conversion, and quantization. In addition, a prediction mode (for example, an intra-picture prediction mode or an inter-picture prediction mode) is determined in units of coding blocks, and prediction blocks included in the coding blocks can share the determined prediction mode. The coded blocks may be square or non-square blocks of any size, ranging from 8x8 to even 64x64, and may be square or non-square blocks of 128x128, 256x256, or more.

Specifically, the coding block may be hierarchically partitioned based on at least one of a quad tree (quad tree), a triple tree (triple tree), and a binary tree (binary tree). The quadtree-based partitioning may refer to a manner of partitioning a 2Nx2N coding block into four NxN coding blocks, the ternary tree-based partitioning may refer to a manner of partitioning one coding block into three coding blocks, and the binary tree-based partitioning may refer to a manner of partitioning one coding block into two coding blocks. Even if the ternary tree based partitioning or binary tree based partitioning is performed, the lower bit depth may have a square coding block. Alternatively, after performing the ternary tree based partitioning or binary tree based partitioning, the generation of the square coded block may also be limited at the lower depth.

The partitioning based on the binary tree can be performed in a symmetric manner or in an asymmetric manner. The coding blocks partitioned based on the binary tree may be square blocks or non-square blocks such as rectangles. For example, as shown in fig. 4, the partition type allowing the partition based on the binary tree may include at least one of 2NxN (horizontal non-square coding unit) or Nx2N (vertical non-square coding unit) in a symmetric form (symmetric), nLx2N, nRx2N, 2NxnU, or 2NxnD in an asymmetric form (asymmetric).

Based on the partitioning of the binary tree, either of symmetric or asymmetric morphological partitioning may be restrictively allowed. In this case, configuring the coding tree unit as a square block may correspond to a quad-tree CU partition, and configuring the coding tree unit as a symmetric non-square block may correspond to a binary-tree partition. The composition of the coding tree unit as square blocks and symmetric non-square blocks may correspond to quadtree and binary tree CU partitions.

The binary tree based partitioning may be performed on a coded block for which the quadtree based partitioning is no longer performed. For a coding block partitioned based on a binary tree, it may be set that at least one of a quadtree-based partition, a ternary tree-based partition, or a binary tree-based partition is not performed any more.

Alternatively, for a coding block partitioned based on a binary tree, while allowing partitioning based on a ternary tree or partitioning based on a binary tree, only either one of horizontal or vertical partitioning may be restrictively allowed.

For example, the coding block divided based on the binary tree may be restricted in further division or in further division direction according to the position, index (index), form, further division form of the adjacent partitions, and the like of the coding block divided based on the binary tree. For example, when the index of the coding block in the front of the coding sequence of the two coding blocks generated by the binary tree based division is set to 0 (hereinafter referred to as coding block index 0) and the index of the coding block in the back of the coding sequence is set to 1 (hereinafter referred to as coding block index 1), when the binary tree based division is applied to both the coding blocks of coding block index 0 and coding block index 1, the binary tree based division direction of the coding block of coding block index 1 can be determined according to the binary tree based division direction of the coding block of coding block index 0. Specifically, in the case where the binary tree based division direction of the coding block with the coding block index of 0 is to divide the coding block with the coding block index of 0 into square partitions, the binary tree based division of the coding block with the coding block index of 1 may be restricted to have a direction different from the binary tree based division of the coding block with the coding block index of 1. That is, the coding blocks of coding block index 0 and coding block index 1 can be restricted from being divided into square partitions. In this case, encoding/decoding of information indicating the binary tree split direction of the coding block having the coding block index of 1 can be omitted. This is because the coding blocks of coding block index 0 and coding block index 1 are each divided into square partitions, exhibiting the same effect as dividing the upper depth block based on the quadtree, and allowing the coding block index 0 and coding block index 1 to be each divided into square partitions is not preferable in terms of coding efficiency.

The ternary tree based partitioning means that the coding block is partitioned into three partitions in the horizontal or vertical direction. The three partitions generated by the ternary tree based partitioning may each have a different size. Alternatively, two of the partitions generated by the partition based on the ternary tree may have the same size, and the remaining one may have a different size. For example, the width ratio or the height ratio of the partitions generated by dividing the coding block may be set to 1: n:1, 1:1: n, n:1:1, or m: n:1 along the dividing direction. Where m and n are 1 or a real number greater than 1, and may be an integer such as 2, for example.

The ternary tree based partitioning may be performed on coding blocks for which the quaternary tree based partitioning is no longer performed. For an encoded block partitioned based on a ternary tree, it may be set that at least one of a quaternary tree based partition, a ternary tree based partition, or a binary tree based partition is no longer performed.

Or, the coding block divided based on the ternary tree is allowed to be divided based on the ternary tree or divided based on the binary tree, and meanwhile, only any one of the division in the horizontal direction or the vertical direction is allowed in a limiting way.

For example, the coding block divided based on the ternary tree may be restricted in further division or in further division direction according to the position, index, form, size, further division form of the adjacent partitions, and the like of the coding block divided based on the ternary tree. For example, the partition having the largest size in the coding block generated by the partition based on the ternary tree may be limited to either the horizontal direction partition or the vertical direction partition. Specifically, for a partition having the largest size among the coding blocks generated by the partition based on the ternary tree, binary tree partition in the same direction as the ternary tree partition direction of the upper depth partition or ternary tree partition direction in the same direction may not be allowed. In this case, encoding/decoding of information indicating the binary tree division direction or the ternary tree division direction may be omitted for the largest partition of the coding blocks divided based on the ternary tree.

Binary tree or ternary tree based partitioning may be restricted according to the size or shape of the current block. Wherein the size of the current block can be represented based on at least one of the width, the height, the minimum/maximum value of the width/height, the sum of the width and the height, the product of the width and the height, or the number of samples contained in the current block. As an example, in case at least one of the width or the height of the current block is larger than a predefined value, binary tree or ternary tree based partitioning may not be allowed. The value defined in advance may be an integer of 16, 32, 64, 128, or the like. As another example, in case the ratio of the width to the height of the current block is greater than a predefined value or less than a predefined value, binary tree or ternary tree based partitioning may not be allowed. In case the predefined value is 1, a binary tree or ternary tree based partitioning may be allowed only for the case where the current block is a square block of the same width and height.

The lower depth division may be determined depending on the division form of the upper depth. For example, when two or more depths allow binary tree based segmentation, only binary tree based segmentation having the same form as that of binary tree segmentation of an upper bit depth may be allowed at a lower bit depth. For example, in the case where the binary tree based segmentation is performed in the 2NxN form at the upper bit depth, the binary tree based segmentation of the 2NxN form may be performed also at the lower bit depth. Alternatively, in the case where the binary tree based segmentation is performed in the Nx2N mode at the upper bit depth, the binary tree based segmentation of the Nx2N mode may be allowed at the lower bit depth as well.

In contrast, at the lower bit depth, only binary tree based partitioning whose morphology differs from that of binary tree partitioning at the upper bit depth may be allowed.

For sequences, slices, coding tree units or coding units, they may be restricted to use only binary tree based segmentation of a specific modality or ternary tree based segmentation of a specific modality. As an example, the coding tree unit may be restricted to allow only binary tree based partitioning of 2NxN or Nx2N morphology. The allowable partition type may be defined in advance in an encoder or a decoder, or information on the allowable partition type or the unallowable partition type may be encoded and signaled in a bit stream (Signaling).

Fig. 5 is a diagram showing an example of binary tree based segmentation that allows only a specific modality. Fig. 5 (a) shows an example of binary tree based segmentation limited to only the Nx2N mode, and fig. 5 (b) shows an example of binary tree based segmentation limited to only the 2NxN mode. In order to achieve adaptive partitioning based on the above-described quad tree or binary tree, information indicating partitioning based on the quad tree, information on the size/depth of a coding block for which partitioning based on the quad tree is allowed, information indicating partitioning based on the binary tree, information on the size/depth of a coding block for which partitioning based on the binary tree is allowed, information on the size/depth of a coding block for which partitioning based on the binary tree is not allowed, or information on whether partitioning based on the binary tree is vertical or horizontal, or the like may be utilized.

Further, the number of times binary tree division/ternary tree division is allowed, the depth of binary tree division/ternary tree division is allowed, the number of depths of binary tree division/ternary tree division is allowed, or the like may be acquired for the coding tree unit or the prescribed coding unit. The above information may be encoded in coding tree units or coding unit units and transmitted to a decoder through a bitstream.

As an example, the syntax 'max _ binary _ depth _ idx _ minus1' indicating the maximum depth at which binary tree partitioning is allowed may be encoded/decoded by a bitstream. In this case, max _ binary _ depth _ idx _ minus1+1 may refer to the maximum depth at which binary tree partitioning is allowed.

As can be seen from an observation of the example shown in fig. 6, binary tree splitting is performed for the coding unit of depth 2 and the coding unit of depth 3. Thus, at least one of information indicating the number of times (twice) binary tree splitting is performed in the coding tree unit, information indicating the maximum depth (depth 3) in which binary tree splitting is allowed in the coding tree unit, or information indicating the number of depths (two, depth 2 and depth 3) in which binary tree splitting is allowed in the coding tree unit can be encoded/decoded by a bit stream.

As another example, at least one of the number of times the binary tree division/ternary tree division is allowed, the depth of the binary tree division/ternary tree division is allowed, or the number of depths of the binary tree division/ternary tree division is allowed may be acquired in terms of a sequence, a picture, or a slice. For example, the information may be encoded in units of a sequence, a picture, or a slice and transmitted through a bitstream. Alternatively, the number of depths allowing binary/ternary tree division or depths allowing binary/ternary tree division is defined in advance in terms of a sequence, a picture, or a slice. Thus, at least one of the number of binary/ternary tree splits, the maximum depth at which binary/ternary tree splits are allowed, or the number of depths at which binary/ternary tree splits are allowed of the first and second stripes may be different. As an example, in the first stripe, binary tree splitting is allowed only at one depth, whereas in the second stripe, binary tree splitting is allowed at two depths.

As another other example, at least one of the number of times the binary/ternary tree division is allowed, the depth of the binary/ternary tree division is allowed, or the number of depths of the binary/ternary tree division is allowed may be set to be different according to a temporal level identifier (temporalld) of a slice or a picture. Wherein the temporal level identifier (temporalld) is for identifying each of a plurality of layers of the image having Scalability (Scalability) of at least one of view (view), space (spatial), temporal (temporal) or picture quality (quality).

As shown in fig. 3, a first coding block 300 with a split depth (split depth) of k may be split into a plurality of second coding blocks based on a quadtree (quad tree). For example, the second encoding blocks 310 to 340 may be square blocks having a size of half the width and height of the first encoding block, and the division depth of the second encoding block is increased to k + 1.

The second coding block 310 with the split depth of k +1 may be split into a plurality of third coding blocks with the split depth of k + 2. The division of the second encoding block 310 may be selectively performed using any one of a quad tree or a binary tree according to the division manner. Wherein the division manner may be decided based on at least one of information indicating the quadtree-based division or information indicating the binary tree-based division.

In the case of dividing the second encoding block 310 based on the quadtree, the second encoding block 310 may be divided into four third encoding blocks 310a having a size of half the width and height of the second encoding block, and the division depth of the third encoding blocks 310a is increased to k + 2. In contrast, in the case where the second encoding block 310 is partitioned based on a binary tree, the second encoding block 310 may be partitioned into two third encoding blocks. At this time, the two third encoding blocks are respectively non-square blocks in which any one of the width and the height of the second encoding block is half the size, and the division depth is increased to k + 2. The second coding block may be decided as a non-square block of either the horizontal or vertical direction according to the splitting direction, which may then be decided based on information about whether the binary tree based splitting is vertical or horizontal.

On the other hand, the second coding block 310 may be decided as an end coding block that is no longer based on a quadtree or binary tree split, in which case the corresponding coding block may be used as a prediction block or a transform block.

The third coding block 310a may be decided to be an end coding block in the same way as the division of the second coding block 310, or further divided based on a quad tree or a binary tree.

On the other hand, the third coding block 310b divided based on the binary tree may be further divided into a vertical coding block 310b-2 or a horizontal coding block 310b-3 based on the binary tree, and the division depth of the corresponding coding block is increased to k + 3. Alternatively, the third encoding block 310b may be decided not to be based on the end encoding block 310b-1 of the binary tree partition any more, in which case the corresponding encoding block 310b-1 may be used as a prediction block or a transform block. The above-described segmentation process may be restrictively performed based on at least one of information on a size/depth of a coding block allowing the quadtree-based segmentation, information on a size/depth of a coding block allowing the binary tree-based segmentation, or information on a size/depth of a coding block not allowing the binary tree-based segmentation.

The size that the coding blocks can have may be limited to a prescribed number, or the size of the coding blocks within a prescribed unit may have a fixed value. As an example, the size of the intra-sequence coding blocks or the size of the intra-picture coding blocks may be limited to 256x256, 128x128, or 32x 32. The information indicating the size of the coding block within a sequence or picture may be signaled by a sequence header or a picture header.

Based on the results of quadtree, binary tree, and ternary tree partitioning, the coding unit may be square or rectangular of any size.

The encoded block may be encoded/decoded using at least one of a skip mode, intra-picture prediction, inter-picture prediction, or skip method.

As another example, intra prediction or inter prediction may be performed in the same size as or smaller than a coding block by division of the coding block. For this reason, when the coding Block is decided, a Prediction Block (Prediction Block) may be decided by Prediction division of the coding Block. Predictive division of an encoded block may be performed by a partition mode (Part _ mode) indicating a division form of the encoded block. The size or shape of the prediction block may be decided according to the partition mode of the coding block. As an example, the size of the prediction block decided according to the partition mode may have a value equal to or smaller than the size of the coding block.

Fig. 7 is a diagram showing partition patterns applicable to a coding block when the coding block is coded by inter-picture prediction.

In the case where an encoding block is encoded by inter-picture prediction, any one of 8 partition modes may be applied to the encoding block as shown in the example of fig. 7.

In case of encoding an encoding block by intra-picture prediction, a partition mode PART _2Nx2N or PART _ NxN may be applied to the encoding block.

PART _ NxN may be applicable in case that the coded block has a minimum size. Wherein the minimum size of the coding block can be defined in advance in the encoder as well as in the decoder. Alternatively, information about the minimum size of the coding block may be signaled via a bit stream. As an example, the minimum size of the coding block may be signaled by a slice header, and thus, the minimum size of the coding block may be defined for each slice.

In general, the size of the prediction block may have a size from 64x64 to 4x 4. However, in the case of encoding an encoding block by inter-picture prediction, when motion compensation is performed, a prediction block may not have a size of 4 × 4 in order to reduce a memory bandwidth (memory bandwidth).

Fig. 8 shows the types of intra prediction modes defined in advance in the image encoder/decoder, to which the present invention is applied.

The image encoder/decoder may perform intra prediction using any one of the previously defined intra prediction modes. The pre-defined intra prediction modes for intra prediction may be composed of a non-directional prediction mode (e.g., a planar mode, a DC mode) and 33 directional prediction modes (directional prediction modes).

Alternatively, in order to improve the accuracy of intra prediction, a greater number of directional prediction modes than 33 directional prediction modes may be utilized. That is, the angle (angle) of the directional prediction mode may be further subdivided to define M extended directional prediction modes (M >33), and a directional prediction mode having a predetermined angle may be derived and used by using at least one of the 33 directional prediction modes defined in advance.

Specifically, a greater number of intra prediction modes than the 35 intra prediction modes shown in fig. 8 may be utilized. The intra prediction modes using more than the 35 intra prediction modes shown in fig. 8 may be referred to as extended intra prediction modes.

Fig. 9 shows an example of the extended intra prediction mode, and the extended intra prediction mode may be composed of 2 non-directional prediction modes and 65 extended directional prediction modes. The extended intra prediction mode may be equally used for the luminance component and the color difference component, or the intra prediction modes may be used in different numbers for the components. For example, 67 extended intra prediction modes may be used in the luminance component, and 35 intra prediction modes may be used in the color difference component.

Alternatively, intra prediction may be performed using a different number of intra prediction modes according to a color difference format (format). For example, in the case of the 4:2:0 format, intra prediction may be performed using 67 intra prediction modes in the luminance component and 35 intra prediction modes in the color difference component, and in the case of the 4:4:4 format, intra prediction may be performed using 67 intra prediction modes for both the luminance component and the color difference component.

Alternatively, intra prediction may be performed using a different number of intra prediction modes according to the size and/or shape of the block. That is, intra prediction may be performed using 35 intra prediction modes or 67 intra prediction modes according to the size and/or shape of a PU or CU. For example, in the case where the size of a CU or PU is less than 64x64 or is asymmetric partition (asymmetric partition), intra prediction may be performed using 35 intra prediction modes, and in the case where the size of a CU or PU is equal to or greater than 64x64, intra prediction may be performed using 67 intra prediction modes. 65 directional Intra prediction modes may be allowed in Intra _2Nx2N, and only 35 directional Intra prediction modes may be allowed in Intra _ NxN.

The size of the block to which the extended intra prediction mode is applicable may be differently set for each sequence, picture, or slice. As an example, in the first slice, the extended intra prediction mode may be set to be applicable to blocks (e.g., CUs or PUs) larger than 64x64, and in the second slice, the extended intra prediction mode is set to be applicable to blocks larger than 32x 32. Information indicating the size of a block to which the extended intra prediction mode is applicable may be signaled for each sequence, picture, or slice unit. For example, the information indicating the size of the block to which the extended intra prediction mode is applied may be defined as "log 2_ extended _ intra _ mode _ size _ minus 4" obtained by taking a logarithmic value for the size of the block and subtracting an integer of 4. For example, a value of log2_ extended _ intra _ mode _ size _ minus4 of 0 may indicate that the extended intra prediction mode can be applied to a block having a size of 16x16 or more or a block having a size larger than 16x16, and a value of log2_ extended _ intra _ mode _ size _ minus4 of 1 may indicate that the extended intra prediction mode can be applied to a block having a size of 32x32 or more or a block having a size larger than 32x 32.

As described above, the number of intra prediction modes can be determined in consideration of at least one of color difference components, color difference formats, block sizes, and shapes. Not limited to the above example, the intra prediction mode candidates (for example, the number of MPMs) used to determine the intra prediction mode of the encoding/decoding target block may be determined according to at least one of the color difference components, the color difference format, the block size, and the form. Further, a larger number of intra prediction modes than those shown in fig. 8 may also be utilized. For example, the directional prediction modes shown in fig. 8 may be further subdivided to use 129 directional prediction modes and 2 non-directional prediction modes. As in the above example, whether to use more intra prediction modes than the number shown in fig. 8 may be decided in consideration of at least one of color difference components, block sizes, or morphologies.

A method of deciding an intra prediction mode of an encoding/decoding object block and a method of performing intra prediction using the decided intra prediction mode will be described with reference to the drawings described later.

Fig. 10 is a flowchart schematically illustrating an intra prediction method according to an embodiment of the present invention.

Referring to fig. 10, an intra prediction mode of a current block may be decided (S1000).

Specifically, the intra prediction mode of the current block may be derived based on the candidate list and the index. Wherein the candidate list may include a plurality of candidates, and the plurality of candidates may be decided based on intra prediction modes of peripheral blocks adjacent to the current block. The peripheral block may include at least one of blocks located at an upper end, a lower end, a left side, a right side, or a corner of the current block. The index may specify any one of a plurality of candidates belonging to the candidate list. The candidates specified by the above-described index may be set as the intra prediction mode of the current block.

The intra prediction mode used for intra prediction for the peripheral block may be set as a candidate. As an example, the candidates may be derived based on intra prediction modes of a left block, an upper block, a left lower corner neighboring block, a right upper corner neighboring block, and a left upper corner neighboring block of the current block. If the peripheral blocks are encoded by intra prediction, candidates for the current block may be derived using intra prediction modes of Collocated blocks (Collocated blocks) of the peripheral blocks.

Further, an intra prediction mode having a similar directivity to that of the intra prediction mode of the peripheral block may be set as a candidate. The intra prediction modes having similar directivity can be determined by adding or subtracting a predetermined constant value to or from the intra prediction modes of the peripheral blocks. The predetermined constant value may be 1, 2, or an integer greater than or equal to 1, and may be determined adaptively according to the number of usable intra prediction modes. For example, the predetermined constant value may be set to 1 when the number of available intra prediction modes is 35, and may be set to 2 when the number of available intra prediction modes is 67. When the number of available intra prediction modes is 131, the predetermined constant value may be set to 4.

The candidate list may also include a default mode. The default mode may include at least one of a Planar mode (Planar mode), a DC mode, a vertical mode, a horizontal mode, a top right diagonal mode, a top left diagonal mode. The default mode may be adaptively added in consideration of the maximum number of candidates that the candidate list of the current block can contain.

The maximum number of candidates that can be included in the candidate list may be 3, 4, 5, 6, 7, or more. The maximum number of candidates that can be included in the candidate list may be a fixed value set in advance in the image encoder/decoder, or may be variably determined based on the attribute of the current block. The attribute may indicate a position, size, and shape of the block, the number and type of intra prediction modes that can be used for the block, a color difference attribute, a color difference format, and the like. Alternatively, information indicating the maximum number of candidates that can be included in the candidate list may be separately signaled, and the maximum number of candidates that can be included in the candidate list may be variably determined using the information. The information indicating the maximum number of candidates may be signaled in at least one of a sequence level, a picture level, a slice level, or a block level.

The candidates included in the candidate list may be sorted in a predefined order. As an example, the candidates may be arranged in the candidate list in the order of the left block, the upper block, the left lower block, the right upper block, and the left upper block. Alternatively, the order of arrangement of the candidates may be variably decided according to the size or shape of the current block. As an example, if the current block is a non-square block having a height greater than a width, the intra prediction modes of the column blocks may be arranged at a higher priority than the intra prediction mode of the left block.

In the case where the extended intra prediction mode and the 35 intra prediction modes defined in advance are selectively used, candidates may be derived by converting intra prediction modes of the peripheral blocks into indexes corresponding to the extended intra prediction mode or into indexes corresponding to the 35 intra prediction modes. For the conversion of the index, a table defined in advance may be used, or a scaling operation based on a predetermined value may be used. Among them, the previously defined table may define a mapping relationship between different intra prediction mode groups (e.g., extended intra prediction mode and 35 intra prediction modes).

For example, in the case where the left peripheral block uses 35 intra prediction modes and the intra prediction mode of the left peripheral block is 10 (horizontal mode), it may be converted into an index 16 corresponding to the horizontal mode in the extended intra prediction mode.

Alternatively, in the case where the extended intra prediction mode is used for the upper-end peripheral block, and the intra prediction mode index of the upper-end peripheral block is 50 (vertical mode), it may be converted into the index 26 corresponding to the vertical mode in 35 intra prediction modes.

The intra prediction mode may be derived independently of each other for each of the luminance component and the color difference component based on the above-described intra prediction mode decision method, and the color difference component may be derived depending on the intra prediction mode of the luminance component.

Specifically, the intra prediction mode of the color difference component can be determined based on the intra prediction mode of the luminance component, as shown in table 1 below.

[ Table 1]

In table 1, intra _ chroma _ pred _ mode indicates information signaled to specify an intra prediction mode for a color difference component, and intrapredmode indicates an intra prediction mode for a luminance component.

When the candidate list is decided, information indicating whether the same candidate as the intra prediction mode of the current block is included in the candidate list may be decoded. In the case where the above information indicates that the same candidate as the intra prediction mode of the current block is included in the candidate list, index information (e.g., MPM _ index) indicating any one of the candidates may be decoded. The intra prediction mode of the current block may be set to be the same as the candidate intra prediction mode referred to by the above index information.

In contrast, in the case where the above-described information indicates that the same candidate as the intra prediction mode of the current block is not included in the candidate list, residual intra prediction mode information (e.g., rem _ intra _ mode) that specifies any one of the residual intra prediction modes other than the candidate may be decoded. The intra prediction mode of the current block may be decided based on the intra prediction mode to which the above-described residual intra prediction mode information refers. For example, the intra prediction mode indicated by the residual intra prediction mode may be compared with the candidates to determine the current intra prediction mode. For example, in the case where the candidate intra prediction mode is smaller than the intra prediction mode indicated by the residual intra prediction mode, 1 may be added to the residual intra prediction mode to derive the intra prediction mode of the current block.

Referring to fig. 10, reference samples for intra prediction of a current block may be derived (S1010).

Specifically, reference samples for intra prediction may be derived based on the peripheral samples of the current block. The surrounding samples may represent reconstructed samples of the surrounding block described above, which may be reconstructed samples before the loop filter is applied or reconstructed samples after the loop filter is applied.

The peripheral samples reconstructed before the current block may be used as reference samples, and the peripheral samples filtered based on a prescribed intra filter may be used as reference samples. Filtering the surrounding samples with an intra filter may be referred to as reference sample smoothing. The intra filter may include at least one of a first intra filter applied to a plurality of peripheral samples located at the same horizontal line or a second intra filter applied to a plurality of peripheral samples located at the same vertical line. Either the first intra filter or the second intra filter may be selectively applied according to the positions of the peripheral samples, or 2 intra filters may be repeatedly used. At this time, the filter coefficient of at least one of the first intra filter or the second intra filter may be (1, 2, 1), but is not limited thereto.

The filtering may be adaptively performed based on at least one of an intra prediction mode of the current block or a size of a transform block with respect to the current block. For example, in the case where the intra prediction mode of the current block is a DC mode, a vertical mode, or a horizontal mode, filtering may not be performed. In the case where the size of the above conversion block is NxM, filtering may not be performed. Wherein N and M may be the same or different values, and may be any one of 4, 8, 16, or more. As an example, in the case where the size of the conversion block is 4x, filtering may not be performed. Alternatively, the filtering may be selectively performed based on a result of comparison between a difference between the intra prediction mode and the vertical mode (or the horizontal mode) of the current block and a previously defined threshold (threshold). For example, the filtering may be performed only in the case where a difference between the intra prediction mode and the vertical mode of the current block is greater than a threshold value. As shown in table 2, the above threshold may be defined in terms of the size of the conversion block.

[ Table 2]

8x8 conversion	16x16 conversion	32x32 conversion
			Threshold value
7	1	0

The above-described intra filter may be decided as any one of a plurality of intra filter candidates defined in advance in the image encoder/decoder. To this end, a further index specifying the intra filter of the current block among the plurality of intra filter candidates may be signaled. Alternatively, the intra filter may be decided based on at least one of the size/shape of the current block, the size/shape of the transform block, information on the filtering strength (strength), or the variation (variation) of the surrounding samples.

The intra prediction of the current block may be performed using a plurality of reference sample lines. As an example, it can be performed using 2 or more reference sample lines.

Whether to perform intra prediction using a plurality of reference sample lines may be adaptively determined according to the size, morphology, intra prediction mode, or the like of the current block. As an example, in the case where the intra prediction mode of the current block is a non-directional intra prediction mode or an intra prediction mode in a specific direction, it may be restricted that intra prediction is performed using a plurality of reference sample lines. The specific direction may include a vertical direction, a horizontal direction, a diagonal direction, or the like.

Referring to fig. 10, intra prediction may be performed using an intra prediction mode of a current block and reference samples (S1020).

That is, the prediction sample of the current block may be acquired using the intra prediction mode decided in S1000 and the reference sample derived in S1010. In the case where intra prediction is performed using a plurality of reference sample lines, prediction samples may be obtained based on weighting of reference samples belonging to different reference sample lines. As an example, the prediction sample may be derived based on a weighting of a first reference sample belonging to a first reference sample line and a second reference sample belonging to a second reference sample line. In this case, the weights applied to the first reference sample and the second reference sample may have the same value or different values depending on the distance from the prediction target sample. For example, a higher weight may be given to the one of the first reference sample and the second reference sample that is closer to the prediction target sample.

In the case of intra prediction alone, since boundary samples of peripheral blocks are used, there is a possibility that the image quality of a predicted image is degraded. Therefore, a correction process for the prediction samples generated by the above prediction process may also be included, which is described in detail below with reference to fig. 11. It is obvious that the correction process described below is applicable not only to intra-predicted samples but also to inter-predicted samples or reconstructed samples.

FIG. 11 is a flowchart illustrating a method for correcting a prediction sample of a current block based on difference information of neighboring samples according to an embodiment of the present invention.

The prediction sample of the current block may be corrected based on difference information of a plurality of peripheral samples with respect to the current block. The correction may be performed for all prediction samples belonging to the current block, or may be performed for only prediction samples belonging to a predetermined partial region. The partial region may be one line or a plurality of lines or columns, or may be a region set in advance for correction in the image encoder/decoder. As an example, a line/column at the boundary of the current block or a plurality of lines/columns from the boundary of the current block may be compensated. Alternatively, the partial region may be variably decided based on at least one of the size/shape of the current block or the intra prediction mode.

The peripheral samples may belong to at least one of peripheral blocks located at upper, left, and upper left corners of the current block. The number of the peripheral samples used for correction may be 2, 3, 4, or more. The positions of the neighboring samples can be variably determined based on the position of the prediction sample as a correction target in the current block. Alternatively, a part of the peripheral samples may have a fixed position regardless of the position of the prediction sample as the correction target, and the remaining part may have a variable position depending on the position of the prediction sample as the correction target.

The difference information of the peripheral samples may represent the difference samples between the peripheral samples, or may represent a value obtained by scaling the difference samples to a predetermined constant value (for example, 1, 2, 3, or the like). The predetermined constant value may be determined in consideration of the position of the prediction sample to be corrected, the position of the column or row to which the prediction sample to be corrected belongs, the position of the prediction sample in the column or row, and the like.

For example, in the case where the intra prediction mode of the current block is the vertical mode, the final prediction sample may be obtained as in the following equation 1 using a differential sample between the peripheral sample p (-1, y) adjacent to the left side boundary of the current block and the upper-left peripheral sample p (-1, -1).

[ mathematical formula 1]

P′(0，y)＝P(0，y)+((p(-1，y)-p(-1，-1))＞＞1fory＝0...N-1

For example, in the case where the intra prediction mode of the current block is the horizontal mode, the final prediction sample may be obtained as shown in the following equation 2 using a differential sample between the upper-left upper-end peripheral sample p (-1, -1) and the peripheral sample p (x, -1) adjacent to the upper-end boundary of the current block.

[ mathematical formula 2]

P′(x，0)＝p(x，0)+((p(x，-1)-p(-1，-1))＞＞1forx＝0...N-1

For example, in the case where the intra prediction mode of the current block is a vertical mode, the final prediction sample may be obtained using a differential sample between a peripheral sample p (-1, y) adjacent to the left side boundary of the current block and an upper-left peripheral sample p (-1, -1). In this case, the difference sample may be added to the prediction sample, or the difference sample may be scaled to a predetermined constant value and then added to the prediction sample. The prescribed constant value utilized for scaling may be decided differently depending on the column and/or row. For example, the prediction samples may be corrected as shown in the following

equations

3 and 4.

[ mathematical formula 3]

P′(0，y)＝P(0，y)+((p(-1，y)-p(-1，-1))＞＞1fory＝0...N-1

[ mathematical formula 4]

P′(1，y)＝P(1，y)+((p(-1，y)-p(-1，-1))＞＞2fory＝0...N-1

For example, in the case where the intra prediction mode of the current block is the horizontal mode, the final prediction sample may be obtained using a differential sample between the peripheral sample p (x, -1) adjacent to the upper boundary of the current block and the upper-left peripheral sample p (-1, -1), as described in the vertical mode. For example, the prediction samples may be corrected as shown in the following

equations

5 and 6.

[ math figure 5]

P′(x，0)＝p(x，0)+((p(x，-1)-p(-1，-1))＞＞1forx＝0...N-1

[ mathematical formula 6]

P′(x，1)＝p(x，1)+((p(x，-1)-p(-1，-1))＞＞2forx＝0...N-1

In case that the intra prediction mode of the current block is the directional prediction mode, the intra prediction of the current block may be performed based on the directivity of the directional prediction mode. As an example, table 3 shows intra direction parameters (intraPredAng) from mode 2 to mode 34, which are directional intra prediction modes shown in fig. 8.

[ Table 3]

In table 3, 33 directional intra prediction modes are illustrated as an example, but a greater or lesser number of directional intra prediction modes than this may be defined.

The intra direction parameter for the current block may be decided based on a lookup table defining a mapping relationship of the directional intra prediction mode and the intra direction parameter. Alternatively, the intra direction parameter for the current block may also be decided based on information signaled through a bitstream.

The intra prediction of the current block may be performed using at least one of the left-side reference sample or the upper-side reference sample according to the directivity of the directional intra prediction mode. Here, the upper-end reference sample may represent a reference sample (e.g., -1) to (2W-1, -1)) having a smaller y-axis coordinate than the prediction target sample (x, 0) included in the uppermost row within the current block, and the left-side reference sample may represent a reference sample (e.g., -1) to (-1, 2H-1)) having a smaller x-axis coordinate than the prediction target sample (0, y) included in the leftmost column within the current block.

The reference samples of the current block may be one-dimensionally arranged according to the directionality of the intra prediction mode. Specifically, when both the upper-side reference sample and the left-side reference sample need to be used for intra prediction of the current block, the reference sample of each prediction target sample is selected after the upper-side reference sample and the left-side reference sample are aligned in a line in the vertical or horizontal direction.

For example, in the case where the intra direction parameter is a negative number (for example, in the case of the intra prediction mode corresponding to the mode 11 to the mode 25 in table 3), the one-dimensional reference sample group (P _ ref _1D) may be configured by rearranging the upper-end reference sample and the left-side reference sample in the horizontal or vertical direction.

Fig. 12 and 13 show a one-dimensional reference sample group in which reference samples are rearranged in a line.

It is possible to decide whether to rearrange the reference samples in the vertical direction or in the horizontal direction according to the directionality of the intra prediction mode. For example, in the case where the intra prediction mode index is between 11 and 18, the one-dimensional reference sample group in which the left-side reference sample and the upper-side reference sample are arranged in the vertical direction may be generated by rotating the upper-side reference sample of the current block in the counterclockwise direction as in the example shown in fig. 12.

In contrast, in the case where the intra prediction mode index is between 19 to 25, the left reference sample of the current block may be rotated in the clockwise direction as in the example shown in fig. 13 to generate a one-dimensional reference sample group in which the left reference sample and the upper reference sample are arranged in the horizontal direction.

In the case where the intra direction parameter of the current block is not a negative number, the intra prediction for the current block may be performed using only the left side reference sample or the upper side reference sample. In this way, for an intra prediction mode in which the intra direction parameter is not negative, a one-dimensional reference sample group can be generated using only the left side reference sample or the upper side reference sample.

A reference sample decision index ididx for specifying at least one reference sample used in prediction of a prediction target sample may be derived based on the intra direction parameter. Furthermore, a weight-related parameter i for determining a weight to be applied to each reference sample may be derived based on the intra-frame direction parameter_fact. For example, the following

equations

7 and 8 show examples of deriving the reference sample determination index and the weight correlation parameter.

[ math figure 7]

iIdx＝(y+1)*(P_ang/33)

ifact＝[(y+1)*P_ang]31

As shown in math figure 7, ididx and i are variably decided according to the slope of the directional intra prediction mode_fact. At this time, the reference sample designated by the ididx may correspond to an integer pel (integer pel).

The index may be determined based on reference samples, and at least one reference sample may be assigned for each prediction target sample. For example, the index may be determined based on the reference sample, and the position of the reference sample in the one-dimensional reference sample group used for predicting the current intra prediction target sample may be specified. A prediction image (i.e., a prediction sample) for the prediction target sample can be generated based on the reference sample at the specified position.

When the intra prediction mode of the current block is considered, in the case where the prediction target sample can be predicted by only one reference sample, a prediction image for the prediction target sample can be generated based on the reference sample specified by the intra prediction mode of the current block.

As an example, if an integer pel (integer pel) (i.e., a reference sample at an integer position) passes through a one-dimensional reference sample group according to an angle of an intra prediction mode or a virtual angle line (integer line) of a slope of the intra prediction mode, a prediction image for a prediction target sample may be generated by copying the reference sample at the integer pel position or considering a position between the reference sample at the integer pel position and the prediction target sample. As an example, the following equation 8 shows an example of generating a prediction image P (x, y) for a prediction target sample by copying the reference sample P _ ref _1D (x + ididx +1) in the one-dimensional reference sample group specified by the intra prediction mode of the current block.

[ mathematical formula 8]

P(x，y)＝P_ref_1D(x+iIdx+1)

When it is determined that the prediction target sample cannot be predicted by only one reference sample in consideration of the intra prediction mode of the current block, prediction on the prediction target sample may be performed using a plurality of reference samples. Specifically, prediction for a prediction target sample may be performed by performing linear interpolation or Tap filter (Tap filter) -based interpolation on a reference sample at a predetermined position and an adjacent reference sample adjacent to the reference sample at the predetermined position according to the intra prediction mode of the current block. The number of taps of the interpolation filter may be a natural number of 2 or more. Specifically, the number of taps of the Tap filter (Tap filter) may be 2, 3, 4, 5, or 6 or an integer greater than or equal to that of the reference samples to be interpolated.

For example, if an integer pel (i.e., a reference sample at an integer position) does not pass through a virtual angular line (angular line) according to an angle of an intra prediction mode or a slope of the intra prediction mode, a prediction image for a prediction target sample may be generated by interpolating the reference sample on the angular line and a reference sample adjacent to the left/right or upper/lower of the reference sample. As an example, the following equation 9 shows an example of generating a prediction sample P (x, y) for a prediction target sample by interpolating two or more reference samples.

[ mathematical formula 9]

P(x，y)＝(32-i_fact)/32*P_ref_1D(x+iIdx+1)+i_fact32*P_ref_1D(x+iIdx+2)

The coefficients of the interpolation filter may be based onWeight-related parameter i_factAnd (6) determining. As an example, the interpolation filter may be determined based on a distance between fractional pixels (fractional pels) located on an angle line (angular line) and integer pixels (i.e., integer positions of respective reference samples).

The following equation 10 shows a case where the tap number of the tap filter is 4.

[ mathematical formula 10]

P(x，y)＝f(0)*P_ref_1D(x+iIdx-1)+f(1)*P_ref_1D(x+Idx)+f(2)*P_ref_1D(x+Idx+1)+f(3)*P_ref_1D(x+iIdx+2)

When a multi-tap filter is used, samples that do not correspond to the position of the left or upper reference sample may be replaced with the reference sample that is most adjacent at that position. For example, in equation 9, when the sample at the P _ ref _1D (x + ididx-1) position does not correspond to the upper-end reference sample, the sample may be replaced with the reference sample at the P _ ref _1D (x + idx) position. Alternatively, when the sample at the position P _ ref _1D (x + idix +2) does not correspond to the upper-end reference sample, the sample may be replaced with the reference sample at the position P _ ref _1D (x + idix + 1).

The multi-tap filter may be applied to a plurality of reference samples arranged in a column along the horizontal or vertical direction. Alternatively, the multi-tap filter may be applied to a predetermined polygonal form such as a quadrangle. The modality to which the multi-tap filter is applied may be variably decided according to the size, the modality, or the intra prediction mode of the current block.

As shown in equations 8 to 10, the generation of prediction samples by interpolating reference samples using the directivity of intra prediction can be referred to as an intra prediction sample interpolation method.

When the intra-frame prediction sample interpolation method is used, the tap filter with a large number of taps does not guarantee improvement of prediction accuracy. For example, if the size of the current block is an asymmetric coding unit having a height or width significantly larger than that of another as 2x16, or a small-sized block as 4x4, using a tap filter of 4 taps or more instead results in a result of excessively smoothing the prediction image. For this, the kind of the tap filter may be adaptively decided according to the size, shape, or intra prediction mode of the current block. Wherein, the kind of the tap filter can be distinguished according to at least one of the number of taps, the filter coefficient, the filter strength (strong/weak), and the filter direction. The number of filter taps, the filter coefficient, and the like may be variably determined according to the filter strength. Further, the direction to which the tap filter is applied, such as horizontal interpolation, vertical interpolation, or horizontal and vertical interpolation, can be determined according to the type of the tap filter. The applicable direction of the tap filter may be variably set in line units (rows or columns) or sample units of the current block.

Specifically, the kind of tap filter to be used may be decided based on the width or height of the current block. As an example, in the case where at least one value of the width or the height of the current block is smaller than a value defined in advance, the intra prediction sample interpolation method may be performed using a 2-tap filter instead of the 4-tap filter. In contrast, in the case where the width and height of the current block are both a value defined in advance or more, the intra prediction sample interpolation method may be performed using a 4-tap filter. The value defined in advance may be 4, 8, 16, or the like.

Alternatively, the kind of the tap filter to be used may be decided according to whether the width and the height of the current block are the same value. As an example, in the case where the width and height of the current block are different values, the intra prediction sample interpolation method may be performed using a 2-tap filter instead of a 4-tap filter. In contrast, in the case where the width and the height of the current block have the same value, the intra prediction sample interpolation method may be performed using a 4-tap filter.

Alternatively, the kind of tap filter to be used may be decided according to the ratio of the width to the height of the current block. As an example, in the case where the ratio of the width (w) to the height (h) of the current block (i.e., w/h or h/w) is less than a threshold value defined in advance, the intra prediction sample interpolation method may be performed using a 2-tap filter instead of a 4-tap filter. In contrast, in the case where the ratio of the width to the height of the current block is above a previously defined threshold, the intra prediction sample interpolation method may be performed using a 4-tap filter.

Alternatively, the kind of the tap filter may be decided according to an intra prediction mode, a shape, or a size of the current block. For example, when the current block is a coding unit of 2 × 16 type and the intra prediction mode of the current block is an intra prediction mode belonging to a horizontal range, the intra prediction sample interpolation method may be performed using a tap filter of n taps. In contrast, in the case where the current block is a coding unit of 2 × 16 type and the intra prediction mode of the current block is an intra prediction mode belonging to a vertical direction range, the intra prediction sample interpolation method may be performed using a tap filter of which the number of taps is m.

In contrast, in the case where the current block is a coding unit of the 16 × 2 mode and the intra prediction mode of the current block is an intra prediction mode belonging to a horizontal direction range, the intra prediction sample interpolation method may be performed using a tap filter of which the number of taps is n. In contrast, in the case where the current block is a coding unit of the 16 × 2 mode and the intra prediction mode of the current block is an intra prediction mode belonging to the vertical direction range, the intra prediction sample interpolation method may be performed using a tap filter of which the number of taps is m.

Here, the horizontal direction range may indicate a predetermined range including the horizontal direction intra prediction mode, and the vertical direction range may indicate a predetermined range including the vertical direction intra prediction mode. As an example, when based on 35 intra prediction modes, the horizontal direction range may represent intra prediction modes from mode 11 to mode 18, and the vertical direction range may represent intra prediction modes from mode 19 to mode 27.

Further, n and m are constants greater than 0, and n and m may have different values. Alternatively, at least one of the filter coefficients or the filter strengths of the n-tap filter and the m-tap filter may be set to be different while n and m are set to have the same value.

When the directional prediction mode or the DC mode is used, there is a possibility that image quality is deteriorated at a block boundary. In contrast, in the case of the planar mode, there is an advantage in that the image quality degradation at the block boundary is relatively smaller than that in the above-described prediction mode.

The plane prediction can be performed by generating a first prediction image in the horizontal direction and a second prediction image in the vertical direction using the reference samples, and then performing weighted prediction on the first prediction image and the second prediction image.

Wherein the first prediction image may be generated based on a reference sample adjacent to the current block disposed in the horizontal direction of the prediction target sample. As an example, the first prediction image may be generated based on a weighting of a reference sample placed in the horizontal direction of the prediction target sample. In this case, the weight applied to each reference sample may be determined in consideration of the distance to the prediction target sample, the size of the current block, and the like. The samples located in the horizontal direction may include a left-side reference sample (i.e., a left-side reference sample having the same y-coordinate as the prediction target sample) and a right-side reference sample (i.e., a right-side reference sample having the same y-coordinate as the prediction target sample) that are placed on the same horizontal line as the prediction target sample. At this time, the right reference sample may be derived from the upper reference sample of the current block. For example, the right-side reference sample may be derived by copying the value of the upper-end reference sample placed on the same vertical line, or may be derived by weighting or averaging a plurality of upper-end reference samples. Among them, the upper end reference sample, which is positioned on the same vertical line as the right side reference sample, may include a reference sample adjacent to the right upper end corner of the current block (i.e., an upper end reference sample having the same x-coordinate as the right side reference sample). Alternatively, the position of the upper reference sample used to derive the right reference sample may be determined differently according to the shape and size of the current block or the position of the prediction target sample.

The second prediction image may be generated based on a reference sample adjacent to the current block placed in the vertical direction of the prediction target sample. As an example, the second prediction image may be generated based on a weight of a reference sample placed in a vertical direction of the prediction target sample. In this case, the weight applied to each reference sample may be determined in consideration of the distance to the prediction target sample, the size of the current block, and the like. The samples located in the vertical direction may include an upper end reference sample (i.e., an upper end reference sample having the same x-coordinate as the prediction target sample) and a lower end reference sample (i.e., a lower end reference sample having the same x-coordinate as the prediction target sample) that are placed on the same vertical line as the prediction target sample. At this time, the lower reference sample may be derived from the left reference sample of the current block. For example, the lower reference sample may be derived by copying the value of the left reference sample placed on the same horizontal line, or may be derived by weighting or averaging a plurality of left reference samples. Wherein the left reference sample, which is positioned on the same horizontal line as the lower reference sample, may include a reference sample adjacent to the left lower corner of the current block (i.e., a left reference sample having the same y-coordinate as the lower reference sample). Alternatively, the position of the upper reference sample used to derive the lower reference sample may be determined differently according to the size and shape of the current block or the position of the prediction target sample.

Alternatively, at least one of the right side reference sample or the lower side reference sample may be derived using both the left side reference sample and the upper side reference sample.

For example, the weighted or average of the upper reference sample and the left reference sample of the current block may be determined as at least one of the right reference sample and the lower reference sample.

Alternatively, after a right-side lower end reference sample adjacent to a right-side lower end corner of the current block is derived using the left-side lower end reference sample and the right-side upper end reference sample, the right-side reference sample and the lower end reference sample may be derived using the derived right-side lower end reference sample. The right lower reference sample may be derived based on a weighted or average of the right upper reference sample and the left reference sample of the current block. In this case, the weights applied to the right upper end reference sample and the left side reference sample may have the same value or may be determined based on the width/height of the current block.

When the right lower end reference sample is decided, the right lower end reference sample and the right upper end reference sample may be interpolated (interpolation) to derive the right reference sample, and the right lower end reference sample and the left lower end reference sample may be interpolated to derive the lower end reference sample. At this time, the coefficient of the interpolation filter may be determined based on the size of the current block, the shape of the current block, the distance from the right lower end reference sample, the distance from the right upper end reference sample, or the distance from the left lower end reference sample, or the like.

In order to derive the right-side reference sample or the left-side reference sample, a fixed-position reference sample or a reference sample adaptively selected according to the position of the prediction target sample may be used. For example, the right reference sample may be derived using a right upper end reference sample regardless of the position of the prediction target sample, or derived using a left reference sample (for example, a reference sample having the same y-axis coordinate as the prediction target sample) or an upper end reference sample (for example, a reference sample having the same x-axis coordinate as the prediction target sample) selected according to the position of the prediction target sample. Alternatively, the lower end reference sample may be derived using a left lower end reference sample regardless of the position of the prediction target sample, or derived using a left side reference sample (for example, a reference sample having the same y-axis coordinate as the prediction target sample) or an upper end reference sample (for example, a reference sample having the same x-axis coordinate as the prediction target sample) selected according to the position of the prediction target sample.

Fig. 14 is a diagram showing an example in which a right-side reference sample or a lower-end reference sample is derived using a plurality of reference samples. Assume that the current block is a block having a size of WxH.

Referring to (a) of fig. 14, first, the right lower reference sample P (W, H) may be generated based on a weighted or average value of the right upper reference sample P (W, -1) and the left lower sample P (-1, H) of the current block. In this case, the weights applied to the right upper end reference sample and the left side reference sample may be set to be the same, or may be determined based on the width (W) and the height (H) of the current block. For example, when the current block is non-square, the weight applied to the right upper end reference sample may be determined as W/(W + H), and the weight applied to the left lower end reference sample may be determined as H/(W + H).

Further, the right reference sample P (W, y) for the target prediction sample (x, y) may be generated based on the right lower end reference sample P (W, H) and the right upper end reference sample P (W, -1). For example, the right prediction sample P (W, y) may be calculated as a weighted or average of the right lower reference sample P (W, H) and the right upper reference sample P (W, -1). Further, the lower end reference sample P (x, H) for the target prediction sample (x, y) may be generated based on the right lower end reference sample P (W, H) and the left lower end reference sample P (-1, H). For example, the lower reference sample P (x, H) may be calculated as a weighted or average of the right lower reference sample P (W, H) and the left reference sample P (-1, H).

As shown in fig. 14 (b), when the right side reference sample and the lower side reference sample are generated, the first prediction sample P for the prediction target sample may be generated using the generated reference samples_h(x, y) and a second prediction sample P_v(x, y). At this time, the first prediction sample P_h(x, y) may be generated based on the weighting of the left reference sample P (-1, y) and the right reference sample P (W, y), and the second prediction sample P_v(x, y) may be generated based on the weighting of the upper reference sample P (x, -1) and the lower reference sample P (x, H).

As shown in the example of fig. 15, in the case where the current block is a non-square block of (N/2) xN size, the right-side reference sample may be derived based on the right-side upper-end reference sample P (N/2, -1), and the lower-end reference sample may be derived based on the left-side lower-end reference sample P (-1, N).

Alternatively, the right or lower reference sample may be derived based on at least one of a weighted, average, minimum, or maximum value of the right upper reference sample P (N/2, -1) and the left lower reference sample P (-1, N). For example, the right side reference sample may be derived by interpolating the right side lower end reference sample and the right side upper end reference sample after deriving the right side reference sample based on a weighted or average of P (N/2, -1) and P (-1, N), or deriving the right side lower end reference sample P (N/2, N) based on P (N/2, -1) and P (-1, N). Alternatively, the lower-end reference sample may be derived by interpolating the right-side lower-end reference sample and the left-side lower-end reference sample after deriving the lower-end reference sample based on a weighted or average of P (N/2, -1) and P (-1, N), or deriving the right-side lower-end reference sample P (N/2, N) based on P (N/2, -1) and P (-1, N).

In contrast, as shown in the example shown in fig. 16, in the case where the current block is a non-square block of Nx (N/2) size, the right-side reference sample may be derived based on the right-side upper-end reference sample P (N, -1), and the lower-end reference sample may be derived based on the left-side lower-end reference sample P (-1, N/2).

Alternatively, the right or lower reference sample may be derived based on at least one of a weighted, average, minimum, or maximum value of the left upper reference sample P (N, -1) and the left lower reference sample P (-1, N/2). For example, the right side reference sample may be derived by interpolating the right side lower end reference sample and the right side upper end reference sample after deriving the right side reference sample based on a weighted or average of P (N, -1) and P (-1, N/2), or deriving the right side lower end reference sample P (N, N/2) based on P (N, -1) and P (-1, N/2). Alternatively, the lower-end reference sample may be derived by interpolating the right-side lower-end reference sample and the left-side lower-end reference sample after deriving the lower-end reference sample based on a weighted or average of P (N, -1) and P (-1, N/2), or deriving the right-side lower-end reference sample P (N, N/2) based on P (N, -1) and P (-1, N/2).

In the examples described with reference to fig. 14 to 16, the following are described: the lower-end reference sample is derived based on at least one of a left-side lower-end reference sample of the current block placed on the same horizontal line as the lower-end reference sample or a right-side upper-end reference sample of the current block placed on the same vertical line as the right-side reference sample, and the right-side reference sample is derived based on at least one of a right-side upper-end reference sample of the current block placed on the same vertical line as the right-side reference sample or a left-side lower-end reference sample of the current block placed on the same horizontal line as the lower-end reference sample. However, unlike the illustrated example, the right-side reference sample may be derived based on at least one of the upper-side middle reference sample or the left-side middle reference sample, or the left-side reference sample may be derived. For example, the lower side sample may be generated by interpolation or extrapolation (extrapolation) of the lower side middle sample and the left side lower side sample after the lower side middle sample is derived using the upper side middle sample and the left side lower side reference sample. Further, the lower samples may be generated by interpolation or extrapolation of the right side mid-sample and the right side upper samples after the right side mid-sample is derived using the left side mid-sample and the right side upper end sample.

The positions of the reference samples used for generating the first predicted image and the second predicted image may be determined differently according to the size or shape of the current block. For example, the position of the upper-side reference sample or the left-side reference sample used to derive the right-side reference sample or the lower-side reference sample may be determined differently according to the size or shape of the current block.

As an example, in the case where the current block is a square block of NxN size, the right-side reference sample may be derived based on the right-side upper-end reference sample P (N, -1), and on the other hand, the lower-end reference sample may be derived based on the left-side lower-end reference sample P (-1, N). Alternatively, in the case where the current block is a square block of NxN size, the right-side reference sample and the lower-side reference sample may be derived based on at least one of a weighted, average, minimum, or maximum value of the right-side upper-end reference sample P (N, -1) and the left-side lower-end reference sample P (-1, N).

In contrast, in the case where the current block is a non-square block of Nx2/N size, the lower middle reference sample P (N/2) may be derived based on the upper middle reference sample P (N/2, -1) and the left lower end reference sample P (-1, N/2), and the lower end reference sample may be derived based on the derived lower middle reference sample. For example, the lower end reference sample may be derived by interpolation (interpolation) or extrapolation (extrapolation) of the lower middle reference sample and the left lower end reference sample. Alternatively, in the case where the current block is a non-square block of N/2xN size, the right middle reference sample P (N/2) may be derived based on the right upper end reference sample P (N/2, -1) and the left middle reference sample P (-1, N/2), and the right reference sample may be derived based on the derived right middle reference sample. For example, the right reference sample may be derived by interpolation or extrapolation of the right middle reference sample and the right upper end reference sample.

The first prediction image can be calculated based on weighted prediction of reference samples placed on the same horizontal line as the prediction target sample. Further, the second prediction image may be calculated based on weighted prediction of reference samples placed on the same vertical line as the prediction target sample.

The first prediction image or the second prediction image may be generated using, without being limited to the above example, an average value, a minimum value, a maximum value, or the like of the reference sample.

The method of deriving the reference sample or the method of deriving the first predicted image or the second predicted image may be set differently depending on whether or not the prediction target sample includes a predetermined region in the current block, the size or shape of the current block, or the like. Specifically, the number of reference samples or the position of the reference sample used when the right or lower reference sample is used may be set differently according to the position of the prediction target sample, or the weight used when the first predicted image or the second predicted image is derived or the number of reference samples may be set differently.

For example, the right reference sample used when generating the first predicted image of the prediction target sample included in the predetermined region may be derived using only the upper reference sample, and the right reference sample used when generating the first predicted image of the prediction target sample included in the samples other than the predetermined region may be derived based on a weighting or an average of the upper reference sample and the left reference sample.

For example, as shown in the example of fig. 15, when the current block is a non-square block having a height larger than a width, the right reference sample of the prediction target sample included in the (x, y) position of the predetermined region in the current block can be derived from P (N/2, -1). For example, the right reference sample of the prediction target sample included in the predetermined region may be generated by copying the value of the P (N/2, -1) reference sample. In contrast, the right-side reference sample of the prediction target sample included in the (x ', y') position outside the predetermined region in the current block may be derived based on the weighted or average values of P (N/2, -1) and P (-1, N). For example, the right reference sample of the prediction target sample included in the outside of the predetermined region may be generated by interpolation of the right lower end reference sample P (N/2, N) and the right upper end reference sample P (N/2, -1) derived based on P (N/2, -1) and P (-1, N).

Alternatively, as shown in the example of fig. 16, in the case where the current block is a non-square block having a width greater than its height, the lower reference sample of the prediction target sample included in the (x, y) position of the predetermined region in the current block may be derived from P (-1, N/2). For example, the lower reference sample of the prediction target sample included in the predetermined region may be generated by copying the value of the P (-1, N/2) reference sample. In contrast, the lower reference sample of the prediction target sample at the (x ', y') position outside the predetermined region in the current block may be derived based on the weighted or average values of P (N, -1) and P (-1, N/2). For example, the lower target reference sample of the prediction target sample included in the outside of the predetermined region may be generated by interpolation of the right lower reference sample P (N, N/2) and the left lower reference sample P (-1, N/2) derived based on P (N, -1) and P (-1, N/2).

As another example, the prediction target sample included in the predetermined region may generate the first prediction image or the second prediction image based on the weighting of the reference sample, and the prediction target sample outside the predetermined region may generate the first prediction image or the second prediction image by referring to the average value, the minimum value, or the maximum value of the reference sample, or by using only one of the positions defined in advance in the reference sample. For example, as shown in the example of fig. 15, when the current block is a non-square block having a height greater than a width, the first predicted image may be generated using only one of the right-side reference sample P (N/2, y) derived from P (N/2, -1) or the left-side reference sample at the position of P (-1, y) included in the prediction target sample at the position (x, y) of the predetermined region in the current block. In contrast, the prediction target sample not included in the (x ', y') position of the predetermined region may be generated based on the weighted or average of the right reference sample P (N/2, y ') derived from P (N/2, -1) and the reference sample at the P (-1, y') position.

Alternatively, as shown in the example of fig. 16, when the current block is a non-square block having a width larger than the height, the second predicted image may be generated using only one of the lower-end reference sample P (x, N/2) derived based on P (-1, N/2) or the upper-end reference sample at the P (x, -1) position, which is the prediction target sample included in the (x, y) position of the predetermined region in the current block. In contrast, the prediction target sample not included in the (x ', y') position of the predetermined region may be generated based on the weighted or average of the lower reference sample P (x ', N/2) derived from P (-1, N/2) and the reference sample at the P (-1, y') position.

In the above-described embodiment, the prescribed region may be one of at least one sample line adjacent to the boundary of the current block or a residual region in addition thereto. Wherein the boundary of the current block may include at least one of a left side boundary, a right side boundary, an upper end boundary, or a lower side boundary. The number or position of the boundaries used when defining the predetermined region may be set differently according to the form of the current block. Alternatively, the prescribed region may be a block shape that contacts a side corner of the current block. At this time, the size and shape of the prescribed region may be decided based on at least one of the size or shape of the current block.

The final predicted image in planar mode can be derived based on a weighted, average, minimum or maximum of the first predicted image and the second predicted image.

As an example, the following equation 11 indicates a first predicted image P_hAnd a second predicted image P_vTo generate an example of the final prediction image P.

[ mathematical formula 11]

P(x，y)＝(w*P_h(x，y)+(1-w)*P_v(x，y)+N)＞＞(log2(N)+1)

In equation 11, the prediction weight w may be different depending on the shape and size of the current block, or the position of the prediction target sample.

As an example, the prediction weight w may be derived in consideration of the width of the current block, the height of the current block, or a width-height ratio, etc. In the case where the current block is a non-square block having a width larger than a height, w may be set so as to give more weight to the first prediction image. Conversely, in the case where the current block is a non-square block whose height is greater than the width, w may be set so as to give more weight to the second prediction image.

As an example, in case that the current block is square, the prediction weight w may have a value of 1/2. In contrast, the prediction weight w may be set to 1/4 in case that the current block is a non-square block (e.g., (N/2) xN) having a height greater than a width, and to 3/4 in case that the current block is a non-square block (e.g., Nx (N/2)) having a width greater than a height.

In addition to the planar mode, the intra prediction may be performed using reference samples other than the left-side reference sample and the upper-end reference sample in the DC mode or the directional intra prediction mode. In the following embodiments, the left-side reference sample and the upper-side reference sample are referred to as a first reference sample, and the reference samples other than the left-side reference sample and the upper-side reference sample are referred to as a second reference sample. As an example, the second reference sample may include a right reference sample and/or a lower reference sample of the current block. Here, the lower reference sample may represent a reference sample having a larger y-axis coordinate than the prediction target sample of the lowermost row within the current block, and the right reference sample may represent a reference sample having a larger x-axis coordinate than the prediction target sample of the rightmost column within the current block.

Whether to perform intra prediction using the second reference sample may be decided based on at least one of the size, shape, intra prediction mode, or position of the prediction target sample of the current block. As an example, whether to perform intra prediction using the second reference sample may be decided based on whether the intra prediction mode of the current block is a vertical mode, a horizontal mode, a diagonal direction mode, or the like. Alternatively, it may be set that: the intra prediction is performed using the second reference sample for the prediction target sample included in the predetermined area within the current block, and the intra prediction is performed using the first reference sample for the prediction target sample not included in the predetermined area within the current block.

Alternatively, information indicating whether or not to use the second reference sample is signaled through a bitstream. The information may be a 1-bit flag, an index used when determining the intra prediction mode of the current block, or the like.

Alternatively, whether to use the second reference sample may be decided based on whether to use the second reference sample in the peripheral blocks of the current block.

The second reference sample may be generated based on the first reference sample. For example, the order of the first reference samples may be changed to form the second reference sample, or the second reference sample may be derived from the first reference sample at a specific position.

Fig. 17 is a diagram of an example for deriving a second reference sample using a first reference sample.

First, a right lower reference sample P (W, H) to be derived may be derived based on the right upper reference sample r (W, -1) and the left lower reference sample r (-1, H) of the current block. Specifically, the right lower end reference sample may be derived by a weighted or average operation of the right upper end reference sample and the left lower end reference sample. An example of deriving the right lower end reference sample is shown in the following equation 12.

[ mathematical formula 12]

As shown in the above equation 12, the right lower end reference sample may be calculated based on the weighting between the right upper end reference sample and the left lower end reference sample. At this time, the weight applied to the right upper end reference sample and the left lower end reference sample may be determined according to the width and the height of the current block. For example, in the case where the current block is square, the same weight may be applied to the right upper end reference sample and the left lower end reference sample, and in the opposite case where the current block is non-square, different weights may be applied to the right upper end reference sample and the left lower end reference sample. The method of setting the weight expressed by equation 12 is only an example of the present invention, and the present invention is not limited thereto. In addition to the example shown in equation 12, the weight may also be decided based on at least one of the size, shape, intra prediction mode, availability of reference samples, availability of peripheral blocks, whether the peripheral blocks are encoded in the intra prediction mode, or the intra prediction mode of the peripheral blocks.

The right reference sample may be derived based on the right upper end reference sample and the right lower end reference sample. As an example, the right side reference sample may be obtained by interpolating the right side upper end reference sample and the right side lower end reference sample. An example of deriving the right-side reference sample is shown in the following equation 13.

[ mathematical formula 13]

As shown in the above equation 13, the right reference sample P_r(W, y) (in this case, y is an integer between 0 and CU height (CU height)) can be obtained by performing weighted prediction on the right upper end reference sample r (W, -1) and the right lower end reference sample P (W, H). At this time, the weight applied to the right upper end reference sample and the right lower end reference sample may be determined based on at least one of the width, the height, or the position of the right reference sample of the current block. For example, as shown in the example of equation 13, a weight of (H-1-y)/H may be applied to the upper right-side reference sample, and a weight of (y +1)/H may be applied to the lower right-side reference sample. The weight setting method expressed by the mathematical expression 13 is only the present inventionThe present invention is not limited to the above examples. In addition to the example shown in equation 13, the weight may also be decided based on at least one of the size, shape, intra prediction mode, availability of reference samples, availability of peripheral blocks, whether the peripheral blocks are encoded in the intra prediction mode or the intra prediction mode of the peripheral blocks.

The lower end reference sample may be derived based on the left lower end reference sample and the right lower end reference sample. As an example, the lower end reference sample may be obtained by interpolating the left lower end reference sample and the right lower end reference sample. An example of deriving the lower reference sample is shown in the following equation 14.

[ mathematical formula 14]

As shown in the above equation 14, the lower reference sample P_b(x, H) (in this case, x is an integer between 0 and CU width (CU width)) can be obtained by performing weighted prediction on the left lower end reference sample r (-1, H) and the right lower end reference sample P (W, H). In this case, the weight applied to the left lower end reference sample and the right lower end reference sample may be determined based on at least one of the width and the height of the current block or the position of the lower end reference sample. For example, as shown in the example of equation 14, a weight of (W-1-x)/W may be applied to the left lower end reference sample, and a weight of (x +1)/H may be applied to the right lower end reference sample. The method of setting the weight expressed by equation 14 is only an example of the present invention, and the present invention is not limited thereto. In addition to the example represented by equation 14, the weight may be determined according to at least one of the size, shape, intra prediction mode, availability of reference samples, availability of peripheral blocks, whether the peripheral blocks are encoded in the intra prediction mode, or the intra prediction mode of the peripheral blocks.

In the case where the current block is non-square, the right-side reference sample and the lower-side reference sample can be derived based on the example shown in fig. 15 and 16.

As shown in the above example, the second reference samples such as the right side upper end reference sample and the left side lower end reference sample can be derived using the first reference samples at fixed positions such as the right side upper end reference sample and the left side lower end reference sample. In contrast to the above example, the second reference sample may be derived using the first reference sample at a different position from the right upper end reference sample and/or the left lower end reference sample. For example, the right-side reference sample and the lower-side reference sample may be derived using a first reference sample, such as an upper-middle reference sample of the current block or a left-middle reference sample of the current block.

Alternatively, the first reference sample used in deriving the second reference sample may be determined according to the intra prediction mode of the current block. As an example, the right reference sample and/or the lower reference sample may be derived based on the left reference sample and/or the upper reference sample specified by the intra prediction mode directionality of the current block.

Alternatively, the second reference sample may be determined using a plurality of left side reference samples and/or a plurality of upper side reference samples. For example, at least one of the right side reference sample, the lower side reference sample, or the right side lower side reference sample may be generated based on a weight, an average value, a maximum value, or a minimum value of a plurality of left side reference samples, or may be generated based on a weight, an average value, a maximum value, or a minimum value of a plurality of upper side reference samples.

Alternatively, the second reference sample may be generated by copying the first reference sample. In this case, the first reference sample used when the second reference sample is generated may have a fixed position, or may be adaptively determined according to the size, shape, intra prediction mode, or position of the second reference sample of the current block.

In the above example, the example in which the lower end reference samples are W and the right side reference samples are H is shown, but a larger number of lower end reference samples and/or right side reference samples than these may be derived. For example, the lower-end reference sample may be derived up to the same vertical line as the rightmost upper-end reference sample r (2W-1, -1), or the right-side reference sample may be derived up to the same horizontal line as the rightmost left-end reference sample r (-1, 2H-1).

In this case, the lower end reference sample having an x coordinate larger than W may be generated by extrapolating the left lower end reference sample and the right lower end reference sample, or by interpolating the right lower end reference sample P (W, H) and the rightmost lower end reference sample P (2W-1, H). The rightmost lower-end reference sample may be generated by copying the rightmost upper-end reference sample r (2W-1, -1), or may be generated by a weighted operation between the rightmost upper-end reference sample and the left lower-end reference sample. The right reference sample having a y-coordinate greater than H may be generated by extrapolating the right upper end reference sample and the right lower end reference sample, or interpolating the right lower end reference sample P (W, H) and the lowermost right reference sample P (W, 2H-1). At this time, the lowermost right reference sample may be generated by copying the lowermost left reference sample r (-1, 2H-1), or by a weighting operation between the lowermost left reference sample and the left upper reference sample.

The first reference samples may be arranged one-dimensionally to generate a first one-dimensional reference sample set, and the second reference samples may be arranged one-dimensionally to generate a second one-dimensional reference sample set. In this case, the first one-dimensional reference sample group may be configured to include not only the first reference sample but also at least one of the second reference samples, and the second one-dimensional reference sample group may be configured to include not only the second reference sample but also at least one of the first reference samples.

As shown in the example shown in fig. 18 (a), the first one-dimensional reference sample group may be composed of a left-side reference sample and an upper-side reference sample of the current block.

In contrast, as shown in the example of (b) of fig. 18, the second one-dimensional reference sample group may be configured to include not only the right-side reference sample and the lower-end reference sample of the current block but also a portion of the left-side reference sample and a portion of the upper-end reference sample.

That is, the left side reference sample having the y-axis coordinate larger than the left side lower end reference sample r (-1, H) and the left side lower end reference sample may be included in both the first one-dimensional reference sample group and the second one-dimensional reference sample group. In addition, the upper-end reference samples having x-axis coordinates larger than the right-side upper-end reference sample r (W, -1) and the right-side upper-end reference sample may be included in both the first one-dimensional reference sample group and the second one-dimensional reference sample group.

Alternatively, the portion of the first reference samples may be included only in the first one-dimensional reference sample group or included only in the second one-dimensional reference sample group based on at least one of a size, a shape, or an intra prediction mode of the current block. In addition to the structure of the one-dimensional reference sample group, the order of arrangement of the reference samples constituting the one-dimensional reference sample group may be variably determined based on at least one of the size, shape, and intra prediction mode of the current block.

For convenience of description, in the embodiments described below, a reference sample group including a left-side reference sample and an upper-end reference sample of the current block is referred to as a first reference sample group (e.g., a first one-dimensional reference sample group), and a reference sample group including a right-side reference sample and a lower-end reference sample of the current block is referred to as a second reference sample group (e.g., a second one-dimensional reference sample group). For example, the first reference sample group and the second reference sample group may be divided according to whether or not the right reference sample and the lower reference sample are included. In order to perform intra prediction of a prediction target sample, a reference sample selected from the first reference sample group is referred to as a first base reference sample, and a reference sample selected from the second reference sample group is referred to as a second base reference sample.

The intra prediction of the current block may be performed using at least one of the first reference sample group or the second reference sample group. For example, the prediction value of the current intra prediction target sample may be obtained based on at least one of a first base reference sample selected from the first reference sample group or a second base reference sample selected from the second reference sample group. In this case, the first base reference sample and/or the second base reference sample may be determined based on at least one of a shape, a size, or an intra prediction mode of the current block. For example, when the intra prediction mode of the current block is determined, the first base reference sample for the prediction target sample may be specified according to the determined intra prediction mode direction, and the second base reference sample for the prediction target sample may be specified in the opposite direction of the determined intra prediction mode.

Alternatively, the position of the second base reference sample may be determined based on the position of the first base reference sample, or the position of the first base reference sample may be determined based on the position of the second base reference sample. For example, a second base reference sample having the same x-coordinate or the same y-coordinate as the first base reference sample may be selected, or a second base reference sample having a value obtained by adding an offset to the x-coordinate or the y-coordinate of the first base reference sample may be selected. The offset may have a fixed value, or may be adaptively determined according to the size, shape, or intra prediction mode of the current block.

Alternatively, the position of the first base reference sample and/or the second base reference sample may be determined based on the position of the prediction target sample. For example, the first base reference sample and/or the second base reference sample having the same x coordinate or the same y coordinate as the prediction target sample may be selected, or the first base reference sample and/or the second base reference sample having a value obtained by adding an offset to the x coordinate or the y coordinate of the prediction target sample may be selected. The offset may have a fixed value, or may be adaptively determined according to the size, shape, or intra prediction mode of the current block.

The prediction value of the prediction target sample may be generated based on at least one of a first prediction image based on the first base reference sample or a second prediction image based on the second base reference sample. In this case, the first predicted image may be generated based on the description given by the above equation 8 to equation 10.

The second prediction image may be generated by interpolating or copying a second base reference sample specified according to a slope of the intra prediction mode of the current block. As an example, equation 15 is a diagram showing an example of deriving the second predicted image by copying the second base reference sample.

[ mathematical formula 15]

P₂(x，y)＝P_2nd_1D(x+iIdx+1+f)

In the above mathematical formula 15, P₂(x, y) denotes a second prediction image, and P _2nd _1D (x + idix +1+ f) denotes a second base reference sample.

When the slope of the intra prediction mode of the current block cannot be expressed using only one second base reference sample, a plurality of second base reference samples may be interpolated to generate a second prediction image. Specifically, when a virtual angle line according to the slope and/or angle of the intra prediction mode does not pass through an integer pel (i.e., a reference sample of an integer position), the second prediction image may be acquired by interpolating the second reference samples adjacent to the left and right or up and down of the corresponding angle line. As an example, equation 16 represents an example of obtaining the second predicted image by interpolating the second reference sample.

[ mathematical formula 16]

The coefficients of the interpolation filter may be based on a weight-dependent parameter i_factTo decide. As an example, the coefficients of the interpolation filter may be determined based on the distance between fractional pixels (fractional pel) and integer pixels (i.e., integer positions of respective reference samples) located on an angle line (angular line).

Although the interpolation filter having the number of taps of 2 is shown in equation 16, an interpolation filter having a number of taps greater than 2 may be used.

A final prediction image of the prediction target sample may be acquired based on at least one of the first prediction image or the second prediction image. For example, the first predicted image may be determined as a final predicted image of the sample to be predicted, or the second predicted image may be determined as a final predicted image of the sample to be predicted. Alternatively, the final predicted image of the prediction target sample may be determined based on a weighted or average operation of the first predicted image and the second predicted image. Equation 17 is an example of obtaining a final prediction sample based on a weighting operation of the first prediction image and the second prediction image.

[ mathematical formula 17]

P(x，y)＝w(x，y)×P₁(x，y)+(1-w(x，y))×P₂(x，y)

In the above mathematical formula 17P₁(x, y) denotes a first predicted image, P₂(x, y) denotes a second prediction image. W (x, y) represents a weight given to the first predicted image.

The weight to be given to the first predicted image and the second predicted image may be determined based on at least one of the position of the prediction target sample, the size, the shape, or the intra prediction mode of the current block. As an example, equation 18 shows an example of determining the weight according to the size of the current block and the position of the prediction target sample.

[ mathematical formula 18]

In the above equation 18, W and H represent the width and height of the current block, respectively, and (x, y) represent the coordinates of the prediction target sample.

As shown in the example of the above mathematical expression 18, it is possible to set: the closer the sample to be predicted is to the upper left corner of the current block, the greater the weight given to the first predicted image, and the closer the sample to be predicted is to the lower right corner of the current block, the greater the weight given to the second predicted image.

Alternatively, the weights may be derived from neighboring blocks of the current block. The neighboring blocks of the current block may include at least one of an upper-side neighboring block, a left-side neighboring block, or a neighboring block adjacent to a corner of the current block (e.g., a left-side upper-side neighboring block, a right-side upper-side neighboring block, or a left-side lower-side neighboring block).

Alternatively, the information for deciding the weight may be signaled through a bit stream. The above information may indicate a weight value applied to the first prediction image or the second prediction image, or may indicate a weight difference value between the current block and the adjacent block.

As shown in the above example, obtaining the final Prediction image through the weighting operation between the first Prediction image and the second Prediction image may be referred to as Bi-intra Prediction (Bi-intra Prediction).

Bi-directional intra prediction may only be applicable for a portion of the area within the current block. In this case, the region to which the bidirectional intra prediction is applied may be defined in advance in an encoder and a decoder. For example, bidirectional intra prediction may be applied to a block of a predetermined size (e.g., 4 × 4) adjacent to the right lower corner in the current block. Alternatively, the region to which bidirectional intra prediction is applied may be adaptively determined according to the size, shape, or intra prediction mode of the current block. Alternatively, information for deciding an area to which bidirectional intra prediction is applied (for example, information indicating the size or position of the area) may be signaled through a bitstream.

Fig. 19 shows an example of an area to which bidirectional intra prediction is applied.

In a region to which bidirectional intra prediction is applied, a final prediction sample can be obtained by performing weighted prediction on the first prediction image and the second prediction image. In contrast, in a region where bidirectional intra prediction is not applicable, the first prediction image or the second prediction image may be decided as a final prediction sample.

In the above example, it is explained that bi-directional intra prediction is performed using a first base reference sample selected from a first sample group and a second base reference sample selected from a second sample group. Unlike the example described above, bidirectional intra prediction may also be performed by selecting a plurality of reference samples in the first sample group, or by selecting a plurality of reference samples in the second sample group. As an example, in the case where the intra prediction mode of the current block is the upper right diagonal direction or the lower left diagonal direction, the bidirectional intra prediction may be performed by selecting the upper reference sample and the left reference sample in the first sample group. That is, the final prediction sample of the current block may be obtained by weighted prediction of a first reference image obtained based on the upper end reference sample and a second reference image obtained based on the lower end reference sample.

Alternatively, bidirectional intra prediction may be performed by selecting the right reference sample and the lower reference sample in the second sample group according to the intra prediction mode.

Bidirectional intra prediction may be defined as an independent intra prediction mode. As an example, a total of 2N +2 intra prediction modes may be defined by defining N directional prediction modes and N bidirectional intra prediction modes corresponding to the N directional prediction modes. For example, a total of 68 intra prediction modes (i.e., 2 non-directional intra prediction modes, 33 directional intra prediction modes, and 33 bidirectional intra prediction modes) may be defined by adding a bidirectional intra prediction mode to the intra prediction mode shown in fig. 8. Of course, a greater or lesser number of directional intra prediction modes or bi-directional intra prediction modes than 33 may also be used.

Alternatively, it may be decided whether to transfer the decided intra prediction mode to a bidirectional prediction mode after deciding the intra prediction mode of the current block. As an example, when the intra prediction mode of the current block is decided, information on whether to use the decided intra prediction mode as a bidirectional intra prediction mode may be decoded. The above information may be a 1-bit flag (e.g., bi _ intra _ flag), but is not limited thereto. A value of the bi _ intra _ flag of 0 indicates that directional intra prediction is performed, and a value of the bi _ intra _ flag of 1 indicates that bi-directional intra prediction is performed. That is, when the value of the bi _ intra _ flag is 0, the first predicted image is determined as the final predicted sample of the current block, and conversely, when the value of the bi _ intra _ flag is 1, the image in which the first predicted image and the second predicted image are weighted-predicted is determined as the final predicted sample of the current block.

Alternatively, whether the current block uses the bidirectional intra prediction mode may be decided according to whether a peripheral block adjacent to the current block uses the bidirectional intra prediction mode. For example, when the intra prediction mode of the current block is the same as the candidate derived based on the intra prediction modes of the peripheral blocks (i.e., the MPM candidate), whether or not the current block uses the bidirectional intra prediction mode may be determined in the same manner as whether or not the peripheral blocks use the bidirectional intra prediction mode.

Alternatively, whether to perform bidirectional intra prediction may be decided based on the size and/or shape of the current block. For example, bidirectional intra prediction may be allowed only for blocks of 32 × 32 or more. Thus, bi-directional intra prediction may not be applicable in the case where the size of the current block is less than 32x32, and conversely, bi-directional intra prediction may be applicable in the case where the size of the current block is 32x32 or more.

As another example, bi-directional intra prediction may be allowed only for square blocks, or only for non-square morphology blocks.

Alternatively, bidirectional intra prediction may be applied to only a part of directional intra prediction modes. As an example, fig. 20 is an example of identifying a directional prediction mode indicating that bidirectional intra prediction is permitted. As shown in the example shown in fig. 20, it may be set to allow bidirectional intra prediction only for a part of intra prediction modes between the horizontal direction and the vertical direction. At this time, in the case where the intra prediction mode within the range is selected, bidirectional intra prediction may be performed by default, and in the case where the intra prediction mode within the range is selected, whether to perform the bidirectional intra prediction mode may be decided based on at least one of information parsed by a bitstream, a size or a shape of the current block.

The intra prediction mode to which the bidirectional intra prediction is applied is not limited to the example shown in fig. 20. The intra prediction mode allowing bidirectional intra prediction may be defined in advance in an encoder and a decoder, or may be adaptively determined according to the size and/or shape of the current block. Alternatively, information for deciding an intra prediction mode allowing bidirectional intra prediction may be signaled through a bitstream.

FIG. 21 is a flowchart illustrating a method of intra prediction of a current block based on a bidirectional intra prediction mode according to the present invention.

First, it may be decided whether bi-directional intra prediction is applicable to the current block (S2110). Whether or not bidirectional intra prediction is applied to the current block may be decided based on information parsed from the bitstream, a shape, a size, or an intra prediction mode of the current block.

As an example, after deciding the intra prediction mode of the current block based on the candidate list and the index, whether to apply bidirectional intra prediction to the current block may be decided based on the size, shape, or information parsed from the bitstream (e.g., bi _ pred _ flag) of the current block. Alternatively, whether or not bidirectional intra prediction is applied to the current block may be decided based on whether or not the intra prediction mode of the current block is a directional prediction mode to which bidirectional intra prediction is applied.

Thereafter, a reference sample of the current block may be derived (S2120). First, first reference samples adjacent to the left side and the upper end of the current block are derived, and when bidirectional intra prediction is applied to the current block, second reference samples adjacent to the right side and the lower end are further derived (S2130).

Next, in a case where the bidirectional intra prediction is not applied to the current block, a first prediction image may be generated based on at least one base reference sample among the first reference samples according to the intra prediction mode of the current block (S2140). In this case, the first prediction image is decided as the final prediction sample of the current block.

In contrast, in the case where bidirectional intra prediction is applied to the current block, the second prediction image may be generated based on at least one base reference sample of the first prediction image and the second reference sample (S2150). The first base reference sample and the second base reference sample may be determined based on the directivity of the intra prediction mode, or may be determined based on the size and shape of the current block or the position of another base reference sample. When the first prediction image and the second prediction image are acquired, the final prediction sample of the current block can be acquired by performing weighted prediction on the first prediction image and the second prediction image.

The application also provides the following scheme:

1. an image decoding method, comprising:

a step of deciding whether or not bidirectional intra prediction is applied to the current block;

deriving a reference sample of the current block; and

and a step of obtaining a prediction sample of the current block based on a first reference sample and a second reference sample specified by an intra prediction mode of the current block when bidirectional intra prediction is applied to the current block.

2. The image decoding method according to claim 1, wherein,

the first reference sample is a left side reference sample or an upper end reference sample of the current block, and the second reference sample is a right side reference sample or a lower end reference sample of the current block.

3. The image decoding method according to claim 2, wherein,

the right side reference sample is generated by interpolating a right side lower end reference sample and a right side upper end reference sample, and the lower end reference sample is generated by interpolating the right side lower end reference sample and a left side lower end reference sample.

4. The image decoding method according to claim 1, wherein,

the first reference sample is selected by applying an intra prediction mode of the current block in a forward direction, and the second reference sample is selected by applying an intra prediction mode of the current block in a reverse direction.

5. The image decoding method according to claim 1, wherein,

the prediction sample is obtained by a weighting operation of a first prediction sample obtained based on the first reference sample and a second prediction sample obtained based on the second reference sample.

6. The image decoding method according to claim 5, wherein,

the weight applied to both the first prediction sample and the second prediction sample is determined based on at least one of a position of a sample to be predicted, a size, a shape, or an intra prediction mode of the current block.

7. The image decoding method according to claim 1, wherein,

the bi-directional intra prediction is applied only to a predetermined area within the current block.

8. The image decoding method according to claim 1, wherein,

the step of determining whether to apply bi-directional intra prediction to the current block comprises:

determining whether the intra prediction mode of the current block is a directional intra prediction mode; and

and decoding information indicating whether the directional intra prediction mode is used as a bidirectional intra prediction mode.

9. The image decoding method according to claim 1, wherein,

whether or not bidirectional intra prediction is applied to the current block is determined based on whether or not bidirectional intra prediction is applied to a peripheral block adjacent to the current block.

10. The image decoding method according to claim 1, wherein,

when the intra prediction mode of the current block is a directional intra prediction mode within a predetermined range, it is determined that bidirectional intra prediction is applied to the current block.

11. An image encoding method, comprising:

deriving a reference sample of the current block; and

when bidirectional intra prediction is applied to the current block, a prediction sample of the current block is acquired based on a first reference sample and a second reference sample specified by an intra prediction mode of the current block.

12. The image encoding method according to claim 11, wherein,

13. The image encoding method according to claim 11, wherein,

14. An image decoding apparatus characterized in that,

the method includes determining whether or not bidirectional intra prediction is applied to a current block, deriving a reference sample of the current block, and acquiring a prediction sample of the current block based on a first reference sample and a second reference sample specified by an intra prediction mode of the current block when bidirectional intra prediction is applied to the current block.

15. An image encoding device is characterized in that,

The above-described embodiment is explained based on a series of steps or a sequence diagram, but this is not a limitation to the time series of the invention, and may be performed simultaneously or in another order as necessary. Note that, in the above-described embodiments, each of the components (for example, a unit, a module, and the like) constituting the block diagram may be realized by a hardware device or software, or may be realized by one hardware device or software by combining a plurality of components. The embodiments described above may be implemented in the form of program instructions that can be executed by various computing mechanisms as elements and stored in a computer-readable storage medium. The computer-readable storage media described above may contain program instructions, data files, data structures, etc., either alone or in combination. Examples of the computer-readable storage medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical storage media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks (magnetic-optical media), and hardware devices, such as ROMs, RAMs, and flash memories, that are configured to store and execute program instructions. The hardware device may be configured to operate as one or more software modules in order to execute the processing according to the present invention, and vice versa.

Industrial applicability

The present invention is applicable to an electronic device capable of encoding/decoding an image.

Claims

1. A method of decoding an image using a decoding apparatus, comprising:

determining an intra prediction mode for a current block in a picture from intra prediction modes predefined in the decoding device, the intra prediction modes including a planar mode, a DC mode, and a bi-directional intra prediction mode;

determining reference samples for intra prediction of the current block based on the intra prediction mode being a bi-directional intra prediction mode; and

generating a prediction block for the current block by performing intra prediction on current samples in the current block based on the reference samples,

wherein the reference samples include an upper end reference sample adjacent to the current block, a left side reference sample adjacent to the current block, and a plurality of right side reference samples adjacent to the current block.

2. The method of claim 1, wherein the position of the upper end reference sample is determined based on the position of the current sample.

3. The method of claim 2, wherein the upper end reference sample has the same x-coordinate as the current sample.

4. The method of claim 1, wherein the position of the left reference sample is determined based on the position of the current sample.

5. The method of claim 4, wherein the left reference sample has the same y-coordinate as the current sample.

6. The method of claim 1, wherein the position of the right reference sample is determined based on a size of the current block.

7. A method of encoding an image using an encoding apparatus, comprising:

acquiring a prediction block of a current block in an image;

acquiring a residual block of the current block based on the original block of the current block and the prediction block; and

encoding the residual block to generate a bitstream,

wherein the intra prediction mode of the current block is determined from intra prediction modes including a planar mode, a DC mode, and a bi-directional intra prediction mode,

wherein the prediction samples of the prediction block are obtained by using reference samples corresponding to an intra prediction mode of the current block;

8. A computer readable medium having stored thereon instructions, which when executed by a processor, perform a method comprising:

determining an intra prediction mode of a current block in an image from intra prediction modes predefined in a decoding apparatus, the intra prediction modes including a planar mode, a DC mode, and a bi-directional intra prediction mode;