KR102342870B1 - Intra prediction mode-based image processing method and apparatus therefor - Google Patents

Abstract

Disclosed are a method for processing an image based on an intra prediction mode and an apparatus therefor. Specifically, a method of processing an image based on an intra prediction mode, the method comprising: inducing an intra prediction mode of a current block; deriving a first reference sample from at least one of left, upper, upper-left, lower-left, and upper-right reference samples of the current block based on the intra prediction mode; deriving a second reference sample from at least one of right, lower, and lower right reference samples of the current block based on the intra prediction mode; dividing the current block into a first sub-region and a second sub-region; generating a prediction sample of the first sub-region by using the first reference sample; and generating a prediction sample of the second sub-region by using the first reference sample and the second reference sample.

Description

Intra prediction mode-based image processing method and apparatus therefor

The present invention relates to a still image or a moving image processing method, and more particularly, to a method for encoding/decoding a still image or moving image based on an intra prediction mode, and an apparatus supporting the same.

Compression coding refers to a series of signal processing techniques for transmitting digitized information through a communication line or storing it in a form suitable for a storage medium. Media such as video, image, and audio may be subjected to compression encoding. In particular, a technique for performing compression encoding on an image is called video image compression.

Next-generation video content will have features of high spatial resolution, high frame rate, and high dimensionality of scene representation. In order to process such content, it will bring a huge increase in terms of memory storage, memory access rate and processing power.

Therefore, there is a need to design a coding tool for more efficiently processing next-generation video content.

An object of the present invention is to propose a linear interpolation intra prediction method for generating a weighted prediction sample based on a distance between a prediction sample and a reference sample.

Another object of the present invention is to propose a method for more accurately generating a prediction sample by combining the conventional general intra prediction and linear interpolation intra prediction.

Another object of the present invention is to propose a method of selectively applying conventional general intra prediction and linear interpolation intra prediction based on a distance between a prediction sample and a reference sample of a reconstructed region.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the description below. will be able

An aspect of the present invention provides a method of processing an image based on an intra prediction mode, the method comprising: inducing an intra prediction mode of a current block; deriving a first reference sample from at least one of left, upper, upper-left, lower-left, and upper-right reference samples of the current block based on the intra prediction mode; deriving a second reference sample from at least one of right, lower, and lower right reference samples of the current block based on the intra prediction mode; dividing the current block into a first sub-region and a second sub-region; generating a prediction sample of the first sub-region by using the first reference sample; and generating a prediction sample of the second sub-region by using the first reference sample and the second reference sample.

Preferably, the first sub-region includes one sample line adjacent to a reference sample determined according to a prediction direction of the intra prediction mode among left, upper, upper-left, lower-left, and upper-right reference samples of the current block. Way.

Preferably, the first sub-region includes a specific number of sample lines adjacent to a reference sample determined according to a prediction direction of the intra prediction mode among left, upper, upper-left, lower-left, and upper-right reference samples of the current block. can do.

Preferably, the specific number may be determined based on at least one of a distance between a current sample in the current block and the first reference sample, a size of the current block, or the intra prediction mode.

Preferably, generating the prediction sample of the second sub-region comprises: generating a first prediction sample using the first reference sample and generating a second prediction sample using the second reference sample; and generating a final prediction sample of the second sub-region by weight summing the first prediction sample and the second prediction sample.

Preferably, the weights respectively applied to the first prediction sample and the second prediction sample are based on a ratio of a distance between a current sample and the first reference sample in the current block and a distance between the current sample and the second reference sample. can be determined by

Another aspect of the present invention provides an apparatus for processing an image based on an intra prediction mode, comprising: a prediction mode inducing unit for inducing an intra prediction mode of a current block; a first reference sample derivation unit for deriving a first reference sample from at least one of left, upper, upper left, lower left, and right reference samples of the current block based on the intra prediction mode; a second reference sample derivation unit for deriving a second reference sample from at least one of right, lower, and lower right reference samples of the current block based on the intra prediction mode; a sub-region dividing unit dividing the current block into a first sub-region and a second sub-region; and a prediction sample generator configured to generate a prediction sample of the first sub-region using the first reference sample and generate a prediction sample of the second sub-region using the first reference sample and the second reference sample. may include

Preferably, the first sub-region includes one sample line adjacent to a reference sample determined according to a prediction direction of the intra prediction mode among left, upper, upper-left, lower-left, and upper-right reference samples of the current block. can

Preferably, the first sub-region includes a specific number of sample lines adjacent to a reference sample determined according to a prediction direction of the intra prediction mode among left, upper, upper-left, lower-left, and upper-right reference samples of the current block. can do.

Preferably, the specific number may be determined based on at least one of a distance between a current sample in the current block and the first reference sample, a size of the current block, or the intra prediction mode.

Preferably, the prediction sample generator generates a first prediction sample by using the first reference sample, generates a second prediction sample by using the second reference sample, and generates the first prediction sample and the second prediction sample. A final prediction sample of the second sub-region may be generated by weighted summing the prediction samples.

Preferably, the weights respectively applied to the first prediction sample and the second prediction sample are based on a ratio of a distance between a current sample and the first reference sample in the current block and a distance between the current sample and the second reference sample. can be determined by

According to an embodiment of the present invention, by generating a prediction sample using a plurality of reference samples determined according to the intra prediction mode, it is possible to improve the compression efficiency compared to the conventional image compression technique.

In addition, according to an embodiment of the present invention, by adaptively determining a reference sample used for prediction based on the distance between the prediction sample and the reference sample of the reconstructed area, the accuracy of the sample value of the reconstructed area is effectively reflected, The accuracy of the prediction can be further improved.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. .

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as a part of the detailed description to facilitate the understanding of the present invention, provide embodiments of the present invention, and together with the detailed description, explain the technical features of the present invention.
1 is a schematic block diagram of an encoder in which encoding of a still image or a moving picture signal is performed as an embodiment to which the present invention is applied.
2 is a schematic block diagram of a decoder in which encoding of a still image or moving image signal is performed as an embodiment to which the present invention is applied.
3 is a diagram for explaining a structure of division of a coding unit that can be applied to the present invention.
4 is a diagram for describing a prediction unit applicable to the present invention.
5 is a diagram illustrating an intra prediction method as an embodiment to which the present invention is applied.
6 illustrates a prediction direction according to an intra prediction mode.
7 and 8 are diagrams for explaining a linear interpolation prediction method as an embodiment to which the present invention is applied.
9 is a diagram for explaining a method for generating a lower right reference sample in a conventional linear interpolation prediction method as an embodiment to which the present invention can be applied.
10 is a diagram for explaining a method of generating right reference samples and lower reference samples according to an embodiment to which the present invention is applied.
11 and 12 are diagrams for comparing and explaining a conventional intra prediction method and a linear interpolation intra prediction method as an embodiment to which the present invention can be applied.
13 is a diagram for explaining a new intra prediction method according to an embodiment of the present invention.
14 is a diagram illustrating an intra prediction method according to an embodiment of the present invention.
15 is a diagram illustrating in more detail an intra prediction unit according to an embodiment of the present invention.
16 is an embodiment to which the present invention is applied, and shows a structure diagram of a content streaming system.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. DETAILED DESCRIPTION The detailed description set forth below in conjunction with the appended drawings is intended to describe exemplary embodiments of the present invention and is not intended to represent the only embodiments in which the present invention may be practiced. The following detailed description includes specific details in order to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without these specific details.

In some cases, well-known structures and devices may be omitted or shown in block diagram form focusing on core functions of each structure and device in order to avoid obscuring the concept of the present invention.

In addition, the terms used in the present invention have been selected as widely used general terms as possible, but specific cases will be described using terms arbitrarily selected by the applicant. In such a case, since the meaning is clearly described in the detailed description of the part, it should not be simply interpreted only by the name of the term used in the description of the present invention, but it should be understood and interpreted as well as the meaning of the term. .

Specific terms used in the following description are provided to help the understanding of the present invention, and the use of these specific terms may be changed to other forms without departing from the technical spirit of the present invention. For example, signals, data, samples, pictures, frames, blocks, etc. may be appropriately substituted and interpreted in each coding process.

Hereinafter, in the present specification, a 'processing unit' means a unit in which encoding/decoding processing such as prediction, transformation, and/or quantization is performed. Hereinafter, for convenience of description, a processing unit may be referred to as a 'processing block' or a 'block'.

The processing unit may be interpreted as including a unit for a luminance component and a unit for a chroma component. For example, the processing unit may correspond to a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

Also, the processing unit may be interpreted as a unit for a luminance component or a unit for a chroma component. For example, the processing unit may be a coding tree block (CTB), a coding block (CB), a prediction block (PU) or a transform block (TB) for a luma component. ) may be applicable. Alternatively, it may correspond to a coding tree block (CTB), a coding block (CB), a prediction block (PU), or a transform block (TB) for a chroma component. Also, the present invention is not limited thereto, and the processing unit may be interpreted as including a unit for a luminance component and a unit for a chroma component.

In addition, the processing unit is not necessarily limited to a square block, and may be configured in a polygonal shape having three or more vertices.

Also, in the present specification, a pixel or a pixel is collectively referred to as a sample. And, using the sample may mean using a pixel value or a pixel value.

1 is a schematic block diagram of an encoder in which encoding of a still image or a moving picture signal is performed as an embodiment to which the present invention is applied.

Referring to FIG. 1 , the encoder 100 includes an image divider 110 , a subtractor 115 , a transform unit 120 , a quantizer 130 , an inverse quantizer 140 , an inverse transform unit 150 , and a filtering unit. 160 , a decoded picture buffer (DPB) 170 , a prediction unit 180 , and an entropy encoding unit 190 may be included. Also, the prediction unit 180 may include an inter prediction unit 181 and an intra prediction unit 182 .

The image dividing unit 110 divides an input video signal (or a picture, a frame) input to the encoder 100 into one or more processing units.

The subtractor 115 subtracts a prediction signal (or a prediction block) output from the prediction unit 180 (ie, the inter prediction unit 181 or the intra prediction unit 182) from the input image signal to make a difference Generate a residual signal (or residual block). The generated differential signal (or differential block) is transmitted to the transform unit 120 .

The transform unit 120 converts the difference signal (or difference block) into a transform technique (eg, DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), GBT (Graph-Based Transform), KLT (Karhunen-Loeve transform)). etc.) to generate a transform coefficient. In this case, the transform unit 120 may generate transform coefficients by performing transform using a transform technique determined according to the prediction mode applied to the residual block and the size of the residual block.

The quantization unit 130 quantizes the transform coefficients and transmits them to the entropy encoding unit 190 , and the entropy encoding unit 190 entropy-codes the quantized signal and outputs it as a bit stream.

Meanwhile, a quantized signal output from the quantization unit 130 may be used to generate a prediction signal. For example, the quantized signal may be restored to a differential signal by applying inverse quantization and inverse transformation through the inverse quantizer 140 and the inverse transform unit 150 in the loop. A reconstructed signal may be generated by adding the reconstructed differential signal to a prediction signal output from the inter prediction unit 181 or the intra prediction unit 182 .

Meanwhile, in the compression process as described above, as adjacent blocks are quantized by different quantization parameters, deterioration in which block boundaries are visible may occur. This phenomenon is called blocking artifacts, and it is one of the important factors for evaluating image quality. A filtering process may be performed to reduce such deterioration. Through this filtering process, the image quality can be improved by removing the blocking degradation and reducing the error of the current picture.

The filtering unit 160 applies filtering to the reconstructed signal and outputs it to the reproduction device or transmits it to the decoded picture buffer 170 . The filtered signal transmitted to the decoded picture buffer 170 may be used as a reference picture in the inter prediction unit 181 . As such, by using the filtered picture as a reference picture in the inter prediction mode, it is possible to improve not only the picture quality but also the encoding efficiency.

The decoded picture buffer 170 may store the filtered picture to be used as a reference picture in the inter prediction unit 181 .

The inter prediction unit 181 performs temporal prediction and/or spatial prediction in order to remove temporal redundancy and/or spatial redundancy with reference to a reconstructed picture. Here, since the reference picture used for prediction is a transformed signal that has undergone quantization and inverse quantization on a block-by-block basis during encoding/decoding at a previous time, blocking artifacts or ringing artifacts may exist. have.

Accordingly, the inter prediction unit 181 may interpolate a signal between pixels in units of sub-pixels by applying a low-pass filter in order to solve performance degradation due to discontinuity or quantization of the signal. Here, the sub-pixel means a virtual pixel generated by applying an interpolation filter, and the integer pixel means an actual pixel existing in the reconstructed picture. As the interpolation method, linear interpolation, bi-linear interpolation, Wiener filter, etc. may be applied.

The interpolation filter may be applied to a reconstructed picture to improve prediction precision. For example, the inter prediction unit 181 generates interpolated pixels by applying an interpolation filter to integer pixels, and uses an interpolated block composed of interpolated pixels as a prediction block. prediction can be made.

The intra prediction unit 182 predicts the current block with reference to samples in the vicinity of the block to be currently encoded. The intra prediction unit 182 may perform the following process to perform intra prediction. First, a reference sample necessary for generating a prediction signal may be prepared. In addition, a prediction signal may be generated using the prepared reference sample. Thereafter, the prediction mode is encoded. In this case, the reference sample may be prepared through reference sample padding and/or reference sample filtering. Since the reference sample has undergone prediction and reconstruction, quantization errors may exist. Accordingly, in order to reduce such an error, a reference sample filtering process may be performed for each prediction mode used for intra prediction.

In particular, the intra prediction unit 182 according to the present invention may perform intra prediction on the current block by linearly interpolating prediction sample values generated based on the intra prediction mode of the current block. A more detailed description of the intra prediction unit 182 will be described later.

A prediction signal (or prediction block) generated by the inter prediction unit 181 or the intra prediction unit 182 is used to generate a reconstructed signal (or reconstructed block) or a differential signal (or a residual block) can be used to create

2 is a schematic block diagram of a decoder in which encoding of a still image or moving image signal is performed as an embodiment to which the present invention is applied.

Referring to FIG. 2 , the decoder 200 includes an entropy decoding unit 210 , an inverse quantization unit 220 , an inverse transform unit 230 , an adder 235 , a filtering unit 240 , and a decoded picture buffer (DPB). Buffer Unit) 250 and the prediction unit 260 may be included. In addition, the prediction unit 260 may include an inter prediction unit 261 and an intra prediction unit 262 .

In addition, a reconstructed video signal output through the decoder 200 may be reproduced through a reproduction device.

The decoder 200 receives a signal (ie, a bit stream) output from the encoder 100 of FIG. 1 , and the received signal is entropy-decoded through the entropy decoding unit 210 .

The inverse quantization unit 220 obtains a transform coefficient from the entropy-decoded signal using the quantization step size information.

The inverse transform unit 230 applies an inverse transform technique to inversely transform transform coefficients to obtain a residual signal (or a residual block).

The adder 235 converts the obtained differential signal (or differential block) into a prediction signal (or a prediction block) output from the prediction unit 260 (ie, the inter prediction unit 261 or the intra prediction unit 262 ). ) to generate a reconstructed signal (or a reconstructed block).

The filtering unit 240 applies filtering to a reconstructed signal (or a reconstructed block) and outputs it to a reproduction device or transmits it to the decoded picture buffer unit 250 . The filtered signal transmitted to the decoded picture buffer unit 250 may be used as a reference picture in the inter prediction unit 261 .

In this specification, the embodiments described in the filtering unit 160, the inter prediction unit 181, and the intra prediction unit 182 of the encoder 100 are the filtering unit 240 of the decoder, the inter prediction unit 261 and the The same may be applied to the intra prediction unit 262 .

In particular, the intra prediction unit 262 according to the present invention may perform intra prediction on the current block by linearly interpolating prediction sample values generated based on the intra prediction mode of the current block. A more detailed description of the intra prediction unit 262 will be described later.

In general, a block-based image compression method is used in a still image or moving image compression technique (eg, HEVC). The block-based image compression method is a method for processing an image by dividing the image into specific block units, and can reduce memory usage and computational amount.

3 is a diagram for explaining a structure of division of a coding unit that can be applied to the present invention.

The encoder divides one image (or picture) into a quadrangular coding tree unit (CTU) unit. In addition, one CTU is sequentially encoded according to a raster scan order.

In HEVC, the size of the CTU may be determined as any one of 64×64, 32×32, and 16×16. The encoder may select and use the size of the CTU according to the resolution of the input image or the characteristics of the input image. The CTU includes a coding tree block (CTB) for a luma component and a CTB for two chroma components corresponding thereto.

One CTU may be divided into a quad-tree structure. That is, one CTU is divided into four units having a square shape and having a half horizontal size and a half vertical size to generate a coding unit (CU). have. The division of such a quad-tree structure may be performed recursively. That is, a CU is hierarchically divided from one CTU into a quad-tree structure.

A CU refers to a basic unit of coding in which an input image processing process, for example, intra/inter prediction is performed. A CU includes a coding block (CB) for a luma component and a CB for two chroma components corresponding thereto. In HEVC, the size of a CU may be set to any one of 64×64, 32×32, 16×16, and 8×8.

3, the quad-tree root node (root node) is related to the CTU. The quad-tree is split until reaching a leaf node, and a leaf node corresponds to a CU.

More specifically, the CTU corresponds to a root node and has the smallest depth (ie, depth=0) value. Depending on the characteristics of the input image, the CTU may not be divided, and in this case, the CTU corresponds to a CU.

The CTU may be divided in the form of a quad tree, and as a result, lower nodes having a depth of 1 (depth=1) are generated. And, a node (ie, a leaf node) that is no longer divided in a lower node having a depth of 1 corresponds to a CU. For example, CU(a), CU(b), and CU(j) corresponding to nodes a, b, and j in FIG. 3(b) are split once in the CTU, and have a depth of 1.

At least one of the nodes having a depth of 1 may be divided again in the form of a quiz tree, and as a result, lower nodes having a depth of 1 (ie, depth=2) are generated. And, a node (ie, a leaf node) that is no longer divided in a lower node having a depth of 2 corresponds to a CU. For example, CU(c), CU(h), and CU(i) corresponding to nodes c, h and i in FIG. 3(b) are divided twice in the CTU, and have a depth of 2.

Also, at least one of the nodes having a depth of 2 may be divided again in a quad tree form, and as a result, lower nodes having a depth of 3 (ie, depth=3) are generated. And, a node (ie, a leaf node) that is no longer divided in a lower node having a depth of 3 corresponds to a CU. For example, CU(d), CU(e), CU(f), and CU(g) corresponding to nodes d, e, f, and g in FIG. 3(b) are divided three times in the CTU, have depth

The encoder may determine the maximum or minimum size of a CU according to a characteristic (eg, resolution) of a video image or in consideration of encoding efficiency. In addition, information about this or information that can induce it may be included in the bitstream. A CU having the largest size may be referred to as a largest coding unit (LCU), and a CU having the smallest size may be referred to as a smallest coding unit (SCU).

Also, a CU having a tree structure may be hierarchically divided with predetermined maximum depth information (or maximum level information). And, each divided CU may have depth information. Since the depth information indicates the number and/or degree of division of the CU, it may include information about the size of the CU.

Since the LCU is divided in the form of a quad tree, the size of the SCU can be obtained using information on the size and maximum depth of the LCU. Or conversely, the size of the LCU can be obtained by using the information on the size of the SCU and the maximum depth of the tree.

For one CU, information indicating whether the CU is split (eg, a split CU flag (split_cu_flag)) may be transmitted to the decoder. This partitioning information is included in all CUs except for the SCU. For example, if the value of the flag indicating whether to split is '1', the CU is divided into 4 CUs again. If the value of the flag indicating whether to split is '0', the CU is not divided any more and processing may be performed.

As described above, a CU is a basic unit of coding in which intra prediction or inter prediction is performed. HEVC divides a CU into prediction units (PUs) in order to more effectively code an input image.

A PU is a basic unit for generating a prediction block, and even within one CU, a prediction block may be generated differently in units of PUs. However, PUs belonging to one CU are not mixed with intra prediction and inter prediction, and PUs belonging to one CU are coded with the same prediction method (ie, intra prediction or inter prediction).

A PU is not divided into a quad-tree structure, and is divided once in a predetermined form in one CU. This will be described with reference to the drawings below.

4 is a diagram for describing a prediction unit applicable to the present invention.

A PU is divided differently depending on whether an intra prediction mode or an inter prediction mode is used as the coding mode of the CU to which the PU belongs.

FIG. 4(a) illustrates a PU when an intra prediction mode is used, and FIG. 4(b) illustrates a PU when an inter prediction mode is used.

Referring to FIG. 4( a ), assuming that the size of one CU is 2N×2N (N=4,8,16,32), one CU has two types (ie, 2N×2N or N ×N).

Here, when the PU is divided into 2N×2N type PU, it means that only one PU exists in one CU.

On the other hand, when divided into N×N PUs, one CU is divided into 4 PUs, and different prediction blocks are generated for each PU unit. However, this PU division may be performed only when the size of the CB with respect to the luminance component of the CU is the minimum size (ie, when the CU is the SCU).

Referring to FIG. 4(b), assuming that the size of one CU is 2N×2N (N=4,8,16,32), one CU has 8 PU types (ie, 2N×2N). , N×N, 2N×N, N×2N, nL×2N, nR×2N, 2N×nU, 2N×nD).

Similar to intra prediction, N×N type PU partitioning may be performed only when the size of a CB with respect to a luminance component of a CU is the minimum size (ie, when the CU is an SCU).

Inter prediction supports PU partitioning in the 2N×N format split in the horizontal direction and in the N×2N format split in the vertical direction.

In addition, PU partitioning in the form of nL×2N, nR×2N, 2N×nU, and 2N×nD, which are asymmetric motion partitioning (AMP) types, is supported. Here, 'n' means a 1/4 value of 2N. However, the AMP cannot be used when the CU to which the PU belongs is a CU of the minimum size.

In order to efficiently encode an input image in one CTU, the optimal division structure of the coding unit (CU), the prediction unit (PU), and the transform unit (TU) undergoes the following process to achieve minimum rate-distortion. It can be determined based on the value. For example, looking at the optimal CU partitioning process within a 64×64 CTU, rate-distortion cost can be calculated while going through the partitioning process from a CU of a 64×64 size to a CU of an 8×8 size. The specific process is as follows.

1) An optimal PU and TU partition structure that generates a minimum rate-distortion value is determined by performing inter/intra prediction, transform/quantization, inverse quantization/inverse transform, and entropy encoding for a CU having a size of 64×64.

2) A 64×64 CU is divided into 4 CUs with a size of 32×32, and an optimal PU and TU partition structure that generates a minimum rate-distortion value for each 32×32 CU is determined.

3) The 32×32 CU is again divided into four 16×16 CUs, and an optimal PU and TU partition structure that generates the minimum rate-distortion value for each 16×16 CU is determined.

4) The 16×16 CU is again divided into 4 8×8 CUs, and an optimal PU and TU partition structure that generates the minimum rate-distortion value for each 8×8 CU is determined.

5) 16×16 block by comparing the sum of the rate-distortion values of 16×16 CUs calculated in step 3) and the rate-distortion values of 4 8×8 CUs calculated in step 4) above. Determine the optimal CU partition structure in The same process is performed for the remaining three 16×16 CUs.

6) 32×32 block by comparing the sum of the rate-distortion values of 32×32 CUs calculated in the process of 2) above and the rate-distortion values of four 16×16 CUs obtained in the process of 5) above Determine the optimal CU partition structure in The same process is performed for the remaining three 32×32 CUs.

7) Finally, by comparing the sum of the rate-distortion values of 64×64 CUs calculated in step 1) and the rate-distortion values of four 32×32 CUs obtained in step 6) above, 64 An optimal CU partition structure is determined within a x64 block.

In the intra prediction mode, a prediction mode is selected in units of PUs, and prediction and reconstruction are performed in units of actual TUs for the selected prediction mode.

TU means a basic unit in which actual prediction and reconstruction are performed. A TU includes a transform block (TB) for a luma component and a TB for two chroma components corresponding thereto.

As in the example of FIG. 3, one CTU is divided into a quad-tree structure to generate a CU, a TU is hierarchically divided from one CU to be coded into a quad-tree structure.

Since a TU is divided into a quad-tree structure, a TU divided from a CU may be divided into smaller sub-TUs again. In HEVC, the size of a TU may be set to any one of 32×32, 16×16, 8×8, and 4×4.

Referring back to FIG. 3 , it is assumed that a root node of the quad-tree is related to a CU. The quad-tree is split until reaching a leaf node, and a leaf node corresponds to a TU.

More specifically, a CU corresponds to a root node and has the smallest depth (ie, depth=0) value. Depending on the characteristics of the input image, a CU may not be divided, and in this case, the CU corresponds to a TU.

A CU may be divided in a quad tree form, and as a result, lower nodes having a depth of 1 (depth=1) are generated. And, a node (ie, a leaf node) that is no longer divided in a lower node having a depth of 1 corresponds to a TU. For example, in FIG. 3(b) , TU(a), TU(b), and TU(j) corresponding to nodes a, b, and j are split once in a CU and have a depth of 1.

At least one of the nodes having a depth of 1 may be divided again in the form of a quiz tree, and as a result, lower nodes having a depth of 1 (ie, depth=2) are generated. And, a node (ie, a leaf node) that is no longer divided in a lower node having a depth of 2 corresponds to a TU. For example, TU(c), TU(h), and TU(i) corresponding to nodes c, h and i in FIG. 3(b) are divided twice in the CU, and have a depth of 2.

Also, at least one of the nodes having a depth of 2 may be divided again in a quad tree form, and as a result, lower nodes having a depth of 3 (ie, depth=3) are generated. And, a node (ie, a leaf node) that is no longer divided in a lower node having a depth of 3 corresponds to a CU. For example, in FIG. 3(b), TU(d), TU(e), TU(f), and TU(g) corresponding to nodes d, e, f, g are divided three times in the CU, have depth

A TU having a tree structure may be hierarchically divided with predetermined maximum depth information (or maximum level information). And, each split TU may have depth information. Since the depth information indicates the number and/or degree of division of the TU, information on the size of the TU may be included.

For one TU, information indicating whether the corresponding TU is split (eg, split TU flag (split_transform_flag)) may be transmitted to the decoder. This division information is included in all TUs except for the TU of the minimum size. For example, if the value of the flag indicating whether to split is '1', the corresponding TU is again divided into 4 TUs, and if the value of the flag indicating whether to split is '0', the corresponding TU is no longer divided.

prediction

In order to reconstruct the current processing unit on which decoding is performed, the current picture including the current processing unit or decoded portions of other pictures may be used.

Prediction of a picture (slice) that uses only the current picture for restoration, that is, a picture (slice) that performs only intra prediction, an intra picture or an I picture (slice), and a picture (slice) using at most one motion vector and reference index to predict each unit A picture (slice) using a predictive picture or a P picture (slice), up to two motion vectors and a reference index may be referred to as a bi-predictive picture or a B picture (slice).

Intra prediction refers to a prediction method for deriving a current processing block from data elements (eg, sample values, etc.) of the same decoded picture (or slice). That is, it refers to a method of predicting the pixel value of the current processing block with reference to the reconstructed regions in the current picture.

Inter prediction refers to a prediction method of deriving a current processing block based on a data element (eg, a sample value or a motion vector) of a picture other than the current picture. That is, it refers to a method of predicting the pixel value of the current processing block with reference to reconstructed areas in another reconstructed picture other than the current picture.

Hereinafter, intra prediction (or intra prediction) will be described in more detail.

Intra prediction (or in-screen prediction)

5 is a diagram illustrating an intra prediction method as an embodiment to which the present invention is applied.

Referring to FIG. 5 , the decoder derives an intra prediction mode of the current processing block ( S501 ).

In intra prediction, a prediction direction with respect to a position of a reference sample used for prediction may be obtained according to a prediction mode. An intra prediction mode having a prediction direction is referred to as an intra directional prediction mode (Intra_Angular prediction mode). On the other hand, as intra prediction modes that do not have a prediction direction, there are an intra planner (INTRA_PLANAR) prediction mode and an intra DC (INTRA_DC) prediction mode.

Table 1 illustrates an intra prediction mode and related names, and FIG. 6 illustrates a prediction direction according to the intra prediction mode.

In intra prediction, prediction of the current processing block is performed based on the derived prediction mode. Since a reference sample used for prediction and a specific prediction method vary depending on the prediction mode, when the current block is encoded in the intra prediction mode, the decoder derives the prediction mode of the current block to perform prediction.

The decoder checks whether neighboring samples of the current processing block can be used for prediction, and configures reference samples to be used for prediction ( S502 ).

In intra prediction, the neighboring samples of the current processing block are samples adjacent to the left boundary of the current processing block of size nS × nS and a total of 2 × nS samples neighboring the bottom-left of the current processing block. It means a sample adjacent to the top boundary, a total of 2×nS samples neighboring to the top-right side, and one sample neighboring to the top-left side of the current processing block.

However, some of the neighboring samples of the current processing block may not yet be decoded or available. In this case, the decoder may construct reference samples to be used for prediction by substituting unavailable samples with available samples.

The decoder may perform filtering of the reference sample based on the intra prediction mode (S503).

Whether to perform filtering of the reference sample may be determined based on the size of the current processing block. In addition, the filtering method of the reference sample may be determined by a filtering flag transmitted from the encoder.

The decoder generates a prediction block for the current processing block based on the intra prediction mode and reference samples (S504). That is, the decoder predicts the current processing block based on the intra prediction mode derived in the intra prediction mode deriving step S501 and the reference samples obtained through the reference sample construction step S502 and the reference sample filtering step S503. Generate blocks (ie, generate prediction samples).

In order to minimize the discontinuity of the boundary between the processing blocks when the current processing block is encoded in the INTRA_DC mode, the left boundary sample of the prediction block (that is, the sample within the prediction block adjacent to the left boundary) and the upper side of the prediction block in step S504 (top) boundary samples (ie, samples in the prediction block adjacent to the upper boundary) may be filtered.

In addition, in step S504 , filtering may be applied to the left boundary sample or the upper boundary sample similarly to the INTRA_DC mode for a vertical mode and a horizontal mode among intra directional prediction modes.

More specifically, when the current processing block is encoded in a vertical mode or a horizontal mode, a value of a prediction sample may be derived based on a reference sample located in the prediction direction. In this case, a boundary sample not located in the prediction direction among the left boundary sample or the upper boundary sample of the prediction block may be adjacent to a reference sample not used for prediction. That is, the distance to the reference sample not used for prediction may be much closer than the distance to the reference sample used for prediction.

Accordingly, the decoder may adaptively apply filtering to the left boundary samples or the upper boundary samples according to whether the intra prediction direction is a vertical direction or a horizontal direction. That is, if the intra prediction direction is a vertical direction, filtering may be applied to left boundary samples, and if the intra prediction direction is a horizontal direction, filtering may be applied to upper boundary samples.

As described above, HEVC uses a total of 35 prediction methods, 33 directional prediction methods and two non-directional prediction methods through intra prediction (or intra prediction), and is encoded in the order of surrounding reference samples (raster scan order). / Assuming that decoding is performed, a prediction sample is generated using an upper reference sample or a left reference sample). Then, the generated prediction sample is copied according to the direction of the intra prediction mode.

Since the prediction sample value is simply copied according to the prediction direction, the accuracy of prediction decreases as the distance from the reference sample increases. That is, when the distance between the reference samples used for prediction and the prediction sample is close, the prediction accuracy is high, but when the distance between the reference samples used for prediction and the prediction sample is long, the prediction accuracy is low.

In order to reduce such a prediction error, the present invention proposes a linear interpolation intra prediction method for generating a weighted prediction sample based on a distance between a prediction sample and a reference sample. In particular, the present invention proposes a method of generating the lower right reference sample more accurately compared to the method of generating the lower right reference sample in the recently discussed linear interpolation prediction method. First, a linear interpolation prediction method will be described with reference to the drawings below.

7 and 8 are diagrams for explaining a linear interpolation prediction method as an embodiment to which the present invention is applied.

Referring to FIG. 7 , although the decoder is mainly described for convenience of description, the linear interpolation prediction method proposed in the present invention may be equally performed in the encoder.

The decoder parses (or checks) a LIP flag indicating whether or not linear interpolation prediction (LIP) (or linear interpolation intra prediction) is applied to the current block from the bit stream received from the encoder (S701).

In an embodiment, the decoder may derive the intra prediction mode of the current block prior to step S701, and may derive the intra prediction mode of the current block after step S701. In other words, the step of inducing the intra prediction mode before or after step S701 may be added. And, the step of inducing the intra prediction mode is parsing the MPM flag indicating whether the MPM (Most Probable Mode) is applied to the current block, and depending on whether the MPM is applied within the MPM candidate or the residual prediction mode candidate. It may include parsing an index indicating a prediction mode applied to intra prediction of the current block.

The decoder generates a lower-right reference sample adjacent to the lower-right side of the current block (S702). The decoder may generate the lower right reference sample using a variety of different methods. A more detailed description of this will be given later.

The decoder generates a right reference sample array or a lower reference sample array using the reconstructed reference sample around the current block and the lower right reference sample generated in step S702 (S703). In the present invention, the right reference sample arrangement may be collectively referred to as a right reference sample, a right reference sample, a right reference sample arrangement, etc., and the lower reference sample arrangement may be collectively referred to as a lower reference sample, a lower reference sample, a lower reference sample arrangement, etc. have. A more detailed description of this will be given later.

The decoder generates a first prediction sample and a second prediction sample based on the prediction direction of the intra prediction mode of the current block ( S704 and S705 ). Here, the first prediction sample (which may be referred to as a first reference sample) and the second prediction sample (which may also be referred to as a second reference sample) are reference samples positioned opposite to each other in the current block with respect to the prediction direction, or Prediction samples generated using reference samples located opposite to each other of the current block are indicated. The first prediction sample uses a first reference sample determined according to the intra prediction mode of the current block among reference samples (left, upper left, and upper reference samples) of the reconstructed region as described with reference to FIGS. 5 and 6 above. indicates a prediction sample generated by the , and the second prediction sample indicates a prediction sample generated using a second reference sample determined according to the intra prediction mode of the current block from among the right reference sample array or the lower reference sample array in step S703.

The decoder generates a final prediction sample by interpolating (or linearly interpolating) the first prediction sample and the second prediction sample generated in steps S704 and S705 ( S706 ). In other words, the decoder may generate the final prediction sample by weight summing the first prediction sample and the second prediction sample based on the distances between the current sample and the prediction samples (or reference samples).

Referring to FIG. 8 , although the decoder is mainly described for convenience of description, the linear interpolation prediction method proposed in the present invention may be performed in the encoder as well.

The decoder may generate the first prediction sample P based on the intra prediction mode. Specifically, the decoder may derive the first prediction sample by interpolating (or linearly interpolating) the A reference sample and the B reference sample determined according to the prediction direction among the upper reference samples. Meanwhile, unlike illustrated in FIG. 8 , when a reference sample determined according to a prediction direction is located at an integer pixel position, interpolation between reference samples may not be performed.

Also, the decoder may generate the second prediction sample P′ based on the intra prediction mode. Specifically, the decoder determines A' reference sample and B' reference sample according to the prediction direction of the intra prediction mode of the current block from among the lower reference samples, and performs a second prediction by linearly interpolating the A' reference sample and B' reference sample. sample can be derived. Meanwhile, unlike illustrated in FIG. 8 , when a reference sample determined according to a prediction direction is located at an integer pixel position, interpolation between reference samples may not be performed.

Then, the decoder determines weights respectively applied to the first prediction sample and the second prediction sample based on the distance between the current sample and the prediction sample (or reference sample), and uses the determined weights to predict the first prediction sample and the second prediction. Samples can be weighted to produce a final prediction sample.

The weight determination methods w1 and w2 shown in FIG. 8 are an example, and when the decoder determines the weights applied to the first prediction sample and the second prediction sample, respectively, as shown in FIG. 8 , the current sample and The vertical distance between the prediction samples (or reference samples) may be used, or the actual distance between the current sample and the prediction samples (or reference samples) may be used. If the actual distance is used, the distance may be calculated based on the actual location of the second reference sample used to generate the second prediction sample and the weight may be determined (or derived).

9 is a diagram for explaining a method for generating a lower right reference sample in a conventional linear interpolation prediction method as an embodiment to which the present invention can be applied.

Referring to FIG. 9 , the encoder/decoder uses the upper right reference sample 901 adjacent to the right and lower left of the current block and the lower right reference sample 902 adjacent to the lower left of the current block. A reference sample 903 may be created.

Referring to FIG. 9( b ), the encoder/decoder determines the rightmost sample (hereinafter, referred to as the upper-right sample) among reference samples neighboring the right-hand side of the current block (for example, refer to the upper-left corner of the current block) A sample that is horizontally two times the width of the current block from the sample, that is, [2*n-1, -1] samples in an n×n block) (904) and a reference next to the lower left of the current block The sample located at the lowermost side among the samples (hereinafter referred to as the lower-left sample) (for example, a sample that is separated by a distance twice the height of the current block in the vertical direction based on the upper-left reference sample of the current block, that is, n×n The lower right reference sample 906 may be generated by using the [-1, 2*n-1] sample) 905 in the block.

10 is a diagram for explaining a method of generating right reference samples and lower reference samples according to an embodiment to which the present invention is applied.

Referring to FIG. 10 , it is assumed that the size of the current block is 2x4. The encoder/decoder may generate a right reference sample and/or a lower reference sample by using the lower right reference sample BR adjacent to the lower right side of the current block and the reconstructed reference sample around the current block.

Specifically, the encoder/decoder may generate a lower reference sample by linearly interpolating a bottom right (BR) and a bottom left (BL) adjacent reference sample to the lower left of the current block. In other words, the encoder/decoder may generate the lower reference samples by performing weighted summation in units of pixels according to a distance ratio for each of the lower right reference sample BR and the lower left reference sample BL.

Also, the encoder/decoder may generate a right reference sample by linearly interpolating a lower right reference sample BR and a top right reference sample TR adjacent to the right of the current block. In other words, the encoder/decoder may generate the lower reference samples by performing weighted summation in units of pixels according to a distance ratio for each of the lower right reference sample BR and the upper right reference sample TR.

As described above, in the linear interpolation prediction method, the encoder/decoder uses a reference sample of an already encoded/decoded and reconstructed region and a predicted (that is, generated through prediction) reference sample of a region that has not yet been encoded/decoded at the current encoding time. A prediction block is generated by a weighted sum based on the distance to the sample. Such a linear interpolation prediction method may be used in combination with an existing intra prediction method, or may be used instead of an existing intra prediction method.

In the present invention, intra prediction other than linear interpolation intra prediction may be referred to as general intra prediction (or general intra prediction). For example, general intra prediction is an intra prediction method used in a conventional image compression technique (eg, HEVC), which is interpolated using one reference sample (or two adjacent integer pixel reference samples) determined according to the prediction direction. It may be an intra prediction method using a reference sample).

When the general intra prediction method and the linear interpolation prediction method are mixed and used, flag information for distinguishing each method may be used. In this case, a problem in that encoding bits increase due to flag signaling may occur. On the other hand, when only the linear interpolation prediction method is used instead of the general intra prediction method, when the linear interpolation prediction method has lower prediction accuracy compared to the general intra prediction method, there may be a problem in that the encoding efficiency is lowered.

Accordingly, the present invention proposes a new intra prediction method in which a general intra prediction method and a linear interpolation prediction method are combined to solve such a problem. The proposed new intra prediction method may be used instead of the general intra prediction method during intra encoding/decoding, or may be used in combination with the general intra prediction method.

According to an embodiment of the present invention, by combining the general intra prediction method and the linear interpolation prediction method, it is possible to solve the problem of low encoding efficiency when the aforementioned linear interpolation prediction method has lower prediction accuracy compared to the general intra prediction method.

In addition, according to an embodiment of the present invention, when the proposed new intra prediction method is used instead of the general intra prediction method, since flag information is not signaled, the problem of increasing the coding bit due to the use of the flag can be solved.

Example 1

An embodiment of the present invention proposes a new intra prediction method in which a general intra prediction method and a linear interpolation intra prediction method are combined. It will be described with reference to the drawings below.

11 and 12 are diagrams for comparing and explaining a conventional intra prediction method and a linear interpolation intra prediction method as an embodiment to which the present invention can be applied.

11 and 12 , it is assumed that the prediction direction of the prediction mode of the current block is positive vertical direction as shown.

Referring to FIG. 11 , when a general intra prediction method is applied, an encoder/decoder may generate a prediction sample by copying a sample value from an upper reference sample determined according to an intra prediction mode. For example, the encoder/decoder may copy the upper reference sample P1 to generate a prediction sample of the C1 sample. In the same way, the encoder/decoder may generate prediction samples of all samples in the current block.

12 , when the linear interpolation prediction method is applied, the encoder/decoder interpolates (or linearly interpolates) sample values of an upper reference sample and a lower reference sample determined according to the intra prediction mode to generate a prediction sample. have. For example, the encoder/decoder may linearly interpolate the upper reference sample P1 and the lower reference sample P′1 to generate a predicted sample of the C1 sample. At this time, linear interpolation (or weighted sum) may be performed by assigning weights of wUP1 and wDOWN1 to the P1 reference sample and the P′1 reference sample, respectively. In the same way, the encoder/decoder may generate prediction samples of all samples in the current block.

The weight determination method (wUP1, wDOWN1, etc.) shown in FIG. 12 is an example, and the decoder is applied to the first prediction sample (P1, P2, etc.) and the second prediction sample (P'1, P'2, etc.), respectively In determining the weight to be used, the vertical distance between the current sample and the prediction sample (or reference sample) may be used as shown in FIG. 12 , or the actual distance between the current sample and the prediction sample (or the reference sample) may be used. If the actual distance is used, the distance may be calculated based on the actual location of the second reference sample used to generate the second prediction sample and the weight may be determined (or derived).

In the new intra prediction method combining the general intra prediction method and the linear interpolation prediction method proposed by the present invention, the above-described general intra prediction method of FIG. 11 and the linear interpolation prediction method of FIG. 12 may be combined and applied. It will be described with reference to the drawings below.

13 is a diagram for explaining a new intra prediction method according to an embodiment of the present invention.

Referring to FIG. 13 , it is assumed that the size of the current block is 4x4 and the prediction direction of the prediction mode of the current block is positive vertical direction as shown.

In an embodiment of the present invention, the encoder/decoder may divide the current block into sub-regions and apply another intra prediction method to the divided sub-regions. Specifically, the encoder/decoder divides the current block into two sub-regions, generates prediction samples by applying the general intra prediction method to the first sub-region, and applies the linear interpolation prediction method to the second sub-region to generate prediction samples. can create

In the example of FIG. 13 , since upper reference samples among reference samples of the reconstructed region according to the prediction direction are used for prediction, the encoder/decoder divides the current block to include samples closest to the upper reference sample in the current block. It is possible to divide the current block into one sub-region and divide the current block into a second sub-region to include the remaining samples. If left reference samples among reference samples of the reconstructed region according to the prediction direction are used for prediction, the encoder/decoder configures the first sub-region to include samples closest to the left reference sample in the current block, The second sub-region may be configured to include the remaining samples.

A first row (ie, an uppermost row including samples C1, C2, C3, and C4) of the current block may be configured as the first sub-region. The encoder/decoder may generate prediction samples of samples in the first sub-region (or first region) by using general intra prediction. That is, the prediction sample of the C1 sample may be generated by copying the value of the P1 reference sample, the prediction sample of the C2 sample may be generated by copying the value of the P2 reference sample, and the prediction sample of the C3 sample may be generated by copying the value of the P3 reference sample. can be generated by copying the values, and the prediction samples of the C4 samples can be generated by copying the values of the P4 reference samples.

In addition, the second to fourth rows of the current block (ie, areas other than the first sub-area) may be configured as a second sub-area (second area). In this case, the encoder/decoder may generate prediction samples of samples in the second sub-region using a linear interpolation prediction method. That is, the prediction sample of the C5 sample in the second row may be generated through linear interpolation by applying the weights of wDOWN5 and wUP5 to the upper reference sample P5 value and the lower reference sample P′5 value, respectively. The prediction sample of the C6 sample in the third row may be generated through linear interpolation by applying the weights of wDOWN6 and wUP6 to the upper reference sample P6 value and the lower reference sample P′6 value, respectively. In the same way, the encoder/decoder may generate prediction samples of samples in the second sub-region.

As described above, in the new intra prediction method proposed by the present invention, a prediction value is generated using an existing intra prediction method in a specific area and a prediction value is generated using a linear interpolation prediction method in the remaining area to generate a final prediction block create

For convenience of explanation, assuming that the prediction direction of the intra prediction mode is vertical (that is, the prediction direction using the upper reference sample in the reconstructed region for prediction), in general, the upper reference sample performs encoding/decoding Since it is a sample value restored through Therefore, as the location of the sample is closer to the upper reference sample, it is more efficient to generate the prediction sample by applying the general intra prediction method to copy the upper reference sample value as it is, compared to applying the linear interpolation prediction.

On the other hand, as the location of the sample is further away from the upper reference sample, the accuracy of prediction according to the application of the general intra prediction method decreases. Therefore, the prediction efficiency can be increased by performing linear interpolation using the upper reference sample and the lower reference sample. .

The method proposed by the present invention may selectively use a general intra prediction method and a linear interpolation prediction method based on a distance from a reconstructed reference sample when performing intra prediction. That is, the encoder/decoder may variably select which prediction method among the general intra prediction method and the linear interpolation prediction method to generate the prediction block in the prediction block according to the distance from the reconstructed reference sample. In the example of FIG. 13 , a 4x4 block is assumed, but the same may be applied to other blocks of various sizes or shapes (eg, 8x8, 16x8, square block, non-square block, etc.).

In an embodiment, the encoder/decoder calculates the current block based on the distance between the prediction sample (or the current sample) and the reference sample of the reconstructed region, a first sub-region to which general intra prediction is applied, and a second sub-region to which linear interpolation prediction is applied. It can be divided into 2 sub-regions. For example, the encoder/decoder may divide the current block into a first sub-region and a second sub-region by comparing the distance between the prediction sample and the reference sample of the reconstructed region with a specific threshold value. For example, the encoder/decoder calculates the distance between the prediction sample and the reference sample of the reconstructed region, configures a sample line (or row, column) whose calculated distance is smaller than a specific threshold as a first sub-region, and the rest The sample lines may be configured as the second sub-region.

Also, according to an embodiment, the encoder/decoder may preset the size of the first sub-region (or the number of sample lines, the number of rows, the number of columns, etc.) according to the size of the current block. For example, when the current block is smaller than a predetermined size, the encoder/decoder uses one sample line (or row, column) in the current block adjacent to the reconstructed reference sample (ie, left or top) determined according to the prediction mode. may be configured as the first sub-region. In addition, when the current block is larger than or equal to a predetermined size, the encoder/decoder converts two sample lines (or rows, columns) adjacent to the reconstructed reference sample (ie, left or top) determined according to the prediction mode to the first sub-region. configurable. As an example, a table in which the number of sample lines included in the first sub-region is determined according to the size of the current block may be stored in the encoder/decoder, and the current block may be divided into the first sub-region and the second sub-region using this table. can

Also, according to an embodiment, the encoder/decoder may divide the current block into a first sub-region to which general intra prediction is applied and a second sub-region to which linear interpolation prediction is applied according to the prediction mode of the current block. As an example, a table in which the number of sample lines included in the first sub-region is determined according to the prediction mode may be stored in the encoder/decoder, and the current block may be divided into a first sub-region and a second sub-region using this . In this case, the table including the range or size information of the first sub-region corresponding to the prediction mode may be derived based on the distance between the prediction sample (or the current sample) and the reference sample of the reconstructed region. And, the distance of the reconstructed region to the reference sample may be calculated using the prediction direction or angle of the prediction mode.

Also, according to an embodiment, the encoder/decoder may divide the current block into a first sub-region to which general intra prediction is applied and a second sub-region to which linear interpolation prediction is applied according to the size and prediction mode of the current block. For example, a table in which the number of sample lines included in the first sub-region is determined according to the size and prediction mode of the current block may be stored in the encoder/decoder, and the current block is stored in the first sub-region and the second sub-region using the table. can be divided into In this case, the table including the range or size information of the first sub-region corresponding to the prediction mode may be derived based on the distance between the prediction sample (or the current sample) and the reference sample of the reconstructed area, and the reconstructed The distance from the reference sample of the region may be calculated using the prediction direction or angle of the prediction mode.

Also, according to an embodiment, a new prediction method combining a general intra prediction method and a linear interpolation prediction method may be used to replace all of the existing directional prediction modes. In this case, the intra prediction mode may be composed of a non-directional mode (eg, a planar mode, a DC mode) and a proposed new prediction directional mode.

Example 2

In an embodiment of the present invention, a new intra prediction method for deriving an intra prediction sample by combining an existing intra prediction method and a linear interpolation prediction method is proposed.

In an embodiment of the present invention, the encoder/decoder may generate a final prediction sample using a prediction sample generated through a conventional intra prediction method and a prediction sample generated through a linear interpolation prediction method.

In an embodiment, the encoder/decoder includes a prediction sample generated through a general intra prediction method (hereinafter referred to as a third prediction sample) and a prediction sample generated through a linear interpolation prediction method (hereinafter referred to as a fourth prediction sample). can be weighted to generate a final prediction sample. The proposed new intra prediction method can be generalized as in Equation 1 below.

Referring to Equation 1, C(i, j) represents an intra prediction sample generated by applying the general intra prediction method described with reference to FIG. 11, and L(i, j) is the linear interpolation prediction method described with reference to FIG. 12 above. represents an intra prediction sample generated by applying . And, (i, j) represents the horizontal and vertical positions (or coordinates) of the corresponding prediction sample in the current block (or prediction block), respectively. As an example, the weight α may be set to a value between 0 and 1. The encoder/decoder may generate a final prediction sample by summing the third prediction sample to which the weight α is applied and the fourth prediction sample to which the weight (1-α) is applied.

Equation 1 described above can be expressed as Equation 2 below in order to eliminate floating point calculations.

Referring to Equation 2, A and B represent weights respectively applied to the third prediction sample and the fourth prediction sample, and both may be expressed as non-negative integers. As an example, the offset value may be set to ^{2 (right_shift-1).} The shift operator a>>b represents the quotient obtained when a is divided by ^{2 b values.} In Equation 2, the ^{condition A+B=2 (right_shift)} may be satisfied. Integer arithmetic can be supported through Equation 2, and thus, arithmetic complexity can be reduced.

Example 3

In an embodiment of the present invention, various embodiments to which the generalized new intra prediction method proposed in Embodiments 1 and 2 described above are applied are proposed.

In an embodiment, the weight value of Equation 1 or 2 may be predefined according to the intra prediction mode. Based on Equation 1, for example, in the case of a planar mode, which is a non-directional mode, the weight α value applied to the general intra prediction sample may be set to ‘0’. In this case, the new intra prediction method may be simply replaced with a linear interpolation prediction method. Also, for example, in the case of the DC mode, which is the non-directional mode, the weight α value applied to the general intra prediction sample may be set to ‘1’. In this case, the new intra prediction method may be replaced with a general intra prediction method. Also, in the case of directional modes, a weight α value predefined according to the prediction mode may be used for intra prediction.

Also, according to an embodiment, the weight value defined in Equation 1 or 2 may be predefined according to the position of the prediction sample in the current processing block. Based on Equation 1, for example, in the case of an upper reference sample and a prediction sample adjacent to the left reference sample, which are reference samples of the reconstructed region, the weight α value applied to the prediction sample generated by the general intra prediction method rather than the other case. This can be set to be relatively larger. Here, a large value of weight α may mean assigning a larger weight to general intra prediction. As an example, as shown in Equation 3 below, the weight α may be modeled to be set differently according to the position of the current sample in the current block (or prediction block).

Here, C(i, j) represents an intra prediction sample generated by applying the general intra prediction method described with reference to FIG. 11, that is, a third prediction sample, and L(i, j) is the linear interpolation described with reference to FIG. 12 above. An intra prediction sample generated by applying a prediction method, that is, a fourth prediction sample is indicated. And, (i, j) represents the horizontal and vertical positions (or coordinates) of the corresponding prediction sample in the current block (or prediction block), respectively. The weight α is a weight applied to the third prediction sample and may be set to a value between 0 and 1. And, the weight (1-α) indicates a weight applied to the fourth prediction sample.

Also, according to an embodiment, the weight value of Equation 1 or 2 may be predefined according to the size or shape of the prediction block. Based on Equation 1, for example, when the size (width×height) of the current block is smaller than a predetermined threshold, the encoder/decoder may set a relatively smaller weight α than otherwise.

In addition, in an embodiment, when the weight value applied to the general intra prediction sample is not '0' or '1', the encoder/decoder performs the general intra prediction method and the proposed new prediction based on additional flag information. A method (or a linear interpolation intra prediction method) may be selected and used. For example, in the case of the planar mode, which is a non-directional mode, the value of α is set to '0', so additional flag information is not required, but in the case of the horizontal mode, the value of α is '0' or '1' If not (eg, set to 0.5), the encoder/decoder applies to the current processing block among the general intra prediction method and the proposed new prediction method (or linear interpolation prediction method) based on the flag information additionally transmitted through the bitstream Intra prediction can be performed by selecting a prediction method that is used.

In this case, a condition for which the signaling of flag information is required may be preset based on a weight value and/or an intra prediction mode. For example, the encoder/decoder may group prediction modes into several classes as follows in order to determine whether to signal the proposed additional flag information.

- Class A = {0, 1, 66}

- Class B = {2, 3, 4,… , 64, 65}

Class A represents a set of prediction modes for which additional flag information is not required, and class B represents a set of prediction modes for which additional flag information is required. Of course, the prediction mode included in each class above is only an example.

14 is a diagram illustrating an intra prediction method according to an embodiment of the present invention.

Referring to FIG. 14 , for convenience of description, the description is based on the decoder, but the intra prediction method proposed in the present invention can be equally applied to the encoder.

First, the decoder derives the intra prediction mode of the current block (S1401).

The decoder derives a first reference sample (or a reference sample array) from at least one reference sample among left, upper, upper-left, lower-left, and upper-right reference samples of the current block based on the intra prediction mode ( S1402 ).

The decoder derives a second reference sample from at least one of right, lower, and lower right reference samples of the current block based on the intra prediction mode (S1403). In this case, the decoder generates a lower-right reference sample adjacent to the lower-right side of the current block as described above with reference to FIGS. 7 and 9, and uses the lower-right reference sample to refer to the right as described with reference to FIGS. 7 and 10 above. A sample or a lower reference sample can be created.

The decoder divides the current block into a first sub-region and a second sub-region (S1404). As described above with reference to FIG. 13 , the decoder may divide the current block into sub-regions and apply another intra prediction method to the divided sub-regions. Specifically, the decoder divides the current block into two sub-regions, generates prediction samples by applying the general intra prediction method to the first sub-region, and generates prediction samples by applying the linear interpolation prediction method to the second sub-region. can do.

As described above, in an embodiment, the first sub-region is the prediction direction of the intra prediction mode among reference samples (ie, left, upper, upper-left, lower-left, and upper-right reference samples) of the reconstructed region around the current block. It may include one sample line (or sample array) adjacent to the reference sample (ie, the first reference sample) determined according to .

Also, as described above, the decoder may variably select which prediction method among the general intra prediction method and the linear interpolation prediction method is applied to generate the prediction block in the prediction block according to the distance from the reconstructed reference sample. In other words, the first sub-region may include a specific number of sample lines adjacent to a reference sample determined according to the prediction direction of the intra prediction mode among left, upper, upper-left, lower-left, and upper-right reference samples of the current block. .

As described above, the specific number may be determined based on at least one of the distance between the current sample and the first reference sample in the current block, the size of the current block, and the intra prediction mode.

The decoder generates a prediction sample of the first sub-region by using the first reference sample (S1405). That is, the decoder may generate prediction samples by applying the general intra prediction method described above with reference to FIGS. 5, 6, and 11 to the samples of the first sub-region.

The decoder generates a prediction sample of the second sub-region by using the first reference sample and the second reference sample (S1406). That is, the decoder may generate prediction samples by applying the linear interpolation prediction method described above with reference to FIGS. 7 to 10 and 12 to the samples of the second sub-region.

As described above, the decoder may generate a first prediction sample using the first reference sample and may generate a second prediction sample using the second reference sample. Then, the first prediction sample and the second prediction sample may be weighted summed (or interpolated or linearly interpolated) to generate a final prediction sample of the second sub-region. In this case, the weights respectively applied to the first prediction sample and the second prediction sample may be determined based on a ratio of the distance between the current sample and the first reference sample in the current block and the distance between the current sample and the second reference sample.

In addition, as described in Examples 2 and 3 above, the decoder may generate a final prediction sample by weighting the prediction sample generated through the general intra prediction method and the prediction sample generated through the linear interpolation prediction method.

15 is a diagram illustrating in more detail an intra prediction unit according to an embodiment of the present invention.

In FIG. 15 , the intra prediction unit is illustrated as one block for convenience of description, but the intra prediction unit may be implemented as a configuration included in an encoder and/or a decoder.

Referring to FIG. 15 , the intra prediction unit implements the functions, processes and/or methods previously proposed in FIGS. 7 to 14 . Specifically, the intra prediction unit includes a prediction mode derivation unit 1501 , a first reference sample derivation unit 1502 , a second reference sample derivation unit 1503 , a sub-region division unit 1504 , and a prediction block generation unit 1505 . can be

First, the prediction mode inducing unit 1501 induces an intra prediction mode of the current block.

The first reference sample derivation unit 1502 selects a first reference sample (or reference sample arrangement) from at least one reference sample among left, upper, upper-left, lower-left, and upper-right reference samples of the current block based on the intra prediction mode. induce

The second reference sample derivation unit 1503 derives a second reference sample from at least one of right, lower, and lower right reference samples of the current block based on the intra prediction mode. In this case, the second reference sample inducing unit 1503 generates a lower right reference sample adjacent to the lower right side of the current block as described above with reference to FIGS. 7 and 9 , and, as described with reference to FIGS. 7 and 10 , the lower right lower end portion as described above with reference to FIGS. 7 and 10 . The reference sample can be used to create either a right reference sample or a bottom reference sample.

The sub-region dividing unit 1504 divides the current block into a first sub-region and a second sub-region. As described above with reference to FIG. 13 , the sub-region dividing unit 1504 may divide the current block into sub-regions and apply another intra prediction method to the divided sub-regions. Specifically, the sub-region dividing unit 1504 divides the current block into two sub-regions, generates prediction samples by applying a general intra prediction method to the first sub-region, and applies a linear interpolation prediction method to the second sub-region. can be applied to generate prediction samples.

As described above, in an embodiment, the first sub-region is the prediction direction of the intra prediction mode among reference samples (ie, left, upper, upper-left, lower-left, and upper-right reference samples) of the reconstructed region around the current block. It may include one sample line (or sample array) adjacent to the reference sample (ie, the first reference sample) determined according to .

Also, as described above, the decoder may variably select which prediction method among the general intra prediction method and the linear interpolation prediction method is applied to generate the prediction block in the prediction block according to the distance from the reconstructed reference sample. In other words, the first sub-region may include a specific number of sample lines adjacent to a reference sample determined according to the prediction direction of the intra prediction mode among left, upper, upper-left, lower-left, and upper-right reference samples of the current block. .

As described above, the specific number may be determined based on at least one of the distance between the current sample and the first reference sample in the current block, the size of the current block, and the intra prediction mode.

The prediction block generator 1505 generates a prediction sample of the first sub-region by using the first reference sample. That is, the prediction block generator 1505 may generate a prediction sample by applying the general intra prediction method described above with reference to FIGS. 5, 6, and 11 to the samples of the first sub-region.

The prediction block generator 1505 generates a prediction sample of the second sub-region by using the first reference sample and the second reference sample. That is, the decoder may generate prediction samples by applying the linear interpolation prediction method described above with reference to FIGS. 7 to 10 and 12 to the samples of the second sub-region.

As described above, the decoder may generate a first prediction sample using the first reference sample and may generate a second prediction sample using the second reference sample. Then, the first prediction sample and the second prediction sample may be weighted summed (or interpolated or linearly interpolated) to generate a final prediction sample of the second sub-region. In this case, the weights respectively applied to the first prediction sample and the second prediction sample may be determined based on the ratio of the distance between the current sample and the first reference sample in the current block and the distance between the current sample and the second reference sample.

16 shows a structure diagram of a content streaming system as an embodiment to which the present invention is applied.

Referring to FIG. 16 , the content streaming system to which the present invention is applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

The encoding server generates a bitstream by compressing content input from multimedia input devices such as a smart phone, a camera, a camcorder, etc. into digital data, and transmits it to the streaming server. As another example, when multimedia input devices such as a smartphone, a camera, a camcorder, etc. directly generate a bitstream, the encoding server may be omitted.

The bitstream may be generated by an encoding method or a bitstream generation method to which the present invention is applied, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

The streaming server transmits multimedia data to the user device based on a user's request through the web server, and the web server serves as a medium informing the user of what kind of service is available. When a user requests a desired service from the web server, the web server transmits it to a streaming server, and the streaming server transmits multimedia data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server serves to control commands/responses between devices in the content streaming system.

The streaming server may receive content from a media repository and/or an encoding server. For example, when content is received from the encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may store the bitstream for a predetermined time.

Examples of the user device include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation system, a slate PC, Tablet PC (tablet PC), ultrabook (ultrabook), wearable device (eg, watch-type terminal (smartwatch), glass-type terminal (smart glass), HMD (head mounted display)), digital TV, desktop There may be a computer, digital signage, and the like.

Each server in the content streaming system may be operated as a distributed server, and in this case, data received from each server may be distributed and processed.

As described above, the embodiments described in the present invention may be implemented and performed on a processor, microprocessor, controller, or chip. For example, the functional units shown in each figure may be implemented and performed on a computer, a processor, a microprocessor, a controller, or a chip.

In addition, the decoder and encoder to which the present invention is applied include a multimedia broadcasting transceiver, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video conversation device, a real-time communication device such as a video communication device, a mobile streaming device, It may be included in a storage medium, a camcorder, a video on demand (VoD) service providing device, an over the top video (OTT) device, an Internet streaming service providing device, a three-dimensional (3D) video device, a videophone video device, and a medical video device, etc. and may be used to process a video signal or a data signal. For example, the OTT video (Over the top video) device may include a game console, a Blu-ray player, an Internet-connected TV, a home theater system, a smart phone, a tablet PC, a digital video recorder (DVR), and the like.

In addition, the processing method to which the present invention is applied may be produced in the form of a program executed by a computer, and may be stored in a computer-readable recording medium. Multimedia data having a data structure according to the present invention may also be stored in a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices and distributed storage devices in which computer-readable data is stored. The computer-readable recording medium is, for example, Blu-ray Disc (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical It may include a data storage device. In addition, the computer-readable recording medium includes a medium implemented in the form of a carrier wave (eg, transmission through the Internet). In addition, the bitstream generated by the encoding method may be stored in a computer-readable recording medium or transmitted through a wired/wireless communication network.

Also, an embodiment of the present invention may be implemented as a computer program product using program codes, and the program code may be executed in a computer according to an embodiment of the present invention. The program code may be stored on a carrier readable by a computer.

The embodiments described above are those in which elements and features of the present invention are combined in a predetermined form. Each component or feature should be considered optional unless explicitly stated otherwise. Each component or feature may be implemented in a form that is not combined with other components or features. It is also possible to configure embodiments of the present invention by combining some elements and/or features. The order of operations described in the embodiments of the present invention may be changed. Some features or features of one embodiment may be included in another embodiment, or may be replaced with corresponding features or features of another embodiment. It is obvious that claims that are not explicitly cited in the claims can be combined to form an embodiment or included as a new claim by amendment after filing.

Embodiments according to the present invention may be implemented by various means, for example, hardware, firmware, software, or a combination thereof. In the case of implementation by hardware, an embodiment of the present invention provides one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs ( field programmable gate arrays), a processor, a controller, a microcontroller, a microprocessor, and the like.

In the case of implementation by firmware or software, an embodiment of the present invention may be implemented in the form of a module, procedure, function, etc. that perform the functions or operations described above. The software code may be stored in the memory and driven by the processor. The memory may be located inside or outside the processor, and may transmit/receive data to and from the processor by various well-known means.

It is apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the essential characteristics of the present invention. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention.

Above, preferred embodiments of the present invention described above are disclosed for the purpose of illustration, and those skilled in the art will improve and change various other embodiments within the spirit and scope of the present invention disclosed in the appended claims below. , substitution or addition may be possible.

Claims (12)

In a method of processing an image based on an intra prediction mode,
deriving an intra prediction mode of the current block;
deriving a first reference sample from at least one of left, upper, upper-left, lower-left, and upper-right reference samples of the current block based on the intra prediction mode;
deriving a second reference sample from at least one of right, lower, and lower right reference samples of the current block based on the intra prediction mode;
dividing the current block into a first sub-region and a second sub-region when the intra prediction mode is a directional prediction mode;
generating a prediction sample of the first sub-region by using the first reference sample; and
and generating a prediction sample of the second sub-region by using the first reference sample and the second reference sample. According to claim 1,
The first sub-region includes one sample line adjacent to a reference sample determined according to a prediction direction of the intra prediction mode among left, upper, upper-left, lower-left, and upper-right reference samples of the current block. According to claim 1,
The first sub-region includes a specific number of sample lines adjacent to a reference sample determined according to a prediction direction of the intra prediction mode among left, upper, upper-left, lower-left, and upper-right reference samples of the current block. 4. The method of claim 3,
The specific number is determined based on at least one of a distance between a current sample in the current block and the first reference sample, a size of the current block, and the intra prediction mode. According to claim 1,
The generating of the prediction sample of the second sub-region comprises:
generating a first prediction sample by using the first reference sample and generating a second prediction sample by using the second reference sample; and
weighted summing the first prediction sample and the second prediction sample to generate a final prediction sample of the second sub-region. 6. The method of claim 5,
Weights respectively applied to the first prediction sample and the second prediction sample are determined based on a ratio of a distance between a current sample and the first reference sample in the current block and a distance between the current sample and the second reference sample Way. In an apparatus for processing an image based on an intra prediction mode,
a prediction mode inducing unit for inducing an intra prediction mode of the current block;
a first reference sample derivation unit for deriving a first reference sample from at least one of left, upper, upper left, lower left, and right reference samples of the current block based on the intra prediction mode;
a second reference sample derivation unit for deriving a second reference sample from at least one of right, lower, and lower right reference samples of the current block based on the intra prediction mode;
a sub-region dividing unit dividing the current block into a first sub-region and a second sub-region when the intra prediction mode is a directional prediction mode; and
and a prediction sample generator configured to generate a prediction sample of the first sub-region using the first reference sample, and generate a prediction sample of the second sub-region using the first reference sample and the second reference sample. device to do. 8. The method of claim 7,
The first sub-region includes one sample line adjacent to a reference sample determined according to a prediction direction of the intra prediction mode among left, upper, upper-left, lower-left, and upper-right reference samples of the current block. 8. The method of claim 7,
The first sub-region includes a specific number of sample lines adjacent to a reference sample determined according to a prediction direction of the intra prediction mode among left, upper, upper-left, lower-left, and upper-right reference samples of the current block. 10. The method of claim 9,
The specific number is determined based on at least one of a distance between a current sample and the first reference sample in the current block, a size of the current block, and the intra prediction mode. 8. The method of claim 7,
The prediction sample generation unit,
generating a first prediction sample by using the first reference sample and generating a second prediction sample by using the second reference sample;
An apparatus for generating a final prediction sample of the second sub-region by weighted summing the first prediction sample and the second prediction sample. 12. The method of claim 11,
Weights respectively applied to the first prediction sample and the second prediction sample are determined based on a ratio of a distance between a current sample and the first reference sample in the current block and a distance between the current sample and the second reference sample Device.

Images