KR102558803B1 - Method and apparatus for prediction of residual signal - Google Patents

Abstract

A method and apparatus for performing prediction on a residual signal are disclosed. An encoding method and an encoding apparatus generate a predicted residual signal by performing additional prediction on a residual signal of a current block using residual signals of neighboring blocks. A decoding method and a decoding apparatus use a residual signal of an already reconstructed neighboring block when generating a reconstructed block of a current block. In encoding and decoding, values of prediction samples used to generate a prediction block may be updated before generation of the prediction block. Encoding efficiency can be improved by predicting the residual signal and updating the prediction sample.

Description

Prediction method and apparatus for residual signal {METHOD AND APPARATUS FOR PREDICTION OF RESIDUAL SIGNAL}

The following embodiments relate to a video decoding method, a decoding apparatus, an encoding method, and an encoding apparatus, and more particularly, to a method and apparatus for predicting a residual signal.

Through the continuous development of the information communication industry, broadcasting services having high definition (HD) resolution have been spread worldwide. Through this diffusion, many users have become accustomed to high-resolution and high-definition images.

In order to satisfy users' demand for high image quality, many organizations are accelerating the development of next-generation video devices. Users' interest in not only High Definition TV (HDTV) and Full HD (FHD) TV, but also Ultra High Definition (UHD) TV, which has a resolution four times higher than that of FHD TV, has increased. In accordance with this increase in interest, image encoding/decoding technology for images with higher resolution and quality is required.

An image encoding/decoding device and method may use inter prediction technology, intra prediction technology, entropy encoding technology, etc. to perform encoding/decoding of high-resolution and high-quality images. The inter-prediction technique may be a technique of predicting a value of a pixel included in a current picture using a temporally previous picture and/or a temporally subsequent picture. Intra-prediction technology may be a technology of predicting a value of a pixel included in a current picture using pixel information in the current picture. The entropy coding technique may be a technique of allocating a short code to a symbol with a high frequency of occurrence and a long code to a symbol with a low frequency of occurrence.

In video encoding and decoding, prediction may mean generating a prediction signal similar to an original signal. Prediction can be largely classified into prediction referring to a spatially reconstructed image, prediction referring to a temporally reconstructed image, and prediction of other symbols.

Intra prediction can be a prediction technique that allows only spatial reference. The current block may be a block that is a current encoding target. Intra prediction may be a method of predicting a current block by referring to reference samples already reconstructed around the current block.

In intra prediction, neighboring reference samples may be predicted and reconstructed brightness values other than brightness values in the original image, and may be values before post-processing filtering is applied. Since the reference sample has been previously encoded and reconstructed, it can be used for prediction of the current block in an encoder and a decoder.

Conceptually, however, intra prediction can be effective only in a flat region having continuity with neighboring reference signals and in a region having constant directionality. For regions without such intra-image characteristics, intra prediction coding efficiency may be considerably lower than that of inter prediction. In particular, when encoding an image, a first picture may be coded only by intra prediction, and a picture may be coded only by intra prediction for random access and error robustness. Therefore, a method capable of improving the efficiency of intra prediction encoding is required.

An embodiment may provide a method and apparatus for updating a reference sample used for intra prediction to be as close as possible to a current block, which is an encoding target, in order to improve intra encoding efficiency.

An embodiment may provide a method and apparatus for predicting a residual signal of a current block using residual signals of neighboring blocks that have already been encoded or decoded.

In one aspect, a video encoding method includes generating a first residual signal of a current block based on a residual signal of a first neighboring block of the current block; and performing encoding of the current block using the first residual signal of the current block.

The video encoding method may further include generating a second residual signal of the current block.

The second residual signal may be a difference between the current block and a predicted block of the current block.

The first residual signal may be generated based on the second residual signal and the residual signal of the neighboring block.

The first residual signal may be generated based on a difference between the second residual signal and the residual signal of the first neighboring block.

The first residual signal may be a difference between the second residual signal and the residual signal of the first neighboring block.

The video encoding method may further include determining whether to perform residual signal prediction.

The generating may be performed when it is determined that the residual signal prediction is performed.

The video encoding method may further include encoding information indicating whether or not the residual signal prediction is performed.

The encoding method may further include encoding an identifier of the first neighboring block.

In another aspect, in a method of decoding an image, generating a prediction block of a current block; and generating a reconstructed block of the current block based on the prediction block, a residual signal of the current block, and a residual signal of a first neighboring block of the current block.

The reconstruction block may be generated based on the sum of the residual signal of the current block and the residual signal of the first neighboring block.

The reconstruction block may be the sum of the prediction block of the current block, the residual signal of the current block, and the residual signal of the first neighboring block.

The image decoding method may further include generating a residual signal of the current block.

The decoding method of the image may further include identifying the first neighboring block.

The first neighboring block may be identified by an identifier of the first neighboring block.

When the identifier of the first neighboring block does not exist, the selected block may be identified as the first neighboring block according to a predefined method.

The method of decoding the image may further include updating a value of a reference sample used to generate the prediction block.

In another aspect, a method of decoding an image includes determining a value of a reference sample based on a block adjacent to a current block; and generating a prediction block of the current block using the reference sample.

The value of the reference sample may be determined based on the gradient pattern of the neighboring block.

When the gradient pattern is symmetrical, the value of the reference sample may be determined based on gradient values of two lines forming symmetry of the gradient pattern.

The value of the reference sample may be determined based on a gradient value between two adjacent reference samples among a plurality of reference samples belonging to one row of the neighboring block.

In the determining step, the value of the reference sample may be changed from a value before update to a value after update.

The value before the update may be a value generated by predicting and restoring a block including the reference sample.

When the intra prediction mode of the current block is the horizontal prediction mode, the reference sample is a sample adjacent to the left of the current block, and the neighboring block is a block obtained by adding an upper adjacent block of the current block and an upper left adjacent block of the current block.

In another aspect, a method of encoding an image includes determining a value of a reference sample based on a block adjacent to a current block; and generating a prediction block of the current block using the reference sample.

In order to improve the efficiency of intra encoding, a method and apparatus for updating a reference sample used for intra prediction to be as close as possible to a current block to be encoded are provided.

A method and apparatus for predicting a residual signal of a current block using residual signals of neighboring blocks that have already been encoded or decoded are provided.

1 is a block diagram showing a configuration according to an embodiment of an encoding device to which the present invention is applied.
2 is a block diagram showing a configuration according to an embodiment of a decoding device to which the present invention is applied.
3 is a diagram schematically illustrating a division structure of an image when encoding and decoding an image.
4A to 7H are diagrams illustrating a form of a prediction unit (PU) that may be included in a coding unit (CU).
5 is a diagram illustrating a form of a transform unit (TU) that may be included in a coding unit (CU).
6 is a diagram for explaining an embodiment of an intra prediction process.
7 is a diagram for explaining an embodiment of an inter prediction process.
8 is a structural diagram of an encoding device according to an embodiment.
9A and 9B are flowcharts of an encoding method according to an exemplary embodiment.
10A is a flowchart of an encoding method according to an exemplary embodiment.
10B is a flowchart of a method of updating a reference sample according to an embodiment.
10C is a flowchart of a method for predicting a residual signal according to an exemplary embodiment.
10D is a flowchart of a method of encoding a current block according to an exemplary embodiment.
11 illustrates a current block and a reference sample according to an example.
12 illustrates a method of updating a reference sample by reflecting a gradient of a neighboring block in a horizontal direction according to an example.
13 illustrates a method of obtaining an inclination pattern according to an example.
14 illustrates a method of updating a reference sample by reflecting a gradient of a neighboring block in a vertical direction according to an example.
15A shows intra prediction with 33 respective modes according to an example.
15B shows intra prediction with 65 respective modes according to an example.
16 illustrates a region of an image according to an example.
17 illustrates a method of calculating a residual signal of a neighboring block according to an example.
18 illustrates a method of calculating a residual signal of a current block according to an example.
19 illustrates a residual signal prediction method according to an example.
20A illustrates a default residual signal according to an example.
20B shows a result of discrete cosine transformation on a default residual signal according to an example.
21A illustrates a proposed residual signal according to an example.
21B shows a result of discrete cosine transformation on the proposed residual signal according to an example.
22 illustrates locations of neighboring blocks according to an example.
23 is a structural diagram of a decryption device according to an embodiment.
24A and 24B are flowcharts of a decoding method according to an exemplary embodiment.
25A is a flowchart of a method for generating a residual signal according to an embodiment.
25B is a flowchart of a method of decoding a current block using a feast signal according to an exemplary embodiment.
25C is a flowchart of a prediction block generation method according to an embodiment.
25D is a flowchart of a recovery block generation method according to an embodiment.
26 is a structural diagram of an electronic device implementing an encoding device according to an embodiment.
27 is a structural diagram of an electronic device implementing a decryption device according to an embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS For detailed descriptions of exemplary embodiments described below, reference is made to the accompanying drawings, which illustrate specific embodiments by way of example. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that the various embodiments are different, but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in one embodiment in another embodiment without departing from the spirit and scope of the invention. Additionally, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the embodiment. Accordingly, the detailed description set forth below is not to be taken in a limiting sense, and the scope of the exemplary embodiments, if properly described, is limited only by the appended claims, along with all equivalents as claimed by those claims.

Like reference numbers in the drawings indicate the same or similar function throughout the various aspects. The shapes and sizes of elements in the drawings may be exaggerated for clarity.

When a component is referred to as being "connected" or "connected" to another component, the two components may be directly connected or connected to each other, but it should be understood that another component may exist in the middle of the two components. In addition, the description of "including" a specific configuration in exemplary embodiments does not exclude configurations other than the specific configuration above, and additional configurations may be included in the practice of exemplary embodiments or the scope of the technical idea of exemplary embodiments.

Terms such as first and second may be used to describe various elements, but the above elements should not be limited by the above terms. The above terms are used to distinguish and refer to one component from another. For example, without departing from the scope of rights, a first element may be termed a second element, and similarly, a second element may also be termed a first element.

In addition, the components appearing in the embodiments are shown independently to represent different characteristic functions, and each component does not mean that each component consists of only a separate hardware or one software unit. That is, each component is listed as each component for convenience of description. For example, at least two of the components may be combined into one component. Also, one component may be divided into a plurality of components. An integrated embodiment and a separate embodiment of each of these components are also included in the scope of rights unless departing from the essence.

In addition, some of the components may not be essential components for performing essential functions, but may be optional components for improving performance. Embodiments may be implemented by including only components essential to implement the essence of the embodiment, and structures excluding optional components, such as components used only for performance improvement, are also included in the scope of rights.

Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily practice the embodiments. In describing the embodiments, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present specification, the detailed description will be omitted.

Hereinafter, an image may mean one picture constituting a video, and may also indicate a video itself. For example, "encoding and/or decoding a video" may mean "encoding and/or decoding a video", and may also mean "encoding and/or decoding one of the images constituting a video".

First, terms used in the embodiments will be described.

Unit: In encoding and decoding of an image, a unit may be a region created by dividing one image. One image may be divided into a plurality of units. In encoding and decoding of an image, a predefined process may be performed for each unit. Depending on the function, units such as blocks, macro blocks, Coding Units (CUs), Prediction Units (PUs), and Transform Units (TUs) are used. One unit can be further divided into sub-units having a smaller size relative to the unit.

- Block division information may include information about the depth of a unit. Depth information may indicate the number and/or degree of division of a unit.

- One unit may be hierarchically divided into a plurality of sub-units while having depth information based on a tree structure. In other words, a unit and a sub-unit generated by division of the unit may correspond to a node and a child node of the node, respectively. Each divided sub-unit may have depth information. Since the depth information of a unit indicates the number and/or degree of division of the unit, the division information of the sub-unit may include information about the size of the sub-unit.

- In the tree structure, the highest node may correspond to the first non-split unit. The highest node may be referred to as a root node. Also, the highest node may have the smallest depth value. In this case, the highest node may have a depth of level 0.

- A node with a depth of level 1 may represent a unit created as the original unit is split once. A node with a depth of level 2 may represent a unit created as the original unit is split twice.

- A leaf node with a depth of level 3 may represent a unit created as the initial unit is split 3 times. A leaf node may be the lowest node and may have the maximum depth value.

- Block: A block can be an MxN array of samples. M and N may each be a positive integer. A block may refer to a two-dimensional array of samples. A sample can be a pixel or a pixel value.

Transform Unit: A transform unit may be a basic unit in residual signal encoding and/or residual signal decoding, such as transform, inverse transform, quantization, inverse quantization, transform coefficient encoding, and transform coefficient decoding. One transform unit may be divided into multiple transform units having smaller sizes.

Parameter Set: A parameter set may correspond to header information among structures in a bitstream. For example, the parameter set may include a sequence parameter set, a picture parameter set, an adaptation parameter set, and the like.

Rate-distortion optimization: An encoding device may use rate-distortion optimization to provide high encoding efficiency by using a combination of coding unit size, prediction mode, prediction unit size, motion information, and transformation unit size.

- The rate-distortion optimization method may calculate a rate-distortion cost of each combination in order to select an optimal combination among the above combinations. The rate-distortion cost can be calculated using Equation 1 below. In general, a combination that minimizes the rate-distortion cost can be selected as an optimal combination in the rate-distortion optimization method.

D can represent distortion. D may be a mean square error of difference values between original transform coefficients and reconstructed transform coefficients within a transform block.

R can represent a rate. R may represent a bit rate using related context information.

λ may represent a Lagrangian multiplier. R may include not only encoding parameter information such as a prediction mode, motion information, and a coded block flag, but also bits generated by encoding transform coefficients.

The encoding device performs processes such as inter prediction and/or intra prediction, transformation, quantization, entropy encoding, inverse quantization, and inverse transformation to calculate accurate D and R, and these processes can greatly increase complexity in the encoding device.

1 is a block diagram showing a configuration according to an embodiment of an encoding device to which the present invention is applied.

The encoding device 100 may be a video encoding device or an image encoding device. A video may include one or more images. The encoding apparatus 100 may sequentially encode one or more images of a video according to time.

1, the encoding apparatus 100 includes a motion estimation unit 111, a motion compensation unit 112, an intra prediction unit 120, a switch 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit ( 180) and a reference picture buffer 190.

The encoding apparatus 100 may perform encoding on an input image in an intra mode and/or an inter mode. Also, the encoding device 100 may generate a bitstream through encoding of an input device and output the generated bitstream. When the intra mode is used, the switch 115 can be switched to intra, and when the inter mode is used, the switch 115 can be switched to inter.

The encoding apparatus 100 may generate a prediction block for an input block of an input image. Also, the encoding apparatus 100 may encode a residual between the input block and the prediction block after the prediction block is generated. The input image may be referred to as a current image that is a current encoding target. An input block may be referred to as a current block that is a target of current encoding.

When the prediction mode is the intra mode, the intra predictor 120 may use pixel values of previously encoded blocks adjacent to the current block as reference pixels. The intra predictor 120 may perform spatial prediction using a reference pixel and generate prediction samples for an input block through spatial prediction.

When the prediction mode is the inter mode, the motion prediction unit 111 may search for a region that best matches the input block from the reference image in the motion prediction process, and derive a motion vector using the searched region. The reference picture may be stored in the reference picture buffer 190, and may be stored in the reference picture buffer 190 when encoding and/or decoding of the reference picture is processed.

The motion compensation unit 112 may generate a prediction block by performing motion compensation using a motion vector. Here, the motion vector may be a 2D vector used for inter prediction. Also, the motion vector may indicate an offset between the current image and the reference image.

The subtractor 125 may generate a residual block using a difference between the input block and the prediction block. A residual block may also be referred to as a residual signal.

The transform unit 130 may generate a transform coefficient by performing a transform on the residual block, and may output the transform coefficient. Here, the transform coefficient may be a coefficient value generated by performing transform on the residual block. When a transform skip mode is applied, the transform unit 130 may skip transform on the residual block.

A quantized transform coefficient level may be generated by applying quantization to the transform coefficient. Hereinafter, in the embodiments, a quantized transform coefficient level may also be referred to as a transform coefficient.

The quantization unit 140 may generate a quantized transform coefficient level by quantizing the transform coefficient according to a quantization parameter, and may output the quantized transform coefficient level. At this time, the quantization unit 140 may quantize the transform coefficient using a quantization matrix.

The entropy encoding unit 150 performs entropy encoding according to a probability distribution on the values calculated by the quantization unit 140 or the encoding parameter values calculated in the encoding process to generate a bitstream and output the bitstream.

The entropy encoding unit 150 may perform entropy encoding on information for decoding an image in addition to pixel information of the image. For example, information for decoding an image may include a syntax element and the like.

The encoding parameter may be information required for encoding and/or decoding. The encoding parameter may include information that is encoded by the encoding device and transmitted to the decoding device, and may include information that can be inferred during an encoding or decoding process. For example, as information transmitted to the decoding device, there is a syntax element.

For example, the coding parameters may include values or statistics such as prediction mode, motion vector, reference picture index, coded block pattern, presence/absence of a residual signal, transform coefficient, quantized transform coefficient, quantization parameter, block size, and block partition information. The prediction mode may indicate an intra prediction mode or an inter prediction mode.

The residual signal may mean a difference between the original signal and the prediction signal. Alternatively, the residual signal may be a signal generated by transforming a difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming and quantizing the difference between the original signal and the predicted signal. A residual block may be a residual signal in units of blocks.

When entropy encoding is applied, a small number of bits may be allocated to a symbol having a high probability of occurrence, and a large number of bits may be allocated to a symbol having a low probability of occurrence. As symbols are represented through such allocation, the size of bitstrings for symbols that are encoding targets can be reduced. Therefore, compression performance of image encoding can be improved through entropy encoding.

Also, encoding methods such as exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) may be used for entropy encoding. For example, the entropy encoding unit 150 may perform entropy encoding using a variable length coding/code (VLC) table. For example, the entropy encoding unit 150 may derive a binarization method for a target symbol. Also, the entropy encoder 150 may derive a probability model of the target symbol/bin. The entropy encoding unit 150 may perform entropy encoding using the derived binarization method or a probability model.

When the encoding apparatus 100 performs encoding through inter prediction, the encoded current image may be used as a reference image for other image(s) to be processed later. Accordingly, the encoding device 100 may decode the encoded current image again and store the decoded image as a reference image. Inverse quantization and inverse transformation of the current image encoded for decoding may be processed.

The quantized coefficient may be inversely quantized in the inverse quantization unit 160. It may be inversely transformed in the inverse transformation unit 170. The inverse quantized and inverse transformed coefficients may be combined with the prediction block through the adder 175. A reconstructed block may be generated by summing the inverse quantized and inverse transformed coefficients and the prediction block.

The reconstruction block may pass through the filter unit 180 . The filter unit 180 may apply at least one of a deblocking filter, a Sample Adaptive Offset (SAO), and an Adaptive Loop Filter (ALF) to a reconstructed block or a reconstructed picture. The filter unit 180 may also be referred to as an adaptive in-loop filter.

The deblocking filter may remove block distortion generated at a boundary between blocks. SAO may add an appropriate offset value to a pixel value to compensate for a coding error. ALF may perform filtering based on a value obtained by comparing the restored image and the original image. A reconstructed block that has passed through the filter unit 180 may be stored in the reference picture buffer 190 .

2 is a block diagram showing a configuration according to an embodiment of a decoding device to which the present invention is applied.

The decoding device 200 may be a video decoding device or an image decoding device.

Referring to FIG. 2 , the decoding apparatus 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra prediction unit 240, a motion compensation unit 250, an adder 255, a filter unit 260, and a reference picture buffer 270.

The decoding device 200 may receive the bitstream output from the encoding device 100. The decoding apparatus 200 may perform decoding on a bitstream in an intra mode or an inter mode. Also, the decoding apparatus 200 may generate a restored image through decoding and output the restored image.

When the prediction mode used for decoding is the intra mode, the switch may be switched to the intra mode. When the prediction mode used for decoding is the inter mode, the switch may be switched to inter mode.

The decoding apparatus 200 may obtain a reconstructed residual block from the input bitstream and generate a prediction block. When the reconstructed residual block and the prediction block are obtained, the decoding apparatus 200 may generate a reconstructed block by adding the reconstructed residual block and the prediction block.

The entropy decoding unit 210 may generate symbols by performing entropy decoding on a bitstream according to a probability distribution. The generated symbols may include symbols in the form of quantized coefficients. Here, the entropy decoding method may be similar to the above-described entropy encoding method. For example, the entropy decoding method may be a reverse process of the above-described entropy encoding method.

The quantized coefficient may be inversely quantized in the inverse quantization unit 220 and inversely transformed in the inverse transformation unit 230 . As a result of inverse quantization and inverse transformation of the quantized coefficients, a reconstructed residual block may be generated. At this time, the inverse quantization unit 220 may apply a quantization matrix to the quantized coefficients.

When the intra mode is used, the intra predictor 240 may generate a prediction block by performing spatial prediction using pixel values of previously encoded blocks adjacent to the current block. When the inter mode is used, the motion compensation unit 250 may generate a prediction block by performing motion compensation using a motion vector and a reference image stored in the reference picture buffer 270 .

The reconstructed residual block and the predicted block may be added through an adder 255. A block generated by adding the reconstructed residual block and the prediction block may pass through the filter unit 260 . The filter unit 260 may apply at least one of a deblocking filter, SAO, and ALF to a reconstructed block or a reconstructed picture. The filter unit 260 may output a reconstructed image. The reconstructed image may be stored in the reference picture buffer 270 and used for inter prediction.

3 is a diagram schematically illustrating a division structure of an image when encoding and decoding an image.

In order to efficiently segment an image, a coding unit (CU) may be used in encoding and decoding. A unit may be a term collectively referring to 1) a syntax element and 2) a block including image samples. For example, “division of a unit” may mean “division of a block corresponding to a unit”.

Referring to FIG. 3 , an image 300 is sequentially divided into units of largest coding units (LCUs), and a division structure is determined in units of LCUs. Here, LCU may be used as the same meaning as Coding Tree Unit (CTU).

The division structure may refer to a distribution of Coding Units (CUs) within the LCU 310 . A CU may be a unit for efficiently encoding an image. This distribution may be determined according to whether one CU is divided into 4 CUs. The horizontal size and vertical size of the CU generated by division may be half of the horizontal size and half of the vertical size of the CU before division, respectively. The divided CU may be recursively divided into four CUs whose horizontal and vertical sizes are reduced by half in the same way.

In this case, the division of the CU may be recursively performed up to a predefined depth. Depth information may be information indicating the size of a CU. Depth information may be stored for each CU. For example, the depth of the LCU may be 0, and the depth of the smallest coding unit (SCU) may be a predefined maximum depth. Here, the LCU may be a coding unit having the largest coding unit size as described above, and the SCU may be a coding unit having the smallest coding unit size.

The division starts from the LCU 310, and the depth of the CU increases by 1 each time the horizontal and vertical sizes of the CU are reduced by half. For each depth, a CU that is not split may have a size of 2Nx2N. In the case of a CU being split, a CU of 2Nx2N size may be split into 4 CUs having a size of NxN. The size of N is halved for each increase in depth by one.

Referring to FIG. 3 , an LCU having a depth of 0 may be 64x64 pixels. 0 may be the minimum depth. A SCU with a depth of 3 may be 8x8 pixels. 3 may be the maximum depth. In this case, a CU of 64x64 pixels, which is an LCU, may be expressed as a depth of 0. A CU of 32x32 pixels can be expressed as a depth of 1. A CU of 16x16 pixels can be expressed as depth 2. A CU of 8x8 pixels, which is an SCU, can be expressed as a depth of 3.

Also, information on whether the CU is split may be expressed through split information of the CU. The division information may be 1 bit of information. All CUs except for the SCU may include partition information. For example, if the value of the split information is 0, the CU may not be split, and if the value of the split information is 1, the CU may be split.

4A to 4H are diagrams illustrating a form of a prediction unit (PU) that may be included in a coding unit (CU).

Among the CUs divided from the LCU, a CU that is not further divided may be divided into one or more prediction units (PUs). This process may also be referred to as segmentation.

A PU may be a basic unit for prediction. A PU may be coded and decoded in any one of skip mode, inter mode, and intra mode. A PU may be divided into various types according to modes.

As shown in FIG. 4A , in skip mode, there may not be a split within a CU. In the skip mode, the 2Nx2N mode 410 having the same size as the CU without division may be supported.

In inter mode, 8 divided types can be supported within the CU. For example, in the inter mode, 2Nx2N mode 410, 2NxN mode 415, Nx2N mode 420, NxN mode 425, 2NxnU mode 430, 2NxnD mode 435, nLx2N mode 440 and nRx2N mode 445 may be supported.

In intra mode, 2Nx2N mode 410 and NxN mode 425 may be supported.

5 is a diagram illustrating a form of a transform unit (TU) that may be included in a coding unit (CU).

A transform unit (TU) may be a basic unit used for transform, quantization, inverse transform, and inverse quantization processes within a CU. A TU may have a square shape or a rectangular shape.

Among the CUs split from the LCU, a CU that is no longer split into CUs may be split into one or more TUs. In this case, the division structure of the TU may be a quad-tree structure. For example, as shown in FIG. 5 , one CU 510 may be divided one or more times according to a quadtree structure. Through division, one CU 510 may be composed of TUs of various sizes.

6 is a diagram for explaining an embodiment of an intra prediction process.

The number of intra prediction modes may be fixed to 35 regardless of the size of a prediction unit.

The prediction mode may include 2 non-directional modes and 33 directional modes as shown in FIG. 6 . The two non-directional modes may include DC mode and Planar mode.

The number of prediction modes may be different according to the type of color component. For example, the number of prediction modes may be different depending on whether the color component is a luma signal or a chroma signal.

A PU may have a square shape, with a size of NxN or a size of 2Nx2N. The size of NxN may include 4x4, 8x8, 16x16, 32x32, and 64x64. A unit of PU may be the size of at least one of CU, PU, and TU.

Intra encoding and/or decoding may be performed using sample values or encoding parameters included in neighboring reconstructed units.

7 is a diagram for explaining an embodiment of an inter prediction process.

The rectangle shown in FIG. 7 may represent an image (or picture). Also, arrows in FIG. 7 may indicate prediction directions. That is, an image may be encoded and/or decoded according to a prediction direction.

Each image (or picture) may be classified into an I-picture (Intra Picture), a P-picture (Uni-prediction Picture), and a B-picture (Bi-prediction Picture) according to an encoding type. Each picture may be coded according to the coding type of each picture.

If an image to be encoded is an I-picture, the image itself may be coded without inter prediction. When an image to be encoded is a P picture, the image may be encoded through inter prediction using a reference image only in the forward direction. If an image to be encoded is a B picture, it may be coded through inter prediction using reference pictures in both the forward and backward directions, and may be coded through inter prediction using reference pictures in one of the forward and backward directions.

Pictures of P pictures and B pictures that are encoded and/or decoded using reference pictures may be regarded as pictures using inter prediction.

Below, inter prediction according to an embodiment is described in detail.

Inter prediction may be performed using a reference picture and motion information. Also, inter prediction may use the above-described skip mode.

A reference picture may be at least one of a picture before the current picture or a picture after the current picture. In this case, inter prediction may perform prediction on a block of the current picture based on the reference picture. Here, the reference picture may mean an image used for prediction of a block.

In this case, the region within the reference picture may be specified by using a reference picture index (refIdx) indicating the reference picture and a motion vector to be described later.

Inter prediction may select a reference picture and a reference block corresponding to a current block within the reference picture, and generate a prediction block for the current block using the selected reference block. The current block may be a block currently being encoded or decoded among blocks of the current picture.

Motion information may be derived during inter prediction by each of the encoding apparatus 100 and the decoding apparatus 200 . Also, the derived motion information may be used to perform inter prediction.

In this case, the encoding apparatus 100 and the decoding apparatus 200 may use motion information of a reconstructed neighboring block and/or motion information of a collocated block (col block) to encode and/or decode. Efficiency can be improved. A collocated block may be a block corresponding to a current block in a previously reconstructed collocated picture (col picture). The reconstructed neighboring block may be a block in the current picture and a block already reconstructed through encoding and/or decoding. Also, the restoration block may be a block adjacent to the current block and/or a block located at an outer corner of the current block. Here, the block located at the outer corner of the current block may be a block vertically adjacent to a neighboring block horizontally adjacent to the current block or a block horizontally adjacent to a neighboring block vertically adjacent to the current block.

Each of the encoding apparatus 100 and the decoding apparatus 200 may determine a block spatially located at a position corresponding to the current block in a collocated picture, and may determine a predefined relative position with respect to the determined block. The predefined relative position may be a position inside and/or outside a block spatially existing at a position corresponding to the current block. Also, each of the encoding apparatus 100 and the decoding apparatus 200 may derive a collocated block based on the determined relative position. Here, the collocated picture may be one picture among at least one reference picture included in the reference picture list.

A method of deriving motion information may change according to a prediction mode of a current block. For example, as a prediction mode applied for inter prediction, there may be an Advanced Motion Vector Predictor (AMVP) and merge.

For example, when AMVP is applied as a prediction mode, each of the encoding apparatus 100 and the decoding apparatus 200 may generate a predictive motion vector candidate list using a motion vector of a reconstructed neighboring block and/or a motion vector of a collocated block. A motion vector of a reconstructed neighboring block and/or a motion vector of a collocated block may be used as a predicted motion vector candidate.

A bitstream generated by the encoding apparatus 100 may include a predictive motion vector index. The predicted motion vector index may indicate an optimal predicted motion vector selected from motion vector predicted candidates included in the predicted motion vector candidate list. The predictive motion vector index may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream.

The decoding apparatus 200 may use the predictive motion vector index to select a predictive motion vector of the current block from among the predictive motion vector candidates included in the predictive motion vector candidate list.

The encoding apparatus 100 may calculate a motion vector difference (MVD) between the motion vector of the current block and the predicted motion vector, and may encode the MVD. A bitstream may include an encoded MVD. The MVD may be transmitted from the encoding device 100 to the decoding device 200 through a bitstream. At this time, the decoding apparatus 200 may decode the received MVD. The decoding apparatus 200 may derive a motion vector of the current block through the sum of the decoded MVD and the predicted motion vector.

A bitstream may include a reference picture index indicating a reference picture. The reference picture index may be transmitted from the encoding device 100 to the decoding device 200 through a bitstream. The decoding apparatus 200 may predict a motion vector of the current block using motion information of neighboring blocks, and may derive a motion vector of the current block using the predicted motion vector and the motion vector difference. The decoding apparatus 200 may generate a prediction block for the current block based on the derived motion vector and reference picture index information.

Another example of a motion information derivation method is merge. Merge may refer to merging of motions of a plurality of blocks. Merge may mean applying motion information of one block to another block as well. When merge is applied, each of the encoding device 100 and the decoding device 200 may generate a merge candidate list using motion information of a reconstructed neighboring block and/or motion information of a collocated block. The motion information may include at least one of 1) a motion vector, 2) an index for a reference picture, and 3) a prediction direction. The prediction direction can be unidirectional or bidirectional.

In this case, merge may be applied in units of CUs or units of PUs. When merging is performed in units of CUs or PUs, the encoding apparatus 100 may transmit predefined information to the decoding apparatus 200 through a bitstream. A bitstream may include predefined information. The predefined information may include 1) information indicating whether merging is to be performed for each block partition, and 2) information about which block to perform merging with among neighboring blocks adjacent to the current block. For example, neighboring blocks of the current block may include a left neighboring block of the current block, an upper neighboring block of the current block, and a temporal neighboring block of the current block.

The merge candidate list may indicate a list in which motion information is stored. Also, the merge candidate list may be generated before merging is performed. Motion information stored in the merge candidate list may be 1) motion information of neighboring blocks adjacent to the current block or 2) motion information of blocks collocated with the current block in the reference image. Also, the motion information stored in the merge candidate list may be new motion information generated by a combination of motion information already existing in the merge candidate list.

The skip mode may be a mode in which information of neighboring blocks is applied to the current block as it is. A skip mode may be one of modes used for inter prediction. When the skip mode is used, the encoding apparatus 100 may transmit only information on which block's motion information is to be used as the current block's motion information to the decoding apparatus 200 through a bitstream. The encoding device 100 may not transmit other information to the decoding device 200. For example, the other information may be syntax information. Syntax information may include motion vector difference information.

In the following embodiments, residual signal prediction is described. In general, in conventional video encoding and/or decoding techniques such as High Efficiency Video Coding (HEVC) and Advanced Video Coding (AVC), a residual signal of a current block is generated to encode and/or decode the current block. When the residual signal is generated, a residual signal predicted once more may be generated through residual signal prediction using residual signals of neighboring blocks of the current block.

Residual signal prediction may be prediction of a residual signal of a current block with respect to residual signals of neighboring blocks. Alternatively, residual signal prediction may be using a difference between the residual signal of the current block and the residual signal of neighboring blocks as a new residual signal of the current block.

A residual signal obtained through residual signal prediction may have an advantage in coding efficiency compared to a residual signal obtained using an existing intra prediction method. For example, the amount of bits generated for the residual signal can be reduced by using the residual signal obtained using residual signal prediction.

In the following embodiments, a residual signal obtained using residual signal prediction may be referred to as a first residual signal, and a residual signal obtained using an existing intra prediction method may be referred to as a second residual signal.

Further, in the following embodiments, updating of reference samples is described. Reference samples are used to generate prediction blocks. Accordingly, if the reference sample has characteristics similar to expected characteristics of a current block to be encoded or decoded, encoding efficiency or decoding efficiency may be improved. Below, embodiments of updating a reference sample according to a predefined condition before generating a prediction block are described.

8 is a structural diagram of an encoding device according to an embodiment.

The encoding device 800 may correspond to the aforementioned encoding device 100. The encoding apparatus 800 includes a motion prediction unit 111, a motion compensation unit 112, an intra prediction unit 120, a switch 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference pick. The buffer 190 may be included, and an intra residual predictor 810 may be further included.

The motion prediction unit 111, the motion compensation unit 112, the intra prediction unit 120, the switch 115, the subtracter 125, the transform unit 130, the quantization unit 140, the entropy encoding unit 150, the inverse quantization unit 160, the inverse transform unit 170, the adder 175, the filter unit 180, and the reference picture buffer 190 are shown in FIG. Functions and/or operations as described above with reference to 1 may be performed. Redundant descriptions are omitted.

In addition, the motion prediction unit 111, the motion compensation unit 112, the intra prediction unit 120, the switch 115, the subtractor 125, the transform unit 130, the quantization unit 140, the entropy encoding unit 150, the inverse quantization unit 160, the inverse transform unit 170, the adder 175, the filter unit 180, and the reference picture buffer 190 may perform functions and/or operations related to the intra residual predictor 810. motion compensation unit 112, intra prediction unit 120, switch 115, subtractor 125, transform unit 130, quantization unit 140, entropy encoding unit 150, inverse quantization unit 160, inverse transform unit 170, adder 175, filter unit 180, reference picture buffer 190, and intra residual prediction unit 810 The function and / or operation of ) is described in detail below.

The intra residual predictor 810 may not be distinguished from the intra predictor 120 . The intra predictor 120 and the intra residual predictor 810 may be integrated into the intra predictor 120 . In embodiments, functions and/or operations described as performed by the intra residual predictor 810 may be performed by the intra predictor 120 .

In addition, the decoding apparatus 2300 corresponding to the encoding apparatus 800 according to an embodiment will be described in detail with reference to FIG. 23 below.

9A and 9B are flowcharts of an encoding method according to an exemplary embodiment.

Hereinafter, the current block may be a block currently being encoded or a block within the current image.

First, referring to FIG. 9A , step 910 may be performed.

Prior to performing step 910, a reference sample may be created. Generation of a reference sample according to an example will be described in detail below with reference to FIG. 11 .

In generation of reference samples, reference sample generation methods of existing video encoding and/or decoding technologies, such as high-efficiency video coding and advanced video coding, may be used.

In step 910, the intra predictor 120 may determine whether to update the reference sample. Here, the update of the reference sample may be to refine a sample value of the reference sample used for generating the prediction block before generating the prediction block of the current block.

If it is determined to perform an update of the reference sample, step 915 may be performed. If it is determined not to perform a reference sample update, steps 920 and 970 may be performed.

In step 915, the intra prediction unit 120 may update the value of the reference sample and determine the value of the reference sample used to generate the prediction block of the current block through the update.

The intra predictor 120 may update values of reference samples according to directional patterns of neighboring samples.

Updating a reference sample according to an example will be described in detail with reference to FIGS. 12, 13 and 14 below.

After step 915 is executed, steps 920 and 970 may be performed.

In step 920, the intra prediction unit 120 may perform intra prediction. The intra prediction unit 120 may generate a prediction block of the current block. The intra prediction unit 120 may generate a prediction block of the current block by using the reference sample according to the intra prediction mode of the current block.

For example, in generating a prediction block, a prediction block generation method of an existing video encoding and/or decoding technology such as HEVC and AVC may be used.

After step 920 is performed, step 930 may be performed.

In operation 930, the intra residual predictor 810 may determine whether to perform residual signal prediction.

If it is determined to perform residual signal prediction, steps 935, 940 and 980 may be performed.

If it is determined not to perform residual signal prediction, steps 960 and 980 may be performed.

In operation 935, the intra residual predictor 810 may determine a first neighboring block among one or more neighboring blocks of the current block. The first neighboring block may be a block used for residual signal prediction. Determination of a neighboring block according to an example will be described in detail below with reference to FIG. 20 .

After step 935 is performed, steps 950 and 985 may be performed.

In operation 940, the intra prediction unit 120 may generate a second residual signal of the current block by using intra prediction. The intra predictor 120 may generate a second residual signal of the current block based on the intra prediction mode and the reference sample of the current block.

The second residual signal may correspond to a residual signal of a current block in existing video encoding and/or decoding technologies such as HEVC and AVC. For example, in generating the second residual signal, a residual signal generation method of an existing video encoding and/or decoding technology such as HEVC and AVC may be used.

Generation of the second residual signal according to an example will be described in detail with reference to FIGS. 16, 17, and 18 below.

After step 940 is performed, step 950 may be performed.

If steps 935 and 940 are performed according to the decision to perform residual signal prediction, step 950 may be performed next.

In operation 950, the intra residual predictor 810 may generate a first residual signal of the current block according to the residual signal prediction.

The intra residual predictor 810 may use the residual signal of the first neighboring block to predict the residual signal.

The intra residual predictor 810 may generate a first residual signal of the current block based on the second residual signal of the current block and the residual signal of the first neighboring block.

The first residual signal may be a difference between the second residual signal of the current block and the residual signal of the first neighboring block. Alternatively, the first residual signal may be a result of subtracting the residual signal of the first neighboring block from the second residual signal of the current block. The intra residual predictor 810 may generate a difference between the second residual signal of the current block and the residual signal of the first neighboring block as the first residual signal.

The first residual signal may be a residual signal of the current block generated by residual signal prediction. Alternatively, the first residual signal may be a residual signal of the current block generated based on residual signals of first neighboring blocks of the current block.

By applying the residual signal prediction to the second residual signal generated in step 940, more efficient encoding of the current block may be possible.

Generation of the first residual signal according to an example will be described in detail below with reference to FIG. 19 .

If it is determined that residual signal prediction is not to be performed, in operation 960, the intra predictor 120 or the intra residual predictor 810 may generate a third residual signal of the current block.

The third residual signal may be a residual signal of a current block in existing video encoding and/or decoding technologies such as HEVC and AVC. For example, in generating the third residual signal, a residual signal generation method of an existing video encoding and/or decoding technology such as HEVC and AVC may be used.

Also, the third residual signal may be the same signal as the second residual signal in step 940 . In other words, steps 940 and 960 may generate the residual signal of the current block in the same way.

In step 970, the intra predictor 120 may encode information indicating whether updating of reference samples is performed.

Unlike conventional intra prediction, when the reference sample is updated, the intra predictor 120 can encode the above information so that the decoding apparatus 2300, which will be described later with reference to FIG. 23, knows whether or not the reference sample is updated. For example, if the update of the reference sample is used, the value of the above information may be set to '1', and if the update of the reference sample is not used, the value of the above information may be set to '0'.

For example, the intra predictor 120 may indicate whether the reference sample is updated using a flag, and whether the reference sample is updated may be indicated through the flag.

In operation 980, the intra residual predictor 810 may encode information indicating whether or not residual signal prediction is performed.

When residual signal prediction is performed, the intra residual predictor 810 may encode the above information so that the decoding apparatus 2300, which will be described later, can know whether residual signal prediction is performed or not with reference to FIG. 23 . For example, if the update of the reference sample is used, the value of the above information may be set to '1', and if the update of the reference sample is not used, the value of the above information may be set to '0'.

For example, the intra predictor 120 may indicate whether the reference sample is updated by using the "intra_residual_prediciton_flag" flag, and may indicate whether the reference sample is updated through the flag.

In operation 985, the intra residual predictor 810 may encode an identifier of a first neighboring block.

The identifier of the first neighboring block may be information enabling the identification of the neighboring block used to predict the residual signal of the current block.

For example, the identifier of the first neighboring block may indicate a neighboring block used to predict a residual signal of the current block among a plurality of neighboring blocks. Alternatively, the identifier of the first neighboring block may be location information indicating a location of a neighboring block used for predicting a residual signal of a current block among a plurality of neighboring blocks. The position may indicate a relative position of the selected neighboring block to the current block. The location may indicate a direction in which the selected neighboring block is adjacent to the current block.

The identifier of the first neighboring block may be configured to indicate the same block in both the encoding device 800 and the decoding device 2300. For example, in relation to the identifier of the first neighboring block, the block size N and the location of the neighboring blocks may be common to the encoding apparatus 800 and the decoding apparatus 2300. In order for the encoding apparatus 800 and the decoding apparatus 2300 to share a common configuration with respect to the identifier of the first neighboring block, the identifier of the first neighboring block may be encoded in a “(neighboring-residual_index) truncated unary ((neighboring-residual_idx) Truncated unary)” method.

When electric vehicle signal prediction is performed, step 990 may be performed after steps 950, 970, 980, and 985 are performed. Also, when residual signal prediction is not performed, step 990 may be performed after steps 960 , 970 , 980 , and 985 are performed.

Next, refer to FIG. 9B.

In operation 990, the encoding apparatus 800 may perform encoding of the current block by using the residual signal of the current block. Step 990 may be performed by at least one of the transform unit 130, the quantization unit 140, and the entropy encoding unit 150.

In step 990, the residual signal used for encoding of the current block may be one of two residual signals.

If it is determined in step 930 to perform residual signal prediction, the first residual signal of the current block generated in step 950 may be a residual signal used in step 990 . In other words, the residual signal generated by residual signal prediction may be used for encoding the current block, and the encoding apparatus 800 may perform encoding of the current block using the first residual signal of the current block.

When it is determined not to perform residual signal prediction in step 930 , the third residual signal of the current block generated in step 960 may be a residual signal used in step 990 .

Step 990 may include step 991 , step 992 and step 993 .

In step 991, the transform unit 130 may generate transform coefficients by performing transform on the residual signal.

In step 992, the quantization unit 140 may generate a quantized transform coefficient level using the transform coefficient.

In step 993, the entropy encoding unit 150 may perform entropy encoding on the transform coefficient level.

Among the steps related to encoding of information described above, steps 970, 980, and 985 may be performed by a different entity and in a different order from those described above. For example, step 993 may include step 970, step 980 and step 985. Also, the entro unit encoder 150 may encode at least one of information indicating whether reference sample update is performed, information indicating whether residual signal prediction is performed, and an identifier of the first neighboring block.

10A is a flowchart of an encoding method according to an exemplary embodiment.

10B is a flowchart of a method of updating a reference sample according to an embodiment.

10C is a flowchart of a method for predicting a residual signal according to an exemplary embodiment.

10D is a flowchart of a method of encoding a current block according to an exemplary embodiment.

Compared to the embodiment described above with reference to FIG. 9 , in the embodiment described with reference to FIGS. 10A , 10B and 10C , updating of a reference sample and prediction of a residual signal can be performed separately.

Hereinafter, the current block may be a block currently being encoded or a block within the current image.

First, reference is made to FIG. 10A.

In step 1010, the intra predictor 120 may generate a reference sample of the current block. The intra prediction unit 120 may generate reference samples around the current block for intra prediction. Generation of a reference sample according to an example will be described in detail below with reference to FIG. 11 .

In generation of reference samples, reference sample generation methods of existing video encoding and/or decoding technologies, such as high-efficiency video coding and advanced video coding, may be used.

The encoding device 800 may selectively provide a function of updating reference samples. When the function of updating the reference sample is used, step 1020 may be performed following step 1010. When the function of updating the reference sample is not used, step 1030 may be performed following step 1010. That is to say, in an embodiment, functions of updating reference samples may be selectively combined.

In step 1020, the intra prediction unit 120 may provide a function related to updating a reference sample.

Next, refer to FIG. 10B.

Referring to FIG. 10B , step 1020 may include step 1021, step 1022, and step 1023.

In step 1021, the intra predictor 120 may determine whether to update the reference sample. Here, the update of the reference sample may be reconstructing a sample value of the reference sample used for generating the prediction block before generating the prediction block of the current block.

If it is determined to perform an update of the reference sample, step 1022 may be performed. If it is determined not to perform a reference sample update, step 1023 may be performed.

In step 1022, the intra prediction unit 120 may update the value of the reference sample and determine the value of the reference sample used to generate the prediction block of the current block through the update.

The intra predictor 120 may update values of reference samples according to directional patterns of neighboring samples.

Updating a reference sample according to an example will be described in detail with reference to FIGS. 12, 13 and 14 below.

After step 1022 is performed, step 1023 may be performed.

In step 1023, the intra prediction unit 120 may encode information indicating whether updating of reference samples is performed.

Unlike conventional intra prediction, when the reference sample is updated, the intra predictor 120 can encode the above information so that the decoding apparatus 2300, which will be described later with reference to FIG. 23, knows whether or not the reference sample is updated. For example, if the update of the reference sample is used, the value of the above information may be set to '1', and if the update of the reference sample is not used, the value of the above information may be set to '0'.

For example, the intra predictor 120 may indicate whether the reference sample is updated using a flag, and whether the reference sample is updated may be indicated through the flag.

When step 1010 or step 1020 is performed, step 1030 may be performed.

Again refer to FIG. 10A.

In step 1030, the intra prediction unit 120 may perform intra prediction. The intra prediction unit 120 may generate a prediction block of the current block. The intra prediction unit 120 may generate a prediction block of the current block by using the reference sample according to the intra prediction mode of the current block.

For example, in generating a prediction block, a prediction block generation method of an existing video encoding and/or decoding technology such as HEVC and AVC may be used.

The encoding device 800 may selectively provide a function of predicting a feast signal. When the function of predicting the feast signal is used, step 1040 may be performed following step 1030. If the function of predicting the feast signal is not used, step 1050 may be performed following step 1030. That is to say, in an embodiment the function of predicting the feast signal may be selectively combined.

When the residual signal of the current block is obtained through intra prediction, the intra residual predictor 810 may perform prediction on the residual signal of the current block using residual signals of neighboring blocks. Since updates are made to the residual signal once the residual signal is generated, a "prediction" on the residual signal may be considered a "re-prediction" on the residual signal.

In step 1040, the intra residual predictor 810 may provide a function related to prediction of a banquet signal.

Referring to FIG. 10C , step 1040 may include step 1041, step 1042, step 1046, and step 1047.

In step 1041, the intra residual predictor 810 may determine whether to perform residual signal prediction.

If it is determined to perform residual signal prediction, step 1042 may be performed.

If it is determined not to perform residual signal prediction, step 1046 may be performed.

In step 1042, the intra residual predictor 810 may predict the residual signal.

Step 1042 may include step 1043 , step 1044 and step 1045 .

In step 1042, steps 1043 and 1044 may be performed. Steps 1043 and 1044 may be performed in a predefined order. For example, step 1043 may be performed before step 1044. Alternatively, step 1044 may be performed before step 1043.

In step 1043, the intra residual predictor 810 may determine a first neighboring block among one or more neighboring blocks of the current block. The first neighboring block may be a block used for residual signal prediction. Determination of a neighboring block according to an example will be described in detail below with reference to FIG. 22 .

In step 1044, the intra prediction unit 120 may generate a second residual signal of the current block by using intra prediction. The intra predictor 120 may generate a second residual signal of the current block based on the intra prediction mode and the reference sample of the current block.

The second residual signal may correspond to a residual signal of a current block in existing video encoding and/or decoding technologies such as HEVC and AVC. For example, in generating the second residual signal, a residual signal generation method of an existing video encoding and/or decoding technology such as HEVC and AVC may be used.

Generation of the second residual signal according to an example will be described in detail with reference to FIGS. 16, 17, and 18 below.

When steps 1043 and 1044 are performed as it is determined that residual signal prediction is to be performed, step 1045 may be performed next.

In operation 1045, the intra residual predictor 810 may generate a first residual signal of the current block according to the residual signal prediction.

The intra residual predictor 810 may use the residual signal of the first neighboring block to predict the residual signal.

The intra residual predictor 810 may generate a first residual signal of the current block based on the second residual signal of the current block and the residual signal of the first neighboring block.

The first residual signal may be a difference between the second residual signal of the current block and the residual signal of the first neighboring block. Alternatively, the first residual signal may be a result of subtracting the residual signal of the first neighboring block from the second residual signal of the current block. The intra residual predictor 810 may generate a difference between the second residual signal of the current block and the residual signal of the first neighboring block as the first residual signal.

The first residual signal may be a residual signal of the current block generated by residual signal prediction. Alternatively, the first residual signal may be a residual signal of the current block generated based on residual signals of first neighboring blocks of the current block.

By applying the residual signal prediction to the second residual signal generated in step 1044, more efficient encoding of the current block may be possible.

Generation of the first residual signal according to an example will be described in detail below with reference to FIG. 19 .

If it is determined that residual signal prediction is not to be performed, in step 1046, the intra predictor 120 or the intra residual predictor 810 may generate a third residual signal of the current block.

The third residual signal may be a residual signal of a current block in existing video encoding and/or decoding technologies such as HEVC and AVC. For example, in generating the third residual signal, a residual signal generation method of an existing video encoding and/or decoding technology such as HEVC and AVC may be used.

Also, the third residual signal may be the same signal as the second residual signal in step 1044. In other words, steps 1044 and 1046 may generate the residual signal of the current block in the same way.

If step 1042 or step 1046 is performed, step 1047 may be performed next.

In step 1047, the intra residual predictor 810 may encode information related to residual signal prediction.

When residual signal prediction is performed, the intra residual predictor 810 may encode the above information so that the decoding apparatus 2300, which will be described later, can know whether residual signal prediction is performed or not with reference to FIG. 21 . For example, if the update of the reference sample is used, the value of the above information may be set to '1', and if the update of the reference sample is not used, the value of the above information may be set to '0'.

For example, the intra predictor 120 may indicate whether the reference sample is updated by using the "intra_residual_prediciton_flag" flag, and may indicate whether the reference sample is updated through the flag.

Next, refer to FIG. 10C.

Step 1047 may include steps 1048 and 1049 .

In operation 1048, the intra residual predictor 810 may encode an identifier of a first neighboring block.

The identifier of the first neighboring block may be information enabling the identification of the neighboring block used to predict the residual signal of the current block.

For example, the identifier of the first neighboring block may indicate a neighboring block used to predict a residual signal of the current block among a plurality of neighboring blocks. Alternatively, the identifier of the first neighboring block may be location information indicating a location of a neighboring block used for predicting a residual signal of a current block among a plurality of neighboring blocks. The position may indicate a relative position of the selected neighboring block to the current block. The location may indicate a direction in which the selected neighboring block is adjacent to the current block.

The identifier of the first neighboring block may be configured to indicate the same block in both the encoding device 800 and the decoding device 2300. For example, in relation to the identifier of the first neighboring block, the block size N and the location of the neighboring blocks may be common to the encoding apparatus 800 and the decoding apparatus 2300. In order for the encoding apparatus 800 and the decoding apparatus 2300 to share a common configuration with respect to the identifier of the first neighboring block, the identifier of the first neighboring block may be encoded in a “(neighboring-residual_index) truncated unary ((neighboring-residual_idx) Truncated unary)” method.

Step 1048 may be omitted. For example, if in step 1041 it is determined that residual signal prediction is not to be performed, then step 1048 may be omitted.

In step 1049, the intra residual predictor 810 may encode information indicating whether residual signal prediction is performed.

In step 1047, steps 1048 and 1049 may be performed. Steps 1048 and 1049 may be performed in a predefined order. For example, step 1048 may be performed before step 1049. Alternatively, step 1049 may be performed before step 1048.

If step 1030 or step 1040 is performed, step 1050 may be performed next.

In operation 1050, the encoding apparatus 800 may perform encoding of the current block by using the residual signal of the current block. Step 1050 may be performed by at least one of the transform unit 130, the quantization unit 140, and the entropy encoding unit 150.

The residual signal used for encoding the current block in step 1050 may be one of two residual signals.

When it is determined to perform residual signal prediction in step 1041 , the first residual signal of the current block generated in step 950 may be a residual signal used in step 1050 . In other words, the residual signal generated by residual signal prediction may be used for encoding the current block, and the encoding apparatus 800 may perform encoding of the current block using the first residual signal of the current block.

When it is determined not to perform residual signal prediction in step 1050 , the third residual signal of the current block generated in step 1046 may be a residual signal used in step 1050 .

Next, refer to FIG. 10D.

Step 1050 may include step 1051 , step 1052 and step 1053 .

In step 1051, the transform unit 130 may generate transform coefficients by performing transform on the residual signal.

In step 1052, the quantization unit 140 may generate a quantized transform coefficient level using the transform coefficient.

In step 1053, the entropy encoding unit 150 may perform entropy encoding on the transform coefficient level.

Among the steps related to encoding of information described above, steps 1023, 1048, and 1049 may be performed by a different entity and in a different order from those described above. For example, step 1053 may include step 1023, step 1048 and step 1049. Also, the entro unit encoder 150 may encode at least one of information indicating whether reference sample update is performed, information indicating whether residual signal prediction is performed, and an identifier of the first neighboring block.

Proposed Encoding Methods

In the embodiment described above with reference to FIG. 10, the encoding method can be classified into two methods. Each method is as follows.

- First method: The first method may be a method of generating a reference sample for intra prediction using an existing method and reconstructing sample values of the reference sample generated using the existing method according to a directional pattern of neighboring samples. Here, the conventional method may refer to a method of generating a reference sample for intra prediction performed by the intra predictor 120 described above with reference to FIG. 1 .

- Second method: The second method may be a method of reducing energy of a residual signal by performing re-prediction using a residual signal of a neighboring block on a residual signal obtained through existing intra prediction. That is to say, in the twice-method, re-prediction at the residual signal level can be performed. Here, the conventional method may include a process of generating a prediction signal of the current block through spatial prediction of a reference sample, which is performed by the intra prediction unit 120 described above with reference to FIG. 1, and a process of obtaining a residual signal of the current block using the prediction signal.

Classification of Examples

The embodiments described above with reference to FIGS. 10A, 10B, 10C, and 10D can be classified into three embodiments. Each embodiment is as follows.

- First Embodiment: The first embodiment is indicated by a filled arrow in FIG. 10A. The first embodiment may include steps 1010, 1020, 1030 and 1050. The first embodiment may perform update of the reference sample, but may not perform prediction of the residual signal.

- Second Embodiment: The second embodiment is indicated by an empty arrow in Fig. 10A. The second embodiment may include steps 1010, 1020, 1030, 1040 and 1050. The second embodiment may perform update of a reference sample and prediction of a residual signal.

- Third Embodiment: The third embodiment is indicated by an arrow with a hatched line inside in FIG. 10A. The third embodiment may include steps 1010, 1030, 1040 and 1050. The third embodiment does not update the reference sample, but can predict the residual signal.

The encoding apparatus 800 may perform encoding of the current block using one of the first embodiment, the second embodiment, and the third embodiment. Alternatively, the encoding apparatus 800 may perform rate-distortion optimization for all three embodiments, and may select a method for deriving the minimum rate-distortion value among the three embodiments. For example, the encoding apparatus 800 may selectively use each of steps 1020 and 1040 to derive a minimum rate-distortion value.

Unit of update of reference sample

In step 910, the intra predictor 120 may determine whether or not to update reference samples for each predefined unit. The predefined unit may be at least one of 1) an entire image sequence (ie, video), 2) one image (ie, picture), 3) a slice, and 4) a coding unit.

For a predefined unit, reference sample update information may be used to indicate whether or not to update a reference sample. The reference sample update information may be information indicating whether the update of the reference sample is performed for a predefined unit. For example, when the value of the reference sample update information is the first value, it may indicate that the reference sample has been updated in encoding of the current block. When the value of the reference sample update information is the second value, it may indicate that the reference sample is not updated in encoding of the current block.

The encoding apparatus 800 may include encoded residual signal prediction information in a bitstream. The decoding apparatus 2300 may determine whether to perform residual signal prediction on the current block using residual signal prediction information.

Below, the update of the reference sample for each of the predefined units is described.

1) Entire video sequence: Whether to update reference samples may be determined for the entire video sequence. In this case, a sequence parameter set may include reference sample update information. If the reference sample update information of the sequence parameter set indicates that the reference sample is updated, the intra predictor 120 may perform intra prediction using a reference sample to which a directional gradient is applied to the entire video sequence.

2) One image: Whether to update the reference sample may be determined for each image. In this case, a picture parameter set may include reference sample update information. If the reference sample update information of the picture parameter set indicates that the reference sample is updated, the intra prediction unit 120 uses a reference sample to which a gradient according to a direction is applied to the entire image corresponding to the picture parameter set. Can perform intra prediction. Alternatively, the intra predictor 120 may identify whether updating of the reference sample is performed using a picture parameter set identifier (ID) specified in the slice header.

3) Slice: One image can be divided into a plurality of slice segments or a plurality of tiles within one slice segment. Whether to update the reference sample may be determined for each slice. In this case, the slice segment header may include reference sample update information. If the reference sample update information of the slice segment header indicates that the reference sample is updated, the intra predictor 120 may perform intra prediction using a reference sample to which a gradient according to a direction is applied to a slice corresponding to the slice segment header.

4) Coding Unit: Whether to update a reference sample may be determined for each coding unit. In this case, reference sample update information may exist for the coding unit. If the reference sample update information for the coding unit indicates that the coding unit is updated, the intra prediction unit 120 uses a reference sample to which a directional gradient is applied for the coding unit corresponding to the reference sample update information. Can perform intra prediction.

As described above, the reference sample update information can be coded within a sequence parameter set, picture parameter set or slice segment header. Also, update information may be coded for a coding unit.

Units of Residual Signal Prediction

Whether to perform the residual signal prediction determined in step 930 may be determined for a predefined unit. The predefined unit may be at least one of 1) an entire image sequence (ie, video), 2) one image (ie, picture), 3) a slice, and 4) a coding unit.

The residual signal prediction information may be information indicating whether residual signal prediction is performed for a predefined unit. For example, when the value of the residual signal prediction information is the first value, it may indicate that residual signal prediction is performed in encoding of the current block. When the value of the residual signal prediction information is the second value, it may indicate that residual signal prediction is not performed in encoding of the current block.

The encoding apparatus 800 may include encoded residual signal prediction information in a bitstream. The decoding apparatus 2300 may determine whether to perform residual signal prediction on the current block using residual signal prediction information.

Below, residual signal prediction for each of the predefined units is described.

1) Entire video sequence: Whether to perform residual signal prediction may be determined for the entire video sequence. In this case, the sequence parameter set may include residual signal prediction information. If the residual signal prediction information of the sequence parameter set indicates that residual signal prediction is performed, the intra residual predictor 810 may perform residual signal prediction for the entire video sequence.

2) One image: Whether to perform residual signal prediction may be determined for each image. In this case, a picture parameter set may include residual signal prediction information. If the residual signal prediction information of the picture parameter set indicates that residual signal prediction is performed, the intra residual predictor 810 may perform residual signal prediction on all images corresponding to the picture parameter set.

3) Slice: One image can be divided into a plurality of slice segments or a plurality of tiles within one slice segment. Whether to perform residual signal prediction may be determined for each slice. In this case, the slice segment header may include residual signal prediction information. If the residual signal prediction information of the slice segment header indicates that residual signal prediction is performed, the intra residual predictor 810 may perform residual signal prediction on the slice corresponding to the slice segment header.

4) Coding Unit: Whether to perform residual signal prediction may be determined for each coding unit. In this case, residual signal prediction information may exist for the coding unit. If the residual signal prediction information for the coding unit indicates that residual signal prediction is to be performed, the intra residual predictor 810 may perform residual signal prediction for the coding unit corresponding to the residual signal prediction information.

As described above, the residual signal prediction information can be coded within a sequence parameter set, a picture parameter set or a slice segment header. Also, update information may be coded for a coding unit.

11 illustrates a current block and a reference sample according to an example.

The processing of the reference sample to be described below may be used to determine the value of the reference sample before updating the reference sample, and may be performed by the intra predictor 120 before step 910 described above with reference to FIG. 9A. Also, processing of a reference sample to be described below may correspond to step 1010 described above with reference to FIG. 10A.

In FIG. 11 , a current block 1100 , a top adjacent line 1110 , a left adjacent line 1120 , and a top left sample 1130 are shown.

The reference sample of the current block 1100 may include the top adjacent line 1110 , the left adjacent line 1120 , and the top left sample 1130 . Alternatively, the reference sample of the current block 1100 may be at least some of samples of the top adjacent line 1110 , the left adjacent line 1120 , and the top left sample 1130 . The intra predictor 120 may select at least some of the pixels of the top adjacent line 1110, the left adjacent line 1120, and the top left sample 1130 as reference samples according to the intra prediction mode of the current block.

The top adjacent line 1110 may be a horizontal line adjacent to the top of the current block 1100 . The left adjacent line 1120 may be a vertical line adjacent to the left side of the current block 1100 . The upper left sample 1130 may be a sample adjacent to the upper left corner of the current block 1100 .

The x-coordinate of the leftmost sample of the top adjacent line 1110 may be the same as the x-coordinate of the leftmost pixel(s) of the current block 1100 . When the size of the current block 1100 is NxN, the length of the upper adjacent line 1110 may be 2N. Here, N may be an integer of 1 or greater. The top adjacent line 1110 may be 2N×1 pixels.

The y coordinate of the topmost sample of the left adjacent line 1120 may be the same as the y coordinate of the topmost pixel(s) of the current block 1100 . When the size of the current block 1100 is NxN, the length of the left adjacent line 1120 may be 2N. The left adjacent line 1120 may be 1x2N of pixels.

The x coordinate of the upper left sample 1130 may be a value obtained by subtracting 1 from the x coordinate of the leftmost pixel(s) of the current block 1100 . The y coordinate of the top left sample 1130 may be a value obtained by subtracting 1 from the y coordinate of the top pixel(s) of the current block 1100 .

The aforementioned reference sample may be used in intra prediction for the current block.

A reference sample used for intra prediction may have a brightness value restored by prediction and restoration, not a brightness value of pixel(s) of the original image. For example, neighboring block(s) of the current block 1100 may be encoded prior to encoding of the current block 1100, and brightness values of pixels of the neighboring blocks may be restored through prediction and restoration in the encoding process. A reference sample may be some of the pixels of these neighboring blocks. Also, the brightness value of the reference sample may be a value before post-processing filtering is applied.

When there are no reference samples available around the current block 1100, the intra predictor 120 uses samples closest to the current block 1100 among available samples (i.e., pixels) in the vicinity. Padding can be performed. A brightness value of the reference sample may be generated by padding the reference sample.

The intra prediction unit 120 may perform reference sample filtering to reduce a prediction error due to a quantization error according to the size of the current block 1100 and the intra prediction mode.

12 illustrates a method of updating a reference sample by reflecting a gradient of a neighboring block in a horizontal direction according to an example.

In FIG. 12 , circles may represent samples (or pixels). A dotted line rectangle may represent a block.

In FIG. 12 , a current block 1210 , a reference sample block 1220 , reference samples 1221 , and neighboring blocks 1230 are illustrated.

The current block is shown as a block having a size of 4x4. The size of a block may mean a width and a height of the block.

The reference sample block 1220 may be a block including reference samples 1221 used for the current block 1210 . The reference sample block 1220 may be a block adjacent to the current block 1210 and having the same size as the current block 1210 .

The reference samples 1221 are illustrated as having values of I', J', K', and L', respectively, through updating of the reference sample. The illustrated reference samples may be samples configured according to a method for generating a reference sample among samples adjacent to the current block according to an intra prediction mode.

12 shows reference samples generated by the intra prediction unit 120 when the intra prediction mode for the current block 1210 is a horizontal prediction mode.

Dotted lines in the neighboring block 1230 may represent horizontal lines of samples in the neighboring block 1230 . Thick solid lines in the neighboring block 1230 may represent gradients of samples included in the horizontal line. ∇ dx can represent the slope value.

12 shows an example in which a sample value constantly increases and then decreases constantly from left to right in a horizontal line of samples.

The neighboring block 1230 may be a block for which reconstruction has already been completed prior to encoding and/or decoding of the current block 1210 .

A neighboring block used for updating the reference sample may be different from a neighboring block used for intra residual prediction of the current block. As described above with reference to FIGS. 9A and 10C , a neighboring block used for intra residual prediction of a current block may be referred to as a first neighboring block. Also, a neighboring block used for updating the reference sample may be referred to as a second block. The first neighboring block and the second neighboring block may be the same as or different from each other. Also, the first neighboring block may include the second neighboring block, or the second neighboring block may include the first neighboring block.

For example, when the intra prediction mode is the horizontal prediction mode, the reference sample may be a sample adjacent to the left of the current block 1210 . Alternatively, the reference sample may be a sample of a vertical line adjacent to the left side of the current block 1210 . The reference sample block 1220 may be a block adjacent to the left of the current block 1210 . In addition, the neighboring block 1230 may be a block obtained by adding a top adjacent block of the current block 1210 and a left top adjacent block of the current block 1210 . The top adjacent block of the current block 1210 may be a block adjacent to the top of the current block 1210 . The block adjacent to the top left of the current block 1210 may be a block adjacent to the top left of the current block 1210 . The top adjacent block and the upper left adjacent block may be adjacent to each other.

When the size of the current block 1210 is NxN, the size of the upper adjacent block and the upper left adjacent block may be NxN, respectively, and the size of the neighboring block 1230 may be 2NxN. In FIG. 12, it is shown that the size of the neighboring block is 8x4.

In addition, when the size of the current block 1210 is NxN, the size of the upper adjacent block and the upper left adjacent block may each be aNxbN, and the size of the neighboring block 1230 may be 2aNxbN. Here, a and b may each be a real number.

Also, each of the top adjacent block, the upper left adjacent block, and the neighboring block 1230 may have a predefined size or a size determined by a predefined method.

The intra prediction unit 120 of the encoding apparatus 800 and the intra prediction unit 240 of the decoding apparatus 2300 to be described later may use upper adjacent blocks, upper left adjacent blocks, and neighboring blocks of the same size. The encoding device 800 may set the size of each of the top adjacent block, the upper left adjacent block, and the neighboring block 1230 . The size of the top adjacent block, the size of the top left adjacent block, and the size of the neighboring block 1230 may also be used in the decoding apparatus 2300 as well. The set size may be transmitted from the encoding device 800 to the decoding device 2300 through a bitstream.

The update of the reference sample may be regarded as restoration of reference samples around the current block for intra prediction.

When intra prediction is performed on the second neighboring block, the current block may also be composed of a texture having a direction similar to that of the second neighboring block according to spatial correlation. To reflect this direction, the intra predictor 120 may update a value of a reference sample required for intra prediction before performing intra prediction on the current block.

The intra prediction unit 120 may reconstruct the reference sample to be similar to the sample of the current block by using a directional gradient pattern according to the directionality of the second neighboring block.

In step 915 described above with reference to FIG. 9A and step 1022 described above with reference to FIG. 10B , the intra predictor 120 may detect the gradient pattern of the second neighboring block and calculate the gradient. A gradient pattern detection method according to an example will be described in detail below with reference to FIG. 13 .

In step 915 described with reference to FIG. 9A and step 1022 described with reference to FIG. 10B , the intra predictor 120 may detect gradient patterns of rows of the second neighboring block. The intra predictor 120 may check whether gradient patterns of rows of the second neighboring block are the same. When the gradient patterns of the rows of the second neighboring block are the same, the intra prediction unit 120 may calculate the gradient of the second neighboring block. Here, the gradient of the second neighboring block may be the gradient of one selected row of the second neighboring block. For example, the gradient of the second neighboring block may be the gradient of a row adjacent to the current block 1210 among rows of the second neighboring block. The intra predictor 120 may determine a gradient of a row adjacent to the current block 1210 among rows of the second neighboring block as a final gradient of the second neighboring block.

In addition, in steps 915 and 1022, the intra predictor 120 may determine or update a value of a reference sample based on neighboring blocks of the current block. The intra predictor 120 may determine or update a value of a reference sample based on a gradient pattern of a second neighboring block.

The value of the reference sample may be changed from a value before update to a value after update by the intra predictor 120, and the value before update of the reference sample may be a value generated by predicting and restoring the reference sample block 1220 including the reference sample. In other words, while the reference sample block 1220 is predicted and reconstructed, a value of the reference sample may be determined, and the determined value may be used as a value before update in updating the reference sample. Prediction and reconstruction methods of existing video encoding and/or decoding technologies such as HEVC and AVC may be used to predict and restore reference samples.

When the gradient ∇ dx is calculated, the intra predictor 120 may determine or update the value of the reference sample in a predefined manner.

For example, when the intra prediction mode is the horizontal prediction mode, the intra prediction unit 120 may determine or update the value of the reference sample in the manner shown in Equation 2 below.

f can represent a function. w may represent a weight. I may be a value before updating of the topmost sample among the reference samples 1221 . I 'may be a value after updating of the topmost sample. J , K , and L may also be pre-update values of corresponding reference samples. J ', K ', and L ' may be values after update of corresponding reference samples.

As described in Equation 2, the intra prediction unit 120 reflects a subtraction according to a gradient with respect to existing reference samples I , J , K and L , thereby reconstructed reference samples I ', J ', K 'and L 'can be generated. Also, the intra predictor 120 may consider a weighting factor w when updating the value of the reference sample. The intra predictor 120 may update the value of the reference sample by subtracting the product of the value determined based on the gradient ∇ dx and the predefined weight factor w from the value of the reference sample before updating. Through updating, the value of the reference sample can be reduced by w * f (∇ dx ).

Reference samples may be plural. A weight factor w may be the same for a plurality of reference samples. Alternatively, the weight factor w may be different from each other for each of the plurality of reference samples. For example, the weight factor w may be different for each position of each reference sample of a plurality of reference samples. The position of the reference sample may be a position relative to the current block 1210 .

The intra prediction unit 120 may combine the aforementioned reference sample update method and the existing method. For example, when the intra prediction unit 120 performs updating of the reference sample described above with reference to FIG. 10A, in step 1010, padding of the reference pixel may not be performed. In addition, when the intra prediction unit 120 performs updating of the reference sample described above with reference to FIG. 10A, in step 1010, smoothing using a low pass filter may not be performed.

13 illustrates a method of obtaining an inclination pattern according to an example.

As described above with reference to FIG. 12 , the second neighboring block may be selected based on the position of the current block. For example, the intra predictor 120 may select, as a second neighboring block, a block at a predefined position according to an intra prediction mode among blocks reconstructed based on the position of the current block. Also, the intra predictor 120 may analyze the gradient pattern of the selected second neighboring block.

As described above with reference to FIG. 12 , the size of the second neighboring block may be determined based on the size of the current block. For example, if the size of the current block is NxN, the size of the second neighboring block may be 2N*N. The encoding device 800 may set the size of the second neighboring block. The set size may also be used in the decryption device 2300 as well. The set size may be transmitted from the encoding device 800 to the decoding device 2300 through a bitstream.

In FIG. 13, one line of the second peripheral block is shown. A line may be a row or column of neighboring blocks. In FIG. 12, the size of the second neighboring block is shown as 8x4, and in FIG. 13, one line of the second neighboring block is shown as 8x1.

The intra predictor 120 may calculate one or more sample gradients for each line. Each sample gradient of the one or more sample gradients may be a gradient between two adjacent samples of a line. Since one line may include a plurality of samples, the intra prediction unit 120 may obtain a plurality of sample gradients for the line. As an example of one or more sample gradients, in FIG. 13 , ∇ dx 1, ∇ dx 2, ∇ dx 3, ∇ dx 4, ∇ dx 5, ∇ dx 6, and ∇ dx 7 are shown. For example, ∇ dx 1 may be the slope between the first sample and the second sample of a line.

The intra predictor 120 may calculate gradients of one or more samples of each line of the second neighboring block.

The intra prediction unit 120 may calculate a line gradient based on one or more sample gradients. For example, the line gradient may be 1) a median value of one or more sample gradients, 2) an average value of one or more sample gradients, or 3) a predefined representative value of one or more sample gradients.

The predefined representative value can always be positive.

When using the median value of one or more sample gradients as the line gradient, Equation 3 below can be used.

∇ dx can represent the slope of the line.

In calculating the line gradient, the intra predictor 120 may apply a weight for each sample gradient to one or more sample gradients.

Through the above-described method, the intra predictor 120 may calculate one or more line gradients of one or more lines of the second neighboring block.

When the line gradient is calculated, the intra predictor 120 may calculate the gradient of the second neighboring block based on one or more line gradients. For example, the gradient of the second neighboring block may be 1) a median value of one or more line gradients, 2) an average value of one or more line gradients, or 3) a predefined representative value of one or more line gradients. Alternatively, the intra predictor 120 may apply a weight for each line gradient to one or more line gradients when calculating the gradient of the second neighboring block.

The calculated gradient of the second neighboring block may be used as a final gradient for updating the reference sample.

For example, as shown in FIG. 13, when sample gradients of a line increase and then decrease, the updated reference samples I', J', K', and L' may be obtained by subtracting the line gradient of the line from each of I, J, K, and L previously generated as reference samples 1221 of the current block 1210 of FIG. 12.

As described above for the sample gradient, the line gradient, and the gradient of the second neighboring block, the intra predictor 120 may determine a reference sample value based on a gradient between two adjacent reference samples among a plurality of reference samples belonging to one row of the second neighboring block.

Alternatively, the intra prediction unit 120 may use the line gradient of one line as the gradient of a reference sample corresponding to the line. For example, lines and reference samples having the same relative positions may correspond to each other. For example, the intra prediction unit 120 may use a line gradient of a topmost line among one or more lines to update a topmost reference sample among one or more reference samples. Alternatively, the intra predictor 120 may use a line gradient of a line adjacent to the current block 1210 among one or more lines to update the reference sample.

In calculating the gradient of the second neighboring block, the intra predictor 120 may select only some line(s) from among all lines of the second neighboring block. The intra predictor 120 may calculate the gradient of the second neighboring block using the line gradient(s) of the selected partial line(s). The intra predictor 120 may select line(s) at a predefined position(s) or a predefined number of line(s) from among all lines of the second neighboring block. The intra predictor 120 may set a method of selecting only some line(s) from among all lines of the second neighboring block. The set method may be transmitted from the encoding device 800 to the decoding device 2300 through a bitstream.

In calculating the line gradient of the line, the intra predictor 120 may select only some samples from among all samples of the line. That is to say, the slope of one or more samples of each line need not necessarily be calculated for all samples of each line. The intra prediction unit 120 may calculate a line gradient for each of the left and right symmetrical regions based on the dotted line in the middle of FIG. 12 . For example, the intra predictor 120 may calculate one or more sample gradients of each line for the left half samples of each line, and calculate the line gradient based on the calculated one or more sample gradients. In addition, the intra prediction unit 120 may calculate one or more sample gradients of each line for the right half samples of each line, and calculate the line gradient based on the calculated one or more sample gradients.

The intra predictor 120 may calculate gradients of one or more samples of each line for samples selected according to a predefined method among samples of each line, and calculate a line gradient based on the calculated gradients of one or more samples. In calculating the line gradient, the size of the line (or the number of samples) may be larger or smaller than the width 2N of the second neighboring block. The intra prediction unit 120 may calculate a line gradient of a line using sample gradient(s) between selected some samples. The intra prediction unit 120 may select samples at predefined positions or a predefined number of samples from among all samples of the line. The intra prediction unit 120 may set a method of selecting only some samples from among all samples of a line. The set method may be transmitted from the encoding device 800 to the decoding device 2300 through a bitstream.

The intra predictor 120 may select some reference sample(s) to which update is to be applied from among all reference samples of the current block. The intra predictor 120 may select the number of reference sample(s) to which update is to be applied among all reference samples of the current block. For example, the intra predictor 120 may update only some of the reference samples 1221 of FIG. 12 .

The intra predictor 120 may select some reference sample(s) to which update is to be applied from among all reference samples of the current block according to characteristics of the current block. Also, the intra predictor 120 may determine the number of some reference sample(s) to which update is to be applied among all reference samples of the current block according to characteristics of the current block.

For example, the property of the current block may be the size of the current block. The intra predictor 120 may select some reference sample(s) to be updated from among all reference samples of the current block using different methods according to the size of the current block.

For example, considering the case where the size of the current block is relatively large, such as 16x16 and 32x32, the larger the size of the current block, the greater the number of reference samples of the current block. When the number of reference samples increases, the degree of correlation with respect to the gradient pattern according to the directionality may be relatively decreased. In this case, the encoding apparatus 800 may determine the number of reference samples to be updated based on the gradient.

A method for selecting some of the reference sample(s) to be updated from among the reference samples and the number of some of the reference sample(s) to be updated from among the reference samples may be transmitted from the encoding device 800 to the decoding device 2300 through a bitstream.

14 illustrates a method of updating a reference sample by reflecting a gradient of a neighboring block in a vertical direction according to an example.

In FIG. 14 , circles may represent samples (or pixels). A dotted line rectangle may represent a block.

In FIG. 14 , a current block 1410 , a reference sample block 1420 , reference samples 1421 , and neighboring blocks 1430 are illustrated.

The current block is shown as a block having a size of 4x4. The size of a block may mean a width and a height of the block.

The reference sample block 1420 may be a block including reference samples 1421 used for the current block 1410 . The reference sample block 1420 may be a block adjacent to the current block 1410 and having the same size as the current block 1410 .

The reference samples 1421 are illustrated as having values of A', B', C', and D', respectively, through updating of the reference sample. The illustrated reference samples may be samples configured according to a method for generating a reference sample among samples adjacent to the current block according to an intra prediction mode.

14 shows reference samples generated by the intra prediction unit 120 when the intra prediction mode for the current block 1410 is a vertical prediction mode.

Dotted lines in the neighboring block 1430 may represent vertical lines of pixels in the neighboring block 1430 . Thick solid lines in the neighboring block 1430 may represent gradients of samples included in the vertical line. ∇ dx can represent the slope value.

14 shows an example in which a sample value constantly increases and then decreases constantly from top to bottom in a vertical line of samples.

The neighboring block 1430 may be a block for which reconstruction has already been completed prior to encoding and/or decoding of the current block 1410 .

A neighboring block used for updating the reference sample may be different from a neighboring block used for intra residual prediction of the current block. As described above with reference to FIG. 9A , a neighboring block used for intra residual prediction of a current block may be referred to as a first neighboring block. Also, a neighboring block used for updating the reference sample may be referred to as a second block. The first neighboring block and the second neighboring block may be the same as or different from each other. Also, the first neighboring block may include the second neighboring block, or the second neighboring block may include the first neighboring block.

For example, when the intra prediction mode is the vertical prediction mode, the reference sample may be a sample adjacent to the top of the current block 1410 . Alternatively, the reference sample may be a sample of a horizontal line adjacent to the top of the current block 1410 . The reference pixel block 1420 may be a block adjacent to the top of the current block 1410 . Also, the neighboring block 1330 may be a block obtained by combining the left side block of the current block 1410 and the upper left adjacent block of the current block 1410 . A block adjacent to the left of the current block 1410 may be a block adjacent to the left of the current block 1410 . The block adjacent to the top left of the current block 1410 may be a block adjacent to the top left of the current block 1410 . The left adjacent block and the upper left adjacent block may be adjacent to each other.

When the size of the current block 1410 is NxN, the size of the left adjacent block and the upper left adjacent block may be NxN, respectively, and the size of the neighboring block 1430 may be Nx2N. In FIG. 11, it is shown that the size of the neighboring block is 8x4.

In addition, when the size of the current block 1410 is NxN, the size of the top adjacent block and the upper left adjacent block may each be aNxbN, and the size of the neighboring block 1430 may be aNx2bN. Here, a and b may each be a real number.

Also, each of the top adjacent block, the upper left adjacent block, and the neighboring block 1430 may have a predefined size or a size determined by a predefined method.

The intra prediction unit 120 of the encoding apparatus 800 and the intra prediction unit 240 of the decoding apparatus 2300 to be described later may use upper adjacent blocks, upper left adjacent blocks, and neighboring blocks of the same size. The encoding apparatus 800 may set the size of each of the top adjacent block, the upper left adjacent block, and the neighboring block 1430 . The size of the top adjacent block, the size of the top left adjacent block, and the size of the neighboring block 1430 may also be used in the decoding apparatus 2300 as well. The set size may be transmitted from the encoding device 800 to the decoding device 2300 through a bitstream.

The update of the reference sample may be regarded as restoration of reference samples around the current block for intra prediction.

When intra prediction is performed on the second neighboring block, the current block may also be composed of a texture having a direction similar to that of the second neighboring block according to spatial correlation. To reflect this direction, the intra predictor 120 may update a value of a reference sample required for intra prediction before performing intra prediction on the current block.

The intra predictor 120 may update the reference sample to be similar to the sample of the current block by using the gradient pattern according to the direction of the second neighboring block.

In step 915 described above with reference to FIG. 9A and step 1022 described above with reference to FIG. 10B , the intra predictor 120 may detect the gradient pattern of the second neighboring block and calculate the gradient.

In step 915 described with reference to FIG. 9A and step 1022 described with reference to FIG. 10B , the intra predictor 120 may detect gradient patterns of columns of the second neighboring block. The intra predictor 120 may check whether gradient patterns of columns of the second neighboring block are the same. When the gradient patterns of the columns of the second neighboring block are the same, the intra predictor 120 may calculate the gradient of the second neighboring block. Here, the gradient of the second neighboring block may be the gradient of one selected row of the second neighboring block. For example, the gradient of the second neighboring block may be the gradient of a column adjacent to the current block 1410 among columns of the second neighboring block. The intra predictor 120 may determine a gradient of a column adjacent to the current block 1410 among columns of the second neighboring block as a final gradient of the second neighboring block.

In addition, in steps 915 and 1022, the intra predictor 120 may determine or update a value of a reference sample based on neighboring blocks of the current block. The intra predictor 120 may determine or update a value of a reference sample based on a gradient pattern of a second neighboring block.

The value of the reference sample may be changed from a value before update to a value after update by the intra predictor 120, and the value of the reference sample before update may be a value generated by predicting and restoring the reference sample block 1420 including the reference sample. In other words, while the reference sample block 1420 is predicted and reconstructed, a value of the reference sample may be determined, and the determined value may be used as a value before update in updating the reference sample. Prediction and reconstruction methods of existing video encoding and/or decoding technologies such as HEVC and AVC may be used to predict and restore reference samples.

When the gradient ∇ dx is calculated, the intra predictor 120 may determine or update the value of the reference sample in a predefined manner.

For example, when the intra prediction mode is the vertical prediction mode, the intra predictor 120 may determine or update the value of the reference sample in the manner shown in Equation 3 below.

f can represent a function. w may represent a weight. A may be a value before updating of the topmost sample among the reference samples 1421 . A 'may be a value after updating of the topmost sample. B , C , and D may also be pre-update values of corresponding reference samples. B ', C ', and D ' may be post-update values of corresponding reference samples.

As described in Equation 4, the intra prediction unit 120 reflects a subtraction according to a gradient with respect to existing reference samples A , B, C , and D , thereby reconstructed reference samples A ', B ', C 'and D ' can be generated. Also, the intra predictor 120 may consider a weighting factor w when updating the value of the reference sample. The intra predictor 120 may update the value of the reference sample by adding the product of the value determined based on the gradient ∇ dx and the predefined weight factor w to the value before the update of the reference sample. Through updating, the value of the reference sample may increase by w * f (∇ dx ).

Reference samples may be plural. A weight factor w may be the same for a plurality of reference samples. Alternatively, the weight factor w may be different from each other for each of the plurality of reference samples. For example, the weight factor w may be different for each position of each reference sample of a plurality of reference samples. The position of the reference sample may be a position relative to the current block 1410 .

The intra prediction unit 120 may combine the aforementioned reference sample update method and the existing method. For example, when the intra prediction unit 120 performs updating of the reference sample described above with reference to FIG. 10A, in step 1010, padding of the reference pixel may not be performed. In addition, when the intra prediction unit 120 performs updating of the reference sample described above with reference to FIG. 10A , smoothing using a low-frequency filter may not be performed in step 1010 .

Types of Oblique Patterns

The inclination pattern for the direction described above with reference to FIGS. 12 to 14 may be classified into predefined types. For example, the gradient pattern may include at least some of 1) increase, 2) decrease, 3) increase and saturation, 4) decrease and saturation, 5) saturation and increase, 6) saturation and decrease, 7) increase and decrease symmetry, and 8) decrease and increase symmetry. Below, the aforementioned types are described.

1) “Increase” indicates a gradient pattern in which the values of samples increase along the direction. The direction may be from left to right. Alternatively, the direction may be from top to bottom.

2) “decrease” may indicate a gradient pattern in which the value of samples decreases along a direction.

3) “Increase and saturation” may represent a gradient pattern in which the values of samples increase in the front part and the values of the samples remain constant in the back part according to the direction.

4) “decrease and saturation” may indicate a gradient pattern in which values of samples decrease in the front part and values of the samples remain constant in the back part according to the direction.

5) “Saturation and increase” may represent a gradient pattern in which the values of samples are kept constant in the front part and the values of the samples increase in the back part according to the direction.

6) “Saturation and decrease” may represent a gradient pattern in which the values of samples are kept constant in the front part and the values of the samples decrease in the back part according to the direction.

7) "Symmetry of increase and decrease" may represent a left-right symmetrical gradient pattern in which values of samples increase in the front part and decrease in the back part according to the direction.

8) “Symmetry of decrease and increase” may indicate a left-right symmetrical gradient pattern in which values of samples decrease in the front part and increase in the value of the samples in the back part according to the direction.

The aforementioned “increase” may be a constant increase or a non-constant increase. Also, the above-mentioned "decrease" may be a constant decrease or a non-constant decrease.

The intra predictor 120 may determine a reference sample value according to the type of the gradient pattern of the second neighboring block of the current block. For example, when the gradient pattern is one of “increase”, “decrease”, “increase and saturation”, “decrease and saturation”, “saturation and increase”, and “saturation and decrease”, the intra prediction unit 120 may determine the value of the reference sample based on the gradient value of the “increase” line or the gradient value of the “decrease” line.

When the gradient pattern is “symmetrical”, the intra prediction unit 120 may determine a value of a reference sample based on gradient values of two symmetrical lines of the gradient pattern. For example, as described above with reference to FIG. 11 , when the type of the inclination pattern is symmetrical, increasing and then decreasing, the update of Equation 2 may be applied. For example, the intra predictor 120 may update the value of the reference sample by subtracting the product of the value determined based on the gradient ∇ dx and the predefined weight w from the value of the reference sample before updating. Through updating, the value of the reference sample can be reduced by w * f (∇ dx ). For example, as described above with reference to FIG. 13 , when the type of the inclination pattern is symmetrical, increasing after decreasing, the update of Equation 3 may be applied. For example, the intra predictor 120 may update the value of the reference sample by adding the product of the value determined based on the gradient ∇ dx and the predefined weight w to the value before updating the reference sample. Through updating, the value of the reference sample may increase by w * f (∇ dx ).

Use of Coding Parameters to Detect Gradient Patterns

In addition to the above-described embodiments, the intra predictor 120 may detect a gradient pattern using an encoding parameter. For example, the encoding parameters may include 1) syntax predefined in relation to intra prediction, 2) encoding variables, 3) current encoding unit, 4) prediction unit, 5) size of the transform unit, and 6) whether or not the encoding unit is split. For example, the intra predictor 120 may determine a value related to a neighboring block using an encoding parameter. The neighboring block may be the neighboring block 1230 described above with reference to FIG. 12 or the neighboring block 1430 described above with reference to FIG. 14 . A value related to a neighboring block may include a size of the neighboring block. The intra predictor 120 may determine a gradient of a neighboring block using an encoding parameter. The intra prediction unit 120 may determine a range of reference samples to which an update is applied using an encoding parameter. Also, the intra predictor 120 may determine a weight factor using an encoding parameter.

Expansion of the use of slope

The intra predictor 120 may apply the gradient of a neighboring block to a pixel value of a prediction block rather than to a reference sample. The intra predictor 120 may fix a value of a reference sample and update a pixel value of a prediction block using a gradient of a neighboring block. By directly updating the pixel value of the prediction block, the same effect as when updating the value of the reference sample can be obtained.

In the above-described embodiment, the update performed on the reference sample according to the gradient of the neighboring block may be equally applied to the prediction block. For example, updates made to one row of reference samples may be equally applied to each row of a plurality of rows of a prediction block. Alternatively, for example, updates made to reference samples of one column may be equally applied to each column of a plurality of columns of the prediction block. For the update of the prediction block of the intra predictor 120, the encoding parameter can be used in the same manner as used for the update of the reference sample. In other words, the intra predictor 120 may update a pixel value of a prediction block based on an encoding parameter.

15A shows intra prediction with 33 respective modes according to an example.

15B shows intra prediction with 65 respective modes according to an example.

In step 920 described with reference to FIG. 9A and step 1030 described with reference to FIG. 10A , the intra prediction unit 120 may generate a prediction block of the current block by performing intra prediction on the current block. In performing intra prediction on the current block, the intra predictor 120 may use updated (or reconstructed) reference sample(s). The intra predictor 120 may generate a prediction block of the current block using the updated (or reconstructed) reference sample(s).

The intra predictor 120 may construct neighboring reference sample(s) for the current block through the above-described update of the reference sample. The intra predictor 120 may generate one or more prediction blocks of the current block for one or more intra modes. One or more intra modes may include an angular mode.

The intra predictor 120 may determine a prediction block having a minimum rate-distortion value with respect to one or more prediction blocks. The intra prediction unit 120 may select an intra mode corresponding to a prediction block as a final intra prediction mode. Alternatively, the intra predictor 120 may select an intra mode that generates a prediction block having a minimum rate-distortion value among one or more intra modes as a final intra prediction mode.

In FIG. 15A, intra modes having 33 respective modes are illustrated. In FIG. 15B, intra modes having 65 respective modes are illustrated. Also, mode 0 may indicate a planer mode. Mode 1 may indicate a DC mode.

16 illustrates a region of an image according to an example.

In FIG. 16 , a portion 1600 of an image is shown. A portion 1600 of the illustrated image may include a current block 1610, a neighboring block 1620, a reconstructed region 1630, a neighboring block reference sample 1631, and a current block reference sample 1632.

The neighboring block reference sample 1631 may be a reference sample used when performing intra prediction on a neighboring block. The current block reference sample 1632 may be a reference sample used when performing intra prediction on the current block.

The sample value of the neighboring block reference sample 1631 may be an updated value when updating of the reference sample in steps 915 and 1022 is performed with respect to the neighboring block 1620 .

The sample value of the current block reference sample 1632 may be an updated value when the update of the reference sample in steps 915 and 1022 is performed with respect to the current block 1610 .

The current block 1610, the neighboring block 1620, the neighboring block reference sample 1631, and the current block reference sample 1632 shown in FIG. 16 may indicate relative positions when both the intra prediction mode of the neighboring block 1620 and the intra prediction mode of the current block 1610 are vertical prediction modes. For example, when the intra prediction mode is the vertical prediction mode, the first neighboring block described in FIGS. 9A and 10C may be a left adjacent block of the current block. The reference sample of the current block may be the pixel(s) in the horizontal line adjacent to the top of the current block. Also, the reference sample of the neighboring block may be pixel(s) in a horizontal line adjacent to an upper end of the neighboring block.

17 illustrates a method of calculating a residual signal of a neighboring block according to an example.

In FIG. 17 , a neighboring block 1710, a prediction block 1720 of the neighboring block, and a residual signal 1730 of the neighboring block are shown. In FIG. 17, the intra neighboring block_residual signal, which is _{the residual signal of the neighboring block,} is explained in the form of a matrix equation.

The neighboring block 1710 may correspond to the first neighboring block described above with reference to FIGS. 9A and 10C . Alternatively, the neighboring block 1710 may correspond to the neighboring block 1620 described above with reference to FIG. 16 .

The residual signal 1730 may correspond to the “residual signal of the first neighboring block” described above with reference to FIGS. 9A and 10C .

As shown in FIG. 17 , a residual signal 1730 may be a difference between a neighboring block 1710 and a prediction block 1720 . Alternatively, the residual signal 1730 may be a result of subtracting the prediction block 1720 from the neighboring block 1710 .

A value of the prediction block 1720 may be determined based on an intra prediction mode for the neighboring block 1710 .

For example, as shown in FIG. 17 , when the intra prediction mode of the neighboring block 1710 is the vertical prediction mode, values of each row of the prediction block 1720 may be values of reference pixels of the neighboring block 1710. In other words, when the intra prediction mode of the neighboring block 1710 is the vertical prediction mode, values of each row of the prediction block 1720 may be values of pixels in a horizontal line adjacent to an upper end of the neighboring block 1710 .

Alternatively, when the intra prediction mode of the neighboring block 1710 is the horizontal prediction mode, values of each column of the prediction block 1720 may be values of reference pixels of the neighboring block 1710. In other words, when the intra prediction mode of the neighboring block 1710 is the horizontal prediction mode, values of each column of the prediction block 1720 may be values of pixels in a vertical line adjacent to the left side of the neighboring block 1710.

In addition to the vertical prediction mode and the horizontal prediction mode, prediction block generation methods of existing video encoding and/or decoding technologies, such as HEVC and AVC, may be used for other intra prediction modes.

18 illustrates a method of calculating a residual signal of a current block according to an example.

In FIG. 18 , a current block 1810, a prediction block 1820 of the current block, and a residual signal 1830 of the current block are illustrated. In FIG. 18, Intra current block_residual signal, which is _{the residual signal of the current block,} is explained in the form of a matrix equation.

The current block 1810 may correspond to the current block described above with reference to FIGS. 9A and 10A. Alternatively, the current block 1810 may correspond to the current block 1610 described above with reference to FIG. 16 .

The residual signal 1830 may correspond to the “second residual signal of the current block” described above with reference to FIGS. 9A and 10C .

As shown in FIG. 18 , the residual signal 1830 may be the difference between the current block 1810 and the prediction block 1820 . Alternatively, the residual signal 1830 may be the result of subtracting the prediction block 1820 from the current block 1810 .

According to FIG. 18, the sum of the values of the residual signal 1830 is 560. In other words, the sum of the levels of the residual signal 1830 is 560.

A value of the prediction block 1820 may be determined based on intra prediction modes for neighboring blocks.

For example, as shown in FIG. 18 , when the intra prediction mode of the current block 1810 is the vertical prediction mode, values of each row of the prediction block 1820 may be values of reference pixels of neighboring blocks. In other words, when the intra prediction mode of the current block 1810 is the vertical prediction mode, values of each row of the prediction block 1820 may be values of pixels in a horizontal line adjacent to an upper end of the current block 1810 .

Alternatively, when the intra prediction mode of the current block 1810 is the horizontal prediction mode, values of each column of the prediction block 1820 may be values of reference pixels of neighboring blocks. In other words, when the intra prediction mode of the current block 1810 is the horizontal prediction mode, values of each column of the prediction block 1820 may be values of pixels in a vertical line adjacent to the left side of the current block 1810 .

The method of calculating the residual signal 1830 of the current block 1810 described with reference to FIG. 18 may correspond to the generation of the second residual signal of the current block in step 940 described above with reference to FIG. 9A and step 1045 described above with reference to FIG. 10C.

Also, the method of calculating the residual signal 1830 of the current block 1810 described with reference to FIG. 18 may correspond to generation of the third residual signal of the current block in steps 960 and 1046 .

A residual signal after residual signal prediction according to embodiments will be described in detail with reference to FIG. 19 below.

19 illustrates a residual signal prediction method according to an example.

In FIG. 19 , a residual signal 1910 of the current block, a residual signal 1920 of neighboring blocks, and a prediction residual signal 1930 of the current block are illustrated. In FIG. 19, the prediction residual signal of the current block, i.e., the intra _{prediction residual signal,} is explained in the form of a matrix equation.

19 may show residual signal prediction in step 950 of FIG. 9 and step 1042 of FIG. 10 .

The residual signal 1910 of the current block may correspond to the residual signal 1830 of the current block described above with reference to FIG. 18 . Alternatively, the residual signal 1910 of the current block may correspond to the “second residual signal of the current block” described above with reference to FIGS. 9A and 10C .

The residual signal 1920 of the neighboring block may correspond to the residual signal 1730 of the neighboring block described above with reference to FIG. 17 . Alternatively, the residual signal 1920 of the neighboring block may correspond to the “residual signal of the first neighboring block” described above with reference to FIGS. 9A and 10C .

The prediction residual signal 1930 of the current block may correspond to the “first residual signal of the current block” described above with reference to FIGS. 9A and 10C .

The prediction residual signal 1930 may be a difference between the residual signal 1910 of the current block and the residual signal 1920 of neighboring blocks. Alternatively, the prediction residual signal 1930 may be a result of subtracting the residual signal 1920 of the neighboring block from the residual signal 1910 of the current block.

The sum of the values of the prediction residual signal 1930 is 267. In other words, the sum of the levels of the prediction residual signal 1930 is 267. By the residual signal prediction, the final sum level of the residual signal of the current block is reduced from 560 to 269. Reduction of the final sum of levels of the residual signal by residual signal prediction may indicate that energy of the residual signal itself used for encoding the current block is reduced. Reducing the energy of the residual signal itself used for encoding may mean that the amount of bits required for encoding is reduced. Accordingly, the encoding apparatus 800 may reduce the bitstream capacity through residual signal prediction.

20A illustrates a default residual signal according to an example.

The default residual of FIG. 20A may be an example of the second residual signal described above with reference to FIGS. 9A and 10C . In other words, the default residual may be the residual signal of the current block before residual signal prediction. Alternatively, the default residual of FIG. 18A may be an example of the residual signal 1910 of the current block described above with reference to FIG. 19 .

20B shows a result of discrete cosine transformation on a default residual signal according to an example.

In FIG. 20B, a result of discrete cosine transform for the default residual signal of FIG. 20A is shown.

21A illustrates a proposed residual signal according to an example.

The proposed residual of FIG. 21A may be an example of the first residual signal described above with reference to FIGS. 9A and 10C. In other words, the proposed residual may be a residual signal of the current block according to residual signal prediction. Alternatively, the proposed residual of FIG. 20A may be an example of the prediction residual signal 1930 of the current block described above with reference to FIG. 19 .

21B shows a result of discrete cosine transformation on the proposed residual signal according to an example.

In FIG. 21B, the result of DCT for the default residual signal of FIG. 21A is shown.

Referring to FIGS. 20A and 21B , it can be confirmed that the energy concentration in the frequency domain is increased by the residual signal prediction. The intra residual predictor 810 may improve the energy concentration in the frequency domain of the residual signal of the current block through residual signal prediction. As the magnitude of the coefficient value in the frequency domain is reduced by residual signal prediction, the probability that the result value generated by quantization will be 0 and the probability that the result value generated by quantization will be close to 0 can be improved.

22 illustrates locations of neighboring blocks according to an example.

In FIG. 16 , a neighboring block 1620 is shown adjacent to the left side of the current block 1610 . In FIG. 16, the case where residual signal prediction is performed using the left adjacent block has been described. However, the location of the first neighboring block described above with reference to FIGS. 9A and 10C may not be limited to the left side of the current block 1610 .

As described above in step 935 of FIG. 9 and step 1043 of FIG. 10 , the intra residual predictor 810 may determine a first neighboring block among one or more neighboring blocks of the current block. The first neighboring block may be a block used for residual signal prediction. The intra residual predictor 810 may determine a first neighboring block among one or more previously reconstructed neighboring blocks of the current block.

As the first neighboring block is determined among the plurality of neighboring blocks, the intra residual predictor 810 may improve spatial correlation of the first neighboring block with respect to the residual signal.

In FIG. 22 , as one or more neighboring blocks of the current block, a lower left adjacent block A ₀ (2221), a left adjacent block A ₁ (2222), an upper right adjacent block B ₁ (2223), a right adjacent block B ₂ (2224), and an upper left adjacent block B ₃ (2225) are shown. As shown, one or more neighboring blocks of the current block may include a lower left adjacent block, a left adjacent block, an upper right adjacent block, a right adjacent block, and an upper left adjacent block. One or more neighboring blocks of the current block are not limited to the blocks at the positions illustrated in FIG. 22 .

In order to improve the efficiency of residual signal prediction, spatial correlation between the current block and neighboring blocks needs to be improved. In order to improve a neighboring block having a high spatial correlation, the intra residual predictor 810 may perform residual signal prediction on each neighboring block among a plurality of neighboring blocks of the current block. When a plurality of residual signals for a plurality of neighboring blocks are generated according to residual signal prediction, the intra residual predictor 810 may select a minimum rate-distortion residual signal having a minimum rate-distortion value from among the plurality of residual signals. Also, the intra residual predictor 810 may select a neighboring block corresponding to a minimum rate-distortion residual signal from among a plurality of neighboring blocks. The intra residual predictor 810 may use the selected neighboring block and the minimum rate-distortion residual signal to encode the current block.

For example, steps 935, 940, and 950 described above with reference to FIG. 9A and steps 1043, 1044, and 1045 described above with reference to FIG. 10C may be performed for each block of a plurality of neighboring blocks of the current block. Here, the plurality of neighboring blocks of the current block may be at least some of one or more previously reconstructed blocks adjacent to the current block. The number of the plurality of neighboring blocks and the positions of the plurality of neighboring blocks may be changed according to the setting of the encoding apparatus 800 .

Through repetition of step 935 or step 1043, the intra residual predictor 810 may sequentially select a plurality of neighboring blocks of the current block as the first block.

Through repetition of step 950 or step 1045, the intra residual predictor 810 may generate a plurality of first residual signals of a plurality of neighboring blocks of the current block. Also, the intra residual predictor 810 may calculate a rate-distortion value for each of the plurality of first residual signals.

The intra residual predictor 810 may determine a minimum rate-distortion residual signal having a minimum rate-distortion value among a plurality of first residual signals, and may determine a neighboring block corresponding to the minimum rate-distortion residual signal as a first neighboring block used for residual signal prediction of the current block.

Step 940 may be repeated as step 950 is repeated, or may be performed only once. Step 1044 may be repeated as step 1055 is repeated, or may be performed only once.

In steps 935 and 1043, a first neighboring block of the current block may be determined for each coding unit. In addition, in steps 985 and 1048, the identifier of the first neighboring block may be encoded for each coding unit.

23 is a structural diagram of a decryption device according to an embodiment.

The decoding device 2300 may correspond to the decoding device 200 described above. The decoding apparatus 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra prediction unit 240, a motion compensation unit 250, an adder 255, a filter unit 260, and a reference picture buffer 270, and may further include an intra residual prediction unit 2310.

The entropy decoding unit 210, the inverse quantization unit 220, the inverse transform unit 230, the intra prediction unit 240, the motion compensation unit 250, the adder 255, the filter unit 260, and the reference picture buffer 270 may perform the same functions and/or operations as described above with reference to FIG. Redundant descriptions are omitted.

In the embodiments described above with reference to FIGS. 9A to 22, the function and / or operation described as being performed by the intra prediction unit 120 of the encoding apparatus 800 may be performed by the intra prediction unit 240 of the decoding apparatus 2300. Also, functions and/or operations described as being performed by the intra residual predictor 810 of the encoding apparatus 800 may be performed by the intra residual predictor 2310 of the decoding apparatus 2300.

In addition, the entropy decoding unit 210, the inverse quantization unit 220, the inverse transform unit 230, the intra prediction unit 240, the motion compensation unit 250, the adder 255, the filter unit 260, and the reference picture buffer 270 may perform functions and/or operations related to the intra residual predictor 2310. The functions and/or operations of the entropy decoding unit 210, the inverse quantization unit 220, the inverse transform unit 230, the intra prediction unit 240, the motion compensation unit 250, the adder 255, the filter unit 260, the reference picture buffer 270, and the intra residual predictor 2310 will be described in detail below.

The intra residual predictor 2310 may not be distinguished from the intra predictor 240 . The intra predictor 240 and the intra residual predictor 2310 may be integrated into the intra predictor 240, and in embodiments, functions and/or operations described as being performed by the intra residual predictor 2310 may be performed by the intra predictor 240.

24A and 24B are flowcharts of a decoding method according to an exemplary embodiment.

Hereinafter, the current block may be a block currently being decoded or may be a block within the current image.

First, referring to FIG. 24A , step 2410 may be performed.

In operation 2410, the decoding apparatus 2310 may generate a residual signal of the current block. Here, the generated residual signal may correspond to the first residual signal of the current block described above with reference to FIGS. 9A and 10C .

Step 2410 may be performed by at least one of the entropy decoding unit 210, the inverse quantization unit 220, and the inverse transform unit 230.

Step 2410 may include step 2411 , step 2412 and step 2413 .

In step 2411, the entropy decoding unit 210 may generate quantized coefficients for the current block.

In operation 2412, the inverse quantization unit 220 may generate inverse quantized coefficients by performing inverse quantization on the quantized coefficients.

In operation 2413, the inverse transform unit 230 may generate a residual signal by performing an inverse transform on the inverse quantized coefficient.

After step 2410 is performed, step 2420 and step 2$40 may be performed.

Next, refer to FIG. 24B.

Prior to performing step 2420, a reference sample may be created. The information related to the generation of the reference sample described above with reference to FIG. 11 may also be applied to this embodiment. Redundant descriptions are omitted.

In step 2420, the intra predictor 240 may determine whether to update the reference sample.

Here, the update of the reference sample may be reconstructing a sample value of the reference sample used for generating the prediction block before generating the prediction block of the current block.

The intra predictor 240 may determine whether or not to update the reference sample by using the information indicating whether or not to update the reference sample described above with reference to FIGS. 9A and 10B. A bitstream transmitted from the encoding device 800 to the decoding device 2300 may include information indicating whether or not updating of reference samples is performed. Information indicating whether updating of reference samples is performed may be encoded in the bitstream.

The intra prediction unit 240 may decode information indicating whether updating of the encoded reference sample is performed. The intra predictor 240 may determine whether or not to update the reference sample by using information indicating whether or not to update the decoded reference sample. If the above information indicates that the reference sample is updated, the intra predictor 240 may update the reference sample. If the above information indicates that the reference sample is not updated, the intra predictor 240 may not update the reference sample.

If it is determined to perform an update of the reference sample, step 2425 may be performed. If it is determined not to perform a reference sample update, step 2430 may be performed.

In step 2425, the intra predictor 240 may update the value of the reference sample and determine the value of the reference sample used to generate the prediction block of the current block through the update.

The contents related to the update of the reference sample described above with reference to FIGS. 12, 13, and 14 may also be applied to this embodiment. In the embodiments described above with reference to FIGS. 12, 13, and 14, the function and / or operation described as being performed by the intra prediction unit 120 of the encoding apparatus 800 may be performed by the intra prediction unit 240 of the decoding apparatus 2300. In addition, the functions and/or operations described as being performed in step 915 described above with reference to FIG. 9A and step 1022 described above with reference to FIG. 10B may also be performed in step 2425 . Redundant descriptions are omitted.

Step 2430 may be performed after step 2425 is performed.

In step 2430, the intra predictor 240 may generate a prediction block of the current block. The intra prediction unit 240 may generate a prediction block of the current block by using the reference sample according to the intra prediction mode of the current block.

For example, in generating a prediction block, a prediction block generation method of an existing video encoding and/or decoding technology such as HEVC and AVC may be used.

After step 2430 is performed, either step 2460 or step 2490 may be performed.

In step 2440, the intra residual predictor 2310 may determine whether to perform residual signal prediction.

The intra residual predictor 2310 may determine whether or not to perform residual signal prediction using the information indicating whether to perform residual signal prediction described above with reference to FIGS. 9A and 10C . The bitstream transmitted from the encoding apparatus 800 to the decoding apparatus 2300 may include information indicating whether residual signal prediction is performed. Information representing whether or not residual signal prediction is performed may be coded within the bitstream.

The intra predictor 240 may decode information indicating whether prediction of the coded residual signal is performed. The intra predictor 240 may determine whether or not to perform prediction of the decoded residual signal using information indicating whether to perform prediction of the decoded residual signal. If the above information indicates that residual signal prediction is performed, the intra prediction unit 240 may perform residual signal prediction. If the above information indicates that residual signal prediction is not performed, the intra predictor 240 may not perform residual signal prediction.

If it is determined to perform residual signal prediction, step 2450 may be performed.

If it is determined not to perform residual signal prediction, step 2490 may be performed.

In operation 2450, the intra residual predictor 2310 may identify a first neighboring block. The first neighboring block may be a block used for residual signal prediction and may be a block located in the vicinity of the current block. The information related to the determination of the first neighboring block described above with reference to FIG. 22 may also be applied to the present embodiment.

The intra residual predictor 2310 may identify the first neighboring block using the identifier of the first neighboring block described above with reference to FIGS. 9A and 10C . The bitstream transmitted from the encoding device 800 to the decoding device 2300 may include an identifier of the first neighboring block. The identifier of the first neighboring block may be encoded within the bitstream.

The intra residual predictor 2310 may decode the encoded ID of the first neighboring block. The intra residual predictor 2310 may identify the first neighboring block using the decoded identifier of the first neighboring block.

The identifier of the first neighboring block may be information enabling the identification of the neighboring block used to predict the residual signal of the current block.

For example, the identifier of the first neighboring block may indicate a neighboring block used to predict a residual signal of the current block among a plurality of neighboring blocks. Alternatively, the identifier of the first neighboring block may be location information indicating a location of a neighboring block used for predicting a residual signal of a current block among a plurality of neighboring blocks. The position may indicate a relative position of the selected neighboring block to the current block. The location may indicate a direction in which the selected neighboring block is adjacent to the current block.

The location and number of neighboring blocks may be defined according to encoding parameters.

The identifier of the first neighboring block may be configured to indicate the same block in both the encoding device 800 and the decoding device 2300. For example, in relation to the identifier of the first neighboring block, the block size N and the location of the neighboring blocks may be common to the encoding apparatus 800 and the decoding apparatus 2300. In order for the encoding apparatus 800 and the decoding apparatus 2300 to share a common configuration with respect to the identifier of the first neighboring block, the identifier of the first neighboring block may be entropy-decoded using a “(neighboring-residual_index) truncated unary ((neighboring-residual_idx) truncated unary)” method.

After steps 2450 and 2430 are performed, step 2460 may be performed next.

In step 2460, the intra residual predictor 2310 may generate a reconstructed block of the current block.

The intra residual predictor 2310 may generate a reconstructed block of the current block based on the prediction block, the residual signal of the current block, and the residual signal of the first neighboring block.

The residual signal of the current block used in step 2460 may be a residual signal generated by residual signal prediction in the encoding apparatus 800 . In other words, the residual signal of the current block used in step 2460 may correspond to the first residual signal described above with reference to FIGS. 9A and 10C.

The first neighboring block may be a block that has already been restored before decoding the current block. Accordingly, the residual signal of the first neighboring block may have already been obtained by the intra residual predictor 2310 before decoding the current block.

The reconstructed block of the current block may be the sum of the prediction block of the current block, the residual signal of the current block, and the residual signal of neighboring blocks. Also, a reconstructed block of the current block may be generated based on a sum of a residual signal of the current block and a residual signal of the first neighboring block.

The reconstructed block may be the sum of 1) the prediction block, 2) the residual signal of the current block, and 3) the residual signal of the first neighboring block. A prediction block may be obtained according to an intra prediction mode. The residual signal of the current block may be transmitted from the encoding apparatus 800 to the decoding apparatus 2300. A residual signal of the first neighboring block may be obtained by residual signal prediction. For example, when residual signal prediction is not performed, the residual signal of the first neighboring block may be a zero signal.

In some cases, when the intra residual predictor 2310 generates the residual signal of the first neighboring block according to the residual signal prediction, the intra predictor 240 may generate a reconstructed block of the current block by summing the prediction block of the current block, the residual signal of the current block, and the residual signals of the neighboring blocks.

If step 2430 is performed and, in step 2440, it is determined not to perform residual signal prediction, step 2490 may be performed.

In operation 2490, the intra predictor 240 or the intra residual predictor 2310 may generate a reconstructed block of the current block based on the prediction block and the residual signal of the current block.

The residual signal of the current block used in step 2490 may be a residual signal generated without performing residual signal prediction in the encoding apparatus 800 . In other words, the residual signal of the current block used in step 2490 may correspond to the third residual signal described above with reference to FIGS. 9A and 10C.

The third residual signal may be a residual signal of a current block in existing video encoding and/or decoding technologies such as HEVC and AVC. For example, in generating the third residual signal, a residual signal generation method of an existing video encoding and/or decoding technology such as HEVC and AVC may be used.

25A is a flowchart of a method for generating a residual signal according to an embodiment.

Hereinafter, the current block may be a block currently being decoded or may be a block within the current image.

First, referring to FIG. 25A , step 2510 may be performed.

In operation 2510, the decoding apparatus 2310 may generate a residual signal of the current block. Here, the generated residual signal may correspond to the first residual signal of the current block described above with reference to FIGS. 9A and 10C .

Step 2510 may be performed by at least one of the entropy decoding unit 210, the inverse quantization unit 220, and the inverse transform unit 230.

Step 2510 may include step 2411 , step 2412 and step 2413 .

In step 2511, the entropy decoding unit 210 may generate quantized coefficients for the current block.

In operation 2512, the inverse quantization unit 220 may generate inverse quantized coefficients by performing inverse quantization on the quantized coefficients.

In operation 2513, the inverse transform unit 230 may generate a residual signal by performing an inverse transform on the inverse quantized coefficient.

After step 2510 is performed, step 2520 to be described later with reference to FIG. 25B may be performed.

25B is a flowchart of a method of decoding a current block using a feast signal according to an exemplary embodiment.

In step 2520, the intra predictor 240 may decode information indicating whether updating of the encoded reference sample is performed.

A bitstream transmitted from the encoding device 800 to the decoding device 2300 may include information indicating whether or not updating of reference samples is performed. Information indicating whether updating of reference samples is performed may be encoded in the bitstream.

In operation 2530, the intra predictor 240 may decode information indicating whether prediction of the coded residual signal is performed.

The bitstream transmitted from the encoding apparatus 800 to the decoding apparatus 2300 may include information indicating whether residual signal prediction is performed. Information representing whether or not residual signal prediction is performed may be coded within the bitstream.

In operation 2540, the intra residual predictor 2310 may decode the encoded ID of the first neighboring block.

The bitstream transmitted from the encoding device 800 to the decoding device 2300 may include an identifier of the first neighboring block. The identifier of the first neighboring block may be encoded within the bitstream.

In operation 2550, the intra predictor 240 and the intra residual predictor 2310 may decode the residual signal. The intra predictor 240 and the intra residual predictor 2310 may generate a reconstructed block by decoding the residual signal.

Step 2550 may include steps 2561, 2562, 2563, and 2564 that will be described later with reference to FIG. 25C.

Also, step 2550 may include steps 2571, 2572, 2573, and 2574 to be described later with reference to FIG. 25D.

25C is a flowchart of a prediction block generation method according to an embodiment.

A reference sample may be created by steps 2561, 2562 and 2563. The information related to the generation of the reference sample described above with reference to FIG. 11 may also be applied to this embodiment. Redundant descriptions are omitted.

In step 2561, the intra predictor 240 may determine whether to update the reference sample.

Here, the update of the reference sample may be reconstructing a sample value of the reference sample used for generating the prediction block before generating the prediction block of the current block.

The intra predictor 240 may determine whether or not to update the reference sample by using the information indicating whether or not to update the reference sample described above with reference to FIGS. 9A and 10B.

The intra predictor 240 may determine whether or not to update the reference sample by using information indicating whether or not to update the decoded reference sample. If the above information indicates that the reference sample is updated, the intra predictor 240 may update the reference sample. If the above information indicates that the reference sample is not updated, the intra predictor 240 may not update the reference sample.

If it is determined to perform an update of the reference sample, step 2563 may be performed. If it is determined not to perform a reference sample update, step 2564 may be performed.

In step 2563, the intra predictor 240 may update the value of the reference sample and determine the value of the reference sample used to generate the prediction block of the current block through the update.

The contents related to the update of the reference sample described above with reference to FIGS. 12, 13, and 14 may also be applied to this embodiment. In the embodiments described above with reference to FIGS. 12, 13, and 14, the function and / or operation described as being performed by the intra prediction unit 120 of the encoding apparatus 800 may be performed by the intra prediction unit 240 of the decoding apparatus 2300. In addition, the function and/or operation described as being performed in step 915 described above with reference to FIG. 9A and step 1022 described above with reference to FIG. 10B may also be performed in step 2563 . Redundant descriptions are omitted.

Step 2564 may be performed after step 2563 is performed.

In step 2564, the intra predictor 240 may generate a prediction block of the current block. The intra prediction unit 240 may generate a prediction block of the current block by using the reference sample according to the intra prediction mode of the current block.

For example, in generating a prediction block, a prediction block generation method of an existing video encoding and/or decoding technology such as HEVC and AVC may be used.

For example, after step 2564 is performed, step 2571 to be described later with reference to FIG. 25D may be performed.

25D is a flowchart of a recovery block generation method according to an embodiment.

In step 2571, the intra residual predictor 2310 may determine whether to perform residual signal prediction.

The intra residual predictor 2310 may determine whether or not to perform residual signal prediction using the information indicating whether to perform residual signal prediction described above with reference to FIGS. 9A and 10C .

The intra predictor 240 may determine whether or not to perform prediction of the decoded residual signal using information indicating whether to perform prediction of the decoded residual signal. If the above information indicates that residual signal prediction is performed, the intra prediction unit 240 may perform residual signal prediction. If the above information indicates that residual signal prediction is not performed, the intra predictor 240 may not perform residual signal prediction.

If it is determined to perform residual signal prediction, step 2572 may be performed.

If it is determined not to perform residual signal prediction, step 2574 may be performed.

In step 2572, the intra residual predictor 2310 may identify a first neighboring block. The first neighboring block may be a block used for residual signal prediction and may be a block located in the vicinity of the current block. The information related to the determination of the first neighboring block described above with reference to FIG. 22 may also be applied to the present embodiment.

The intra residual predictor 2310 may identify the first neighboring block using the identifier of the first neighboring block described above with reference to FIGS. 9A and 10C .

The intra residual predictor 2310 may identify the first neighboring block using the decoded identifier of the first neighboring block.

The identifier of the first neighboring block may be information enabling the identification of the neighboring block used to predict the residual signal of the current block.

For example, the identifier of the first neighboring block may indicate a neighboring block used to predict a residual signal of the current block among a plurality of neighboring blocks. Alternatively, the identifier of the first neighboring block may be location information indicating a location of a neighboring block used for predicting a residual signal of a current block among a plurality of neighboring blocks. The position may indicate a relative position of the selected neighboring block to the current block. The location may indicate a direction in which the selected neighboring block is adjacent to the current block.

The location and number of neighboring blocks may be defined according to encoding parameters.

The identifier of the first neighboring block may be configured to indicate the same block in both the encoding device 800 and the decoding device 2300. For example, in relation to the identifier of the first neighboring block, the block size N and the location of the neighboring blocks may be common to the encoding apparatus 800 and the decoding apparatus 2300. In order for the encoding apparatus 800 and the decoding apparatus 2300 to share a common configuration with respect to the identifier of the first neighboring block, the identifier of the first neighboring block may be entropy-decoded using a “(neighboring-residual_index) truncated unary ((neighboring-residual_idx) truncated unary)” method.

If step 2572 is performed, step 2573 may be performed next.

In step 2573, the intra residual predictor 2310 may generate a reconstructed block of the current block.

The intra residual predictor 2310 may generate a reconstructed block of the current block based on the prediction block, the residual signal of the current block, and the residual signal of the first neighboring block.

The residual signal of the current block used in step 2573 may be a residual signal generated by residual signal prediction in the encoding apparatus 800 . In other words, the residual signal of the current block used in step 2573 may correspond to the first residual signal described above with reference to FIGS. 9A and 10C.

The first neighboring block may be a block that has already been restored before decoding the current block. Accordingly, the residual signal of the first neighboring block may have already been obtained by the intra residual predictor 2310 before decoding the current block.

The reconstructed block of the current block may be the sum of the prediction block of the current block, the residual signal of the current block, and the residual signal of neighboring blocks. Also, a reconstructed block of the current block may be generated based on a sum of a residual signal of the current block and a residual signal of the first neighboring block.

The reconstructed block may be the sum of 1) the prediction block, 2) the residual signal of the current block, and 3) the residual signal of the first neighboring block. A prediction block may be obtained according to an intra prediction mode. The residual signal of the current block may be transmitted from the encoding apparatus 800 to the decoding apparatus 2300. A residual signal of the first neighboring block may be obtained by residual signal prediction. For example, when residual signal prediction is not performed, the residual signal of the first neighboring block may be a zero signal.

In some cases, when the intra residual predictor 2310 generates the residual signal of the first neighboring block according to the residual signal prediction, the intra predictor 240 may generate a reconstructed block of the current block by summing the prediction block of the current block, the residual signal of the current block, and the residual signals of the neighboring blocks.

If at step 2570 it is determined not to perform residual signal prediction, then step 2574 may be performed.

In operation 2574, the intra predictor 240 or the intra residual predictor 2310 may generate a reconstructed block of the current block based on the prediction block and the residual signal of the current block.

The residual signal of the current block used in step 2574 may be a residual signal generated without performing residual signal prediction in the encoding apparatus 800 . In other words, the residual signal of the current block used in step 2574 may correspond to the third residual signal described above with reference to FIGS. 9A and 10C.

The third residual signal may be a residual signal of a current block in existing video encoding and/or decoding technologies such as HEVC and AVC. For example, in generating the third residual signal, a residual signal generation method of an existing video encoding and/or decoding technology such as HEVC and AVC may be used.

Implicit transmission of information related to residual signal prediction

As described above, the bitstream may include 1) information indicating whether residual signal prediction is to be performed and 2) an identifier of a first neighboring block used for residual signal prediction of a current block. In other words, the above information and the above identifier may be explicitly transmitted to the decoding apparatus 2300 by the encoding apparatus 800.

On the other hand, if a condition predefined for reducing the amount of bits required for the above information is met, 1) information indicating whether or not residual signal prediction is performed may be omitted. Even if the above information is omitted, the decoding apparatus 2300 may derive the above information when a predefined condition is satisfied. In other words, the above information can be transmitted implicitly.

In addition, when a predefined condition for reducing the amount of bits required for the identifier is satisfied, 2) the identifier of the first neighboring block used for predicting the residual signal of the current block may be omitted. Even if the above information is omitted, the decoding device 2300 may derive the above identifier when a predefined condition is satisfied. That is to say, the above identifier may be sent explicitly.

In step 930 described above with reference to FIG. 9A and step 1041 described above with reference to FIG. 10C , when a predefined condition is satisfied, the intra residual predictor 810 determines to perform residual signal prediction on the current block. Alternatively, when a predefined condition is satisfied, the intra residual predictor 810 may determine not to perform residual signal prediction on the current block.

In addition, in steps 935 and 1043, when a predefined condition is satisfied, the intra residual predictor 810 may determine the predefined block as the first neighboring block. Also, steps 980, 985, 1048 and 1049 may be selectively performed.

In steps 980 and 1049, the intra residual predictor 810 may selectively encode information indicating whether residual signal prediction is to be performed. Alternatively, when a predefined condition is satisfied, the intra residual predictor 810 may omit encoding of information indicating whether or not residual signal prediction is performed.

In operations 985 and 1048, the intra residual predictor 810 may selectively encode the identifier of the first neighboring block. Alternatively, when a predefined condition is satisfied, the intra residual predictor 810 may omit coding of the identifier of the first neighboring block.

In step 2440 described above with reference to FIG. 24B , when a predefined condition is satisfied, the intra residual predictor 2310 may determine to perform residual signal prediction. Alternatively, when a predefined condition is satisfied, the intra residual predictor 2310 may determine not to perform residual signal prediction. For example, when there is no information indicating whether residual signal prediction is performed, the intra residual predictor 2310 may determine to perform residual signal prediction.

In addition, in operation 2445, when a predefined condition is satisfied, the intra residual predictor 2310 may identify the selected block as a first neighboring block according to a predefined method. For example, when the identifier of the first neighboring block does not exist, the intra residual predictor 2310 may identify the selected block as the first neighboring block according to a predefined method.

For example, when the directionality of the current block and the directionality of the first neighboring block are the same, high efficiency can be expected in predicting the residual signal. Accordingly, when a block having the same intra prediction mode as the intra prediction mode of the current block exists in the vicinity of the current block, residual signal prediction of the current block may be performed on the block having the same intra prediction mode.

In step 930 described above with reference to FIG. 9A and step 1041 described above with reference to FIG. 10C , when a block having the same intra prediction mode as the intra prediction mode of the current block exists around the current block, the intra residual predictor 810 may determine to perform residual signal prediction on the current block. For example, when the MPM flag has a value of true (or “1”), the intra residual predictor 2310 may detect that a block having the same intra prediction mode as that of the current block exists in the vicinity of the current block, and may determine to perform residual signal prediction on the current block. Accordingly, the intra residual predictor 2310 may identify whether residual signal prediction is used based on the value of the MPM flag.

In addition, in steps 935 and 1043, when blocks having the same intra prediction mode as the intra prediction mode of the current block exist in the vicinity of the current block, the intra residual predictor 810 may determine the block having the same intra prediction mode as the first neighboring block. When a plurality of blocks having the same intra prediction mode as the intra prediction mode of the current block exist around the current block, the intra residual predictor 810 may determine a block selected according to a predefined priority among the plurality of blocks as the first neighboring block.

In steps 980 and 1049, when a block having the same intra prediction mode as the intra prediction mode of the current block exists around the current block, the intra residual predictor 810 may omit encoding of information indicating whether or not residual signal prediction is performed.

In steps 985 and 1048, when blocks having the same intra prediction mode as the intra prediction mode of the current block exist in the vicinity of the current block, the intra residual predictor 810 may omit coding of the identifier of the first neighboring block.

In step 2440 described above with reference to FIG. 24B, if a block having the same intra prediction mode as the intra prediction mode of the current block exists around the current block, the intra residual predictor 2310 may determine to perform residual signal prediction. Alternatively, when there is no information indicating whether or not residual signal prediction is performed, the intra residual predictor 2310 may determine to perform residual signal prediction when a block having the same intra prediction mode as the intra prediction mode of the current block exists around the current block. Alternatively, when information indicating whether residual signal prediction is not present, the intra residual predictor 2310 may determine not to perform residual signal prediction if there is no block having the same intra prediction mode as the intra prediction mode of the current block in the vicinity of the current block.

In addition, in step 2445, if blocks having the same intra prediction mode as the intra prediction mode of the current block exist in the vicinity of the current block, the intra residual predictor 2310 may identify the block having the same intra prediction mode as the first neighboring block. Alternatively, when the identifier of the first block does not exist, the intra residual predictor 2310 may identify the block having the same intra prediction mode as the first neighboring block if there are blocks having the same intra prediction mode as the intra prediction mode of the current block in the vicinity of the current block. Alternatively, when the identifier of the first block does not exist and a plurality of blocks having the same intra prediction mode as the intra prediction mode of the current block exist around the current block, the intra residual predictor 810 may determine a block selected according to a predefined priority among the plurality of blocks as the first neighboring block.

When intra prediction is performed, the intra prediction mode may be transmitted from the encoding apparatus 800 to the decoding apparatus 2300 through a Most Probable Mode (MPM) flag and an MPM index. In other words, the intra prediction mode can be entropy encoded by the MPM flag and the MPM index. MPM can represent all three intra prediction modes. Intra prediction modes represented by MPM are called MPM candidate modes. The intra prediction unit 240 may identify MPM candidate modes through prediction blocks in a picture around the current block.

If the intra prediction mode of the current block is the same as one of the three intra prediction modes identified by MPM, the value of the MPM flag may be true (or “1”). Also, when the value of the MPM flag is true (or “1”), the encoding device 800 may transmit the MPM index to the decoding device 2300. The MPM index may indicate which mode among MPM candidate modes is an intra prediction mode of the current block.

The intra residual predictor 2310 may perform intra prediction according to the syntax definition. When the final intra prediction mode obtained after performing intra prediction is the same as one of the MPM candidate modes, the intra residual predictor 2310 may obtain a residual signal of the current block and perform prediction on the residual signal of the current block using the residual signal of the first neighboring block. When the intra prediction mode of the current block is the same as the intra prediction mode of the first neighboring block, the intra residual predictor 2310 may identify the position of the first neighboring block for prediction of the residual signal of the current block through the MPM index.

Unit of update of reference sample

In step 2420, the intra predictor 240 may determine whether or not to update reference samples for each predefined stage. The predefined unit may be at least one of 1) an entire image sequence (ie, video), 2) one image (ie, picture), 3) a slice, and 4) a coding unit.

For a predefined unit, reference sample update information may be used to indicate whether or not to update a reference sample. The reference sample update information may be information indicating whether the update of the reference sample is performed for a predefined unit. For example, when the value of the reference sample update information is the first value, it may indicate that the reference sample should be updated in decoding the current block. When the value of the reference sample update information is the second value, it may indicate that the update of the reference sample is not performed in decoding the current block.

The encoding apparatus 800 may include encoded residual signal prediction information in a bitstream. The decoding apparatus 2300 may determine whether to perform residual signal prediction on the current block using residual signal prediction information.

Below, the update of the reference sample for each of the predefined units is described.

1) Entire video sequence: Whether to update reference samples may be determined for the entire video sequence. In this case, a sequence parameter set may include reference sample update information. If the reference sample update information of the sequence parameter set indicates that the reference sample is updated, the intra prediction unit 240 may perform intra prediction using a reference sample to which a gradient according to a direction is applied to the entire video sequence.

2) One image: Whether to update the reference sample may be determined for each image. In this case, a picture parameter set may include reference sample update information. If the reference sample update information of the picture parameter set indicates that the reference sample is updated, the intra prediction unit 240 uses a reference sample to which a directional gradient is applied to the entire image corresponding to the picture parameter set. Can perform intra prediction.

3) Slice: One image can be divided into a plurality of slice segments or a plurality of tiles within one slice segment. Whether to update the reference sample may be determined for each slice. In this case, the slice segment header may include reference sample update information. If the reference sample update information of the slice segment header indicates that the reference sample is updated, the intra prediction unit 240 may perform intra prediction using a reference sample to which a gradient according to a direction is applied for a slice corresponding to the slice segment header.

4) Coding Unit: Whether to update a reference sample may be determined for each coding unit. In this case, reference sample update information may exist for the coding unit. If the reference sample update information for the coding unit indicates that the coding unit is updated, the intra prediction unit 240 uses a reference sample to which a directional gradient is applied for the coding unit corresponding to the reference sample update information. Can perform intra prediction.

As described above, the reference sample update information can be coded within a sequence parameter set, picture parameter set or slice segment header. Also, update information may be coded for a coding unit.

Units of Residual Signal Prediction

In step 2440, the intra residual predictor 2310 may determine whether to perform residual signal prediction for each predefined unit. The predefined unit may be at least one of 1) an entire image sequence (ie, video), 2) one image (ie, picture), 3) a slice, and 4) a coding unit.

The residual signal prediction information may be information indicating whether residual signal prediction is performed for a predefined unit. For example, when the value of the residual signal prediction information is the first value, it may indicate that residual signal prediction should be performed in decoding the current block. When the value of the residual signal prediction information is the second value, it may indicate that residual signal prediction is not performed in decoding the current block.

The encoding apparatus 800 may include encoded residual signal prediction information in a bitstream. The decoding apparatus 2300 may determine whether to perform residual signal prediction on the current block using residual signal prediction information.

Below, residual signal prediction for each of the predefined units is described.

1) Entire video sequence: Whether to perform residual signal prediction may be determined for the entire video sequence. In this case, the sequence parameter set may include residual signal prediction information. If the residual signal prediction information of the sequence parameter set indicates that residual signal prediction is performed, the intra residual predictor 2310 may perform residual signal prediction for the entire video sequence. All blocks encoded by intra prediction in a video sequence may be decoded using feast signal prediction or decoded without using residual signal prediction, depending on whether or not electric signal prediction is performed.

2) One image: Whether to perform residual signal prediction may be determined for each image. In this case, a picture parameter set may include residual signal prediction information. If the residual signal prediction information of the picture parameter set indicates that residual signal prediction is performed, the intra residual predictor 2310 may perform residual signal prediction on all images corresponding to the picture parameter set. All blocks coded by intra prediction within one destination may be decoded using feast signal prediction or decoded without using residual signal prediction, depending on whether or not electric signal prediction is performed.

3) Slice: One image can be divided into a plurality of slice segments or a plurality of tiles within one slice segment. Whether to perform residual signal prediction may be determined for each slice. In this case, the slice segment header may include residual signal prediction information. If the residual signal prediction information of the slice segment header indicates that residual signal prediction is performed, the intra residual predictor 2310 may perform residual signal prediction on the slice corresponding to the slice segment header. When the slice segment header includes residual signal prediction information, all blocks coded by intra prediction at the slice level may be decoded using feast signal prediction or residual signal prediction depending on whether or not full signal prediction is performed. Can be decoded without using.

4) Coding Unit: Whether to perform residual signal prediction may be determined for each coding unit. In this case, residual signal prediction information may exist for the coding unit. If the residual signal prediction information for the coding unit indicates that the coding unit is to be updated, the intra residual predictor 2310 may perform residual signal prediction for the coding unit corresponding to the residual signal prediction information.

As described above, the residual signal prediction information can be coded within a sequence parameter set, a picture parameter set or a slice segment header. Also, update information may be coded for a coding unit.

26 is a structural diagram of an electronic device implementing an encoding device according to an embodiment.

According to an embodiment, the motion estimation unit 111, the motion compensation unit 112, the intra prediction unit 120, the switch 115, the subtractor 125, the transform unit 130, the quantization unit 140, the entropy encoding unit 150, the inverse quantization unit 160, the inverse transform unit 170, the adder 175, and the filter unit 1 of the encoding apparatus 800 80), at least a part of the reference picture buffer 190 and the intra residual predictor 810 may be program modules, and may communicate with an external device or system. Program modules may be included in the encoding device 800 in the form of an operating system, application program modules, and other program modules.

Program modules may be physically stored on various known storage devices. In addition, at least some of these program modules may be stored in a remote storage device capable of communicating with the encoding device 800 .

Program modules may include, but are not limited to, routines, subroutines, programs, objects, components, and data structures that perform functions or operations according to an embodiment or implement abstract data types according to an embodiment.

Program modules may be composed of instructions or codes executed by at least one processor of the encoding device 800 .

The encoding device 800 may be implemented as the electronic device 2600 shown in FIG. 26 . The electronic device 2600 may be a general-purpose computer system operating as the encoding device 800.

As shown in FIG. 26, an electronic device 2600 may include at least one processor 2621, memory 2623, user interface (UI) input device 2626, UI output device 2627, and storage 2628 communicating with each other through a bus 2622. In addition, the electronic device 2600 may further include a network interface 2629 connected to the network 2630 . The processor 2621 may be a semiconductor device that executes processing instructions stored in a central processing unit (CPU), memory 2623, or storage 2628. Memory 2623 and storage 2628 may be various forms of volatile or non-volatile storage media. For example, the memory may include at least one of a ROM 2624 and a RAM 2625 .

The encoding device 800 may be implemented in a computer system including a recording medium that can be read by a computer.

The recording medium may store at least one module required for the electronic device 2600 to operate as the encoding device 800 . The memory 2623 may store at least one module and may be configured to be executed by at least one processor 2621 .

A function related to communication of data or information of the encoding device 800 may be performed through the network interface 2629 .

27 is a structural diagram of an electronic device implementing a decryption device according to an embodiment.

According to an embodiment, at least some of the entropy decoding unit 210, the inverse quantization unit 220, the inverse transform unit 230, the intra prediction unit 240, the motion compensation unit 250, the adder 255, the filter unit 260, the reference picture buffer 270, and the intra residual prediction unit 2310 of the decoding apparatus 2300 may be program modules, Can communicate with a device or system. Program modules may be included in the decryption device 2300 in the form of an operating system, application program modules, and other program modules.

Program modules may be physically stored on various known storage devices. Also, at least some of these program modules may be stored in a remote storage device capable of communicating with the decryption device 2300 .

Program modules may include, but are not limited to, routines, subroutines, programs, objects, components, and data structures that perform functions or operations according to an embodiment or implement abstract data types according to an embodiment.

The program modules may include instructions or codes executed by at least one processor of the decoding device 2300 .

The decryption device 2300 may be implemented as the electronic device 2700 shown in FIG. 27 . The electronic device 2700 may be a general-purpose computer system operating as the decryption device 2300.

As shown in FIG. 27 , an electronic device 2700 may include at least one processor 2721, a memory 2723, a user interface (UI) input device 2726, a UI output device 2727, and a storage 2728 that communicate with each other through a bus 2722. In addition, the electronic device 2700 may further include a network interface 2729 connected to the network 2730. The processor 2721 may be a semiconductor device that executes processing instructions stored in a central processing unit (CPU), memory 2723, or storage 2728. Memory 2723 and storage 2728 may be various forms of volatile or non-volatile storage media. For example, the memory may include at least one of a ROM 2724 and a RAM 2725 .

The decryption device 2300 may be implemented in a computer system including a recording medium that can be read by a computer.

The recording medium may store at least one module required for the electronic device 2700 to operate as the decryption device 2300 . The memory 2723 may store at least one module and may be configured to be executed by at least one processor 2721 .

A function related to communication of data or information of the decoding apparatus 2300 may be performed through the network interface 2729 .

In the foregoing embodiments, the methods are described on the basis of a flowchart as a series of steps or units, but the invention is not limited to the order of steps, and some steps may occur in a different order or concurrently with other steps as described above. In addition, those skilled in the art will understand that the steps shown in the flow chart are not exclusive, other steps may be included, or one or more steps of the flow chart may be deleted without affecting the scope of the present invention.

Embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions such as ROM, RAM, and flash memory. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes such as those produced by a compiler. The hardware device may be configured to act as one or more software modules to perform processing according to the present invention and vice versa.

Although the present invention has been described above by specific details such as specific components and limited embodiments and drawings, this is only provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. Those skilled in the art can make various modifications and variations from these descriptions.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and not only the claims to be described later, but also all modifications equivalent or equivalent to these claims belong to the scope of the spirit of the present invention.

Claims (23)

delete Determining a value of a reference sample for intra prediction of a current block;
generating a prediction block for the current block using the reference sample; and
generating a reconstruction block for the current block using the prediction block and the reconstructed residual block;
Including, The prediction mode of the intra prediction is a DC mode,
The value of the reference sample is determined based on a plurality of pixels in a horizontal line not adjacent to the current block or a plurality of pixels in a vertical line not adjacent to the current block. Determining a value of a reference sample for intra prediction of a current block;
generating a prediction block for the current block using the reference sample; and
generating a reconstruction block for the current block using the prediction block and the reconstructed residual block;
including,
The method of decoding an image in which the value of the reference sample is determined based on at least one pixel of each horizontal line of the plurality of horizontal lines. According to claim 3,
The plurality of horizontal lines are located above the current block. Determining a value of a reference sample for intra prediction of a current block;
generating a prediction block for the current block using the reference sample; and
generating a reconstruction block for the current block using the prediction block and the reconstructed residual block;
The method of decoding an image, wherein the determination of the value of the reference sample is an update of the value of the reference sample. According to claim 5,
wherein the update is performed based on calculation using values of pixels adjacent to a horizontal line included in the current block. delete Determining a value of a reference sample for intra prediction of a current block;
generating a prediction block for the current block using the reference sample; and
generating a reconstruction block for the current block using the prediction block and the reconstructed residual block;
including,
The prediction mode of the intra prediction is a DC mode,
Wherein the value of the reference sample is determined based on a plurality of pixels in a horizontal line not adjacent to the current block or a plurality of pixels in a vertical line not adjacent to the current block. Determining a value of a reference sample for intra prediction of a current block;
generating a prediction block for the current block using the reference sample; and
generating a reconstruction block for the current block using the prediction block and the reconstructed residual block;
including,
The method of encoding an image in which the value of the reference sample is determined based on at least one pixel of each horizontal line of the plurality of horizontal lines. According to claim 9,
The plurality of horizontal lines are an image encoding method located higher than the current block. Determining a value of a reference sample for intra prediction of a current block;
generating a prediction block for the current block using the reference sample; and
generating a reconstruction block for the current block using the prediction block and the reconstructed residual block;
including,
Determination of the value of the reference sample is an update of the value of the reference sample. According to claim 11,
wherein the update is performed based on calculation using values of pixels adjacent to a horizontal line included in the current block. A recording medium on which a bitstream generated by the encoding method according to claim 8 is recorded. delete A computer-readable recording medium storing a bitstream,
The bitstream is
Coded information about the current block
including,
Decoding of the current block is performed using the encoded information;
In the decoding, a value of a reference sample for intra prediction of the current block is determined, a prediction block for the current block is generated using the reference sample, and a reconstructed block for the current block is generated using the prediction block and the reconstructed residual block,
The prediction mode of the intra prediction is a DC mode,
The value of the reference sample is determined based on a plurality of pixels in a horizontal line not adjacent to the current block or a plurality of pixels in a vertical line not adjacent to the current block. Determining a value of a reference sample for intra prediction of a current block;
generating a prediction block for the current block using the reference sample; and
generating a reconstruction block for the current block using the prediction block and the reconstructed residual block;
including,
The value of the reference sample is determined based on at least one pixel of each horizontal line of the plurality of horizontal lines. According to claim 15,
The plurality of horizontal lines are positioned higher than the current block. Determining a value of a reference sample for intra prediction of a current block;
generating a prediction block for the current block using the reference sample; and
generating a reconstruction block for the current block using the prediction block and the reconstructed residual block;
The computer-readable recording medium of claim 1 , wherein the determination of the value of the reference sample is an update of the value of the reference sample. According to claim 18,
The update is performed based on calculation using values of pixels adjacent to a horizontal line included in the current block. delete According to claim 6,
The method of decoding an image in which the update is performed based on whether the current block is divided. According to claim 12,
The method of encoding an image in which the update is performed based on whether the current block is divided. According to claim 19,
The update is performed based on whether the current block is divided or not.