KR20220002207A - Method and apparatus for prediction of residual signal - Google Patents

Abstract

Disclosed are a method and device for performing prediction on a residual signal. An encoding method and an encoding device generate a predicted residual signal by performing additional prediction using a residual signal of a neighboring block on a residual signal of a current block. A decoding method and a decoding device use a residual signal of an already reconstructed neighboring block in generating a reconstructed block of the current block. In encoding and decoding, before generation of a prediction block, a value of a prediction sample used to generate a prediction block may be updated. Efficiency of encoding may be improved through prediction of a residual signal and updating of a prediction sample.

Description

Prediction method and apparatus for residual signal

The following embodiments relate to an image decoding method, a decoding apparatus, an encoding method, and an encoding apparatus, and more particularly, to a method and an apparatus for predicting a residual signal.

With the continuous development of the information and communication industry, a broadcasting service having a high definition (HD) resolution has spread worldwide. Through this proliferation, many users have become accustomed to high-resolution and high-definition images.

In order to satisfy the demand of users for high picture quality, many organizations are spurring the development of next-generation imaging devices. High definition TV (HDTV) and Full HD (FHD) TVs, as well as Ultra High Definition (UHD) TVs having a resolution four times higher than that of FHD TVs Interest has increased, and with this increase in interest, image encoding/decoding technology for an image having higher resolution and image quality is required.

An apparatus and method for encoding/decoding an image include an inter prediction technology, an intra prediction technology, an entropy encoding technology, etc. to perform encoding/decoding on a high-resolution and high-definition image. can be used The inter prediction technique may be a technique of predicting a value of a pixel included in a current picture using a temporally previous picture and/or a temporally later picture. The intra prediction technique may be a technique of predicting a value of a pixel included in the current picture by using information on the pixel in the current picture. The entropy encoding technique may be a technique of allocating a short code to a symbol having a high frequency of occurrence and a technique of assigning a long code to a symbol having a low frequency of occurrence.

In encoding and decoding of an image, prediction may mean generating a prediction signal similar to an original signal. Prediction may be largely classified into prediction referring to a spatially reconstructed image, prediction referring to a temporally reconstructed image, and prediction of other symbols.

Intra prediction may be a prediction technique that allows only spatial reference. The current block may be a block to be currently encoded. Intra prediction may be a method of predicting the current block by referring to reference samples already reconstructed around the current block.

In intra prediction, a neighboring reference sample may be a predicted and reconstructed brightness value rather than a brightness value in the original image, and may be a value before post-processing filtering is applied. Since the reference sample has been previously encoded and reconstructed, it can be used for prediction of the current block by an encoder and a decoder.

However, conceptually, intra prediction may be effective only in a flat region having continuity with respect to a neighboring reference signal and a region having constant directionality. For a region having no such characteristics in an image, the efficiency of intra prediction encoding efficiency may be significantly lower than that of inter prediction encoding efficiency. In particular, when encoding an image, the first picture may have to be encoded only by intra prediction, and the picture may need to be encoded only by intra prediction for random access and error robustness. Accordingly, there is a need for a method capable of improving the encoding efficiency of intra prediction.

An embodiment may provide a method and apparatus for updating a reference sample used for intra prediction to be as close as possible to a current block to be encoded in order to improve intra-encoding efficiency.

An embodiment may provide a method and apparatus for predicting a residual signal of a current block using a residual signal of a neighboring block that has already been encoded or decoded.

In one aspect, an image encoding method comprising: generating a first residual signal of a current block based on a residual signal of a first neighboring block of the current block; and performing encoding of the current block using a first residual signal of the current block.

The video encoding method may further include generating a second residual signal of the current block.

The second residual signal may be a difference between the current block and a prediction block of the current block.

The first residual signal may be generated based on the second residual signal and a residual signal of the neighboring block.

The first residual signal may be generated based on a difference between the second residual signal and a residual signal of the first neighboring block.

The first residual signal may be a difference between the second residual signal and a residual signal of the first neighboring block.

The video encoding method may further include determining whether to perform residual signal prediction.

The generating may be performed when it is determined that the residual signal prediction is to be performed.

The video encoding method may further include encoding information indicating whether the residual signal prediction is performed.

The encoding method may further include encoding the identifier of the first neighboring block.

In another aspect, there is provided a method of decoding an image, the method comprising: generating a prediction block of a current block; and generating a reconstructed block of the current block based on the prediction block, a residual signal of the current block, and a residual signal of a first neighboring block of the current block.

The reconstruction block may be generated based on a sum of the residual signal of the current block and the residual signal of the first neighboring block.

The reconstruction block may be a sum of a prediction block of the current block, a residual signal of the current block, and a residual signal of the first neighboring block.

The image decoding method may further include generating a residual signal of the current block.

The method of decoding the image may further include identifying the first neighboring block.

The first neighboring block may be identified by the identifier of the first neighboring block.

When the identifier of the first neighboring block does not exist, the selected block may be identified as the first neighboring block according to a predefined method.

The image decoding method may further include updating a value of a reference sample used to generate the prediction block.

In still another aspect, there is provided a method of decoding an image, the method comprising: determining a value of a reference sample based on a neighboring block of a current block; and generating a prediction block of the current block by using the reference sample.

The value of the reference sample may be determined based on the gradient pattern of the neighboring block.

When the gradient pattern is symmetrical, the value of the reference sample may be determined based on gradient values of two lines that form symmetry of the gradient pattern.

The value of the reference sample may be determined based on a gradient value between two adjacent reference samples among a plurality of reference samples belonging to one row of the neighboring block.

In the determining, the value of the reference sample may be changed from a value before the update to a value after the update.

The value before the update may be a value generated as the block including the reference sample is predicted and reconstructed.

When the intra prediction mode for the current block is a horizontal prediction mode, the reference sample is a sample adjacent to the left of the current block, and the adjacent block is an upper-left adjacent block of the current block and an upper-left adjacent block of the current block. It may be a combined block.

In still another aspect, there is provided a method for encoding an image, the method comprising: determining a value of a reference sample based on blocks adjacent to a current block; and generating a prediction block of the current block by using the reference sample.

In order to improve the efficiency of intra encoding, a method and apparatus for updating a reference sample used for intra prediction to be as close as possible to a current block to be encoded are provided.

A method and apparatus for predicting a residual signal of a current block by using a residual signal of an already encoded or decoded neighboring block are provided.

1 is a block diagram illustrating a configuration of an encoding apparatus to which the present invention is applied according to an embodiment.
2 is a block diagram illustrating a configuration of a decoding apparatus to which the present invention is applied according to an embodiment.
3 is a diagram schematically illustrating a structure of an image segmentation when an image is encoded and decoded.
4A to 7H are diagrams illustrating a form of a prediction unit (PU) that a coding unit (CU) may include.
5 is a diagram illustrating a form of a transform unit (TU) that may be included in a coding unit (CU).
6 is a diagram for explaining an embodiment of an intra prediction process.
7 is a diagram for explaining an embodiment of an inter prediction process.
8 is a structural diagram of an encoding apparatus according to an embodiment.
9A and 9B are flowcharts of an encoding method according to an embodiment.
10A is a flowchart of an encoding method according to an embodiment.
10B is a flowchart of a method for updating a reference sample according to an embodiment.
10C is a flowchart of a method for predicting a residual signal according to an embodiment.
10D is a flowchart of a method for encoding a current block according to an embodiment.
11 illustrates a current block and a reference sample according to an example.
12 illustrates a method of updating a reference sample by reflecting a horizontal gradient of a neighboring block according to an example.
13 illustrates a method of obtaining a gradient pattern according to an example.
14 illustrates a method of updating a reference sample by reflecting a vertical gradient of a neighboring block according to an example.
15A shows intra prediction with 33 respective modes according to an example.
15B shows intra prediction with 65 respective modes according to an example.
16 illustrates a region of an image according to an example.
17 illustrates a method of calculating a residual signal of a neighboring block according to an example.
18 illustrates a method of calculating a residual signal of a current block according to an example.
19 illustrates a residual signal prediction method according to an example.
20A illustrates a default residual signal according to an example.
20B illustrates a result of a discrete cosine transform on a default residual signal according to an example.
21A illustrates a proposed residual signal according to an example.
21B shows the result of discrete cosine transform on the proposed residual signal according to an example.
22 illustrates a location of a neighboring block according to an example.
23 is a structural diagram of a decoding apparatus according to an embodiment.
24A and 24B are flowcharts of a decoding method according to an embodiment.
25A is a flowchart of a method for generating a residual signal according to an embodiment.
25B is a flowchart of a method of decoding a current block using a feast signal according to an embodiment.
25C is a flowchart of a method for generating a prediction block according to an embodiment.
25D is a flowchart of a method for generating a reconstructed block according to an embodiment.
26 is a structural diagram of an electronic device implementing an encoding device according to an embodiment.
27 is a structural diagram of an electronic device implementing a decoding device according to an embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] Reference is made to the accompanying drawings, which illustrate specific embodiments by way of example. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that various embodiments are different, but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein with respect to one embodiment may be implemented in other embodiments without departing from the spirit and scope of the invention. In addition, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the embodiment. Accordingly, the detailed description set forth below is not intended to be taken in a limiting sense, and the scope of exemplary embodiments, if properly described, is limited only by the appended claims, along with all scope equivalents to those as claimed.

Like reference numerals in the drawings refer to the same or similar functions throughout the various aspects. The shapes and sizes of elements in the drawings may be exaggerated for clearer description.

When a component is referred to as being “connected” or “connected” to another component, the two components may be directly connected or connected to each other, but in the above 2 It should be understood that other components may exist in the middle of the components. In addition, the description of "including" a specific configuration in the exemplary embodiments does not exclude components other than the above specific configuration, and additional configuration is the implementation of the exemplary embodiments or the technical spirit of the exemplary embodiments. It means that it can be included in the scope.

Terms such as first and second may be used to describe various components, but the above components should not be limited by the above terms. The above terms are used to distinguish one component from another component. For example, without departing from the scope of rights, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

In addition, the components shown in the embodiments are shown independently to represent different characteristic functions, and it does not mean that each component is made of separate hardware or only one software component unit. That is, each component is listed as each component for convenience of description. For example, at least two components among the components may be combined into one component. Also, one component may be divided into a plurality of components. Integrated embodiments and separate embodiments of each of these components are also included in the scope of rights without departing from the essence.

In addition, some of the components are not essential components to perform essential functions, but may be optional components only to improve performance. Embodiments may be implemented including only components essential for implementing the essence of the embodiment, and for example, structures in which optional components are excluded, such as components used only for performance improvement, are also included in the scope of rights.

Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings in order to enable those of ordinary skill in the art to easily implement the embodiments. In describing the embodiments, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present specification, the detailed description thereof will be omitted.

Hereinafter, an image may mean one picture constituting a video, or may indicate a video itself. For example, "encoding and/or decoding of an image" may mean "encoding and/or decoding of a video", and may mean "encoding and/or decoding of one image among images constituting a video". may be

First, terms used in the embodiments will be described.

Unit: In encoding and decoding of an image, a unit may be an area generated by segmentation of one image. One image may be divided into a plurality of units. In encoding and decoding of an image, a predefined process may be performed for each unit. Depending on the function, units such as a block, a macro block, a coding unit (CU), a prediction unit (PU), and a transform unit (TU) are used. One unit may be further divided into sub-units having a smaller size compared to the unit.

- The block division information may include information about the depth of the unit. The depth information may indicate the number and/or degree to which a unit is divided.

- One unit may be hierarchically divided into a plurality of sub-units while having depth information based on a tree structure. In other words, a unit and a sub-unit generated by division of the unit may correspond to a node and a child node of the node, respectively. Each divided sub-unit may have depth information. Since the depth information of the unit indicates the number and/or degree of division of the unit, the division information of the sub-unit may include information about the size of the sub-unit.

- In the tree structure, the highest node may correspond to the first undivided unit. The uppermost node may be referred to as a root node. Also, the highest node may have a minimum depth value. In this case, the highest node may have a depth of level 0.

- A node with a depth of level 1 may represent a unit created as the original unit is split once. A node with a depth of level 2 may represent a unit generated as the original unit is split twice.

- A leaf node having a depth of level 3 may indicate a unit generated as the initial unit is divided three times. A leaf node may be the lowest node and may have a maximum depth value.

- Block: A block may be an MxN array of samples. M and N may each be a positive integer. A block can often mean an array of two-dimensional samples. A sample may be a pixel or a pixel value.

Transform Unit: A transform unit may be a basic unit in residual signal encoding and/or residual signal decoding such as transform, inverse transform, quantization, inverse quantization, transform coefficient encoding, and transform coefficient decoding. . One transform unit may be divided into multiple transform units having a smaller size.

Parameter Set: A parameter set may correspond to header information among structures in a bitstream. For example, the parameter set may include a sequence parameter set, a picture parameter set, an adaptation parameter set, and the like.

Rate-distortion optimization: The encoding apparatus uses a combination of a size of a coding unit, a prediction mode, a size of a prediction unit, motion information, and a size of a transform unit to provide high encoding efficiency. Distortion optimization can be used.

- The rate-distortion optimization method may calculate a rate-distortion cost of each combination in order to select an optimal combination from among the above combinations. The rate-distortion cost can be calculated using Equation 1 below. In general, the combination in which the rate-distortion cost is minimized may be selected as the optimal combination in the rate-distortion optimization method.

D may represent distortion. D may be a mean square error of squares of difference values between original transform coefficients and reconstructed transform coefficients in a transform block.

R may represent a rate. R may represent a bit rate using related context information.

λ may represent a Lagrangian multiplier. R may include not only encoding parameter information such as prediction mode, motion information, and a coded block flag, but also bits generated by encoding a transform coefficient.

The encoding apparatus performs processes such as inter prediction and/or intra prediction, transformation, quantization, entropy encoding, inverse quantization, and inverse transformation to calculate accurate D and R. These processes may significantly increase the complexity of the encoding apparatus. have.

1 is a block diagram illustrating a configuration of an encoding apparatus to which the present invention is applied according to an embodiment.

The encoding apparatus 100 may be a video encoding apparatus or an image encoding apparatus. A video may include one or more images. The encoding apparatus 100 may sequentially encode one or more images of a video according to time.

Referring to FIG. 1 , the encoding apparatus 100 includes a motion prediction unit 111 , a motion compensation unit 112 , an intra prediction unit 120 , a switch 115 , a subtractor 125 , a transform unit 130 , and a quantization unit. It may include a unit 140 , an entropy encoder 150 , an inverse quantizer 160 , an inverse transform unit 170 , an adder 175 , a filter unit 180 , and a reference picture buffer 190 .

The encoding apparatus 100 may encode an input image in an intra mode and/or an inter mode. Also, the encoding apparatus 100 may generate a bitstream through encoding for an input device, and may output the generated bitstream. When the intra mode is used, the switch 115 may be switched to the intra mode, and when the inter mode is used, the switch 115 may be switched to the inter mode.

The encoding apparatus 100 may generate a prediction block for the input block of the input image. Also, after the prediction block is generated, the encoding apparatus 100 may encode a residual between the input block and the prediction block. The input image may be referred to as a current image that is a current encoding target. The input block may be referred to as a current block that is a current encoding target.

When the prediction mode is the intra mode, the intra prediction unit 120 may use a pixel value of an already-encoded block adjacent to the current block as a reference pixel. The intra prediction unit 120 may perform spatial prediction using a reference pixel, and may generate prediction samples for an input block through spatial prediction.

When the prediction mode is the inter mode, the motion prediction unit 111 may search for a region that best matches the input block from the reference image in the motion prediction process, and may derive a motion vector using the searched region. . The reference picture may be stored in the reference picture buffer 190 , and may be stored in the reference picture buffer 190 when encoding and/or decoding of the reference picture is processed.

The motion compensator 112 may generate a prediction block by performing motion compensation using a motion vector. Here, the motion vector may be a two-dimensional vector used for inter prediction. Also, the motion vector may indicate an offset between the current image and the reference image.

The subtractor 125 may generate a residual block by using the difference between the input block and the prediction block. The residual block may be referred to as a residual signal.

The transform unit 130 may perform transform on the residual block to generate a transform coefficient, and may output a transform coefficient. Here, the transform coefficient may be a coefficient value generated by performing transform on the residual block. When a transform skip mode is applied, the transform unit 130 may skip transform on the residual block.

A quantized transform coefficient level may be generated by applying quantization to the transform coefficient. Hereinafter, in embodiments, a quantized transform coefficient level may also be referred to as a transform coefficient.

The quantization unit 140 may generate a quantized transform coefficient level by quantizing the transform coefficient according to the quantization parameter, and may output the quantized transform coefficient level. In this case, the quantization unit 140 may quantize the transform coefficients using a quantization matrix.

The entropy encoding unit 150 may generate a bitstream by performing entropy encoding according to a probability distribution on the values calculated by the quantization unit 140 or encoding parameter values calculated in the encoding process, etc. can be printed out.

The entropy encoding unit 150 may perform entropy encoding on information for decoding an image in addition to information on pixels of the image. For example, information for decoding an image may include a syntax element or the like.

The encoding parameter may be information required for encoding and/or decoding. The encoding parameter may include information encoded by the encoding apparatus and transmitted to the decoding apparatus, and may include information that can be inferred during encoding or decoding. For example, as information transmitted to the decoding apparatus, there is a syntax element.

For example, the coding parameter includes a prediction mode, a motion vector, a reference picture index, a coding block pattern, presence or absence of a residual signal, a transform coefficient, a quantized transform coefficient, a quantization parameter, a block size, and a block partition. It may include values or statistics such as information. The prediction mode may indicate an intra prediction mode or an inter prediction mode.

The residual signal may mean a difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming a difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming and quantizing a difference between the original signal and the predicted signal. The residual block may be a residual signal in units of blocks.

When entropy encoding is applied, a small number of bits may be allocated to a symbol having a high probability of occurrence, and a large number of bits may be allocated to a symbol having a low probability of occurrence. As a symbol is expressed through this allocation, the size of a bitstring for symbols to be encoded may be reduced. Accordingly, compression performance of image encoding may be improved through entropy encoding.

In addition, encoding methods such as exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be used for entropy encoding. For example, the entropy encoding unit 150 may perform entropy encoding using a Variable Length Coding/Code (VLC) table. For example, the entropy encoder 150 may derive a binarization method for a target symbol. Also, the entropy encoder 150 may derive a probability model of a target symbol/bin. The entropy encoding unit 150 may perform entropy encoding using the derived binarization method or a probability model.

When the encoding apparatus 100 performs encoding through inter prediction, the encoded current image may be used as a reference image for other image(s) to be processed later. Accordingly, the encoding apparatus 100 may decode the encoded current image again and store the decoded image as a reference image. Inverse quantization and inverse transformation of the encoded current image for decoding may be processed.

The quantized coefficient may be inversely quantized in the inverse quantization unit 160 . It may be inversely transformed by the inverse transform unit 170 . The inverse quantized and inverse transformed coefficients may be summed with the prediction block through the adder 175. A reconstructed block may be generated by summing the inverse quantized and inverse transformed coefficients and the prediction block.

The reconstruction block may go through the filter unit 180 . The filter unit 180 may apply at least one of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to a reconstructed block or a reconstructed picture. The filter unit 180 may be referred to as an adaptive in-loop filter.

The deblocking filter may remove block distortion generated at the boundary between blocks. The SAO may add an appropriate offset value to the pixel value to compensate for the coding error. The ALF may perform filtering based on a comparison value between the reconstructed image and the original image. The reconstructed block that has passed through the filter unit 180 may be stored in the reference picture buffer 190 .

2 is a block diagram illustrating a configuration of a decoding apparatus to which the present invention is applied according to an embodiment.

The decoding apparatus 200 may be a video decoding apparatus or an image decoding apparatus.

Referring to FIG. 2 , the decoding apparatus 200 includes an entropy decoding unit 210 , an inverse quantization unit 220 , an inverse transform unit 230 , an intra prediction unit 240 , a motion compensator 250 , and an adder 255 . , a filter unit 260 and a reference picture buffer 270 may be included.

The decoding apparatus 200 may receive the bitstream output from the encoding apparatus 100 . The decoding apparatus 200 may decode the bitstream in an intra mode or an inter mode. Also, the decoding apparatus 200 may generate a reconstructed image through decoding and may output the reconstructed image.

When the prediction mode used for decoding is the intra mode, the switch may be switched to the intra mode. When the prediction mode used for decoding is the inter mode, the switch may be switched to the inter mode.

The decoding apparatus 200 may obtain a reconstructed residual block from the input bitstream and generate a prediction block. When the reconstructed residual block and the prediction block are obtained, the decoding apparatus 200 may generate a reconstructed block by adding the reconstructed residual block and the prediction block.

The entropy decoding unit 210 may generate symbols by performing entropy decoding according to a probability distribution on the bitstream. The generated symbols may include symbols in the form of quantized coefficients. Here, the entropy decoding method may be similar to the entropy encoding method described above. For example, the entropy decoding method may be a reverse process of the entropy encoding method described above.

The quantized coefficient may be inverse quantized by the inverse quantization unit 220 and inversely transformed by the inverse transform unit 230 . As a result of inverse quantization and inverse transformation of the quantized coefficient, a reconstructed residual block may be generated. In this case, the inverse quantizer 220 may apply a quantization matrix to the quantized coefficients.

When the intra mode is used, the intra prediction unit 240 may generate a prediction block by performing spatial prediction using pixel values of an already encoded block around the current block. When the inter mode is used, the motion compensator 250 may generate a prediction block by performing motion compensation using a motion vector and a reference image stored in the reference picture buffer 270 .

The reconstructed residual block and the prediction block may be added through an adder 255 . A block generated by adding the reconstructed residual block and the prediction block may pass through the filter unit 260 . The filter unit 260 may apply at least one of a deblocking filter, SAO, and ALF to a reconstructed block or a reconstructed picture. The filter unit 260 may output a restored image. The reconstructed image may be stored in the reference picture buffer 270 and used for inter prediction.

3 is a diagram schematically illustrating a structure of an image segmentation when an image is encoded and decoded.

In order to efficiently segment an image, a coding unit (CU) may be used in encoding and decoding. A unit may be a term that collectively refers to a block including 1) a syntax element and 2) image samples. For example, "division of a unit" may mean "division of a block corresponding to a unit".

Referring to FIG. 3 , an image 300 is sequentially divided in units of a Largest Coding Unit (LCU), and a division structure is determined in units of LCUs. Here, LCU may be used in the same meaning as a Coding Tree Unit (CTU).

The partition structure may mean a distribution of coding units (CUs) within the LCU 310 . A CU may be a unit for efficiently encoding an image. This distribution may be determined according to whether one CU is divided into four CUs. The horizontal size and vertical size of the CU generated by division may be half of the horizontal size and half of the vertical size of the CU before division, respectively. The divided CU may be recursively divided into four CUs whose horizontal size and vertical size are reduced by half in the same manner.

In this case, the division of the CU may be performed recursively to a predefined depth. The depth information may be information indicating the size of a CU. Depth information may be stored for each CU. For example, the depth of the LCU may be 0, and the depth of a Smallest Coding Unit (SCU) may be a predefined maximum depth. Here, the LCU may be a coding unit having the largest coding unit size as described above, and the SCU may be a coding unit having the smallest coding unit size.

The division starts from the LCU 310, and the depth of the CU increases by 1 whenever the horizontal size and the vertical size of the CU are halved by the division. For each depth, an undivided CU may have a size of 2Nx2N. In the case of a divided CU, a CU having a size of 2Nx2N may be divided into four CUs having a size of NxN. The size of N is halved for each increase in depth by 1.

Referring to FIG. 3 , an LCU having a depth of 0 may be 64x64 pixels. 0 may be the minimum depth. An SCU of depth 3 may be 8x8 pixels. 3 may be the maximum depth. In this case, a CU of 64x64 pixels, which is an LCU, may be expressed as depth 0. A CU of 32x32 pixels may be expressed as depth 1. A CU of 16x16 pixels may be expressed as depth 2. A CU of 8x8 pixels, which is an SCU, may be expressed as depth 3 .

Also, information on whether a CU is split may be expressed through split information of the CU. The division information may be 1-bit information. All CUs except for the SCU may include partition information. For example, if the value of the partitioning information is 0, the CU may not be partitioned, and if the value of the partitioning information is 1, the CU may be partitioned.

4A to 4H are diagrams illustrating a form of a prediction unit (PU) that a coding unit (CU) may include.

A CU that is no longer split among CUs split from an LCU may be divided into one or more prediction units (PUs). This process may also be referred to as segmentation.

A PU may be a basic unit for prediction. The PU may be encoded and decoded in any one of a skip mode, an inter mode, and an intra mode. A PU may be divided into various forms according to modes.

As shown in FIG. 4A , in the skip mode, there may not be a partition in the CU. In the skip mode, a 2Nx2N mode 410 having the same size as a CU without division may be supported.

In the inter mode, 8 types of divided types within a CU may be supported. For example, in the inter mode, 2Nx2N mode 410, 2NxN mode 415, Nx2N mode 420, NxN mode 425, 2NxnU mode 430, 2NxnD mode 435, nLx2N mode 440, and nRx2N mode Mode 445 may be supported.

In the intra mode, the 2Nx2N mode 410 and the NxN mode 425 may be supported.

5 is a diagram illustrating a form of a transform unit (TU) that may be included in a coding unit (CU).

A transform unit (TU) may be a basic unit used for processes of transform, quantization, inverse transform, and inverse quantization within a CU. A TU may have a square shape or a rectangular shape.

Among CUs split from an LCU, a CU that is no longer split into CUs may be split into one or more TUs. In this case, the partition structure of the TU may be a quad-tree structure. For example, as shown in FIG. 5 , one CU 510 may be divided one or more times according to the quadtree structure. Through division, one CU 510 may be configured with TUs of various sizes.

6 is a diagram for explaining an embodiment of an intra prediction process.

The number of intra prediction modes may be fixed to 35 regardless of the size of the prediction unit.

The prediction mode may include two non-directional modes and 33 directional modes as shown in FIG. 6 . The two non-directional modes may include a DC mode and a planar mode.

The number of prediction modes may be different depending on the type of color component. For example, the number of prediction modes may be different depending on whether the color component is a luma signal or a chroma signal.

The PU may have a square shape, having a size of NxN or a size of 2Nx2N. The size of NxN may include 4x4, 8x8, 16x16, 32x32 and 64x64. The unit of PU may be the size of at least one of a CU, a PU, and a TU.

Intra encoding and/or decoding may be performed using sample values or encoding parameters included in neighboring reconstructed units.

7 is a diagram for explaining an embodiment of an inter prediction process.

The rectangle shown in FIG. 7 may represent an image (or a picture). Also, an arrow in FIG. 7 may indicate a prediction direction. That is, the image may be encoded and/or decoded according to the prediction direction.

Each image (or picture) may be classified into an I picture (intra picture), a P picture (uni-prediction picture), and a B picture (bi-prediction picture) according to an encoding type. Each picture may be encoded according to the encoding type of each picture.

When an image to be encoded is an I picture, the image may be encoded with respect to the image itself without inter prediction. When an image to be encoded is a P picture, the image may be encoded through inter prediction using a reference image only in the forward direction. When an image to be encoded is a B picture, it may be encoded through inter prediction using reference pictures in both forward and backward directions, and may be encoded through inter prediction using reference pictures in one of the forward and backward directions.

Images of P-pictures and B-pictures that are encoded and/or decoded using a reference image may be regarded as images for which inter prediction is used.

Hereinafter, inter prediction according to an embodiment will be described in detail.

Inter prediction may be performed using a reference picture and motion information. In addition, inter prediction may use the skip mode described above.

The reference picture may be at least one of a previous picture of the current picture or a subsequent picture of the current picture. In this case, the inter prediction may perform prediction on the block of the current picture based on the reference picture. Here, the reference picture may mean an image used for prediction of a block.

In this case, the region in the reference picture may be specified by using a reference picture index (refIdx) indicating the reference picture and a motion vector to be described later.

Inter prediction may select a reference picture and a reference block corresponding to the current block within the reference picture, and may generate a prediction block for the current block using the selected reference block. The current block may be a block to be currently encoded or decoded among blocks of the current picture.

Motion information may be derived during inter prediction by each of the encoding apparatus 100 and the decoding apparatus 200 . In addition, the derived motion information may be used to perform inter prediction.

In this case, the encoding apparatus 100 and the decoding apparatus 200 increase encoding and/or decoding efficiency by using motion information of a reconstructed neighboring block and/or motion information of a collocated block. can be improved The collocated block may be a block corresponding to the current block in an already reconstructed collocated picture (col picture). The reconstructed neighboring block may be a block in the current picture and already reconstructed through encoding and/or decoding. Also, the reconstructed block may be a neighboring block adjacent to the current block and/or a block located at an outer corner of the current block. Here, the block located at the outer corner of the current block may be a block vertically adjacent to a neighboring block horizontally adjacent to the current block or a block horizontally adjacent to a neighboring block vertically adjacent to the current block.

Each of the encoding apparatus 100 and the decoding apparatus 200 may determine a block existing at a position spatially corresponding to the current block in the collocated picture, and may determine a predefined relative position based on the determined block. The predefined relative position may be a position inside and/or outside a block that exists spatially at a position corresponding to the current block. Also, each of the encoding apparatus 100 and the decoding apparatus 200 may derive a collocated block based on the determined and predefined relative positions. Here, the collocated picture may be one picture from among at least one reference picture included in the reference picture list.

A method of deriving motion information may change according to a prediction mode of the current block. For example, as a prediction mode applied for inter prediction, there may be an advanced motion vector predictor (AMVP) and a merge.

For example, when AMVP is applied as a prediction mode, each of the encoding apparatus 100 and the decoding apparatus 200 uses a motion vector of a reconstructed neighboring block and/or a motion vector of a collocated block as a prediction motion vector candidate. You can create a list. A motion vector of a reconstructed neighboring block and/or a motion vector of a collocated block may be used as a prediction motion vector candidate.

The bitstream generated by the encoding apparatus 100 may include a prediction motion vector index. The prediction motion vector index may indicate an optimal prediction motion vector selected from prediction motion vector candidates included in the prediction motion vector candidate list. The prediction motion vector index may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through the bitstream.

The decoding apparatus 200 may select the prediction motion vector of the current block from among the prediction motion vector candidates included in the prediction motion vector candidate list by using the prediction motion vector index.

The encoding apparatus 100 may calculate a motion vector difference (MVD) between the motion vector of the current block and the predicted motion vector, and may encode the MVD. The bitstream may include an encoded MVD. The MVD may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. In this case, the decoding apparatus 200 may decode the received MVD. The decoding apparatus 200 may derive the motion vector of the current block through the sum of the decoded MVD and the predicted motion vector.

The bitstream may include a reference picture index indicating a reference picture, and the like. The reference picture index may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. The decoding apparatus 200 may predict the motion vector of the current block using motion information of the neighboring block, and may derive the motion vector of the current block using the predicted motion vector and the motion vector difference. The decoding apparatus 200 may generate a prediction block for the current block based on the derived motion vector and reference picture index information.

Another example of a method of deriving motion information is a merge. Merge may refer to merging of motions for a plurality of blocks. Merge may mean applying motion information of one block to another block as well. When merge is applied, each of the encoding apparatus 100 and the decoding apparatus 200 may generate a merge candidate list using motion information of a reconstructed neighboring block and/or motion information of a collocated block. have. The motion information may include at least one of 1) a motion vector, 2) an index to a reference image, and 3) a prediction direction. The prediction direction may be unidirectional or bidirectional.

In this case, the merge may be applied in units of CUs or units of PUs. When the merge is performed in units of CUs or PUs, the encoding apparatus 100 may transmit predefined information to the decoding apparatus 200 through a bitstream. The bitstream may include predefined information. The predefined information may include 1) information indicating whether to perform a merge for each block partition, and 2) information on which block to be merged with among neighboring blocks adjacent to the current block. For example, the neighboring blocks of the current block may include a left neighboring block of the current block, an upper neighboring block of the current block, and a temporal neighboring block of the current block.

The merge candidate list may indicate a list in which motion information is stored. Also, the merge candidate list may be generated before merging is performed. The motion information stored in the merge candidate list may be 1) motion information of a neighboring block adjacent to the current block or 2) motion information of a block collocated to the current block in a reference image. Also, the motion information stored in the merge candidate list may be new motion information generated by a combination of motion information already existing in the merge candidate list.

The skip mode may be a mode in which information of a neighboring block is applied to the current block as it is. The skip mode may be one of modes used for inter prediction. When the skip mode is used, the encoding apparatus 100 may transmit only information about which block of motion information to use as the motion information of the current block to the decoding apparatus 200 through the bitstream. The encoding apparatus 100 may not transmit other information to the decoding apparatus 200 . For example, the other information may be syntax information. The syntax information may include motion vector difference information.

Residual signal prediction is described in the embodiments below. In general, in existing video encoding and/or decoding technologies such as High Efficiency Video Coding (HEVC) and Advanced Video Coding (AVC), in order to encode and/or decode a current block, the current block A residual signal of When the residual signal is generated, the residual signal predicted once more may be generated through residual signal prediction using the residual signal of a block adjacent to the current block.

The residual signal prediction may be to predict a residual signal of a current block with respect to a residual signal of a neighboring block. Alternatively, the residual signal prediction may be to use a difference between the residual signal of the current block and the residual signal of a neighboring block as a new residual signal of the current block.

A residual signal obtained through residual signal prediction may have an advantage in terms of encoding efficiency compared to a residual signal obtained using a conventional intra prediction method. For example, the amount of bit generation for the residual signal can be reduced by using a residual signal obtained using residual signal prediction.

In the following embodiments, a residual signal obtained using residual signal prediction may be referred to as a first residual signal, and a residual signal obtained using a conventional intra prediction method may be referred to as a second residual signal.

In addition, the updating of the reference sample is described in the embodiments below. A reference sample is used to generate a predictive block. Accordingly, if the reference sample has characteristics similar to the expected characteristics of the current block to be encoded or decoded, the efficiency of encoding or decoding may be improved. Hereinafter, embodiments of updating a reference sample according to a predefined condition before generation of a prediction block are described.

8 is a structural diagram of an encoding apparatus according to an embodiment.

The encoding apparatus 800 may correspond to the above-described encoding apparatus 100 . The encoding apparatus 800 includes a motion prediction unit 111 , a motion compensation unit 112 , an intra prediction unit 120 , a switch 115 , a subtractor 125 , a transform unit 130 , a quantization unit 140 , and entropy. It may include an encoder 150 , an inverse quantizer 160 , an inverse transform unit 170 , an adder 175 , a filter unit 180 , and a reference picture buffer 190 , and an intra residual predictor 810 . may include more.

Motion prediction unit 111, motion compensation unit 112, intra prediction unit 120, switch 115, subtractor 125, transform unit 130, quantization unit 140, entropy encoding unit 150, The inverse quantizer 160, the inverse transform unit 170, the adder 175, the filter unit 180, and the reference picture buffer 190 may perform the same functions and/or operations as described above with reference to FIG. 1 . . A duplicate description will be omitted.

In addition, the motion predictor 111 , the motion compensator 112 , the intra predictor 120 , the switch 115 , the subtractor 125 , the transform unit 130 , the quantizer 140 , and the entropy encoder 150 . ), the inverse quantizer 160 , the inverse transform unit 170 , the adder 175 , the filter unit 180 , and the reference picture buffer 190 perform functions and/or operations related to the intra residual prediction unit 810 . can Motion compensator 112, intra prediction unit 120, switch 115, subtractor 125, transform unit 130, quantization unit 140, entropy encoding unit 150, inverse quantization unit 160, The functions and/or operations of the inverse transform unit 170 , the adder 175 , the filter unit 180 , the reference picture buffer 190 , and the intra residual prediction unit 810 will be described in detail below.

The intra residual predictor 810 may not be distinguished from the intra predictor 120 . The intra prediction unit 120 and the intra residual prediction unit 810 may be integrated into the intra prediction unit 120 . In embodiments, functions and/or operations described as being performed by the intra residual predictor 810 may be performed by the intra predictor 120 .

Also, the decoding apparatus 2300 corresponding to the encoding apparatus 800 according to an embodiment will be described in detail below with reference to FIG. 23 .

9A and 9B are flowcharts of an encoding method according to an embodiment.

Hereinafter, the current block may be a block to be currently encoded or may be a block in the current image.

First, referring to FIG. 9A , step 910 may be performed.

Prior to performing step 910, a reference sample may be generated. Generation of a reference sample according to an example will be described in detail below with reference to FIG. 11 .

In generating the reference sample, a reference sample generating method of an existing image encoding and/or decoding technique, such as high-efficiency video coding and enhanced video coding, may be used.

In operation 910 , the intra predictor 120 may determine whether to update the reference sample. Here, the updating of the reference sample may be refining the sample value of the reference sample used for generation of the prediction block before generation of the prediction block of the current block.

If it is determined to perform an update of the reference sample, step 915 may be performed. If it is determined not to perform the reference sample update, steps 920 and 970 may be performed.

In operation 915 , the intra prediction unit 120 may update the value of the reference sample, and through the update, determine the value of the reference sample used to generate the prediction block of the current block.

The intra prediction unit 120 may update the value of the reference sample according to the directional pattern of the surrounding sample.

The update of the reference sample according to an example will be described in detail below with reference to FIGS. 12, 13 and 14 .

After step 915 is executed, steps 920 and 970 may be performed.

In operation 920, the intra prediction unit 120 may perform intra prediction. The intra prediction unit 120 may generate a prediction block of the current block. The intra prediction unit 120 may generate the prediction block of the current block by using the reference sample according to the intra prediction mode for the current block.

For example, in generating a prediction block, a method of generating a prediction block of an existing image encoding and/or decoding technology such as HEVC and AVC may be used.

After step 920 is performed, step 930 may be performed.

In operation 930 , the intra residual predictor 810 may determine whether to perform residual signal prediction.

If it is determined to perform residual signal prediction, steps 935 , 940 , and 980 may be performed.

If it is determined not to perform residual signal prediction, steps 960 and 980 may be performed.

In operation 935 , the intra residual predictor 810 may determine a first neighboring block from among one or more neighboring blocks of the current block. The first neighboring block may be a block used for residual signal prediction. Determination of a neighboring block according to an example will be described in detail below with reference to FIG. 20 .

After step 935 is performed, steps 950 and 985 may be performed.

In operation 940 , the intra prediction unit 120 may generate a second residual signal of the current block using intra prediction. The intra prediction unit 120 may generate a second residual signal of the current block based on the intra prediction mode and the reference sample for the current block.

The second residual signal may correspond to a residual signal of a current block in existing image encoding and/or decoding techniques such as HEVC and AVC. For example, in generating the second residual signal, a residual signal generating method of an existing image encoding and/or decoding technique such as HEVC and AVC may be used.

Generation of the second residual signal according to an example will be described in detail below with reference to FIGS. 16 , 17 , and 18 .

After step 940 is performed, step 950 may be performed.

If steps 935 and 940 are performed as it is determined that residual signal prediction is to be performed, then step 950 may be performed.

In operation 950 , the intra residual predictor 810 may generate a first residual signal of the current block according to residual signal prediction.

The intra residual predictor 810 may use the residual signal of the first neighboring block to predict the residual signal.

The intra residual predictor 810 may generate a first residual signal of the current block based on the second residual signal of the current block and the residual signal of the first neighboring block.

The first residual signal may be a difference between the second residual signal of the current block and the residual signal of the first neighboring block. Alternatively, the first residual signal may be a result of subtracting the residual signal of the first neighboring block from the second residual signal of the current block. The intra residual predictor 810 may generate a difference between the second residual signal of the current block and the residual signal of the first neighboring block as the first residual signal.

The first residual signal may be a residual signal of a current block generated by residual signal prediction. Alternatively, the first residual signal may be a residual signal of a current block generated based on a residual signal of a first neighboring block of the current block.

By applying residual signal prediction to the second residual signal generated in operation 940 , more efficient encoding may be possible with respect to the current block.

Generation of the first residual signal according to an example will be described in detail below with reference to FIG. 19 .

If it is determined that the residual signal prediction is not to be performed, in operation 960 , the intra predictor 120 or the intra residual predictor 810 may generate a third residual signal of the current block.

The third residual signal may be a residual signal of a current block in conventional image encoding and/or decoding techniques such as HEVC and AVC. For example, in generating the third residual signal, a residual signal generating method of an existing image encoding and/or decoding technique such as HEVC and AVC may be used.

Also, the third residual signal may be the same signal as the second residual signal in step 940 . In other words, steps 940 and 960 may generate the residual signal of the current block in the same manner.

In operation 970 , the intra prediction unit 120 may encode information indicating whether the update of the reference sample is performed.

Unlike the existing intra prediction, when the reference sample is updated, the intra prediction unit 120 provides the information so that the decoding apparatus 2300, which will be described later with reference to FIG. 23, knows whether or not the reference sample is updated. can be coded. For example, if the update of the reference sample is used, the value of the information may be set to '1', and if the update of the reference sample is not used, the value of the information may be set to '0'.

For example, the intra predictor 120 may indicate whether the reference sample is updated using a flag, and whether the reference sample is updated may be indicated through the flag.

In operation 980 , the intra residual predictor 810 may encode information indicating whether residual signal prediction is performed.

When the residual signal prediction is performed, the intra residual predictor 810 may encode the information so that the decoding apparatus 2300, which will be described later with reference to FIG. 23, knows whether the residual signal is predicted. For example, if the update of the reference sample is used, the value of the information may be set to '1', and if the update of the reference sample is not used, the value of the information may be set to '0'.

For example, the intra prediction unit 120 may indicate whether the reference sample is updated by using the "intra_residual_prediciton_flag" flag, and whether the reference sample is updated may be indicated through the flag.

In operation 985 , the intra residual predictor 810 may encode the identifier of the first neighboring block.

The identifier of the first neighboring block may be information that enables the neighboring block used for prediction of the residual signal of the current block to be identified.

For example, the identifier of the first neighboring block may indicate a neighboring block used for prediction of a residual signal of a current block among a plurality of neighboring blocks. Alternatively, the identifier of the first neighboring block may be position information indicating a position of a neighboring block used for prediction of a residual signal of a current block among a plurality of neighboring blocks. The position may indicate a relative position of the selected neighboring block with respect to the current block. The position may indicate a direction in which a neighboring block selected with respect to the current block is adjacent.

The identifier of the first neighboring block may be configured to indicate the same block in both the encoding apparatus 800 and the decoding apparatus 2300 . For example, in relation to the identifier of the first neighboring block, the size N of the block and the position of the neighboring block may have to be common to the encoding apparatus 800 and the decoding apparatus 2300 . In order for the encoding apparatus 800 and the decoding apparatus 2300 to share a common configuration with respect to the identifier of the first neighboring block, the identifier of the first neighboring block is "(neighboring-residual_index) truncation unary ((neighboring) -residual_idx) Truncated unary)" may be encoded.

If the streetcar signal prediction is performed, step 990 may be performed after steps 950 , 970 , 980 , and 985 are performed. Also, if residual signal prediction is not performed, step 990 may be performed after step 960 , step 970 , step 980 , and step 985 are performed.

Next, reference is made to FIG. 9B.

In operation 990 , the encoding apparatus 800 may encode the current block using the residual signal of the current block. Operation 990 may be performed by at least one of the transform unit 130 , the quantizer 140 , and the entropy encoder 150 .

In operation 990 , the residual signal used for encoding of the current block may be one of two residual signals.

When it is determined to perform residual signal prediction in operation 930 , the first residual signal of the current block generated in operation 950 may be the residual signal used in operation 990 . That is, the residual signal generated by the residual signal prediction may be used for encoding the current block, and the encoding apparatus 800 may perform encoding of the current block using the first residual signal of the current block.

When it is determined not to perform residual signal prediction in operation 930 , the third residual signal of the current block generated in operation 960 may be the residual signal used in operation 990 .

Step 990 may include steps 991 , 992 , and 993 .

In operation 991 , the transform unit 130 may perform transform on the residual signal to generate transform coefficients.

In operation 992, the quantization unit 140 may generate a quantized transform coefficient level by using the transform coefficient.

In operation 993, the entropy encoding unit 150 may perform entropy encoding on the transform coefficient level.

Among the steps related to the above-described encoding of information, steps 970, 980, and 985 may be performed by a different subject and in a different order than those described above. For example, step 993 may include steps 970 , 980 , and 985 . Also, the entropy encoder 150 may encode at least one of information indicating whether the reference sample is updated, information indicating whether residual signal prediction is performed, and the identifier of the first neighboring block.

10A is a flowchart of an encoding method according to an embodiment.

10B is a flowchart of a method for updating a reference sample according to an embodiment.

10C is a flowchart of a method for predicting a residual signal according to an embodiment.

10D is a flowchart of a method for encoding a current block according to an embodiment.

Compared to the embodiment described above with reference to FIG. 9 , in the embodiment described with reference to FIGS. 10A, 10B and 10C , the update of the reference sample and the prediction of the residual signal may be performed separately.

Hereinafter, the current block may be a block to be currently encoded or may be a block in the current image.

First, refer to FIG. 10A.

In operation 1010, the intra prediction unit 120 may generate a reference sample of the current block. The intra prediction unit 120 may generate a reference sample around the current block for intra prediction. Generation of a reference sample according to an example will be described in detail below with reference to FIG. 11 .

In generating the reference sample, a reference sample generating method of an existing image encoding and/or decoding technique, such as high-efficiency video coding and enhanced video coding, may be used.

The encoding apparatus 800 may selectively provide a function of updating a reference sample. If the function of updating the reference sample is used, step 1020 may be performed following step 1010 . If the function of updating the reference sample is not used, step 1030 may be performed following step 1010 . That is to say, the function of updating the reference sample in the embodiment can be selectively combined.

In operation 1020 , the intra prediction unit 120 may provide a function related to the update of the reference sample.

Next, reference is made to FIG. 10B.

Referring to FIG. 10B , step 1020 may include steps 1021 , 1022 , and 1023 .

In operation 1021 , the intra prediction unit 120 may determine whether to update the reference sample. Here, the updating of the reference sample may be reconstructing the sample value of the reference sample used for generation of the prediction block before generation of the prediction block of the current block.

If it is determined to perform an update of the reference sample, step 1022 may be performed. If it is determined not to perform the reference sample update, step 1023 may be performed.

In operation 1022, the intra prediction unit 120 may update the value of the reference sample, and through the update, determine the value of the reference sample used to generate the prediction block of the current block.

The intra prediction unit 120 may update the value of the reference sample according to the directional pattern of the surrounding sample.

The update of the reference sample according to an example will be described in detail below with reference to FIGS. 12, 13 and 14 .

After step 1022 is performed, step 1023 may be performed.

In operation 1023 , the intra predictor 120 may encode information indicating whether the update of the reference sample is performed.

Unlike the existing intra prediction, when the reference sample is updated, the intra prediction unit 120 provides the information so that the decoding apparatus 2300, which will be described later with reference to FIG. 23, knows whether or not the reference sample is updated. can be coded. For example, if the update of the reference sample is used, the value of the information may be set to '1', and if the update of the reference sample is not used, the value of the information may be set to '0'.

For example, the intra predictor 120 may indicate whether the reference sample is updated using a flag, and whether the reference sample is updated may be indicated through the flag.

If step 1010 or step 1020 is performed, step 1030 may be performed.

Reference is again made to FIG. 10A .

In operation 1030 , the intra prediction unit 120 may perform intra prediction. The intra prediction unit 120 may generate a prediction block of the current block. The intra prediction unit 120 may generate a prediction block of the current block by using a reference sample according to an intra prediction mode for the current block.

For example, in generating a prediction block, a method of generating a prediction block of an existing image encoding and/or decoding technology such as HEVC and AVC may be used.

The encoding apparatus 800 may selectively provide a function of prediction of the residual signal. If the function of prediction of the residual signal is used, step 1040 may be performed following step 1030 . If the function of prediction of the residual signal is not used, step 1050 may be performed following step 1030 . That is to say, the function of prediction of the feast signal in an embodiment can be selectively combined.

When the residual signal of the current block is obtained through intra prediction, the intra residual predictor 810 may perform prediction on the residual signal of the current block using the residual signal of the neighboring block. Since updates are made to the residual signal once it is generated, a "prediction" for the residual signal may be considered a "reprediction" for the residual signal.

In operation 1040 , the intra residual predictor 810 may provide a function related to prediction of the residual signal.

Referring to FIG. 10C , step 1040 may include steps 1041 , 1042 , 1046 , and 1047 .

In operation 1041 , the intra residual predictor 810 may determine whether to perform residual signal prediction.

If it is determined to perform residual signal prediction, step 1042 may be performed.

If it is determined not to perform residual signal prediction, step 1046 may be performed.

In operation 1042 , the intra residual predictor 810 may predict the residual signal.

Step 1042 may include step 1043 , step 1044 , and step 1045 .

In step 1042, steps 1043 and 1044 may be performed. Steps 1043 and 1044 may be performed in a predefined order. For example, step 1043 may be performed prior to step 1044 . Alternatively, step 1044 may be performed prior to step 1043 .

In operation 1043 , the intra residual predictor 810 may determine a first neighboring block from among one or more neighboring blocks of the current block. The first neighboring block may be a block used for residual signal prediction. Determination of a neighboring block according to an example will be described in detail below with reference to FIG. 22 .

In operation 1044 , the intra prediction unit 120 may generate a second residual signal of the current block by using the intra prediction. The intra prediction unit 120 may generate a second residual signal of the current block based on the intra prediction mode and the reference sample for the current block.

The second residual signal may correspond to a residual signal of a current block in existing image encoding and/or decoding techniques such as HEVC and AVC. For example, in generating the second residual signal, a residual signal generating method of an existing image encoding and/or decoding technique such as HEVC and AVC may be used.

Generation of the second residual signal according to an example will be described in detail below with reference to FIGS. 16 , 17 , and 18 .

If steps 1043 and 1044 are performed as it is determined that residual signal prediction is to be performed, then step 1045 may be performed.

In operation 1045 , the intra residual predictor 810 may generate a first residual signal of the current block according to residual signal prediction.

The intra residual predictor 810 may use the residual signal of the first neighboring block to predict the residual signal.

The intra residual predictor 810 may generate a first residual signal of the current block based on the second residual signal of the current block and the residual signal of the first neighboring block.

The first residual signal may be a difference between the second residual signal of the current block and the residual signal of the first neighboring block. Alternatively, the first residual signal may be a result of subtracting the residual signal of the first neighboring block from the second residual signal of the current block. The intra residual predictor 810 may generate a difference between the second residual signal of the current block and the residual signal of the first neighboring block as the first residual signal.

The first residual signal may be a residual signal of a current block generated by residual signal prediction. Alternatively, the first residual signal may be a residual signal of a current block generated based on a residual signal of a first neighboring block of the current block.

By applying residual signal prediction to the second residual signal generated in operation 1044 , more efficient encoding may be possible with respect to the current block.

Generation of the first residual signal according to an example will be described in detail below with reference to FIG. 19 .

If it is determined that the residual signal prediction is not to be performed, in operation 1046 , the intra predictor 120 or the intra residual predictor 810 may generate a third residual signal of the current block.

The third residual signal may be a residual signal of a current block in conventional image encoding and/or decoding techniques such as HEVC and AVC. For example, in generating the third residual signal, a residual signal generating method of an existing image encoding and/or decoding technique such as HEVC and AVC may be used.

Also, the third residual signal may be the same signal as the second residual signal in step 1044 . In other words, steps 1044 and 1046 may generate the residual signal of the current block in the same way.

Once step 1042 or step 1046 is performed, step 1047 may then be performed.

In operation 1047 , the intra residual prediction unit 810 may perform encoding of information related to prediction of a residual signal.

When the residual signal prediction is performed, the intra residual predictor 810 may encode the above information so that the decoding apparatus 2300, which will be described later with reference to FIG. 21 , knows whether the residual signal is predicted. For example, if the update of the reference sample is used, the value of the information may be set to '1', and if the update of the reference sample is not used, the value of the information may be set to '0'.

For example, the intra prediction unit 120 may indicate whether the reference sample is updated by using the "intra_residual_prediciton_flag" flag, and whether the reference sample is updated may be indicated through the flag.

Next, reference is made to FIG. 10C.

Step 1047 may include steps 1048 and 1049 .

In operation 1048 , the intra residual predictor 810 may encode the identifier of the first neighboring block.

The identifier of the first neighboring block may be information that enables the neighboring block used for prediction of the residual signal of the current block to be identified.

For example, the identifier of the first neighboring block may indicate a neighboring block used for prediction of a residual signal of a current block among a plurality of neighboring blocks. Alternatively, the identifier of the first neighboring block may be position information indicating a position of a neighboring block used for prediction of a residual signal of a current block among a plurality of neighboring blocks. The position may indicate a relative position of the selected neighboring block with respect to the current block. The position may indicate a direction in which a neighboring block selected with respect to the current block is adjacent.

The identifier of the first neighboring block may be configured to indicate the same block in both the encoding apparatus 800 and the decoding apparatus 2300 . For example, in relation to the identifier of the first neighboring block, the size N of the block and the position of the neighboring block may have to be common to the encoding apparatus 800 and the decoding apparatus 2300 . In order for the encoding apparatus 800 and the decoding apparatus 2300 to share a common configuration with respect to the identifier of the first neighboring block, the identifier of the first neighboring block is "(neighboring-residual_index) truncation unary ((neighboring) -residual_idx) Truncated unary)" may be encoded.

Step 1048 may be omitted. For example, if at step 1041 it is determined that residual signal prediction is not to be performed, then step 1048 may be omitted.

In operation 1049 , the intra residual predictor 810 may encode information indicating whether residual signal prediction is performed.

In step 1047, steps 1048 and 1049 may be performed. Steps 1048 and 1049 may be performed in a predefined order. For example, step 1048 may be performed prior to step 1049 . Alternatively, step 1049 may be performed prior to step 1048 .

After step 1030 or step 1040 is performed, step 1050 may be performed next.

In operation 1050, the encoding apparatus 800 may encode the current block using the residual signal of the current block. Step 1050 may be performed by at least one of the transform unit 130 , the quantizer 140 , and the entropy encoder 150 .

The residual signal used for encoding of the current block in operation 1050 may be one of two residual signals.

When it is determined to perform residual signal prediction in operation 1041 , the first residual signal of the current block generated in operation 950 may be the residual signal used in operation 1050 . That is, the residual signal generated by the residual signal prediction may be used for encoding the current block, and the encoding apparatus 800 may perform encoding of the current block using the first residual signal of the current block.

When it is determined not to perform residual signal prediction in operation 1050 , the third residual signal of the current block generated in operation 1046 may be the residual signal used in operation 1050 .

Next, reference is made to FIG. 10D.

Step 1050 may include step 1051 , step 1052 , and step 1053 .

In operation 1051 , the transform unit 130 may perform transform on the residual signal to generate transform coefficients.

In operation 1052 , the quantization unit 140 may generate a quantized transform coefficient level by using the transform coefficient.

In operation 1053 , the entropy encoding unit 150 may perform entropy encoding on the transform coefficient level.

Among the steps related to the above-described encoding of information, steps 1023, 1048, and 1049 may be performed by a different subject and in a different order than those described above. For example, step 1053 may include step 1023 , step 1048 , and step 1049 . Also, the entropy encoder 150 may encode at least one of information indicating whether the reference sample is updated, information indicating whether residual signal prediction is performed, and the identifier of the first neighboring block.

Proposed encoding methods

In the embodiment described above with reference to FIG. 10, the encoding method may be classified into two methods. Each method is as follows.

- First method: The first method may be a method of generating a reference sample of intra prediction by the conventional method, and reconstructing the sample value of the reference sample generated by the existing method according to the directional pattern of the surrounding sample. have. Here, the conventional method may refer to a method of generating a reference sample for intra prediction performed by the intra prediction unit 120 described above with reference to FIG. 1 .

- Second method: The second method may be a method of reducing the energy of the residual signal by performing re-prediction using the residual signal of the neighboring block on the residual signal obtained through the existing intra prediction. That is to say, in the double-method, re-prediction at the residual signal level can be performed. Here, the conventional method is a process of generating a prediction signal of a current block through spatial prediction of a reference sample, performed by the intra prediction unit 120 described above with reference to FIG. 1 , and a current using the prediction signal. It may include a process of obtaining a residual signal of the block.

Classification of Examples

The embodiment described above with reference to FIGS. 10A, 10B, 10C and 10D may be classified into three embodiments. Each example is as follows.

- Example 1 : Example 1 is indicated by a filled arrow in FIG. 10A . The first embodiment may include steps 1010 , 1020 , 1030 , and 1050 . The first embodiment may update a reference sample, but may not perform prediction of a residual signal.

- Second embodiment: The second embodiment is indicated by an arrow with an empty inside in FIG. 10A. The second embodiment may include steps 1010 , 1020 , 1030 , 1040 , and 1050 . The second embodiment may perform updating of a reference sample and prediction of a residual signal.

- Third Embodiment: The third embodiment is indicated by an arrow with an oblique line drawn therein in FIG. 10A . A third embodiment may include steps 1010 , 1030 , 1040 , and 1050 . The third embodiment does not update the reference sample, but predicts the residual signal.

The encoding apparatus 800 may encode the current block using one of the first, second, and third embodiments. Alternatively, the encoding apparatus 800 may perform rate-distortion optimization for all three embodiments, and may select a method for deriving a minimum rate-distortion value among the three embodiments. For example, the encoding apparatus 800 may selectively use each of steps 1020 and 1040 to derive a minimum rate-distortion value.

Unit of update of reference sample

In operation 910 , the intra predictor 120 may determine whether to update the reference sample for each predefined unit. The predefined unit may be at least one of 1) an entire image sequence (ie, video), 2) one image (ie, picture), 3) a slice, and 4) a coding unit.

For a predefined unit, reference sample update information may be used to indicate whether the reference sample is updated. The reference sample update information may be information indicating whether the reference sample is updated with respect to a predefined unit. For example, that the value of the reference sample update information is the first value may indicate that the reference sample is updated in encoding of the current block. When the value of the reference sample update information is the second value, it may indicate that the reference sample is not updated in the encoding of the current block.

The encoding apparatus 800 may include the encoded residual signal prediction information in the bitstream. The decoding apparatus 2300 may determine whether residual signal prediction is performed on the current block by using the residual signal prediction information.

In the following, the update of the reference sample for each of the predefined units is described.

1) Entire image sequence: Whether to update the reference sample may be determined for the entire image sequence. In this case, the sequence parameter set may include reference sample update information. When the reference sample update information of the sequence parameter set indicates that the reference sample is to be updated, the intra prediction unit 120 may perform intra prediction using the reference sample to which the gradient according to the directionality is applied with respect to the entire image sequence.

2) One image: Whether to update the reference sample may be determined for each image. In this case, the picture parameter set may include reference sample update information. When the reference sample update information of the picture parameter set indicates that the reference sample is to be updated, the intra prediction unit 120 performs intra prediction using the reference sample to which the gradient according to the directionality is applied to the entire image corresponding to the picture parameter set. can be done Alternatively, the intra prediction unit 120 may identify whether the update of the reference sample is performed using a picture parameter set identifier (ID) specified in the slice header.

3) Slice: One image may be divided into a plurality of slice segments or a plurality of tiles within one slice segment. Whether to update the reference sample may be determined for each slice. In this case, the slice segment header may include reference sample update information. If the reference sample update information of the slice segment header indicates that the reference sample is to be updated, the intra prediction unit 120 performs intra prediction using the reference sample to which the gradient according to the directionality is applied on the slice corresponding to the slice segment header. can

4) Coding unit: Whether to update the reference sample may be determined for each coding unit. In this case, reference sample update information may be present for the coding unit. When the reference sample update information for the coding unit indicates that the coding unit is updated, the intra prediction unit 120 performs intra prediction using the reference sample to which the gradient according to the directionality is applied with respect to the coding unit corresponding to the reference sample update information. can be done

As described above, the reference sample update information may be coded in a sequence parameter set, a picture parameter set, or a slice segment header. Also, update information may be coded for a coding unit.

Unit of Residual Signal Prediction

Whether to perform the residual signal prediction determined in operation 930 may be determined for a predefined unit. The predefined unit may be at least one of 1) an entire image sequence (ie, video), 2) one image (ie, picture), 3) a slice, and 4) a coding unit.

The residual signal prediction information may be information indicating whether residual signal prediction is performed with respect to a predefined unit. For example, when the value of the residual signal prediction information is the first value, it may indicate that the residual signal prediction is performed in the encoding of the current block. The second value of the residual signal prediction information may indicate that residual signal prediction is not performed in encoding of the current block.

The encoding apparatus 800 may include the encoded residual signal prediction information in the bitstream. The decoding apparatus 2300 may determine whether residual signal prediction is performed on the current block by using the residual signal prediction information.

In the following, the residual signal prediction for each of the predefined units is described.

1) Entire image sequence: Whether to perform residual signal prediction may be determined for the entire image sequence. In this case, the sequence parameter set may include residual signal prediction information. When the residual signal prediction information of the sequence parameter set indicates that residual signal prediction is performed, the intra residual predictor 810 may perform residual signal prediction on the entire image sequence.

2) One image: Whether to perform residual signal prediction may be determined for each image. In this case, the picture parameter set may include residual signal prediction information. When the residual signal prediction information of the picture parameter set indicates that residual signal prediction is performed, the intra residual prediction unit 810 may perform residual signal prediction on the entire image corresponding to the picture parameter set.

3) Slice: One image may be divided into a plurality of slice segments or a plurality of tiles within one slice segment. Whether to perform residual signal prediction may be determined for each slice. In this case, the slice segment header may include residual signal prediction information. When the residual signal prediction information of the slice segment header indicates that residual signal prediction is performed, the intra residual predictor 810 may perform residual signal prediction on a slice corresponding to the slice segment header.

4) Coding unit: Whether to perform residual signal prediction may be determined for each coding unit. In this case, residual signal prediction information may be present for the coding unit. When the residual signal prediction information for the coding unit indicates that residual signal prediction is performed, the intra residual prediction unit 810 may perform residual signal prediction on the coding unit corresponding to the residual signal prediction information.

As described above, the residual signal prediction information may be coded within a sequence parameter set, a picture parameter set, or a slice segment header. Also, update information may be coded for a coding unit.

11 illustrates a current block and a reference sample according to an example.

The processing on the reference sample, which will be described below, may be used to determine the value of the reference sample before the update of the reference sample, and is performed by the intra prediction unit 120 before the step 910 described above with reference to FIG. 9A . can be performed. In addition, processing on a reference sample, which will be described below, may correspond to step 1010 described above with reference to FIG. 10A .

In FIG. 11 , the current block 1100 , the top adjacent line 1110 , the left adjacent line 1120 , and the top left sample 1130 are shown.

The reference sample of the current block 1100 may include an upper adjacent line 1110 , a left adjacent line 1120 , and an upper left sample 1130 . Alternatively, the reference sample of the current block 1100 may be at least some of samples of the top adjacent line 1110 , the left adjacent line 1120 , and the left top sample 1130 . The intra prediction unit 120 may select at least some of pixels of the upper adjacent line 1110 , the left adjacent line 1120 , and the upper left sample 1130 as reference samples according to the intra prediction mode for the current block.

The top adjacent line 1110 may be a horizontal line adjacent to the top of the current block 1100 . The left adjacent line 1120 may be a vertical line adjacent to the left of the current block 1100 . The upper left sample 1130 may be a sample adjacent to the upper left of the current block 1100 .

The x-coordinate of the leftmost sample of the upper adjacent line 1110 may be the same as the x-coordinate of the leftmost pixel(s) of the current block 1100 . When the size of the current block 1100 is NxN, the length of the upper adjacent line 1110 may be 2N. Here, N may be an integer of 1 or more. The top adjacent line 1110 may be 2Nx1 pixels.

The y-coordinate of the uppermost sample of the left adjacent line 1120 may be the same as the y-coordinate of the uppermost pixel(s) of the current block 1100 . When the size of the current block 1100 is NxN, the length of the left adjacent line 1120 may be 2N. The left adjacent line 1120 may be 1x2N pixels.

The x-coordinate of the upper-left sample 1130 may be a value obtained by subtracting 1 from the x-coordinate of the leftmost pixel(s) of the current block 1100 . The y-coordinate of the upper-left sample 1130 may be a value obtained by subtracting 1 from the y-coordinate of the uppermost pixel(s) of the current block 1100 .

The aforementioned reference sample may be used in intra prediction for the current block.

A reference sample used for intra prediction may have a brightness value reconstructed by prediction and reconstruction, not a brightness value of the pixel(s) of the original image. For example, before encoding of the current block 1100, the neighboring block(s) of the current block 1100 may be encoded, and the brightness value of the pixel of the neighboring block may be restored through prediction and restoration in the encoding process. can The reference sample may be some of the pixels of these neighboring blocks. Also, the brightness value of the reference sample may be a value before post-processing filtering is applied.

When there is no reference sample available in the vicinity of the current block 1100 , the intra prediction unit 120 uses samples closest to the current block 1100 among available samples (ie, pixels) in the vicinity of the reference sample. Padding may be performed. A brightness value of a reference sample may be generated by reference sample padding.

The intra prediction unit 120 may perform reference sample filtering to reduce a prediction error due to a quantization error according to the size of the current block 1100 and the intra prediction mode.

12 illustrates a method of updating a reference sample by reflecting a horizontal gradient of a neighboring block according to an example.

A circle in FIG. 12 may represent a sample (or pixel). A dotted rectangle may represent a block.

In FIG. 12 , a current block 1210 , a reference sample block 1220 , reference samples 1221 , and a neighboring block 1230 are illustrated.

The current block is shown as a block having a size of 4x4. The size of the block may mean the width and height of the block.

The reference sample block 1220 may be a block including reference samples 1221 used for the current block 1210 . The reference sample block 1220 may be a block adjacent to the current block 1210 and having the same size as the size of the current block 1210 .

Reference samples 1221 are shown to have values of I', J', K', and L', respectively, through updating of the reference sample. The illustrated reference samples may be samples constructed according to a method of generating a reference sample among samples surrounding the current block according to the intra prediction mode.

In FIG. 12 , reference samples generated by the intra prediction unit 120 are illustrated when the intra prediction mode for the current block 1210 is a horizontal prediction mode.

A dotted line in the neighboring block 1230 may indicate a horizontal line of samples in the neighboring block 1230 . A thick solid line in the peripheral block 1230 may indicate the inclination of samples included in the horizontal line. ∇ dx may represent a gradient value.

In FIG. 12 , an example in which a sample value constantly increases and then decreases constantly from left to right in a horizontal line of samples is illustrated.

The neighboring block 1230 may be a block that has already been reconstructed prior to encoding and/or decoding of the current block 1210 .

A neighboring block used for updating a reference sample may be different from a neighboring block used for intra-residual prediction of the current block. As described above with reference to FIGS. 9A and 10C , a neighboring block used for intra-residual prediction of a current block may be referred to as a first neighboring block. Also, a neighboring block used for updating the reference sample may be referred to as a second block. The first neighboring block and the second neighboring block may be the same as or different from each other. Also, the first neighboring block may include the second neighboring block, or the second neighboring block may include the first neighboring block.

For example, when the intra prediction mode is the horizontal prediction mode, the reference sample may be a sample adjacent to the left side of the current block 1210 . Alternatively, the reference sample may be a sample of a vertical line adjacent to the left of the current block 1210 . The reference sample block 1220 may be a block adjacent to the left side of the current block 1210 . Also, the neighboring block 1230 may be a block obtained by adding an upper adjacent block of the current block 1210 and an upper left adjacent block of the current block 1210 . The block adjacent to the top of the current block 1210 may be a block adjacent to the top of the current block 1210 . The upper left adjacent block of the current block 1210 may be a block adjacent to the upper left of the current block 1210 . The top adjacent block and the top left adjacent block may be adjacent to each other.

When the size of the current block 1210 is NxN, the size of the upper neighboring block and the size of the upper left neighboring block may be NxN, respectively, and the size of the neighboring block 1230 may be 2NxN. In FIG. 12, it is shown that the size of the neighboring block is 8x4.

Also, when the size of the current block 1210 is NxN, the size of the upper adjacent block and the size of the upper left adjacent block may be aNxbN, respectively, and the size of the neighboring block 1230 may be 2aNxbN. Here, a and b may each be a real number.

Also, the upper adjacent block, the upper left adjacent block, and the adjacent block 1230 may each have a predefined size or a size determined by a predefined method.

The intra prediction unit 120 of the encoding apparatus 800 and the intra prediction unit 240 of the decoding apparatus 2300 to be described later may use upper adjacent blocks of the same size, upper left adjacent blocks, and adjacent blocks. The encoding apparatus 800 may set the respective sizes of the upper adjacent block, the upper left adjacent block, and the adjacent block 1230 . The size of the upper adjacent block, the size of the upper left adjacent block, and the size of the adjacent block 1230 may be equally used in the decoding apparatus 2300 . The set size may be transmitted from the encoding apparatus 800 to the decoding apparatus 2300 through a bitstream.

The update of the reference sample may be regarded as reconstruction of the reference sample in the vicinity of the current block for intra prediction.

When intra prediction is performed on the second neighboring block, the current block may also be configured with a texture having a direction similar to that of the texture of the second neighboring block according to spatial correlation. To reflect this direction, the intra prediction unit 120 may update a value of a reference sample required for intra prediction before performing intra prediction on the current block.

The intra prediction unit 120 may reconstruct the reference sample to be similar to the sample of the current block by using a directional gradient pattern of the second neighboring block.

In step 915 described above with reference to FIG. 9A and step 1022 described above with reference to FIG. 10B , the intra prediction unit 120 may detect a gradient pattern of the second neighboring block and calculate a gradient . A method of detecting a gradient pattern according to an example will be described in detail below with reference to FIG. 13 .

In step 915 described above with reference to FIG. 9A and step 1022 described above with reference to FIG. 10B , the intra prediction unit 120 may detect gradient patterns of rows of the second neighboring block. The intra predictor 120 may check whether gradient patterns of rows of the second neighboring block are the same. When the gradient patterns of the rows of the second neighboring block are the same, the intra predictor 120 may calculate the gradient of the second neighboring block. Here, the gradient of the second neighboring block may be the gradient of one selected row of the second neighboring block. For example, the gradient of the second neighboring block may be the gradient of a row adjacent to the current block 1210 among the rows of the second neighboring block. The intra predictor 120 may determine a gradient of a row adjacent to the current block 1210 among rows of the second neighboring block as a final gradient of the second neighboring block.

Also, in steps 915 and 1022 , the intra prediction unit 120 may determine or update a value of a reference sample based on a neighboring block of the current block. The intra prediction unit 120 may determine or update the value of the reference sample based on the gradient pattern of the second neighboring block.

The value of the reference sample may be changed from the value before update to the value after update by the intra predictor 120 , and the value before update of the reference sample is changed as the reference sample block 1220 including the reference sample is predicted and reconstructed. It can be a generated value. That is, while the reference sample block 1220 is predicted and reconstructed, a value of the reference sample may be determined, and the determined value may be used as a value before the update in updating the reference sample. In prediction and reconstruction of the reference sample, prediction and reconstruction methods of existing image encoding and/or decoding techniques such as HEVC and AVC may be used.

When the gradient ∇ dx is calculated, the intra predictor 120 may determine or update the value of the reference sample in a predefined manner.

For example, when the intra prediction mode is the horizontal prediction mode, the intra prediction unit 120 may determine or update the value of the reference sample in the same manner as in Equation 2 below.

f can represent a function. w may represent a weight. I may be a value before the update of the uppermost sample among the reference samples 1221 . I ' may be a value after the update of the uppermost sample. J , K , L may also be the values before update of the corresponding reference sample. J ', K ', L ' may be values after update of the corresponding reference sample.

As described in Equation 2, the intra prediction unit 120 reconstructs reference samples I ', J ', K by reflecting the subtraction according to the gradient with respect to the existing reference samples I , J , K and L . ' and L ' can be created. Also, the intra predictor 120 may consider a weighting factor w when updating the value of the reference sample. The intra prediction unit 120 may update the value of the reference sample by subtracting a product of a value determined based on the gradient ∇ dx and a predefined weight factor w from a value before the update of the reference sample. Through the update, the value of the reference sample may decrease by w * f (∇ dx ).

The reference sample may be plural. The weight factor w may be the same for a plurality of reference samples. Alternatively, the weight factor w may be different from each other for each of the plurality of reference samples. For example, the weight factor w may be different for each reference sample of the plurality of reference samples for each location of each reference sample. The position of the reference sample may be a position relative to the current block 1210 .

The intra prediction unit 120 may combine the above-described method of updating the reference sample and the existing method. For example, when the intra prediction unit 120 updates the reference sample described above with reference to FIG. 10A , padding of the reference pixel may not be performed in operation 1010 . In addition, when the intra prediction unit 120 updates the reference sample described above with reference to FIG. 10A , smoothing using a low pass filter may not be performed in step 1010 . .

13 illustrates a method of obtaining a gradient pattern according to an example.

As described above with reference to FIG. 12 , the second neighboring block may be selected based on the position of the current block. For example, the intra prediction unit 120 may select a block at a predefined position according to the intra prediction mode from among blocks reconstructed based on the position of the current block as the second neighboring block. Also, the intra prediction unit 120 may analyze the gradient pattern of the selected second neighboring block.

As described above with reference to FIG. 12 , the size of the second neighboring block may be determined based on the size of the current block. For example, if the size of the current block is NxN, the size of the second neighboring block may be 2N*N. The encoding apparatus 800 may set the size of the second neighboring block. The set size may be equally used in the decoding apparatus 2300 . The set size may be transmitted from the encoding apparatus 800 to the decoding apparatus 2300 through a bitstream.

In FIG. 13 , one line of the second peripheral block is illustrated. A line may be a row or a column of a neighboring block. In FIG. 12 , the size of the second neighboring block is shown to be 8x4, and in FIG. 13 , one line of the second neighboring block is shown to be 8x1.

The intra prediction unit 120 may calculate one or more sample gradients for each line. Each sample slope of the one or more sample slopes may be a slope between two adjacent samples of the line. Since one line may include a plurality of samples, the intra prediction unit 120 may obtain a plurality of sample gradients with respect to the line. As an example of one or more sample gradients, ∇ dx 1, ∇ dx 2, ∇ dx 3, ∇ dx 4, ∇ dx 5, ∇ dx 6 and ∇ dx 7 are shown in FIG. 13 . For example, ∇ dx 1 may be the gradient between the first sample and the second sample of the line.

The intra prediction unit 120 may calculate, for each line of the second neighboring block, one or more sample gradients of each line.

The intra predictor 120 may calculate a line gradient based on one or more sample gradients. For example, the line gradient may be 1) a median value of one or more sample gradients, 2) an average value of one or more sample gradients, or 3) a predefined representative value of one or more sample gradients.

The predefined representative value may always have a positive value.

When an intermediate value of one or more sample gradients is used as a line gradient, Equation 3 below may be used.

∇ dx may represent a line gradient.

In calculating the line gradient, the intra predictor 120 may apply a weight to each of the sample gradients with respect to one or more sample gradients.

Through the above-described method, the intra prediction unit 120 may calculate one or more line inclinations of one or more lines of the second neighboring block.

When the line gradient is calculated, the intra predictor 120 may calculate the gradient of the second neighboring block based on one or more line gradients. For example, the gradient of the second neighboring block may be 1) a median value of one or more line gradients, 2) an average value of one or more line gradients, or 3) a predefined representative value of one or more line gradients. Alternatively, when calculating the gradient of the second neighboring block, the intra predictor 120 may apply a weight to each of the line gradients with respect to one or more line gradients.

The calculated gradient of the second neighboring block may be used as a final gradient for updating the reference sample.

For example, as shown in FIG. 13 , when the sample slopes of the line increase and then decrease, each of I, J, K and L already generated as reference samples 1221 in the current block 1210 of FIG. 12 , respectively. The updated reference samples I', J', K' and L' can be obtained by subtracting the line slope of the line from

As described above with respect to the sample gradient, the line gradient, and the gradient of the second neighboring block, the intra prediction unit 120 determines the gradient between two adjacent reference samples among a plurality of reference samples belonging to one row of the second neighboring block. A value of the reference sample may be determined based on the value.

Alternatively, the intra prediction unit 120 may use a line gradient of one line as a gradient of a reference sample corresponding to the line. For example, a line and a reference sample having the same relative position may correspond to each other. For example, the intra predictor 120 may use the line gradient of the topmost line among the one or more lines to update the topmost reference sample among the one or more reference samples. Alternatively, the intra prediction unit 120 may use a line gradient of a line adjacent to the current block 1210 among one or more lines to update the reference sample.

In calculating the gradient of the second neighboring block, the intra prediction unit 120 may select only some line(s) from among all the lines of the second neighboring block. The intra prediction unit 120 may calculate the slope of the second neighboring block by using the line slope(s) of the selected partial line(s). The intra prediction unit 120 may select a line(s) at a predefined location(s) or a predefined number of line(s) among all lines of the second neighboring block. The intra prediction unit 120 may set a method of selecting only some line(s) among all lines of the second neighboring block. The set method may be transmitted from the encoding apparatus 800 to the decoding apparatus 2300 through a bitstream.

The intra prediction unit 120 may select only some samples from among all samples of the line in calculating the line inclination of the line. In other words, the slope of more than one sample of each line need not necessarily be calculated for all samples of each line. The intra prediction unit 120 may calculate a line inclination for each of the regions symmetrical to the left and right with respect to the dotted line in the middle of FIG. 12 . For example, the intra prediction unit 120 may calculate one or more sample gradients of each line with respect to samples of the left half of each line, and may calculate a line gradient based on the calculated one or more sample gradients. . Also, the intra prediction unit 120 may calculate one or more sample gradients of each line with respect to samples of the right half of each line, and may calculate a line gradient based on the calculated one or more sample gradients.

The intra prediction unit 120 may calculate one or more sample gradients of each line with respect to samples selected according to a predefined method among samples of each line, and calculate a line gradient based on the calculated one or more sample gradients. can In the calculation of the line gradient, the size (or the number of samples) of the line may be larger or smaller than the width 2N of the second neighboring block. The intra predictor 120 may calculate a line gradient of a line using sample gradient(s) between selected partial samples. The intra prediction unit 120 may select samples at predefined positions or a predefined number of samples from among all samples of the line. The intra prediction unit 120 may set a method of selecting only some samples among all samples of the line. The set method may be transmitted from the encoding apparatus 800 to the decoding apparatus 2300 through a bitstream.

The intra prediction unit 120 may select some reference sample(s) to which an update is to be applied from among all reference samples of the current block. The intra prediction unit 120 may select the number of reference sample(s) to which an update is to be applied from among all reference samples of the current block. For example, the intra prediction unit 120 may update only some of the reference samples 1221 of FIG. 12 .

The intra prediction unit 120 may select some reference sample(s) to which an update is to be applied from among all reference samples of the current block according to the characteristics of the current block. Also, the intra prediction unit 120 may determine the number of partial reference sample(s) to which update is to be applied among all reference samples of the current block according to the characteristics of the current block.

For example, the characteristic of the current block may be the size of the current block. The intra prediction unit 120 may select some reference sample(s) to which an update is to be applied from among all reference samples of the current block using different methods according to the size of the current block.

For example, considering a case in which the size of the current block is relatively large, such as 16x16 and 32x32, the number of reference samples of the current block may increase as the size of the current block increases. As the number of reference samples increases, the correlation with respect to the gradient pattern according to the direction may be relatively decreased. In this case, the encoding apparatus 800 may determine the number of reference samples to be updated based on the gradient.

A method of selecting some reference sample(s) to which an update is to be applied among reference samples and the number of some reference sample(s) to which an update is to be applied among reference samples are determined from the encoding apparatus 800 through the bitstream. may be transmitted to 2300 .

14 illustrates a method of updating a reference sample by reflecting a vertical gradient of a neighboring block according to an example.

14 , a circle may represent a sample (or pixel). A dotted rectangle may represent a block.

In FIG. 14 , a current block 1410 , a reference sample block 1420 , reference samples 1421 , and a neighboring block 1430 are illustrated.

The current block is shown as a block having a size of 4x4. The size of the block may mean the width and height of the block.

The reference sample block 1420 may be a block including reference samples 1421 used for the current block 1410 . The reference sample block 1420 may be a block adjacent to the current block 1410 and having the same size as the size of the current block 1410 .

Reference samples 1421 are shown to have values of A', B', C', and D', respectively, through updating of the reference sample. The illustrated reference samples may be samples constructed according to a method of generating a reference sample among samples surrounding the current block according to the intra prediction mode.

In FIG. 14 , reference samples generated by the intra prediction unit 120 are illustrated when the intra prediction mode for the current block 1410 is a vertical prediction mode.

A dotted line in the peripheral block 1430 may indicate a vertical line of pixels in the peripheral block 1430 . A thick solid line in the peripheral block 1430 may indicate the inclination of samples included in the vertical line. ∇ dx may represent a gradient value.

In FIG. 14 , in a vertical line of samples, an example in which a sample value is constantly increased from top to bottom and then decreased constantly is shown.

The neighboring block 1430 may be a block that has already been reconstructed prior to encoding and/or decoding of the current block 1410 .

A neighboring block used for updating a reference sample may be different from a neighboring block used for intra-residual prediction of the current block. As described above with reference to FIG. 9A , a neighboring block used for intra-residual prediction of the current block may be referred to as a first neighboring block. Also, a neighboring block used for updating the reference sample may be referred to as a second block. The first neighboring block and the second neighboring block may be the same as or different from each other. Also, the first neighboring block may include the second neighboring block, or the second neighboring block may include the first neighboring block.

For example, when the intra prediction mode is the vertical prediction mode, the reference sample may be a sample adjacent to the top of the current block 1410 . Alternatively, the reference sample may be a sample of a horizontal line adjacent to the top of the current block 1410 . The reference pixel block 1420 may be a block adjacent to the top of the current block 1410 . Also, the neighboring block 1330 may be a block obtained by adding a left block of the current block 1410 and an upper left neighboring block of the current block 1410 . The left adjacent block of the current block 1410 may be a block adjacent to the left of the current block 1410 . The upper left adjacent block of the current block 1410 may be a block adjacent to the upper left of the current block 1410 . The left adjacent block and the upper left adjacent block may be adjacent to each other.

When the size of the current block 1410 is NxN, the size of the left neighboring block and the size of the upper left neighboring block may each be NxN, and the size of the neighboring block 1430 may be Nx2N. In FIG. 11, it is shown that the size of the neighboring block is 8x4.

Also, when the size of the current block 1410 is NxN, the size of the upper adjacent block and the size of the upper left adjacent block may be aNxbN, respectively, and the size of the neighboring block 1430 may be aNx2bN. Here, a and b may each be a real number.

Also, the upper adjacent block, the upper left adjacent block, and the adjacent block 1430 may each have a predefined size or a size determined by a predefined method.

The intra prediction unit 120 of the encoding apparatus 800 and the intra prediction unit 240 of the decoding apparatus 2300 to be described later may use upper adjacent blocks of the same size, upper left adjacent blocks, and adjacent blocks. The encoding apparatus 800 may set the respective sizes of the upper adjacent block, the upper left adjacent block, and the adjacent block 1430 . The size of the upper adjacent block, the size of the upper left adjacent block, and the size of the adjacent block 1430 may be equally used in the decoding apparatus 2300 . The set size may be transmitted from the encoding apparatus 800 to the decoding apparatus 2300 through a bitstream.

The update of the reference sample may be regarded as reconstruction of the reference sample in the vicinity of the current block for intra prediction.

When intra prediction is performed on the second neighboring block, the current block may also be configured with a texture having a direction similar to that of the texture of the second neighboring block according to spatial correlation. To reflect this direction, the intra prediction unit 120 may update a value of a reference sample required for intra prediction before performing intra prediction on the current block.

The intra prediction unit 120 may update the reference sample to be similar to the sample of the current block by using the gradient pattern according to the directionality of the second neighboring block.

In step 915 described above with reference to FIG. 9A and step 1022 described above with reference to FIG. 10B , the intra prediction unit 120 may detect a gradient pattern of the second neighboring block and calculate a gradient .

In step 915 described above with reference to FIG. 9A and step 1022 described above with reference to FIG. 10B , the intra prediction unit 120 may detect gradient patterns of columns of the second neighboring block. The intra predictor 120 may check whether gradient patterns of columns of the second neighboring block are the same. When the gradient patterns of columns of the second neighboring block are the same, the intra predictor 120 may calculate the gradient of the second neighboring block. Here, the gradient of the second neighboring block may be the gradient of one selected column of the second neighboring block. For example, the gradient of the second neighboring block may be the gradient of a column adjacent to the current block 1410 among columns of the second neighboring block. The intra prediction unit 120 may determine a gradient of a column adjacent to the current block 1410 among columns of the second neighboring block as the final gradient of the second neighboring block.

Also, in steps 915 and 1022 , the intra prediction unit 120 may determine or update a value of a reference sample based on a neighboring block of the current block. The intra prediction unit 120 may determine or update the value of the reference sample based on the gradient pattern of the second neighboring block.

The value of the reference sample may be changed from the value before update to the value after update by the intra predictor 120 , and the value before update of the reference sample is changed as the reference sample block 1420 including the reference sample is predicted and reconstructed. It can be a generated value. That is, while the reference sample block 1420 is predicted and reconstructed, a value of the reference sample may be determined, and the determined value may be used as a value before the update in updating the reference sample. In prediction and reconstruction of the reference sample, prediction and reconstruction methods of existing image encoding and/or decoding techniques such as HEVC and AVC may be used.

When the gradient ∇ dx is calculated, the intra predictor 120 may determine or update the value of the reference sample in a predefined manner.

For example, when the intra prediction mode is the vertical prediction mode, the intra prediction unit 120 may determine or update the value of the reference sample in the same manner as in Equation 3 below.

f can represent a function. w may represent a weight. A may be a value before the update of the uppermost sample among the reference samples 1421 . A ' may be a value after the update of the uppermost sample. B , C , and D may also be values before updating of the corresponding reference sample. B ', C ', D ' may be values after updating of the corresponding reference sample.

As described in Equation 4, the intra prediction unit 120 reconstructs the reference samples A ', B ', C by reflecting the subtraction according to the gradient with respect to the existing reference samples A , B , C and D . ' and D ' can be created. Also, the intra predictor 120 may consider a weighting factor w when updating the value of the reference sample. The intra predictor 120 may update the value of the reference sample by adding a product of a value determined based on the gradient ∇ dx and a predefined weight factor w to the value before the update of the reference sample. Through the update, the value of the reference sample may increase by w * f (∇ dx ).

The reference sample may be plural. The weight factor w may be the same for a plurality of reference samples. Alternatively, the weight factor w may be different from each other for each of the plurality of reference samples. For example, the weight factor w may be different for each reference sample of the plurality of reference samples for each location of each reference sample. The position of the reference sample may be a position relative to the current block 1410 .

The intra prediction unit 120 may combine the above-described method of updating the reference sample and the existing method. For example, when the intra prediction unit 120 updates the reference sample described above with reference to FIG. 10A , padding of the reference pixel may not be performed in operation 1010 . Also, when the intra prediction unit 120 updates the reference sample described above with reference to FIG. 10A , in operation 1010 , smoothing using the low frequency filter may not be performed.

Types of gradient patterns

The gradient patterns for the directionality described above with reference to FIGS. 12 to 14 may be classified into predefined types. For example, a gradient pattern can be 1) increase, 2) decrease, 3) increase and saturation, 4) decrease and saturate, 5) saturate and increase, 6) saturate and decrease, 7) at least some of the symmetry of increase and decrease and 8) the symmetry of decrease and increase. In the following, the above-mentioned types will be described.

1) "Increase" indicates a gradient pattern in which the values of the samples increase according to the direction. The direction may be from left to right. Alternatively, the direction may be a direction from the top to the bottom.

2) “Decrease” may indicate a gradient pattern in which values of samples decrease according to a direction.

3) "Increase and saturate" may indicate a gradient pattern in which values of samples increase in the front part and the values of samples are kept constant in the rear part according to the direction.

4) “Reduction and saturation” may indicate a gradient pattern in which values of samples are decreased in the front part and values of samples are kept constant in the rear part according to the direction.

5) "Saturation and increase" may indicate a gradient pattern in which values of samples are kept constant in the front part and the values of samples increase in the rear part according to the direction.

6) “Saturation and decrease” may indicate a gradient pattern in which values of samples are kept constant in the front part and the values of samples decrease in the rear part according to directions.

7) “Symmetry of increase and decrease” may indicate a left-right symmetrical gradient pattern in which values of samples increase in the front part and decrease in values in the rear part according to the direction.

8) "Symmetry of decrease and increase" may indicate a left-right symmetrical gradient pattern in which values of samples decrease in the front part and increase in values in the rear part according to the direction.

The above-mentioned "increase" may be a constant increase or a non-constant increase. Also, the above-mentioned “reduction” may be a constant decrease or a non-constant decrease.

The intra prediction unit 120 may determine the value of the reference sample according to the type of the gradient pattern of the second neighboring block of the current block. For example, when the gradient pattern is one of “increase”, “decrease”, “increase and saturate”, “decrease and saturate”, “saturate and increase”, and “saturate and decrease”, the intra prediction unit 120 may The value of the reference sample may be determined based on the slope value of the line of “increase” or the slope value of the line of “decrease”.

When the gradient pattern is “symmetric”, the intra predictor 120 may determine a value of the reference sample based on gradient values of two lines that form symmetry of the gradient pattern. For example, as described above with reference to FIG. 11 , when the type of the gradient pattern is symmetric which increases and then decreases, the update of Equation 2 may be applied. For example, the intra predictor 120 may update the value of the reference sample by subtracting a product of a value determined based on the gradient ∇ dx and a predefined weight w from a value before the update of the reference sample. Through the update, the value of the reference sample may decrease by w * f (∇ dx ). For example, as described above with reference to FIG. 13 , when the type of the gradient pattern is symmetric increasing after decreasing, the update of Equation 3 may be applied. For example, the intra prediction unit 120 may update the value of the reference sample by adding a product of a value determined based on the gradient ∇ dx and a predefined weight w to the value before the update of the reference sample. Through the update, the value of the reference sample may increase by w * f (∇ dx ).

Use of encoding parameters to detect gradient patterns

In addition to the above-described embodiments, the intra prediction unit 120 may detect a gradient pattern using an encoding parameter. For example, the encoding parameters include: 1) a syntax predefined in relation to intra prediction, 2) an encoding variable, 3) a current encoding unit, 4) a prediction unit, 5) a size of a transform unit, and 6) an encoding Whether or not the unit is divided may be included. For example, the intra predictor 120 may determine a value related to a neighboring block by using the encoding parameter. The neighboring block may be the neighboring block 1230 described above with reference to FIG. 12 or the neighboring block 1430 described above with reference to FIG. 14 . The value related to the neighboring block may include the size of the neighboring block. The intra predictor 120 may determine the gradient of the neighboring block by using the encoding parameter. The intra predictor 120 may determine the range of the reference sample to which the update is applied by using the encoding parameter. Also, the intra predictor 120 may determine the weight factor using the encoding parameter.

Expansion of use of slope

The intra prediction unit 120 may apply the gradient of the neighboring block to the pixel value of the prediction block, not the reference sample. The intra prediction unit 120 may fix the value of the reference sample and update the pixel value of the prediction block by using the gradient of the neighboring block. By directly updating the pixel value of the prediction block, the same effect as when the value of the reference sample is updated can be obtained.

In the above-described embodiment, the update performed on the reference sample according to the gradient of the neighboring block may be equally applied to the prediction block. For example, the update performed on the reference samples of one row may be equally applied to each row of a plurality of rows of the prediction block. Alternatively, for example, an update performed on reference samples in one column may be equally applied to each column of a plurality of columns of the prediction block. The encoding parameters can be used for the update of the prediction block of the intra prediction unit 120 in the same manner as that used for the update of the reference sample. That is, the intra prediction unit 120 may update the pixel value of the prediction block based on the encoding parameter.

15A shows intra prediction with 33 respective modes according to an example.

15B shows intra prediction with 65 respective modes according to an example.

In step 920 described above with reference to FIG. 9A and step 1030 described above with reference to FIG. 10A, the intra prediction unit 120 performs intra prediction on the current block to generate a prediction block of the current block. have. In performing intra prediction on the current block, the intra prediction unit 120 may use updated (or reconstructed) reference sample(s). The intra prediction unit 120 may generate a prediction block of the current block by using the updated (or reconstructed) reference sample(s).

The intra prediction unit 120 may configure reference sample(s) surrounding the current block through the update of the reference sample described above. The intra prediction unit 120 may generate one or more prediction blocks of the current block for one or more intra modes. The one or more intra modes may include an angular mode.

The intra prediction unit 120 may determine a prediction block having a minimum rate-distortion value with respect to one or more prediction blocks. The intra prediction unit 120 may select the intra mode corresponding to the prediction block as the final intra prediction mode. Alternatively, the intra prediction unit 120 may select an intra mode generating a prediction block having a minimum rate-distortion value among one or more intra modes as the final intra prediction mode.

In FIG. 15A, intra modes having 33 respective modes are illustrated. In FIG. 15B, intra modes having 65 respective modes are illustrated. Also, mode 0 may represent a planar mode. Mode 1 may represent a DC mode.

16 illustrates a region of an image according to an example.

In FIG. 16 , a portion 1600 of an image is shown. A portion 1600 of the illustrated image may include a current block 1610 , a neighboring block 1620 , a reconstructed region 1630 , a neighboring block reference sample 1631 , and a current block reference sample 1632 .

The neighboring block reference sample 1631 may be a reference sample used when performing intra prediction on the neighboring block. The current block reference sample 1632 may be a reference sample used when intra prediction is performed on the current block.

The sample value of the neighboring block reference sample 1631 may be an updated value when the updating of the reference sample in steps 915 and 1022 is performed with respect to the neighboring block 1620 .

The sample value of the current block reference sample 1632 may be an updated value when the update of the reference sample in steps 915 and 1022 is performed with respect to the current block 1610 .

The current block 1610 , the neighboring block 1620 , the neighboring block reference sample 1631 , and the current block reference sample 1632 shown in FIG. 16 are the intra prediction mode of the neighboring block 1620 and the intra prediction mode of the current block 1610 . It may indicate a relative position when all prediction modes are vertical prediction modes. For example, when the intra prediction mode is the vertical prediction mode, the first neighboring block described in FIGS. 9A and 10C may be a left neighboring block of the current block. The reference sample of the current block may be pixel(s) in a horizontal line adjacent to the top of the current block. Also, the reference sample of the neighboring block may be pixel(s) in a horizontal line adjacent to the top of the neighboring block.

17 illustrates a method of calculating a residual signal of a neighboring block according to an example.

In FIG. 17 , a neighboring block 1710 , a prediction block 1720 of the neighboring block, and a residual signal 1730 of the neighboring block are illustrated. In FIG. 17 , the Intra neighboring block_residual signal, which is the residual signal of the _{neighboring block} , has been described in the form of a matrix formula.

The neighboring block 1710 may correspond to the first neighboring block described above with reference to FIGS. 9A and 10C . Alternatively, the neighboring block 1710 may correspond to the neighboring block 1620 described above with reference to FIG. 16 .

The residual signal 1730 may correspond to the “residual signal of the first neighboring block” described above with reference to FIGS. 9A and 10C .

As shown in FIG. 17 , the residual signal 1730 may be a difference between the neighboring block 1710 and the prediction block 1720 . Alternatively, the residual signal 1730 may be a result of subtracting the prediction block 1720 from the neighboring block 1710 .

The value of the prediction block 1720 may be determined based on the intra prediction mode for the neighboring block 1710 .

For example, as shown in FIG. 17 , when the intra prediction mode of the neighboring block 1710 is the vertical prediction mode, the value of each row of the prediction block 1720 may be the value of reference pixels of the neighboring block 1710 . . That is, when the intra prediction mode of the neighboring block 1710 is the vertical prediction mode, values of each row of the prediction block 1720 may be values of pixels in a horizontal line adjacent to the top of the neighboring block 1710 .

Alternatively, when the intra prediction mode of the neighboring block 1710 is the horizontal prediction mode, the values of each column of the prediction block 1720 may be values of reference pixels of the neighboring block 1710 . That is, when the intra prediction mode of the neighboring block 1710 is the horizontal prediction mode, values of each column of the prediction block 1720 may be values of pixels in a vertical line adjacent to the left side of the neighboring block 1710 .

In addition to the vertical prediction mode and the horizontal prediction mode, the prediction block generation method of existing image encoding and/or decoding techniques such as HEVC and AVC may be used for other intra prediction modes.

18 illustrates a method of calculating a residual signal of a current block according to an example.

18 , a current block 1810 , a prediction block 1820 of the current block, and a residual signal 1830 of the current block are illustrated. In FIG. 18 , the Intra current block_residual signal, which is the residual signal of the _{current block} , has been described in the form of a matrix expression.

The current block 1810 may correspond to the current block described above with reference to FIGS. 9A and 10A . Alternatively, the current block 1810 may correspond to the current block 1610 described above with reference to FIG. 16 .

The residual signal 1830 may correspond to the “second residual signal of the current block” described above with reference to FIGS. 9A and 10C .

18 , the residual signal 1830 may be the difference between the current block 1810 and the prediction block 1820 . Alternatively, the residual signal 1830 may be a result of subtracting the prediction block 1820 from the current block 1810 .

According to FIG. 18 , the sum of the values of the residual signal 1830 is 560. In other words, the level sum of the residual signal 1830 is 560.

The value of the prediction block 1820 may be determined based on the intra prediction mode for the neighboring block.

For example, as shown in FIG. 18 , when the intra prediction mode of the current block 1810 is the vertical prediction mode, the value of each row of the prediction block 1820 may be the value of reference pixels of the neighboring block. That is, when the intra prediction mode of the current block 1810 is the vertical prediction mode, the values of each row of the prediction block 1820 may be values of pixels in a horizontal line adjacent to the top of the current block 1810 .

Alternatively, when the intra prediction mode of the current block 1810 is the horizontal prediction mode, the value of each column of the prediction block 1820 may be the value of reference pixels of a neighboring block. That is, when the intra prediction mode of the current block 1810 is the horizontal prediction mode, the values of each column of the prediction block 1820 may be values of pixels in a vertical line adjacent to the left side of the current block 1810 .

The method of calculating the residual signal 1830 of the current block 1810 described with reference to FIG. 18 is the first step of the current block in step 940 described above with reference to FIG. 9A and step 1045 described above with reference to FIG. 10C. 2 It can correspond to the generation of a residual signal.

In addition, the method of calculating the residual signal 1830 of the current block 1810 described with reference to FIG. 18 may correspond to the generation of the third residual signal of the current block in steps 960 and 1046 .

A residual signal after prediction of the residual signal according to embodiments is described in detail below with reference to FIG. 19 .

19 illustrates a residual signal prediction method according to an example.

19 , the residual signal 1910 of the current block, the residual signal 1920 of the neighboring block, and the prediction residual signal 1930 of the current block are shown. In FIG. 19, the intra _{prediction residual signal, which} is the prediction residual signal of the current block, has been described in the form of a matrix expression.

19 may represent the residual signal prediction of step 950 of FIG. 9 and step 1042 of FIG. 10 .

The residual signal 1910 of the current block may correspond to the residual signal 1830 of the current block described above with reference to FIG. 18 . Alternatively, the residual signal 1910 of the current block may correspond to the “second residual signal of the current block” described above with reference to FIGS. 9A and 10C .

The residual signal 1920 of the neighboring block may correspond to the residual signal 1730 of the neighboring block described above with reference to FIG. 17 . Alternatively, the residual signal 1920 of the neighboring block may correspond to the “residual signal of the first neighboring block” described above with reference to FIGS. 9A and 10C .

The prediction residual signal 1930 of the current block may correspond to the “first residual signal of the current block” described above with reference to FIGS. 9A and 10C .

The prediction residual signal 1930 may be a difference between the residual signal 1910 of the current block and the residual signal 1920 of the neighboring block. Alternatively, the prediction residual signal 1930 may be a result of subtracting the residual signal 1920 of a neighboring block from the residual signal 1910 of the current block.

The sum of the values of the prediction residual signal 1930 is 267. In other words, the level sum of the prediction residual signal 1930 is 267. By the residual signal prediction, the level sum of the final residual signal of the current block is reduced from 560 to 269. The reduction in the level sum of the final residual signal by the residual signal prediction may indicate that the energy of the residual signal itself used for encoding of the current block is reduced. Reducing the energy of the residual signal itself used for encoding may mean that the amount of bits required for encoding is reduced. Accordingly, the encoding apparatus 800 may reduce the capacity of the bitstream through residual signal prediction.

20A illustrates a default residual signal according to an example.

The default residual of FIG. 20A may be an example of the second residual signal described above with reference to FIGS. 9A and 10C . In other words, the default residual may be a residual signal of the current block before residual signal prediction. Alternatively, the default residual of FIG. 18A may be an example of the residual signal 1910 of the current block described above with reference to FIG. 19 .

20B illustrates a result of a discrete cosine transform on a default residual signal according to an example.

In FIG. 20B, the result of discrete cosine transform on the default residual signal of FIG. 20A is shown.

21A illustrates a proposed residual signal according to an example.

The proposed residual of FIG. 21A may be an example of the first residual signal described above with reference to FIGS. 9A and 10C . In other words, the proposed residual may be a residual signal of the current block according to residual signal prediction. Alternatively, the proposed residual of FIG. 20A may be an example of the prediction residual signal 1930 of the current block described above with reference to FIG. 19 .

21B shows the result of discrete cosine transform on the proposed residual signal according to an example.

In FIG. 21B, the result of the discrete cosine transform on the default residual signal of FIG. 21A is shown.

Referring to FIGS. 20A to 21B , it can be confirmed that the energy concentration in the frequency domain is increased by the residual signal prediction. The intra residual predictor 810 may improve the energy concentration of the residual signal of the current block in the frequency domain through residual signal prediction. As the magnitude of the coefficient value in the frequency domain is reduced by the residual signal prediction, the probability that the result value generated by quantization becomes 0 and the probability that the result value generated by quantization becomes close to 0 may be improved.

22 illustrates a location of a neighboring block according to an example.

In FIG. 16 , the neighboring block 1620 is shown adjacent to the left side of the current block 1610 . In FIG. 16 , a case in which residual signal prediction is performed using the left adjacent block has been described. However, the position of the first neighboring block described above with reference to FIGS. 9A and 10C may not be limited to the left side of the current block 1610 .

As described above in operation 935 of FIG. 9 and operation 1043 of FIG. 10 , the intra residual predictor 810 may determine a first neighboring block from among one or more neighboring blocks of the current block. The first neighboring block may be a block used for residual signal prediction. The intra residual predictor 810 may determine a first neighboring block from among one or more previously reconstructed neighboring blocks of the current block.

As the first neighboring block is determined from among the plurality of neighboring blocks, the intra residual predictor 810 may improve the spatial correlation of the first neighboring block with respect to the residual signal.

In FIG. 22 , as one or more neighboring blocks of the current block, the lower left neighboring block A ₀ 2221 , the left neighboring block A ₁ 2222 , the upper right neighboring block B ₁ 2223 , and the right neighboring block B ₂ 2224 ) and upper left adjacent block B ₃ (2225) are shown. As illustrated, the one or more neighboring blocks of the current block may include a lower left neighboring block, a left neighboring block, an upper right neighboring block, a right neighboring block, and an upper left neighboring block. One or more neighboring blocks of the current block are not limited to the blocks of the position illustrated in FIG. 22 .

In order to improve the residual signal prediction efficiency, the spatial correlation between the current block and the neighboring blocks needs to be improved. In order to improve a neighboring block having high spatial correlation, the intra residual prediction unit 810 may perform residual signal prediction on each neighboring block of a plurality of neighboring blocks of the current block. When a plurality of residual signals for a plurality of neighboring blocks are generated according to residual signal prediction, the intra residual predictor 810 selects a minimum rate-distortion residual signal having a minimum rate-distortion value from among the plurality of residual signals. can Also, the intra residual predictor 810 may select a neighboring block corresponding to the minimum rate-distortion residual signal from among the plurality of neighboring blocks. The intra residual predictor 810 may use the selected neighboring block and the minimum rate-distortion residual signal for encoding the current block.

For example, steps 935, 940, and 950 described above with reference to FIG. 9A and steps 1043, 1044, and 1045 described above with reference to FIG. 10C are currently It may be performed for each block of a plurality of neighboring blocks of the block. Here, the plurality of neighboring blocks of the current block may be at least some of one or more previously reconstructed blocks adjacent to the current block. The number of the plurality of neighboring blocks and positions of the plurality of neighboring blocks may be changed according to a setting of the encoding apparatus 800 .

Through repetition of operation 935 or operation 1043 , the intra residual predictor 810 may sequentially select a plurality of neighboring blocks of the current block as the first block.

Through repetition of operation 950 or operation 1045 , the intra residual predictor 810 may generate a plurality of first residual signals of a plurality of neighboring blocks of the current block. Also, the intra residual predictor 810 may calculate a rate-distortion value for each of the plurality of first residual signals.

The intra residual predictor 810 may determine a minimum rate-distortion residual signal having a minimum rate-distortion value from among the plurality of first residual signals, and select a neighboring block corresponding to the minimum rate-distortion residual signal of the current block. It may be determined as the first neighboring block used for residual signal prediction.

Step 940 may be repeated together as step 950 is repeated, or may be performed only once. Step 1044 may be repeated together as step 1055 is repeated, or may be performed only once.

In steps 935 and 1043 , a first neighboring block of the current block may be determined for each coding unit. Also, in steps 985 and 1048 , the identifier of the first neighboring block may be encoded for each coding unit.

23 is a structural diagram of a decoding apparatus according to an embodiment.

The decoding apparatus 2300 may correspond to the above-described decoding apparatus 200 . The decoding apparatus 200 includes an entropy decoding unit 210 , an inverse quantization unit 220 , an inverse transform unit 230 , an intra prediction unit 240 , a motion compensator 250 , an adder 255 , and a filter unit 260 . and a reference picture buffer 270 , and may further include an intra residual predictor 2310 .

Entropy decoding unit 210 , inverse quantization unit 220 , inverse transform unit 230 , intra prediction unit 240 , motion compensator 250 , adder 255 , filter unit 260 and reference picture buffer 270 . ) may perform the same functions and/or operations as described above with reference to FIG. 2 . A duplicate description will be omitted.

In the embodiments described above with reference to FIGS. 9A to 22 , the functions and/or operations described as being performed by the intra prediction unit 120 of the encoding apparatus 800 are performed by the intra prediction unit ( 240) can be performed. Also, functions and/or operations described as being performed by the intra residual predicting unit 810 of the encoding apparatus 800 may be performed by the intra residual predicting unit 2310 of the decoding apparatus 2300 .

In addition, the entropy decoding unit 210, the inverse quantization unit 220, the inverse transform unit 230, the intra prediction unit 240, the motion compensation unit 250, the adder 255, the filter unit 260 and the reference picture buffer. The 270 may perform functions and/or operations related to the intra residual predictor 2310 . Entropy decoding unit 210 , inverse quantization unit 220 , inverse transform unit 230 , intra prediction unit 240 , motion compensation unit 250 , adder 255 , filter unit 260 , reference picture buffer 270 . ) and functions and/or operations of the intra residual prediction unit 2310 will be described in detail below.

The intra residual predictor 2310 may not be distinguished from the intra predictor 240 . The intra prediction unit 240 and the intra residual prediction unit 2310 may be integrated into the intra prediction unit 240 , and in embodiments, a function described as being performed by the intra residual prediction unit 2310 and/or The operation may be performed by the intra prediction unit 240 .

24A and 24B are flowcharts of a decoding method according to an embodiment.

Hereinafter, the current block may be a block currently being decoded, or may be a block in the current image.

First, referring to FIG. 24A , step 2410 may be performed.

In operation 2410, the decoding apparatus 2310 may generate a residual signal of the current block. Here, the generated residual signal may correspond to the first residual signal of the current block described above with reference to FIGS. 9A and 10C .

Operation 2410 may be performed by at least one of the entropy decoding unit 210 , the inverse quantization unit 220 , and the inverse transform unit 230 .

Step 2410 may include steps 2411 , 2412 , and 2413 .

In operation 2411, the entropy decoder 210 may generate a quantized coefficient for the current block.

In operation 2412 , the inverse quantization unit 220 may generate an inverse quantized coefficient by performing inverse quantization on the quantized coefficient.

In operation 2413 , the inverse transform unit 230 may generate a residual signal by performing an inverse transform on the inverse quantized coefficients.

After step 2410 is performed, step 2420 and step 2$40 may be performed.

Next, reference is made to FIG. 24B.

Prior to performing step 2420, a reference sample may be generated. The content related to the generation of the reference sample described above with reference to FIG. 11 may also be applied to the present embodiment. A duplicate description is omitted.

In operation 2420, the intra predictor 240 may determine whether to update the reference sample.

Here, the updating of the reference sample may be reconstructing the sample value of the reference sample used for generation of the prediction block before generation of the prediction block of the current block.

The intra predictor 240 may determine whether to update the reference sample by using the information indicating whether or not to update the reference sample described above with reference to FIGS. 9A and 10B . The bitstream transmitted from the encoding apparatus 800 to the decoding apparatus 2300 may include information indicating whether the reference sample is updated. Information indicating whether the update of the reference sample is performed may be encoded in the bitstream.

The intra prediction unit 240 may decode information indicating whether to update the coded reference sample. The intra predictor 240 may determine whether to update the reference sample by using information indicating whether to update the decoded reference sample. If the above information indicates that the reference sample is updated, the intra prediction unit 240 may update the reference sample. If the above information indicates that the update of the reference sample is not performed, the intra prediction unit 240 may not perform the update of the reference sample.

If it is determined to perform an update of the reference sample, step 2425 may be performed. If it is determined not to perform the reference sample update, step 2430 may be performed.

In operation 2425, the intra prediction unit 240 may update the value of the reference sample, and through the update, determine the value of the reference sample used to generate the prediction block of the current block.

The contents related to the update of the reference sample described above with reference to FIGS. 12, 13 and 14 may also be applied to the present embodiment. In the embodiments described above with reference to FIGS. 12 , 13 and 14 , the functions and/or operations described as being performed by the intra prediction unit 120 of the encoding apparatus 800 are performed by the intra prediction unit of the decoding apparatus 2300 . (240). In addition, functions and/or operations described as being performed in step 915 described above with reference to FIG. 9A and step 1022 described above with reference to FIG. 10B may also be performed in step 2425 . A duplicate description is omitted.

After step 2425 is performed, step 2430 may be performed.

In operation 2430, the intra prediction unit 240 may generate a prediction block of the current block. The intra prediction unit 240 may generate the prediction block of the current block by using the reference sample according to the intra prediction mode for the current block.

For example, in generating a prediction block, a method of generating a prediction block of an existing image encoding and/or decoding technology such as HEVC and AVC may be used.

After step 2430 is performed, step 2460 or step 2490 may be performed.

In operation 2440 , the intra residual predictor 2310 may determine whether to perform residual signal prediction.

The intra residual prediction unit 2310 may determine whether to perform the residual signal prediction by using the information indicating whether the residual signal prediction described above with reference to FIGS. 9A and 10C is performed. The bitstream transmitted from the encoding apparatus 800 to the decoding apparatus 2300 may include information indicating whether residual signal prediction is performed. Information indicating whether residual signal prediction is performed may be encoded in a bitstream.

The intra prediction unit 240 may decode information indicating whether prediction of the encoded residual signal is performed. The intra prediction unit 240 may determine whether to perform the residual signal prediction by using information indicating whether the decoded residual signal prediction is performed. If the above information indicates that residual signal prediction is performed, the intra prediction unit 240 may perform residual signal prediction. If the above information indicates that residual signal prediction is not performed, the intra prediction unit 240 may not perform residual signal prediction.

If it is determined to perform residual signal prediction, step 2450 may be performed.

If it is determined not to perform residual signal prediction, step 2490 may be performed.

In operation 2450 , the intra residual predictor 2310 may identify a first neighboring block. The first neighboring block may be a block used for prediction of the residual signal, and may be a block located near the current block. The contents related to the determination of the first neighboring block described above with reference to FIG. 22 may also be applied to the present embodiment.

The intra residual predictor 2310 may identify the first neighboring block using the identifier of the first neighboring block described above with reference to FIGS. 9A and 10C . The bitstream transmitted from the encoding apparatus 800 to the decoding apparatus 2300 may include the identifier of the first neighboring block. The identifier of the first neighboring block may be encoded in the bitstream.

The intra residual predictor 2310 may decode the encoded identifier of the first neighboring block. The intra residual predictor 2310 may identify the first neighboring block by using the identifier of the decoded first neighboring block.

The identifier of the first neighboring block may be information that enables the neighboring block used for prediction of the residual signal of the current block to be identified.

For example, the identifier of the first neighboring block may indicate a neighboring block used for prediction of a residual signal of a current block among a plurality of neighboring blocks. Alternatively, the identifier of the first neighboring block may be position information indicating a position of a neighboring block used for prediction of a residual signal of a current block among a plurality of neighboring blocks. The position may indicate a relative position of the selected neighboring block with respect to the current block. The position may indicate a direction in which a neighboring block selected with respect to the current block is adjacent.

The position and number of neighboring blocks may be defined according to encoding parameters.

The identifier of the first neighboring block may be configured to indicate the same block in both the encoding apparatus 800 and the decoding apparatus 2300 . For example, in relation to the identifier of the first neighboring block, the size N of the block and the position of the neighboring block may have to be common to the encoding apparatus 800 and the decoding apparatus 2300 . In order for the encoding apparatus 800 and the decoding apparatus 2300 to share a common configuration with respect to the identifier of the first neighboring block, the identifier of the first neighboring block is "(neighboring-residual_index) truncation unary ((neighboring) -residual_idx) Truncated unary)" may be entropy-decoded.

After steps 2450 and 2430 are performed, step 2460 may be performed next.

In operation 2460, the intra residual predictor 2310 may generate a reconstructed block of the current block.

The intra residual predictor 2310 may generate a reconstructed block of the current block based on the prediction block, the residual signal of the current block, and the residual signal of the first neighboring block.

The residual signal of the current block used in operation 2460 may be a residual signal generated by prediction of the residual signal in the encoding apparatus 800 . In other words, the residual signal of the current block used in operation 2460 may correspond to the first residual signal described above with reference to FIGS. 9A and 10C .

The first neighboring block may be a block that has already been restored before the current block is decoded. Accordingly, the residual signal of the first neighboring block may also be already obtained by the intra residual predictor 2310 before decoding of the current block.

The reconstruction block of the current block may be the sum of the prediction block of the current block, the residual signal of the current block, and the residual signal of the neighboring block. Also, the reconstruction block of the current block may be generated based on the sum of the residual signal of the current block and the residual signal of the first neighboring block.

The reconstruction block may be a sum of 1) a prediction block, 2) a residual signal of the current block, and 3) a residual signal of a first neighboring block. The prediction block may be obtained according to the intra prediction mode. The residual signal of the current block may be transmitted from the encoding apparatus 800 to the decoding apparatus 2300 . The residual signal of the first neighboring block may be obtained by residual signal prediction. For example, when residual signal prediction is not performed, the residual signal of the first neighboring block may be a zero signal.

In some cases, when the intra-residual prediction unit 2310 generates the residual signal of the first neighboring block according to the residual signal prediction, the intra-prediction unit 240 determines the prediction block of the current block, the residual signal of the current block, and the neighboring block. A reconstructed block of the current block may be generated by summing the residual signals.

Step 2430 may be performed, and if it is determined at step 2440 not to perform residual signal prediction, then step 2490 may be performed.

In operation 2490, the intra prediction unit 240 or the intra residual prediction unit 2310 may generate a reconstructed block of the current block based on the prediction block and the residual signal of the current block.

The residual signal of the current block used in operation 2490 may be a residual signal generated without performing residual signal prediction in the encoding apparatus 800 . That is, the residual signal of the current block used in operation 2490 may correspond to the third residual signal described above with reference to FIGS. 9A and 10C .

The third residual signal may be a residual signal of a current block in conventional image encoding and/or decoding techniques such as HEVC and AVC. For example, in generating the third residual signal, a residual signal generating method of an existing image encoding and/or decoding technique such as HEVC and AVC may be used.

25A is a flowchart of a method for generating a residual signal according to an embodiment.

Hereinafter, the current block may be a block currently being decoded, or may be a block in the current image.

First, referring to FIG. 25A , step 2510 may be performed.

In operation 2510 , the decoding apparatus 2310 may generate a residual signal of the current block. Here, the generated residual signal may correspond to the first residual signal of the current block described above with reference to FIGS. 9A and 10C .

Step 2510 may be performed by at least one of the entropy decoding unit 210 , the inverse quantization unit 220 , and the inverse transform unit 230 .

Step 2510 may include steps 2411 , 2412 , and 2413 .

In operation 2511, the entropy decoder 210 may generate a quantized coefficient for the current block.

In operation 2512 , the inverse quantization unit 220 may generate an inverse quantized coefficient by performing inverse quantization on the quantized coefficient.

In operation 2513 , the inverse transform unit 230 may generate a residual signal by performing inverse transform on the inverse quantized coefficients.

After step 2510 is performed, step 2520 to be described later with reference to FIG. 25B may be performed.

25B is a flowchart of a method of decoding a current block using a feast signal according to an embodiment.

In operation 2520 , the intra predictor 240 may decode information indicating whether or not to update the encoded reference sample.

The bitstream transmitted from the encoding apparatus 800 to the decoding apparatus 2300 may include information indicating whether the reference sample is updated. Information indicating whether the update of the reference sample is performed may be encoded in the bitstream.

In operation 2530 , the intra prediction unit 240 may decode information indicating whether prediction of the encoded residual signal is performed.

The bitstream transmitted from the encoding apparatus 800 to the decoding apparatus 2300 may include information indicating whether residual signal prediction is performed. Information indicating whether residual signal prediction is performed may be encoded in a bitstream.

In operation 2540, the intra residual predictor 2310 may decode the encoded identifier of the first neighboring block.

The bitstream transmitted from the encoding apparatus 800 to the decoding apparatus 2300 may include the identifier of the first neighboring block. The identifier of the first neighboring block may be encoded in the bitstream.

In operation 2550 , the intra predictor 240 and the intra residual predictor 2310 may decode the residual signal. The intra prediction unit 240 and the intra residual prediction unit 2310 may generate a reconstructed block by decoding the residual signal.

Step 2550 may include steps 2561 , 2562 , 2563 , and 2564 that will be described later with reference to FIG. 25C .

Also, step 2550 may include steps 2571 , 2572 , 2573 , and 2574 that will be described later with reference to FIG. 25D .

25C is a flowchart of a method for generating a prediction block according to an embodiment.

A reference sample may be generated by steps 2561 , 2562 and 2563 . The content related to the generation of the reference sample described above with reference to FIG. 11 may also be applied to the present embodiment. A duplicate description is omitted.

In operation 2561 , the intra predictor 240 may determine whether to update the reference sample.

Here, the updating of the reference sample may be reconstructing the sample value of the reference sample used for generation of the prediction block before generation of the prediction block of the current block.

The intra predictor 240 may determine whether to update the reference sample by using the information indicating whether or not to update the reference sample described above with reference to FIGS. 9A and 10B .

The intra predictor 240 may determine whether to update the reference sample by using information indicating whether to update the decoded reference sample. If the above information indicates that the reference sample is updated, the intra prediction unit 240 may update the reference sample. If the above information indicates that the update of the reference sample is not performed, the intra prediction unit 240 may not perform the update of the reference sample.

If it is determined to perform an update of the reference sample, step 2563 may be performed. If it is determined not to perform the reference sample update, step 2564 may be performed.

In operation 2563 , the intra prediction unit 240 may update the value of the reference sample, and through the update, determine the value of the reference sample used to generate the prediction block of the current block.

The contents related to the update of the reference sample described above with reference to FIGS. 12, 13 and 14 may also be applied to the present embodiment. In the embodiments described above with reference to FIGS. 12 , 13 and 14 , the functions and/or operations described as being performed by the intra prediction unit 120 of the encoding apparatus 800 are performed by the intra prediction unit of the decoding apparatus 2300 . (240). In addition, functions and/or operations described as being performed in step 915 described above with reference to FIG. 9A and step 1022 described above with reference to FIG. 10B may also be performed in step 2563 . A duplicate description is omitted.

After step 2563 is performed, step 2564 may be performed.

In operation 2564 , the intra prediction unit 240 may generate a prediction block of the current block. The intra prediction unit 240 may generate the prediction block of the current block by using the reference sample according to the intra prediction mode for the current block.

For example, in generating a prediction block, a method of generating a prediction block of an existing image encoding and/or decoding technology such as HEVC and AVC may be used.

For example, after step 2564 is performed, step 2571 to be described later with reference to FIG. 25D may be performed.

25D is a flowchart of a method for generating a reconstructed block according to an embodiment.

In operation 2571 , the intra residual predictor 2310 may determine whether to perform residual signal prediction.

The intra residual prediction unit 2310 may determine whether to perform the residual signal prediction by using the information indicating whether the residual signal prediction described above with reference to FIGS. 9A and 10C is performed.

The intra prediction unit 240 may determine whether to perform the residual signal prediction by using information indicating whether the decoded residual signal prediction is performed. If the above information indicates that residual signal prediction is performed, the intra prediction unit 240 may perform residual signal prediction. If the above information indicates that residual signal prediction is not performed, the intra prediction unit 240 may not perform residual signal prediction.

If it is determined to perform residual signal prediction, step 2572 may be performed.

If it is determined not to perform residual signal prediction, step 2574 may be performed.

In operation 2572 , the intra residual predictor 2310 may identify a first neighboring block. The first neighboring block may be a block used for prediction of the residual signal, and may be a block located near the current block. The contents related to the determination of the first neighboring block described above with reference to FIG. 22 may also be applied to the present embodiment.

The intra residual predictor 2310 may identify the first neighboring block using the identifier of the first neighboring block described above with reference to FIGS. 9A and 10C .

The intra residual predictor 2310 may identify the first neighboring block by using the identifier of the decoded first neighboring block.

The identifier of the first neighboring block may be information that enables the neighboring block used for prediction of the residual signal of the current block to be identified.

For example, the identifier of the first neighboring block may indicate a neighboring block used for prediction of a residual signal of a current block among a plurality of neighboring blocks. Alternatively, the identifier of the first neighboring block may be position information indicating a position of a neighboring block used for prediction of a residual signal of a current block among a plurality of neighboring blocks. The position may indicate a relative position of the selected neighboring block with respect to the current block. The position may indicate a direction in which a neighboring block selected with respect to the current block is adjacent.

The position and number of neighboring blocks may be defined according to encoding parameters.

The identifier of the first neighboring block may be configured to indicate the same block in both the encoding apparatus 800 and the decoding apparatus 2300 . For example, in relation to the identifier of the first neighboring block, the size N of the block and the position of the neighboring block may have to be common to the encoding apparatus 800 and the decoding apparatus 2300 . In order for the encoding apparatus 800 and the decoding apparatus 2300 to share a common configuration with respect to the identifier of the first neighboring block, the identifier of the first neighboring block is "(neighboring-residual_index) truncation unary ((neighboring) -residual_idx) Truncated unary)" may be entropy-decoded.

Once step 2572 is performed, step 2573 may then be performed.

In operation 2573 , the intra residual predictor 2310 may generate a reconstructed block of the current block.

The intra residual predictor 2310 may generate a reconstructed block of the current block based on the prediction block, the residual signal of the current block, and the residual signal of the first neighboring block.

The residual signal of the current block used in operation 2573 may be a residual signal generated by prediction of the residual signal in the encoding apparatus 800 . In other words, the residual signal of the current block used in step 2573 may correspond to the first residual signal described above with reference to FIGS. 9A and 10C .

The first neighboring block may be a block that has already been restored before the current block is decoded. Accordingly, the residual signal of the first neighboring block may also be already obtained by the intra residual predictor 2310 before decoding of the current block.

The reconstruction block of the current block may be the sum of the prediction block of the current block, the residual signal of the current block, and the residual signal of the neighboring block. Also, the reconstruction block of the current block may be generated based on the sum of the residual signal of the current block and the residual signal of the first neighboring block.

The reconstruction block may be a sum of 1) a prediction block, 2) a residual signal of the current block, and 3) a residual signal of a first neighboring block. The prediction block may be obtained according to the intra prediction mode. The residual signal of the current block may be transmitted from the encoding apparatus 800 to the decoding apparatus 2300 . The residual signal of the first neighboring block may be obtained by residual signal prediction. For example, when residual signal prediction is not performed, the residual signal of the first neighboring block may be a zero signal.

In some cases, when the intra-residual prediction unit 2310 generates the residual signal of the first neighboring block according to the residual signal prediction, the intra-prediction unit 240 determines the prediction block of the current block, the residual signal of the current block, and the neighboring block. A reconstructed block of the current block may be generated by summing the residual signals.

If it is determined at step 2570 not to perform residual signal prediction, then step 2574 may be performed.

In operation 2574, the intra prediction unit 240 or the intra residual prediction unit 2310 may generate a reconstructed block of the current block based on the prediction block and the residual signal of the current block.

The residual signal of the current block used in operation 2574 may be a residual signal generated without performing residual signal prediction in the encoding apparatus 800 . In other words, the residual signal of the current block used in step 2574 may correspond to the third residual signal described above with reference to FIGS. 9A and 10C .

The third residual signal may be a residual signal of a current block in conventional image encoding and/or decoding techniques such as HEVC and AVC. For example, in generating the third residual signal, a residual signal generating method of an existing image encoding and/or decoding technique such as HEVC and AVC may be used.

Implicit transmission of information related to residual signal prediction

As described above, the bitstream may include 1) information indicating whether residual signal prediction is performed and 2) an identifier of the first neighboring block used for residual signal prediction of the current block. That is, the information and the identifier may be explicitly transmitted to the decoding apparatus 2300 by the encoding apparatus 800 .

On the other hand, when a predefined condition is satisfied in order to reduce the amount of bits required for the above information, 1) information indicating whether residual signal prediction is performed may be omitted. Even if the above information is omitted, the decoding apparatus 2300 may derive the above information when a predefined condition is satisfied. That is to say, the above information may be transmitted implicitly.

In addition, when a predefined condition is satisfied to reduce the amount of bits required for the identifier, 2) the identifier of the first neighboring block used for prediction of the residual signal of the current block may be omitted. Even if the above information is omitted, the decoding apparatus 2300 may derive the identifier when a predefined condition is satisfied. That is to say, the identifier may be transmitted explicitly.

In step 930 described above with reference to FIG. 9A and step 1041 described above with reference to FIG. 10C, when a predefined condition is satisfied, the intra residual prediction unit 810 predicts the residual signal for the current block. can decide what to do. Alternatively, when a predefined condition is satisfied, the intra residual predictor 810 may determine not to perform residual signal prediction on the current block.

In addition, when a predefined condition is satisfied in steps 935 and 1043 , the intra residual predictor 810 may determine the predefined block as the first neighboring block. Also, steps 980 , 985 , 1048 , and 1049 may be selectively performed.

In steps 980 and 1049 , the intra residual prediction unit 810 may selectively encode information indicating whether residual signal prediction is performed. Alternatively, when a predefined condition is satisfied, the intra residual predictor 810 may omit encoding of information indicating whether residual signal prediction is performed.

In steps 985 and 1048 , the intra residual predictor 810 may selectively encode the identifier of the first neighboring block. Alternatively, when a predefined condition is satisfied, the intra residual predictor 810 may omit encoding of the identifier of the first neighboring block.

In operation 2440 described above with reference to FIG. 24B , when a predefined condition is satisfied, the intra residual predictor 2310 may determine to perform residual signal prediction. Alternatively, when a predefined condition is satisfied, the intra residual predictor 2310 may determine not to perform residual signal prediction. For example, when information indicating whether residual signal prediction is not present, the intra residual predictor 2310 may determine to perform residual signal prediction.

Also, when a predefined condition is satisfied in operation 2445 , the intra residual predictor 2310 may identify the selected block as the first neighboring block according to a predefined method. For example, when the identifier of the first neighboring block does not exist, the intra residual predictor 2310 may identify the selected block as the first neighboring block according to a predefined method.

For example, when the directionality of the current block and the directionality of the first neighboring block are the same, high efficiency in prediction of the residual signal may be expected. Accordingly, when a block having the same intra prediction mode as the intra prediction mode of the current block exists around the current block, the residual signal prediction of the current block may be performed with respect to the block having the same intra prediction mode.

In step 930 described above with reference to FIG. 9A and step 1041 described above with reference to FIG. 10C, if there is a block having the same intra prediction mode as the intra prediction mode of the current block in the vicinity of the current block, intra The residual predictor 810 may determine to perform residual signal prediction on the current block. For example, if the MPM flag has a value of true (or "1"), the intra residual prediction unit 2310 indicates that a block having the same intra prediction mode as the intra prediction mode of the current block exists in the vicinity of the current block. can be detected, and it can be determined to perform residual signal prediction on the current block. Accordingly, the intra residual predictor 2310 may identify whether residual signal prediction is used based on the value of the MPM flag.

In addition, in steps 935 and 1043, when a block having the same intra prediction mode as the intra prediction mode of the current block exists around the current block, the intra residual prediction unit 810 performs the same intra prediction A block having a mode may be determined as a first neighboring block. When a plurality of blocks having the same intra prediction mode as the intra prediction mode of the current block exist around the current block, the intra residual prediction unit 810 selects a block selected according to a predefined priority among the plurality of blocks. It can be determined by 1 surrounding block.

In steps 980 and 1049, when a block having the same intra prediction mode as the intra prediction mode of the current block exists around the current block, the intra residual prediction unit 810 determines whether residual signal prediction is performed It is possible to omit encoding of information indicating

In steps 985 and 1048, when a block having the same intra prediction mode as the intra prediction mode of the current block exists in the vicinity of the current block, the intra residual prediction unit 810 determines the identifier of the first neighboring block. Encoding can be omitted.

In step 2440 described above with reference to FIG. 24B , when a block having the same intra prediction mode as the intra prediction mode of the current block exists around the current block, the intra residual prediction unit 2310 performs residual signal prediction can decide what to do. Alternatively, when there is no information indicating whether residual signal prediction is present, the intra-residual prediction unit 2310 predicts the residual signal when a block having the same intra-prediction mode as the intra-prediction mode of the current block exists in the vicinity of the current block. can decide what to do. Alternatively, when information indicating whether residual signal prediction is not present, the intra-residual prediction unit 2310 predicts the residual signal if a block having the same intra-prediction mode as the intra-prediction mode of the current block does not exist in the vicinity of the current block. You can decide not to do it.

In addition, in step 2445, when a block having the same intra prediction mode as the intra prediction mode of the current block exists around the current block, the intra residual prediction unit 2310 selects the block having the same intra prediction mode. It can be identified by the first neighboring block. Alternatively, when the identifier of the first block does not exist, the intra-residual prediction unit 2310 selects the same intra-prediction mode when a block having the same intra-prediction mode as the intra-prediction mode of the current block exists in the vicinity of the current block. A block having a block may be identified as a first neighboring block. Alternatively, when the identifier of the first block does not exist and a plurality of blocks having the same intra prediction mode as the intra prediction mode of the current block exist around the current block, the intra residual prediction unit 810 may select the plurality of blocks. Among them, the selected block may be determined as the first neighboring block according to a predefined priority.

When intra prediction is performed, the intra prediction mode may be transmitted from the encoding apparatus 800 to the decoding apparatus 2300 through a Most Probable Mode (MPM) flag and an MPM index. In other words, the intra prediction mode may be entropy-encoded by the MPM flag and the MPM index. MPM may indicate all three intra prediction modes. The intra prediction modes indicated by the MPM are called MPM candidate modes. The intra prediction unit 240 may identify MPM candidate modes through a prediction block in a picture adjacent to the current block.

If the intra prediction mode of the current block is the same as one of three intra prediction modes identified by the MPM, the value of the MPM flag may be true (or "1"). Also, when the value of the MPM flag is true (or "1"), the encoding apparatus 800 may transmit the MPM index to the decoding apparatus 2300 . The MPM index may indicate which mode among MPM candidate modes is the intra prediction mode of the current block.

The intra residual prediction unit 2310 may perform intra prediction according to the definition of the syntax. When the final intra prediction mode obtained after performing intra prediction is the same as one of the MPM candidate modes, the intra residual prediction unit 2310 may obtain a residual signal of the current block, and the residual signal of the first neighboring block can be used to predict the residual signal of the current block. When the intra prediction mode of the current block is the same as the intra prediction mode of the first neighboring block, the intra residual prediction unit 2310 may identify the position of the first neighboring block for prediction of the residual signal of the current block through the MPM index. have.

Unit of update of reference sample

In operation 2420, the intra predictor 240 may determine whether to update the reference sample for each predefined stage. The predefined unit may be at least one of 1) an entire image sequence (ie, video), 2) one image (ie, picture), 3) a slice, and 4) a coding unit.

For a predefined unit, reference sample update information may be used to indicate whether the reference sample is updated. The reference sample update information may be information indicating whether the reference sample is updated with respect to a predefined unit. For example, when the value of the reference sample update information is the first value, it may indicate that the reference sample should be updated in decoding the current block. When the value of the reference sample update information is the second value, it may indicate that the update of the reference sample is not performed in decoding the current block.

The encoding apparatus 800 may include the encoded residual signal prediction information in the bitstream. The decoding apparatus 2300 may determine whether residual signal prediction is performed on the current block by using the residual signal prediction information.

In the following, the update of the reference sample for each of the predefined units is described.

1) Entire image sequence: Whether to update the reference sample may be determined for the entire image sequence. In this case, the sequence parameter set may include reference sample update information. When the reference sample update information of the sequence parameter set indicates that the reference sample is to be updated, the intra prediction unit 240 may perform intra prediction using the reference sample to which the gradient according to the directionality is applied to the entire image sequence.

2) One image: Whether to update the reference sample may be determined for each image. In this case, the picture parameter set may include reference sample update information. If the reference sample update information of the picture parameter set indicates that the reference sample is to be updated, the intra prediction unit 240 performs intra prediction using the reference sample to which the gradient according to the directionality is applied to the entire image corresponding to the picture parameter set. can be done

3) Slice: One image may be divided into a plurality of slice segments or a plurality of tiles within one slice segment. Whether to update the reference sample may be determined for each slice. In this case, the slice segment header may include reference sample update information. If the reference sample update information of the slice segment header indicates that the reference sample is to be updated, the intra prediction unit 240 performs intra prediction using the reference sample to which the gradient according to the directionality is applied on the slice corresponding to the slice segment header. can

4) Coding unit: Whether to update the reference sample may be determined for each coding unit. In this case, reference sample update information may be present for the coding unit. When the reference sample update information for the coding unit indicates that the coding unit is updated, the intra prediction unit 240 performs intra prediction using the reference sample to which the gradient according to the directionality is applied with respect to the coding unit corresponding to the reference sample update information. can be done

As described above, the reference sample update information may be coded in a sequence parameter set, a picture parameter set, or a slice segment header. Also, update information may be coded for a coding unit.

Unit of Residual Signal Prediction

In operation 2440 , the intra residual predictor 2310 may determine whether to perform residual signal prediction for each predefined unit. The predefined unit may be at least one of 1) an entire image sequence (ie, video), 2) one image (ie, picture), 3) a slice, and 4) a coding unit.

The residual signal prediction information may be information indicating whether residual signal prediction is performed with respect to a predefined unit. For example, the first value of the residual signal prediction information may indicate that residual signal prediction should be performed in decoding the current block. The second value of the residual signal prediction information may indicate that residual signal prediction is not performed in decoding the current block.

The encoding apparatus 800 may include the encoded residual signal prediction information in the bitstream. The decoding apparatus 2300 may determine whether residual signal prediction is performed on the current block by using the residual signal prediction information.

In the following, the residual signal prediction for each of the predefined units is described.

1) Entire image sequence: Whether to perform residual signal prediction may be determined for the entire image sequence. In this case, the sequence parameter set may include residual signal prediction information. When the residual signal prediction information of the sequence parameter set indicates that the residual signal prediction is performed, the intra residual predictor 2310 may perform the residual signal prediction on the entire image sequence. All blocks encoded by intra prediction in an image sequence may be decoded using residual signal prediction or may be decoded without using residual signal prediction, depending on whether or not full signal prediction is performed.

2) One image: Whether to perform residual signal prediction may be determined for each image. In this case, the picture parameter set may include residual signal prediction information. When the residual signal prediction information of the picture parameter set indicates that residual signal prediction is performed, the intra residual prediction unit 2310 may perform residual signal prediction on the entire image corresponding to the picture parameter set. All blocks encoded by intra prediction within one destination may be decoded using residual signal prediction or decoded without using residual signal prediction, depending on whether or not full signal prediction is performed.

3) Slice: One image may be divided into a plurality of slice segments or a plurality of tiles within one slice segment. Whether to perform residual signal prediction may be determined for each slice. In this case, the slice segment header may include residual signal prediction information. When the residual signal prediction information of the slice segment header indicates that residual signal prediction is performed, the intra residual predictor 2310 may perform residual signal prediction on a slice corresponding to the slice segment header. When the slice segment header includes residual signal prediction information, all blocks encoded by intra prediction at the slice level are decoded using residual signal prediction or residual signal prediction is not used, depending on whether full signal prediction is performed. can be decrypted.

4) Coding unit: Whether to perform residual signal prediction may be determined for each coding unit. In this case, residual signal prediction information may be present for the coding unit. When the residual signal prediction information for the coding unit indicates that the coding unit is updated, the intra residual prediction unit 2310 may perform residual signal prediction on the coding unit corresponding to the residual signal prediction information.

As described above, the residual signal prediction information may be coded within a sequence parameter set, a picture parameter set, or a slice segment header. Also, update information may be coded for a coding unit.

26 is a structural diagram of an electronic device implementing an encoding device according to an embodiment.

According to an embodiment, the motion prediction unit 111 , the motion compensation unit 112 , the intra prediction unit 120 , the switch 115 , the subtractor 125 , the transform unit 130 , and the quantization of the encoding apparatus 800 . unit 140 , entropy encoding unit 150 , inverse quantization unit 160 , inverse transform unit 170 , adder 175 , filter unit 180 , reference picture buffer 190 , and intra residual prediction unit 810 . At least a portion of the module may be program modules, and may communicate with an external device or system. The program modules may be included in the encoding device 800 in the form of an operating system, an application program module, and other program modules.

Program modules may be physically stored on various known storage devices. In addition, at least some of these program modules may be stored in a remote storage device capable of communicating with the encoding device 800 .

Program modules include routines, subroutines, programs, objects, components, and data that perform functions or operations according to an embodiment or implement abstract data types according to an embodiment. It may include, but is not limited to, a data structure and the like.

The program modules may include instructions or codes that are executed by at least one processor of the encoding apparatus 800 .

The encoding device 800 may be implemented as the electronic device 2600 illustrated in FIG. 26 . The electronic device 2600 may be a general-purpose computer system operating as the encoding device 800 .

As shown in FIG. 26 , the electronic device 2600 includes at least one processor 2621 , a memory 2623 , and a user interface (UI) input device that communicate with each other through a bus 2622 . 2626 ), a UI output device 2627 , and storage 2628 . Also, the electronic device 2600 may further include a network interface 2629 connected to the network 2630 . The processor 2621 may be a semiconductor device that executes processing instructions stored in a central processing unit (CPU), the memory 2623 , or the storage 2628 . Memory 2623 and storage 2628 may be various forms of volatile or non-volatile storage media. For example, the memory may include at least one of a ROM 2624 and a RAM 2625 .

The encoding apparatus 800 may be implemented in a computer system including a recording medium that can be read by a computer.

The recording medium may store at least one module required for the electronic device 2600 to operate as the encoding device 800 . The memory 2623 may store at least one module and may be configured to be executed by the at least one processor 2621 .

A function related to data or information communication of the encoding device 800 may be performed through the network interface 2629 .

27 is a structural diagram of an electronic device implementing a decoding device according to an embodiment.

According to an embodiment, the entropy decoding unit 210 , the inverse quantization unit 220 , the inverse transform unit 230 , the intra prediction unit 240 , the motion compensator 250 , and the adder 255 of the decoding apparatus 2300 . , at least some of the filter unit 260 , the reference picture buffer 270 , and the intra residual prediction unit 2310 may be program modules, and may communicate with an external device or system. The program modules may be included in the decryption apparatus 2300 in the form of an operating system, an application program module, and other program modules.

Program modules may be physically stored on various known storage devices. Also, at least some of these program modules may be stored in a remote storage device capable of communicating with the decryption device 2300 .

Program modules include routines, subroutines, programs, objects, components, and data that perform functions or operations according to an embodiment or implement abstract data types according to an embodiment. It may include, but is not limited to, a data structure and the like.

The program modules may include instructions or codes that are executed by at least one processor of the decoding apparatus 2300 .

The decoding device 2300 may be implemented as the electronic device 2700 illustrated in FIG. 27 . The electronic device 2700 may be a general-purpose computer system operating as the decryption device 2300 .

As shown in FIG. 27 , the electronic device 2700 includes at least one processor 2721 , a memory 2723 , and a user interface (UI) input device that communicate with each other through a bus 2722 . 2726 ), a UI output device 2727 , and storage 2728 . Also, the electronic device 2700 may further include a network interface 2729 connected to the network 2730 . The processor 2721 may be a semiconductor device that executes processing instructions stored in a central processing unit (CPU), the memory 2723 , or the storage 2728 . Memory 2723 and storage 2728 may be various forms of volatile or non-volatile storage media. For example, the memory may include at least one of a ROM 2724 and a RAM 2725 .

The decryption apparatus 2300 may be implemented in a computer system including a recording medium that can be read by a computer.

The recording medium may store at least one module required for the electronic device 2700 to operate as the decryption device 2300 . The memory 2723 may store at least one module and may be configured to be executed by the at least one processor 2721 .

A function related to communication of data or information of the decryption apparatus 2300 may be performed through the network interface 2729 .

In the above-described embodiments, the methods are described on the basis of a flowchart as a series of steps or units, but the present invention is not limited to the order of steps, and some steps may occur in a different order or at the same time as other steps as described above. can In addition, those of ordinary skill in the art will recognize that the steps shown in the flowchart are not exclusive, other steps may be included, or that one or more steps in the flowchart may be deleted without affecting the scope of the present invention. You will understand.

The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the computer software field. Examples of the computer-readable recording medium include a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as a CD-ROM and DVD, and a magneto-optical medium such as a floppy disk. media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules for carrying out the processing according to the present invention, and vice versa.

In the above, the present invention has been described with specific matters such as specific components and limited embodiments and drawings, but these are provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , various modifications and variations can be devised from these descriptions by those of ordinary skill in the art to which the present invention pertains.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and not only the claims described below but also all modifications equivalently or equivalently to the claims described below belong to the scope of the spirit of the present invention. will do it

Claims (19)

In the video decoding method,
determining a value of a reference sample for prediction for a current block;
generating a prediction block for the current block using the reference sample; and
generating a reconstructed block for the current block using the prediction block
A video decoding method comprising: According to claim 1,
The value of the reference samples is determined based on a plurality of pixels in a horizontal line not adjacent to the current block or a plurality of pixels in a vertical line not adjacent to the current block. According to claim 1,
The value of the reference samples is determined based on at least one pixel of each horizontal line of the plurality of horizontal lines. 4. The method of claim 3,
The plurality of horizontal lines are positioned higher than the current block. According to claim 1,
The determining of the value of the reference sample is an update of the value of the sample. 6. The method of claim 5,
and the updating is performed based on a calculation using values of adjacent pixels in a horizontal line included in the current block. In the video encoding method,
determining a value of a reference sample for prediction for a current block;
generating a prediction block for the current block using the reference sample; and
generating a reconstructed block for the current block using the prediction block
An image encoding method comprising a. 8. The method of claim 7,
The value of the reference samples is determined based on a plurality of pixels in a horizontal line not adjacent to the current block or a plurality of pixels in a vertical line not adjacent to the current block. 8. The method of claim 7,
The value of the reference samples is determined based on at least one pixel of each horizontal line of a plurality of horizontal lines. 10. The method of claim 9,
The plurality of horizontal lines are positioned higher than the current block. 8. The method of claim 7,
and the determining of the value of the reference sample is an update of the value of the sample. 12. The method of claim 11,
and the updating is performed based on a calculation using values of adjacent pixels in a horizontal line included in the current block. A recording medium for recording a bitstream generated by the encoding method according to claim 7. A computer-readable recording medium for storing a bitstream, comprising:
The bitstream is
Encoded information about the current block
including,
Decoding is performed on the current block using the encoded information,
In the decoding, a value of a reference sample for prediction with respect to the current block is determined, a prediction block for the current block is generated using the reference sample, and the prediction block is applied to the current block using the prediction block. A computer-readable recording medium on which the reconstructed blocks are generated. 15. The method of claim 14,
The value of the reference samples is determined based on a plurality of pixels in a horizontal line not adjacent to the current block or a plurality of pixels in a vertical line not adjacent to the current block. 15. The method of claim 14,
The value of the reference samples is determined based on at least one pixel of each horizontal line of the plurality of horizontal lines. 16. The method of claim 15,
The plurality of horizontal lines are located above the current block. 15. The method of claim 14,
and determining the value of the reference sample is an update of the value of the sample. 19. The method of claim 18,
wherein the updating is performed based on a calculation using values of adjacent pixels in a horizontal line included in the current block.

Images