CN115460409A - Method and apparatus for encoding and decoding video by using prediction - Google Patents

Abstract

Disclosed are a method and apparatus for encoding and decoding video by using prediction. In encoding a current block, a residual signal of the current block is generated based on the current block, a first prediction, and a second prediction. In addition, information on the encoded residual signal is generated by encoding the residual signal. The second prediction is a prediction for a portion of the current block. In decoding a current block, a reconstructed residual signal for the current block is generated. A reconstructed block for the current block is generated based on the reconstructed residual signal, the second prediction, and the first prediction. The second prediction is a prediction for a portion of the current block.

Description

Method and apparatus for encoding and decoding video by using prediction

The present application is a divisional application of an inventive patent application having an application date of 2017, 26.01/h, application number of "201780008673.3", entitled "method and apparatus for encoding and decoding video by using prediction".

Technical Field

The following embodiments relate generally to a video decoding method and apparatus and a video encoding method and apparatus, and more particularly, to a method and apparatus for performing encoding and decoding on video based on prediction of a target block.

Background

With the continuous development of the information and communication industry, broadcasting services having High Definition (HD) resolution have been popularized throughout the world. Through this popularity, a large number of users have become accustomed to high resolution and high definition images and/or videos.

In order to meet the demand of users for high definition, a large number of mechanisms have accelerated the development of next-generation imaging devices. In addition to the increasing interest of users in High Definition TV (HDTV) and Full High Definition (FHD) TV, the interest in Ultra High Definition (UHD) TV, which has a resolution more than four times that of full high definition (FUD) TV, has also increased. With this increase in interest, image encoding/decoding techniques for images with higher resolution and higher definition are required.

The image encoding/decoding apparatus and method may use an inter prediction technique, an intra prediction technique, an entropy encoding technique, etc. in order to perform encoding/decoding on high resolution and high definition images. The inter prediction technique may be a technique for predicting values of pixels included in a current picture using temporally preceding pictures and/or temporally succeeding pictures. The intra prediction technique may be a technique for predicting values of pixels included in a current picture using information on the pixels in the current picture. The entropy coding technique may be a technique for allocating short codes to more frequently occurring symbols and long codes to less frequently occurring symbols.

In the image encoding and decoding processes, prediction may mean generating a prediction signal similar to an original signal. Predictions can be classified primarily as: prediction with reference to spatially reconstructed images, prediction with reference to temporally reconstructed images, and prediction with reference to other symbols. In other words, the temporal reference may indicate that the temporally reconstructed image is referred to, and the spatial reference may indicate that the spatially reconstructed image is referred to.

The current block may be a block that is a target to be currently encoded or decoded. The current block may be referred to as a "target block" or a "target unit". In the encoding process, the current block may be referred to as an "encoding target block" or an "encoding target unit". In the decoding process, the current block may be referred to as a "decoding target block" or a "decoding target unit".

Inter prediction may be a technique for predicting a current block using temporal and spatial references. Intra-prediction may be a technique for predicting a current block using only spatial references.

When performing intra prediction to reduce spatial repetition, the image encoding/decoding technique encodes a current block using any one of a plurality of prediction modes. For example, in High Efficiency Video Coding (HEVC), 35 intra prediction modes are used.

An encoding apparatus generates a prediction block of a current block using all available prediction modes among a plurality of prediction modes, and selects one prediction mode exhibiting the best result as a prediction mode for encoding the current block from among the all available prediction modes. However, even if a prediction mode showing the best result is used, there is still an error between the original block and the prediction block. Such errors are represented as residual blocks.

For some pixels, there is a large error between the original block and the predicted block. Due to this large error, spatial repetition may not be sufficiently reduced even after applying a transform and/or quantization to the residual block.

Since the intra prediction is a prediction of the entire block in one direction, a large error may occur in a portion of the block.

Disclosure of Invention

Technical problem

Embodiments are directed to a method and apparatus for reducing prediction error based on a prediction unit occurring due to prediction.

Embodiments are directed to a method and apparatus for reducing a prediction error occurring due to a prediction direction of a prediction mode.

Embodiments are directed to a method and apparatus for performing an optional second prediction on a prediction error corresponding to a first prediction made in a direction of a first prediction mode.

Embodiments are directed to providing a method and apparatus for partitioning a current block into two partitions having the same number of pixels and performing a second prediction on the two partitions.

Embodiments are directed to a method and apparatus for using a second prediction for one of a plurality of partitions of a current block, in which a prediction error caused by a first prediction is large.

Embodiments are directed to providing a method and apparatus for reducing a prediction error in a current block by using a second prediction.

Embodiments are directed to providing a method and apparatus for providing a higher compression effect to a current block by using a second prediction.

Solution scheme

According to an aspect, there is provided an encoding method comprising: generating a residual signal of the current block based on the current block, the first prediction, and the second prediction; generating information on the encoded residual signal by performing encoding on the residual signal, wherein the second prediction is a prediction for a portion of the current block.

There is provided a decoding apparatus including: a reconstructed residual signal generating unit for generating a reconstructed residual signal for the current block; a reconstructed block generation unit for generating a reconstructed block for the current block based on the reconstructed residual signal, a second prediction and the first prediction, wherein the second prediction is a prediction for a portion of the current block.

There is provided a decoding method comprising: generating a reconstructed residual signal for a current block; a reconstructed block for the current block is generated based on the reconstructed residual signal, a second prediction, and the first prediction, wherein the second prediction is a prediction for a portion of the current block.

The first prediction and the second prediction may both be intra prediction.

The portion may have a non-square shape.

The portion may be determined based on a prediction mode of the first prediction.

The portion may be determined based on a straight line passing through the center of the current block in the prediction direction of the first prediction.

The first prediction and the second prediction may have different prediction directions.

The portion may be one of two portions generated by partitioning the current block.

The two portions may have the same number of pixels.

Each of the two portions may have a non-square shape.

The two portions may be determined based on a straight line passing through the center of the current block in the prediction direction of the first prediction.

The portion may be selected based on selected portion information indicating one of the two portions.

When the prediction mode of the first prediction is a non-directional mode, the second prediction may not be used.

When the second prediction usage information indicating whether the second prediction will be used to encode the current block indicates that the second prediction will not be used, the second prediction may not be used.

When the second prediction is not used, the first prediction may be a prediction of the entire current block.

A second prediction signal generated via a second prediction for the portion may be added to the reconstructed residual signal.

The first prediction may be a prediction for a remaining portion of the current block except for the portion.

A first prediction signal generated via a first prediction for the remaining portion may be added to a reconstructed residual signal.

The reconstructed block may be generated based on a second prediction signal generated via a second prediction for the portion and a first prediction signal generated via a first prediction for the remaining portion.

Advantageous effects

A method and apparatus for reducing prediction error based on a prediction unit occurring due to prediction are provided.

A method and apparatus for reducing a prediction error occurring due to a prediction direction of a prediction mode are provided.

A method and apparatus for performing selective second prediction of a prediction error corresponding to a first prediction made in a direction of a first prediction mode are provided.

A method and apparatus for partitioning a current block into two partitions having the same number of pixels and performing second prediction on the two partitions are provided.

A method and apparatus for using a second prediction for one of a plurality of partitions of a current block, for which a prediction error caused by a first prediction is large, are provided.

Embodiments are provided that aim to provide a method and apparatus for reducing prediction error in a current block by using a second prediction.

A method and apparatus for providing a higher compression effect for a current block by using a second prediction are provided.

Drawings

Fig. 1 is a block diagram showing a configuration of an embodiment of an encoding apparatus to which the present invention is applied;

fig. 2 is a block diagram showing a configuration of an embodiment of a decoding apparatus to which the present invention is applied;

fig. 3 is a diagram schematically showing a partition structure of an image when the image is encoded and decoded;

fig. 4 is a diagram illustrating a shape of a Prediction Unit (PU) that a Coding Unit (CU) can include;

fig. 5 is a diagram illustrating a shape of a Transform Unit (TU) that can be included in a CU;

FIG. 6 is a diagram for explaining an embodiment of an intra prediction process;

fig. 7 is a diagram for explaining positions of reference samples used in an intra prediction process;

FIG. 8 is a diagram for explaining an embodiment of an inter prediction process;

fig. 9 is a diagram illustrating a prediction error when an original image is predicted in a vertical direction according to an embodiment;

fig. 10 is a configuration diagram of an encoding device according to an embodiment;

FIG. 11 is a flow diagram of an encoding method according to an embodiment;

fig. 12 is a diagram illustrating first prediction and second prediction using intra prediction according to an embodiment;

FIG. 13 is a flow diagram of a method of residual block generation according to an embodiment;

FIG. 14 is a flow diagram of a reconstructed block generation method according to an embodiment;

FIG. 15 is a diagram illustrating an encoding process according to an embodiment;

fig. 16 is a configuration diagram of a decoding apparatus according to an embodiment;

fig. 17 is a flowchart of a decoding method according to an embodiment.

Best mode for carrying out the invention

The following exemplary embodiments will be described in detail with reference to the accompanying drawings showing specific embodiments. These embodiments are described so that those of ordinary skill in the art to which this disclosure pertains will be readily able to practice them. It should be noted that the various embodiments are different from one another, but are not necessarily mutually exclusive. For example, particular shapes, structures and characteristics described herein may be implemented as other embodiments without departing from the spirit and scope of the embodiments in connection with one embodiment. In addition, it is to be understood that the location or arrangement of individual components within each disclosed embodiment can be modified without departing from the spirit and scope of the embodiments. Therefore, the appended detailed description is not intended to limit the scope of the disclosure, and the scope of exemplary embodiments is defined only by the appended claims and equivalents thereof, as they are properly described.

In the drawings, like numerals are used to designate the same or similar functions in various respects. The shapes, sizes, and the like of components in the drawings may be exaggerated for clarity of the description.

It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. Further, it should be noted that, in the exemplary embodiments, the expression for describing a component "including" a specific component means that another component may be included in a practical scope or technical spirit of the exemplary embodiments, but does not exclude the presence of components other than the specific component.

Terms such as "first" and "second" may be used to describe various components, but the components are not limited by the terms. The terms are only used to distinguish one component from another component. For example, a first component may be termed a second component without departing from the scope of the present description. Similarly, the second component may be referred to as the first component.

Further, components described in the embodiments are independently illustrated to represent different feature functions, but this does not mean that each component is formed of a separate hardware or software. That is, a plurality of components are individually arranged and included for convenience of description. For example, at least two of the plurality of components may be integrated into a single component. Instead, one component may be divided into a plurality of components. Embodiments in which a plurality of components are integrated or embodiments in which some components are separated are included in the scope of the present specification as long as they do not depart from the essence of the present specification.

Further, some components are not essential components for performing essential functions, but may be optional components for only improving performance. Embodiments may be implemented using only the necessary components to implement the essence of the embodiments. For example, a structure including only necessary components excluding optional components only for improving performance is also included in the scope of the embodiments.

The embodiments will be described in detail below with reference to the accompanying drawings so that those skilled in the art to which the embodiments pertain can easily practice the embodiments. In the following description of the embodiments, a detailed description of known functions or configurations incorporated herein will be omitted.

Hereinafter, "image" may represent a single picture constituting a part of a video, or may represent the video itself. For example, "encoding and/or decoding of an image" may mean "encoding and/or decoding of a video", and may also mean "encoding and/or decoding of any one of a plurality of images constituting a video".

Hereinafter, the terms "video" and "moving picture" may be used to have the same meaning and may be interchanged with each other.

Hereinafter, the terms "image", "picture", "frame", and "screen" may be used to have the same meaning and may be interchanged with one another.

In the following embodiments, particular information, data, flags, elements, and attributes may have their respective values. A value of 0 corresponding to each of the information, data, flags, elements, and attributes may indicate a logical false or a first predefined value. In other words, the value "0" (logical false) and the first predefined value are interchangeable with each other. A value of "1" corresponding to each of the information, data, flags, elements, and attributes may indicate a logical true or a second predefined value. In other words, the value "1" (logically true) and the second predefined value are interchangeable with each other.

When a variable such as i or j is used to indicate a row, column, or index, the value i may be an integer 0 or an integer greater than 0, or may be an integer 1 or an integer greater than 1. In other words, in an embodiment, each of the rows, columns, and indexes may count from 0, or may count from 1.

Hereinafter, terms to be used in the embodiments will be described.

A unit: the "unit" may represent a unit of image encoding and decoding. The terms "unit" and "block" may have the same meaning as each other. Furthermore, the terms "unit" and "block" are interchangeable with one another.

A cell (or block) may be a matrix of M × N samples. M and N may be positive integers, respectively. The term "cell" may generally denote an array of two-dimensional (2D) spots. The term "samples" may be pixels or pixel values.

The terms "pixel" and "sample" may be used to have the same meaning and may be interchanged with one another.

During the encoding and decoding of an image, a "unit" may be a region produced by partitioning an image. A single image may be partitioned into multiple cells. In encoding and decoding an image, a process predefined for each unit may be performed according to the type of the unit. According to functions, the types of units may be classified into macro-units, coding Units (CUs), prediction Units (PUs), and Transform Units (TUs). A single cell may be further partitioned into lower level cells having a size smaller than the size of the cell.

The cell partition information may comprise information about the depth of the cell. The depth information may indicate the number of times and/or degree to which the unit is partitioned.

A single cell may be hierarchically partitioned into a plurality of lower level cells, while the plurality of lower level cells have tree structure based depth information. In other words, a cell and a lower level cell generated by partitioning the cell may correspond to a node and a child node of the node, respectively. Each partitioned lower level cell may have depth information. The depth information of a cell indicates the number of times and/or degree to which the cell is partitioned, and thus, the partition information of a lower cell may include information on the size of the lower cell.

In the tree structure, the top node may correspond to the initial node before partitioning. The top node may be referred to as the "root node". Further, the root node may have a minimum depth value. Here, the depth of the top node may be level "0".

A node with a depth of level "1" may represent a unit generated when an initial unit is partitioned once. A node with a depth of level "2" may represent a cell that is generated when an initial cell is partitioned twice.

A leaf node with a depth of level "n" may represent a unit that is generated when an initial unit is partitioned n times.

A leaf node may be the bottom node, which cannot be partitioned further. The depth of a leaf node may be a maximum level. For example, the predefined value for the maximum level may be 3.

-a Transform Unit (TU): a TU may be a basic unit of residual signal encoding and/or residual signal decoding, such as transform, inverse transform, quantization, inverse quantization, transform coefficient encoding, and transform coefficient decoding. A single TU may be partitioned into multiple TUs, wherein each TU of the multiple TUs has a smaller size.

-a Prediction Unit (PU): the PU may be a basic unit in the performance of prediction or compensation. A PU may be partitioned into multiple partitions by performing the partitioning. The plurality of partitions may also be basic units in the performance of prediction or compensation. The partitions generated via partitioning a PU may also be prediction units.

-reconstructed neighboring cells: the reconstructed neighboring cells may be cells that have been previously encoded or decoded and reconstructed in the vicinity of the encoding target cell or the decoding target cell. The reconstructed neighboring cells may be spatially neighboring cells to the target cell or temporally neighboring cells to the target cell.

-prediction unit partitioning: the prediction unit partition may represent a shape in which the PU is partitioned.

-a set of parameters: the parameter set may correspond to information regarding a header of a structure of the bitstream. For example, the parameter sets may include sequence parameter sets, picture parameter sets, adaptation parameter sets, and the like.

-rate distortion optimization: the encoding device may use rate-distortion optimization to provide higher coding efficiency by utilizing a combination of: the size of the CU, the prediction mode, the size of the prediction unit, the motion information, and the size of the TU.

-rate distortion optimization scheme: the scheme may calculate the rate-distortion cost of each combination to select the optimal combination from among the combinations. The rate-distortion cost may be calculated using equation 1 below. In general, the combination that minimizes the rate-distortion cost can be selected as the optimal combination under the rate-distortion optimization method.

[ equation 1]

D+λ*R

Here, D may represent distortion. D may be the average of the squares of the differences between the original transform coefficients and the reconstructed transform coefficients in the transform block (mean square error).

R denotes a code rate, which can represent a bit rate using relevant context information.

λ represents the lagrange multiplier. R may include not only coding parameter information such as a prediction mode, motion information, and a coding block flag, but also bits generated as a result of coding transform coefficients.

The encoding apparatus performs processes such as inter-prediction and/or intra-prediction, transformation, quantization, entropy coding, inverse quantization, and inverse transformation in order to calculate accurate D and R, but these processes may greatly increase the complexity of the encoding apparatus.

-a reference picture: the reference picture may be an image used for inter prediction or motion compensation. The reference picture may be a picture including a reference unit that is referred to by the target unit to perform inter prediction or motion compensation. The terms "picture" and "image" may have the same meaning. Thus, the terms "picture" and "image" are interchangeable with one another.

-reference picture list: the reference picture list may be a list including reference pictures used for inter prediction or motion compensation. The type of the reference picture list may be a merged List (LC), a list 0 (L0), a list 1 (L1), or the like.

-Motion Vector (MV): the MV may be a 2D vector used for inter prediction. For example, can be expressed as (mv) _x ，mv _y ) Represents the MV. mv _x Can indicate the horizontal component, mv _y A vertical component may be indicated.

The MV may represent the offset between the target picture and the reference picture.

-search scope: the search range may be a 2D region in which a search for MVs is performed during inter prediction. For example, the size of the search range may be M × N. M and N may be positive integers, respectively.

Fig. 1 is a block diagram showing the configuration of an embodiment of an encoding apparatus to which the present invention is applied.

The encoding apparatus 100 may be a video encoding apparatus or an image encoding apparatus. A video may comprise one or more images (pictures). The encoding device 110 may encode one or more images of a video sequentially over time.

Referring to fig. 1, the encoding apparatus 100 includes an inter prediction unit 110, an intra prediction unit 120, a switch 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filtering unit 180, and a reference picture buffer 190.

The encoding apparatus 100 may perform encoding on an input image in an intra mode and/or an inter mode. The input image may be referred to as a "current image" as a target to be currently encoded.

In addition, the encoding apparatus 100 may generate a bitstream including information on encoding by encoding an input image, and may output the generated bitstream.

When the intra mode is used, the switch 115 may switch to the intra mode. When the inter mode is used, the switch 115 may switch to the inter mode.

The encoding apparatus 100 may generate a prediction block for an input block in an input image. Also, after the prediction block is generated, the encoding apparatus 100 may encode a residual between the input block and the prediction block. An input block may be referred to as a "current block" as a target to be currently encoded.

When the prediction mode is the intra mode, the intra prediction unit 120 may use pixel values of previously encoded neighboring blocks around the current block as reference pixels. The intra prediction unit 120 may perform spatial prediction on the current block using the reference pixels and generate prediction samples for the current block via the spatial prediction.

The inter prediction unit 110 may include a motion prediction unit and a motion compensation unit.

When the prediction mode is an inter mode, the motion prediction unit may search for a region that most matches the current block in a reference picture in a motion prediction process, and may derive a motion vector for the current block and the found region. The reference image may be stored in the reference picture buffer 190. More specifically, when encoding and/or decoding of a reference image is processed, the reference image may be stored in the reference picture buffer 190.

The motion compensation unit may generate the prediction block by performing motion compensation using the motion vector. Here, the motion vector may be a two-dimensional (2D) vector for inter prediction. In addition, the motion vector may represent an offset between the current picture and the reference picture.

Subtractor 125 may generate a residual block, where the residual block is the residual between the input block and the prediction block. The residual block is also referred to as a "residual signal".

The transform unit 130 may generate transform coefficients by transforming the residual block, and may output the generated transform coefficients. Here, the transform coefficient may be a coefficient value generated by transforming the residual block. When the transform skip mode is used, the transform unit 130 may omit an operation of transforming the residual block.

By performing quantization on the transform coefficients, quantized transform coefficient levels may be generated. Here, in the embodiment, the quantized transform coefficient level may also be referred to as a "transform coefficient".

The quantization unit 140 may generate quantized transform coefficient levels by quantizing the transform coefficients according to a quantization parameter. The quantization unit 140 may output quantized transform coefficient levels. In this case, the quantization unit 140 may quantize the transform coefficient using a quantization matrix.

The entropy encoding unit 150 may generate a bitstream by performing probability distribution-based entropy encoding based on the values calculated by the quantization unit 140 and/or encoding parameter values calculated in the encoding process. The entropy encoding unit 150 may output the generated bitstream.

In addition to the pixel information of the image, the entropy encoding unit 150 may perform entropy encoding on information required to decode the image. For example, information required for decoding an image may include syntax elements and the like.

The encoding parameters may be information required for encoding and/or decoding. The encoding parameters may include information encoded by the encoding apparatus and transmitted to the decoding apparatus, and may also include information derived in the encoding or decoding process. For example, the information transmitted to the decoding apparatus may include a syntax element.

For example, the encoding apparatus may include values or statistical information such as a prediction mode, a motion vector, a reference picture index, a coded block pattern, the presence or absence of a residual signal, a transform coefficient, a quantized transform coefficient, a quantization parameter, a block size, and block partition information. The prediction mode may be an intra prediction mode or an inter prediction mode.

The residual signal may represent the difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming a difference between the original signal and the prediction signal. Alternatively, the residual signal may be a signal generated by transforming and quantizing the difference between the original signal and the prediction signal. The residual block may be a block-based residual signal.

When entropy coding is applied, fewer bits may be allocated to more frequently occurring symbols and more bits may be allocated to less frequently occurring symbols. Since the symbols are represented by this allocation, the size of the bit string for the target symbol to be encoded can be reduced. Accordingly, the compression performance of video encoding can be improved by entropy encoding.

Also, for entropy encoding, an encoding method such as exponential golomb, context-adaptive variable length coding (CAVLC), or context-adaptive binary arithmetic coding (CABAC) may be used. For example, entropy encoding unit 150 may perform entropy encoding using a variable length coding/code (VLC) table. For example, the entropy encoding unit 150 may derive a binarization method for the target symbol. Furthermore, entropy encoding unit 150 may derive a probability model for the target symbol/bin. The entropy encoding unit 150 may perform entropy encoding using the derived binarization method or probability model.

Since the encoding apparatus 100 performs encoding via inter prediction, the encoded current picture can be used as a reference picture for another picture to be subsequently processed. Accordingly, the encoding apparatus 100 may decode the encoded current picture and store the decoded picture as the reference picture. For decoding, inverse quantization and inverse transformation of the encoded current image may be performed.

The quantized coefficients may be inverse quantized by inverse quantization unit 160 and inverse transformed by inverse transformation unit 170. The coefficients that have been inverse quantized and inverse transformed may be added to the prediction block by adder 175. The inverse quantized and inverse transformed coefficients and the prediction block are added, and then a reconstructed block may be generated.

The reconstructed block may be filtered by the filtering unit 180. Filtering unit 180 may apply one or more of a deblocking filter, a Sample Adaptive Offset (SAO) filter, and an Adaptive Loop Filter (ALF) to the reconstructed block or reconstructed picture. The filtering unit 180 may also be referred to as an "adaptive in-loop filter".

The deblocking filter may remove block distortion occurring at the boundary of the block. The SAO filter may add appropriate offset values to the pixel values in order to compensate for the coding error. ALF may perform filtering based on the comparison between the reconstructed block and the original block. The reconstructed block that has been filtered by the filtering unit 180 may be stored in the reference picture buffer 190.

Fig. 2 is a block diagram showing the configuration of an embodiment of a decoding apparatus to which the present invention is applied.

The decoding apparatus 200 may be a video decoding apparatus or an image decoding apparatus.

Referring to fig. 2, the decoding apparatus 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra prediction unit 240, an inter prediction unit 250, an adder 255, a filtering unit 260, and a reference picture buffer 270.

The decoding apparatus 200 may receive the bitstream output from the encoding apparatus 100. The decoding apparatus 200 may perform decoding on the bitstream in an intra mode and/or an inter mode. Further, the decoding apparatus 200 may generate a reconstructed image via decoding, and may output the reconstructed image.

For example, an operation of switching to an intra mode or an inter mode based on a prediction mode for decoding may be performed by a switch. When the prediction mode for decoding is an intra mode, the switch may be operated to switch to the intra mode. When the prediction mode for decoding is an inter mode, the switch may be operated to switch to the inter mode.

The decoding apparatus 200 may acquire a reconstructed residual block from an input bitstream and may generate a prediction block. When the reconstructed residual block and the prediction block are acquired, the decoding apparatus 200 may generate a reconstructed block by adding the reconstructed residual block to the prediction block.

The entropy decoding unit 210 may generate symbols by performing entropy decoding on the bitstream based on the probability distribution. The generated symbols may comprise quantized coefficient format symbols. Here, the entropy decoding method may be similar to the entropy encoding method described above. That is, the entropy decoding method may be the inverse process of the entropy encoding method described above.

The quantized coefficients may be inverse quantized by the inverse quantization unit 220. Also, the inverse quantized coefficients may be inverse transformed by the inverse transform unit 230. As a result of inverse quantization and inverse transformation of the quantized coefficients, a reconstructed residual block may be generated. Here, the inverse quantization unit 220 may apply a quantization matrix to the quantized coefficients.

When the intra mode is used, the intra prediction unit 240 may generate a prediction block by performing spatial prediction using pixel values of previously encoded neighboring blocks around the current block.

The inter prediction unit 250 may include a motion compensation unit. When the inter mode is used, the motion compensation unit 250 may generate a prediction block by performing motion compensation using a motion vector and a reference picture. The reference image may be stored in the reference picture buffer 270.

The reconstructed residual block and the prediction block may be added to each other by an adder 255. The adder 255 may generate a reconstructed block by adding the reconstructed residual block and the prediction block.

The reconstructed block may be filtered by the filtering unit 160. The filtering unit 260 may apply one or more of a deblocking filter, an SAO filter, and an ALF to the reconstructed block or the reconstructed picture. The filtering unit 260 may output a reconstructed image (picture). The reconstructed image may be stored in the reference picture buffer 270 and then may be used for inter prediction.

Fig. 3 is a diagram schematically showing an image partition structure when an image is encoded and decoded.

In order to partition an image efficiently, a Coding Unit (CU) may be used in encoding and decoding. The term "unit" may be used to collectively specify 1) a block comprising image samples and 2) syntax elements. For example, "partition of a unit" may represent "partition of a block corresponding to the unit".

Referring to fig. 3, the picture 200 is sequentially partitioned into units corresponding to a maximum coding unit (LCU), and a partition structure of the picture 300 may be determined according to the LCU. Here, the LCU may be used to have the same meaning as a Coding Tree Unit (CTU).

The partition structure may represent the distribution of Coding Units (CUs) in the LCU 310 for efficient encoding of the image. Such a distribution may be determined according to whether a single CU is to be partitioned into four CUs. The horizontal and vertical sizes of each CU resulting from the partitioning may be half of the horizontal and vertical sizes of the CU before being partitioned. Each partitioned CU may be recursively partitioned into four CUs, and in the same manner, the horizontal and vertical sizes of the four CUs are halved.

Here, partitioning of a CU may be performed recursively until a predefined depth. The depth information may be information indicating a size of the CU. Depth information may be stored for each CU. For example, the depth of the LCU may be 0, and the depth of the minimum coding unit (SCU) may be a predefined maximum depth. Here, as described above, the LCU may be a CU having a largest coding unit size, and the SCU may be a CU having a smallest coding unit size.

Partitioning begins at LCU 310, and the depth of a CU may increase by "1" whenever the horizontal and vertical dimensions of the CU are halved by partitioning. For each depth, a CU that is not partitioned may have a size of 2N × 2N. Further, in the case where CUs are partitioned, a CU of size 2N × 2N may be partitioned into four CUs each of size N × N. Dimension N may be halved each time the depth is increased by 1.

Referring to fig. 3, an LCU having a depth of 0 may have 64 × 64 pixels. 0 may be a minimum depth. An SCU with a depth of 3 may have 8 x 8 pixels. 3 may be the maximum depth. Here, a CU having 64 × 64 pixels as an LCU may be represented by depth 0. A CU with 32 × 32 pixels may be represented by depth 1. A CU with 16 × 16 pixels may be represented by depth 2. A CU with 8 × 8 pixels as an SCU may be represented by depth 3.

Also, information on whether a corresponding CU is partitioned may be represented by partition information of the CU. The partition information may be 1-bit information. All CUs except the SCU may include partition information. For example, when a CU is not partitioned, the value of the partition information of the CU may be 0. When a CU is partitioned, the value of the partition information of the CU may be 1.

Fig. 4 is a diagram illustrating a shape of a Prediction Unit (PU) that a Coding Unit (CU) can include.

Among CUs partitioned from the LCU, CUs that are no longer partitioned may be divided into one or more Prediction Units (PUs). This division may also be referred to as "partitioning".

A PU may be the basic unit for prediction. A PU may be encoded and decoded in any one of skip mode, inter mode, and intra mode. The PUs may be partitioned into various shapes according to various modes.

In skip mode, there may be no partition in a CU. In the skip mode, the 2N × 2N mode 410 may be supported without partitioning, wherein the size of the PU and the size of the CU are identical to each other in the 2N × 2N mode.

In inter mode, there may be 8 types of partition shapes in a CU. For example, in the inter mode, a 2N × 2N mode 410, a 2N × N mode 415, an N × 2N mode 420, an N × N mode 425, a 2N × nU mode 430, a 2N × nD mode 435, an nL × 2N mode 440, and an nR × 2N mode 445 may be supported.

In intra mode, a 2N × 2N mode 410 and an N × N mode 425 may be supported.

In the 2N × 2N mode 410, PUs of size 2N × 2N may be encoded. A PU of size 2N × 2N may represent a PU of the same size as the CU. For example, a PU of size 2N × 2N may have a size 64 × 64, 32 × 32, 16 × 16, or 8 × 8.

In the nxn mode 425, PUs of size nxn may be encoded.

For example, in intra prediction, when the size of a PU is 8 × 8, four partitioned PUs may be encoded. The size of each partitioned PU may be 4 x 4.

When a PU is encoded in intra mode, the PU may be encoded using any one of a plurality of intra prediction modes. For example, HEVC techniques may provide 35 intra prediction modes, a PU may be encoded under any one of the 35 intra prediction modes.

Which of the 2 nx 2N mode 410 and the nxn mode 425 is to be used to encode the PU may be determined based on a rate-distortion cost.

The encoding apparatus 100 may perform an encoding operation on PUs having a size of 2N × 2N. Here, the encoding operation may be an operation of encoding the PU in each of a plurality of intra prediction modes that can be used by the encoding apparatus 100. Through the encoding operation, an optimal intra prediction mode for a PU of size 2N × 2N may be obtained. The optimal intra prediction mode may be an intra prediction mode in which a minimum rate-distortion cost occurs when a PU having a size of 2N × 2N is encoded, among a plurality of intra prediction modes that can be used by the encoding apparatus 100.

Further, the encoding apparatus 100 may sequentially perform an encoding operation on the respective PUs obtained by performing the N × N partitioning. Here, the encoding operation may be an operation of encoding the PU in each of a plurality of intra prediction modes that can be used by the encoding apparatus 100. Through the encoding operation, an optimal intra prediction mode for a PU of size N × N may be obtained. The optimal intra prediction mode may be an intra prediction mode in which a minimum rate-distortion cost occurs when a PU of size N × N is encoded, among a plurality of intra prediction modes that can be used by the encoding apparatus 100.

The encoding apparatus 100 may determine which one of a PU of size 2N × 2N and a PU of size N × N is to be encoded based on a comparison result between a rate distortion cost of the PU of size 2N × 2N and a rate distortion cost of the PU of size N × N.

Fig. 5 is a diagram illustrating a shape of a Transform Unit (TU) that can be included in a CU.

A Transform Unit (TU) may be a basic unit used in a CU for processes such as transform, quantization, inverse transform, inverse quantization, entropy coding, and entropy decoding. The TU may have a square or rectangular shape.

Among CUs partitioned from an LCU, CUs that are no longer partitioned as CUs may be partitioned into one or more TUs. Here, the partition structure of the TU may be a quad tree structure. For example, as shown in fig. 5, a single CU 510 may be partitioned one or more times according to a quadtree structure. With such partitioning, a single CU 510 may be composed of TUs having various sizes.

In the encoding apparatus 100, a Coding Tree Unit (CTU) of size 64 × 64 may be partitioned into a plurality of smaller CUs using a recursive tree structure. A single CU may be partitioned into four CUs having the same size. Each CU may be recursively partitioned and may have a quadtree structure.

A CU may have a given depth. When a CU is partitioned, the CU resulting from the partitioning operation may have a depth increased by 1 from the depth of the partitioned CU.

For example, the depth of a CU may have a value ranging from 0 to 3. The size of a CU may range from size 64 × 64 to size 8 × 8, depending on the depth of the CU.

By recursively partitioning the CUs, the optimal partitioning method that incurs the smallest rate-distortion cost can be selected.

Fig. 6 is a diagram for explaining an embodiment of an intra prediction process.

An arrow radially extending from the center of the graph in fig. 6 may represent a prediction direction of the intra prediction mode. Further, numerals shown near the arrows may represent examples of mode values allocated to the intra prediction mode or to the prediction direction of the intra prediction mode.

Intra-coding and/or decoding may be performed using reference samples of cells neighboring the target cell. The neighboring cells may be neighboring reconstruction cells. For example, intra-coding and/or decoding may be performed using values of reference samples included in each neighboring reconstruction unit or encoding parameters of the neighboring reconstruction unit.

The encoding apparatus 100 and/or the decoding apparatus 200 may generate the prediction block by performing intra prediction on the target unit based on the information regarding the samples in the current picture. When the intra prediction is performed, the encoding apparatus 100 and/or the decoding apparatus 200 may generate a prediction block for a target unit by performing the intra prediction based on information on samples in a current picture. When the intra prediction is performed, the encoding apparatus 100 and/or the decoding apparatus 200 may perform directional prediction and/or non-directional prediction based on the at least one reconstructed reference sample.

The prediction block may represent a block generated as a result of performing intra prediction. The prediction block may correspond to at least one of a CU, a PU, and a TU.

The units of the prediction block may have a size corresponding to at least one of the CU, the PU, and the TU. The prediction block may have a square shape with a size of 2N × 2N or N × N. The size N × N may include sizes 4 × 4, 8 × 8, 16 × 16, 32 × 32, 64 × 64, and so on.

Alternatively, the prediction block may be a square block having a size of 2 × 2, 4 × 4, 16 × 16, 32 × 32, 64 × 64, or the like, or a rectangular block having a size of 2 × 8, 4 × 8, 2 × 16, 4 × 16, 8 × 16, or the like.

The intra prediction may be performed according to an intra prediction mode for the target unit. The number of intra prediction modes that the target unit may have may be a predefined fixed value, and may be a value differently determined according to the properties of the prediction block. For example, the properties of the prediction block may include the size of the prediction block, the type of prediction block, and the like.

For example, the number of intra prediction modes may be fixed to 35 regardless of the size of the prediction unit. Alternatively, the number of intra prediction modes may be, for example, 3, 5, 9, 17, 34, 35, or 36.

As shown in fig. 6, the intra prediction modes may include two non-directional modes and 33 directional modes. The two non-directional modes may include a DC mode and a planar mode.

For example, in the vertical mode with the mode value of 26, prediction may be performed in the vertical direction based on the pixel values of the reference sampling points. For example, in the horizontal mode with the mode value of 10, prediction may be performed in the horizontal direction based on the pixel values of the reference sampling points. For example, in the vertical mode with the mode value of 26, prediction may be performed in the vertical direction based on the pixel value of the reference sample point.

Even in a directional mode other than the above-described modes, the encoding apparatus 100 and the decoding apparatus 200 may perform intra prediction on a target unit using reference samples according to an angle corresponding to the directional mode.

The intra prediction mode located on the right side with respect to the vertical mode may be referred to as a "vertical-right mode". The intra prediction mode located below the horizontal mode may be referred to as a "horizontal-below mode". For example, in fig. 6, the intra prediction mode having one of the mode values 27, 28, 29, 30, 31, 32, 33, and 34 may be a vertical-right mode 613. The intra prediction mode having a mode value of one of 2, 3, 4, 5, 6, 7, 8, and 9 may be a horizontal-lower mode 616.

The non-directional mode may include a DC mode and a planar mode. For example, the mode value of the DC mode may be 1. The mode value of the planar mode may be 0.

The directional pattern may include an angular pattern. Among the plurality of intra prediction modes, the modes other than the DC mode and the planar mode may be directional modes.

In the DC mode, a prediction block may be generated based on an average of pixel values of a plurality of reference samples. For example, the pixel values of the prediction block may be determined based on an average of the pixel values of the plurality of reference samples.

The number of intra prediction modes and the mode values of the respective intra prediction modes described above are merely exemplary. The number of intra prediction modes described above and the mode values of the respective intra prediction modes may be defined differently according to embodiments, implementations, and/or requirements.

The number of intra prediction modes may be different according to the type of color component. For example, the number of prediction modes may differ depending on whether the color component is a luminance (luma) signal or a chrominance (chroma) signal.

Fig. 7 is a diagram for explaining positions of reference samples used in an intra prediction process.

Fig. 7 illustrates positions of reference samples used for intra prediction of a target unit. Referring to fig. 7, the reconstructed reference samples for intra prediction of the current block may include, for example, a lower-left reference sample 731, a left-side reference sample 733, an upper-left reference sample 735, an upper reference sample 737, and an upper-right reference sample 739.

For example, the left reference sampling point 733 may represent a reconstructed reference sampling point adjacent to the left side of the target unit. The upper reference sample 737 may represent a reconstructed reference sample adjacent to the top of the target unit. The top left reference sample 735 may represent a reconstructed reference sample located at the top left corner of the target unit. The lower-left reference sampling point 731 may represent a reference sampling point located below the left side sampling point line among sampling points located on the same line as the left side sampling point line composed of the left side reference sampling points 733. The upper right reference spot 739 may represent a reference spot located on the right side of the upper spot line among spots located on the same line as the upper spot line composed of the upper reference spot 737.

When the size of the target unit is N × N, the numbers of the lower-left reference sample 731, the left-side reference sample 733, the upper reference sample 737, and the upper-right reference sample 739 may all be N.

By performing intra prediction on the target unit, a prediction block may be generated. The process of generating the prediction block may include determining values of pixels in the prediction block. The sizes of the target unit and the prediction block may be the same.

The reference samples used for intra prediction of the target unit may be changed according to an intra prediction mode of the target unit. The direction of the intra prediction mode may represent a dependency relationship between the reference sample and pixels of the prediction block. For example, a value specifying a reference sample may be used as a value of one or more specified pixels in the prediction block. In this case, the specified reference samples and the one or more specified pixels in the prediction block may be samples and pixels located on a straight line along a direction of the intra prediction mode. In other words, the value of the specified reference sample point may be copied as the value of the pixel located in the direction opposite to the direction of the intra prediction mode. Alternatively, the value of a pixel in the prediction block may be a value of a reference sample point located in the direction of the intra prediction mode with respect to the position of the pixel.

For example, when the intra prediction mode of the target unit is the vertical mode having a mode value of 26, the upper reference samples 737 may be used for intra prediction. When the intra prediction mode is a vertical mode, the value of a pixel in the prediction block may be the value of a reference pixel located vertically above the position of the pixel. Therefore, the upper reference samples 737 adjacent to the top of the target unit may be used for intra prediction. Further, the values of the pixels in a row of the prediction block may be the same as the values of the pixels of the upper reference sample 737.

For example, when the intra prediction mode of the current block is a horizontal mode having a mode value of 10, the left reference sample 733 may be used for intra prediction. When the intra prediction mode is a horizontal mode, the value of a pixel in the prediction block may be the value of a reference pixel horizontally located to the left of the pixel. Accordingly, the left reference sample 733 adjacent to the left side of the target unit may be used for intra prediction. Also, the values of pixels in one column of the prediction block may be the same as those of the pixel of the left reference sample 733.

For example, when the mode value of the intra prediction mode of the current block is 18, at least some of the left reference samples 733, the upper-left reference samples 735, and at least some of the upper reference samples 737 may be used for intra prediction. When the mode value of the intra prediction mode is 18, the value of a pixel in the prediction block may be the value of a reference pixel located diagonally at the upper left corner of the pixel.

Further, when an intra prediction mode having a mode value corresponding to 27, 28, 29, 30, 31, 32, 33, or 34 is used, at least some of the upper right reference samples 739 may be used for intra prediction.

Further, when an intra prediction mode having a mode value corresponding to 2, 3, 4, 5, 6, 7, 8, or 9 is used, at least some of the lower-left reference samples 731 may be used for intra prediction.

Further, when an intra prediction mode having a mode value corresponding to any one of 11 to 25 is used, the upper left reference sample 735 may be used for intra prediction.

The number of reference samples used to determine the pixel value of one pixel in the prediction block may be 1 or 2 or more.

As described above, the pixel values of the pixels in the prediction block may be determined according to the positions of the pixels and the positions of the reference samples indicated by the direction of the intra prediction mode. When the position of the pixel and the position of the reference sample point indicated by the direction of the intra prediction mode are integer positions, the value of one reference sample point indicated by the integer position may be used to determine the pixel value of the pixel in the prediction block.

When the position of the pixel and the position of the reference sample indicated by the direction of the intra prediction mode are not integer positions, an interpolated reference sample based on two reference samples closest to the position of the reference sample may be generated. The values of the interpolated reference samples may be used to determine pixel values for pixels in the prediction block. In other words, when the position of the pixel in the prediction block and the position of the reference sampling point indicated by the direction of the intra prediction mode indicate a position between two reference sampling points, an interpolation value based on values of the two sampling points may be generated.

The prediction block generated through prediction may be different from the original target unit. In other words, there may be a prediction error, which is a difference between the target unit and the prediction unit, and there may also be a prediction error between the pixel of the target unit and the pixel of the prediction block. For example, in the case of directional intra prediction, the longer the distance between the pixels of the predicted block and the reference sample point, the larger the prediction error that may occur. Such a prediction error may cause discontinuity between the generated prediction block and the neighboring block.

To reduce the prediction error, a filtering operation for the prediction block may be used. The filtering operation may be configured to adaptively apply a filter to a region in the prediction block that is considered to have a large prediction error. For example, a region considered to have a large prediction error may be a boundary of a prediction block. In addition, a region that is considered to have a large prediction error in a prediction block may be different according to an intra prediction mode, and characteristics of a filter may also be different according to the intra prediction mode.

Fig. 8 is a diagram for explaining an embodiment of an intra prediction process.

The rectangle shown in fig. 8 may represent an image (or picture). Further, in fig. 8, an arrow may indicate a prediction direction. That is, each image may be encoded and/or decoded according to a prediction direction.

Images (or pictures) can be classified into intra pictures (I pictures), unidirectional predictive pictures or predictive coded pictures (P pictures), and bidirectional predictive pictures or bidirectional predictive coded pictures (B pictures) according to coding types. Each picture can be coded according to its coding type.

When the image that is the target to be encoded is an I picture, the image itself may be encoded without inter prediction. When an image that is a target to be encoded is a P picture, the image can be encoded via inter prediction using only a reference picture in the forward direction. When the image to be encoded is a B picture, the image may be encoded via inter prediction using reference pictures in both the forward and reverse directions, or may be encoded via inter prediction using a reference picture in one of the forward and reverse directions.

P-pictures and B-pictures encoded and/or decoded using reference pictures may be considered images using inter prediction.

Hereinafter, inter prediction in the inter mode according to the embodiment will be described in detail.

In the inter mode, the encoding apparatus 100 and the decoding apparatus 200 may perform prediction and/or motion compensation on the encoding target unit and the decoding target unit. For example, the encoding apparatus 100 or the decoding apparatus 200 may perform prediction and/or motion compensation by using motion information of neighboring reconstructed blocks as motion information of an encoding target unit or a decoding target unit. Here, the encoding target unit or the decoding target unit may represent a prediction unit and/or a prediction unit partition.

Inter prediction may be performed using a reference picture and motion information. Further, inter prediction may use the skip mode described above.

The reference picture may be at least one of pictures preceding or succeeding the current picture. Here, the inter prediction may perform prediction on a block in the current picture based on the reference picture. Here, the reference picture may represent an image used to predict a block.

Here, the area in the reference picture can be specified by using a reference picture index refIdx indicating the reference picture and a motion vector, which will be described later.

Inter prediction may select a reference picture and a reference block in the reference picture corresponding to the current block, and may generate a prediction block for the current block using the selected reference block. The current block may be a block that is a target to be currently encoded or decoded, among blocks in the current picture.

The motion information may be derived by each of the encoding apparatus 100 and the decoding apparatus 200 during inter prediction. Furthermore, the derived motion information may be used to perform inter prediction.

Here, the encoding apparatus 100 and the decoding apparatus 200 may improve encoding efficiency and/or decoding efficiency by using motion information of neighboring reconstructed blocks and/or motion information of co-located blocks (col blocks). The col block may be a block corresponding to the current block in a co-located picture (col picture) that has been previously reconstructed.

The neighboring reconstructed block may be a block that is present in the current picture and may be a block that has been previously reconstructed via encoding and/or decoding. The reconstructed block may be a neighboring block adjacent to the current block and/or a block located at an outer corner of the current block. Here, the "block located at an outer corner of the current block" may mean a block vertically adjacent to a neighboring block horizontally adjacent to the current block, or a block horizontally adjacent to a neighboring block vertically adjacent to the current block.

For example, the neighboring reconstruction unit (block) may be a unit located at the left side of the target unit, a unit located above the target unit, a unit located at the lower left corner of the target unit, a unit located at the upper right corner of the target unit, or a unit located at the upper left corner of the target unit.

Each of the encoding apparatus 100 and the decoding apparatus 200 may determine a block existing in a col picture at a position spatially corresponding to the current block, and may determine a predefined relative position based on the determined block. The predefined relative position may be a position inside and/or outside the block that is present at a position spatially corresponding to the current block. Further, each of the encoding apparatus 100 and the decoding apparatus 200 may derive the col block based on the predefined relative position that has been determined. Here, a col picture can be any one of one or more reference pictures included in a reference picture list.

The block in the reference picture may be present in the reconstructed reference picture at a position spatially corresponding to the position of the current block. In other words, the position of the current block in the current picture and the position of the block in the reference picture may correspond to each other. Hereinafter, motion information of a block included in a reference picture may be referred to as "temporal motion information".

The method for deriving motion information may vary according to the prediction mode of the current block. For example, as a prediction mode applied to inter prediction, there may be an Advanced Motion Vector Predictor (AMVP) mode, a merge mode, and the like.

For example, when the AMVP mode is used as the prediction mode, each of the encoding apparatus 100 and the decoding apparatus 200 may generate the prediction motion vector candidate list using motion vectors of neighboring reconstructed blocks and/or motion vectors of col blocks. Motion vectors of neighboring reconstructed blocks and/or motion vectors of col blocks may be used as prediction motion vector candidates.

The bitstream generated by the encoding apparatus 100 may include a prediction motion vector index. The predicted motion vector index may represent a best predicted motion vector selected from predicted motion vector candidates included in the predicted motion vector candidate list. The predictive motion vector index may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream.

The decoding apparatus 200 may select a predicted motion vector of the current block from among predicted motion vector candidates included in the predicted motion vector candidate list using the predicted motion vector index.

The encoding apparatus 100 may calculate a Motion Vector Difference (MVD) between the motion vector of the current block and the prediction motion vector, and may encode the MVD. The bitstream may include coded MVDs. The MVD may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. Here, the decoding apparatus 200 may decode the received MVD. The decoding apparatus 200 may derive a motion vector of the current block using the sum of the decoded MVD and the prediction motion vector.

The bitstream may include a reference picture index or the like for indicating a reference picture. The reference picture index may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. The decoding apparatus 200 may predict a motion vector of the current block using motion information of neighboring blocks, and may derive the motion vector of the current block using a difference (MVD) between the predicted motion vector and the motion vector. The decoding apparatus 200 may generate a prediction block for the current block based on the derived motion vector and the reference picture index information.

Since motion information of neighboring reconstructed blocks can be used to encode and decode a target unit, the encoding apparatus 100 may not separately encode the motion information of the target unit in a specific inter prediction mode. If the motion information of the target unit is not encoded, the number of bits transmitted to the decoding apparatus 200 can be reduced and encoding efficiency can be improved. For example, there may be a skip mode and/or a merge mode that is an inter prediction mode that does not encode motion information of the target unit. Here, each of the encoding apparatus 100 and the decoding apparatus 200 may use an identifier and/or an index indicating one neighboring reconstructed block of a plurality of neighboring reconstructed blocks, the motion information of which is to be used as the motion information of the target unit.

There is a merging method as another example of a method of deriving motion information. The term "merging" may denote merging of motions of a plurality of blocks. The term "merge" may mean that motion information of one block is also applied to other blocks. When the merging is applied, each of the encoding apparatus 100 and the decoding apparatus 200 may generate a merge candidate list using motion information of neighboring reconstructed blocks and/or motion information of a col block. The motion information may include at least one of: 1) motion vectors, 2) indices of reference pictures and 3) prediction directions. The prediction direction may be unidirectional or bidirectional.

Here, merging may be applied on a CU basis or a PU basis. When the merging is performed on a CU basis or a PU basis, the encoding apparatus 100 may transmit predefined information to the decoding apparatus 200 through a bitstream. The bitstream may include predefined information. The predefined information may include: 1) Information on whether to perform merging for the respective block partitions, and 2) information on a neighboring block to be used to perform merging among a plurality of neighboring blocks neighboring the current block. For example, the neighboring blocks of the current block may include a left neighboring block of the current block, an upper neighboring block of the current block, a temporally neighboring block of the current block, and the like.

The merge candidate list may represent a list in which a plurality of pieces of motion information are stored. Further, the merge candidate list may be generated before performing the merge. The motion information stored in the merge candidate list may be 1) motion information of a neighboring block adjacent to the current block and 2) motion information of a co-located block corresponding to the current block in a reference picture. Further, the motion information stored in the merge candidate list may be new motion information generated by combining pieces of motion information previously existing in the merge candidate list.

The skip mode may be a mode in which information on a neighboring block is applied to the current block without change. The skip mode may be one of a plurality of modes for inter prediction. When the skip mode is used, the encoding apparatus 100 may transmit only information on a block, the motion information of which is to be used as the motion information of the current block, to the decoding apparatus 200 through a bitstream. The encoding apparatus 100 may not transmit other information to the decoding apparatus 200. For example, the other information may be syntax information. The syntax information may include Motion Vector Difference (MVD) information.

FIG. 9 illustrates a second prediction of a partition of a current block according to an embodiment.

The current block shown in fig. 9 may be a PU.

When optimal intra prediction is performed on a current block, prediction is performed on the entire current block in only one direction, and thus a large prediction error may occur on a portion of the current block.

The scheme for partitioning the current block in a non-square shape may be applied to a portion where a large prediction error occurs.

In other words, if the second prediction in another prediction mode is performed on one part having a large prediction error after the first prediction has been performed on the square prediction unit, the prediction error in the current block may be reduced.

For example, both the first prediction mode and the second prediction mode may be intra prediction modes.

In fig. 9, the current block may have a square shape. The prediction direction of the first prediction performed on the entire current block is shown. For example, the prediction mode of the first prediction may be a vertical mode having a mode value of 26.

The current block may be partitioned into a first partition and a second partition. Each of the first and second portions may have a non-square shape.

In fig. 9, the prediction direction of the second prediction for the first part is shown, and the prediction direction of the second prediction for the second part is shown. For example, the prediction mode for the second prediction of the first part may be a horizontal mode having a mode value of 10. The prediction mode for the second prediction of the second part may be the lowest diagonal mode with a mode value of 34.

The shape of the portion having a large prediction error may correspond to the direction of the best prediction mode for the current block. In the following embodiments, a method and apparatus for improving prediction performance and reducing a prediction error by effectively utilizing a tendency of a shape of a portion having a large prediction error to correspond to a direction of an optimal prediction mode for a current block may be described.

Fig. 10 is a configuration diagram of an encoding device according to an embodiment.

The encoding device 1000 may be a general-purpose computer system that performs encoding.

As shown in fig. 10, the encoding apparatus 1000 may include at least one processor 1010, a memory 1030, a User Interface (UI) input device 1050, a UI output device 1060, and a storage 1040, which communicate with each other through a bus 1090. The encoding device 1000 may also include a communication unit 1020 connected to the network 1099. Processor 1010 may be a Central Processing Unit (CPU) or semiconductor device for executing processing instructions stored in memory 1030 or storage 1040. Each of the memory 1010 and the storage 1040 may be any of various types of volatile or non-volatile storage media. For example, the memory may include at least one of a Read Only Memory (ROM) 1031 and a Random Access Memory (RAM) 1032.

The processor 1010 may include the inter prediction unit 110, the intra prediction unit 120, the switch 115, the subtractor 125, the transform unit 130, the quantization unit 140, the entropy encoding unit 150, the inverse quantization unit 160, the inverse transform unit 170, the adder 175, the filtering unit 180, and the reference picture buffer 190 of the encoding apparatus 100.

The storage medium may store at least one module for the operation of the encoding apparatus 1000. Memory 1030 may be configured to store at least one module and allow the module to be executed by at least one processor 1010.

According to an embodiment, at least some of the inter prediction unit 110, the intra prediction unit 120, the switch 115, the subtractor 125, the transform unit 130, the quantization unit 140, the entropy encoding unit 150, the inverse quantization unit 160, the inverse transform unit 170, the adder 175, the filtering unit 180, and the reference picture buffer 190 of the encoding apparatus 1000 may be program modules and may communicate with an external device or system. The program modules may be included in the encoding device 1000 in the form of an operating system, application program modules, and other program modules.

The program modules may be physically stored in various types of well-known storage devices. Furthermore, at least some of the program modules may also be stored in a remote memory storage device that is capable of communicating with the encoding apparatus 1000.

Program modules may include, but are not limited to, routines, subroutines, programs, objects, components, and data structures for performing functions or operations in accordance with the embodiments or for implementing abstract data types in accordance with the embodiments.

The program modules may be implemented using instructions or code executed by processor 1010.

Functions related to the communication of data or information by the encoding device 1000 may be performed by the communication unit 1020.

Fig. 11 is a flow chart of an encoding method according to an embodiment.

The encoding method according to the embodiment may be performed by the encoding apparatus 100 or the encoding apparatus 1000.

In step 1110, the prediction part may generate a residual signal of the current block based on the current block, the first prediction, and the second prediction.

The prediction part may include an inter prediction unit 110, an intra prediction unit 120, and a subtractor 125.

For example, the residual signal may represent a residual block.

For example, the residual block may be a difference between the current block and the prediction block. Alternatively, a residual block may be generated based on a difference between the current block and the prediction block.

For example, the prediction block may be a sum of the first prediction block and the second prediction block. The residual block may be obtained by subtracting a sum of the first prediction block and the second prediction block from the current block.

The first prediction block may be a block generated via first prediction. The second prediction block may be a block generated via second prediction. When the second prediction is not used, the second prediction block may be a null block. The empty block may be a block in which all pixels have a value of 0.

The prediction part may generate a residual block for the current block by performing inter prediction or intra prediction.

In an embodiment, the prediction part may generate the second prediction block based on the current block and/or the first prediction block.

In an embodiment, the prediction part may generate a residual signal of the current block by performing a first prediction on the current block and performing a second prediction on a first prediction error that is a result of the first prediction. Here, the first prediction error may be a difference between the current block and the first prediction signal. The residual signal may be a difference between the first prediction error and the second prediction signal.

In an embodiment, the first prediction and the second prediction may be different types of predictions.

In an embodiment, the first prediction and the second prediction may be the same type of prediction. For example, the first prediction and the second prediction may both be intra-prediction. When the first prediction is intra prediction, the second prediction may also be set to intra prediction.

In an embodiment, the prediction direction of the first prediction and the prediction direction of the second prediction may be the same as each other. The prediction direction of the second prediction may be set to the same prediction direction as the first prediction.

For example, in an embodiment, the first prediction and the second prediction may both be inter-prediction.

In an embodiment, the block that is the target of the first prediction and the block that is the target of the second prediction may be different from each other. Here, the content indicated by the expression "a block as a target of prediction" may be different. Here, the expression "a block that is a target of prediction" may denote a block for which a prediction block is to be generated via "prediction".

For example, the type of the block that is a target of the first prediction and the type of the block that is a target of the second prediction may be different from each other. The types of blocks may include original blocks, luminance (luma) blocks, chrominance (chroma) blocks, depth blocks, residual blocks, and the like.

For example, the block that is the target of the first prediction may be the current block or the original block.

For example, a first prediction error generated via a first prediction may represent a first block of prediction errors. The first prediction error may be a first residual signal. Further, the first residual signal may represent the first residual block. The block that is the target of the second prediction may be the first prediction error block or the first residual block. In this case, the first residual signal may be generated via a first prediction, and the second prediction error may be generated via a second prediction. The second prediction error may be a second residual signal. The second residual signal may represent a second residual block.

The reference block may be used for prediction. In an embodiment, the first reference block used for the first prediction and the second reference block used for the second prediction may be different from each other. The reference blocks may differ in the type and/or location of the reference block.

In an embodiment, a position of a first reference block for the first prediction and a position of a second reference block for the second prediction may be different from each other. Here, the position of the first reference block may be a position relative to a position of a block that is a target of the first prediction. The position of the second reference block may be a position relative to a position of a block that is a target of the second prediction.

In an embodiment, the first reference block for the first prediction may comprise a plurality of reference blocks. The second reference block used for the second prediction may include a plurality of reference blocks. At least some of the plurality of first reference blocks used for the first prediction and the plurality of second reference blocks used for the second prediction may be different from each other.

The positions of at least some of the plurality of first reference blocks used for the first prediction and the plurality of second reference blocks used for the second prediction may be included in only one of the positions of the plurality of first reference blocks and the positions of the plurality of second reference blocks.

In an embodiment, a type of a first reference block used for the first prediction and a type of a second reference block used for the second prediction may be different from each other. The types of blocks may include a reconstructed block, a reconstructed luma block, a reconstructed chroma block, a reconstructed depth block, a reconstructed first residual block, and a reconstructed second residual block.

For example, the first reference block used for the first prediction may be a reconstructed block. The second reference block used for the second prediction may be the reconstructed first residual block. The first reference sample used for the first prediction may be a pixel in the reconstructed block. The second reference samples used for the second prediction may be pixels in the reconstructed first residual block.

For example, the first reference block used for the first prediction may be a neighboring reconstructed block adjacent to the current block. The second reference block used for the second prediction may be a reconstructed first residual block adjacent to the current block.

The neighboring reconstructed residual block adjacent to the current block may be obtained by adding a second prediction block of the neighboring reconstructed block to the reconstructed residual block for the neighboring reconstructed block. Alternatively, the neighboring reconstructed residual block adjacent to the current block may be a difference between the neighboring reconstructed block and the first prediction block of the neighboring reconstructed block.

For example, when the target of the first prediction is the current block, neighboring reconstructed blocks of the current block may be used as reference blocks in order to perform prediction of the current block. When the target of the second prediction is the first residual block, a neighboring reconstructed residual block of the current block or the first residual block may be used as a second reference block in order to perform the prediction of the first residual block.

In an embodiment, the first predicted region and the second predicted region may be different from each other. Here, the "predicted area" may mean an area that generates a predicted value in a block that is a target of prediction. Alternatively, the "predicted region" may represent a region to which a prediction value generated through prediction is allocated among prediction blocks corresponding to a block that is a target of prediction.

In an embodiment, the second prediction may be a prediction for a portion of the current block. The first prediction may be a prediction of the entire current block. Alternatively, the first prediction may be a prediction for the remaining portion of the current block except for the portion predicted via the second prediction.

For example, the second predicted region may be a part of the current block. The first predicted region may be the entire current block. Alternatively, the first predicted region may be a remaining portion of the current block except for the second predicted portion.

For example, the second predicted region may be a non-square portion of the current block. The first predicted region may be the entire current block. Alternatively, the first predicted region may be a remaining non-square region in the current block except for the second predicted non-square region.

For example, a prediction value determined via the first prediction may be allocated only to pixels that fall within a region of the first prediction in a block that is a target of the first prediction. A prediction value determined via the first prediction may not be allocated to pixels that fall outside the area of the first prediction in the block that is the target of the first prediction. Alternatively, a predefined value may be assigned to a pixel that falls outside the area of the first prediction in the block that is the target of the first prediction. The predefined value may be 0.

For example, a prediction value determined via the second prediction may be allocated only to pixels that fall within a region of the second prediction in a block that is a target of the second prediction. A prediction value determined via the second prediction may not be allocated to pixels that fall outside the area of the second prediction in the block that is the target of the second prediction. Alternatively, a predefined value may be assigned to a pixel that falls outside the area of the second prediction in the block that is the target of the second prediction. The predefined value may be, for example, 0.

In an embodiment, the region of the first prediction may be determined based on a type of the first prediction. For example, the region of the first prediction may be determined based on whether the first prediction is inter-prediction. Alternatively, the first predicted region may be determined based on whether the first prediction is intra-prediction. Alternatively, the first predicted region may be determined based on a prediction direction of the first prediction.

In an embodiment, the area of the second prediction may be determined based on the type of the second prediction. For example, the region of the second prediction may be determined based on whether the second prediction is inter prediction. Alternatively, the second predicted region may be determined based on whether the second prediction is intra-prediction. Alternatively, the area of the second prediction may be determined based on the prediction direction of the second prediction.

In step 1120, the encoding part may perform encoding on the residual signal. The encoding part may generate information on the encoded residual signal by performing encoding on the residual signal.

The encoding part may include a transform unit 130, a quantization unit 140, and an entropy encoding unit 150.

Step 1120 may include steps 1121, 1122, and 1123.

In step 1121, the transform unit 130 may generate transform coefficients for the residual signal.

In step 1122, the quantization unit 1140 may generate quantized transform coefficients by performing quantization on the transform coefficients.

In step 1123, the entropy encoding unit 150 may generate information on the encoded residual signal by performing entropy encoding on the quantized transform coefficients.

In step 1130, the entropy encoding unit 150 may generate a bitstream of information regarding the encoded residual signal.

The bitstream may include information on the encoded residual signal and may further include prediction-related information.

The entropy encoding unit 150 may include prediction-related information in the bitstream. The prediction related information may be entropy encoded.

For example, the prediction-related information may include prediction scheme information indicating a scheme used to encode the current block.

For example, the prediction scheme information may indicate which one of intra prediction and inter prediction is used to encode the current block. Alternatively, the prediction scheme information may indicate whether the current block has been encoded via intra prediction. Alternatively, the prediction scheme information may indicate whether the current block has been encoded via inter prediction.

For example, the prediction-related information may include intra prediction mode information indicating a prediction mode of intra prediction.

For example, the prediction related information may include second prediction usage information, wherein the second prediction usage information indicates whether the second prediction is to be used to encode the current block.

For example, the prediction-related information may include first prediction mode information indicating a prediction mode of the first prediction and second prediction mode information indicating a prediction mode of the second prediction.

For example, the prediction-related information may include selected partition information indicating a selected partition to which the second prediction is to be applied, wherein the selected partition is one of two partitions generated by partitioning the current block.

For example, the prediction-related information may include current block type information indicating a type of the current block, first prediction block type information indicating a type of the first prediction block, second prediction block type information indicating a type of the second prediction block, first reference block type information indicating a type of the first reference block, second reference block type information indicating a type of the second reference block, first reference sample type information indicating a type of the first reference sample, and second reference sample type information indicating a type of the second reference sample.

For example, the prediction-related information may include first prediction region information indicating a region of the first prediction and/or second prediction region information indicating a region of the second prediction.

The prediction related information may include information for prediction that has been described in the embodiments. The entropy encoding unit 150 may include pieces of prediction-related information in a bitstream according to an order described in the embodiments or an order generated in the embodiments.

In step 1140, the communication unit 1020 may transmit the bitstream to the decoding apparatus 200 or the decoding apparatus 1500.

In step 1150, the reconstructed residual signal generation unit may generate a reconstructed residual signal based on the information on the residual signal.

The reconstructed residual signal generation unit may include an inverse quantization unit 160 and an inverse transformation unit 170.

Step 1150 may include step 1151 and step 1152.

In step 1151, the inverse quantization unit 160 may generate a reconstructed transform coefficient by performing inverse quantization on the quantized transform coefficient.

In step 1152, the transform unit 130 may generate a reconstructed residual signal by performing inverse transform on the reconstructed transform coefficients.

In step 1160, the reconstructed block generation unit may generate a reconstructed block based on the reconstructed residual signal, the first prediction, and the second prediction.

The reconstruction block generation unit may include an adder 175.

In an example, a sum of the reconstructed residual signal and the prediction signal may represent a reconstructed block. Alternatively, the reconstructed block may be generated based on the sum of the reconstructed residual signal and the prediction signal.

In an example, the reconstructed block may be a sum of the reconstructed residual block and the prediction block. The prediction block may be a sum of the first prediction block and the second prediction block. The second prediction signal may be a signal generated by performing a second prediction on a portion of the current block. The first prediction signal may be a signal generated by performing a first prediction on the current block or a signal generated by performing a first prediction on the remaining portion of the current block except for the second predicted portion.

Alternatively, the reconstructed block may be generated based on a sum of the reconstructed residual block and the prediction block.

For example, the reconstructed block may be a block generated based on the reconstructed residual signal, the first prediction block, and the second prediction block. The reconstructed block generation unit may generate a reconstructed first residual signal by adding the reconstructed residual signal to the second prediction signal. The reconstructed block generation unit may generate a reconstructed block by adding the reconstructed first residual signal to the first prediction signal.

Alternatively, the reconstructed block generation unit may generate the reconstructed first residual block by adding the reconstructed residual block to the second prediction block. The reconstructed block generation unit may generate a reconstructed block by adding the reconstructed first residual block to the first prediction block.

The reconstructed block and the reconstructed first residual signal may be used as a reference block for encoding other blocks.

FIG. 12 illustrates a partition operation on a current block according to an embodiment.

In fig. 12, a current block of size N × N is shown.

The current block may be partitioned into two parts. The two portions may be determined based on a straight line passing through a center of the current block along a prediction direction of the first prediction. In fig. 12, the prediction direction of the first prediction may be a prediction direction of a prediction mode having a mode value of 3.

The first prediction may be a prediction using a first best prediction mode obtained by intra-predicting the current block.

The current block may be partitioned into two partitions (partition regions) using the prediction mode of the first prediction such that the two partitions have the same number of pixels.

Here, the reason why the current block is partitioned in the prediction direction of the first prediction is: when prediction of a current block is performed using intra prediction, there may be a tendency for a large prediction error to occur in a section specified by a prediction direction.

In fig. 12, the two portions are indicated by S1 and S2, respectively. Among the points whose x-coordinate value is i, a point whose y-coordinate value is j2 may be included in S1. Among the points whose x-coordinate value is i, a point whose y-coordinate value is j1 may be included in S2.

The prediction direction of the prediction mode may be calculated using equations 2, 3, 4, 5, and 6 below and table 1 below.

Since the prediction mode having a mode value of 0 or 1 is a non-directional prediction mode, the prediction mode may be excluded from the partition.

[ equation 2]

"predMode" may represent a number for prediction mode. "vertical" may indicate that the prediction mode is a vertical mode. The "horizontal" may indicate that the prediction mode is a horizontal mode.

[ equation 3]

absAngle＝|predAngle|

[ equation 4]

[ equation 5]

[ equation 6]

Tan＝TanSign×TanValue

The value of absAngle can be defined by table 1 below.

TABLE 1

absAngle	0	1	2	3	4	5	6	7	8
										Value of	0	2	5	9	13	17	21	26	32

When the value of Tan of the prediction mode is calculated, a pixel located at the coordinate (i, j) in the current block may be included in at least one of the section S0 and the section S1 by equation 7 below:

[ equation 7]

When the mode value of the prediction mode predMode is 3, 18, or 34, the value of Tan may be-1 or +1. In this case, pixels on a straight line partitioning the current block may be included in each of the two partitions S0 and S1.

In other words, when the current block is partitioned into two partitions along a straight line passing through the center of the current block, pixels located on the straight line in the current block may be included in each of the two partitions.

Fig. 13 is a flowchart of a residual block generating method according to an embodiment.

Step 1110 described above with reference to fig. 11 may include steps 1310, 1320, 1330, 1335, 1340, 1345, 1350, 1355, 1360, 1370, and 1380.

In an embodiment, the block that is the target of the first prediction may be a current block. The first prediction may be intra prediction.

In step 1310, the prediction part may generate a first prediction signal by performing a first prediction on the current block.

The first prediction signal may represent a first prediction block.

The prediction part may perform the first prediction in each of a plurality of prediction modes in order to find an optimal prediction mode for the first prediction to encode the current block. That is, step 1310 may be performed in each of a plurality of prediction modes.

The multiple prediction modes may correspond to different prediction directions.

The prediction part may generate a first prediction error based on the current block and the first prediction signal.

The first prediction error may represent a first residual block.

The first residual block may represent a difference (residual) between the current block and the first prediction block. That is, the prediction part may generate the first prediction error by subtracting the first prediction block represented by the first prediction signal from the current block.

The prediction section may calculate a rate-distortion cost for the first prediction using the first prediction error.

The prediction section may store the calculated first prediction error and/or the calculated rate-distortion cost for the first prediction in the memory 1030, the storage 1040, or a buffer.

The prediction section may perform the first prediction in each of a plurality of prediction modes in order to find an optimal prediction mode of the first prediction to encode the current block. The prediction unit may calculate a plurality of rate-distortion costs for the plurality of prediction modes using the first prediction errors in the plurality of prediction modes. The prediction part may store a minimum rate-distortion cost among the calculated plurality of rate-distortion costs as a minimum first rate-distortion cost RDcost _1. For example, RDcost _1 may be the rate-distortion cost of the mode having the smallest rate-distortion cost in the main intra prediction.

The prediction unit may store RDcost _1 and a first prediction error corresponding to RDcost _1.

In step 1320, the prediction part may determine a first prediction mode of the first prediction. The first prediction mode may indicate a prediction direction of the first prediction.

The prediction section may determine a prediction mode having a smallest rate-distortion cost among the plurality of prediction modes as the first prediction mode. Hereinafter, the prediction mode of the first prediction may be a prediction mode having a smallest rate-distortion cost among a plurality of prediction modes. The prediction direction of the first prediction may be a prediction direction of a prediction mode of the first prediction.

At step 1330, it may be determined whether to perform a second prediction on the current block.

In an embodiment, the determination at step 1330 may not be a final decision as to whether or not to encode the current block using the second prediction. In step 1130, the prediction part may determine whether the second prediction can be performed on the current block.

For example, if it is determined that the second prediction is not performed, the current block may be encoded without using the second prediction. If it is determined that the second prediction is performed, a second prediction signal and a second prediction error may be generated via the second prediction, but it may be determined whether to encode the current block using the second prediction by comparing rate-distortion costs in step 1360.

The residual block for the current block may be a block represented by the first prediction error if it is determined that the second prediction is not performed. In other words, the first prediction error may be provided as a residual block.

If it is determined that the second prediction is performed, the residual block for the current block may be a block represented by a second prediction error, which will be described later. In other words, the second prediction error may be provided as a residual block.

The prediction part may determine whether to perform the second prediction based on a predefined condition.

In an embodiment, the prediction section may determine whether to perform the second prediction based on a prediction mode of the first prediction.

For example, when the prediction mode of the first prediction is a non-directional mode, the prediction part may not perform the second prediction. Alternatively, the prediction section may perform the second prediction when the prediction mode of the first prediction is the directional mode.

When the prediction mode of the first prediction is a non-directional mode among the plurality of prediction modes, the current block may not be partitioned in the prediction direction since the direction of the prediction mode does not exist.

If it is determined that the second prediction is to be performed, step 1335 may be performed.

If it is determined that the second prediction is not to be performed, step 1370 or step 1120 may be performed. When step 1120 is performed, the residual signal at step 1120 may be a first prediction error. That is, when only the first prediction is performed without performing the second prediction, a first prediction error generated through the first prediction may be used as a residual signal for the current block.

In step 1335, the prediction part may partition the current block based on the first prediction mode. The prediction part may generate two partitions by partitioning the current block based on the first prediction mode.

In an embodiment, the prediction part may generate the two non-square portions by partitioning the current block along a straight line passing through a center of the current block in a prediction direction of the first prediction. Hereinafter, the two non-square portions may be designated as a "first portion" and a "second portion".

When the current block is partitioned, the second prediction may be performed on each partition resulting from the partition operation.

In step 1340, the predictor may generate a second prediction signal for the first portion by performing a second prediction on the first portion.

The second prediction signal for the first portion may represent a second prediction block for the first portion.

The second prediction for the first portion may be intra prediction.

The prediction section may perform the second prediction in each of the plurality of prediction modes in order to find an optimal prediction mode to encode the first portion. In other words, step 1340 may be performed in each of a plurality of prediction modes.

The multiple prediction modes may correspond to different prediction directions.

The prediction part may generate a second prediction error with respect to the first part based on the first part and the second prediction signal.

The second prediction error for the first portion may represent a second residual block for the first portion.

The second residual block for the first portion may represent a difference (residual) between the first portion and the second prediction block. That is, the prediction part may generate the second prediction error for the first portion by subtracting a second prediction block for the first portion from the first portion, wherein the second prediction block for the first portion is represented by the second prediction signal for the first portion.

The prediction section may calculate a rate-distortion cost of the second prediction for the first part using the second prediction error for the first part.

The prediction section may store the calculated second prediction error for the first portion and/or the calculated rate-distortion cost for the second prediction for the first portion in the memory 1030, the storage 1040, or a buffer.

The prediction part may perform the second prediction in each of the plurality of prediction modes in order to find an optimal prediction mode for the second prediction of the first part to encode the first part. The prediction section may calculate a plurality of rate-distortion costs for the plurality of prediction modes using a plurality of second prediction errors for the plurality of prediction modes. The prediction part may store a minimum rate-distortion cost among the calculated plurality of rate-distortion costs as a minimum second rate-distortion cost RDcost _2 of the first part. For example, RDcost _2 may be the rate-distortion cost of the mode having the smallest rate-distortion cost in the secondary intra prediction of the first portion.

The prediction part may store RDcost _2 of the first part and a second prediction error of the first part corresponding to RDcost _2 of the first part.

At step 1345, the prediction component may determine a second prediction mode for a second prediction of the first portion. The second prediction mode for the first portion may indicate a prediction direction for a second prediction for the first portion.

The prediction part may determine a prediction mode having a smallest rate-distortion cost among the plurality of prediction modes as the second prediction mode of the first part. Hereinafter, the prediction mode for the second prediction of the first part may be a prediction mode having a smallest rate-distortion cost among a plurality of prediction modes. The prediction direction of the second prediction for the first part may be a prediction direction of a prediction mode of the second prediction for the first part.

In step 1350, the prediction unit may generate a second prediction signal for the second portion by performing a second prediction on the second portion.

The second prediction signal for the second portion may represent a second prediction block for the second portion.

The second prediction for the second portion may be intra prediction.

The prediction part may perform the second prediction in each of the multiple prediction modes in order to find an optimal prediction mode to encode the second part. In other words, step 1340 may be performed in each of a plurality of prediction modes.

The multiple prediction modes may correspond to different prediction directions.

The prediction section may generate a second prediction error for the second portion based on the second portion and the second prediction signal.

The second prediction error for the second portion may represent a second residual block for the second portion.

The second residual block for the second portion may represent a difference (residual) between the second portion and the second prediction block. That is, the prediction part may generate a second prediction error for the second portion by subtracting a second prediction block for the second portion from the second portion, wherein the second prediction block for the second portion is represented by a second prediction signal for the second portion.

The prediction section may calculate a rate-distortion cost for the second prediction for the second part using the second prediction error for the second part.

The prediction section may store the calculated second prediction error for the second part and/or the calculated rate-distortion cost for the second prediction for the second part in the memory 1030, the storage 1040, or a buffer.

The prediction section may perform the second prediction in each of the plurality of prediction modes in order to find an optimal prediction mode for the second prediction of the second part to encode the second part. The prediction section may calculate a plurality of rate-distortion costs for the plurality of prediction modes using a plurality of second prediction errors for the plurality of prediction modes. The prediction part may store a minimum rate-distortion cost among the calculated plurality of rate-distortion costs as a minimum second rate-distortion cost RDcost _2 of the second part. For example, RDcost _2 may be the rate-distortion cost of the mode having the smallest rate-distortion cost in the secondary intra prediction of the second part.

The prediction part may store RDcost _2 of the second part and a second prediction error of the second part corresponding to RDcost _2 of the second part.

At step 1355, the prediction part may determine a second prediction mode for a second prediction of the second part. The second prediction mode for the second portion may indicate a prediction direction for a second prediction for the second portion.

The prediction section may determine a prediction mode having a smallest rate-distortion cost among the plurality of prediction modes as the second prediction mode of the second section. Hereinafter, the prediction mode for the second prediction of the second part may be a prediction mode having a smallest rate-distortion cost among a plurality of prediction modes. The prediction direction of the second prediction for the second part may be a prediction direction of a prediction mode of the second prediction for the second part.

In step 1360, the predictor may determine whether the current block is encoded using the second prediction.

The prediction part may determine whether to encode the current block using the second prediction based on a predefined condition.

For example, when the rate-distortion cost is further reduced by using the second prediction, the prediction section may determine to use the second prediction. When the rate-distortion cost is not further reduced even if the second prediction is used, the prediction section may determine not to use the second prediction.

For example, the prediction section may determine to use the second prediction when the minimum rate-distortion cost calculated when the second prediction is used is smaller than the minimum rate-distortion cost calculated when the second prediction is not used.

In an embodiment, the prediction part may determine whether to use the second prediction based on a result of comparison between a rate-distortion cost of the prediction mode of the first prediction, a rate-distortion cost of the prediction mode of the second prediction for the first part, and a rate-distortion cost of the prediction mode of the second prediction for the second part.

In an embodiment, the predictor may select one of the portions to be encoded from between the first portion and the second portion. The prediction part may select one part to be encoded from between the first part and the second part based on a result of comparison between a rate-distortion cost of the second prediction mode for the first part and a rate-distortion cost of the second prediction mode for the second part.

For example, the prediction part may select one part having a higher rate-distortion cost from between the first part and the second part. The prediction part may select the first part when a rate-distortion cost of the second prediction mode for the first part is higher than a rate-distortion cost of the second prediction mode for the second part. The prediction part may select the second part when a rate-distortion cost of the second prediction mode for the first part is lower than a rate-distortion cost of the second prediction mode for the second part.

For example, when a rate-distortion cost of a second prediction mode for the first part is the same as a rate-distortion cost of a second prediction mode for the second part, the prediction part may select the first part if the second prediction mode of the first part is different from the first prediction mode, and may select the second part if the second prediction mode of the second part is different from the first prediction mode. In other words, the prediction section may select one section from between the first section and the second section using the rate-distortion cost as a primary criterion and the prediction mode as a secondary criterion.

Hereinafter, the portion selected by the prediction section among the first portion and the second portion is referred to as a "selected portion". Further, "second prediction" represents a second prediction for the selected portion of the second prediction for the first portion and the second prediction for the second portion. The portion of the first portion and the second portion that is not selected by the prediction section is referred to as a remaining portion.

In an embodiment, the predictor may determine whether to use the second prediction based on a second prediction mode of the selected portion and the first prediction mode of the current block.

For example, when the second prediction mode of the selected portion is the same as the first prediction mode of the current block, the result obtained when the second prediction is used may be the same as the result obtained when the second prediction is not used. Therefore, in this case, the effect obtained by using the second prediction may not exist. The predictor may determine not to use the second prediction for the selected partition if the second prediction mode of the selected partition is the same as the first prediction mode of the current block.

In an embodiment, the predictor may determine to use the second prediction for the selected partition if a rate-distortion cost of the prediction for the current block is reduced by the second prediction for the selected partition. The predictor may determine not to use the second prediction for the selected partition if the rate-distortion cost of the prediction for the current block is not reduced by the second prediction for the selected partition.

In an example, when the rate-distortion cost of the second prediction for the selected portion is RDcost _ SS, the rate-distortion cost of the first prediction for the remaining portion is RDcost _ SNS, and the rate-distortion cost of the first prediction for the current block is RDcost, the prediction part may determine to use the second prediction for the selected portion if a sum of RDcost _ SS and RDcost _ SNS is less than RDcost. Further, the prediction section may determine not to use the second prediction for the selected portion if the sum of RDcost _ SS and RDcost _ SNS is equal to or greater than RDcost.

In an example, it is determined not to use the second prediction for the selected portion if the rate-distortion cost of the second prediction for the selected portion is less than the rate-distortion cost of the first prediction for the selected portion. Further, it is determined not to use the second prediction for the selected portion if the rate-distortion cost of the second prediction for the selected portion is equal to or greater than the rate-distortion cost of the first prediction for the selected portion.

If it is determined that the current block is not encoded using the second prediction, step 1370 may be performed.

If it is determined that the current block is encoded using the second prediction, step 1380 may be performed.

At step 1370, the prediction section may perform a setting indicating that the second prediction will not be used.

The prediction section may set a value of the second prediction use information such that the second prediction use information indicates that the second prediction will not be used. For example, the second prediction usage information may be an extra secondary intra prediction (ASIP) flag.

For example, when the value of the second prediction usage information is 0, the second prediction usage information may indicate that the second prediction will not be used.

The prediction section may set first prediction mode information to a prediction mode of the first prediction.

When step 1370 is performed, the residual signal of step 1120 may be a residual signal of the current block. The residual signal of the current block may be a first prediction error. In other words, when the second prediction is not performed on the current block, a first prediction error generated based on the first prediction may be used as a residual signal of the current block.

At step 1380, the prediction component may perform a setting indicating that the second prediction is to be used.

In an embodiment, the prediction section may set a value of the second prediction use information such that the second prediction use information indicates that the second prediction is to be used.

In an embodiment, the second prediction usage information may indicate that the second prediction is to be used when a value of the second prediction usage information is, for example, 1.

The prediction section may set a value of the selected portion information such that the selected portion information indicates the selected portion to which the second prediction is applied. For example, the selected part information may be a selected part flag (SSF).

For example, the predictor may set a value of the selected partition information such that the selected partition information indicates one of two partitions generated by partitioning the current block. When the selected portion is the portion S1 described above with reference to fig. 11, the prediction section may set the value of the selected portion information to 0. When the selected portion is the portion S2 described above with reference to fig. 11, the prediction section may set the value of the selected portion information to 1.

The prediction section may set first prediction mode information to a prediction mode of the first prediction. The prediction section may set the second prediction mode information to the prediction mode of the second prediction.

When step 1370 is performed, the residual signal at step 1120 may be a residual signal of the current block. The residual signal of the current block may be a signal corresponding to a difference between the current block and the prediction block. The prediction block may be a sum of a prediction block generated via the second prediction for the selected portion and a prediction block generated via the first prediction for the remaining portion.

In other words, when the second prediction is performed with respect to the current block, a prediction error generated based on the first prediction and the second prediction may be used as a residual signal.

Alternatively, when step 1370 is performed, the residual signal at step 1120 may be a residual signal for the selected portion and a residual signal for the remaining portion. The residual signal for the selected portion may be a signal corresponding to a difference between the selected portion and a second prediction block generated via second prediction for the selected portion. The residual signal for the remaining portion may be a signal corresponding to a difference between the remaining portion and a third prediction block generated via first prediction for the remaining portion.

Fig. 14 is a flowchart of a reconstruction block generation method according to an embodiment.

Step 1160, described above with reference to FIG. 11, may include the following steps 1410, 1420, and 1430.

In step 1410, the reconstructed block generation unit may determine whether the second prediction has been used to encode the current block.

If it is determined that the second prediction has been used to encode the current block, step 1420 may be performed.

If it is determined that the second prediction is not used to encode the current block, step 1430 may be performed.

In step 1420, the reconstructed block generation unit may add the second prediction signal for the selected portion to the reconstructed residual signal.

The second prediction signal may be a signal generated via a second prediction for the selected portion.

In step 1430, the reconstructed block generation unit may generate a reconstructed block by adding the first prediction signal for the current block or the remaining part to the reconstructed residual signal.

In an embodiment, the reconstructed block generation unit may generate the reconstructed block by adding the first prediction signal for the current block to the reconstructed residual signal when the second prediction is not used. The first prediction signal may be a signal generated via a first prediction with respect to the current block.

In an embodiment, the reconstructed block generation unit may generate the reconstructed block by adding the first prediction signal for the remaining portion to the reconstructed residual signal when the second prediction is not used. The first prediction signal may be a signal generated via a first prediction for the remaining portion.

For example, when the second prediction is used, the signal indicative of the reconstructed block may be the sum of the reconstructed residual signal, the second prediction signal for the selected portion, and the first prediction signal for the remaining portion. When the second prediction is used, the second prediction signal for the selected portion and the first prediction signal for the remaining portion may be added to the reconstructed residual signal, as described above with reference to steps 1420 and 1430. Alternatively, when the second prediction is used, a reconstructed block may be generated based on the reconstructed residual signal, the second prediction signal for the selected portion, and the first prediction signal for the remaining portion.

For example, when the second prediction is not used, the signal indicating the reconstructed block may be the sum of the reconstructed residual signal and the first prediction signal for the current block. Alternatively, when the second prediction is not used, a reconstructed block may be generated based on the reconstructed residual signal and the first prediction signal for the current block.

Fig. 15 is a configuration diagram of a decoding apparatus according to an embodiment.

The decoding apparatus 1500 may be a general-purpose computer system that performs decoding.

As shown in fig. 15, the decoding apparatus 1500 may include at least one processor 1510, a memory 1530, a User Interface (UI) input device 1550, a UI output device 1560, and a storage 1540 that communicate with each other through a bus 1590. The decoding apparatus 1500 may also include a communication unit 1520 connected to the network 1599. The processor 1510 may be a CPU or semiconductor device for executing processing instructions stored in the memory 1530 or in the storage 1540. Memory 1530 and storage 1540 can each be any of a variety of types of volatile or non-volatile storage media. For example, the memory may include at least one of ROM 1531 and RAM 1532.

The processor 1510 may include the entropy decoding unit 210, the inverse quantization unit 220, the inverse transform unit 230, the intra prediction unit 240, the inter prediction unit 250, the adder 255, the filtering unit 260, and the reference picture buffer 270 of the decoding apparatus 200.

The storage medium may store at least one module used in the operation of the decoding apparatus 1500. The memory 1530 may be configured to store at least one module and allow the module to be executed by the at least one processor 1510.

According to an embodiment, at least some of the entropy decoding unit 210, the inverse quantization unit 220, the inverse transform unit 230, the intra prediction unit 240, the inter prediction unit 250, the adder 255, the filtering unit 260, and the reference picture buffer 270 of the decoding apparatus 1500 may be program modules and may communicate with an external device or system. Program modules may be included in the decoding device 1500 in the form of an operating system, application program modules, and other program modules.

Program modules may be physically stored in various types of well-known memory devices. Furthermore, at least some of the program modules may also be stored in a remote memory storage device that is capable of communicating with the decoding apparatus 1500.

Program modules may include, but are not limited to, routines, subroutines, programs, objects, components, and data structures for performing functions or operations in accordance with the embodiments or for implementing abstract data types in accordance with the embodiments.

Program modules may be implemented using instructions or code executed by processor 1510.

Functions related to communication of data or information of the decoding apparatus 1500 may be performed by the communication unit 1520.

Fig. 16 is a flow chart of a decoding method according to an embodiment.

The decoding method according to the embodiment may be performed by the decoding apparatus 200 or the decoding apparatus 1500.

The communication unit 1520 may receive a bitstream from the encoding apparatus 100 or the encoding apparatus 1000 in step 1610.

The bitstream may include information on the encoded residual signal, and may further include prediction-related information.

The information on the encoded residual signal may include quantized transform coefficients that have been entropy encoded.

The prediction related information may be entropy encoded.

In step 1620, the entropy-decoding unit 210 may generate a quantized transform coefficient by performing entropy decoding on the bitstream. In addition, the entropy-decoding unit 210 may generate prediction-related information by performing entropy decoding on the bitstream.

In an example, the prediction-related information may include prediction scheme information indicating a scheme used to encode the current block.

In an example, the prediction scheme information may indicate which one of intra prediction and inter prediction is used to encode the current block. Alternatively, the prediction scheme information may indicate whether the current block has been encoded via intra prediction. Alternatively, the prediction scheme information may indicate whether the current block has been encoded via inter prediction.

In an example, the prediction-related information may include intra prediction mode information indicating a prediction mode of intra prediction.

In an example, the prediction related information may include second prediction usage information, wherein the second prediction usage information indicates whether the second prediction is to be used to encode the current block.

In an example, the prediction related information may include first prediction mode information indicating a prediction mode of the first prediction and second prediction mode information indicating a prediction mode of the second prediction.

In an example, the prediction related information may include selected partition information indicating a selected partition to which the second prediction is to be applied, wherein the selected partition is one of two partitions generated by partitioning the current block.

In an example, the prediction related information may include current block type information indicating a type of the current block, first prediction block type information indicating a type of the first prediction block, second prediction block type information indicating a type of the second prediction block, first reference block type information indicating a type of the first reference block, second reference block type information indicating a type of the second reference block, first reference sample point type information indicating a type of the first reference sample point, and second reference sample point type information indicating a type of the second reference sample point.

In an example, the prediction-related information may include first prediction region information indicating a first predicted region and/or second prediction region information indicating a second predicted region.

Further, the prediction-related information may include information for prediction described in the embodiments. The pieces of prediction-related information may be included in the bitstream according to the order described in the embodiments or the order generated in the embodiments. The reconstructed residual signal generation unit may acquire the pieces of prediction-related information from the bitstream according to an order described in the embodiments or an order generated in the embodiments.

In step 1630, the reconstructed residual signal generation unit may generate a reconstructed residual signal for the current block based on the quantized transform coefficients.

The reconstructed residual signal generation unit may include an inverse quantization unit 220 and an inverse transformation unit 230.

Step 1630 may include step 1631 and step 1632.

In step 1631, the inverse quantization unit 220 may generate reconstructed transform coefficients by performing inverse quantization on the quantized transform coefficients.

In step 1632, the inverse transform unit 230 may generate a reconstructed residual signal by performing inverse transform on the inverse quantized transform coefficients.

At steps 1610, 1720, and 1730, a reconstructed residual signal for the current block may be generated.

In step 1640, the reconstructed block generation unit may generate a reconstructed block based on the reconstructed residual signal, the second prediction, and the first prediction.

The reconstructed block generation unit may include an adder 225, an intra prediction unit 240, an inter prediction unit 250, a filter unit 260, and a reference picture buffer 270.

The reconstructed residual signal may be a reconstructed residual block.

In an example, the reconstructed block may be a sum of the reconstructed residual block and the prediction block. Alternatively, the reconstructed block may be generated based on the sum of the reconstructed residual signal and the prediction block.

In an example, the prediction block may be a sum of the first prediction block and the second prediction block. Alternatively, the prediction signal may be the sum of the first prediction signal and the second prediction signal. The second prediction signal may be a signal generated via a second prediction with respect to the selected portion of the current block. The first prediction signal may be a signal generated via a first prediction with respect to the current block or the remaining portion of the current block.

Alternatively, the reconstructed block may be generated based on a sum of the reconstructed residual block and the prediction block.

For example, the reconstructed block may be a block generated based on the reconstructed residual signal, the first prediction block, and the second prediction block. The reconstructed block generation unit may generate a reconstructed first residual signal by adding the reconstructed residual signal to the second prediction signal. The reconstructed block generation unit may generate a reconstructed block by adding the reconstructed first residual signal to the first prediction signal.

Alternatively, the reconstructed block generation unit may generate the reconstructed first residual block by adding the reconstructed residual block to the second prediction block. The reconstructed block generation unit may generate the reconstructed block by adding the reconstructed first residual block to the first prediction block.

In an embodiment, the second prediction signal may be a null signal or the second prediction block may be a null block when the second prediction is not used. The empty block may be a block in which all pixels have a value of 0.

The reconstructed block generation unit may generate the reconstructed block by performing inter prediction or intra prediction.

In an embodiment, the first prediction and the second prediction may be different types of predictions.

In an embodiment, the first prediction and the second prediction may be the same type of prediction. For example, the first prediction and the second prediction may both be intra-prediction. When the first prediction is intra prediction, the second prediction may also be set to intra prediction.

In an embodiment, the prediction direction of the first prediction and the prediction direction of the second prediction may be the same as each other. The prediction direction of the second prediction may be set to the same prediction direction as the first prediction.

For example, in an embodiment, the first prediction and the second prediction may both be inter-prediction.

In an embodiment, the block that is the target of the first prediction and the block that is the target of the second prediction may be different from each other. Here, the content indicated by the expression "a block as a target of prediction" may be different. Here, the expression "a block that is a target of prediction" may denote a block for which a prediction block is to be generated via "prediction".

For example, the type of the block that is the target of the first prediction and the type of the block that is the target of the second prediction may be different from each other. The types of blocks may include original blocks, luminance (luma) blocks, chrominance (chroma) blocks, depth blocks, residual blocks, and the like.

For example, the block that is the target of the first prediction may be a reconstructed block.

The reference block may be used for prediction. In an embodiment, the first reference block used for the first prediction and the second reference block used for the second prediction may be different from each other. The reference blocks may differ in the type and/or location of the reference blocks.

In an embodiment, a position of a first reference block for the first prediction and a position of a second reference block for the second prediction may be different from each other. Here, the position of the first reference block may be a position relative to a position of a block that is a target of the first prediction. The position of the second reference block may be a position relative to a position of a block that is a target of the second prediction.

In an embodiment, the first reference block used for the first prediction may comprise a plurality of reference blocks. The second reference block used for the second prediction may include a plurality of reference blocks. At least some of the plurality of first reference blocks used for the first prediction and the plurality of second reference blocks used for the second prediction may be different from each other.

The position of at least one of the plurality of first reference blocks used for the first prediction and the plurality of second reference blocks used for the second prediction may be included in only one of the positions of the plurality of first reference blocks and the positions of the plurality of second reference blocks.

In an embodiment, a type of a first reference block used for the first prediction and a type of a second reference block used for the second prediction may be different from each other. The types of blocks may include a reconstructed block, a reconstructed luma block, a reconstructed chroma block, a reconstructed depth block, a reconstructed first residual block, and a reconstructed second residual block.

For example, the first reference block used for the first prediction may be a reconstructed block. The second reference block used for the second prediction may be the reconstructed first residual block. The first reference sample used for the first prediction may be a pixel in the reconstructed block. The second reference samples used for the second prediction may be pixels in the reconstructed first residual block.

For example, the first reference block used for the first prediction may be a neighboring reconstructed block adjacent to the current block. The second reference block for the second prediction may be a reconstructed first residual block adjacent to the current block.

The neighboring reconstructed residual block adjacent to the current block may be obtained by adding a second prediction block of the neighboring reconstructed block to the reconstructed residual block for the neighboring reconstructed block. Alternatively, the neighboring reconstructed residual block adjacent to the current block may be a difference between the neighboring reconstructed block and the first prediction block of the neighboring reconstructed block.

For example, when the target of the first prediction is the current block, neighboring reconstructed blocks of the current block may be used as reference blocks in order to perform the prediction of the current block. When the second prediction is targeted for the first residual block, a neighboring reconstructed residual block of the current block or the first residual block may be used as a second reference block in order to perform the prediction of the first residual block.

In an embodiment, the first predicted region and the second predicted region may be different from each other. Here, the "predicted area" may mean an area that generates a predicted value in a block that is a target of prediction. Alternatively, the "predicted region" may represent a region to which a prediction value generated through prediction is allocated in a prediction block corresponding to a block that is a target of prediction.

In an embodiment, the second prediction may be a prediction for a selected portion of the current block. The first prediction may be a prediction of the entire current block. Alternatively, the first prediction may be a prediction for the remaining portion of the current block other than the selected portion for the second prediction.

In an example, the second predicted region may be a selected portion of the current block. The first predicted region may be the entire current block. Alternatively, the first predicted region may be a remaining portion of the current block except for the selected portion of the second prediction.

In an example, the second predicted region may be a non-square portion of the current block. The first predicted region may be the entire current block. Alternatively, the first predicted region may be a remaining non-square region in the current block except for the second predicted non-square region.

In an example, a prediction value determined via the first prediction may be allocated only to pixels that fall within a region of the first prediction in a block that is a target of the first prediction. A prediction value determined via the first prediction may not be allocated to pixels that fall outside the area of the first prediction in the block that is the target of the first prediction. Alternatively, a predefined value may be assigned to a pixel that falls outside the area of the first prediction in the block that is the target of the first prediction. The predefined value may be, for example, 0.

In an example, a prediction value determined via the second prediction may be allocated only to pixels that fall within a region of the second prediction in a block that is a target of the second prediction. A prediction value determined via the second prediction may not be allocated to pixels that fall outside the area of the second prediction in the block that is the target of the second prediction. Alternatively, a predefined value may be assigned to a pixel that falls outside the area of the second prediction in the block that is the target of the second prediction. The predefined value may be, for example, 0.

In an embodiment, the region of the first prediction may be determined based on a type of the first prediction. For example, the region of the first prediction may be determined based on whether the first prediction is inter-prediction. Alternatively, the first predicted region may be determined based on whether the first prediction is intra-prediction. Alternatively, the first predicted region may be determined based on a prediction direction of the first prediction.

In an embodiment, the area of the second prediction may be determined based on the type of the second prediction. For example, the region of the second prediction may be determined based on whether the second prediction is inter prediction. Alternatively, the second predicted region may be determined based on whether the second prediction is intra-prediction. Alternatively, the second predicted region may be determined based on a prediction direction of the second prediction.

Fig. 17 is a flowchart of a reconstructed block generation method according to an embodiment.

Step 1640, described above with reference to fig. 16, may include the following steps 1710, 1730, 1740, and 1750.

The second prediction may be selectively performed based on predefined conditions.

In step 1710, the reconstructed block generation unit may determine whether to generate a reconstructed block using the second prediction.

The reconstructed block generation unit may determine whether to generate the reconstructed block using the second prediction based on a predetermined condition.

In an embodiment, the reconstructed block generation unit may determine whether to use the second prediction based on a prediction mode of the first prediction.

For example, the reconstructed block generation unit may acquire first prediction mode information indicating a prediction mode of the first prediction from the bitstream. When the prediction mode of the first prediction is a non-directional mode, the reconstructed block generation unit may not use the second prediction.

In an embodiment, after determining whether to use the second prediction using the prediction mode, the position of the current block, and/or the number of neighboring reconstructed blocks, the reconstructed block generation unit may determine whether to use the second prediction based on whether the second prediction has been used to encode the current block.

For example, the reconstructed block generation unit may acquire the second prediction use information from the bitstream. The reconstructed block generation unit may use the second prediction when the second prediction use information indicates that the second prediction is to be used. When the second prediction usage information indicates that the second prediction will not be used, the reconstructed block generation unit may not use the second prediction.

When the second prediction is to be used to encode the current block, step 1720 may be performed.

When the second prediction is not to be used for encoding the current block, step 1740 may be performed.

In step 1720, the reconstructed block generation unit may generate a second prediction signal by performing the second prediction.

The reconstruction block generation unit may generate a second prediction signal by performing a second prediction on the selected portion.

The second prediction may correspond to the second prediction for the selected portion in encoding the current block described above with reference to fig. 11 and 13. For example, the second prediction may be a prediction for a selected portion of the current block. The second prediction signal may correspond to the second prediction for the selected portion described above with reference to fig. 11 and 13.

The second prediction signal may represent a second prediction block.

The first prediction and the second prediction may both be intra-prediction, and the second prediction mode of the second prediction may be different from the first prediction mode of the first prediction. Alternatively, the prediction direction of the second prediction may be different from the second prediction direction.

The reference block used for the first prediction and the reference block used for the second prediction may be reconstructed neighboring blocks adjacent to the current block.

In an embodiment, the selected partition on which the second prediction is performed may be one of two partitions generated by partitioning the current block.

In an embodiment, each of the two portions may have a non-square shape. The selected portion may have a non-square shape.

The selected portion of the second prediction to be performed may be determined based on a prediction direction of a prediction mode of the first prediction.

In an embodiment, the partitioning of the current block may be performed along the prediction direction according to the scheme described above with reference to fig. 12. The two portions may be determined based on a straight line passing through a center of the current block in a prediction direction of the first prediction. Each of the two partitions may be one of two regions generated by partitioning the current block along a straight line passing through the center of the current block in the prediction direction of the first prediction.

In an embodiment, the two portions created by partitioning may be equal in size. Alternatively, the two portions may have the same number of pixels.

In an embodiment, each of the two portions resulting from partitioning may have a non-square shape.

In an embodiment, the selected portion of the two portions to which the second prediction is to be applied may be selected based on the selected portion information. The reconstruction block generation unit may acquire the selected partial information from the bitstream. The selected portion may be selected based on selected portion information indicating one of the two portions. One of the two portions that is not indicated by the selected portion information may be a remaining portion.

For example, the value of the selected portion information may indicate one of the two portions to which the second prediction is to be applied. When the value of the selected part information is 0, the selected part may be S1 described above with reference to fig. 11. When the value of the selected part information is 1, the selected part may be S2 described above with reference to fig. 11.

In step 1730, the reconstructed block generation unit may add the second prediction signal for the selected portion to the reconstructed residual signal.

The second prediction signal may be a signal generated via a second prediction for the selected portion.

In step 1740, the reconstructed block generation unit may generate a first prediction signal for the current block or a remaining portion of the current block.

The reconstructed block generation unit may generate a first prediction signal by performing a first prediction on the current block or the remaining portion.

The first prediction signal may represent a first prediction block.

In the first prediction, reconstructed neighboring blocks of the current block may be used as reference blocks. Furthermore, the versions in the reconstructed neighboring blocks of the current block may be used as reference samples.

In an embodiment, the first prediction signal for the current block may be generated by performing the first prediction on the current block when the second prediction is not used. The first prediction signal may be a signal generated through a first prediction of the current block. When the second prediction is not used to decode the current block, the first prediction may be a prediction of the entire current block.

In an embodiment, the first prediction signal for the remaining portion is generated by performing the first prediction on the remaining portion when the second prediction is not used. The first prediction signal may be a signal generated via a first prediction for the remaining portion. When the second prediction is used to decode the current block, the first prediction may be a prediction for the remaining portion of the current block except for the selected portion used for the second prediction.

In step 1750, the reconstructed block generation unit may generate a reconstructed block by adding the first prediction signal for the current block or the remaining portion to the reconstructed residual signal.

In an embodiment, the reconstructed block generation unit may generate the reconstructed block by adding the first prediction signal for the current block to the reconstructed residual signal when the second prediction is not used. The first prediction signal may be a signal generated via a first prediction with respect to the current block.

In an embodiment, the reconstructed block generation unit may generate the reconstructed block by adding the first prediction signal for the remaining portion to the reconstructed residual signal when the second prediction is not used. The first prediction signal may be a signal generated via a first prediction for the remaining portion.

According to steps 1710, 1720, 1730, 1740 and 1750 described above, when the second prediction is used, the signal representing the reconstructed block may be the sum of the reconstructed residual signal, the second prediction signal for the selected portion and the first prediction signal for the remaining portion. When the second prediction is used, the second prediction signal for the selected portion and the first prediction signal for the remaining portion may be added to the reconstructed residual signal, as described above with reference to steps 1730 and 1750. Alternatively, when second prediction is used, a reconstructed block may be generated based on the reconstructed residual signal, the second prediction signal for the selected portion, and the first prediction signal for the remaining portion.

According to steps 1710, 1720, 1730, 1740, and 1750 described above, when the second prediction is not used, the signal representing the reconstructed block may be the sum of the reconstructed residual signal and the first prediction signal for the current block. Alternatively, when the second prediction is not used, a reconstructed block may be generated based on the reconstructed residual signal and the first prediction signal for the current block.

The reconstructed block may be used as a reference block for decoding other blocks.

According to the above-described embodiments, when the second prediction is used, the selected portion to which the first prediction is applied and the remaining portion to which the second prediction is applied may be separated.

In an embodiment, when the second prediction is used, the first prediction may be applied to the entire current block. In this case, a reconstructed block for the selected portion to which the second prediction is applied may be generated based on a sum of the reconstructed reference signal, a second prediction signal generated via the second prediction, and a first prediction signal generated via the first prediction. Further, a reconstructed block for a remaining portion to which the second prediction is not applied may be generated based on a sum of the reconstructed residual block and a first prediction signal generated via the first prediction.

In this case, the second prediction may be a prediction of a residual. The second prediction may be a prediction of a first residual signal, wherein the first residual signal is a difference between the current block and the first prediction signal. The reference block used for the second prediction may be a reconstructed residual block, and the reference pixel used for the second prediction may be a pixel in the reconstructed residual block. In other words, a second residual signal, which is a difference between the first residual signal and the second prediction signal, may be generated via second prediction of the first residual signal, and encoding information of the current block may be generated using the second residual signal.

Alternatively, in this case, the reconstructed block for the selected portion to which the second prediction is applied may be a weighted sum of the reconstructed residual block, the second prediction signal and the first prediction signal. The prediction related information may comprise a first weight for the reconstructed residual signal, a second weight for the second prediction signal and a third weight for the first prediction signal. It can be considered that: not using the second prediction means that the second weight for the second prediction signal is set to 0. Alternatively, it may be considered that: for the remaining portion to which the second prediction is not applied, a second weight for the second prediction is set to 0.

The above-described description of the encoding of the current block may also be applied to the decoding of the current block. Duplicate description is omitted. Furthermore, the above-described description of decoding of the current block may also be applied to encoding of the current block. Duplicate description will be omitted.

In the above-described embodiments, although the method has been described based on the flowchart as a series of steps or units, the present invention is not limited to the order of the steps, and some steps may be performed in an order different from the order of the steps already described or simultaneously performed with other steps. Furthermore, those skilled in the art will understand that: the steps shown in the flowcharts are not exclusive and other steps may be included, or one or more steps in the flowcharts may be deleted without departing from the scope of the invention.

The embodiments according to the present invention described above can be implemented as programs that can be executed by various computer apparatuses and can be recorded on computer-readable storage media. The computer readable storage medium may include program instructions, data files, and data structures, either alone or in combination. The program instructions recorded on the storage medium may be specially designed and configured for the present invention, or may be known or available to those having ordinary skill in the computer software art. Examples of the computer storage medium may include all types of hardware devices specifically configured to record and execute program instructions, such as magnetic media (such as hard disks, floppy disks, and magnetic tapes), optical media (such as Compact Disks (CD) -ROMs, and Digital Versatile Disks (DVDs)), magneto-optical media (such as floppy disks, ROMs, RAMs, and flash memories). Examples of program instructions include both machine code, such as created by a compiler, and high-level language code that may be executed by the computer using an interpreter. The hardware devices may be configured to operate as one or more software modules in order to perform the operations of the present invention, and vice versa.

As described above, although the present invention has been described based on specific details (such as detailed components and a limited number of embodiments and drawings), the specific details are provided only for easy understanding of the present invention, the present invention is not limited to these embodiments, and those skilled in the art will practice various changes and modifications according to the above description.

Therefore, it should be understood that the spirit of the present embodiments is not limited to the above-described embodiments, and the appended claims and their equivalents and modifications fall within the scope of the present invention.

Claims (1)

1. A decoding apparatus, comprising:

a reconstructed residual signal generation unit for generating a reconstructed residual signal for the current block;

a reconstructed block generating unit for generating a reconstructed block for the current block based on the reconstructed residual signal, the second prediction, and the first prediction,

wherein the second prediction is a prediction for a portion of the current block.

Images