CN115733977A9

CN115733977A9 - Method and apparatus for encoding and decoding video by using prediction

Info

Publication number: CN115733977A9
Application number: CN202211613078.1A
Authority: CN
Inventors: 林成昶; 高现硕; 姜晶媛; 李镇浩; 全东山; 金晖容; 全炳宇; 金男昱; 田丞秀; 崔振秀
Original assignee: Sungkyunkwan University School Industry Cooperation; Electronics and Telecommunications Research Institute ETRI
Current assignee: Sungkyunkwan University School Industry Cooperation; Electronics and Telecommunications Research Institute ETRI
Priority date: 2016-01-27
Filing date: 2017-01-26
Publication date: 2023-07-11
Also published as: CN116016912A; WO2017131473A1; CN115733977A; CN115941949A; CN115941949A9; CN116016911A; CN115733979A9; CN115733978A; CN115733979A; CN115941948A

Abstract

A method and apparatus for encoding and decoding video by using prediction are disclosed. In encoding the current block, a residual signal of the current block is generated based on the current block, the first prediction, and the second prediction. In addition, information on a residual signal encoded by encoding the residual signal is generated. In decoding the current block, a restored residual signal of the current block is generated. A restored block of the current block is generated based on the reconstructed residual signal, the second prediction, and the first prediction.

Description

Method and apparatus for encoding and decoding video by using prediction

The present application is a divisional application of patent application entitled "method and apparatus for encoding and decoding video by using prediction" having application date of 2017, 1, 26, and application number of 201780009090.2.

Technical Field

The following embodiments relate generally to a video decoding method and apparatus and a video encoding method and apparatus, and more particularly, to a method and apparatus for performing encoding and decoding on a video based on prediction of a target block.

Background

With the continued development of the information and communication industry, broadcast services with High Definition (HD) resolution have been popular throughout the world. Through this popularity, a large number of users have become accustomed to high resolution and high definition images and/or videos.

In order to meet the demands of users for high definition, a large number of institutions have accelerated the development of next-generation imaging devices. In addition to the increased interest of users in High Definition TV (HDTV) and Full High Definition (FHD) TV, there has been an increase in Ultra High Definition (UHD) TV, which has a resolution that is four times or more that of full high definition (FUD) TV. With this interest increasing, image encoding/decoding techniques for images with higher resolution and higher definition are required.

The image encoding/decoding apparatus and method may use an inter prediction technique, an intra prediction technique, an entropy encoding technique, or the like in order to perform encoding/decoding on high resolution and high definition images. The inter prediction technique may be a technique for predicting values of pixels included in a current picture using a temporally preceding picture and/or a temporally following picture. The intra prediction technique may be a technique for predicting a value of a pixel included in a current picture using information about the pixel in the current picture. Entropy coding techniques may be techniques for assigning short codes to more frequently occurring symbols and long codes to less frequently occurring symbols.

In image encoding and decoding processes, prediction may mean generating a predicted signal similar to the original signal. Predictions can be categorized primarily as: prediction of reference spatially reconstructed images, prediction of reference temporally reconstructed images, and prediction with reference to other symbols. In other words, the temporal reference may represent that the temporally reconstructed image is referenced and the spatial reference may represent that the spatially reconstructed image is referenced.

The current block may be a block that is a target to be currently encoded or decoded. The current block may be referred to as a "target block" or "target unit". In the encoding process, the current block may be referred to as an "encoding target block" or "encoding target unit". In the decoding process, the current block may be referred to as a "decoding target block" or a "decoding target unit".

Inter prediction may be a technique for predicting a current block using a temporal reference and a spatial reference. Intra prediction may be a technique for predicting a current block using only spatial references.

When intra prediction is performed to reduce spatial repetition, an image encoding/decoding technique encodes a current block using any one of a plurality of prediction modes. For example, in High Efficiency Video Coding (HEVC), 35 intra prediction modes are used.

The encoding apparatus generates a prediction block of the current block using all available prediction modes among a plurality of prediction modes, and selects one prediction mode that exhibits the best result from among the all available prediction modes as a prediction mode for encoding the current block. However, even if a prediction mode exhibiting the best result is used, there is still an error between the original block and the predicted block. Such errors are denoted as residual blocks.

For some pixels, there is a large error between the original block and the predicted block. Due to this large error, spatial repetition may not be sufficiently reduced even after transform and/or quantization is applied to the residual block.

Disclosure of Invention

Technical problem

Embodiments are directed to a method and apparatus for reducing prediction errors based on a prediction unit that occur due to prediction.

Embodiments are directed to a method and apparatus for reducing a prediction error occurring due to a prediction direction of a prediction mode.

Embodiments are directed to a method and apparatus for performing selective second prediction of a prediction error corresponding to a first prediction performed in a direction of a first prediction mode.

Embodiments are directed to a method and apparatus for performing selective second prediction on a portion of prediction errors corresponding to first prediction.

Embodiments are directed to a method and apparatus for performing selective second prediction on a portion having severe distortion in a prediction error corresponding to the first prediction.

Embodiments aim to provide a method and apparatus for preventing additional prediction errors from occurring when performing a second prediction on all prediction errors corresponding to a first prediction by performing a selective second prediction on only a part of the prediction errors corresponding to the first prediction.

Embodiments aim to provide a method and apparatus that substantially reduces spatial repetition by transform and quantization processes by reducing the large prediction error that occurs in the prediction direction.

Solution scheme

According to an aspect, there is provided an encoding method comprising: generating a residual signal of the current block based on the current block, the first prediction, and the second prediction; information about the encoded residual signal is generated by performing encoding on the residual signal, wherein the second prediction is a prediction of a prediction error in the first prediction.

According to another aspect, there is provided a decoding apparatus including: the reconstructed residual signal generating unit generates a reconstructed residual signal of the current block; the reconstructed block generation unit generates a reconstructed block for the current block based on the reconstructed residual signal, the second prediction, and the first prediction.

According to another aspect, there is provided a decoding method comprising: generating a reconstructed residual signal for the current block; a reconstructed block for the current block is generated based on the reconstructed residual signal, the second prediction, and the first prediction.

The reconstructed block may correspond to a sum of a reconstructed residual signal, a first prediction signal generated via a first prediction, and a second prediction signal generated via a second prediction.

The first prediction and the second prediction may both be intra predictions.

The second prediction and the first prediction may have the same prediction direction.

At least a portion of the plurality of first reference blocks for the first prediction and the plurality of second reference blocks for the second prediction may be different from each other.

The type of the first reference block for the first prediction may be different from the type of the second reference block for the second prediction.

The first reference block for the first prediction may be a neighboring reconstructed block neighboring the current block.

The second reference block for the second prediction may be a neighboring reconstructed residual block neighboring the current block.

The neighboring reconstructed residual block may be a difference between the neighboring reconstructed block and a first prediction block for the neighboring reconstructed block.

The region of the reference sample for the second prediction may be a portion of the region of the reference sample for the first prediction.

The region to which the second prediction is to be applied may be specified based on a range of reference samples to be used for the second prediction.

The region to which the second prediction is to be applied may be specified based on the prediction direction of the second prediction.

The range of reference samples may be specified based on information indicating a first reference sample to be used for the second prediction and information indicating the number of reference samples to be used for the second prediction.

The region to which the second prediction is to be applied may be a region generated when the reference sample falling within the range moves in the prediction direction of the second prediction.

Values of reference samples falling outside the range may be considered as null or 0.

The second prediction may not be used when the second prediction use information indicating whether the second prediction is to be used for encoding the current block indicates that the second prediction is not to be used.

When the prediction mode of the first prediction is a non-directional mode, the second prediction may not be used.

When the current block is adjacent to the upper boundary and the left boundary, the second prediction may not be used.

The boundary may be a boundary of a picture, a boundary of a slice, or a boundary of a parallel block.

Whether the second prediction is to be used may be determined based on the number of reconstructed blocks neighboring the current block.

When the second prediction is not used, a reconstructed block may be generated based on the reconstructed residual signal and a first prediction signal generated via the first prediction.

Advantageous effects

A method and apparatus for reducing prediction errors based on a prediction unit due to prediction are provided.

A method and apparatus for reducing prediction errors occurring due to a prediction direction of a prediction mode are provided.

A method and apparatus for performing selective second prediction of a prediction error corresponding to a first prediction performed in a direction of a first prediction mode are provided.

A method and apparatus for performing selective second prediction on a portion of prediction errors corresponding to a first prediction are provided.

A method and apparatus for performing selective second prediction on a portion having severe distortion in a prediction error corresponding to the first prediction are provided.

A method and apparatus for preventing an additional prediction error from occurring when performing a second prediction on all prediction errors corresponding to a first prediction by performing a selective second prediction on only a part of the prediction errors corresponding to the first prediction are provided.

Drawings

Fig. 1 is a block diagram showing the configuration of an embodiment of an encoding apparatus to which the present invention is applied;

fig. 2 is a block diagram showing the configuration of an embodiment of a decoding apparatus to which the present invention is applied;

fig. 3 is a diagram schematically showing a partition structure of an image when the image is encoded and decoded;

fig. 4 is a diagram showing the shape of a Prediction Unit (PU) that an encoding unit (CU) can include;

fig. 5 is a diagram illustrating the shape of a Transform Unit (TU) that can be included in a CU;

fig. 6 is a diagram for explaining an embodiment of an intra prediction process;

fig. 7 is a diagram for explaining the positions of reference samples used in an intra prediction process;

fig. 8 is a diagram for explaining an embodiment of an inter prediction process;

fig. 9 is a diagram illustrating a prediction error when an original image is predicted in a vertical direction according to an embodiment;

fig. 10 is a configuration diagram of an encoding apparatus according to an embodiment;

FIG. 11 is a flow chart of an encoding method according to an embodiment;

fig. 12 is a diagram illustrating first prediction and second prediction using intra prediction according to an embodiment;

fig. 13 is a flowchart of a residual block generation method according to an embodiment;

FIG. 14 is a flow chart of a method of generating a reconstructed block according to an embodiment;

Fig. 15 is a diagram showing an encoding process according to an embodiment;

fig. 16 is a configuration diagram of a decoding apparatus according to an embodiment;

FIG. 17 is a flow chart of a decoding method according to an embodiment;

FIG. 18 is a flow chart of a method of generating a reconstructed block according to an embodiment;

fig. 19 is a diagram illustrating a decoding process according to an embodiment.

Best mode for carrying out the invention

The following exemplary embodiments will be described in detail with reference to the accompanying drawings showing specific embodiments. These embodiments are described so that those of ordinary skill in the art to which the present disclosure pertains will be readily able to practice them. It should be noted that the various embodiments are different from each other, but need not be mutually exclusive. For example, the specific shapes, structures, and characteristics described herein may be implemented as other embodiments without departing from the spirit and scope of the various embodiments associated with one embodiment. Further, it is to be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the embodiments. Accordingly, the following detailed description is not intended to limit the scope of the disclosure, and the scope of the exemplary embodiments is defined only by the appended claims and their equivalents (as long as they are properly described).

In the drawings, like reference numerals are used to designate the same or similar functions in all respects. The shapes, sizes, etc. of components in the drawings may be exaggerated to make the description clear.

It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. Furthermore, it should be noted that, in the exemplary embodiments, the expression for describing "including" a specific component means that another component may be included in the practical scope or technical spirit of the exemplary embodiments, but does not exclude the presence of components other than the specific component.

Terms such as "first" and "second" may be used to describe various components, but the components are not limited by the terms. The terms are used only to distinguish one component from another. For example, a first component may be referred to as a second component without departing from the scope of the present description. Similarly, the second component may be referred to as a first component.

Furthermore, the components described in the embodiments are shown separately to represent different characteristic functions, but this does not mean that each component is formed of one separate hardware or software. That is, for convenience of description, a plurality of components are individually arranged and included. For example, at least two of the plurality of components may be integrated into a single component. Instead, one component may be divided into a plurality of components. Embodiments in which multiple components are integrated or embodiments in which some components are separated are included in the scope of the present specification as long as they do not depart from the essence of the present specification.

Further, some components are not necessary components for performing necessary functions, but may be optional components for merely improving performance. Embodiments may be implemented using only the necessary components to implement the essence of the embodiments. For example, structures that include only necessary components and exclude only optional components for improving performance are also included within the scope of the embodiments.

The embodiments will be described in detail below with reference to the drawings so that those skilled in the art to which the embodiments pertain can easily practice the embodiments. In the following description of the embodiments, a detailed description of known functions or configurations that are considered to obscure the gist of the present specification will be omitted.

Hereinafter, "image" may represent a single picture that forms part of a video, or may represent the video itself. For example, "encoding and/or decoding an image" may mean "encoding and/or decoding a video" and may also mean "encoding and/or decoding any one of a plurality of images constituting a video".

Hereinafter, the terms "video" and "moving picture" may be used to have the same meaning and may be interchangeable with each other.

Hereinafter, the terms "image", "picture", "frame", and "screen" may be used to have the same meaning and are interchangeable with each other.

In the following embodiments, specific information, data, flags, elements, and attributes may have their respective values. A value of 0 corresponding to each of the information, data, flags, elements, and attributes may indicate a logical false or a first predefined value. In other words, the value "0" (logical false) and the first predefined value may be interchanged with each other. The value "1" corresponding to each of the information, data, flags, elements, and attributes may indicate a logical true or a second predefined value. In other words, the value "1" (logical true) and the second predefined value may be interchanged with each other.

When a variable such as i or j is used to indicate a row, column, or index, the value i may be an integer of 0 or an integer greater than 0, or may be an integer of 1 or an integer greater than 1. In other words, in an embodiment, each of the rows, columns, and indexes may be counted starting from 0, or may be counted starting from 1.

Next, terms to be used in the embodiments will be described.

A unit: "unit" may mean a unit of image encoding and decoding. The meaning of the terms "unit" and "block" may be identical to each other. Furthermore, the terms "unit" and "block" are interchangeable with each other.

The cells (or blocks) may be an mxn sample matrix. M and N may each be positive integers. The term "cell" may generally refer to an array of two-dimensional (2D) samples. The term "sample" may be a pixel or a pixel value.

The terms "pixel" and "sample" may be used with the same meaning and are interchangeable with each other.

During the encoding and decoding of an image, a "unit" may be a region created by partitioning an image. A single image may be partitioned into multiple units. In encoding and decoding an image, a process predefined for each unit may be performed according to the type of the unit. The types of units may be classified into macro units, coding Units (CUs), prediction Units (PUs), and Transform Units (TUs) according to functions. The individual cells may be further partitioned into lower level cells having smaller dimensions than the size of the cell.

The unit partition information may comprise information about the depth of the unit. The depth information may represent the number and/or degree to which the unit is partitioned.

A single unit may be hierarchically partitioned into a plurality of lower layer units, while the plurality of lower layer units have tree-structure-based depth information. In other words, a cell and a lower level cell generated by partitioning the cell may correspond to a node and a child node of the node, respectively. Each partitioned lower level unit may have depth information. The depth information of a unit indicates the number and/or degree of times the unit is partitioned, and thus the partition information of a lower layer unit may include information about the size of the lower layer unit.

In a tree structure, the top node may correspond to the initial node before partitioning. The top node may be referred to as the "root node". Further, the root node may have a minimum depth value. Here, the depth of the top node may be level "0".

A node of depth level "1" may represent a cell that is generated when the initial cell is partitioned once. A node of depth level "2" may represent a cell that is generated when an initial cell is partitioned twice.

Leaf nodes of depth level "n" may represent units that are generated when an initial unit is partitioned n times.

The leaf node may be a bottom node, which cannot be partitioned further. The depth of the leaf node may be a maximum level. For example, the predefined value for the maximum level may be 3.

-a Transform Unit (TU): the TUs may be basic units of residual signal encoding and/or residual signal decoding (such as transform, inverse transform, quantization, inverse quantization, transform coefficient encoding, and transform coefficient decoding). A single TU may be partitioned into multiple TUs, wherein each of the multiple TUs has a smaller size.

-a Prediction Unit (PU): a PU may be a basic unit in the execution of prediction or compensation. A PU may be partitioned into multiple partitions by performing the partitioning. The plurality of partitions may also be basic units in the execution of prediction or compensation. The partition generated via partitioning the PU may also be a prediction unit.

-reconstructed neighboring units: the reconstructed neighboring unit may be a unit that has been previously encoded or decoded and reconstructed in the vicinity of the encoding target unit or decoding target unit. The reconstructed neighboring cell may be a cell that is spatially neighboring the target cell or a cell that is temporally neighboring the target cell.

-prediction unit partition: the prediction unit partition may represent the shape of the PU being partitioned.

-parameter set: the parameter set may correspond to information about a header of a structure of the bitstream. For example, the parameter sets may include a sequence parameter set, a picture parameter set, an adaptation parameter set, and the like.

Rate distortion optimization: the encoding device may use rate distortion optimization to provide higher encoding efficiency by utilizing a combination of: the size of a CU, prediction mode, size of a prediction unit, motion information, and size of a TU.

-rate distortion optimization scheme: the scheme may calculate the rate distortion costs for each combination to select the optimal combination from among the combinations. The rate distortion cost may be calculated using equation 1 below. In general, the combination that minimizes the rate-distortion cost may be selected as the optimal combination under the rate-distortion optimization method.

[ equation 1]

D+λ*R

Here, D may represent distortion. D may be an average (mean square error) of the squares of the differences between the original transform coefficients and the reconstructed transform coefficients in the transform block.

R represents a code rate, which may represent a bit rate using the relevant context information.

Lambda represents the lagrangian multiplier. R may include not only coding parameter information such as a prediction mode, motion information, and a coding block flag, but also bits generated due to coding of transform coefficients.

The encoding apparatus performs processes such as inter prediction and/or intra prediction, transformation, quantization, entropy coding, inverse quantization, and inverse transformation in order to calculate accurate D and R, but these processes greatly increase the complexity of the encoding apparatus.

-a reference picture: the reference picture may be an image used for inter prediction or motion compensation. The reference picture may be a picture including a reference unit that is referenced by the target unit to perform inter prediction or motion compensation. The terms "picture" and "image" may have the same meaning. Accordingly, the terms "picture" and "image" are interchangeable with each other.

-reference picture list: the reference picture list may be a list including reference pictures used for inter prediction or motion compensation. The type of reference picture list may be a merged List (LC), list 0 (L0), list 1 (L1), etc.

-Motion Vector (MV): the MV may be a 2D vector for inter prediction. For example, it may be as follows (mv) _x ，mv _y ) Is expressed as MV. mv (mv) _x Can indicate the horizontal component, mv _y The vertical component may be indicated.

The MV may represent an offset between the target picture and the reference picture.

Search range: the search range may be a 2D region in which a search for MVs is performed during inter prediction. For example, the size of the search range may be mxn. M and N may each be positive integers.

Fig. 1 is a block diagram showing the configuration of an embodiment of an encoding apparatus to which the present invention is applied.

The encoding apparatus 100 may be a video encoding apparatus or an image encoding apparatus. The video may include one or more images (pictures). The encoding device 110 may encode one or more images of the video sequentially over time.

Referring to fig. 1, the encoding apparatus 100 includes an inter prediction unit 110, an intra prediction unit 120, a switcher 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filtering unit 180, and a reference picture buffer 190.

The encoding apparatus 100 may perform encoding on an input image in an intra mode and/or an inter mode. The input image may be referred to as a "current image" as a target to be currently encoded.

Further, the encoding apparatus 100 may generate a bitstream including information about encoding by encoding an input image, and may output the generated bitstream.

When the intra mode is used, the switcher 115 can switch to the intra mode. When the inter mode is used, the switcher 115 can switch to the inter mode.

The encoding apparatus 100 may generate a prediction block for an input block in an input image. Further, after the prediction block is generated, the encoding apparatus 100 may encode a residual between the input block and the prediction block. The input block may be referred to as a "current block" as a target to be currently encoded.

When the prediction mode is an intra mode, the intra prediction unit 120 may use pixel values of previously encoded neighboring blocks around the current block as reference pixels. Intra-prediction unit 120 may perform spatial prediction on the current block using the reference pixels and generate prediction samples for the current block via spatial prediction.

The inter prediction unit 110 may include a motion prediction unit and a motion compensation unit.

When the prediction mode is an inter mode, the motion prediction unit may search a reference image for a region that best matches the current block in the motion prediction process, and may derive a motion vector for the current block and the found region. The reference picture may be stored in a reference picture buffer 190. More specifically, when encoding and/or decoding of a reference picture is processed, the reference picture may be stored in the reference picture buffer 190.

The motion compensation unit may generate the prediction block by performing motion compensation using the motion vector. Here, the motion vector may be a two-dimensional (2D) vector for inter prediction. Further, the motion vector may represent an offset between the current image and the reference image.

The subtractor 125 may generate a residual block, where the residual block is a residual between the input block and the prediction block. The residual block is also referred to as a "residual signal".

The transform unit 130 may generate transform coefficients by transforming the residual block, and may output the generated transform coefficients. Here, the transform coefficient may be a coefficient value generated by transforming the residual block. When the transform skip mode is used, the transform unit 130 may omit an operation of transforming the residual block.

By quantizing the transform coefficients, quantized transform coefficient levels may be generated. Here, in an embodiment, the quantized transform coefficient level may also be referred to as a "transform coefficient".

The quantization unit 140 may generate quantized transform coefficient levels by quantizing the transform coefficients according to quantization parameters. The quantization unit 140 may output quantized transform coefficient levels. In this case, the quantization unit 140 may quantize the transform coefficient using a quantization matrix.

The entropy encoding unit 150 may generate a bitstream by performing entropy encoding based on probability distribution based on the values calculated by the quantization unit 140 and/or the encoding parameter values calculated in the encoding process. The entropy encoding unit 150 may output the generated bitstream.

In addition to pixel information of an image, the entropy encoding unit 150 may perform entropy encoding on information required to decode the image. For example, information required for decoding an image may include syntax elements and the like.

The encoding parameters may be information required for encoding and/or decoding. The encoding parameters may include information encoded by the encoding device and transmitted to the decoding device, and may also include information derived during the encoding or decoding process. For example, the information transmitted to the decoding device may include syntax elements.

For example, the encoding device may include values or statistical information such as prediction modes, motion vectors, reference picture indexes, encoded block patterns, presence or absence of residual signals, transform coefficients, quantized transform coefficients, quantization parameters, block sizes, and block partition information. The prediction mode may be an intra prediction mode or an inter prediction mode.

The residual signal may represent a difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming a difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming and quantizing a difference between the original signal and the predicted signal. The residual block may be a block-based residual signal.

When entropy coding is applied, fewer bits may be allocated to more frequently occurring symbols and more bits may be allocated to less frequently occurring symbols. Since the symbol is represented by this allocation, the size of the bit string for the target symbol to be encoded can be reduced. Accordingly, the compression performance of video coding can be improved by entropy coding.

Further, for entropy encoding, an encoding method such as exponential golomb, context Adaptive Variable Length Coding (CAVLC), or Context Adaptive Binary Arithmetic Coding (CABAC) may be used. For example, the entropy encoding unit 150 may perform entropy encoding using a variable length coding/coding (VLC) table. For example, the entropy encoding unit 150 may derive a binarization method for the target symbol. Furthermore, entropy encoding unit 150 may derive a probability model for the target symbol/binary bit. The entropy encoding unit 150 may perform entropy encoding using a derived binarization method or probability model.

Since the encoding apparatus 100 performs encoding via inter prediction, the encoded current image may be used as a reference image for another image to be subsequently processed. Accordingly, the encoding apparatus 100 may decode the encoded current image and store the decoded image as a reference image. For decoding, inverse quantization and inverse transformation of the encoded current image may be performed.

The quantized coefficients may be inverse quantized by the inverse quantization unit 160 and may be inverse transformed by the inverse transformation unit 170. The coefficients that have been inverse quantized and inverse transformed may be added to the prediction block by adder 175. The inverse quantized and inverse transformed coefficients and the prediction block are added, and then a reconstructed block may be generated.

The reconstructed block may be filtered by a filtering unit 180. The filtering unit 180 may apply one or more of a deblocking filter, a Sample Adaptive Offset (SAO) filter, and an Adaptive Loop Filter (ALF) to the reconstructed block or the reconstructed picture. The filtering unit 180 may also be referred to as an "adaptive in-loop filter".

The deblocking filter may remove block distortion that occurs at the boundaries of the blocks. The SAO filter may add the appropriate offset value to the pixel value to compensate for the coding error. The ALF may perform filtering based on a comparison result between the reconstructed block and the original block. The reconstructed block that has been filtered by the filtering unit 180 may be stored in a reference picture buffer 190.

Fig. 2 is a block diagram showing the configuration of an embodiment of a decoding apparatus to which the present invention is applied.

The decoding apparatus 200 may be a video decoding apparatus or an image decoding apparatus.

Referring to fig. 2, the decoding apparatus 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transformation unit 230, an intra prediction unit 240, an inter prediction unit 250, an adder 255, a filtering unit 260, and a reference picture buffer 270.

The decoding apparatus 200 may receive the bit stream output from the encoding apparatus 100. The decoding apparatus 200 may perform decoding on the bit stream in an intra mode and/or an inter mode. Further, the decoding apparatus 200 may generate a reconstructed image via decoding, and may output the reconstructed image.

For example, an operation of switching to an intra mode or an inter mode based on a prediction mode for decoding may be performed by a switch. When the prediction mode for decoding is an intra mode, the switch may be operated to switch to the intra mode. When the prediction mode for decoding is an inter mode, the switch may be operated to switch to the inter mode.

The decoding apparatus 200 may acquire a reconstructed residual block from an input bitstream and may generate a prediction block. When the reconstructed residual block and the prediction block are acquired, the decoding apparatus 200 may generate the reconstructed block by adding the reconstructed residual block and the prediction block.

The entropy decoding unit 210 may generate symbols by performing entropy decoding on the bitstream based on the probability distribution. The generated symbols may include quantized coefficient format symbols. Here, the entropy decoding method may be similar to the entropy encoding method described above. That is, the entropy decoding method may be an inverse of the entropy encoding method described above.

The quantized coefficients may be dequantized by dequantization unit 220. Further, the inversely quantized coefficients may be inversely transformed by the inverse transformation unit 230. As a result of the inverse quantization and inverse transformation of the quantized coefficients, a reconstructed residual block may be generated. Here, the inverse quantization unit 220 may apply a quantization matrix to the quantized coefficients.

When the intra mode is used, the intra prediction unit 240 may generate a prediction block by performing spatial prediction using pixel values of previously encoded neighboring blocks around the current block.

The inter prediction unit 250 may include a motion compensation unit. When the inter mode is used, the motion compensation unit 250 may generate a prediction block by performing motion compensation using a motion vector and a reference image. The reference picture may be stored in a reference picture buffer 270.

The reconstructed residual block and the prediction block may be added to each other by an adder 255. The adder 255 may generate a reconstructed block by adding the reconstructed residual block and the prediction block.

The reconstructed block may be filtered by a filtering unit 160. The filtering unit 260 may apply one or more of a deblocking filter, an SAO filter, and an ALF to the reconstructed block or the reconstructed picture. The filtering unit 260 may output a reconstructed image (picture). The reconstructed image may be stored in a reference picture buffer 270 and may then be used for inter prediction.

Fig. 3 is a diagram schematically showing an image partition structure when an image is encoded and decoded.

In order to partition an image efficiently, a Coding Unit (CU) may be used in encoding and decoding. The term "unit" may be used to collectively designate 1) a block including image samples and 2) a syntax element. For example, "partition of a unit" may represent "partition of a block corresponding to the unit".

Referring to fig. 3, an image 200 is sequentially partitioned into units corresponding to a Largest Coding Unit (LCU), and a partition structure of the image 300 may be determined according to the LCU. Here, LCUs may be used to have the same meaning as Code Tree Units (CTUs).

The partition structure may represent a distribution of Coding Units (CUs) in LCU 310 for efficiently encoding an image. Such a distribution may be determined according to whether a single CU is to be partitioned into four CUs. The horizontal and vertical sizes of each CU resulting from partitioning may be half the horizontal and vertical sizes of the CU before being partitioned. Each partitioned CU may be recursively partitioned into four CUs, and in the same manner, the horizontal and vertical sizes of the four CUs are halved.

Here, partitioning of CUs may be performed recursively until a predefined depth. The depth information may be information indicating the size of the CU. Depth information may be stored for each CU. For example, the depth of the LCU may be 0 and the depth of the Smallest Coding Unit (SCU) may be a predefined maximum depth. Here, as described above, the LCU may be a CU having a maximum coding unit size, and the SCU may be a CU having a minimum coding unit size.

Partitioning begins at LCU 310, and the depth of a CU may be increased by "1" each time the horizontal and vertical dimensions of the CU are halved by partitioning. For each depth, a CU that is not partitioned may have a size of 2n×2n. Further, in the case where a CU is partitioned, a CU having a size of 2n×2n may be partitioned into four CUs each having a size of n×n. The dimension N may be halved each time the depth increases by 1.

Referring to fig. 3, an LCU of depth 0 may have 64×64 pixels. 0 may be the minimum depth. An SCU of depth 3 may have 8 x 8 pixels. 3 may be the maximum depth. Here, a CU having 64×64 pixels as an LCU may be represented by a depth of 0. A CU with 32 x 32 pixels may be represented by a depth of 1. A CU with 16 x 16 pixels may be represented by a depth of 2. A CU with 8×8 pixels as SCU may be represented by depth 3.

Further, information on whether the corresponding CU is partitioned may be represented by partition information of the CU. The partition information may be 1-bit information. All CUs except SCU may include partition information. For example, when a CU is not partitioned, the value of partition information of the CU may be 0. When a CU is partitioned, the value of partition information of the CU may be 1.

Fig. 4 is a diagram illustrating the shape of a Prediction Unit (PU) that an encoding unit (CU) can include.

Among CUs partitioned from LCUs, a CU that is no longer partitioned may be partitioned into one or more Prediction Units (PUs). This partitioning may also be referred to as "partitioning".

A PU may be a base unit for prediction. The PU may be encoded and decoded in any one of a skip mode, an inter mode, and an intra mode. The PU may be partitioned into various shapes according to various modes.

In skip mode, no partition may be present in the CU. In the skip mode, the 2n×2n mode 410 may be supported without partitioning, wherein the size of the PU and the size of the CU are the same as each other in the 2n×2n mode.

In inter mode, there may be 8 types of partition shapes in a CU. For example, in the inter mode, a 2n×2n mode 410, a 2n×n mode 415, an n×2n mode 420, an n×n mode 425, a 2n×nu mode 430, a 2n×nd mode 435, an nl×2n mode 440, and an nr×2n mode 445 may be supported.

In intra mode, a 2nx2n mode 410 and an nxn mode 425 may be supported.

In the 2n×2n mode 410, PUs of size 2n×2n may be encoded. A PU of size 2N x 2N may represent a PU of the same size as a CU. For example, a PU of size 2N x 2N may have a size of 64 x 64, 32 x 32, 16 x 16, or 8 x 8.

In the nxn mode 425, a PU of size nxn may be encoded.

For example, in intra prediction, four partitioned PUs may be encoded when the PU size is 8 x 8. The size of each partitioned PU may be 4 x 4.

When encoding a PU in intra mode, the PU may be encoded using any of a plurality of intra prediction modes. For example, HEVC techniques may provide 35 intra-prediction modes, and a PU may be encoded in any of the 35 intra-prediction modes.

Which of the 2nx2n mode 410 and the nxn mode 425 is to be used to encode the PU may be determined based on the rate distortion cost.

The encoding apparatus 100 may perform an encoding operation on a PU having a size of 2nx2n. Here, the encoding operation may be an operation of encoding the PU in each of a plurality of intra prediction modes that can be used by the encoding apparatus 100. By the encoding operation, the optimal intra prediction mode for a PU of size 2n×2n can be obtained. The optimal intra prediction mode may be an intra prediction mode that exhibits a minimum rate distortion cost when encoding a PU having a size of 2nx2n among a plurality of intra prediction modes that can be used by the encoding apparatus 100.

Further, the encoding apparatus 100 may sequentially perform encoding operations on respective PUs obtained by performing nxn partitioning. Here, the encoding operation may be an operation of encoding the PU in each of a plurality of intra prediction modes that can be used by the encoding apparatus 100. By the encoding operation, the best intra prediction mode for a PU of size nxn can be obtained. The optimal intra prediction mode may be an intra prediction mode that exhibits a minimum rate distortion cost when encoding a PU of size nxn among a plurality of intra prediction modes that can be used by the encoding apparatus 100.

The encoding apparatus 100 may determine which one of a PU of a size 2nx2n and a PU of a size nxn is to be encoded based on a comparison result between a rate distortion cost of the PU of the size 2nx2n and a rate distortion cost of the PU of the size nxn.

Fig. 5 is a diagram illustrating the shape of a Transform Unit (TU) that can be included in a CU.

A Transform Unit (TU) may be a basic unit in a CU used for processes such as transform, quantization, inverse transform, inverse quantization, entropy encoding, and entropy decoding. TUs may have a square or rectangular shape.

Among CUs partitioned from LCUs, a CU that is no longer partitioned into CUs may be partitioned into one or more TUs. Here, the partition structure of the TUs may be a quadtree structure. For example, as shown in FIG. 5, a single CU 510 may be partitioned one or more times according to a quadtree structure. With such partitioning, a single CU 510 may be composed of TUs having various sizes.

Fig. 6 is a diagram for explaining an embodiment of an intra prediction process.

The arrow extending radially from the center of the graph in fig. 6 may represent the prediction direction of the intra prediction mode. Further, numbers shown near the arrow may represent examples of mode values assigned to intra prediction modes or prediction directions of intra prediction modes.

Intra-coding and/or decoding may be performed using reference samples of units adjacent to the target unit. The neighboring unit may be a neighboring reconstruction unit. For example, intra-frame encoding and/or decoding may be performed using values of reference samples included in each neighboring reconstruction unit or encoding parameters of neighboring reconstruction units.

The encoding apparatus 100 and/or the decoding apparatus 200 may generate a prediction block by performing intra prediction on a target unit based on information about samples in a current picture. When intra prediction is performed, the encoding apparatus 100 and/or the decoding apparatus 200 may generate a prediction block for a target unit by performing intra prediction based on information about samples in a current picture. When intra prediction is performed, the encoding apparatus 100 and/or the decoding apparatus 200 may perform directional prediction and/or non-directional prediction based on at least one reconstructed reference sample.

The prediction block may represent a block generated as a result of performing intra prediction. The prediction block may correspond to at least one of a CU, PU, and TU.

The unit of the prediction block may have a size corresponding to at least one of the CU, PU, and TU. The prediction block may have a square shape with a size of 2n×2n or n×n. The dimensions N x N may include dimensions 4 x 4, 8 x 8, 16 x 16, 32 x 32, 64 x 64, etc.

Alternatively, the prediction block may be a square block of size 2×2, 4×4, 16×16, 32×32, 64×64, etc. or a rectangular block of size 2×8, 4×8, 2×16, 4×16, 8×16, etc.

Intra prediction may be performed according to an intra prediction mode for the target unit. The number of intra prediction modes that the target unit may have may be a predefined fixed value, and may be a value that is differently determined according to the properties of the prediction block. For example, the attribute of the prediction block may include the size of the prediction block, the type of the prediction block, and the like.

For example, the number of intra prediction modes may be fixed to 35 regardless of the size of the prediction unit. Alternatively, the number of intra prediction modes may be, for example, 3, 5, 9, 17, 34, 35, or 36.

As shown in fig. 6, the intra prediction modes may include two non-directional modes and 33 directional modes. The two non-directional modes may include a DC mode and a planar mode.

For example, in a vertical mode with a mode value of 26, prediction may be performed in the vertical direction based on the pixel value of the reference sample. For example, in a horizontal mode with a mode value of 10, prediction may be performed in the horizontal direction based on the pixel value of the reference sample. For example, in a vertical mode with a mode value of 26, prediction may be performed in the vertical direction based on the pixel value of the reference sample.

Even in the directional mode other than the above-described modes, the encoding apparatus 100 and the decoding apparatus 200 can perform intra prediction on the target unit using the reference samples according to the angle corresponding to the directional mode.

The intra prediction mode located at the right side with respect to the vertical mode may be referred to as a "vertical-right mode". The intra prediction mode located below the horizontal mode may be referred to as a "horizontal-below mode". For example, in fig. 6, an intra prediction mode in which the mode value is one of 27, 28, 29, 30, 31, 32, 33, and 34 may be the vertical-right mode 613. The intra prediction mode whose mode value is one of 2, 3, 4, 5, 6, 7, 8, and 9 may be the horizontal-down mode 616.

The non-directional modes may include a DC mode and a planar mode. For example, the mode value of the DC mode may be 1. The mode value of the planar mode may be 0.

The orientation pattern may include an angular pattern. Among the plurality of intra prediction modes, modes other than the DC mode and the plane mode may be a directional mode.

In the DC mode, a prediction block may be generated based on an average of pixel values of a plurality of reference samples. For example, the pixel values of the prediction block may be determined based on an average of the pixel values of the plurality of reference samples.

The number of intra prediction modes and the mode values of the respective intra prediction modes described above are merely exemplary. The number of intra prediction modes described above and the mode values of the respective intra prediction modes may be defined differently according to embodiments, implementations, and/or requirements.

The number of intra prediction modes may be different according to the type of color component. For example, the number of prediction modes may be different depending on whether the color component is a luminance (luma) signal or a chrominance (chroma) signal.

Fig. 7 is a diagram for explaining the positions of reference samples used in an intra prediction process.

Fig. 7 shows the positions of reference samples for intra prediction of a target unit. Referring to fig. 7, reconstructed reference samples for intra-prediction of a current block may include, for example, a lower left reference sample 731, a left reference sample 733, an upper left reference sample 735, an upper reference sample 737, and an upper right reference sample 739.

For example, the left reference sample 733 may represent a reconstructed reference sample adjacent to the left side of the target unit. The upper reference sample 737 may represent a reconstructed reference sample adjacent to the top of the target unit. The upper left corner reference sample 735 may represent a reconstructed reference sample located at the upper left corner of the target unit. The lower left reference sample point 731 may represent a reference sample point located below a left sample point line composed of the left reference sample point 733 among sample points located on the same line as the left sample point line. The upper right reference sample point 739 may represent a reference sample point located right of an upper sample point line composed of the upper reference sample points 737 among sample points located on the same line as the upper sample point line.

When the size of the target unit is n×n, the numbers of the lower left reference sample 731, the left reference sample 733, the upper reference sample 737, and the upper right reference sample 739 may all be N.

By performing intra prediction on the target unit, a prediction block may be generated. The process of generating the prediction block may include determining values of pixels in the prediction block. The target unit and the prediction block may be the same size.

The reference points used for intra-prediction of the target unit may vary according to the intra-prediction mode of the target unit. The direction of the intra prediction mode may represent a dependency relationship between the reference sample point and the pixels of the prediction block. For example, the value of the specified reference sample point may be used as the value of one or more specified pixels in the prediction block. In this case, the one or more specified pixels in the specified reference sample and prediction block may be samples and pixels located on a straight line along a direction of the intra prediction mode. In other words, the value of the specified reference sample point may be copied as a value of a pixel located in a direction opposite to the direction of the intra prediction mode. Alternatively, the value of a pixel in the prediction block may be the value of a reference sample located in the direction of the intra prediction mode with respect to the position of the pixel.

In one example, when the intra prediction mode of the target unit is a vertical mode with a mode value of 26, the upper reference sample 737 may be used for intra prediction. When the intra prediction mode is a vertical mode, the value of a pixel in the prediction block may be the value of a reference pixel located vertically above the position of the pixel. Thus, the upper reference sample 737 adjacent to the top of the target unit may be used for intra prediction. Furthermore, the values of the pixels in a row of the prediction block may be the same as the values of the pixels of the upper reference sample 737.

In one example, when the intra prediction mode of the current block is a horizontal mode with a mode value of 10, the left reference sample 733 may be used for intra prediction. When the intra prediction mode is a horizontal mode, the value of a pixel in the prediction block may be the value of a reference pixel horizontally located to the left of the pixel. Accordingly, the left reference sample 733 adjacent to the left side of the target unit may be used for intra prediction. Further, the values of pixels in a column of the prediction block may be the same as the values of pixels of the left reference sample 733.

In one example, when the mode value of the intra prediction mode of the current block is 18, at least some of the left reference samples 733, the upper left reference samples 735, and at least some of the upper reference samples 737 may be used for intra prediction. When the mode value of the intra prediction mode is 18, the value of a pixel in the prediction block may be the value of a reference pixel diagonally located at the upper left corner of the pixel.

Further, when an intra prediction mode having a mode value corresponding to 27, 28, 29, 30, 31, 32, 33, or 34 is used, at least some of the upper right reference samples 739 may be used for intra prediction.

Further, when an intra prediction mode having a mode value corresponding to 2, 3, 4, 5, 6, 7, 8, or 9 is used, at least some of the lower left reference sample points 731 may be used for intra prediction.

In addition, when an intra prediction mode having a mode value corresponding to any one of 11 to 25 is used, the upper left corner reference sample 735 may be used for intra prediction.

The number of reference samples used to determine the pixel value of one pixel in the prediction block may be 1 or 2 or more.

As described above, the pixel values of the pixels in the prediction block may be determined according to the positions of the pixels and the positions of the reference samples indicated by the direction of the intra prediction mode. When the position of the pixel and the position of the reference sample point indicated by the direction of the intra prediction mode are integer positions, the value of one reference sample point indicated by the integer position may be used to determine the pixel value of the pixel in the prediction block.

When the position of the pixel and the position of the reference sample indicated by the direction of the intra prediction mode are not integer positions, an interpolated reference sample based on two reference samples closest to the position of the reference sample may be generated. The values of the interpolated reference samples may be used to determine pixel values for pixels in the prediction block. In other words, when the position of a pixel in the prediction block and the position of a reference sample indicated by the direction of the intra prediction mode indicate the position between two reference samples, an interpolation value based on the values of the two samples may be generated.

The prediction block generated via prediction may be different from the original target unit. In other words, there may be a prediction error, which is a difference between the target unit and the prediction unit, and there may also be a prediction error between a pixel of the target unit and a pixel of the prediction block. For example, in the case of directional intra prediction, the longer the distance between the pixels of the prediction block and the reference sample, the greater the prediction error that may occur. Such prediction errors may cause discontinuities between the generated prediction block and neighboring blocks.

In order to reduce prediction errors, a filtering operation for the prediction block may be used. The filtering operation may be configured to adaptively apply the filter to regions of the prediction block that are considered to have large prediction errors. For example, the region considered to have a large prediction error may be a boundary of a prediction block. In addition, regions considered to have a large prediction error in a prediction block may differ according to an intra prediction mode, and characteristics of a filter may also differ according to an intra prediction mode.

Fig. 8 is a diagram for explaining an embodiment of an intra prediction process.

The rectangle shown in fig. 8 may represent an image (or screen). Further, in fig. 8, an arrow may indicate a prediction direction. That is, each image may be encoded and/or decoded according to a prediction direction.

The picture (or picture) may be classified into an intra picture (I picture), a unidirectional predicted picture or a predictive coded picture (P picture), and a bidirectional predicted picture or a bidirectional predictive coded picture (B picture) according to the type of coding. Each picture may be encoded according to the type of encoding of each picture.

When the image that is the target to be encoded is an I picture, the image itself may be encoded without inter prediction. When the image that is the target to be encoded is a P picture, the image may be encoded via inter prediction using only reference pictures in the forward direction. When the image that is the target to be encoded is a B picture, the image may be encoded via inter prediction using reference pictures in both the forward direction and the reverse direction, or may be encoded via inter prediction using reference pictures in one of the forward direction and the reverse direction.

P-pictures and B-pictures encoded and/or decoded using reference pictures may be considered as pictures using inter-prediction.

Hereinafter, inter prediction in inter mode according to an embodiment will be described in detail.

In the inter mode, the encoding apparatus 100 and the decoding apparatus 200 may perform prediction and/or motion compensation on the encoding target unit and the decoding target unit. For example, the encoding apparatus 100 or the decoding apparatus 200 may perform prediction and/or motion compensation by using motion information of neighboring reconstructed blocks as motion information of an encoding target unit or a decoding target unit. Here, the encoding target unit or the decoding target unit may represent a prediction unit and/or a prediction unit partition.

Inter prediction may be performed using reference pictures and motion information. Furthermore, inter prediction may use the skip mode described above.

The reference picture may be at least one of a picture before or after the current picture. Here, the inter prediction may perform prediction on a block in a current picture based on a reference picture. Here, the reference picture may represent an image used to predict a block.

Here, the region in the reference picture may be specified by using a reference picture index refIdx indicating the reference picture and a motion vector, which will be described later.

Inter prediction may select a reference picture and a reference block corresponding to a current block in the reference picture, and may generate a prediction block for the current block using the selected reference block. The current block may be a block that is a target to be currently encoded or decoded among blocks in the current picture.

The motion information may be derived by each of the encoding apparatus 100 and the decoding apparatus 200 during inter prediction. Furthermore, the derived motion information may be used to perform inter prediction.

Here, the encoding apparatus 100 and the decoding apparatus 200 may improve encoding efficiency and/or decoding efficiency by using motion information of neighboring reconstructed blocks and/or motion information of co-located blocks (col blocks). The col block may be a block corresponding to the current block in a co-located picture (col picture) that has been previously reconstructed.

The neighboring reconstructed block may be a block existing in the current picture and may be a block that has been previously reconstructed via encoding and/or decoding. The reconstructed block may be a neighboring block adjacent to the current block and/or a block located at an outer corner of the current block. Here, the "block located at the outer corner of the current block" may represent a block vertically adjacent to a neighboring block horizontally adjacent to the current block, or a block horizontally adjacent to a neighboring block vertically adjacent to the current block.

For example, the neighboring reconstruction unit (block) may be a unit located at the left side of the target unit, a unit located above the target unit, a unit located at the lower left corner of the target unit, a unit located at the upper right corner of the target unit, or a unit located at the upper left corner of the target unit.

Each of the encoding apparatus 100 and the decoding apparatus 200 may determine a block existing at a position spatially corresponding to the current block in the col picture, and may determine a predefined relative position based on the determined block. The predefined relative position may be an inner and/or outer position of the block that is present at a position spatially corresponding to the current block. Furthermore, each of the encoding device 100 and the decoding device 200 may derive a col block based on the predefined relative position that has been determined. Here, the col picture may be any one picture of one or more reference pictures included in the reference picture list.

The block in the reference picture may exist at a position in the reconstructed reference picture that spatially corresponds to the position of the current block. In other words, the position of the current block in the current picture and the position of the block in the reference picture may correspond to each other. Hereinafter, motion information of a block included in a reference picture may be referred to as "temporal motion information".

The method for deriving motion information may vary according to the prediction mode of the current block. For example, as a prediction mode applied to inter prediction, there may be an Advanced Motion Vector Predictor (AMVP) mode, a merge mode, or the like.

For example, when AMVP mode is used as the prediction mode, each of the encoding apparatus 100 and the decoding apparatus 200 may generate a prediction motion vector candidate list using motion vectors of neighboring reconstructed blocks and/or motion vectors of col blocks. The motion vectors of neighboring reconstructed blocks and/or the motion vectors of col blocks may be used as prediction motion vector candidates.

The bitstream generated by the encoding apparatus 100 may include a prediction motion vector index. The prediction motion vector index may represent a best prediction motion vector selected from among prediction motion vector candidates included in the prediction motion vector candidate list. The prediction motion vector index may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream.

The decoding apparatus 200 may select the predicted motion vector of the current block from among the predicted motion vector candidates included in the predicted motion vector candidate list using the predicted motion vector index.

The encoding apparatus 100 may calculate a Motion Vector Difference (MVD) between a motion vector of the current block and a predicted motion vector, and may encode the MVD. The bitstream may include an encoded MVD. The MVD may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. Here, the decoding apparatus 200 may decode the received MVD. The decoding apparatus 200 may derive a motion vector of the current block using the sum of the decoded MVD and the predicted motion vector.

The bitstream may include a reference picture index or the like for indicating a reference picture. The reference picture index may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. The decoding apparatus 200 may predict a motion vector of the current block using motion information of neighboring blocks, and may derive the motion vector of the current block using a difference (MVD) between the predicted motion vector and the motion vector. The decoding apparatus 200 may generate a prediction block for the current block based on the derived motion vector and the reference picture index information.

Since the motion information of the neighboring reconstructed block may be used to encode and decode the target unit, the encoding apparatus 100 may not separately encode the motion information of the target unit in a specific inter prediction mode. If the motion information of the target unit is not encoded, the number of bits transmitted to the decoding apparatus 200 may be reduced, and encoding efficiency may be improved. For example, there may be a skip mode and/or a merge mode that is an inter prediction mode that does not encode motion information of a target unit. Here, each of the encoding apparatus 100 and the decoding apparatus 200 may use an identifier and/or index indicating one neighboring reconstruction block of the plurality of neighboring reconstruction blocks, wherein motion information of the one neighboring reconstruction block is to be used as motion information of the target unit.

There is a merging method as another example of a method of deriving motion information. The term "merge" may refer to the merging of motions of multiple blocks. The term "merge" may mean that motion information of one block is also applied to other blocks. When merging is applied, each of the encoding apparatus 100 and the decoding apparatus 200 may generate a merge candidate list using motion information of neighboring reconstructed blocks and/or motion information of col blocks. The motion information may include at least one of: 1) motion vectors, 2) indices of reference pictures, and 3) prediction directions. The prediction direction may be unidirectional or bidirectional.

Here, merging may be applied on a CU basis or a PU basis. When merging is performed on a CU basis or a PU basis, the encoding apparatus 100 may transmit predefined information to the decoding apparatus 200 through a bitstream. The bitstream may include predefined information. The predefined information may include: 1) Information on whether to perform merging for each block partition, and 2) information on a neighboring block to be used for performing merging among a plurality of neighboring blocks neighboring to the current block. For example, neighboring blocks of the current block may include a left neighboring block of the current block, an upper neighboring block of the current block, a temporally neighboring block of the current block, and the like.

The merge candidate list may represent a list in which pieces of motion information are stored. Further, a merge candidate list may be generated before performing the merge. The motion information stored in the merge candidate list may be 1) motion information of a neighboring block adjacent to the current block and 2) motion information of a co-located block corresponding to the current block in the reference picture. Further, the motion information stored in the merge candidate list may be new motion information generated by combining pieces of motion information previously existing in the merge candidate list.

The skip mode may be a mode in which information on neighboring blocks is applied to the current block without change. The skip mode may be one of a plurality of modes for inter prediction. When the skip mode is used, the encoding apparatus 100 may transmit only information on a block whose motion information is to be used as motion information of the current block to the decoding apparatus 200 through a bitstream. The encoding apparatus 100 may not transmit other information to the decoding apparatus 200. For example, the other information may be syntax information. The syntax information may include Motion Vector Difference (MVD) information.

Fig. 9 illustrates a prediction error when an original image is predicted in a vertical direction according to an embodiment.

As described above, when encoding an original image is performed via intra prediction, a single PU and a single prediction mode may be selected after operations on all possible PUs and prediction modes are performed. When prediction of the original image is performed based on the selected PU and the selected prediction mode, a tendency of distortion of the prediction error to remain in the direction of the prediction mode is found.

Thus, a first intra prediction (i.e., a primary prediction) may be performed on the PU using one prediction mode selected from a plurality of prediction modes, and a second intra prediction (i.e., a secondary prediction) may be performed. Here, the second intra prediction may be intra prediction limited to a pixel region specified in the direction of the prediction mode of the first intra prediction. In other words, the prediction mode of the second intra prediction may be the same as the prediction mode of the first intra prediction. Alternatively, the first intra prediction may be applied to the entire PU, but the second intra prediction may be applied to only a portion of the PU.

The encoding device may determine the portion of the PU to which the second intra prediction is to be applied. For example, when the second intra prediction is applied to a plurality of candidate regions, the encoding apparatus may calculate respective rate-distortion costs for the plurality of candidate regions. The encoding device may determine the candidate region with the smallest rate-distortion cost as part of the PU to which the second intra prediction is to be applied.

A PU may include a plurality of pixels. The plurality of candidate regions may be part of a plurality of pixels in the PU, which may be determined according to a particular scheme. The plurality of candidate regions may be determined, for example, by a prediction direction of the first intra prediction.

By applying the first intra prediction and the second intra prediction, prediction errors in the PU may be further reduced compared to the case where only the first intra prediction is applied.

The following embodiments may provide a method and apparatus for improving the performance of intra prediction and reducing prediction errors by effectively utilizing the above trend.

Fig. 10 is a configuration diagram of an encoding apparatus according to an embodiment.

The encoding apparatus 1000 may be a general-purpose computer system that performs encoding.

As shown in fig. 10, the encoding apparatus 100 may include at least one processor 1010, a memory 1030, a User Interface (UI) input device 1050, a UI output device 1060, and a storage 1040 in communication with each other via a bus 1090. The electronic device 1000 may also include a communication unit 1020 that is connected to a network 1099. The processor 1010 may be a Central Processing Unit (CPU) or semiconductor device for executing processing instructions stored in the memory 1030 or storage 1040. Each of memory 1010 and storage 1040 may be any of various types of volatile or non-volatile storage media. For example, the memory may include at least one of Read Only Memory (ROM) 1031 and Random Access Memory (RAM) 1032.

The processor 1010 may include an inter prediction unit 110, an intra prediction unit 120, a switcher 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filtering unit 180, and a reference picture buffer 190 of the encoding apparatus 100.

The storage medium may store at least one module for operation of the encoding apparatus 1000. The memory 1030 may be configured to store at least one module and allow the module to be executed by the at least one processor 1010.

According to an embodiment, at least some of the inter prediction unit 110, the intra prediction unit 120, the switcher 115, the subtractor 125, the transform unit 130, the quantization unit 140, the entropy encoding unit 150, the inverse quantization unit 160, the inverse transform unit 170, the adder 175, the filtering unit 180, and the reference picture buffer 190 of the encoding apparatus 1000 may be program modules and may communicate with external devices or systems. The program modules may be included in the encoding device 1000 in the form of an operating system, application program modules, and other program modules.

The program modules may be physically stored in various types of well known storage devices. Furthermore, at least some of the program modules may also be stored in a remote storage device capable of communicating with the encoding apparatus 1000.

Program modules may include, but are not limited to, routines, subroutines, programs, objects, components, and data structures for performing functions or operations according to embodiments or for implementing abstract data types according to embodiments.

The program modules may be implemented using instructions or code executed by the processor 1010.

The functions related to the communication of data or information of the encoding apparatus 1000 may be performed by the communication unit 1020.

Fig. 11 is a flowchart of an encoding method according to an embodiment.

The encoding method according to the embodiment may be performed by the encoding apparatus 100 or the encoding apparatus 1000.

In step 1110, the prediction part may generate a residual signal of the current block based on the current block, the first prediction, and the second prediction.

The prediction part may include an inter prediction unit 110, an intra prediction unit 120, and a subtractor 125.

In one example, the residual signal may represent a residual block.

In one example, the residual block may be a difference between the current block and the prediction block. Alternatively, the residual block may be generated based on a difference between the current block and the prediction block.

In one example, the prediction block may be a sum of the first prediction block and the second prediction block. The residual block may be obtained by subtracting the sum of the first prediction block and the second prediction block from the current block.

In one example, the second prediction may be a prediction of a prediction error in the first prediction.

The first prediction block may be a block generated via the first prediction. The second prediction block may be a block generated via a second prediction. When the second prediction is not used, the second prediction block may be a null block. A null block may be a block with a value of 0 for all pixels.

The prediction part may generate a residual block for the current block by performing inter prediction or intra prediction.

In an embodiment, the prediction part may generate the second prediction block based on the current block and/or the first prediction block.

In an embodiment, the prediction part may generate the residual signal of the current block by performing a first prediction on the current block and performing a second prediction on a first prediction error that is a result of the first prediction. The first prediction error may be a difference between the current block and the first prediction signal. The residual signal may be a difference between the first prediction error and the second prediction signal.

In an embodiment, the first prediction and the second prediction may be different types of predictions.

In an embodiment, the first prediction and the second prediction may be the same type of prediction. For example, the first prediction and the second prediction may both be intra predictions. When the first prediction is intra prediction, the second prediction may also be set to intra prediction.

In an embodiment, the prediction direction of the first prediction and the prediction direction of the second prediction may be the same as each other. The prediction direction of the second prediction may be set to the same prediction direction as the first prediction.

For example, in an embodiment, the first prediction and the second prediction may both be inter-prediction.

In an embodiment, the block as a target of the first prediction and the block as a target of the second prediction may be different from each other. Here, the content represented by the expression "block as a target of prediction" may be different. Here, the expression "a block as a target of prediction" may denote a block in which a prediction block is to be generated via "prediction".

For example, the type of the block as the target of the first prediction and the type of the block as the target of the second prediction may be different from each other. The types of blocks may include an original block, a luminance (luma) block, a chrominance (chroma) block, a depth block, a residual block, and the like.

For example, the block that is the target of the first prediction may be the current block or the original block.

For example, a first prediction error generated via a first prediction may represent a first prediction error block. The first prediction error may be a first residual signal. Further, the first residual signal may represent a first residual block. The block targeted for the second prediction may be the first prediction error block or the first residual block. In this case, the first residual signal may be generated via a first prediction, and the second prediction error may be generated via a second prediction. The second prediction error may be a second residual signal. The second residual signal may represent a second residual block.

The reference block may be used for prediction. In an embodiment, the first reference block for the first prediction and the second reference block for the second prediction may be different from each other. The reference blocks may differ in the type and/or location of the reference blocks.

In an embodiment, the position of the first reference block for the first prediction and the position of the second reference block for the second prediction may be different from each other. Here, the position of the first reference block may be a position relative to the position of the block that is the target of the first prediction. The position of the second reference block may be a position relative to a position of the block that is a target of the second prediction.

In an embodiment, the first reference block for the first prediction may include a plurality of reference blocks. The second reference block for the second prediction may include a plurality of reference blocks. At least some of the plurality of first reference blocks for the first prediction and the plurality of second reference blocks for the second prediction may be different from each other.

The positions of at least some of the plurality of first reference blocks for the first prediction and the plurality of second reference blocks for the second prediction may be included only in one of the positions of the plurality of first reference blocks and the positions of the plurality of second reference blocks.

In an embodiment, the type of the first reference block for the first prediction and the type of the second reference block for the second prediction may be different from each other. The types of blocks may include a reconstructed block, a reconstructed luma block, a reconstructed chroma block, a reconstructed depth block, a reconstructed first residual block, and a reconstructed second residual block.

For example, the first reference block for the first prediction may be a reconstructed block. The second reference block for the second prediction may be the reconstructed first residual block. The first reference sample for the first prediction may be a pixel in the reconstructed block. The second reference sample for the second prediction may be a pixel in the reconstructed first residual block.

For example, the first reference block for the first prediction may be a neighboring reconstructed block adjacent to the current block. The second reference block for the second prediction may be a reconstructed first residual block adjacent to the current block.

An adjacent reconstructed residual block adjacent to the current block may be obtained by adding the second prediction block of the adjacent reconstructed block to the reconstructed residual block for the adjacent reconstructed block. Alternatively, the neighboring reconstructed residual block neighboring the current block may be a difference between the neighboring reconstructed block and the first prediction block of the neighboring reconstructed block.

For example, when the target of the first prediction is a current block, a neighboring reconstructed block of the current block may be used as a reference block in order to perform prediction of the current block. When the second prediction targets the first residual block, a neighboring reconstructed residual block of the current block or the first residual block may be used as the second reference block in order to perform prediction of the first residual block.

In an embodiment, the first predicted region and the second predicted region may be different from each other. Here, the "predicted region" may represent a region in which a predicted value is generated in a block that is a target of prediction. Alternatively, the "predicted region" may represent a region allocated with a prediction value generated via prediction in a prediction block corresponding to a block that is a target of prediction.

For example, the first prediction may be a prediction of a region of the current block. The second prediction may be a prediction of a portion of the region. At least a portion of the region of the current block may not be second predicted. The block targeted for the first prediction may be a block within a rectangular region. A second prediction may be performed on a portion of the rectangular region. At least a portion of the rectangular region may not be the target of the second prediction.

Considering the case where the region of the block as the target of the first prediction is the same as the region of the block as the target of the second prediction, the first predicted region may be the entire block as the target of the first prediction. The second predicted region may be a portion of a block that is a target of the second prediction. Alternatively, at least a portion of the blocks targeted for the second prediction may not be subjected to the second prediction. In other words, the first prediction may be performed on the entire block that is the target of the first prediction. The second prediction may be performed on a portion of the blocks that are the target of the second prediction.

In an example, the prediction value determined via the first prediction may be allocated only to pixels falling within the region of the first prediction in the block that is the target of the first prediction. The prediction value determined via the first prediction may not be allocated to pixels that fall outside the region of the first prediction in the block that is the target of the first prediction. Alternatively, a predefined value may be assigned to pixels that fall outside the region of the first prediction in the block that is the target of the first prediction. The predefined value may be 0.

In one example, the prediction value determined via the second prediction may be allocated only to pixels falling within the region of the second prediction in the block that is the target of the second prediction. The prediction value determined via the second prediction may not be allocated to pixels that fall outside the region of the second prediction in the block that is the target of the second prediction. Alternatively, a predefined value may be assigned to pixels that fall outside the region of the second prediction in the block that is the target of the second prediction. The predefined value may be, for example, 0.

In one example, when the size of the block that is the target of the first prediction is n×n, the respective prediction values determined via the first prediction may be allocated to n×n pixels. In other words, the prediction value determined via the first prediction may be allocated to all pixels in the block that is the target of the first prediction.

In one example, when the size of the block that is the target of the second prediction is nxn, the prediction value determined via the first prediction may not be allocated to one or more of the nxn pixels. Alternatively, when the size of the block as the target of the second prediction is n×n, a predefined value may be allocated to one or more of the n×n pixels. The predefined value may be, for example, 0.

In an embodiment, the region of the first prediction may be determined based on the type of the first prediction. For example, the region of the first prediction may be determined based on whether the first prediction is inter prediction. Alternatively, the region of the first prediction may be determined based on whether the first prediction is intra prediction. Alternatively, the region of the first prediction may be determined based on the prediction direction of the first prediction.

In an embodiment, the region of the second prediction may be determined based on the type of the second prediction. For example, the region of the second prediction may be determined based on whether the second prediction is inter prediction. Alternatively, the region of the second prediction may be determined based on whether the second prediction is intra prediction. Alternatively, the region of the second prediction may be determined based on the prediction direction of the second prediction.

In an embodiment, the second prediction may be performed using a second reference sample that falls within the selectable range.

In an embodiment, the region of the first reference sample for the first prediction and the region of the second reference sample for the second prediction may be different from each other.

As described above, the first predicted region and the second predicted region may be different from each other. The region of the first reference sample may be a region of the reference sample required to generate a prediction value for the region of the first prediction via the first prediction. The region of the second reference sample may be a region of the reference sample required to generate a prediction value for the region of the second prediction via the second prediction.

As described above, the region of the first prediction may be the entire block that is the target of the first prediction. The second predicted region may be a portion of a block that is a target of the second prediction. In this case, the region of the second reference sample point may be a part of the region of the first reference sample point.

In step 1120, the encoding part may perform encoding on the residual signal. The encoding part may generate information about the encoded residual signal by performing encoding on the residual signal.

The encoding part may include a transform unit 130, a quantization unit 140, and an entropy encoding unit 150.

Step 1120 may include

steps

1121, 1122, and 1123.

In step 1121, the transform unit 130 may generate transform coefficients for the residual signal.

In step 1122, the quantization unit 1140 may generate quantized transform coefficients by performing quantization on the transform coefficients.

In step 1123, the entropy encoding unit 150 may generate information about the encoded residual signal by performing entropy encoding on the quantized transform coefficients.

In step 1130, the entropy encoding unit 150 may generate a bitstream of information about the encoded residual signal.

The bitstream may include information about the encoded residual signal and may also include prediction related information.

The entropy encoding unit 150 may include prediction related information in the bitstream. The prediction related information may be entropy encoded.

In an example, the prediction related information may include prediction scheme information indicating a scheme used to encode the current block.

In an example, the prediction scheme information may indicate which of intra prediction and inter prediction is used to encode the current block. Alternatively, the prediction scheme information may indicate whether the current block has been encoded via intra prediction. Alternatively, the prediction scheme information may indicate whether the current block has been encoded via inter prediction.

In an example, the prediction related information may include intra prediction mode information indicating a prediction mode of intra prediction.

In an example, the prediction related information may include second prediction use information, wherein the second prediction use information indicates whether the second prediction is to be used for encoding the current block.

In an example, the prediction related information may include current block type information indicating a type of a current block, first prediction block type information indicating a type of a first prediction block, second prediction block type information indicating a type of a second prediction block, first reference block type information indicating a type of a first reference block, second reference block type information indicating a type of a second reference block, first reference sample point type information indicating a type of a first reference sample point, and second reference sample point type information indicating a type of a second reference sample point.

In an example, the prediction related information may include first prediction region information indicating a region of the first prediction and/or second prediction region information indicating a region of the second prediction.

The prediction related information may include information for the prediction that has been described in the embodiment. The entropy encoding unit 150 may include a plurality of pieces of prediction related information in the bitstream according to the order described in the embodiment or the order generated in the embodiment.

In step 1140, the communication unit 1020 may transmit a bitstream to the decoding apparatus 200 or the decoding apparatus 1700.

In step 1150, the reconstructed residual signal generation unit may generate a reconstructed residual signal based on the information about the residual signal.

The reconstructed residual signal generation unit may include an inverse quantization unit 160 and an inverse transformation unit 170.

Step 1150 may include step 1151 and step 1152.

In step 1151, the inverse quantization unit 160 may generate reconstructed transform coefficients by performing inverse quantization on the quantized transform coefficients.

In step 1152, the transform unit 130 may generate a reconstructed residual signal by performing inverse transform on the reconstructed transform coefficients.

In step 1160, the reconstructed block generation unit may generate a reconstructed block based on the reconstructed residual signal, the first prediction, and the second prediction.

The reconstructed block generation unit may include an adder 175.

In an example, the sum of the reconstructed residual signal and the prediction signal may represent a reconstructed block. Alternatively, the reconstructed block may be generated based on a sum of the reconstructed residual signal and the prediction signal.

In an example, the reconstructed block may be a sum of a reconstructed residual block and a prediction block. The prediction block may be a sum of the first prediction block and the second prediction block.

Alternatively, the reconstructed block may be generated based on a sum of the reconstructed residual block and the prediction block.

In an example, the reconstructed block may be a block generated based on the reconstructed residual block, the second prediction block, and the first prediction block. The reconstructed block generation unit may generate the reconstructed first residual signal by adding the reconstructed residual signal to the second prediction signal. The reconstructed block generation unit may generate the reconstructed block by adding the reconstructed first residual signal and the first prediction signal.

Alternatively, the reconstructed block generation unit may generate the reconstructed first residual block by adding the reconstructed residual block to the second prediction block. The reconstructed block generation unit may generate the reconstructed block by adding the reconstructed first residual block and the first prediction block.

The reconstructed block and the reconstructed first residual signal may be used as a reference block for encoding other blocks.

Fig. 12 shows a first prediction and a second prediction using intra prediction according to an embodiment.

In step 1110 described above with reference to fig. 11, the first prediction and the second prediction may both be intra predictions. In fig. 12, an example of encoding the current block when both the first prediction and the second prediction are intra predictions is shown.

The current block may be a PU. In fig. 12, PU of size 2n×2n as a current block is depicted.

First, in order to encode the current block, the prediction part may determine which one of a plurality of prediction modes is to be used to perform the first prediction.

The prediction section may perform the first prediction using each of a plurality of prediction modes. For example, prediction directions of the plurality of prediction modes may be different from each other.

In the first prediction, a reconstructed block adjacent to the current block may be used as a reference block. Furthermore, pixels adjacent to the current block in the reconstructed block may be used as reference samples.

The prediction unit may generate a plurality of first prediction blocks for a plurality of prediction modes via the first prediction.

Furthermore, the prediction unit may generate a plurality of first prediction errors for a plurality of prediction modes via the first prediction. For a single prediction mode, a first prediction block and a first prediction error may be calculated.

The first prediction error may represent a difference (residual) between the current block and the first prediction block. The first prediction error may represent a first residual block.

When the first prediction error is generated, the prediction part may select a region to which the second prediction is to be applied from the first residual block by referring to the first prediction error. The selected region may be part of the first residual block.

Next, in order to encode the current block, the prediction part may determine a portion of the first residual block to which the second prediction is to be applied.

The second prediction may be applied to a portion, i.e., a selected region, of the first residual block. In order to select a region to which the second prediction is to be applied, various methods may be used.

The prediction region for the second prediction may be specified by a range of reference samples used for the second prediction.

In the second prediction, a reconstructed residual block adjacent to the current block may be used as a reference block. Furthermore, pixels adjacent to the current block in the reconstructed residual block may be used as reference samples.

The prediction direction of the first prediction and the prediction direction of the second prediction may be identical to each other. In other words, the intra prediction mode of the first prediction and the intra prediction mode of the second prediction may be identical to each other.

In fig. 12, an intra prediction mode having a mode value of 30 is exemplarily shown.

In general, if the prediction direction of the first prediction and the prediction direction of the second prediction are the same, the range of the first predicted reference sample and the range of the second predicted reference sample may be the same as each other.

The second prediction may be applied to a portion of the first residual block. The range of second predicted reference samples may be a fraction of the range of first predicted reference samples.

In fig. 12, the range of the second predicted reference sample point is shown as the range from α to α+β.

α may represent a first reference sample used for the second prediction among reference samples in the prediction direction of the second prediction. Furthermore, α may represent the starting point of the second prediction.

α+β may represent the last reference sample to be used for the second prediction among the reference samples in the prediction direction of the second prediction. Alternatively, α+β may represent the last point of the second prediction.

Beta may represent the number of reference samples to be used for the second prediction. Alternatively, β may represent the range of the second prediction or the length of the second prediction.

Once the range of reference points is determined, a second predicted region may be specified based on the determined range of reference points and the prediction direction. In fig. 12, an area designated by a broken line in the residual block is shown. The second predicted region may be a region generated when the reference sample point falling within the range from α to α+β moves in the prediction direction.

The prediction section may select a range of reference samples to be used for the second prediction from among the reference samples for the prediction direction. The prediction section may perform the second prediction using reference samples falling within the selected range among all reference samples in the prediction direction. In this case, all the reference samples may correspond to the reference samples used for the first prediction.

In the second prediction, the values of the reference samples falling outside the selected range may be regarded as null or 0. For example, the value of the reference sample before α may be considered 0 and the value of the reference sample after α+β may be considered 0.

The prediction unit may generate a plurality of second prediction blocks for the plurality of candidate regions via the second prediction.

The plurality of candidate regions may be different from each other. The plurality of candidate regions may be different portions in the first residual block.

For example, the region of the first residual block may be designated as a reference sample used for the second prediction. In this case, a portion of the first residual block or the second predicted region in the first residual block may be represented as an ordered pair (α, β). The plurality of ordered pairs (α, β) corresponding to the plurality of candidate regions may be different from each other. In other words, the ordered pair (α, β) may represent the range of reference samples to be used for the second prediction.

The plurality of candidate regions may all be different regions that can be selected using a specific scheme for the first residual block. For example, the specific scheme may be a scheme using a predicted direction.

For example, the region of the first residual block may be designated as a reference sample used for the second prediction. When the number of reference samples that can be used for the entire first residual block in the prediction mode of the second prediction is n, α may be equal to or greater than 0 and less than or equal to n-1. Beta may be equal to or greater than 0 and less than or equal to n-1-alpha. Furthermore, when the number of reference samples that can be used for the entire first residual block in the prediction mode of the second prediction is n, the number of ordered pairs (α, β) may be (n+1) n/2. In other words, when the number of reference samples that can be used for the entire first residual block in the prediction mode of the second prediction is n, the number of the plurality of candidate regions may be (n+1) n/2.

For example, the plurality of candidate regions may respectively correspond to a range of reference samples to be used for the second prediction.

Alpha and beta may be indices of a list of reference samples in each specified prediction direction.

In order to use α and β to indicate the position of a reference sample in the list, the order of the reference samples must be defined.

In an embodiment, the smaller the X-coordinate of the reference sample point, the earlier the sequential position of the reference sample point can be considered first. Next, among the reference points having the same X-coordinate value, the larger the Y-coordinate value of the reference point, the earlier the sequential position of the reference point. For example, the sequential positions of the reference samples may be specified in the order of the lower left reference sample 731, the left side reference sample 733, the upper left corner reference sample 735, the upper reference sample 737, and the upper right reference sample 739. In the lower left reference sample 731, the reference sample located at the lower part may have an earlier sequential position than that of the reference sample located at the upper part. In the left reference sample 733, the reference sample located at the lower portion may have an earlier sequential position than that of the reference sample located at the upper portion. In the upper reference sample 737, the reference sample located on the left side may have an earlier sequential position than the sequential position of the reference sample located on the right side. In the upper right reference sample 739, the reference sample located on the left side may have an earlier sequential position than the sequential position of the reference sample located on the right side.

The order of the reference points described above is merely an example, and other schemes based orders may also be used. For example, the order of the reference samples may be specified in the reverse order of the order described above.

Furthermore, the order of the reference points may be assigned to the reference points determined according to the prediction direction. In other words, among the above-described lower left reference sample 731, left side reference sample 733, upper left corner reference sample 735, upper reference sample 737, and upper right reference sample 739, only some reference samples determined according to the prediction direction of the second prediction may be used for prediction of the entire first residual block. The order may be assigned to certain reference points.

When the prediction direction of the second prediction is, for example, a vertical direction, the reference sample that can be used for the entire first residual block may be the upper reference sample 737. When the size of the current block is 2n×2n, the sequential position of the leftmost reference sample among the upper reference samples 737 may be 0, and the sequential position of the rightmost reference sample among the upper reference samples 737 may be 2N-1.

Alternatively, when the prediction direction of the second prediction is, for example, a horizontal direction, the reference sample that can be used for the entire first residual block may be the left reference sample 733. When the size of the current block is 2n×2n, the sequential position of the lowermost reference sample among the left reference samples 737 may be 0, and the sequential position of the uppermost reference sample among the left reference samples 737 may be 2N-1.

The prediction unit may generate a plurality of second prediction errors for the plurality of candidate regions via the second prediction. For one region, a second prediction block and a second prediction error may be calculated.

The second prediction error may represent a difference (residual) between the first residual block and the second prediction block. The second prediction error may represent a second residual block.

As described above, the prediction unit may generate a plurality of first prediction errors for a plurality of prediction modes. Further, for each of the plurality of first prediction errors, the prediction section may generate a plurality of second prediction errors for a plurality of candidate regions in the first residual block indicated by the first prediction error via the second prediction.

Through the above-described process, the prediction section may generate a plurality of second prediction errors for all candidate regions of the plurality of prediction modes.

The prediction section may select a second prediction error having a minimum rate distortion cost from among the generated plurality of second prediction errors. The candidate region selected from the plurality of candidate regions corresponding to the second prediction error having the smallest rate-distortion cost may be a portion of the current block to which the second prediction is to be applied. Further, the prediction mode corresponding to the second prediction error having the minimum rate distortion cost selected from among the plurality of prediction modes may be a prediction mode to be used in the first prediction. In other words, the prediction part may determine the prediction mode of the first prediction and the region of the second prediction to be used for encoding the current block according to the prediction mode and the region of the prediction mode where the second prediction error has the minimum rate distortion cost.

Alternatively, the prediction section may select a prediction mode having the smallest rate-distortion cost from among a plurality of prediction modes. In this case, the second prediction may be performed on the first prediction error or the first residual block generated in the selected prediction mode. The prediction unit may select a second prediction error having a minimum rate-distortion cost from among the plurality of second prediction errors. In other words, the prediction section may determine the prediction mode to be used for the first prediction for encoding the current block according to which one of the plurality of first prediction errors has the minimum rate-distortion cost, and may determine the region to be used for the second prediction for encoding the current block according to which one of the plurality of second prediction errors has the minimum rate-distortion cost.

The prediction-related information, which has been described above with reference to fig. 11, may include information about selective secondary intra prediction.

For example, the prediction-related information may include an ssip_flag, which is a flag indicating whether Selective Secondary Intra Prediction (SSIP) is used.

For example, the prediction related information may include information indicating the above α and information indicating the above β. In other words, the prediction related information may include information indicating a first reference sample to be used for the second prediction and information indicating the number of reference samples to be used for the second prediction. Alternatively, the prediction related information may include information indicating a first reference sample to be used for the second prediction and information indicating a last reference sample to be used for the second prediction.

Alternatively, the prediction related information may include information about a point at which the second prediction starts and a point at which the second prediction is to be performed. The prediction related information may include information indicating a start point of the second prediction and an end point of the second prediction.

For example, the information indicating α or the information indicating the start point of the second prediction may be ssip_start_point, and the information indicating β or the information indicating the point at which the second prediction is to be performed may be ssip_length.

By the above-described determination process, the prediction error in the current block can be further reduced as compared with the existing scheme in which only the first prediction is performed. For the trend that the distortion of the prediction error remains in the direction of the prediction mode, which has been described above with reference to fig. 9, the second prediction may effectively reduce the prediction error. In other words, by effectively utilizing the tendency of the distortion of the prediction error to remain in the direction of the prediction mode via the second prediction, the prediction performance can be improved and the prediction error can be reduced.

Fig. 13 is a flowchart of a residual block generation method according to an embodiment.

Step 1110 described above with reference to fig. 11 may include

steps

1310, 1320, 1330, 1340, 1350, 1360, 1370, and 1380.

In an embodiment, the block that is the target of the first prediction may be the current block. The first prediction may be intra prediction.

In step 1310, the prediction part may generate a first prediction signal by performing a first prediction on the current block.

The first prediction signal may represent a first prediction block.

As described above with reference to fig. 12, the prediction part may perform the first prediction in each of a plurality of prediction modes so as to find the best prediction mode for the first prediction to encode the current block. That is, steps 1310, 1320, and 1325 may be performed in each of a plurality of prediction modes.

The multiple prediction modes may correspond to different prediction directions.

In step 1320, the prediction part may generate a first prediction error based on the current block and the first prediction signal.

The first prediction error may represent a first prediction error block or a first residual block.

The first residual block may represent a difference (residual) between the current block and the first prediction block. That is, the prediction part may generate the first prediction error by subtracting the first prediction block represented by the first prediction signal from the current block.

In step 1325, the prediction unit may calculate a rate-distortion cost for the first prediction using the first prediction error.

The prediction portion may store the calculated first prediction error and/or the calculated rate-distortion cost for the first prediction in the memory 1030, the storage 1040, or the buffer.

As described above with reference to fig. 12, the prediction part may perform the first prediction in each of a plurality of prediction modes so as to find the best prediction mode of the first prediction to encode the current block. The prediction section may calculate a plurality of rate-distortion costs for the plurality of prediction modes using the first prediction errors in the plurality of prediction modes. The prediction section may store a minimum rate-distortion cost among the calculated plurality of rate-distortion costs as the minimum first rate-distortion cost rdcost_1. For example, rdcost_1 may be the rate distortion cost of the mode with the smallest rate distortion cost in the main intra prediction.

The prediction unit may store rdcost_1 and a first prediction error corresponding to rdcost_1.

The second prediction may be selectively performed based on a predetermined condition.

At step 1330, the predictor may determine whether to perform the second prediction.

In an embodiment, the determination at step 1330 may not be a final decision as to whether to encode the current block using the second prediction. In step 1130, the prediction part may determine whether the second prediction can be performed on the current block or the first residual block.

For example, if it is determined that the second prediction is not performed, the current block may be encoded without using the second prediction. If it is determined to perform the second prediction, a second prediction signal and a second prediction error may be generated via the second prediction, but it may be determined whether to encode the current block using the second prediction by comparing the rate-distortion costs at step 1360.

As described above with reference to fig. 12, the prediction part may determine whether to perform the second prediction after performing the first prediction in each of a plurality of prediction modes in order to find the best month mode for performing the first prediction of the encoding of the current block. In other words, step 1330 may be performed in each of a plurality of prediction modes.

If it is determined that the second prediction is not performed, the residual block for the current block may be a block represented by the first prediction error. In other words, the first prediction error may be provided as a residual block.

If it is determined to perform the second prediction, the residual block for the current block may be a block represented by a second prediction error signal, which will be described later. In other words, the second prediction error signal may be provided as a residual block.

The prediction part may determine whether to perform the second prediction based on a predefined condition.

In an embodiment, the prediction section may determine whether to perform the second prediction based on a prediction mode of the first prediction.

For example, when the prediction mode of the first prediction is a non-directional mode, the prediction part may not perform the second prediction. Alternatively, the prediction section may perform the second prediction when the prediction mode of the first prediction is the directional mode.

When the prediction mode of the first prediction is a non-directional mode among a plurality of prediction modes, since the direction of the prediction mode does not exist, it may not be possible to perform prediction within an optional region or range. Thus, the second prediction may be performed only when the prediction mode of the first prediction is the directional mode.

In an embodiment, the prediction part may determine whether to perform the second prediction based on the position of the current block.

For example, when the current block is adjacent to the boundary, the prediction part may not perform the second prediction. For example, when the current block is adjacent to the upper boundary and the left boundary, the prediction part may not perform the second prediction. The boundary may be a boundary of a picture, a boundary of a slice, or a boundary of a parallel block.

When the current block is adjacent to the upper and left boundaries, the first residual block for the second prediction may be insufficient. When the first residual block is insufficient, a second reference sample required to perform the second prediction may not be completely generated.

In an embodiment, the prediction part may determine whether to perform the second prediction based on the number of reconstructed blocks adjacent to the current block.

For example, when the number of reconstructed blocks adjacent to the current block is less than or equal to 2, the prediction part may not perform the second prediction.

If it is determined to perform the second prediction, step 1340 may be performed.

If it is determined that the second prediction is not performed, step 1370 or step 1120 may be performed. When step 1120 is performed, the residual signal at step 1120 may be a first prediction error. In other words, when only the first prediction is performed and the second prediction is not performed, the first prediction error generated via the first prediction may be used as a residual signal of the current block.

In step 1340, the prediction unit may generate a second prediction signal by performing a second prediction on the first prediction error.

The second prediction signal may represent a second prediction block.

The second prediction may be an intra prediction, and a prediction direction of the second prediction may be the same as a prediction direction of the first prediction.

The prediction part may perform the second prediction on a part of the region of the first residual block. The portion of the region may be specified by a residual sample to be used for the second prediction.

As described above with reference to fig. 12, the prediction part may perform the second prediction on each of the plurality of candidate regions so as to find the best region of the second prediction to encode the current block.

Further, as described above with reference to fig. 12, the prediction part may perform the second prediction on each candidate region in each prediction mode so as to find the best prediction mode of the first prediction and the best region of the second prediction to encode the current block. That is, step 1340 may be performed for each candidate region in each prediction mode.

Here, the best prediction mode of the first prediction and the best region of the second prediction may be prediction modes and regions corresponding to candidate regions having the smallest rate-distortion cost among the plurality of candidate regions in the plurality of prediction modes. In other words, the prediction part may determine the prediction mode of the first prediction and the region of the second prediction for encoding the current block based on a specific prediction mode and a specific region in which the minimum rate distortion cost occurs when the first prediction and the second prediction are performed among the plurality of candidate regions in the plurality of prediction modes.

The plurality of candidate regions may correspond to ranges of reference samples to be used for the second prediction, respectively.

The prediction section may select a range of reference samples to be used for the second prediction from among the reference samples in the prediction direction. The prediction section performs a second prediction using reference points falling within the selected range among all reference points in the prediction direction. In the second prediction, the value of the reference sample falling outside the selected range may be regarded as 0.

In step 1350, the prediction unit may generate a second prediction error based on the first prediction error and the second prediction signal.

The second prediction error may represent a second prediction error block or a second residual block.

The second residual block may represent a difference (residual) between the first residual block and the second prediction block. That is, the prediction section may generate the second prediction error signal by subtracting the second prediction block represented by the second prediction signal from the first residual block represented by the first prediction error.

In step 1355, the predictor may use the second prediction error to calculate a rate-distortion cost for the second prediction.

The predictor may store the calculated second prediction error and/or the calculated rate-distortion cost for the second prediction in memory 1030, storage 1040, or buffer.

As described above with reference to fig. 12, the prediction part may perform the second prediction on each of the plurality of regions so as to find the best region of the second prediction to encode the current block. The prediction section may calculate a plurality of rate distortion costs for the plurality of regions using a plurality of second prediction errors in the plurality of regions. The prediction section may store a minimum rate-distortion cost among the calculated plurality of rate-distortion costs as a minimum second rate-distortion cost rdcost_2. For example, rdcost_2 may be the rate distortion cost of the region with the smallest rate distortion cost in the selective secondary intra prediction.

The prediction part may store rdcost_2 and a second prediction error corresponding to rdcost_2.

The second prediction may be selectively used based on predefined conditions.

In step 1360, the prediction part may determine whether to encode the current block using the second prediction.

The prediction part may determine whether to encode the current block using the second prediction based on a predefined condition.

For example, when the rate distortion cost is further reduced by using the second prediction, the prediction section may determine to use the second prediction. When the rate distortion cost is not further reduced even though the second prediction is used, the prediction part may determine not to use the second prediction.

For example, the prediction part may determine to use the second prediction when the minimum rate-distortion cost calculated when the second prediction is used is smaller than the minimum rate-distortion cost calculated when the second prediction is not used.

In an embodiment, the prediction part may determine whether to encode the current block using the second prediction based on a comparison result between rdcost_1 and rdcost_2.

A rdcost_2 lower than rdcost_1 means that: when the second prediction is used together, the rate-distortion cost of encoding the current block may be further reduced compared to the case where only the first prediction is used.

For example, when rdcost_1 is less than or equal to rdcost_2, the prediction part may determine not to encode the current block using the second prediction. When rdcost_1 is greater than rdcost_2, the prediction part may determine to encode the current block using the second prediction.

If it is determined that the current block is not encoded using the second prediction, step 1370 may be performed.

If it is determined that the current block is encoded using the second prediction, step 1380 may be performed.

In step 1370, the prediction portion may perform a setting indicating that the second prediction is not to be used.

The prediction section may set a value of the second prediction use information such that the second prediction use information indicates that the second prediction is not to be used. For example, when the value of the second prediction use information is 0, the second prediction use information may indicate that the second prediction is not to be used.

When step 1370 is performed, the residual signal at step 1120 may be a first prediction error. In other words, when the second prediction is not performed on the current block, the first prediction error generated based on the first prediction may be used as a residual signal.

In step 1380, the predictor may perform a setting indicating that the second prediction is to be used.

The prediction section may set a value of the second prediction use information such that the second prediction use information indicates that the second prediction is to be used. For example, when the value of the second prediction use information is 1, the second prediction use information may indicate that the second prediction is to be used.

When step 1380 is performed, the residual signal at step 1120 may be a second prediction error. In other words, when the second prediction is performed on the current block, a second prediction error generated based on the first prediction and the second prediction may be used as the residual signal.

Fig. 14 is a flowchart of a reconstructed block generation method according to an embodiment.

Step 1160 described above with reference to fig. 11 may include the following

steps

1410, 1420, and 1430.

In step 1410, the reconstructed block generation unit may determine whether the second prediction has been used to encode the current block.

If it is determined that the second prediction has been used to encode the current block, step 1420 may be performed.

If it is determined that the second prediction is not used to encode the current block, step 1430 may be performed. When the second prediction is not used, the reconstructed first prediction error of the current block may be a reconstructed residual signal.

In step 1420, the reconstructed block generation unit may generate a reconstructed first prediction error based on the reconstructed residual signal and the second prediction signal.

The reconstructed block generation unit may generate a reconstructed first prediction error of the current block based on the residual signal of the current block and a second prediction signal for a unit of a region of the current block.

The reconstructed first prediction error may be a sum of the reconstructed residual signal and the second prediction signal. Alternatively, the reconstructed first prediction error may be generated based on the reconstructed residual signal and the second prediction signal.

In step 1430, the reconstructed block generation unit may generate the reconstructed block based on the reconstructed first prediction error and the first prediction signal.

The reconstructed block generation unit may generate a signal representing the reconstructed block based on the reconstructed first prediction error of the current block and the first prediction signal of the current block.

In other words, when the second prediction is used, the signal representing the reconstructed block may be the sum of the reconstructed residual signal, the second prediction signal and the first prediction signal. Alternatively, when the second prediction is used, a signal representing the reconstructed block may be generated based on the reconstructed residual signal, the second prediction signal, and the first prediction signal.

In other words, when the second prediction is not used, the signal representing the reconstructed block may be the sum of the reconstructed residual signal and the first prediction signal. Alternatively, when the second prediction is not used, a signal representing the reconstructed block may be generated based on the reconstructed residual signal and the first prediction signal.

The signal representing the reconstructed block may be a sum of the reconstructed first prediction error and the first prediction signal. Furthermore, a reconstructed first prediction error may be generated based on the reconstructed residual signal and the second prediction signal.

Fig. 15 shows an encoding process according to an embodiment.

In fig. 15, the symbol "Σ" with "+" and "-" attached may indicate that an error signal between two signals is generated. The error signal may be generated by subtracting the input signal received with the symbol "-" from the input signal received with the symbol "+".

The symbol "Σ" with "+" and "+" attached may indicate that a signal corresponding to the sum of two signals is generated. The signal being the sum of the two signals may be generated by adding the input signals received with the symbols "+", respectively.

The prediction part described above with reference to fig. 11 may include a first predictor and a second predictor.

The process within the dashed line may indicate SSIP.

The diamond may indicate a conditional branch.

In an equation such as "a=b," a "may represent a target to which a value is to be assigned, and" B "may represent a value to be assigned to the target.

"TQ" may represent transformation and quantization. (TQ) ^-1 Inverse transforms and inverse quantizations may be represented.

e1 may represent a signal corresponding to the first prediction error.

e2 may represent a signal corresponding to the second prediction error.

e may represent the residual signal.

Fig. 16 is a configuration diagram of a decoding apparatus according to an embodiment.

The decoding device 1600 may be a general-purpose computer system that performs decoding.

As shown in fig. 16, the decoding apparatus 1600 may include at least one processor 1610, a memory 1630, a User Interface (UI) input device 1650, a UI output device 1660, and a storage 1640 in communication with each other through a bus 1690. The decoding device 1600 may also include a communication unit 1620 connected to the network 1699. Processor 1610 may be a CPU or semiconductor device for executing processing instructions stored in memory 1630 or storage 1640. Each of memory 1630 and storage 1640 may be any of various types of volatile or non-volatile storage media. For example, the memory may include at least one of ROM 1631 and RAM 1632.

The processor 1610 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transformation unit 230, an intra prediction unit 240, an inter prediction unit 250, an adder 255, a filtering unit 260, and a reference picture buffer 270 of the decoding apparatus 200.

The storage medium may store at least one module used for operation of the decoding device 1600. The memory 1630 may be configured to store at least one module and allow the module to be executed by the at least one processor 1610.

According to an embodiment, at least some of the entropy decoding unit 210, the inverse quantization unit 220, the inverse transformation unit 230, the intra prediction unit 240, the inter prediction unit 250, the adder 255, the filtering unit 260, and the reference picture buffer 270 of the decoding apparatus 1600 may be program modules and may communicate with external devices or systems. Program modules may be included in the decoding device 1600 in the form of operating systems, application program modules, and other program modules.

Program modules may be physically stored in various types of well known storage devices. Furthermore, at least some of the program modules may also be stored in a remote storage device capable of communicating with the decoding apparatus 1600.

Program modules may be implemented using instructions or code executed by processor 1610.

The functions related to the communication of data or information of the decoding apparatus 1600 may be performed by the communication unit 1620.

Fig. 17 is a flowchart of a decoding method according to an embodiment.

The decoding method according to the embodiment may be performed by the decoding apparatus 200 or the decoding apparatus 1600.

In step 1710, the communication unit 1620 may receive a bit stream from the encoding apparatus 100 or the encoding apparatus 1100.

The bitstream may include information about the encoded residual signal, and may further include prediction related information.

The information about the encoded residual signal may comprise quantized transform coefficients that have been entropy encoded.

The prediction related information may be entropy decoded.

At step 1720, the entropy decoding unit 210 may generate quantized transform coefficients by performing entropy decoding on the bitstream. Further, the entropy decoding unit 210 may generate prediction related information by performing entropy decoding on the bitstream.

For example, the prediction related information may include prediction scheme information indicating a scheme used to encode the current block.

For example, the prediction scheme information may indicate which of intra prediction and inter prediction is used to encode the current block. Alternatively, the prediction scheme information may indicate whether the current block has been encoded via intra prediction. Alternatively, the prediction scheme information may indicate whether the current block has been encoded via inter prediction.

For example, the prediction related information may include intra prediction mode information indicating a prediction mode of intra prediction.

For example, the prediction related information may include second prediction use information indicating whether the second prediction is to be used for encoding the current block.

For example, the prediction related information may include current block type information indicating a type of a current block, first prediction block type information indicating a type of a first prediction block, second prediction block type information indicating a type of a second prediction block, first reference block type information indicating a type of a first reference block, second reference block type information indicating a type of a second reference block, first reference sample type information indicating a type of a first reference sample, and second reference sample type information indicating a type of a second reference sample.

For example, the prediction related information may include first prediction region information indicating a region of the first prediction and/or second prediction region information indicating a region of the second prediction.

Furthermore, the prediction related information may include information for prediction described in the embodiment. The pieces of prediction related information may be included in the bitstream according to the order described in the embodiment or the order generated in the embodiment.

In step 1730, the reconstructed residual signal generation unit may generate a reconstructed residual signal for the current block based on the quantized transform coefficients.

The reconstructed residual signal generation unit may include an inverse quantization unit 220 and an inverse transformation unit 230.

Step 1730 may include step 1731 and step 1732.

In step 1731, the inverse quantization unit 220 may generate inverse quantized transform coefficients by performing inverse quantization on the quantized transform coefficients.

In step 1732, the inverse transform unit 230 may generate a reconstructed residual signal by performing inverse transform on the inverse quantized transform coefficients.

In

steps

1710, 1720, and 1730, a reconstructed residual signal for the current block may be generated.

In step 1740, the reconstructed block generation unit may generate a reconstructed block based on the reconstructed residual signal, the second prediction, and the first prediction.

The reconstructed block generation unit may include an adder 225, an intra prediction unit 240, an inter prediction unit 250, a filter unit 260, and a reference picture buffer 270.

The reconstructed residual signal may be a reconstructed residual block.

For example, the reconstructed block may be a sum of a reconstructed residual block and a prediction block. Alternatively, the reconstructed block may be generated based on a sum of the reconstructed residual signal and the prediction block.

For example, the prediction block may be a sum of the first prediction block and the second prediction block. Alternatively, the prediction signal may be a sum of the first prediction signal and the second prediction signal.

For example, the reconstructed block may be a sum of the reconstructed block residual, the first prediction block, and the second prediction block. The reconstructed signal may be a sum of the reconstructed residual signal, the first prediction signal and the second prediction signal. Alternatively, the reconstructed block may correspond to a sum of the reconstructed residual block, the first prediction block, and the second prediction block.

The first prediction block may be a block generated via the first prediction or a block represented by a prediction signal generated by the first prediction. The second prediction block may be a block generated via the second prediction or a block represented by a prediction signal generated by the second prediction.

When the second prediction is not used, the second prediction signal may be a null signal and the second prediction block may be a null block. A null block may be a block where the values of all pixels are 0.

For example, when the second prediction is used to encode the current block, the reconstructed residual signal may be the reconstructed second prediction error, which has been described above with reference to fig. 14. Alternatively, when the second prediction is used to encode the current block, the reconstructed residual signal may be a signal obtained by reconstructing the second prediction error, which has been described above with reference to fig. 11, using inverse quantization and inverse transformation.

For example, when the second prediction is not used to encode the current block, the reconstructed residual signal may be the reconstructed first prediction error, which has been described above with reference to fig. 14. Alternatively, when the second prediction is not used to encode the current block, the reconstructed residual signal may be a signal obtained by reconstructing the first prediction error, which has been described above with reference to fig. 11, using inverse quantization and inverse transformation.

The reconstructed block generation unit may generate the reconstructed block by performing inter prediction or intra prediction.

For example, the block that is the target of the first prediction may be a reconstructed block. The block that is the target of the second prediction may be a prediction error block.

The position of at least one of the plurality of first reference blocks for the first prediction and the plurality of second reference blocks for the second prediction may be included only in one of the positions of the plurality of first reference blocks and the plurality of second reference blocks.

For example, when the target of the first prediction is a current block, a neighboring reconstructed block of the current block may be used as a reference block in order to perform prediction of the current block. When the second prediction is aimed at the first residual block, a neighboring reconstructed residual block of the current block or the first residual block may be used as the second reference block in order to perform prediction of the first residual block.

Considering the case where the region of the block as the target of the first prediction is the same as the region of the block as the target of the second prediction, the first predicted region may be the entire block as the target of the first prediction. The second predicted region may be a portion of a block that is a target of the second prediction. Alternatively, at least a portion of the block targeted for the second prediction may not be subjected to the second prediction. In other words, the first prediction may be performed on the entire block that is the target of the first prediction. The second prediction may be performed on a portion of the blocks that are the target of the second prediction.

For example, the prediction value determined via the first prediction may be allocated only to pixels falling within the region of the first prediction in the block that is the target of the first prediction. The prediction value determined via the first prediction may not be allocated to pixels falling outside the region of the first prediction in the block that is the target of the first prediction. Alternatively, the predefined value may be assigned to pixels falling outside the area of the first prediction in the block that is the target of the first prediction. For example, the predefined value may be 0.

For example, the prediction value determined via the second prediction may be allocated only to pixels falling within the region of the second prediction in the block that is the target of the second prediction. The prediction value determined via the second prediction may not be allocated to pixels falling outside the region of the second prediction in the block that is the target of the second prediction. Alternatively, the predefined value may be assigned to pixels falling outside the area of the second prediction in the block that is the target of the second prediction. The predefined value may be, for example, 0.

For example, when the size of a block that is the target of the first prediction is n×n, each prediction value determined via the first prediction may be allocated to n×n pixels. In other words, the prediction value determined via the first prediction may be allocated to all pixels in the block that is the target of the first prediction.

For example, when the size of the block that is the target of the second prediction is n×n, the prediction value determined via the first prediction may not be allocated to one or more of the n×n pixels. Alternatively, when the size of the block as the target of the second prediction is n×n, a predefined value may be allocated to one or more of the n×n pixels. For example, the predefined value may be 0.

As described above, the first predicted region and the second predicted region may be different from each other. The region of the first reference sample may be a region of the reference sample required to generate a prediction value for the region of the first prediction via the first prediction. The region of the second reference sample may be a region of the reference sample that is required to generate a prediction value for the region of the second prediction via the second prediction.

Fig. 18 is a flowchart of a reconstructed block generation method according to an embodiment.

Step 1740 described above with reference to fig. 17 may include the following

steps

1810, 1820, 1830, 1840 and 1850.

The second prediction may be selectively performed based on predefined conditions.

In step 1810, the reconstructed block generation unit may determine whether to generate the reconstructed block using the second prediction.

The reconstructed block generation unit may determine whether to generate the reconstructed block using the second prediction based on a predetermined condition.

In an embodiment, the reconstructed block generation unit may determine whether to use the second prediction based on a prediction mode of the first prediction.

For example, the reconstructed block generation unit may acquire intra prediction mode information indicating a prediction mode of intra prediction from the bitstream. When the prediction mode of the first prediction is a non-directional mode, the reconstructed block generation unit may not use the second prediction.

In an embodiment, the reconstructed block generation unit may determine whether to use the second prediction based on a position of the current block.

For example, the reconstructed block generation unit may not use the second prediction when the current block is adjacent to the boundary. For example, when the current block is adjacent to the upper boundary and the left boundary, the prediction part may not use the second prediction. The boundary may be a boundary of a picture, a boundary of a slice, or a boundary of a parallel block.

In an embodiment, the reconstructed block generation unit may determine whether to use the second prediction based on the number of reconstructed blocks adjacent to the current block. The reconstructed block may be a PU.

For example, when the number of reconstructed blocks adjacent to the current block is less than or equal to 2, the reconstructed block generation unit may not use the second prediction.

In an embodiment, after determining whether to use the second prediction using the prediction mode, the position of the current block, and/or the number of neighboring reconstructed blocks, the reconstructed block generation unit may determine whether to use the second prediction based on whether the second prediction has been used to encode the current block.

For example, the reconstructed block generation unit may acquire second prediction use information from the bitstream. The reconstructed block generation unit may use the second prediction when the second prediction use information indicates that the second prediction is to be used. The reconstructed block generation unit may not use the second prediction when the second prediction use information indicates that the second prediction is not to be used.

When the second prediction has been used to encode the current block, step 1820 may be performed.

Step 1840 may be performed when the second prediction is not used to encode the current block. When the second prediction is not used, the reconstructed first prediction error of the current block may be a reconstructed residual signal.

In step 1820, the reconstructed block generating unit may generate a second prediction signal by performing a second prediction.

The second prediction may correspond to the second prediction in the process of encoding the current block, which has been described above with reference to fig. 11 and 13. For example, the second prediction signal may correspond to the second prediction in the process of encoding the current block, which has been described above with reference to fig. 11 and 13.

The second prediction signal may represent a second prediction block.

The reference block in the second prediction may be a reconstructed residual block. For example, the reconstructed block generation unit may generate the second prediction signal by performing a second prediction using a reconstructed residual block for a block adjacent to the current block as a reference block.

In an embodiment, as described above, the second predicted region may be a portion of the region of the current block. The reconstructed block generation unit may acquire second prediction region information indicating a prediction region of the second prediction from the bitstream. The reconstructed block generation unit may perform a second prediction on the second prediction region.

In an embodiment, the region to which the second prediction is to be applied may be specified based on the range of reference samples to be used for the second prediction. The region to which the second prediction is to be applied may be specified based on the prediction direction of the second prediction. For example, the region to which the second prediction is to be applied may be specified based on the range of the reference sample to be used for the second prediction and the prediction direction of the second prediction.

In an embodiment, the reconstructed block generation unit may perform the second prediction using reference samples falling within the selected range among all reference pixels in the prediction direction. In the second prediction, the value of the reference sample falling outside the selected range may be regarded as 0.

In an embodiment, the reconstructed block generation unit may acquire information α indicating a first reference sample to be used for the second prediction and information β indicating the number of reference samples to be used for the second prediction from the bitstream. The range of reference samples to be used for the second prediction may be specified by α and β.

In an embodiment, the range of reference samples may be specified by information α indicating a first reference sample to be used for the second prediction and information β indicating the number of reference samples to be used for the second prediction. The second predicted region may be a region generated when the reference sample point falling within the range from α to α+β moves in the prediction direction.

In step 1830, the reconstructed block generation unit may generate a reconstructed first prediction error based on the reconstructed residual signal and the second prediction signal. The reconstructed block generation unit may generate the reconstructed first prediction error by adding the second prediction signal to the reconstructed residual signal.

The reconstructed block generation unit may generate a reconstructed first prediction error for the current block based on the reconstructed residual signal and the second prediction signal for a portion of the region.

The reconstructed first prediction error may be a sum of the reconstructed residual signal and the second prediction signal. Alternatively, the reconstructed first prediction error may be generated based on the reconstructed residual signal and the second prediction signal. Alternatively, the reconstructed first prediction error may be generated based on a sum of the reconstructed residual signal and the second prediction signal.

In step 1840, the reconstructed block generation unit may generate a first prediction signal by performing a first prediction on the current block.

The first prediction signal may represent a first prediction block.

In the first prediction, a reconstructed block adjacent to the current block may be used as a reference block. Furthermore, pixels in a reconstructed block adjacent to the current block may be used as reference samples.

In step 1850, the reconstructed block generation unit may generate a signal representing the reconstructed block based on the reconstructed first prediction error and the first prediction signal. The reconstructed block generation unit may generate a signal representing the reconstructed block by adding the first prediction signal and the reconstructed first prediction error.

According to

steps

1810, 1820, 1830, 1840 and 1850 described above, when the second prediction is used, the reconstructed block may be a sum of the reconstructed residual signal, the second prediction signal and the first prediction signal. Alternatively, when the second prediction is used, a signal representing the reconstructed block may be generated based on the reconstructed residual signal, the second prediction signal, and the first prediction signal.

According to

steps

1810, 1820, 1830, 1840 and 1850 described above, when the second prediction is not used, the signal representing the reconstructed block may be the sum of the reconstructed residual signal and the first prediction signal. Alternatively, when the second prediction is not used, a signal representing the reconstructed block may be generated based on the sum of the reconstructed residual signal and the first prediction signal.

The reconstructed block and the reconstructed first residual signal may be used as reference blocks for decoding other blocks.

Fig. 19 shows a decoding process according to an embodiment.

In fig. 19, the symbol "Σ" with "+" and "+" attached may indicate that a signal corresponding to the sum of two signals is generated. The signal being the sum of the two signals may be generated by adding the input signals received with the symbols "+", respectively.

The reconstructed block generation unit described above with reference to fig. 19 may include a first predictor and a second predictor.

The process within the dashed line may indicate SSIP.

The diamond may indicate a conditional branch.

(TQ) ^-1 Inverse transforms and inverse quantizations may be represented.

According to the embodiments described above, when the second prediction is used, the selected portion to which the first prediction is applied and the remaining portion to which the second prediction is applied may be separated.

In an embodiment, the first prediction may be applied to the entire current block when the second prediction is used. In this case, the reconstructed block for the selected portion to which the second prediction is applied may be generated based on a sum of the reconstructed reference signal, the second prediction signal generated via the second prediction, and the first prediction signal generated via the first prediction. Furthermore, a reconstructed block for the remaining portion to which the second prediction is not applied may be generated based on a sum of the reconstructed residual block and a first prediction signal generated via the first prediction.

In this case, the second prediction may be a prediction of the residual. The second prediction may be a prediction of a first residual signal, wherein the first residual signal is a difference between the current block and the first prediction signal. The reference block for the second prediction may be a reconstructed residual block and the reference pixels for the second prediction may be pixels in the reconstructed residual block. In other words, a second residual signal, which is a difference between the first residual signal and the second prediction signal, may be generated via a second prediction of the first residual signal, and encoding information of the current block may be generated using the second residual signal.

Alternatively, in this case, the reconstructed block for the selected portion to which the second prediction is applied may be a weighted sum of the reconstructed residual block, the second prediction signal and the first prediction signal. The prediction related information may include a first weight for the reconstructed residual signal, a second weight for the second prediction signal, and a third weight for the first prediction signal. It can be considered that: not using the second prediction means that the second weight for the second prediction signal is set to 0. Alternatively, it can be considered that: for the remaining portion to which the second prediction is not applied, the second weight for the second prediction is set to 0.

The above description of the encoding of the current block may also be applied to the decoding of the current block. Duplicate description is omitted. Furthermore, the above description of decoding of the current block may also be applied to encoding of the current block. Duplicate description will be omitted.

In the above-described embodiments, although the method has been described based on a flowchart as a series of steps or units, the present invention is not limited to the order of the steps, and some steps may be performed in a different order from the order of the steps that have been described or simultaneously with other steps. Furthermore, those skilled in the art will appreciate that: the steps shown in the flowcharts are not exclusive and may include other steps, or one or more steps in the flowcharts may be deleted without departing from the scope of the present invention.

The embodiments according to the present invention described above may be implemented as programs capable of being executed by various computer apparatuses, and may be recorded on a computer-readable storage medium. The computer readable storage medium may include program instructions, data files, and data structures, alone or in combination. The program instructions recorded on the storage medium may be specially designed or configured for the present invention, or may be known or available to those having ordinary skill in the computer software arts. Examples of computer storage media may include all types of hardware devices that are specially configured for recording and executing program instructions, such as magnetic media (such as hard disks, floppy disks, and magnetic tape), optical media (such as Compact Discs (CD) -ROMs, and Digital Versatile Discs (DVDs)), magneto-optical media (such as floppy disks, ROMs, RAMs, and flash memory). Examples of program instructions include both machine code, such as produced by a compiler, and high-level language code that can be executed by the computer using an interpreter. The hardware devices may be configured to operate as one or more software modules to perform the operations of the invention, and vice versa.

As described above, although the present invention has been described based on specific details (such as detailed components and a limited number of embodiments and drawings), the specific details are provided only for easy understanding of the present invention, and the present invention is not limited to these embodiments, and various changes and modifications will be practiced by those skilled in the art in light of the above description.

It is, therefore, to be understood that the spirit of the present embodiments is not to be limited to the above-described embodiments and that the appended claims and equivalents thereof, and modifications thereto, fall within the scope of the invention.

Claims

1. A video decoding method, comprising:

determining a current block in a current picture;

performing prediction with respect to the current block to generate a prediction block;

a reconstructed block is generated based on the prediction block.

2. The video decoding method of claim 1, wherein,

the predictions include a first prediction and a second prediction,

the region to which the predicted value generated by the first prediction is assigned is different from the region to which the predicted value generated by the second prediction is assigned,

the region of the first reference sample for the first prediction is different from the region of the second reference sample for the second prediction,

the second prediction is applied to a partial region of the current block.

3. The video decoding method of claim 2, wherein,

the size of the second block generated by the second prediction is 2×8.

4. The video decoding method of claim 1, wherein,

the predictions include a first prediction and a second prediction,

the first reference block for the first prediction and the second reference block for the second prediction are different from each other,

at least one of the first prediction and the second prediction is inter prediction,

the first region for the first prediction and the second region for the second prediction are different from each other,

the first region is a region to which a prediction value generated by the first prediction is allocated among regions of the current block,

the second region is a region to which a prediction value generated by the second prediction is allocated among regions of the current block,

the first prediction value determined through the first prediction is applied only to pixels in a first region among regions of the current block,

the first prediction value determined through the first prediction is not applied to pixels outside the first region among regions of the current block,

the second prediction value determined by the second prediction is applied only to pixels in a second region among regions of the current block, and

the second prediction value determined through the second prediction is not applied to pixels outside the second region among regions of the current block.

5. The video decoding method of claim 1, wherein,

the predictions include a first prediction and a second prediction,

the first prediction is a prediction for a luminance block,

the second prediction is a prediction for a chroma block,

the type of the first reference block used for the first prediction is different from the type of the second reference block used for the second prediction,

the first reference block is a reconstructed luma block,

the second reference block is a reconstructed chroma block,

generating a first prediction block by first prediction

A second prediction block is generated by a second prediction,

the type of the first predicted block is different from the type of the second predicted block, and

the second prediction block is generated based on the first prediction block.

6. The video decoding method of claim 1, wherein,

generating a plurality of prediction errors for a plurality of different candidate regions of the second prediction,

selecting a minimum prediction error from the plurality of prediction errors,

a region for a second prediction is determined based on a candidate region of the plurality of different candidate regions having the selected minimum prediction error.

7. The video decoding method of claim 1, wherein,

the predictions include a first prediction and a second prediction,

the first prediction is an inter-frame prediction,

The second prediction is an intra-frame prediction,

a first prediction block is generated by performing a first prediction,

a second prediction block is generated by performing a second prediction,

the reconstructed block is generated using a first weight for a first prediction block and a second weight for a second prediction block.

8. The video decoding method of claim 1, wherein,

the predictions include a first prediction and a second prediction,

the first prediction is an inter-frame prediction,

the second prediction is an inter-frame prediction,

the reconstructed block is generated using bi-directional inter prediction,

a first prediction block is generated by performing a first prediction,

a second prediction block is generated by performing a second prediction,

the reconstructed block is generated using weights for the first prediction block and the second prediction block,

weight information in the bitstream is used to indicate the weights.

9. A video encoding method, comprising:

determining a current block in a current picture;

performing prediction with respect to the current block to generate a prediction block; and

a reconstructed block is generated based on the prediction block.

10. The video encoding method of claim 9, wherein,

the predictions include a first prediction and a second prediction,

the second prediction is applied to a partial region of the current block.

11. The video encoding method of claim 10, wherein,

the size of the second block generated by the second prediction is 2×8.

12. The video encoding method of claim 10, wherein,

the predictions include a first prediction and a second prediction,

13. The video encoding method of claim 9, wherein,

the predictions include a first prediction and a second prediction,

the first prediction is a prediction for a luminance block,

the second prediction is a prediction for a chroma block,

the first reference block is a reconstructed luma block,

the second reference block is a reconstructed chroma block,

a first prediction block is generated by a first prediction,

a second prediction block is generated by a second prediction,

the second prediction block is generated based on the first prediction block.

14. The video encoding method of claim 9, wherein,

selecting a minimum prediction error from the plurality of prediction errors,

15. The video encoding method of claim 9, wherein,

the predictions include a first prediction and a second prediction,

the first prediction is an inter-frame prediction,

the second prediction is an intra-frame prediction,

a first prediction block is generated by performing a first prediction,

a second prediction block is generated by performing a second prediction,

16. The video encoding method of claim 9, wherein,

the predictions include a first prediction and a second prediction,

the first prediction is an inter-frame prediction,

the second prediction is an inter-frame prediction,

the reconstructed block is generated using bi-directional inter prediction,

a first prediction block is generated by performing a first prediction,

a second prediction block is generated by performing a second prediction,

weight information indicating the weights is included in the bitstream.

17. A computer-readable recording medium storing a bitstream generated by the video encoding method of claim 9.

18. A computer-readable recording medium storing a bitstream comprising computer-executable code, wherein the computer-executable code, when executed, causes a video decoding apparatus to perform the steps of:

Determining a current block in a current picture;

decoding prediction mode information in the computer executable code;

determining a prediction mode based on the prediction mode information;

performing prediction for a current block based on the prediction mode to generate a prediction block; and

a reconstructed block is generated based on the prediction block.

19. The computer-readable recording medium of claim 18, wherein,

the predictions include a first prediction and a second prediction,

the second prediction is applied to a partial region of the current block.

20. The computer-readable recording medium of claim 19, wherein,

the size of the second block generated by the second prediction is 2×8.

21. The computer-readable recording medium of claim 18, wherein,

the predictions include a first prediction and a second prediction,

22. The computer-readable recording medium of claim 18, wherein,

the predictions include a first prediction and a second prediction,

the first prediction is a prediction for a luminance block,

the second prediction is a prediction for a chroma block,

The first reference block is a reconstructed luma block,

the second reference block is a reconstructed chroma block,

generating a first prediction block by first prediction

A second prediction block is generated by a second prediction,

the second prediction block is generated based on the first prediction block.

23. The computer-readable recording medium of claim 18, wherein,

selecting a minimum prediction error from the plurality of prediction errors,

24. The computer-readable recording medium of claim 18, wherein,

the predictions include a first prediction and a second prediction,

the first prediction is an inter-frame prediction,

the second prediction is an intra-frame prediction,

a first prediction block is generated by performing a first prediction,

a second prediction block is generated by performing a second prediction,

25. The computer-readable recording medium of claim 18, wherein,

The predictions include a first prediction and a second prediction,

the first prediction is an inter-frame prediction,

the second prediction is an inter-frame prediction,

the reconstructed block is generated using bi-directional inter prediction,

a first prediction block is generated by performing a first prediction,

a second prediction block is generated by performing a second prediction,

weight information in the bitstream is used to indicate the weights.

26. A computer-readable recording medium storing a bitstream, the bitstream comprising:

prediction mode information, wherein,

determining a current block in a current picture;

a prediction mode is determined based on the prediction mode information,

performing prediction for the current block based on the prediction mode to generate a prediction block,

a reconstructed block is generated based on the prediction block.

27. A video decoding device, comprising:

a prediction unit for performing prediction with respect to a current block to generate a prediction block; and

a reconstructed block generation unit for generating a reconstructed block based on the prediction block, wherein,

the current block is a specific region in the current picture.

28. A video encoding apparatus, comprising:

the current block is a specific region in the current picture.

29. A computer-readable recording medium storing a bit stream generated by the video encoding apparatus of claim 28.

30. A computer-readable recording medium storing a bitstream including computer-executable code,

wherein the computer executable code, when executed, causes a video decoding apparatus to perform the steps of:

determining a current block in a current picture;

decoding prediction mode information in the computer executable code;

determining a prediction mode for prediction of the current block based on the prediction mode information;

generating a prediction block by performing the prediction; and

a reconstructed block is generated based on the prediction block.

31. A computer-readable recording medium storing a bitstream, the bitstream comprising:

prediction mode information, wherein,

a current block in the current picture is determined,

a prediction mode for prediction of the current block is determined based on the prediction mode information,

generating a prediction block by performing the prediction, and

a reconstructed block is generated based on the prediction block.