WO2017131473A1 - Method and device for encoding and decoding video by using prediction - Google Patents

Abstract

A method and a device for performing encoding and decoding by using first prediction and second prediction are disclosed. In the encoding of a current block, a residual signal of the current block is generated on the basis of the current block, the first prediction, and the second prediction. In addition, information on the residual signal encoded by encoding the residual signal is generated. In the decoding of the current block, a restored residual signal of the current block is generated. A restored block of the current block is generated on the basis of the restored residual signal, the second prediction, and the first prediction.

Description

Method and apparatus for encoding and decoding video using prediction

The following embodiments relate to a video decoding method, a decoding device, an encoding method, and an encoding device, and more particularly, to a method and an apparatus for encoding and decoding a video using prediction on a target block.

Through the continuous development of the information and telecommunications industry, broadcasting services having high definition (HD) resolution have spread worldwide. This proliferation has resulted in many users becoming accustomed to high resolution, high quality images and / or video.

In order to satisfy users' demand for high image quality, many organizations are spurring the development of next generation video devices. Users are interested in Ultra High Definition (UHD) TVs, which have four times the resolution of FHD TVs, as well as High Definition TV (HDTV) and Full HD (FHD) TVs. This has increased, and as the interest increases, an image encoding / decoding technique for an image having higher resolution and image quality is required.

An image encoding / decoding apparatus and method include an inter prediction technique, an intra prediction technique, an entropy encoding technique, etc. in order to perform encoding / decoding of high resolution and high quality images. Can be used. The inter prediction technique may be a technique for predicting a value of a pixel included in a current picture using a temporally previous picture and / or temporally following picture. An intra prediction technique may be a technique of predicting a value of a pixel included in a current picture by using information of a pixel in a current picture. The entropy encoding technique may be a technique of allocating a short code to a symbol having a high appearance frequency and a long code to a symbol having a low appearance frequency.

In encoding and decoding of an image, prediction may mean generating a prediction signal similar to the original signal. Predictions can be broadly classified into predictions referring to spatially reconstructed images, predictions referring to temporal reconstructed images, and predictions for other symbols. In other words, a temporal reference may refer to a temporally reconstructed image, and a spatial reference may refer to a spatially reconstructed image.

The current block may be a block that is currently subjected to encoding or decoding. The current block may be named a target block or a target unit. In encoding, the current block may be called an encoding target block or an encoding target unit. In decoding, the current block may be called a decoding target block or a decoding target unit.

Inter prediction may be a technique for predicting the current block using temporal and spatial references. Intra prediction may be a technique for predicting the current block using only spatial references.

The image encoding / decoding technique encodes the current block using one prediction mode among a plurality of prediction modes when performing intra prediction to reduce spatial repetition. For example, High Efficiency Video Coding (HEVC) technology uses 35 intra prediction modes.

The encoding apparatus generates prediction blocks of the current block using all available prediction modes among the plurality of prediction modes, and selects one prediction modes that produce the best result as a prediction mode of encoding of the current block. However, even using the prediction mode that produces the best results, there is still an error between the original block and the prediction block. This error is represented as a residual block.

For some of the pixels, there is a large error between the original block and the prediction block. This large error may not result in sufficient spatial iteration reduction even after transform and / or quantization is applied to the residual block.

One embodiment may provide a method and apparatus for reducing a prediction error of a prediction unit generated by the prediction.

One embodiment may provide a method and apparatus for reducing a prediction error caused by a prediction direction of a prediction mode.

An embodiment may provide a method and apparatus for performing a second prediction selectively in a direction of a first prediction mode with respect to a prediction error according to the first prediction.

An embodiment may provide a method and apparatus for performing a selective second prediction on a part of a prediction error according to a first prediction.

An embodiment may provide a method and apparatus for performing a selective second prediction on a portion with a severe distortion among prediction errors according to the first prediction.

One embodiment provides a method and apparatus for preventing an additional prediction error that may occur when performing a second prediction on the entirety of the prediction error by performing a second prediction selectively on a part of the prediction error according to the first prediction. Can provide.

One embodiment can provide a method and apparatus for obtaining sufficient spatial iteration reduction through the progress of transform quantization by reducing large prediction errors generated in the prediction direction.

The method of claim 1, further comprising: generating a residual signal of the current block based on the current block, the first prediction, and the second prediction; And generating information on the encoded residual signal by performing encoding on the residual signal, wherein the second prediction is a prediction for a prediction error of the first prediction.

On the other side, the restored residual signal generator for generating a restored residual signal for the current block; And a reconstructed block generator that generates a reconstructed block for the current block based on the reconstructed residual signal, the second prediction, and the first prediction.

In another aspect, generating a reconstructed residual signal for a current block; And generating a reconstructed block for the current block based on the reconstructed residual signal, the second prediction, and the first prediction.

The reconstructed block may correspond to the sum of the reconstructed residual signal, the first prediction signal generated by the first prediction, and the second prediction signal generated by the second prediction.

The first prediction and the second prediction may be intra prediction.

The prediction direction of the second prediction may be the same as the first prediction direction.

At least some of the plurality of first reference blocks of the first prediction and the plurality of second reference blocks of the second prediction may be different from each other.

The type of the first reference block of the first prediction and the type of the second reference block of the second prediction may be different from each other.

The first reference block of the first prediction may be a reconstructed neighboring block neighboring the current block.

The second reference block of the second prediction may be a reconstructed neighbor residual block neighboring the current block.

The reconstructed neighboring residual block may be a difference between the reconstructed neighboring block and the first prediction block of the reconstructed neighboring block.

The area of the reference sample of the second prediction may be part of the area of the reference sample of the first prediction.

The region to which the second prediction is to be applied may be specified based on a range of reference samples used for the second prediction.

An area to which the second prediction is to be applied may be specified based on a prediction direction of the second prediction.

And the range of reference samples is specified by information indicating a first reference sample to be used for the second prediction and information indicating the number of reference samples to be used for the second prediction.

The region to which the second prediction is to be applied may be a region generated as the reference samples of the range proceed in the prediction direction of the second prediction.

Values of reference samples outside this range may be considered null or zero.

When the second prediction usage information indicating whether the second prediction is used in encoding the current block indicates that the second prediction is not used, the second prediction may not be used.

If the prediction mode of the first prediction is a non-directional mode, the second prediction may not be used.

If the current block is adjacent to an upper boundary and a left boundary, the second prediction may not be used.

The boundary may be a boundary of a picture, a boundary of a slice, or a boundary of a tile.

Whether the second prediction is to be used may be determined based on the number of reconstructed blocks adjacent to the current block.

When the second prediction is not used, the reconstructed block may be generated based on the reconstructed residual signal and the first prediction signal by the first prediction.

A method and apparatus are provided for reducing a prediction error of a prediction unit generated by prediction.

A method and apparatus are provided for reducing a prediction error caused by a prediction direction of a prediction mode.

A method and apparatus are provided for performing selective second prediction in a direction of a first prediction mode with respect to a prediction error according to the first prediction.

A method and apparatus are provided for performing a selective second prediction on some of the prediction errors according to the first prediction.

Provided are a method and an apparatus for performing selective second prediction on a portion of a distortion that is severe among prediction errors according to a first prediction.

A method and apparatus are provided for preventing additional prediction errors that may occur when performing a second prediction on the entirety of the prediction error by performing a second prediction selectively for only a part of the prediction errors according to the first prediction.

A method and apparatus are provided for obtaining sufficient spatial iteration reduction through the progress of transform quantization by reducing large prediction errors generated in the prediction direction.

1 is a block diagram illustrating a configuration of an encoding apparatus according to an embodiment of the present invention.

2 is a block diagram illustrating a configuration of a decoding apparatus according to an embodiment of the present invention.

3 is a diagram schematically illustrating a division structure of an image when encoding and decoding an image.

4 is a diagram illustrating a form of a prediction unit PU that a coding unit CU may include.

FIG. 5 is a diagram illustrating a form of a transform unit (TU) that may be included in a coding unit (CU).

6 is a diagram for explaining an embodiment of an intra prediction process.

7 is a diagram for describing a position of a reference sample used in an intra prediction process.

8 is a diagram for explaining an embodiment of an inter prediction process.

9 illustrates a prediction error when the original image is predicted in the vertical direction according to an example.

10 is a structural diagram of an encoding apparatus according to an embodiment.

11 is a flowchart of an encoding method according to an embodiment.

12 illustrates first prediction and second prediction using intra prediction, according to an embodiment.

13 is a flowchart of a method of generating a residual block, according to an exemplary embodiment.

14 is a flowchart of a method of generating a restored block, according to an example.

15 illustrates an encoding process according to an embodiment.

16 is a structural diagram of a decoding apparatus according to an embodiment.

17 is a flowchart of a decoding method according to an embodiment.

18 is a flowchart of a method of generating a restored block, according to an exemplary embodiment.

19 illustrates a decoding process according to an embodiment.

DETAILED DESCRIPTION For the following detailed description of exemplary embodiments, reference is made to the accompanying drawings that show, by way of illustration, specific embodiments. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that the various embodiments are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein may be embodied in other embodiments without departing from the spirit and scope of the invention with respect to one embodiment. In addition, it is to be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the embodiments. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the exemplary embodiments, if properly described, is defined only by the appended claims, along with the full scope of equivalents to which such claims are entitled.

Like reference numerals in the drawings refer to the same or similar functions throughout the several aspects. Shape and size of the elements in the drawings may be exaggerated for clarity.

When a component is said to be "connected" or "connected" to another component, the above two components may be directly connected to or connected to each other, It is to be understood that other components may exist in the middle of the two components. In addition, the description "including" a specific configuration in the exemplary embodiments does not exclude a configuration other than the specific configuration described above, the additional configuration is the implementation of the exemplary embodiments or the technical spirit of the exemplary embodiments. It can be included in the range.

Terms such as first and second may be used to describe various components, but the above components should not be limited by the above terms. The above terms are used to distinguish one component from another component. For example, without departing from the scope of rights, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

In addition, the components shown in the embodiments are shown independently to represent different characteristic functions, and do not mean that each component is composed of only separate hardware or one software component unit. That is, each component is listed as each component for convenience of description. For example, at least two of the components may be combined into one component. In addition, one component may be divided into a plurality of components. The integrated and separated embodiments of each of these components are also included in the scope of the present invention without departing from the essence.

In addition, some of the components may not be essential components for performing essential functions, but may be optional components for improving performance. Embodiments may be implemented including only components necessary to implement the nature of the embodiments, and structures including the optional components, such as, for example, components used only for performance improvement, are also included in the scope of rights.

Hereinafter, in order to enable those skilled in the art to easily implement the embodiments, embodiments will be described in detail with reference to the accompanying drawings. In describing the embodiments, when it is determined that the detailed description of the related well-known configuration or function may obscure the subject matter of the present specification, the detailed description thereof will be omitted.

In the following description, an image may mean one picture constituting a video and may represent a video itself. For example, "encoding and / or decoding of an image" may mean "encoding and / or decoding of a video" and may mean "encoding and / or decoding of one of images constituting the video." It may be.

Hereinafter, "video" and "motion picture" may be used interchangeably and may be used interchangeably.

Hereinafter, "image", "picture", "frame" and "screen" may be used in the same sense, and may be used interchangeably.

In embodiments, each of the specified information, data, flags and elements, attributes, etc. may have a value. The value "0" of information, data, flags and elements, attributes, etc. may represent a logical false or first predefined value. In other words, the value "0", logic false and the first predefined value can be used interchangeably. The value "1" of information, data, flags and elements, attributes, etc. may represent logical true or second predefined values. In other words, the value "1", the logical true and the second predefined value can be used interchangeably.

A variable such as i or j may be used to indicate a row, column or index, and the value of i may be an integer of 0 or more and may be an integer of 1 or more. In other words, in embodiments, rows, columns, indexes, etc. may be counted from zero and counted from one.

In the following, terms used in the embodiments are described.

Unit: A “unit” may represent a unit of encoding and decoding of an image. The meanings of the unit and the block may be the same. In addition, the terms "unit" and "block" may be used interchangeably.

The unit (or block) may be an M × N array of samples. M and N may each be a positive integer. A unit can often mean an array of two-dimensional samples. The sample may be a pixel or pixel value.

-The terms "pixel" and "sample" can be used interchangeably and can be used interchangeably.

In encoding and decoding of an image, a unit may be an area generated by division of one image. One image may be divided into a plurality of units. In the encoding and decoding of the image, a predefined process for the unit may be performed according to the type of the unit. Depending on the function, the type of unit may be classified into a macro unit, a coding unit (CU), a prediction unit (PU), a transform unit (TU), and the like. One unit may be further divided into subunits having a smaller size than the unit.

The unit division information may include information about the depth of the unit. The depth information may indicate the number and / or degree of division of the unit.

One unit may be divided into a plurality of sub-units hierarchically with depth information based on a tree structure. In other words, the unit and the lower unit generated by the division of the unit may correspond to the node and the child node of the node, respectively. Each divided subunit may have depth information. Since the depth information of the unit indicates the number and / or degree of division of the unit, the division information of the lower unit may include information about the size of the lower unit.

In the tree structure, the highest node may correspond to the first unit that is not split. The highest node may be referred to as a root node. In addition, the highest node may have a minimum depth value. At this time, the highest node may have a depth of level 0.

A node with a depth of level 1 may represent a unit created as the first unit is divided once. A node with a depth of level 2 may represent a unit created as the first unit is split twice.

A node with a depth of level n may represent a unit generated as the first unit is divided n times.

The leaf node may be the lowest node or may be a node that cannot be further divided. The depth of the leaf node may be at the maximum level. For example, the predefined value of the maximum level may be three.

Transform Unit: A transform unit may be a basic unit in residual signal coding and / or residual signal decoding such as transform, inverse transform, quantization, inverse quantization, transform coefficient encoding, and transform coefficient decoding. . One transform unit may be divided into a plurality of transform units having a smaller size.

Prediction Unit A prediction unit may be a basic unit in performing prediction or compensation. The prediction unit can be a number of partitions by partitioning. Multiple partitions may also be the basic unit in performing prediction or compensation. The partition generated by the partitioning of the prediction unit may also be the prediction unit.

Reconstructed Neighbor Unit: The reconstructed neighbor unit may be a unit that has already been encoded or decoded around the encoding target unit or the decoding target unit. The reconstructed neighbor unit may be a spatial neighbor unit or a temporal neighbor unit to the target unit.

Prediction unit partition: A prediction unit partition may mean a form in which a prediction unit is divided.

Parameter Set: A parameter set may correspond to header information among structures in the bitstream. For example, the parameter set may include a sequence parameter set, a picture parameter set, an adaptation parameter set, and the like.

Rate-distortion optimization: The encoding apparatus uses a combination of the size of the coding unit, the prediction mode, the size of the prediction unit, the motion information, and the size of the transform unit to provide high coding efficiency. Distortion optimization can be used.

The rate-distortion optimization method can calculate the rate-distortion cost of each combination in order to select the optimal combination among the above combinations. The rate-distortion cost may be calculated using Equation 1 below. In general, a combination in which the rate-distortion cost is minimized may be selected as an optimal combination in the rate-distortion optimization scheme.

D may represent distortion. D may be the mean square error of the squares of difference values between the original transform coefficients and the reconstructed transform coefficients in the transform block.

R can represent the rate. R may indicate a bit rate using the associated context information.

λ may represent a Lagrangian multiplier. R may include not only encoding parameter information such as a prediction mode, motion information, and a coded block flag, but also bits generated by encoding of transform coefficients.

The encoding apparatus performs processes such as inter prediction and / or intra prediction, transformation, quantization, entropy encoding, inverse quantization, and inverse transformation to calculate accurate D and R, which can greatly increase the complexity in the encoding apparatus. have.

Reference picture: The reference picture may be an image used for inter prediction or motion compensation. The reference picture may be a picture including a reference unit referenced by the target unit for inter prediction or motion compensation. The meanings of the picture and the image may be the same. In addition, the terms "picture" and "image" may be used interchangeably.

Reference picture list: The reference picture list may be a list including reference pictures used for inter prediction or motion compensation. The type of the reference picture list may be List Combined (LC), List 0 (List 0; L0), List 1 (List 1; L1), and the like.

Motion Vector (MV): The motion vector may be a two-dimensional vector used in inter prediction. For example, MV may be expressed in the form of (mv _x , mv _y ). mv _x may represent a horizontal component and mv _y may represent a vertical component.

The MV may indicate an offset between the target picture and the reference picture.

Search range: The search range may be a two-dimensional area in which a search for MV is performed during inter prediction. For example, the size of the search region may be M × N. M and N may each be a positive integer.

1 is a block diagram illustrating a configuration of an encoding apparatus according to an embodiment of the present invention.

The encoding apparatus 100 may be a video encoding apparatus or an image encoding apparatus. The video may include one or more images. The encoding apparatus 110 may sequentially encode one or more images of the video over time.

Referring to FIG. 1, the encoding apparatus 100 may include an inter predictor 110, an intra predictor 120, a switch 115, a subtractor 125, a transformer 130, a quantizer 140, and entropy decoding. The unit 150 may include an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190.

The encoding apparatus 100 may encode the input image in an intra mode and / or an inter mode. The input image may be referred to as a current image that is a target of current encoding.

In addition, the encoding apparatus 100 may generate a bitstream including encoding information by encoding the input image, and may output the generated bitstream.

When the intra mode is used, the switch 115 can be switched to intra. When the inter mode is used, the switch 115 can be switched to inter.

The encoding apparatus 100 may generate a prediction block for the input block of the input image. In addition, after the prediction block is generated, the encoding apparatus 100 may encode a residual between the input block and the prediction block. The input block may be referred to as a current block that is a target of current encoding.

When the prediction mode is the intra mode, the intra prediction unit 120 may use a pixel value of an already encoded block in the vicinity of the current block as a reference pixel. The intra predictor 120 may perform spatial prediction on the current block by using the reference pixel, and generate prediction samples on the current block through spatial prediction.

The inter predictor 110 may include a motion predictor and a motion compensator.

When the prediction mode is the inter mode, the motion predictor may search an area that best matches the current block from the reference image in the motion prediction process, and may derive a motion vector for the current block and the searched area. The reference picture may be stored in the reference picture buffer 190 and may be stored in the reference picture buffer 190 when encoding and / or decoding of the reference picture is processed.

The motion compensator may generate a prediction block by performing motion compensation using a motion vector. Here, the motion vector may be a two-dimensional vector used for inter prediction. In addition, the motion vector may indicate an offset between the current picture and the reference picture.

The subtractor 125 may generate a residual block that is a difference between the input block and the prediction block. The residual block may be referred to as a residual signal.

The transform unit 130 may generate transform coefficients by performing transform on the residual block, and output the generated transform coefficients. Here, the transform coefficient may be a coefficient value generated by performing transform on the residual block. When the transform skip mode is applied, the transform unit 130 may omit the transform on the residual block.

Quantized transform coefficient levels may be generated by applying quantization to the transform coefficients. In the following embodiments, the quantized transform coefficient level may also be referred to as transform coefficient.

The quantization unit 140 may generate a quantized transform coefficient level by quantizing the transform coefficients according to the quantization parameter. The quantization unit 140 may output the generated quantized transform coefficient level. In this case, the quantization unit 140 may quantize the transform coefficients using the quantization matrix.

The entropy decoder 150 may generate a bitstream by performing entropy encoding according to a probability distribution based on the values calculated by the quantizer 140 and / or encoding parameter values calculated in the encoding process. . The entropy decoder 150 may output the generated bitstream.

The entropy decoder 150 may perform entropy encoding on information for decoding an image in addition to information about pixels of an image. For example, the information for decoding the image may include a syntax element.

The encoding parameter may be information required for encoding and / or decoding. The encoding parameter may include information encoded by the encoding apparatus and transmitted to the decoding apparatus, and may include information that may be inferred in the encoding or decoding process. For example, there is a syntax element as information transmitted to the decoding apparatus.

For example, coding parameters include prediction modes, motion vectors, reference picture indexes, coding block patterns, residual signals, transform coefficients, quantized transform coefficients, quantization parameters, block sizes, block partitions. It may include values or statistics such as information. The prediction mode may indicate an intra prediction mode or an inter prediction mode.

The residual signal may mean a difference between the original signal and the prediction signal. Alternatively, the residual signal may be a signal generated by transforming a difference between the original signal and the prediction signal. Alternatively, the residual signal may be a signal generated by transforming and quantizing the difference between the original signal and the prediction signal. The residual block may be a residual signal in block units.

When entropy coding is applied, a small number of bits may be allocated to a symbol having a high occurrence probability, and a large number of bits may be allocated to a symbol having a low occurrence probability. As the symbol is represented through this assignment, the size of the bitstring for the symbols to be encoded may be reduced. Therefore, compression performance of image encoding may be improved through entropy encoding.

In addition, for entropy coding, such as exponential golomb, context-adaptive variable length coding (CAVLC) and context-adaptive binary arithmetic coding (CABAC), etc. An encoding method can be used. For example, the entropy decoder 150 may perform entropy encoding by using a variable length coding (VLC) table. For example, the entropy decoder 150 may derive a binarization method for the target symbol. In addition, the entropy decoder 150 may derive a probability model of the target symbol / bin. The entropy decoder 150 may perform entropy encoding using the derived binarization method or the probability model.

Since encoding is performed through inter prediction by the encoding apparatus 100, the encoded current image may be used as a reference image with respect to other image (s) to be processed later. Therefore, the encoding apparatus 100 may decode the encoded current image again and store the decoded image as a reference image. Inverse quantization and inverse transform on the encoded current image may be processed for decoding.

The quantized coefficients may be inversely quantized in the inverse quantization unit 160 and inversely transformed in the inverse transformer 170. The inverse quantized and inverse transformed coefficients may be summed with the prediction block via the adder 175. A reconstructed block may be generated by adding the inverse quantized and inverse transformed coefficients and the prediction block.

The restored block may pass through the filter unit 180. The filter unit 180 may apply at least one or more of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the reconstructed block or the reconstructed picture. have. The filter unit 180 may be referred to as an adaptive in-loop filter.

The deblocking filter may remove block distortion generated at boundaries between blocks. The SAO may add an appropriate offset value to the pixel value to compensate for coding errors. The ALF may perform filtering based on a value obtained by comparing the reconstructed image and the original image. The reconstructed block that has passed through the filter unit 180 may be stored in the reference picture buffer 190.

2 is a block diagram illustrating a configuration of a decoding apparatus according to an embodiment of the present invention.

The decoding apparatus 200 may be a video decoding apparatus or an image decoding apparatus.

Referring to FIG. 2, the decoding apparatus 200 may include an entropy decoder 210, an inverse quantizer 220, an inverse transformer 230, an intra predictor 240, an inter predictor 250, and an adder 255. The filter unit 260 may include a reference picture buffer 270.

The decoding apparatus 200 may receive a bitstream output from the encoding apparatus 100. The decoding apparatus 200 may perform intra mode and / or inter mode decoding on the bitstream. In addition, the decoding apparatus 200 may generate a reconstructed image by decoding, and output the generated reconstructed image.

For example, switching to the intra mode or the inter mode according to the prediction mode used for decoding may be made by a switch. When the prediction mode used for decoding is an intra mode, the switch may be switched to intra. When the prediction mode used for decoding is an inter mode, the switch may be switched to inter.

The decoding apparatus 200 may obtain a reconstructed residual block from the input bitstream, and generate a prediction block. When the reconstructed residual block and the prediction block are obtained, the decoding apparatus 200 may generate the reconstructed block by adding the reconstructed residual block and the prediction block.

The entropy decoder 210 may generate symbols by performing entropy decoding on the bitstream based on the probability distribution. The generated symbols may include symbols in the form of quantized coefficients. Here, the entropy decoding method may be similar to the entropy encoding method described above. For example, the entropy decoding method may be an inverse process of the above-described entropy encoding method.

The quantized coefficient may be inverse quantized by the inverse quantization unit 220. In addition, the inverse quantized coefficient may be inversely transformed by the inverse transformer 230. As a result of inverse quantization and inverse transformation of the quantized coefficients, a reconstructed residual block may be generated. In this case, the inverse quantization unit 220 may apply a quantization matrix to the quantized coefficients.

When the intra mode is used, the intra predictor 240 may generate a predictive block by performing spatial prediction using pixel values of blocks already decoded around the current block.

The inter predictor 250 may include a motion compensator. When the inter mode is used, the motion compensator may generate a prediction block by performing motion compensation using a motion vector and a reference image. The reference picture may be stored in the reference picture buffer 270.

The reconstructed residual block and the prediction block may be added through the adder 255. The adder 255 may generate the reconstructed block by adding the reconstructed residual block and the predictive block.

The restored block may pass through the filter unit 260. The filter unit 260 may apply at least one or more of the deblocking filter, SAO, and ALF to the reconstructed block or the reconstructed picture. The filter unit 260 may output the restored image. The reconstructed picture may be stored in the reference picture buffer 270 and used for inter prediction.

3 is a diagram schematically illustrating a division structure of an image when encoding and decoding an image.

In order to efficiently divide an image, a coding unit (CU) may be used in encoding and decoding. A unit may be a term that collectively refers to 1) a block including image samples and 2) a syntax element. For example, "division of a unit" may mean "division of a block corresponding to a unit".

Referring to FIG. 3, the image 300 may be sequentially divided in units of a largest coding unit (LCU), and the division structure of the image 300 may be determined according to the LCU. Here, the LCU may be used as the same meaning as a coding tree unit (CTU).

The partition structure may mean a distribution of a coding unit (CU) for efficiently encoding an image in the LCU 310. This distribution may be determined according to whether to divide one CU into four CUs. The horizontal size and the vertical size of the CU generated by the split may be half of the horizontal size and half of the vertical size, respectively, before the split. The partitioned CU may be recursively divided into four CUs whose width and length are reduced by half in the same manner.

At this time, partitioning of the CU may be performed recursively up to a predetermined depth. Depth information may be information indicating the size of a CU. Depth information may be stored for each CU. For example, the depth of the LCU may be zero, and the depth of the smallest coding unit (SCU) may be a predefined maximum depth. Here, the LCU may be a CU having a maximum coding unit size as described above, and the SCU may be a CU having a minimum coding unit size.

The division may start from the LCU 310, and the depth of the CU may increase by one whenever the horizontal and vertical sizes of the CU are reduced by half by the division. For each depth, the CU that is not divided may have a size of 2N × 2N. In addition, in the case of a partitioned CU, a CU of 2N × 2N size may be divided into four CUs having an N × N size. The size of N can be reduced by half for every 1 increase in depth.

Referring to FIG. 3, an LCU having a depth of 0 may be 64 × 64 pixels. 0 may be the minimum depth. An SCU of depth 3 may be 8x8 pixels. 3 may be the maximum depth. At this time, a CU of 64x64 pixels, which is an LCU, may be represented by a depth of zero. A CU of 32x32 pixels may be represented by depth one. A CU of 16 × 16 pixels may be represented by depth two. A CU of 8x8 pixels, which is an SCU, may be represented by depth three.

In addition, information on whether the CU is split may be expressed through split information of the CU. The split information may be 1 bit of information. All CUs except the SCU may include partition information. For example, when the CU is not split, the value of partition information of the CU may be 0, and when the CU is split, the value of partition information of the CU may be 1.

4 is a diagram illustrating a form of a prediction unit PU that a coding unit CU may include.

A CU that is no longer split among CUs split from the LCU may be split into one or more prediction units (PUs). This partition may also be called a partition.

The PU may be a basic unit for prediction. The PU may be encoded and decoded in any one of a skip mode, an inter mode, and an intra mode. PU may be divided into various types according to each mode.

In skip mode, there may be no partition in the CU. In the skip mode, 2N × 2N mode 410 having the same size of PU and CU without splitting may be supported.

In the inter mode, eight divided forms in a CU may be supported. For example, in the inter mode, 2Nx2N mode 410, 2NxN mode 415, Nx2N mode 420, NxN mode 425, 2NxnU mode 430, 2NxnD mode 435, nLx2N mode 440, and nRx2N Mode 445 may be supported.

In intra mode, 2Nx2N mode 410 and NxN mode 425 may be supported.

In the 2Nx2N mode 410, a PU having a size of 2Nx2N may be encoded. A PU having a size of 2N × 2N may mean a PU having a size equal to the size of a CU. For example, a PU having a size of 2N × 2N may have a size of 64 × 64, 32 × 32, 16 × 16, or 8 × 8.

In the NxN mode 425, a PU having a size of NxN may be encoded.

For example, in intra prediction, when the size of the PU is 8x8, four divided PUs may be encoded. The size of the partitioned PU may be 4 × 4.

When the PU is encoded by the intra mode, the PU may be encoded using one intra prediction mode among the plurality of intra prediction modes. For example, High Efficiency Video Coding (HEVC) technology can provide 35 intra prediction modes, and the PU can be coded in one of the 35 intra prediction modes.

Which of the 2Nx2N mode 410 and NxN mode 425 is to be coded may be determined by the rate-distortion cost.

The encoding apparatus 100 may perform an encoding operation on a PU having a size of 2N × 2N. Here, the encoding operation may be to encode the PU in each of a plurality of intra prediction modes that the encoding apparatus 100 may use. Through the coding operation, an optimal intra prediction mode for a 2N × 2N size PU may be derived. The optimal intra prediction mode may be an intra prediction mode that generates a minimum rate-distortion cost for encoding a 2N × 2N size PU among a plurality of intra prediction modes that can be used by the encoding apparatus 100.

In addition, the encoding apparatus 100 may sequentially perform encoding operations on each PU of the PUs divided by N × N. Here, the encoding operation may be to encode the PU in each of a plurality of intra prediction modes that the encoding apparatus 100 may use. Through the coding operation, an optimal intra prediction mode for a N × N size PU may be derived. The optimal intra prediction mode may be an intra prediction mode that generates a minimum rate-distortion cost for encoding of a PU of an N × N size among a plurality of intra prediction modes that can be used by the encoding apparatus 100.

The encoding apparatus 100 may determine which of 2Nx2N size PU and NxN size PU to encode based on a comparison of the rate-distortion cost of the 2Nx2N size PU and the rate-distortion costs of the NxN size PUs.

FIG. 5 is a diagram illustrating a form of a transform unit (TU) that may be included in a coding unit (CU).

A transform unit (TU) may be a basic unit used for a process of transform, quantization, inverse transform, inverse quantization, entropy encoding, and entropy decoding in a CU. The TU may have a square shape or a rectangular shape.

Of the CUs partitioned from the LCU, a CU that is no longer split into CUs may be split into one or more TUs. In this case, the partition structure of the TU may be a quad-tree structure. For example, as shown in FIG. 5, one CU 510 may be divided one or more times according to the quad-tree structure. Through division, one CU 510 may be configured with TUs of various sizes.

6 is a diagram for explaining an embodiment of an intra prediction process.

Arrows outward from the center of the graph of FIG. 6 may indicate prediction directions of intra prediction modes. In addition, the number displayed near the arrow may represent an example of a mode value allocated to the intra prediction mode or the prediction direction of the intra prediction mode.

Intra encoding and / or decoding may be performed using reference samples of units around the target unit. Peripheral units may be peripheral restored units. For example, intra encoding and / or decoding may be performed using a value or encoding parameter of a reference sample included in a neighboring reconstructed unit.

The encoding apparatus 100 and / or the decoding apparatus 200 may generate the prediction block by performing intra prediction on the target unit based on the information of the sample in the current picture. When performing intra prediction, the encoding apparatus 100 and / or the decoding apparatus 200 may generate a prediction block for a target unit by performing intra prediction based on information of a sample in a current picture. When performing intra prediction, the encoding apparatus 100 and / or the decoding apparatus 200 may perform directional prediction and / or non-directional prediction based on at least one reconstructed reference sample.

The prediction block may mean a block generated as a result of performing intra prediction. The prediction block may correspond to at least one of a CU, a PU, and a TU.

The unit of a prediction block may be the size of at least one of a CU, a PU, and a TU. The prediction block may have a square shape, having a size of 2N × 2N or a size of N × N. The size of NxN may include 4x4, 8x8, 16x16, 32x32 and 64x64.

Alternatively, the prediction block may be a block in the form of a square having a size of 2x2, 4x4, 16x16, 32x32, or 64x64, or a rectangular block having a size of 2x8, 4x8, 2x16, 4x16, and 8x16.

Intra prediction may be performed according to an intra prediction mode for the target unit. The number of intra prediction modes that the target unit may have may be a predefined fixed value or may be a value determined differently according to the attributes of the prediction block. For example, the attributes of the prediction block may include the size of the prediction block and the type of the prediction block.

For example, the number of intra prediction modes may be fixed to 35 regardless of the size of the prediction unit. Or, for example, the number of intra prediction modes may be 3, 5, 9, 17, 34, 35, 36, or the like.

The intra prediction mode may include two non-directional modes and 33 directional modes as shown in FIG. 6. Two non-directional modes may include a DC mode and a planar mode.

For example, in the vertical mode having a mode value of 26, prediction may be performed in the vertical direction based on the pixel value of the reference sample. For example, in the horizontal mode having a mode value of 10, prediction may be performed in the horizontal direction based on the pixel value of the reference sample. For example, in the vertical mode having a mode value of 26, prediction may be performed in the vertical direction based on the pixel value of the reference sample.

Even in the directional mode other than the above-described mode, the encoding apparatus 100 and the decoding apparatus 200 may perform intra prediction on the target unit using the reference sample according to the angle corresponding to the directional mode.

The intra prediction mode located on the right side of the vertical mode may be referred to as a vertical right mode. The intra prediction mode located at the bottom of the horizontal mode may be referred to as a horizontal-below mode. For example, in FIG. 6, intra prediction modes in which the mode value is one of 27, 28, 29, 30, 31, 32, 33, and 34 may be vertical right modes 613. Intra prediction modes with a mode value of one of 2, 3, 4, 5, 6, 7, 8, and 9 may be horizontal bottom modes 616.

The non-directional mode may include a DC mode and a planar mode. For example, the mode value of the DC mode may be 1. The mode value of the planner mode may be zero.

The directional mode may include an angular mode. Among the plurality of intra prediction modes, a mode other than the DC mode and the planner mode may be a directional mode.

In the DC mode, a prediction block may be generated based on an average of pixel values of the plurality of reference samples. For example, the value of a pixel of the prediction block may be determined based on an average of pixel values of the plurality of reference samples.

The number of intra prediction modes described above and the mode value of each intra prediction modes may be exemplary only. The number of intra prediction modes described above and the mode value of each intra prediction modes may be defined differently according to an embodiment, implementation, and / or need.

The number of intra prediction modes may differ depending on the type of color component. For example, the number of prediction modes may vary depending on whether the color component is a luma signal or a chroma signal.

7 is a diagram for describing a position of a reference sample used in an intra prediction process.

7 shows the location of a reference sample used for intra prediction of a target unit. Referring to FIG. 7, a reconstructed reference pixel used for intra prediction of a current block includes, for example, lower-left reference samples 731 and left reference samples 733. , Upper-left corner reference sample 735, upper-reference samples 737, upper-right reference samples 739, and the like.

For example, the left reference samples 733 may refer to a reconstructed reference pixel adjacent to the left side of the target unit. The top reference samples 737 may refer to a reconstructed reference pixel adjacent to the top of the target unit. The upper left corner reference pixel 735 may mean a restored reference pixel located at the upper left corner of the target unit. Also, the lower left reference samples 731 may refer to a reference sample located at the bottom of the left sample line among samples positioned on the same line as the left sample line composed of the left reference samples 733. The upper right reference samples 739 may refer to reference samples positioned to the right of the upper pixel line among samples positioned on the same line as the upper sample line formed of the upper reference samples 737.

When the size of the target unit is N × N, the lower left reference samples 731, the left reference samples 733, the upper reference samples 737, and the upper right reference samples 739 may each be N pieces.

The prediction block may be generated through intra prediction on the target unit. Generation of the predictive block may include determining a value of pixels of the predictive block. The size of the target unit and the prediction block may be the same.

The reference sample used for intra prediction of the target unit may vary according to the intra prediction mode of the target unit. The direction of the intra prediction mode may indicate a dependency relationship between the reference samples and the pixels of the prediction block. For example, the value of the specified reference sample can be used as the value of the specified one or more pixels of the prediction block. In this case, the specified one or more specified pixels of the specified reference sample and prediction block may be samples and pixels designated by a straight line in the direction of the intra prediction mode. In other words, the value of the specified reference sample may be copied to the value of the pixel located in the reverse direction of the intra prediction mode. Alternatively, the pixel value of the prediction block may be a value of a reference sample located in the direction of the intra prediction mode based on the position of the pixel.

For example, when the intra prediction mode of the target unit is a vertical mode having a mode value of 26, the upper reference samples 737 may be used for intra prediction. When the intra prediction mode is the vertical mode, the value of the pixel of the prediction block may be the value of the reference pixel located vertically above the position of the pixel. Thus, the top reference samples 737 adjacent to the top of the target unit can be used for intra prediction. Also, the values of the pixels of one row of the prediction block may be the same as the values of the top reference samples 737.

For example, when the intra prediction mode of the current block is a horizontal mode with a mode value of 10, the left reference samples 733 may be used for intra prediction. When the intra prediction mode is the horizontal mode, the pixel value of the prediction block may be a value of a reference pixel located horizontally on the left side with respect to the pixel. Thus, left reference samples 733 adjacent to the target unit to the left may be used for intra prediction. In addition, the values of the pixels of one column of the prediction block may be the same as the values of the left reference samples 733.

For example, if the mode value of the intra prediction mode of the current block is 18, at least some of the left reference samples 733, the upper left corner reference sample 735, and the at least some intra prediction of the top reference samples 737 are included. Can be used. When the mode value of the intra prediction mode is 18, the value of the pixel of the prediction block may be the value of the reference pixel located at the top left diagonally with respect to the pixel.

In addition, when an intra prediction mode having a mode value of 27, 28, 29, 30, 31, 32, 33, or 34 is used, at least some of the upper right reference pixels 439 may be used for intra prediction.

In addition, when an intra prediction mode having a mode value of 2, 3, 4, 5, 6, 7, 8, or 9 is used, at least some of the lower left reference pixels 431 may be used for intra prediction.

In addition, when an intra prediction mode having a mode value of 11 to 25 is used, the upper left corner reference samples 735 may be used for intra prediction.

The reference sample used to determine the pixel value of one pixel of the prediction block may be one, or may be two or more.

As described above, the pixel value of the pixel of the prediction block may be determined according to the position of the reference sample indicated by the position of the pixel and the direction of the intra prediction mode. If the position of the reference sample indicated by the position of the pixel and the direction of the intra prediction mode is an integer position, the value of one reference sample indicated by the integer position may be used to determine the pixel value of the pixel of the prediction block.

If the position of the reference sample indicated by the position of the pixel and the direction of the intra prediction mode is not an integer position, an interpolated reference sample may be generated based on the two reference samples closest to the position of the reference sample. have. The value of the interpolated reference sample can be used to determine the pixel value of the pixel of the prediction block. In other words, when the position of the reference sample indicated by the position of the pixel of the prediction block and the direction of the intra prediction mode indicates between the two reference samples, an interpolated value is generated based on the values of the two samples. Can be.

The prediction block generated by the prediction may not be the same as the original target unit. In other words, a prediction error that is a difference between the target unit and the prediction block may exist, and the prediction error may exist between the pixel of the target unit and the pixel of the prediction block. For example, in the case of directional intra prediction, the greater the distance between the pixel and the reference sample of the prediction block, the larger prediction error may occur. Discontinuity may occur between the prediction block and the neighboring block generated by such a prediction error.

Filtering on the prediction block may be used to reduce the prediction error. The filtering may be to adaptively apply a filter to a region that is considered to have a large prediction error in the prediction block. For example, an area considered to have a large prediction error may be a boundary of a prediction block. In addition, according to the intra prediction mode, an area considered to have a large prediction error among the prediction blocks may be different, and characteristics of the filter may be different.

8 is a diagram for explaining an embodiment of an inter prediction process.

The rectangle illustrated in FIG. 8 may represent an image (or picture). In addition, arrows in FIG. 8 may indicate prediction directions. That is, the image may be encoded and / or decoded according to the prediction direction.

Each picture (or picture) may be classified into an I picture (Intra Picture), a P picture (Uni-prediction Picture), and a B picture (Bi-prediction Picture) according to an encoding type. Each picture may be encoded according to an encoding type of each picture.

When the image to be encoded is an I picture, the image may be encoded with respect to the image itself without inter prediction. When the image to be encoded is a P picture, the image may be encoded through inter prediction using a reference picture only in the forward direction. When the image to be encoded is a B picture, it may be encoded through inter prediction using reference pictures in both the forward and reverse directions, and may be encoded through inter prediction using the reference picture in one of the forward and reverse directions.

The P picture and the B picture encoded and / or decoded using the reference picture may be regarded as an image using inter prediction.

In the following, inter prediction in inter mode according to an embodiment is described in detail.

In the inter mode, the encoding apparatus 100 and the decoding apparatus 200 may perform prediction and / or motion compensation on the encoding target unit and the decoding target unit. For example, the encoding apparatus 100 or the decoding apparatus 200 may perform prediction and / or motion compensation by using the reconstructed motion information of the neighboring unit as the motion information of the encoding target unit or the decoding target unit. Here, the encoding target unit or the decoding target unit may mean a prediction unit and / or a prediction unit partition.

Inter prediction may be performed using a reference picture and motion information. In addition, inter prediction may use the skip mode described above.

The reference picture may be at least one of a previous picture of the current picture or a subsequent picture of the current picture. In this case, the inter prediction may perform prediction on a block of the current picture based on the reference picture. Here, the reference picture may mean an image used for prediction of a block.

In this case, an area in the reference picture may be specified by using a reference picture index refIdx indicating a reference picture, a motion vector to be described later, and the like.

The inter prediction may select a reference picture and a reference block corresponding to the current block within the reference picture, and generate the prediction block for the current block using the selected reference block. The current block may be a block that is a target of current encoding or decoding among blocks of the current picture.

The motion information may be derived during inter prediction by each of the encoding apparatus 100 and the decoding apparatus 200. In addition, the derived motion information may be used to perform inter prediction.

In this case, the encoding apparatus 100 and the decoding apparatus 200 may use encoding information and / or decoding efficiency by using motion information of a restored neighboring block and / or motion information of a collocated block (col block). Can improve. The call block may be a block corresponding to the current block in a collocated picture (col picture).

The reconstructed neighboring block may be a block in the current picture and may be a block already reconstructed through encoding and / or decoding. The reconstructed block may be a neighboring block adjacent to the current block and / or a block located at an outer corner of the current block. Here, the block located at the outer corner of the current block may be a block vertically adjacent to a neighboring block horizontally adjacent to the current block or a block horizontally adjacent to a neighboring block vertically adjacent to the current block.

For example, a restored peripheral unit may be a unit located to the left of the target unit, a unit located at the top of the target unit, a unit located at the lower left corner of the target unit, a unit located at the upper right corner of the target unit, or an upper left of the target unit. It may be a unit located at the corner.

Each of the encoding apparatus 100 and the decoding apparatus 200 may determine a block existing at a position corresponding to a current block spatially in the call picture, and may determine a predetermined relative position based on the determined block. The predefined relative position may be a position inside and / or outside of a block that exists spatially at a position corresponding to the current block. In addition, each of the encoding apparatus 100 and the decoding apparatus 200 may derive a call block based on the determined predetermined relative position. Here, the call picture may be one picture among at least one reference picture included in the reference picture list.

The block in the reference picture may exist at a position spatially corresponding to the position of the current block in the reconstructed reference picture. In other words, the position of the current block in the current picture and the position of the block in the reference picture may correspond to each other. Hereinafter, motion information of a block included in the reference picture may be referred to as temporal motion information.

The method of deriving the motion information may vary according to the prediction mode of the current block. For example, as a prediction mode applied for inter prediction, there may be an advanced motion vector predictor (AMVP) and merge.

For example, when AMVP is applied as the prediction mode, each of the encoding apparatus 100 and the decoding apparatus 200 may predict the motion vector candidate using the motion vector of the reconstructed neighboring block and / or the motion vector of the call block. You can create a list. The motion vector of the reconstructed neighboring block and / or the motion vector of the collocated block may be used as a prediction motion vector candidate.

The bitstream generated by the encoding apparatus 100 may include a predicted motion vector index. The prediction motion vector index may indicate an optimal prediction motion vector selected from the prediction motion vector candidates included in the prediction motion vector candidate list. The predicted motion vector index may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through the bitstream.

The decoding apparatus 200 may select the prediction motion vector of the current block from the prediction motion vector candidates included in the prediction motion vector candidate list by using the prediction motion vector index.

The encoding apparatus 100 may calculate a motion vector difference (MVD) between the motion vector and the predictive motion vector of the current block, and may encode the MVD. The bitstream may include encoded MVD. The MVD may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. At this time, the decoding apparatus 200 may decode the received MVD. The decoding apparatus 200 may derive the motion vector of the current block through the sum of the decoded MVD and the predictive motion vector.

The bitstream may include a reference picture index and the like indicating the reference picture. The reference picture index may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. The decoding apparatus 200 may predict the motion vector of the current block by using the motion information of the neighboring block, and may derive the motion vector of the current block by using the predicted motion vector and the motion vector difference. The decoding apparatus 200 may generate a prediction block for the current block based on the derived motion vector and the reference picture index information.

Since the motion information of the neighboring unit reconstructed with respect to the encoding target unit and the decoding target unit may be used, in a specific inter prediction mode, the encoding apparatus 100 may not separately encode the motion information for the target unit. If the motion information of the target unit is not encoded, the amount of bits transmitted to the decoding apparatus 200 may be reduced, and encoding efficiency may be improved. For example, the inter prediction mode in which the motion information of the target unit is not encoded may include a skip mode and / or a merge mode. In this case, the encoding apparatus 100 and the decoding apparatus 200 may use an identifier and / or an index indicating which unit of the reconstructed neighboring units is used as the movement information of the target unit.

Another example of a method of deriving motion information is merge. Merge may mean merging of motions for a plurality of blocks. Merge may mean applying motion information of one block to other blocks. When the merge is applied, each of the encoding apparatus 100 and the decoding apparatus 200 may generate a merge candidate list using the motion information of the reconstructed neighboring block and / or the motion information of the call block. have. The motion information may include at least one of 1) a motion vector, 2) an index for a reference image, and 3) a prediction direction. The prediction direction may be unidirectional or bidirectional.

In this case, the merge may be applied in a CU unit or a PU unit. When merging is performed in a CU unit or a PU unit, the encoding apparatus 100 may transmit predefined information to the decoding apparatus 200 through a bitstream. The bitstream may include predefined information. The predefined information may include 1) information indicating whether to merge for each block partition, and 2) information about which one of neighboring blocks adjacent to the current block to merge with. For example, the neighboring blocks of the current block may include a left neighboring block of the current block, a top neighboring block of the current block, a temporal neighboring block of the current block, and the like.

The merge candidate list may represent a list in which motion information is stored. In addition, the merge candidate list may be generated before the merge is performed. The motion information stored in the merge candidate list may be 1) motion information of a neighboring block adjacent to the current block or 2) collocated block motion information corresponding to the current block in the reference image. In addition, the motion information stored in the merge candidate list may be new motion information generated by a combination of motion information already present in the merge candidate list.

The skip mode may be a mode in which information of neighboring blocks is applied to the current block as it is. The skip mode may be one of modes used for inter prediction. When the skip mode is used, the encoding apparatus 100 may transmit only information on which block motion information to use as the motion information of the current block to the decoding apparatus 200 through the bitstream. The encoding apparatus 100 may not transmit other information to the decoding apparatus 200. For example, the other information may be syntax information. The syntax information may include motion vector difference information.

9 illustrates a prediction error when the original image is predicted in the vertical direction according to an example.

As described above, when encoding is performed on the original image with intra prediction, one PU and a prediction mode may be selected after the calculation for all possible PUs and prediction modes is performed. When the prediction on the original image is performed according to the selected PU and the prediction mode, a tendency for distortion of the prediction error remains in the direction of the prediction mode.

Therefore, after performing the first first intra prediction using the selected prediction mode of one of the plurality of prediction modes for the PU, the second second intra prediction may be performed. Here, the secondary intra prediction may be intra prediction defined in the region of pixels specified in the direction of the prediction mode of the first intra prediction. In other words, the prediction mode of the secondary intra prediction may be the same as the prediction mode of the first intra prediction. On the other hand, the first intra prediction is applied to the whole of the PU, but the secondary intra prediction may be applied to only a part of the PU.

The encoding apparatus may determine a part of the PU to which secondary intra prediction is to be applied. For example, the encoding apparatus may calculate a rate-distortion cost when secondary intra prediction is applied to the candidate region for each candidate region of the plurality of candidate regions. The encoding apparatus may determine the candidate region of the smallest rate-distortion cost as part of the PU to which secondary intra prediction is to be applied.

The PU may include a plurality of pixels. The plurality of candidate regions may be portions determined by a specified manner of the plurality of pixels of the PU. For example, the plurality of candidate regions may be determined by the prediction direction of the first intra prediction.

By applying the first intra prediction and the second intra prediction, the prediction error for the PU can be further reduced compared to the case where only the first intra prediction is applied.

The following embodiments can present a method and apparatus for improving the performance of intra prediction and reducing the prediction error by efficiently using the aforementioned tendency.

10 is a structural diagram of an encoding apparatus according to an embodiment.

The encoding apparatus 1000 may be a general-purpose computer system that performs encoding.

As shown in FIG. 10, the encoding apparatus 1000 may include at least one processor 1010, a memory 1030, a user interface (UI) input device 1050, which communicates with each other via a bus 1090, and the like. UI output device 1060 and storage 1040. In addition, the encoding apparatus 1000 may further include a communication unit 1020 connected to the network 1099. The processor 1010 may be a semiconductor device that executes processing instructions stored in a central processing unit (CPU), a memory 1030, or a storage 1040. Memory 1030 and storage 1040 may be various types of volatile or nonvolatile storage media. For example, the memory may include at least one of a ROM 1031 and a RAM 1032.

The processor 1010 is an inter predictor 110, an intra predictor 120, a switch 115, a subtractor 125, a transformer 130, a quantizer 140, and an entropy decoder of the encoding apparatus 100. 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190.

The recording medium may store at least one module for the operation of the encoding apparatus 1000. The memory 1030 may store at least one module and may be configured to be executed by the at least one processor 1010.

According to an embodiment, the inter prediction unit 110, the intra prediction unit 120, the switch 115, the subtractor 125, the transform unit 130, the quantization unit 140, and the entropy decoding of the encoding apparatus 100 may be used. At least some of the unit 150, the inverse quantizer 160, the inverse transform unit 170, the adder 175, the filter unit 180, and the reference picture buffer 190 may be program modules, and may be an external device or system. Communicate with The program modules may be included in the encoding apparatus 1000 in the form of an operating system, an application module, and other program modules.

The program modules may be physically stored on various known storage devices. In addition, at least some of these program modules may be stored in a remote storage device that can communicate with the encoding apparatus 1000.

Program modules perform routines or subroutines, programs, objects, components, and data to perform functions or operations, or to implement abstract data types, according to one embodiment. Data structures and the like, but is not limited thereto.

The program modules may be composed of instructions or code performed by the processor 1010.

Functions related to communication of data or information of the encoding apparatus 1000 may be performed through the communication unit 1020.

11 is a flowchart of an encoding method according to an embodiment.

The encoding method of the embodiment may be performed by the encoding apparatus 100 or the encoding apparatus 1000.

In operation 1110, the prediction unit may generate a residual signal of the current block based on the current block, the first prediction, and the second prediction.

The predictor may include an inter predictor 110, an intra predictor 120, and a subtractor 125.

For example, the residual signal may represent a residual block.

For example, the residual block may be the difference between the current block and the prediction block. Alternatively, the residual block may be generated based on the difference between the current block and the prediction block.

For example, the prediction block may be the sum of the first prediction block and the second prediction block. The residual block may be obtained by subtracting the sum of the first prediction block and the second prediction block from the current block.

For example, the second prediction may be a prediction for a prediction error of the first prediction.

The first prediction block may be a block generated by the first prediction. The second prediction block may be a block generated by the second prediction. If the second prediction is not used, the second prediction block may be a null block. The null block may be a block in which the values of all pixels of the block are "0".

The prediction unit may generate a residual block of the current block by performing inter prediction or intra prediction.

In an embodiment, the prediction unit may generate a second prediction block based on the current block and / or the first prediction block.

In one embodiment, the prediction unit may generate a residual signal of the current block by performing a first prediction on the current block and performing a second prediction on the first prediction error that is a result of the first prediction. Here, the first prediction error may be a difference between the current block and the first prediction signal. The residual signal may be a difference between the first prediction error and the second prediction signal.

In one embodiment, the first prediction and the second prediction may be different kinds of predictions.

In one embodiment, the first prediction and the second prediction may be the same kind of predictions. For example, each of the first prediction and the second prediction may be intra prediction. If the first prediction is intra prediction, the second prediction may also be set to intra prediction.

In one embodiment, the prediction direction of the first prediction and the prediction direction of the second prediction may be the same. The prediction direction of the second prediction may be set to be the same as the prediction direction of the first prediction.

For example, in one embodiment, each of the first prediction and the second prediction may be inter prediction.

In one embodiment, a block that is the target of the first prediction and a block that is the target of the second prediction may be different from each other. Here, the "block to be predicted" may mean to which block the "prediction" generates a prediction block.

For example, the type of the block targeted for the first prediction and the type of the block targeted for the second prediction may be different from each other. The type of block may be an original block, a luma block, a chroma block, a depth block, a residual block, or the like.

For example, the block that is the target of the first prediction may be the current block or the original block.

For example, the first prediction error generated by the first prediction may represent the first prediction error block. The first prediction error may be a first residual signal. Also, the first residual signal may represent a first residual block. The block that is the target of the second prediction may be a first prediction error block or a first residual block. In this case, the first residual signal may be generated by the first prediction, and the second prediction error may be generated by the second prediction. The second prediction error may be a second residual signal. The second residual signal may represent a second residual block.

Reference blocks may be used for prediction. In one embodiment, the first reference block of the first prediction and the second reference block of the second prediction may be different from each other. The reference blocks may differ from each other in terms of type and / or location.

In one embodiment, the position of the first reference block of the first prediction and the position of the second reference block of the second prediction may be different from each other. Here, the position of the first reference block may be a position relative to the block that is the target of the first prediction. The position of the second reference block may be a position relative to the block that is the target of the second prediction.

In one embodiment, the first reference block of the first prediction may be plural. The second reference block of the second prediction may be plural. At least some of the plurality of first reference blocks of the first prediction and the plurality of second reference blocks of the second prediction may be different from each other.

The position of at least one of the plurality of first reference blocks of the first prediction and the plurality of second reference blocks of the second prediction is the positions of the plurality of first reference blocks and the positions of the plurality of second reference blocks. Can only be included in one of the

In one embodiment, the type of the first reference block of the first prediction and the type of the second reference block of the second prediction may be different from each other. The type of block may be a reconstructed block, a reconstructed luma block, a reconstructed chroma block, a reconstructed depth block, a reconstructed first residual block, a reconstructed second residual block, or the like.

For example, the first reference block of the first prediction may be a reconstructed block. The second reference block of the second prediction may be a reconstructed first residual block. The first reference sample of the first prediction may be a pixel of the reconstructed block. The second reference sample of the second prediction may be a pixel of the reconstructed first residual block.

For example, the first reference block of the first prediction may be a reconstructed neighboring block neighboring the current block. The second reference block of the second prediction may be a reconstructed first residual block neighboring the current block.

The reconstructed neighbor residual block neighboring the current block may be obtained by adding the reconstructed residual block of the reconstructed neighboring block to the second prediction block of the reconstructed neighboring block. Alternatively, the reconstructed neighbor residual block neighboring the current block may be a difference between the reconstructed neighboring block and the first prediction block of the reconstructed neighboring block.

For example, when the object of the first prediction is the current block, a reconstructed block around the current block may be used as a reference block for prediction of the current block. When the target of the second prediction is the first residual block, a reconstructed first residual block around the current block or the first residual block may be used as the second reference block for prediction on the first residual block.

In one embodiment, the region of the first prediction and the region of the second prediction may be different from each other. Here, the "prediction area" may indicate an area in which a prediction value is generated among blocks that are to be predicted. Alternatively, the “area of prediction” may indicate an area to which a prediction value generated by prediction is allocated among prediction blocks corresponding to a block that is a prediction target.

For example, the first prediction may be a prediction for an area of the current block. The second prediction may be a prediction for a portion of the above area. At least some of the area of the current block may be excluded from the second prediction. The block that is the target of the first prediction may be a block within a rectangular area. The second prediction may be made for some of the above rectangular regions. At least some of the areas of the quadrangle may not be the target of the second prediction.

When the area of the block that is the target of the first prediction and the area of the block that is the target of the second prediction are considered to be the same, the area of the first prediction may be the entire block that is the target of the first prediction. The area of the second prediction may be part of a block that is the target of the second prediction. Alternatively, the second prediction may be excluded for at least a portion of the block that is the target of the second prediction. In other words, the first prediction may be performed on the entire block that is the target of the first prediction. The second prediction may be performed on the selected portion of the block that is the target of the second prediction.

For example, prediction values determined by the first prediction may be assigned only to pixels in the region of the first prediction among blocks that are the targets of the first prediction. The prediction value according to the first prediction may not be allocated to pixels outside the region of the first prediction among the blocks that are the targets of the first prediction. Alternatively, a predefined value may be assigned to pixels outside the region of the first prediction among blocks that are the target of the first prediction. The predefined value may be "0".

For example, prediction values determined by the second prediction may be assigned only to pixels in the region of the second prediction among blocks that are the target of the second prediction. Pixels outside the region of the second prediction among the blocks that are the target of the second prediction may not be assigned a prediction value by the second prediction. Alternatively, a predefined value may be assigned to pixels outside the region of the second prediction among blocks that are the target of the second prediction. For example, the predefined value may be "0".

For example, when the size of the block that is the target of the first prediction is NxN, each of the NxN pixels may be assigned a prediction value determined by the first prediction. In other words, the prediction value determined by the first prediction may be assigned to all pixels of the block that are the targets of the first prediction.

For example, when the size of the block that is the target of the second prediction is N × N, one or more pixels among the N × N pixels may not be assigned a prediction value determined by the first prediction. Alternatively, when the size of the block that is the target of the second prediction is N × N, one or more pixels among the N × N pixels may be assigned a predefined value. For example, the predefined value may be "0".

In one embodiment, the region of the first prediction may be determined based on the type of the first prediction. For example, the region of the first prediction may be determined based on whether the first prediction is inter prediction. Alternatively, the region of the first prediction may be determined based on whether the first prediction is intra prediction. Alternatively, the region of the first prediction may be determined based on the prediction direction of the first prediction.

In one embodiment, the region of the second prediction may be determined based on the type of the second prediction. For example, the region of the second prediction may be determined based on whether the second prediction is inter prediction. Alternatively, the region of the second prediction may be determined based on whether the second prediction is intra prediction. Alternatively, the region of the second prediction may be determined based on the prediction direction of the second prediction.

In one embodiment, the second prediction may be performed using a second reference sample within an optional range.

In one embodiment, the region of the first reference sample used for the first prediction and the region of the second reference sample used for the second prediction may be different from each other.

As described above, the region of the first prediction and the region of the second prediction may be different from each other. The area of the first reference sample may be an area of reference samples required for generating prediction values for the area of the first prediction through the first prediction. The area of the second reference sample may be an area of reference samples required for generating prediction values for the area of the second prediction through the second prediction.

As described above, the region of the first prediction may be the entirety of the block that is the target of the first prediction. The area of the second prediction may be part of a block that is the target of the second prediction. In this case, the region of the second reference sample may be part of the region of the first reference sample.

In operation 1120, the encoder may perform encoding on the residual signal. The encoder may generate information about the encoded residual signal by performing encoding on the residual signal.

The encoder may include a transformer 130, a quantizer 140, and an entropy encoder 150.

Step 1120 may include steps 1121, 1122, and 1123.

In operation 1121, the transformer 130 may generate transform coefficients of the residual signal.

In operation 1122, the quantization unit 140 may generate quantized transform coefficients by performing quantization on the transform coefficients.

In operation 1123, the entropy encoder 150 may generate information on the encoded residual signal by performing entropy encoding on the quantized transform coefficients.

In operation 1130, the entropy encoder 150 may generate a bitstream including information about the encoded residual signal.

The bitstream may include information about the encoded residual signal, and may include information related to prediction.

The entropy encoder 150 may include information related to prediction in the bitstream. Related information of the prediction may be entropy coded.

For example, the information related to the prediction may include prediction scheme information indicating how the current block is encoded.

For example, the prediction scheme information may indicate whether the current block is encoded by intra prediction or intra prediction. Alternatively, the prediction scheme information may indicate whether the current block is encoded by intra prediction. Alternatively, the prediction scheme information may indicate whether the current block is encoded by inter prediction.

For example, the information related to the prediction may include intra prediction mode information indicating a prediction mode of the intra prediction.

For example, the information related to the prediction may include second prediction usage information indicating whether the second prediction is used in encoding the current block.

For example, the information related to the prediction includes current block type information indicating the type of the current block, first prediction block type information indicating the type of the first prediction block, second prediction block type information indicating the type of the second prediction block, First reference block type information indicating a type of a reference block, second reference block type information indicating a type of a second reference block, first reference sample type information indicating a type of a first reference sample, and a type of a second reference sample Second reference sample type information indicating the information may be included.

For example, the information related to the prediction may include first prediction region information indicating the region of the first prediction and / or second prediction region information indicating the second prediction region of the second prediction.

Information related to the prediction may include information used for the prediction described in the embodiments. The entropy encoder 150 may include the information related to the prediction in the bitstream in the order described in the embodiment or the generation order according to the embodiment.

In operation 1140, the communication unit 1020 may transmit the bitstream to the decoding apparatus 200 or the decoding apparatus 1700.

In operation 1150, the reconstructed residual signal generator may generate a reconstructed residual signal based on the information about the residual signal.

The reconstructed residual signal generator may include an inverse quantizer 160 and an inverse transformer 170.

Step 1150 may include steps 1151 and 1152.

In operation 1151, the inverse quantization unit 160 may generate the reconstructed transform coefficients by performing inverse quantization on the quantized transform coefficients.

In operation 1152, the transformer 130 may generate the reconstructed residual signal by performing transform on the reconstructed transform coefficients.

In operation 1160, the reconstructed block generator may generate a reconstructed block based on the reconstructed residual signal, the first prediction, and the second prediction.

The restored block generator may include an adder 175.

For example, the sum of the reconstructed residual signal and the prediction signal may represent a reconstructed block. Alternatively, the reconstructed block may be generated based on the sum of the reconstructed residual signal and the prediction signal.

For example, the reconstructed block may be the sum of the reconstructed residual block and the prediction block. The prediction block may be the sum of the first prediction block and the second prediction block. Alternatively, the reconstructed block may be generated based on the sum of the reconstructed residual block and the prediction block.

For example, the reconstructed block may be a block generated by the reconstructed residual block, the second prediction block, and the first prediction block. The reconstructed block generator may generate the reconstructed first residual signal by adding the reconstructed residual signal and the second prediction signal. The reconstructed block generator may generate the reconstructed block by adding the reconstructed first residual signal and the first prediction signal.

Alternatively, the reconstructed block generator may generate the reconstructed first residual block by adding the reconstructed residual block and the second prediction block. The reconstructed block generator may generate the reconstructed block by adding the reconstructed first residual block and the first prediction block.

The reconstructed block and the reconstructed first residual signal may be used as reference blocks for encoding other blocks.

12 illustrates first prediction and second prediction using intra prediction, according to an embodiment.

In step 1110 described above with reference to FIG. 11, the first prediction and the second prediction may be intra prediction. 12 illustrates an example of encoding of a current block when the first prediction and the second prediction are intra prediction.

The current block may be a PU. In FIG. 12, a PU having a size of 2N × 2N is illustrated as a current block.

First, for encoding the current block, the prediction unit may determine which prediction mode from among a plurality of prediction modes to use the first prediction.

The prediction unit may perform the first prediction by using each of the plurality of prediction modes. For example, the prediction directions of the plurality of prediction modes may be different.

In the first prediction, reconstructed blocks around the current block can be used as reference blocks. Also, pixels of reconstructed blocks around the current block can be used as reference samples.

The prediction unit may generate first prediction blocks of the plurality of prediction modes through the first prediction.

Also, the prediction unit may generate first prediction errors of the plurality of prediction modes through the first prediction. For one prediction mode, the first prediction block and the first prediction error can be calculated.

The first prediction error may indicate a difference between the current block and the first prediction block. The first prediction error may represent the first residual block.

When the first prediction error is generated, the prediction unit may select a region to which the second prediction is to be applied in the first residual block by referring to the first prediction error. The selected area may be part of the first residual block.

Next, for encoding the current block, the prediction unit may determine which part of the first residual block to apply the second prediction to.

The second prediction may be applied to the portion of the first residual block that is the selected region. Various methods can be used to select the region to which the second prediction is to be applied.

The prediction region of the second prediction may be specified by the range of reference samples used for the second prediction.

In the second prediction, reconstructed residual blocks neighboring the current block may be used as reference blocks. Also, pixels of reconstructed residual blocks neighboring the current block can be used as reference samples.

The prediction direction of the first prediction and the prediction direction of the second prediction may be the same. In other words, the intra prediction mode of the first prediction and the intra prediction mode of the second prediction may be the same.

In FIG. 12, an intra prediction mode having a mode value of 30 is illustrated as an example.

In general, if the prediction direction of the first prediction and the prediction direction of the second prediction are the same, the range of reference samples of the first prediction and the reference samples of the second prediction may be the same.

The second prediction may be applied to a portion of the first residual block. The range of reference samples of the second prediction may be part of the range of reference samples of the first prediction.

In FIG. 12, the range of reference samples of the second prediction is shown as a range between α and α + β.

α may indicate the first reference sample to be used for the second prediction among the reference samples for the prediction direction of the second prediction. Alternatively, α may indicate the starting position of the second prediction.

α + β may indicate the last reference sample to be used for the second prediction among the reference samples for the prediction direction of the second prediction. Alternatively, α + β may indicate the last position of the second prediction.

β may represent the number of reference samples to be used for the second prediction. Alternatively, β may represent the range of the second prediction or the length of the second prediction.

Once the range of reference samples is determined, an area of the second prediction may be specified according to the determined range of reference samples and the prediction direction. In FIG. 12, regions specified by dotted lines in the residual block are shown. The region of the second prediction may be a region generated as the reference samples in the range between α and α + β advance in the prediction direction.

The prediction unit may select a range of reference samples to be used for the second prediction among the reference samples for the prediction direction. The prediction unit may perform the second prediction by using the reference samples in the selected range among the entire reference samples for the prediction direction. Here, the entire reference samples may correspond to the reference samples used in the first prediction.

For the second prediction, the value of the reference sample outside of the selected range may be considered null or "0". For example, the value of the previous reference sample of α may be considered zero, and the value of the subsequent reference sample of α + β may be considered zero.

The prediction unit may generate second prediction blocks for the plurality of candidate regions through the second prediction.

The plurality of candidate regions may be different from each other. The plurality of candidate regions may be different portions of the first residual block.

For example, the region of the first residual block can be specified as reference samples used for the second prediction. In this case, a part of the first residual block or the region of the second prediction for the first residual block may be represented by an ordered pair (α, β). Order pairs of (α, β) corresponding to the plurality of candidate regions may be different from each other. In other words, the ordered pairs α and β may indicate a range of reference samples to be used for the second prediction.

The plurality of candidate regions may be all different regions that may be selected by the manner specified for the first residual block. For example, the specified scheme may be a prediction direction.

For example, the region of the first residual block can be specified as reference samples used for the second prediction. When the number of reference samples that can be used for the whole of the first residual block in n in the prediction mode of the second prediction is n, α may be greater than or equal to 0 and less than or equal to n−1. β may be 0 or more and n-1-α or less. Further, when there are n reference samples that can be used for the entirety of the first residual block in the prediction mode of the second prediction, the number of ordered pairs of (α, β) may be (n + 1) n / 2. In other words, when there are n reference samples that can be used for the entirety of the first residual block in the prediction mode of the second prediction, the number of the plurality of candidate regions may be (n + 1) n / 2.

For example, the plurality of candidate regions may respectively correspond to ranges of reference samples to be used for the second prediction.

α and β may each be an index of a list of reference samples in a specified prediction direction.

In order to indicate the position in the list of reference samples using α and β, etc., the order of the reference samples may be determined.

In one embodiment, first of all, the smaller the value of the X coordinate of the reference sample, the faster the order of the reference sample. Next, among the same reference samples of the values of the X coordinates, the larger the value of the Y coordinate of the reference sample, the faster the order of the reference samples. For example, the lower left reference samples 731, the left reference samples 733, the upper left-left corner reference sample 735, and the upper reference samples. The order of the reference sample may be given in the order of 737 and the above right-above reference samples 739. Among the lower left reference samples 731, the lower reference sample may have a faster order than the upper reference sample. Among the left reference samples 733, the bottom reference sample may have a faster order than the top reference sample. Among the top reference samples 737, the reference sample located on the left side may have a faster order than the reference sample located on the right side. Among the upper right reference samples 739, the left reference sample may have a faster order than the right reference sample.

The order of reference samples described above is for one example, and other manners of order may be used. For example, the order of the reference samples may be given in the reverse order described above.

Further, the order of the reference samples may be given for the reference samples determined according to the prediction direction. In other words, the above-described lower-left reference samples 731, left-reference samples 733, upper-left corner reference sample 735, and upper reference samples Only some of the reference samples 737 and upper-right reference samples 739 determined according to the prediction direction of the second prediction may be used for the prediction of the entirety of the first residual block. Order may be given for some of the determined reference samples.

For example, when the prediction direction of the second prediction is the vertical direction, the reference samples that may be used for the entirety of the first residual block may be upper reference samples 737. When the size of the current block is 2N × 2N, the order of the leftmost reference sample among the top reference samples 737 may be 0, and the order of the rightmost reference sample among the top reference samples 737 may be 2N−1. Can be.

Or, for example, when the prediction direction of the second prediction is the horizontal direction, reference samples that may be used for the entirety of the first residual block may be left reference samples 733. When the size of the current block is 2N × 2N, the order of the lowest reference sample of the left reference samples 737 may be 0, and the order of the highest reference sample of the left reference samples 737 may be 2N-1.

The prediction unit may generate second prediction errors for the plurality of candidate regions through the second prediction. For one region, the second prediction block and the second prediction error can be calculated.

The second prediction error may indicate a residual between the first residual block and the second prediction block. The second prediction error may represent the second residual block.

As described above, the prediction unit may generate a plurality of first prediction errors for the plurality of prediction modes. Further, for each first prediction error of the plurality of first prediction errors, the prediction unit may generate the first prediction error through the second prediction. Second prediction errors of the plurality of candidate regions of the first residual block indicated by the first prediction error may be generated.

Through the above-described processes, the prediction unit may generate second prediction errors for all candidate regions of the plurality of prediction modes.

The prediction unit may select a second prediction error having a minimum rate-distortion cost among the generated second prediction errors. The candidate region corresponding to the selected second prediction error among the plurality of candidate regions may be a part to which the second prediction of the current block is applied. In addition, a prediction mode corresponding to the selected second prediction error among the plurality of prediction modes may be a prediction mode to be used in the first prediction. In other words, the prediction unit may determine the prediction mode of the first prediction and the region of the second prediction to be used for encoding the current block, depending on whether the second prediction error for which region of the prediction mode has a minimum rate-distortion cost. have.

Or, first, the prediction unit may select a prediction mode having a minimum rate-distortion cost among the plurality of prediction modes. In this case, the second prediction may be performed on the first prediction error or the first residual block generated by the selected prediction mode. Next, the prediction unit may select a second prediction error having a minimum rate-distortion cost among the second prediction errors. In other words, the prediction unit may determine the prediction mode of the first prediction to be used for encoding the current block according to which first prediction error has the minimum rate-distortion cost, and which second prediction error is the minimum rate-distortion cost. It is possible to determine the region of the second prediction to be used for encoding the current block according to whether

The information related to the prediction described above with reference to FIG. 11 may include information about the selective intra-picture prediction described above.

For example, the information related to the prediction may include SSIP_flag, which is a flag indicating whether to use the optional Secondary Intra-Prediction (SSIP).

For example, the information related to the prediction may include information representing α described above and information representing β described above. In other words, the information related to the prediction may include information indicating the first reference sample to be used for the second prediction and information indicating the number of reference samples to be used for the second prediction. Alternatively, the information related to the prediction may include information indicating the first reference sample to be used for the second prediction and information indicating the last reference sample to be used for the second prediction.

Alternatively, the information related to the prediction may include information indicating a position at which the second prediction starts and to which position the second prediction proceeds. The information related to the prediction may include information indicating a position where the second prediction starts and a position where the second prediction ends.

For example, the information indicating α or the information indicating the position where the second prediction starts may be SSIP_start_point, and the information indicating β or the information indicating to which position the second prediction may proceed may be SSIP_length.

Through the above-described determination, the prediction error of the current block can be further reduced compared to performing only the existing first prediction. The second prediction may effectively reduce the prediction error with respect to the tendency of the distortion of the prediction error to remain in the direction of the prediction mode described above with reference to FIG. 9. In other words, by effectively using the tendency of the distortion of the prediction error to remain in the direction of the prediction mode through the second prediction, the performance of the prediction may be improved and the prediction error may be reduced.

13 is a flowchart of a method of generating a residual block, according to an exemplary embodiment.

Step 1110 described above with reference to FIG. 11 may include steps 1310, 1320, 1330, 1340, 1350, 1360, 1370, and 1380.

In one embodiment, the block that is the target of the first prediction may be the current block. The first prediction may be intra prediction.

In operation 1310, the prediction unit may generate a first prediction signal by performing a first prediction on the current block.

The first prediction signal may represent the first prediction block.

As described above with reference to FIG. 12, the prediction unit may perform the first prediction for each prediction mode of the plurality of prediction modes to find the prediction mode of the first prediction best for the encoding of the current block. In other words, steps 1310, 1320, and 1325 may be performed for each prediction mode of the plurality of prediction modes.

The plurality of prediction modes may correspond to different prediction directions.

In operation 1320, the prediction unit may generate a first prediction error based on the current block and the first prediction signal.

The first prediction error may represent a first prediction error block or a first residual block.

The first residual block may represent a residual of the current block and the first prediction block. In other words, the prediction unit may generate the first prediction error by subtracting the first prediction block indicated by the first prediction signal from the current block.

In operation 1325, the prediction unit may calculate the rate-distortion cost of the first prediction using the first prediction error.

The prediction unit may store the calculated first prediction error and / or the rate-distortion cost of the first prediction in the memory 1030, the storage 1040, or a buffer.

As described above with reference to FIG. 12, the prediction unit may perform the first prediction for each prediction mode of the plurality of prediction modes to find the prediction mode of the first prediction best for the encoding of the current block. The prediction unit may calculate rate-distortion costs of the plurality of prediction modes using the first prediction errors of the plurality of prediction modes. The prediction unit may store the minimum rate-distortion cost of the calculated rate-distortion costs as the minimum first rate-distortion cost RDcost_1. For example, RDcost_1 may be the rate-distortion cost of the mode with the minimum rate-distortion cost in the first intra picture prediction.

The prediction unit may store a first prediction error and RDcost_1 corresponding to RDcost_1.

The second prediction can optionally be performed based on the predefined conditions.

In operation 1330, the prediction unit may determine whether to perform a second prediction.

In one embodiment, the determination of step 1330 may not ultimately determine that the current block is encoded using the second prediction. In operation 1130, the prediction unit may determine whether a second prediction is possible with respect to the current block or the first residual block.

For example, if it is determined not to perform the second prediction, the current block may be encoded without using the second prediction. When it is determined to perform the second prediction, the second prediction signal and the second prediction error may be generated by the second prediction, but in encoding of the current block through comparison of the rate-distortion costs in step 1360. It may be determined whether to use the second prediction.

As described above with reference to FIG. 12, the prediction unit performs a second prediction after performing the first prediction on each prediction mode of the plurality of prediction modes to find the prediction mode of the first prediction best for the encoding of the current block. You can decide whether or not to do so. In other words, step 1330 may be performed for each prediction mode of the plurality of prediction modes.

If it is determined not to perform the second prediction, the residual block of the current block may be a block indicated by the first prediction error. In other words, the first prediction error may be provided as a residual block.

When it is determined to perform the second prediction, the residual block of the current block may be a block indicated by the second prediction error signal to be described later. In other words, the second prediction error signal may be provided as a residual block.

The prediction unit may determine whether to perform the second prediction based on the predefined condition.

In an embodiment, the prediction unit may determine whether to perform the second prediction based on the prediction mode of the first prediction.

For example, if the prediction mode of the first prediction is the non-directional mode, the prediction unit may not perform the second prediction. Alternatively, the prediction unit may perform the second prediction when the prediction mode of the first prediction is the directional mode.

When the prediction mode of the first prediction is a non-directional mode among the plurality of prediction modes, it may not be possible to perform prediction on a selective region or range because there is no direction of the prediction mode. Therefore, the second prediction may be performed only when the prediction mode of the first prediction is the directional mode.

In an embodiment, the prediction unit may determine whether to perform the second prediction based on the position of the current block.

For example, the prediction unit may not perform the second prediction when the current block is adjacent to the boundary. For example, the prediction unit may not perform the second prediction when the current block is adjacent to the upper boundary and the left boundary. The boundary may be a boundary of a picture, a boundary of a slice, or a boundary of a tile.

If the current block is adjacent to the upper boundary and the left boundary, the first residual block for the second prediction may not be sufficient. If the first residual block is lacking, a second reference sample for performing the second prediction cannot be generated intact.

In an embodiment, the prediction unit may determine whether to perform the second prediction based on the number of reconstructed blocks adjacent to the current block.

For example, if the number of reconstructed blocks adjacent to the current block is two or less, the prediction unit may not perform the second prediction.

If it is determined to perform the second prediction, step 1340 may be performed.

If it is determined not to perform the second prediction, step 1370 or step 1120 may be performed. When step 1120 is performed, the residual signal of step 1120 may be a first prediction error. In other words, when only the first prediction is performed without the second prediction, the first prediction error generated by the first prediction may be used as the residual signal of the current block.

In operation 1340, the prediction unit may generate a second prediction signal by performing a second prediction on the first prediction error.

The second prediction signal may represent the second prediction block.

The second prediction may be intra prediction, and the prediction direction of the second prediction may be the same as the prediction direction of the first prediction.

The predictor may perform second prediction on a portion of the first residual block. Some area may be specified by reference samples to be used in the second prediction.

The area of the reference samples of the second prediction may be part of the area of the reference samples of the first prediction.

As described above with reference to FIG. 12, the prediction unit may perform second prediction on each candidate region of the plurality of candidate regions to find an optimal second prediction region for encoding a current block.

In addition, as described above with reference to FIG. 12, the prediction unit is configured for each candidate region of each prediction mode in order to find the prediction mode of the optimal first prediction and the region of the optimal second prediction for encoding the current block. 2 Predictions can be performed. In other words, step 1340 may be performed for each candidate region of each prediction mode.

Here, the prediction mode of the optimal first prediction and the region of the optimal second prediction may be prediction modes and regions corresponding to candidate regions having a minimum rate-distortion cost among the plurality of candidate regions of the plurality of prediction modes. have. In other words, the prediction unit encodes the current block based on which prediction mode and which region among the plurality of candidate regions of the plurality of prediction modes is the minimum rate-distortion cost incurred when the first prediction and the second prediction are performed. The prediction mode of the first prediction and the region of the second prediction may be determined.

The plurality of candidate regions may respectively correspond to ranges of reference samples to be used for the second prediction.

The prediction unit may select a range of reference samples to be used for the second prediction among the reference samples for the prediction direction. The prediction unit may perform the second prediction by using the reference samples in the selected range among the entire reference samples for the prediction direction. For the second prediction, the value of the reference sample outside the selected range may be considered "0".

In operation 1350, the prediction unit may generate a second prediction error based on the first prediction error and the second prediction signal.

The second prediction error may represent a second prediction error block or a second residual block.

The second residual block may represent a residual of the first residual block and the second prediction block. In other words, the prediction unit may generate the second prediction error signal by subtracting the second prediction block indicated by the second prediction signal from the first residual block indicated by the first prediction error.

In operation 1355, the prediction unit may calculate the rate-distortion cost of the second prediction using the second prediction error.

The prediction unit may store the calculated second prediction error and / or the rate-distortion cost of the second prediction in the memory 1030, the storage 1040, or a buffer.

As described above with reference to FIG. 12, the prediction unit may perform second prediction on each region of the plurality of regions to find an optimal second prediction region for encoding the current block. The prediction unit may calculate rate-distortion costs of the plurality of regions using second prediction errors of the plurality of regions. The prediction unit may store the minimum rate-distortion cost of the calculated rate-distortion costs as the minimum second rate-distortion cost RDcost_2. For example, RDcost_2 may be the rate-distortion cost of the region with the lowest rate-distortion cost in the optional second intra prediction.

The prediction unit may store a second prediction error and RDcost_2 corresponding to RDcost_2.

The second prediction can optionally be used based on the predefined conditions.

In operation 1360, the prediction unit may determine whether to use the second prediction in encoding of the current block.

The prediction unit may determine whether to use the second prediction in encoding of the current block based on the predefined condition.

For example, the prediction unit may determine to use the second prediction if the rate-distortion cost is further reduced by using the second prediction. The prediction unit may determine not to use the second prediction if the rate-distortion cost is not further reduced even if the second prediction is used.

For example, the prediction unit may determine to use the second prediction when the minimum rate-distortion cost when the second prediction is used is smaller than the minimum rate-distortion cost when the second prediction is not used.

In an embodiment, the prediction unit may determine whether to use the second prediction in encoding of the current block based on the comparison between RDcost_1 and RDcost_2.

The fact that RDcost_2 is smaller than RDcost_1 may indicate that the rate-distortion cost of encoding of the current block may be further reduced by using the second prediction than when using only the first prediction.

For example, if RDcost_1 is less than or equal to RDcost_2, the prediction unit may determine not to use the second prediction for encoding the current block. If RDcost_1 is greater than RDcost_2, the prediction unit may determine to use the second prediction for encoding the current block.

If the second prediction is not used for encoding the current block, step 1370 may be performed.

When the second prediction is used for encoding the current block, step 1380 may be performed.

In operation 1370, the prediction unit may perform a setting indicating that the second prediction is not used.

The prediction unit may set a value of the second prediction usage information to indicate that the second prediction usage information does not use the second prediction. For example, if the value of the second prediction usage information is "0", the second prediction usage information may indicate that the second prediction usage information is not used.

When step 1370 is performed, the residual signal of step 1120 may be a first prediction error. In other words, when the second prediction is not performed on the current block, the first prediction error generated based on the first prediction may be used as the residual signal.

In operation 1380, the prediction unit may perform setting indicating to use the second prediction.

The prediction unit may set a value of the second prediction usage information to indicate that the second prediction usage information uses the second prediction. For example, if the value of the second prediction usage information is "1", the second prediction usage information may indicate that the second prediction usage information is used.

When step 1380 is performed, the residual signal of step 1120 may be a second prediction error. In other words, when the second prediction is performed on the current block, the second prediction error generated based on the first prediction and the second prediction may be used as the residual signal.

14 is a flowchart of a method of generating a restored block, according to an example.

Step 1160 described above with reference to FIG. 11 may include steps 1410, 1420, and 1430 below.

In operation 1410, it may be determined whether a second prediction is used for encoding of the reconstructed block generator current block.

If second prediction is used for encoding of the current block, step 1420 may be performed.

If the second prediction is not used for encoding of the current block, step 1430 may be performed. If the second prediction is not used, the reconstructed first prediction error of the current block may be a reconstructed residual signal.

In operation 1420, the reconstructed block generator may generate a reconstructed first prediction error based on the reconstructed residual signal and the second prediction signal.

The reconstructed block generator may generate a reconstructed first prediction error of the current block based on the residual signal of the current block and the second prediction signal for a portion of the current block.

The reconstructed first prediction error may be the sum of the reconstructed residual signal and the second prediction signal. Alternatively, the reconstructed first prediction error may be generated based on the reconstructed residual signal and the second prediction signal.

In operation 1430, the reconstructed block generator may generate a reconstructed block based on the reconstructed first prediction error and the first prediction signal.

The reconstructed block generator may generate a signal of the reconstructed block based on the reconstructed first prediction error of the current block and the first prediction signal of the current block.

In other words, when the second prediction is used, the signal representing the reconstructed block may be the sum of the reconstructed residual signal, the second prediction signal, and the first prediction signal. Alternatively, when the second prediction is used, a signal representing the reconstructed block may be generated based on the reconstructed residual signal, the second prediction signal, and the first prediction signal.

In other words, when the second prediction is not used, the signal representing the reconstructed block may be the sum of the reconstructed residual signal and the first prediction signal. Alternatively, when the second prediction is not used, a signal representing the reconstructed block may be generated based on the reconstructed residual signal and the first prediction signal.

The signal of the reconstructed block may be the sum of the reconstructed first prediction error and the first prediction signal. Alternatively, the reconstructed first prediction error may be generated based on the reconstructed residual signal and the second prediction signal.

15 illustrates an encoding process according to an embodiment.

In Figure 15, plus and minus

The symbol may represent generating an error signal for two signals. An error signal may be generated by subtracting an input signal input with a minus sign from an input signal input with a plus sign.

With + and +

The symbol may represent generating a signal that combines two signals. A signal that is the sum of the two signals can be generated by adding input signals that are respectively input as + symbols.

The prediction unit described above with reference to FIG. 11 may include a first predictor and a second predictor.

The process within the dashed line may represent the SSIP.

Rhombus may represent a conditional divergence.

In an expression such as "A = B", "A" may represent a target to which a value is assigned, and "B" may represent a value to be assigned to a target.

TQ may represent transform and quantization. (TQ) ⁻¹ may represent inverse transformation and inverse quantization.

e ₁ may represent a signal of the first prediction error.

May represent a signal of the reconstructed first prediction error.

e ₂ may represent a signal of the second prediction error.

May represent a signal of the reconstructed second prediction error.

Although not shown in FIG. 15, when the second prediction is used,

Is

May be the same as If no second prediction is used,

Is

May be the same as

e may represent a residual signal.

May represent the reconstructed residual signal.

May represent the first prediction signal.

May represent a second prediction signal.

16 is a structural diagram of a decoding apparatus according to an embodiment.

The decryption apparatus 1600 may be a general purpose computer system for performing decryption.

As illustrated in FIG. 16, the decoding apparatus 1600 may include at least one processor 1610, a memory 1630, a user interface (UI) input device 1650, which communicates with each other via a bus 1690, UI output device 1660 and storage 1640. In addition, the decoding apparatus 1600 may further include a communication unit 1620 connected to the network 1699. The processor 1610 may be a semiconductor device that executes processing instructions stored in a central processing unit (CPU), a memory 1630, or a storage 1640. Memory 1630 and storage 1640 may be various forms of volatile or nonvolatile storage media. For example, the memory may include at least one of a ROM 1631 and a RAM 1632.

The processor 1610 may include an entropy decoder 210, an inverse quantizer 220, an inverse transformer 230, an intra predictor 240, an inter predictor 250, an adder 255 of the decoding apparatus 200. The filter unit 260 and the reference picture buffer 270 may be included.

The recording medium may store at least one module for the operation of the decoding apparatus 1600. The memory 1630 may store at least one module and may be configured to be executed by the at least one processor 1610.

According to an embodiment, the entropy decoder 210, the inverse quantizer 220, the inverse transformer 230, the intra predictor 240, the inter predictor 250, and the adder 255 of the decoder 1600 may be described. At least some of the filter unit 260 and the reference picture buffer 270 may be program modules, and may communicate with an external device or system. The program modules may be included in the decryption apparatus 1600 in the form of an operating system, an application program module, and other program modules.

The program modules may be physically stored on various known storage devices. In addition, at least some of these program modules may be stored in a remote storage device that can communicate with the decryption apparatus 1600.

Program modules perform routines or subroutines, programs, objects, components, and data to perform functions or operations, or to implement abstract data types, according to one embodiment. Data structures and the like, but is not limited thereto.

The program modules may be composed of instructions or code performed by the processor 1610.

Functions related to communication of data or information of the decoding apparatus 1600 may be performed through the communication unit 1620.

17 is a flowchart of a decoding method according to an embodiment.

The decoding method of the embodiment may be performed by the decoding apparatus 200 or the decoding apparatus 1600.

In operation 1710, the communication unit 1620 may receive a bitstream from the encoding apparatus 100 or the encoding apparatus 1100.

The bitstream may include information about the encoded residual signal, and may include information related to prediction.

The information about the encoded residual signal may include entropy coded quantized transform coefficients.

Information related to the prediction may be entropy decoded.

In operation 1720, the entropy decoder 210 may generate quantized transform coefficients by performing entropy decoding on the bitstream. In addition, the entropy decoding unit 210 may generate information related to prediction by performing entropy decoding on the bitstream.

For example, the information related to the prediction may include prediction scheme information indicating how the current block is encoded.

For example, the prediction scheme information may indicate whether the current block is encoded by intra prediction or intra prediction. Alternatively, the prediction scheme information may indicate whether the current block is encoded by intra prediction. Alternatively, the prediction scheme information may indicate whether the current block is encoded by inter prediction.

For example, the information related to the prediction may include intra prediction mode information indicating a prediction mode of the intra prediction.

For example, the information related to the prediction may include second prediction usage information indicating whether the second prediction is used in encoding the current block.

For example, the information related to the prediction includes current block type information indicating the type of the current block, first prediction block type information indicating the type of the first prediction block, second prediction block type information indicating the type of the second prediction block, First reference block type information indicating a type of a reference block, second reference block type information indicating a type of a second reference block, first reference sample type information indicating a type of a first reference sample, and a type of a second reference sample Second reference sample type information indicating the information may be included.

For example, the information related to the prediction may include first prediction region information indicating the region of the first prediction and / or second prediction region information indicating the region of the second prediction.

In addition, the information related to the prediction may include information used for the prediction described in the embodiments. Information related to the prediction may be included in the bitstream according to the order described in the embodiment or the order of generation according to the embodiment.

In operation 1730, the reconstructed residual signal generator may generate a reconstructed residual signal for the current block based on the quantized transform coefficients.

The reconstructed residual signal generator may include an inverse quantizer 220 and an inverse transformer 230.

Step 1730 may include steps 1731 and 1732.

In operation 1731, the inverse quantization unit 220 may generate inverse quantized transform coefficients by performing inverse quantization on the quantized transform coefficients.

In operation 1732, the transformer 230 may generate the reconstructed residual signal by performing inverse transform on the inverse quantized transform coefficients.

Through steps 1710, 1720, and 1730, a reconstructed residual signal for the current block may be generated.

In operation 1740, the reconstructed block generator may generate a reconstructed block based on the reconstructed residual signal, the second prediction, and the first prediction.

The reconstructed block generator may include an adder 225, an intra predictor 240, an inter predictor 250, a filter 260, and a reference picture buffer 270.

The recovered residual signal may be a recovered residual block.

For example, the reconstructed block may be the sum of the reconstructed residual block and the prediction block. Alternatively, the reconstructed block may be generated based on the sum of the reconstructed residual block and the prediction block.

For example, the prediction block may be the sum of the first prediction block and the second prediction block. Alternatively, the prediction signal may be the sum of the first prediction signal and the second prediction signal.

For example, the reconstructed block may be the sum of the reconstructed residual block, the first prediction block, and the second prediction block. The reconstructed signal may be the sum of the reconstructed residual signal, the first prediction signal, and the second prediction signal. Alternatively, the reconstructed block may correspond to the sum of the reconstructed residual block, the first prediction block, and the second prediction block.

The first prediction block may be a block generated by the first prediction or a block indicated by the prediction signal generated by the first prediction. The second prediction block may be a block generated by the second prediction or a block indicated by the prediction signal generated by the second prediction.

If the second prediction is not used, the second prediction signal may be a null signal and the second prediction block may be a null block. The null block may be a block in which the values of all pixels of the block are "0".

For example, when the second prediction is used for encoding of the current block, the reconstructed residual signal may be the reconstructed second prediction error described above with reference to FIG. 14. Alternatively, when the second prediction is used for encoding of the current block, the reconstructed residual signal may be a signal in which the second prediction error described above with reference to FIG. 11 is reconstructed by inverse quantization and inverse transformation.

For example, the reconstructed residual signal in which the second prediction is not used for the encoding of the current block may be the reconstructed first prediction error described above with reference to FIG. 14. Alternatively, when the second prediction is not used for encoding of the current block, the reconstructed residual signal may be a signal in which the first prediction error described above with reference to FIG. 11 is reconstructed by inverse quantization and inverse transformation.

The reconstructed block generator may generate a reconstructed block by performing inter prediction or intra prediction.

In one embodiment, the first prediction and the second prediction may be different kinds of predictions.

In one embodiment, the first prediction and the second prediction may be the same kind of predictions. For example, each of the first prediction and the second prediction may be intra prediction. If the first prediction is intra prediction, the second prediction may also be set to intra prediction.

In one embodiment, the prediction direction of the first prediction and the prediction direction of the second prediction may be the same. The prediction direction of the second prediction may be set to be the same as the prediction direction of the first prediction.

For example, in one embodiment, each of the first prediction and the second prediction may be inter prediction.

In one embodiment, a block that is the target of the first prediction and a block that is the target of the second prediction may be different from each other. Here, the "block to be predicted" may mean to which block the "prediction" generates a prediction block.

For example, the type of the block targeted for the first prediction and the type of the block targeted for the second prediction may be different from each other. The type of block may be an original block, a luma block, a chroma block, a depth block, a residual block, or the like.

For example, the block targeted for the first prediction may be a reconstructed block. The block that is the target of the second prediction may be a prediction error block.

Reference blocks may be used for prediction. In one embodiment, the first reference block of the first prediction and the second reference block of the second prediction may be different from each other. The reference blocks may differ from each other in terms of type and / or location.

In one embodiment, the position of the first reference block of the first prediction and the position of the second reference block of the second prediction may be different from each other. Here, the position of the first reference block may be a position relative to the block that is the target of the first prediction. The position of the second reference block may be a position relative to the block that is the target of the second prediction.

In one embodiment, the first reference block of the first prediction may be plural. The second reference block of the second prediction may be plural. At least some of the plurality of first reference blocks of the first prediction and the plurality of second reference blocks of the second prediction may be different from each other.

The position of at least one of the plurality of first reference blocks of the first prediction and the plurality of second reference blocks of the second prediction is the positions of the plurality of first reference blocks and the positions of the plurality of second reference blocks. Can only be included in one of the

In one embodiment, the type of the first reference block of the first prediction and the type of the second reference block of the second prediction may be different from each other. The type of block may be a reconstructed block, a reconstructed luma block, a reconstructed chroma block, a reconstructed depth block, a reconstructed first residual block, a reconstructed second residual block, or the like.

For example, the first reference block of the first prediction may be a reconstructed block. The second reference block of the second prediction may be a reconstructed first residual block. The first reference sample of the first prediction may be a pixel of the reconstructed block. The second reference sample of the second prediction may be a pixel of the reconstructed first residual block.

For example, the first reference block of the first prediction may be a reconstructed neighboring block neighboring the current block. The second reference block of the second prediction may be a reconstructed first residual block neighboring the current block.

The reconstructed neighbor residual block neighboring the current block may be obtained by adding the reconstructed residual block of the reconstructed neighboring block to the second prediction block of the reconstructed neighboring block. Alternatively, the reconstructed neighbor residual block neighboring the current block may be a difference between the reconstructed neighboring block and the first prediction block of the reconstructed neighboring block.

For example, when the object of the first prediction is the current block, a reconstructed block around the current block may be used as a reference block for prediction of the current block. When the target of the second prediction is the first residual block, a reconstructed first residual block around the current block or the first residual block may be used as the second reference block for prediction on the first residual block.

In one embodiment, the region of the first prediction and the region of the second prediction may be different from each other. Here, the "prediction area" may indicate an area in which a prediction value is generated among blocks that are to be predicted. Alternatively, the “area of prediction” may indicate an area to which a prediction value generated by prediction is allocated among prediction blocks corresponding to a block that is a prediction target.

For example, the first prediction may be a prediction for an area of the current block. The second prediction may be a prediction for a portion of the above area. At least some of the area of the current block may be excluded from the second prediction. The block that is the target of the first prediction may be a block within a rectangular area. The second prediction may be made for some of the above rectangular regions. At least some of the areas of the quadrangle may not be the target of the second prediction.

When the area of the block that is the target of the first prediction and the area of the block that is the target of the second prediction are considered to be the same, the area of the first prediction may be the entire block that is the target of the first prediction. The area of the second prediction may be part of a block that is the target of the second prediction. Alternatively, the second prediction may be excluded for at least a portion of the block that is the target of the second prediction. In other words, the first prediction may be performed on the entire block that is the target of the first prediction. The second prediction may be performed on the selected portion of the block that is the target of the second prediction.

For example, prediction values determined by the first prediction may be assigned only to pixels in the region of the first prediction among blocks that are the targets of the first prediction. The prediction value according to the first prediction may not be allocated to pixels outside the region of the first prediction among the blocks that are the targets of the first prediction. Alternatively, a predefined value may be assigned to pixels outside the region of the first prediction among blocks that are the target of the first prediction. For example, the predefined value may be "0".

For example, prediction values determined by the second prediction may be assigned only to pixels in the region of the second prediction among blocks that are the target of the second prediction. Pixels outside the region of the second prediction among the blocks that are the target of the second prediction may not be assigned a prediction value by the second prediction. Alternatively, a predefined value may be assigned to pixels outside the region of the second prediction among blocks that are the target of the second prediction. For example, the predefined value may be "0".

For example, when the size of the block that is the target of the first prediction is NxN, each of the NxN pixels may be assigned a prediction value determined by the first prediction. In other words, the prediction value determined by the first prediction may be assigned to all pixels of the block that are the targets of the first prediction.

For example, when the size of the block that is the target of the second prediction is N × N, one or more pixels among the N × N pixels may not be assigned a prediction value determined by the first prediction. Alternatively, when the size of the block that is the target of the second prediction is N × N, one or more pixels among the N × N pixels may be assigned a predefined value. For example, the predefined value may be "0".

In one embodiment, the region of the first prediction may be determined based on the type of the first prediction. For example, the region of the first prediction may be determined based on whether the first prediction is inter prediction. Alternatively, the region of the first prediction may be determined based on whether the first prediction is intra prediction. Alternatively, the region of the first prediction may be determined based on the prediction direction of the first prediction.

In one embodiment, the region of the second prediction may be determined based on the type of the second prediction. For example, the region of the second prediction may be determined based on whether the second prediction is inter prediction. Alternatively, the region of the second prediction may be determined based on whether the second prediction is intra prediction. Alternatively, the region of the second prediction may be determined based on the prediction direction of the second prediction.

In one embodiment, the second prediction may be performed using a second reference sample within an optional range.

In one embodiment, the region of the first reference sample used for the first prediction and the region of the second reference sample used for the second prediction may be different from each other.

As described above, the region of the first prediction and the region of the second prediction may be different from each other. The area of the first reference sample may be an area of reference samples required for generating prediction values for the area of the first prediction through the first prediction. The area of the second reference sample may be an area of reference samples required for generating prediction values for the area of the second prediction through the second prediction.

As described above, the region of the first prediction may be the entirety of the block that is the target of the first prediction. The area of the second prediction may be part of a block that is the target of the second prediction. In this case, the region of the second reference sample may be part of the region of the first reference sample.

18 is a flowchart of a method of generating a restored block, according to an exemplary embodiment.

Step 1740 described above with reference to FIG. 17 may include the following steps 1810, 1820, 1830, 1840, and 1850.

The second prediction can optionally be performed based on the predefined conditions.

In operation 1810, the reconstructed block generator may determine whether to use the second prediction for generation of the reconstructed block.

The reconstructed block generator may determine whether to use the second prediction for the generation of the reconstructed block based on a predefined condition.

In an embodiment, the reconstructed block generator may determine whether to use the second prediction based on the prediction mode of the first prediction.

For example, the reconstructed block generator may obtain intra prediction mode information indicating a prediction mode of intra prediction from the bitstream. The reconstructed block generator may not use the second prediction if the prediction mode of the first prediction is a non-directional mode.

In an embodiment, the reconstructed block generator may determine whether to use the second prediction based on the position of the current block.

For example, the reconstructed block generator may not use the second prediction when the current block is adjacent to the boundary. For example, the prediction unit may not use the second prediction when the current block is adjacent to the upper boundary and the left boundary. The boundary may be a boundary of a picture, a boundary of a slice, or a boundary of a tile.

In an embodiment, the reconstructed block generator may determine whether to use the second prediction based on the number of reconstructed blocks adjacent to the current block. The recovered block may be a PU.

For example, the reconstructed block generator may not use the second prediction when the number of reconstructed blocks adjacent to the current block is two or less.

In one embodiment, after the determination is made whether to use the second prediction by the prediction mode, the position of the current block, and / or the number of adjacent reconstructed blocks, the reconstructed block generator then performs a second prediction on the encoding of the current block. It may be determined whether to use the second prediction based on whether it is used.

For example, the reconstructed block generator may obtain second prediction usage information from the bitstream. The reconstructed block generator may use the second prediction when the second prediction usage information indicates that the second prediction use information is used. The reconstructed block generator may not use the second prediction when the second prediction usage information indicates that the second prediction usage information does not use the second prediction.

When using the second prediction for encoding the current block, step 1820 may be performed.

When the second prediction is not used for encoding the current block, step 1840 may be performed. When the second prediction is not used, the reconstructed first prediction error of the current block may be a reconstructed residual signal.

In operation 1820, the reconstructed block generator may generate a second prediction signal by performing a second prediction.

The second prediction may correspond to the second prediction in encoding of the current block described above with reference to FIGS. 11 and 13. For example, the second prediction signal may correspond to the second prediction in encoding of the current block described above with reference to FIGS. 11 and 13.

The second prediction signal may represent the second prediction block.

The second prediction may be intra prediction, and the prediction direction of the second prediction may be the same as the prediction direction of the first prediction.

The reference block of the second prediction may be a reconstructed residual block. For example, the reconstructed block generator may generate a second prediction signal by performing a second prediction using a reconstructed residual block of a block neighboring the current block as a reference block.

In one embodiment, as described above, the region of the second prediction may be part of the region of the current block. The reconstructed block generator may obtain second prediction region information indicating a prediction region of the second prediction from the bitstream. The reconstructed block generator may perform second prediction on the second prediction region.

In one embodiment, the region to which the second prediction is to be applied may be specified based on the range of reference samples used for the second prediction. The region to which the second prediction is to be applied may be specified based on the prediction direction of the second prediction. For example, the region to which the second prediction is to be applied may be specified by the range of reference samples used for the second prediction and the prediction direction of the second prediction.

In an embodiment, the reconstructed block generator may perform second prediction using reference samples in a selected range of the entire reference samples for the prediction direction. For the second prediction, the value of the reference sample outside the selected range may be considered "0".

In an embodiment, the reconstructed block generator may obtain information α indicating the first reference sample to be used for the second prediction and information β indicating the number of reference samples to be used for the second prediction from the bitstream. The range of reference samples used for the second prediction may be specified by α and β.

In one embodiment, the range of reference samples may be specified by information α indicating the first reference sample to be used for the second prediction and information β indicating the number of reference samples to be used for the second prediction. The region of the second prediction may be a region generated as the reference samples in the range between α and α + β advance in the prediction direction.

For the second prediction, the value of the reference sample outside of the selected range may be considered null or "0". For example, the value of the previous reference sample of α may be considered zero, and the value of the subsequent reference sample of α + β may be considered zero.

In operation 1830, the reconstructed block generator may generate a reconstructed first prediction error based on the reconstructed residual signal and the second prediction signal. The reconstructed block generator may generate the reconstructed first prediction error by adding the second prediction signal to the reconstructed residual signal.

The reconstructed block generator may generate a reconstructed first prediction error of the current block based on the reconstructed residual signal and the second prediction signal for the partial region.

The reconstructed first prediction error may be the sum of the reconstructed residual signal and the second prediction signal. Alternatively, the reconstructed first prediction error may be generated based on the reconstructed residual signal and the second prediction signal. Alternatively, the reconstructed first prediction error may be generated based on the sum of the reconstructed residual signal and the second prediction signal.

In operation 1840, the reconstructed block generator may generate a first prediction signal by performing a first prediction on the current block.

The first prediction signal may represent the first prediction block.

In the first prediction, reconstructed blocks around the current block can be used as reference blocks. Also, pixels of reconstructed blocks around the current block can be used as reference samples.

In operation 1850, the reconstructed block generator may generate a signal of the reconstructed block based on the reconstructed first prediction error and the first prediction signal. The reconstructed block generator may generate a signal of the reconstructed block by adding the first prediction signal to the reconstructed first prediction error.

According to the steps 1810, 1820, 1830, 1840, and 1850 described above, when the second prediction is used, the reconstructed block may be the sum of the reconstructed residual signal, the second prediction signal, and the first prediction signal. Alternatively, when the second prediction is used, a signal representing the reconstructed block may be generated based on the reconstructed residual signal, the second prediction signal, and the first prediction signal.

According to the above-described steps 1810, 1820, 1830, 1840, and 1850, when the second prediction is not used, the signal representing the reconstructed block may be the sum of the reconstructed residual signal and the first prediction signal. Or, if the second prediction is not used, a signal representing the reconstructed block may be generated based on the sum of the reconstructed residual signal and the first prediction signal.

The reconstructed block and the reconstructed first residual signal may be used as reference blocks for decoding other blocks.

19 illustrates a decoding process according to an embodiment.

In Figure 19, plus and plus

The symbol may represent generating a signal that combines two signals. A signal that is the sum of the two signals can be generated by adding input signals that are respectively input as + symbols.

The reconstructed block generator described above with reference to FIG. 19 may include a first predictor and a second predictor.

The process within the dashed line may represent the SSIP.

Rhombus may represent a conditional divergence.

(TQ) ⁻¹ may represent inverse transformation and inverse quantization.

May be a prediction error. or,

May be a reconstructed residual signal.

May represent a signal of the reconstructed first prediction error.

May represent a signal of the reconstructed second prediction error. If a second prediction is used,

Is

May be the same as Or, if a second prediction is used,

Is

Can be If no second prediction is used,

Is

May be the same as Or, if the second prediction is not used,

Is

Can be

May represent the first prediction signal.

May represent a second prediction signal.

According to the above-described embodiment, when the second prediction is used, the selected part to which the first prediction is applied and the remaining part to which the second prediction is applied may be separated.

In one embodiment, when the second prediction is used, the first prediction may be applied to the entirety of the current block. In this case, the reconstructed block for the selected portion to which the second prediction is applied is based on the sum of the reconstructed residual block, the second prediction signal generated by the second prediction, and the first prediction signal generated by the first prediction. Can be generated. In addition, a reconstructed block for the remaining part to which the second prediction is not applied may be generated based on the sum of the reconstructed residual block and the first prediction signal generated by the first prediction.

In this case, the second prediction may be a prediction for the residual. The second prediction may be a prediction for a first residual signal that is a difference between the current block and the first prediction signal. The reference block of the second prediction may be a reconstructed residual block, and the reference pixel of the second prediction may be a pixel of the reconstructed residual block. In other words, a second residual signal, which is a difference between the first residual signal and the second prediction signal, may be generated through the second prediction on the first residual signal, and encoded information of the current block is generated using the second residual signal. Can be.

Alternatively, in this case, the reconstructed block for the selected portion to which the second prediction is applied may be a weighted-sum of the reconstructed residual block, the second prediction signal, and the first prediction signal. The information related to the prediction may include a first weight of the reconstructed residual block, a second weight of the second prediction signal, and a third weight of the first prediction signal. The fact that the second prediction is not used may be considered that the second weight of the second prediction signal is set to zero. Alternatively, the second weight of the second prediction signal may be considered to be set to 0 with respect to the remaining part to which the second prediction is not applied.

The above description of encoding of the current block may also be applied to decoding of the current block. Duplicate explanations are omitted. In addition, the above description of decoding of the current block may be applied to encoding of the current block. Duplicate explanations are omitted.

In the above-described embodiments, the methods are described based on a flowchart as a series of steps or units, but the present invention is not limited to the order of steps, and certain steps may occur in a different order or simultaneously from other steps as described above. Can be. Also, one of ordinary skill in the art appreciates that the steps shown in the flowcharts are not exclusive, that other steps may be included, or that one or more steps in the flowcharts may be deleted without affecting the scope of the present invention. I can understand.

Embodiments according to the present invention described above may be implemented in the form of program instructions that may be executed by various computer components, and may be recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be those specially designed and configured for the present invention, or may be known and available to those skilled in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs, DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the process according to the invention, and vice versa.

Although the present invention has been described by specific embodiments such as specific components and the like, but the embodiments and the drawings are provided to assist in a more general understanding of the present invention, the present invention is not limited to the above embodiments. For those skilled in the art, various modifications and variations can be made from these descriptions.

Accordingly, the spirit of the present invention should not be limited to the above-described embodiments, and all of the equivalents or equivalents of the claims, as well as the appended claims, fall within the scope of the spirit of the present invention. I will say.

Claims (20)

1. Generating a residual signal of the current block based on the current block, the first prediction, and the second prediction; And

   Generating information on the encoded residual signal by performing encoding on the residual signal

   Including,

   And the second prediction is a prediction for a prediction error of the first prediction.
2. A reconstructed residual signal generator for generating a reconstructed residual signal for the current block; And

   A reconstructed block generator for generating a reconstructed block for the current block based on the reconstructed residual signal, the second prediction, and the first prediction

   It includes, the decoding method.
3. Generating a reconstructed residual signal for the current block; And

   Generating a reconstructed block for the current block based on the reconstructed residual signal, a second prediction and a first prediction

   It includes, the decoding method.
4. The method of claim 3,

   And the reconstructed block corresponds to the sum of the reconstructed residual signal, the first prediction signal generated by the first prediction, and the second prediction signal generated by the second prediction.
5. The method of claim 3,

   And the first prediction and the second prediction are intra prediction.
6. The method of claim 2,

   The prediction direction of the second prediction is the same as the first prediction direction.
7. The method of claim 3,

   And at least some of the plurality of first reference blocks of the first prediction and the plurality of second reference blocks of the second prediction are different from each other.
8. The method of claim 3,

   The kind of the first reference block of the first prediction and the kind of the second reference block of the second prediction are different from each other.
9. The method of claim 3,

   The first reference block of the first prediction is a reconstructed neighbor block neighboring the current block,

   The second reference block of the second prediction is a reconstructed neighbor residual block neighboring the current block,

   And the reconstructed neighbor residual block is a difference between the reconstructed neighboring block and a first prediction block of the reconstructed neighboring block.
10. The method of claim 3,

    And the region of the reference sample of the second prediction is part of the region of the reference sample of the first prediction.
11. The method of claim 3,

    And a region to which the second prediction is to be applied is specified based on a range of reference samples used for the second prediction.
12. The method of claim 3,

    And a region to which the second prediction is to be applied is specified based on a prediction direction of the second prediction.
13. The method of claim 11,

    And the range of reference samples is specified by information indicating a first reference sample to be used for the second prediction and information indicating the number of reference samples to be used for the second prediction.
14. The method of claim 11,

    And the region to which the second prediction is to be applied is a region generated as the reference samples of the range advance in the prediction direction of the second prediction.
15. The method of claim 11,

    And a value of a reference sample outside of the range is considered null or zero.
16. The method of claim 3,

    And the second prediction is not used when the second prediction usage information indicating whether or not the second prediction is used in encoding the current block indicates that the second prediction is not used.
17. The method of claim 3,

    If the prediction mode of the first prediction is a non-directional mode, the second prediction is not used.
18. The method of claim 3,

    If the current block is adjacent to an upper boundary and a left boundary, the second prediction is not used;

    Wherein the boundary is a picture boundary, a slice boundary, or a tile boundary.
19. The method of claim 3,

    Whether the second prediction is to be used is determined based on the number of reconstructed blocks adjacent to the current block.
20. The method of claim 3,

    And when the second prediction is not used, the reconstructed block is generated based on the reconstructed residual signal and the first prediction signal by the first prediction.

Images