KR102356481B1 - Method and Apparatus for Video Encoding and Video Decoding - Google Patents

Abstract

The present invention relates to prediction of video information and video encoding/decoding using the same, wherein the video encoding according to the present invention includes decoding a picture of a reference layer, sampling the decoded picture of the reference layer to derive an inter-layer reference picture constructing a reference picture list including the inter-layer reference picture and the reference picture of the current layer; predicting the current block by performing prediction on the current block of the current layer based on the reference picture list generating a sample; generating a residual sample for the current block based on the prediction sample; and encoding the residual sample.

Description

A video encoding method, a video decoding method, and an apparatus using the same

The present invention relates to a method for video encoding and decoding and an apparatus using the same, and more particularly, to a method of using information of a corresponding reference picture in a reference layer to generate a prediction sample for an encoding or decoding object block of a current layer; and It's about the device.

Recently, as broadcasting services with high definition (HD) resolution are expanding not only domestically but also worldwide, many users are getting used to high-resolution and high-definition images, and accordingly, many institutions are spurring the development of next-generation imaging devices. Also, along with HDTV, as interest in Ultra High Definition (UHD) having a resolution four times higher than that of HDTV is increasing, compression technology for higher resolution and high-definition images is required.

For image compression, inter prediction technology that predicts pixel values included in the current picture from temporally previous and/or subsequent pictures, predicts pixel values included in the current picture using pixel information in the current picture An intra prediction technique, an entropy encoding technique in which a short code is assigned to a symbol having a high frequency of occurrence and a long code is assigned to a symbol having a low frequency of occurrence, etc. may be used.

In video compression technology, there is a technology that provides a constant network bandwidth under a limited operating environment of hardware without considering a flexible network environment. However, a new compression technique is required to compress image data applied to a network environment in which bandwidth is frequently changed, and for this purpose, a scalable video encoding/decoding method may be used.

Accordingly, there is an increasing demand to efficiently transmit data to various transmission environments and various terminals by generating one integrated data that can support various spatial resolutions and various frame-rates. .

In this regard, as a video encoding technology for supporting efficient data transmission, standardization of scalable high efficiency video coding (SHVC) based on HEVC is in progress.

The present invention provides a method and apparatus capable of performing prediction using information of another layer by adding a decoded picture of a lower layer (reference layer) to a list of reference pictures for encoding/decoding target in the current layer. do.

The present invention also provides a method and apparatus for generating a prediction signal such that the difference signal between the original signal and the prediction signal is minimized by performing prediction (eg, motion prediction) on a decoded picture of a lower layer (reference layer). intended to provide

The present invention also provides an adaptive lower layer (reference layer) using encoding information, depth information of a prediction structure, etc. when adding a decoded picture of a lower layer (reference layer) to a reference picture list for a current encoding/decoding target block. An object of the present invention is to provide a method and apparatus for determining a position in a reference picture list of a decoded picture of a reference layer).

An embodiment of the present invention provides a video encoding method, comprising: decoding a picture of a reference layer; deriving an inter-layer reference picture by sampling a decoded picture of the reference layer; reference of the inter-layer reference picture and the current layer Constructing a reference picture list including pictures, performing prediction on the current block of the current layer based on the reference picture list to generate a prediction sample for the current block, based on the prediction sample, The method may include generating a residual sample for the current block and encoding the residual sample.

Another embodiment of the present invention is a video decoding method, comprising: decoding a picture of a reference layer; deriving an inter-layer reference picture by sampling the decoded picture of the reference layer; and reference of the inter-layer reference picture and the current layer Constructing a reference picture list including a picture, performing prediction on the current block of the current layer based on the reference picture list to generate a prediction sample for the current block, and the prediction sample and the current block and generating a reconstructed sample for the current block based on the residual sample for .

Another embodiment of the present invention provides an inter prediction method of video information, comprising the steps of decoding a picture of a reference layer, sampling the decoded picture of the reference layer to derive an inter-layer reference picture, the inter-layer reference picture and Constructing a reference picture list including a reference picture of the current layer and generating a prediction sample for the current block by performing prediction on the current block of the current layer based on the reference picture list have.

According to the present invention, in constructing a reference picture list to perform prediction (eg, motion prediction and motion compensation, etc.) on an encoding/decoding target picture of an upper layer (current layer), decoding of a lower layer (reference layer) A reference picture list may be constructed including the referenced picture.

According to the present invention, when the decoded image of the lower layer (reference layer) is added to the reference picture list, the reference picture list of the decoded image of the lower layer (reference layer) using encoding information, prediction structure depth information, etc. I can adaptively determine my location.

In addition, according to the present invention, a decoded picture of a lower layer (reference layer) is used as a reference picture, and encoding efficiency is increased by adaptively determining the position of a decoded picture of a lower layer (reference layer) in the reference picture. can

1 is a block diagram illustrating a configuration of an image encoding apparatus according to an embodiment.
2 is a block diagram illustrating a configuration of a video decoding apparatus according to an embodiment.
3 is a conceptual diagram schematically illustrating an embodiment of a scalable video coding structure using a plurality of layers to which the present invention can be applied.
4 is a flowchart schematically illustrating a method for performing inter-layer prediction in an encoding apparatus/decoding apparatus according to the present invention.
5 is a diagram schematically illustrating an example of a method of constructing a reference picture list according to an embodiment of the present invention.
6 is a diagram schematically illustrating an example of temporal levels for each picture in a current layer.
7 is a diagram schematically illustrating an example of depth information of a structure of prediction of a current layer.
8 is a flowchart schematically illustrating an example of a video encoding method according to the present invention.
9 is a flowchart schematically illustrating an example of a video decoding method according to the present invention.

In describing the embodiments of the present specification, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present specification, the detailed description thereof will be omitted.

When it is said that a component is “connected” or “connected to” another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be In addition, the description of “including” a specific configuration in the present invention does not exclude configurations other than the corresponding configuration, and it means that additional configurations may be included in the practice of the present invention or the scope of the technical spirit of the present invention.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

In addition, the components shown in the embodiment of the present invention are shown independently to represent different characteristic functions, and it does not mean that each component is made of separate hardware or a single software component. That is, each component is listed as each component for convenience of description, and at least two components of each component are combined to form one component, or one component can be divided into a plurality of components to perform a function, and each Integrated embodiments and separate embodiments of the components are also included in the scope of the present invention without departing from the essence of the present invention.

In addition, some of the components are not essential components for performing essential functions in the present invention, but may be optional components for merely improving performance. The present invention can be implemented by including only essential components to implement the essence of the present invention, except for components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement Also included in the scope of the present invention.

The present invention relates to a technique for encoding and decoding an image having a structure including a plurality of layers, and to a method and an apparatus for performing prediction on an upper layer using information of a lower layer when encoding/decoding a higher layer.

More specifically, according to the present invention, in generating a reference picture list used for motion prediction and/or motion compensation for an encoding/decoding target picture of an upper layer (current layer), a lower layer (reference layer) ), a reference picture list may be generated including the decoded picture.

In generating the reference picture list of the upper layer (the current layer, hereinafter referred to as the 'current layer') including the decoded picture of the lower layer (reference layer), adaptively using encoding information and depth information of the prediction structure, etc. By adding a decoded picture of a lower layer (referred to as a reference layer, hereinafter referred to as a 'reference layer'), encoding efficiency may be increased.

Hereinafter, the present invention will be described in detail with reference to the drawings.

1 is a block diagram illustrating a configuration of an image encoding apparatus according to an embodiment.

A scalable video encoding/decoding apparatus for a multi-layer structure may be implemented by extending a video encoding/decoding apparatus for a single-layer structure.

1 illustrates an embodiment of a video encoding apparatus that is applicable to a multi-layer structure, that is, provides scalability.

Referring to FIG. 1 , the video encoding apparatus 100 includes an inter prediction unit 110 , an intra prediction unit 120 , a switch 115 , a subtractor 125 , a transform unit 130 , a quantization unit 140 , and entropy. It includes an encoder 150 , an inverse quantizer 160 , an inverse transform unit 170 , an adder 175 , a filter unit 180 , and a reference picture buffer 190 .

The video encoding apparatus 100 may encode an input image in an intra mode or an inter mode and output a bitstream. Intra prediction refers to intra prediction, and inter prediction refers to inter prediction. In the intra mode, the switch 115 is switched to the intra mode, and in the inter mode, the switch 115 is switched to the inter mode. After generating a prediction block for a block (current block) of an input picture, the video encoding apparatus 100 may encode a difference between the current block and the prediction block.

In the intra mode, the intra prediction unit 120 may use a pixel value of an already-encoded block around the current block as a reference pixel. The intra prediction unit 120 may perform spatial prediction using a reference call and generate prediction samples for the current block.

In the inter mode, the inter prediction unit 110 may obtain a motion vector specifying a reference block having the smallest difference from the input block (current block) in the reference picture stored in the reference picture buffer 190 . The inter prediction unit 110 may generate a prediction block for the current block by performing motion compensation using the motion vector and the reference picture stored in the reference picture buffer 190 .

In the case of a multi-layer structure, inter prediction applied in the inter mode may include inter-layer prediction. The inter prediction unit 110 may sample a picture of a reference layer to construct an inter-layer reference picture, and may perform inter-layer prediction by including the inter-layer reference picture in a reference picture list. A reference relationship between layers may be signaled through information specifying a dependency between layers.

Meanwhile, when the current layer picture and the reference layer picture have the same size, sampling applied to the reference layer picture may mean copying a sample from the reference layer picture or generating a reference sample by interpolation. When the resolutions of the current layer picture and the reference layer picture are different, sampling applied to the reference layer picture may mean upsampling.

For example, when resolutions between layers are different, an inter-layer reference picture may be configured by upsampling a reconstructed picture of a reference layer between layers supporting resolution-related scalability.

Whether the picture of which layer is used to construct the inter-layer reference picture may be determined in consideration of encoding cost and the like. The encoding apparatus may transmit information specifying a layer to which a picture to be used as an inter-layer reference picture belongs to the decoding apparatus.

Also, a layer referenced in inter-layer prediction, that is, a picture used for prediction of a current block in a reference layer, may be a picture of the same access unit (AU) as the current picture (prediction target picture in the current layer).

The subtractor 125 may generate a residual block (residual signal) by the difference between the current block and the prediction block.

The transform unit 130 may perform transform on the residual block to output a transform coefficient. When a transform skip mode is applied, the transform unit 130 may skip transform for the residual block.

The quantizer 140 may quantize a transform coefficient according to a quantization parameter and output a quantized coefficient.

The entropy encoding unit 150 may entropy-encode the values calculated by the quantization unit 140 or the encoding parameter values calculated during the encoding process according to a probability distribution to output a bitstream. The entropy encoder 150 may entropy-encode information for video decoding (eg, a syntax element, etc.) in addition to pixel information of the video.

The encoding parameter is information necessary for encoding and decoding, and may include information that is encoded in the encoding apparatus and transmitted to the decoding apparatus, such as a syntax element, as well as information that can be inferred during encoding or decoding.

Coding parameters include, for example, intra/inter prediction modes, motion/motion vectors, reference image indexes, coding block patterns, presence or absence of residual signals, transform coefficients, quantized transform coefficients, quantization parameters, block size, block partition information, etc. or statistics.

The residual signal may mean a difference between an original signal and a predicted signal, and a signal in which the difference between the original signal and the predicted signal is transformed or a signal in which the difference between the original signal and the predicted signal is transformed and quantized. may mean The residual signal may be referred to as a residual block in block units.

When entropy encoding is applied, a small number of bits are allocated to a symbol having a high probability of occurrence and a large number of bits are allocated to a symbol having a low probability of occurrence to express the symbol, so that the size of the bit stream for the symbols to be encoded is increased. can be reduced. Accordingly, compression performance of image encoding may be improved through entropy encoding.

For entropy encoding, an encoding method such as exponential golomb, Context-Adaptive Variable Length Coding (CAVLC), or Context-Adaptive Binary Arithmetic Coding (CABAC) may be used. For example, the entropy encoding unit 150 may perform entropy encoding using a variable length coding (VLC) table. In addition, the entropy encoder 150 derives a binarization method of a target symbol and a probability model of a target symbol/bin, and then performs entropy encoding using the derived binarization method or probability model. You may.

The quantized coefficient may be inverse quantized by the inverse quantization unit 160 and inversely transformed by the inverse transform unit 170 . The inverse-quantized and inverse-transformed coefficients may be added to the prediction block through the adder 175 and a reconstructed block may be generated.

The reconstructed block passes through the filter unit 180, and the filter unit 180 applies at least one of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the reconstructed block or the reconstructed picture. can do. The reconstructed block passing through the filter unit 180 may be stored in the reference image buffer 190 .

2 is a block diagram illustrating a configuration of a video decoding apparatus according to an embodiment.

A scalable video encoding/decoding apparatus for a multi-layer structure may be implemented by extending a video encoding/decoding apparatus for a single-layer structure.

2 illustrates an embodiment of a video decoding apparatus applicable to a multi-layer structure, that is, providing scalability.

Referring to FIG. 2 , the video decoding apparatus 200 includes an entropy decoding unit 210 , an inverse quantization unit 220 , an inverse transform unit 230 , an intra prediction unit 240 , an inter prediction unit 250 , and a filter unit. 260 and a reference picture buffer 270 .

The video decoding apparatus 200 may receive a bitstream output from the encoding apparatus, perform decoding in an intra mode or an inter mode, and output a reconstructed image, that is, a reconstructed image.

In the intra mode, the switch may be switched for intra prediction, and in the inter mode, the switch may be switched for inter prediction.

The video decoding apparatus 200 obtains a reconstructed residual block from the received bitstream, generates a prediction block, and adds the reconstructed residual block and the prediction block to generate a reconstructed block, that is, a reconstructed block. have.

The entropy decoding unit 210 may entropy-decode the input bitstream according to a probability distribution and output information such as a quantized coefficient and a syntax element.

The quantized coefficient is inverse quantized by the inverse quantization unit 220 and inversely transformed by the inverse transformation unit 230 . A reconstructed residual block may be generated by inverse quantizing/inverse transforming the quantized coefficient.

In the intra mode, the intra prediction unit 240 may perform spatial prediction using pixel values of an already coded block around the current block, and generate a prediction block for the current block.

In the inter mode, the inter prediction unit 250 may generate a prediction block for the current block by performing motion compensation using a motion vector and a reference picture stored in the reference picture buffer 270 .

In the case of a multi-layer structure, inter prediction applied in the inter mode may include inter-layer prediction. The inter prediction unit 250 may sample a picture of a reference layer to construct an inter-layer reference picture, and may perform inter-layer prediction by including the inter-layer reference picture in a reference picture list. A reference relationship between layers may be signaled through information specifying a dependency between layers.

Meanwhile, when the current layer picture and the reference layer picture have the same size, sampling applied to the reference layer picture may mean copying a sample from the reference layer picture or generating a reference sample by interpolation. When the resolutions of the current layer picture and the reference layer picture are different, sampling applied to the reference layer picture may mean upsampling.

For example, in this case, if inter-layer prediction is applied between layers supporting resolution-related scalability when resolutions between layers are different, an inter-layer reference picture may be configured by upsampling the reconstructed picture of the reference layer.

In this case, information specifying a layer to which a picture to be used as an inter-layer reference picture belongs may be transmitted from the encoding apparatus to the decoding apparatus.

Also, a layer referenced in inter-layer prediction, that is, a picture used for prediction of a current block in a reference layer, may be a picture of the same access unit (AU) as the current picture (prediction target picture in the current layer).

The reconstructed residual block and the prediction block are added by an adder 255 to generate a reconstructed block. In other words, a reconstructed sample or a reconstructed picture is generated by adding the residual sample and the prediction sample.

The reconstructed picture is filtered by the filter unit 260 . The filter unit 260 may apply at least one of a deblocking filter, SAO, and ALF to a reconstructed block or a reconstructed picture. The filter unit 260 outputs a reconstructed or filtered reconstructed picture. The reconstructed image may be stored in the reference picture buffer 270 and used for inter prediction.

Also, the image decoding apparatus 200 may further include a parsing unit (not shown) that parses information related to the encoded image included in the bitstream. The parsing unit may include the entropy decoding unit 210 or may be included in the entropy decoding unit 210 . This parsing unit may also be implemented as one component of the decoding unit.

1 and 2, it has been described that one encoding apparatus/decoding apparatus processes both encoding/decoding for multi-layers, but this is for convenience of description, and the encoding apparatus/decoding apparatus may be configured for each layer.

In this case, the encoding/decoding apparatus of the higher layer may perform encoding/decoding of the corresponding higher layer by using the information of the higher layer and the information of the lower layer. For example, the prediction unit (inter prediction unit) of the higher layer may perform intra prediction or inter prediction on the current block using pixel information or picture information of the upper layer, and receives picture information reconstructed from the lower layer and It may be used to perform inter prediction (inter-layer prediction) on the current block of a higher layer. Here, only inter-layer prediction has been described as an example, but the encoding/decoding apparatus performs encoding/decoding on the current layer using information of another layer, regardless of whether the encoding apparatus/decoding apparatus is configured for each layer or whether one apparatus processes multi-layers. can do.

In the present invention, a layer may include a view. In this case, in the case of inter-layer prediction, the prediction of the upper layer is not simply performed using the information of the lower layer, but information of another layer between the layers specified as having dependence by the information specifying the inter-layer dependence. Inter-layer prediction may be performed using .

3 is a conceptual diagram schematically illustrating an embodiment of a scalable video coding structure using a plurality of layers to which the present invention can be applied. In FIG. 3 , a group of pictures (GOP) indicates a picture group, that is, a group of pictures.

Video data is transmitted from a video encoding apparatus to a video decoding apparatus through a transmission medium or a storage medium. Video data may be different for each transmission medium according to a network environment, and there may also be a difference for each storage medium. A scalable video coding method may be provided to efficiently support these various video data.

In the scalable video coding method, inter-layer sample information (texture information), motion information, a residual signal, and the like are utilized. Through this, it is possible to increase encoding/decoding performance by removing redundancy between layers. The scalable video coding method may provide various scalability in terms of spatial, temporal, and picture quality according to ambient conditions such as a transmission bit rate, a transmission error rate, and a system resource.

Scalable video coding may be performed using a multiple layer structure. The multi-layer structure may include a base layer that can be independently encoded/decoded without using information of other layers. In addition, the multi-layer structure may include an enhancement layer that is encoded/decoded using information of the base layer and/or other layers.

In the layer structure, a layer having high scalability may be referred to as an upper layer, and a layer having low scalability may be referred to as a lower layer. In addition, a layer referenced for encoding/decoding (eg, prediction) of another layer may be referred to as a reference layer, and a layer encoded/decoded (eg, predicted) using another layer may be referred to as a current layer. The reference layer may be a lower layer than the current layer, and the current layer may be a higher layer than the reference layer.

Here, a layer is an image and bit classified based on spatial (eg, image size, resolution), temporal (eg, frame rate), image quality, complexity, view, etc. It may be a set of streams (bitstream).

Also, the plurality of layers may have a dependency on each other. The current layer may be encoded/decoded with reference to the layer in which the dependency exists.

Referring to FIG. 3 , for example, the first enhancement layer may be defined as a first resolution higher than the base resolution of the base layer, a first frame rate higher than the frame rate of the base layer, and a first bit rate higher than the bit rate of the base layer. can In addition, the second enhancement layer is defined as a second resolution higher than the resolution of the first enhancement layer, a second frame rate higher than the frame rate of the first enhancement layer, and a second bit rate higher than the bit rate of the first enhancement layer can be

For example, if the first enhancement layer is defined with high definition (HD), a frame rate of 30 Hz, and a bit rate of 3.9 Mbps, the second enhancement layer is configured with 4K-UHD (ultra high definition), a frame rate of 60 Hz, and a bit rate of 27.2 Mbps. can be defined. The format, frame rate, bit rate, and the like are examples, and may be differently determined as necessary. Also, the number of layers to be used is not limited to the present embodiment and may be determined differently according to circumstances.

For example, if the transmission bandwidth is 4 Mbps, the frame rate of the first enhancement layer HD may be reduced to transmit at 15 Hz or less. The scalable video coding method may provide temporal, spatial, image quality, and inter view scalability by the method described above in the embodiment of FIG. 3 .

In the case of encoding and decoding of video supporting a plurality of layers in the bitstream, that is, scalable coding, there is a strong correlation between the plurality of layers. Duplicate elements can be removed and the encoding performance of an image can be improved.

Performing prediction on a picture of a current layer, which is a target of prediction, using information of another layer is called inter-layer prediction.

Scalable video coding has the same meaning as scalable video coding in terms of encoding and scalable video decoding in terms of decoding.

The plurality of layers may have different at least one of resolution, frame rate, color format, and viewpoint, and layer upsampling or downsampling may be performed to adjust resolution when inter-layer prediction is performed.

In general, inter prediction may use at least one of a picture before or after a picture of the current picture as a reference picture, and may perform prediction on the current block based on the reference picture. An image used for prediction of the current block is referred to as a reference picture or a reference frame.

A region referenced for prediction of the current picture in the reference picture may be specified using a reference picture index (refIdx) indicating the reference picture, a motion vector, and the like.

Inter prediction may generate a prediction block for the current block by selecting a reference picture and a reference block corresponding to the current block in the reference picture.

In inter prediction, the encoding apparatus and the decoding apparatus may derive motion information on the current block and then perform inter prediction and/or motion compensation based on the motion information. In this case, the encoding apparatus and the decoding apparatus perform a col corresponding to the current block in a collocated picture selected from motion information and/or reference pictures of a neighboring block of the current block. Prediction of the current block may be performed using motion information of a collocated block.

1 and 2, in inter prediction (inter picture prediction), at least one of a picture before or after a picture of the current picture is determined as a reference picture, and based on the reference picture, the current picture (current block within the current picture) make predictions on

The encoding apparatus transforms and/or quantizes the residual signal that is the difference between the prediction block and the current block derived as a result of prediction (the difference between the original signal and the prediction signal), then applies entropy decoding and transmits it to the decoding apparatus.

The decoding apparatus may reconstruct the current block (original signal) by inverse quantizing and/or inverse transforming the residual received from the encoding apparatus through entropy decoding, and then adding it to the derived prediction block (prediction signal) as a result of prediction.

When the PCM mode is applied, transform/quantization and inverse quantization/inverse transform may be omitted, and when the transform skip mode is applied, transform and inverse transform may be omitted. In addition, when the skip mode at the coding unit (CU) level is applied as a method of inter prediction, since there is no transmission of a residual, the prediction signal may be used as the original signal.

More specifically, inter prediction will be described. In the case of inter prediction, the encoding apparatus may generate a prediction signal to satisfy at least one of residual minimization and motion vector minimization. For example, the encoding apparatus may generate the prediction signal to minimize the motion vector while minimizing the residual.

The encoding apparatus may transmit information specifying motion information to the decoding apparatus so that the same prediction signal can be derived also in the decoding apparatus. The decoding apparatus may derive motion information based on information received from the encoding apparatus and generate a prediction signal.

In order to specify information of a reference picture in inter prediction, information of neighboring blocks of a current block may be used. For example, in the skip mode, the merge mode, and the AMVP (Advanced Motion Vector Prediction) mode using mvd (Motion Vector Difference) and mvp (Motion Vector Prediction), which are inter prediction methods, motion information of neighboring blocks is Based on this, it is possible to generate a prediction block for the current block.

As described above, in the skip mode, motion information of a neighboring block may be used as motion information of the current block. For example, information specifying information used as the reference picture index and/or motion vector of the current block among the reference picture index and/or motion vector of the neighboring block is transmitted from the encoding device to the decoding device, but additional information such as residual not sent

When the merge mode is applied as the inter prediction mode, the encoding apparatus/decoding apparatus may derive a prediction block (prediction signal) for the current block by using motion information of the neighboring block as it is. The encoding apparatus may transmit information indicating whether the merge mode is applied to the current block, information indicating which neighboring block information is used, residual information on the current block, and the like, to the decoding apparatus.

When the merge mode is applied, the decoding apparatus generates a prediction block (prediction sample) for the current block using motion information of a neighboring block indicated by the encoding apparatus, and restores the current block (reconstructed sample) by adding it to the residual signal. can be induced).

The motion information of the neighboring block used in the skip mode and the merge mode may include a reference picture index and a motion vector of the neighboring block.

If AMVP is applied, the encoding apparatus/decoding apparatus may predict the motion vector of the current block by using the motion vector of the neighboring block. The encoding apparatus transmits, to the decoder, information specifying which neighboring block motion information is used, information specifying a difference (mvd) between the motion vector of the current block and a predicted motion vector, and a reference picture index specifying a reference picture. can

When AMVP is applied, the decoding apparatus may generate a prediction block (prediction sample) for the current block by using motion information (motion vector) of the neighboring block and the reference picture index transmitted from the encoder. For example, the decoding apparatus may generate the motion vector predictor of the current block based on the motion vector of the neighboring block, and may derive the motion vector of the current block by adding the motion vector difference (mvd) received from the encoding apparatus and the motion vector predictor. . The decoding apparatus may derive a prediction block (prediction sample) for the current block by using the derived motion vector and the received reference picture index. The decoding apparatus may derive reconstructed samples for the current picture by reconstructing the current block by adding the prediction sample to the residual.

When inter prediction is performed, motion information used by the decoder device may be derived based on information received from the encoder. For example, when the skip mode is applied, the motion information of the block indicated by the merge index may be used as the motion information of the current block. When the merge mode is applied, the motion information of the block indicated by the merge index may be used as the motion information of the current block. In this case, some of the motion information of temporal neighboring blocks may be fixed to a specific value or derived separately. In the case of AMVP, the motion vector of the current block may be predicted from the motion vector of the neighboring block, and the reference picture index may be separately signaled.

A processing unit in which prediction is performed may be different from a unit in which a method of prediction is determined. For example, whether intra prediction or inter prediction is applied may be determined in units of coding units (CUs), and the modes of intra prediction and inter prediction may be determined in units of prediction units (PUs). Intra prediction may be performed in units of transform units (TUs), and inter prediction may be performed in units of prediction units.

Meanwhile, in scalable video coding to which a multi-layer structure is applied, prediction of a current layer may be performed using information of another layer. For example, in performing prediction on a current block in a current picture of a current layer that is a decoding/encoding target layer, information of another layer (reference layer) may be used.

In this case, picture information of the reference layer may be used in a manner that uses texture information of another layer. In this case, prediction may be performed using only the decoded picture information of the reference layer corresponding to the encoding/decoding target block (the current block).

However, when prediction for the current layer is performed (inter-layer prediction is performed) using texture information of a predetermined region corresponding to the reference layer, signal distortion may occur based on a difference in image size between layers, etc. , the residual signal may increase.

Therefore, in the present invention, as described with reference to FIGS. 1 and 2 , when performing inter-layer prediction, the encoding apparatus/decoding apparatus (eg, the inter-prediction unit in the encoding apparatus/decoding apparatus) is used for inter-layer prediction in the reference layer. An inter-layer reference picture is generated by performing sampling on a picture to be used, and inter prediction is performed on the current block by including the inter-layer reference picture in the reference picture list for the current block.

4 is a flowchart schematically illustrating a method for performing inter-layer prediction in an encoding apparatus/decoding apparatus according to the present invention.

In FIG. 4 , for convenience of explanation, a case in which inter-layer prediction is performed by a prediction unit in an encoding apparatus/decoding apparatus will be described as an example. In this case, the prediction unit may include an inter prediction unit.

Referring to FIG. 4 , the encoding apparatus/decoding apparatus decodes a picture of a reference layer ( S410 ). As described above, the reference layer refers to a layer referenced by an encoding/decoding target block of the current layer. In this case, the prediction unit may decode a picture of the same AU as the current picture (a picture to which the current block in the current layer belongs) from among the pictures of the reference layer and use it for prediction of the current block.

Decoding of the picture of the reference layer referenced by the current encoding target block may be performed based on information about the encoding method of the reference layer transmitted from a video parameter set (VPS) or a sequence parameter set (SPS).

For example, when the reference layer is encoded according to the MPEG-2 method, the encoding apparatus/decoding apparatus may decode the picture of the reference layer according to the MPEG-2 method.

Also, if the reference layer is encoded by the H.264/AVC method, the encoding apparatus/decoding apparatus may decode the picture of the reference layer according to the H.264/AVC method.

Also, if the reference layer is encoded by the HEVC method, the encoding apparatus/decoding apparatus may decode a picture of the reference layer according to the HEVC method.

The encoding apparatus/decoding apparatus may construct a reference picture list for inter prediction of the current block (S420). The prediction unit may generate a reference picture list including the decoded pictures of the reference layer for inter prediction of the current encoding/decoding target block (the current block).

In video encoding/decoding using a multi-layer structure, the prediction unit may perform prediction on an encoding/decoding target block of a higher layer (current layer) using a decoded picture of a reference layer.

In this case, in using the decoded picture of the reference layer, the decoded picture of the reference layer is included in the reference image list for the encoding/decoding target block (current block) of the higher layer (current layer), prediction can be made.

The reference picture list may be constructed when decoding for a slice starts. For example, in the case of a P slice, one reference picture list may be configured, and in the case of a B slice, two reference picture lists may be configured. Accordingly, the reference picture list for the current block may be configured for each slice to which the current block belongs.

In this regard, there is a problem in which position the decoded picture of the reference layer is to be placed in the reference picture list.

According to embodiments of the present invention (1) when constructing the reference picture list, the decoded picture of the reference layer is always added to a fixed position, or (2) when constructing the reference picture list, in the reference picture list The position of the decoded picture of the reference layer is determined using encoding information, or (3) when constructing the reference picture list, the position of the decoded picture of the reference layer in the reference picture list can be determined using encoding structure information. have.

Hereinafter, a method of determining a position of a decoded picture (inter-layer reference picture) of a reference layer in a reference picture list in the present invention will be described in detail.

According to an embodiment of the present invention, whenever the reference picture list is generated, the decoded picture of the reference layer may be added at a fixed position.

5 is a diagram schematically illustrating an example of a method of constructing a reference picture list according to an embodiment of the present invention.

Referring to FIG. 5 , <1> is a reference picture list configured without considering decoded pictures of a reference layer, and <2> is a reference picture list configured including decoded pictures of a reference layer.

In the example of FIG. 5 , when the reference picture list composed of only pictures of the current layer is equal to <1>, the decoded picture (inter-layer reference picture) of the reference layer may be added to the second position of the reference picture list.

Here, for convenience of explanation, it has been described that the inter-layer reference picture is located at the second position of the reference picture list, but the present invention is not limited thereto, and the inter-layer reference picture (decoded picture of the reference layer) is the reference picture list. It may be added to my other (not the second) fixed predetermined location. In this case, the position of the inter-layer reference picture in the reference picture list may be the same position with respect to the entire video.

In FIG. 5 , a reference picture list L0 is a reference picture list used for inter prediction of a P slice or a reference picture list used as a first reference picture list in inter prediction of a B slice. The reference picture list L1 is a second reference picture list used for inter prediction of a B slice. In addition, a P slice (predictive slice) is a slice on which intra prediction is performed or inter prediction is performed using up to one motion vector and reference picture index per prediction block. A B slice (bi-predictive slice) is a slice on which intra prediction is performed or prediction is performed using up to two motion vectors and reference picture indices per prediction block. In this regard, an I slice (intra slice) is a slice to which only intra prediction is applied.

In the example of FIG. 5 , the position of the inter-layer reference picture in the reference picture list L0 was described as being the same as the position of the inter-layer reference picture in the reference picture list L1, but the present invention is not limited thereto, and the position of the inter-layer reference picture is referenced The picture list L0 may be different from the reference picture list L1. For example, for L0, the inter-layer reference picture may be placed in a position following the reference picture set preceding the current picture in the reference picture list, and for L1, the inter-layer reference picture may be placed after the long-term reference picture set in the reference picture list can be placed in the position of

In addition, in the example of FIG. 5 , it is stated that the inter-layer reference picture is in the second position in the reference picture list. may mean that For example, with respect to L0, the inter-layer reference picture may be placed in a position following the reference picture set preceding the current picture in the reference picture list.

For example, when the current block (the current block to be encoded/decoded) belongs to the B slice, as shown in FIG. 5 , the bidirectional reference picture lists L0 and L1 may exist. In this case, the decoded picture (inter-layer reference picture) of the reference layer may be added to the reference picture list (L0, L1) in both directions (L0 direction and L1 direction) at a fixed position.

Also, for example, when the current block belongs to a B slice, a bidirectional reference picture list may exist as shown in FIG. 5 . In this case, the inter-layer reference picture may be added with a fixed position to only one list (ie, L0 or L1) among the reference picture lists in both directions.

Also, for example, when the current block is an I slice or a P slice, the decoded picture (inter-layer reference picture) of the reference layer may be added only to the reference picture list L0 at a fixed position.

Also, for example, information (eg, LIST0ILRPflag, LIST1ILRPflag) indicating which reference picture list to use for inter-layer prediction of the current block or current picture, transmitted from the SPS, adds the inter-layer reference picture only to the list specified. You may.

In addition, if the position of the inter-layer reference picture (decoded picture of the reference layer) in the reference picture list is fixed at the N-th position, another reference picture at the N-th position in the reference picture list constructed without considering inter-layer prediction If is, the inter-layer reference picture may be placed at the N-th position, and the order may be pushed back one by one from the picture at the N-th position in the reference picture list configured without considering inter-layer prediction.

In addition, when the position of the inter-layer reference picture (the decoded picture of the reference layer) in the reference picture list is fixed at the N-th position, the reference picture at the N-th position in the reference picture list constructed without considering inter-layer prediction may be replaced with an inter-layer reference picture.

According to another embodiment of the present invention, whenever the reference picture list is generated, the position of the decoded picture (inter-layer reference picture) of the reference layer may be determined in the reference picture list by using encoding information.

In the present embodiment, the position of the inter-layer reference picture in the reference picture list for the current block may be determined using temporal level information of the current picture to be encoded/decoded (the current picture).

6 is a diagram schematically illustrating an example of temporal levels for each picture in a current layer. The temporal level of the picture may be transmitted from the encoding apparatus through a syntax element such as nuh_temporal_id.

Considering the case where the temporal level is determined as shown in FIG. 6 as an example, when the temporal level of the current picture is 1, the inter-layer reference picture is located at the first position (the value of the reference picture index is 0) in the reference picture list for the current block. can be placed

When determining the position of the inter-layer reference picture (the decoded picture of the reference layer) according to the temporal level, the inter-layer reference picture for the current picture (the encoding/decoding target picture) having the highest temporal level It may not be included in the reference picture list.

As another example, for example, a picture of a layer having a temporal level of a specific level or higher may not be included in the reference picture list as an inter-layer reference picture.

Meanwhile, the position of the inter-layer reference picture in the reference picture list may be determined according to the range of the temporal level. For example, if the temporal level of the current picture is 2 or more, the inter-layer reference picture may be located at the end of the reference picture list, and if the temporal level of the current picture is less than 2, the inter-layer reference picture is placed at the first position of the reference picture list can also be placed on

In adding an inter-layer reference picture to the reference picture list, if a bi-predictive reference picture list exists, it is implicitly inter-layered only in one direction of the reference picture list (reference picture list L0 or reference picture list L1). A reference picture may be added. For example, when the temporal level of the current picture is 0, the inter-layer reference picture is added to the first position in both directions list (reference picture list L0 and reference picture list L1), and the temporal level of the current picture is greater than 0 In this case, the inter-layer reference picture list may be added at the first position only to the list in one direction (the reference picture list L0 and the reference picture list L1). Although it has been described herein that the inter-layer reference picture is added at the first position, the inter-layer reference picture may be added at a position other than the first position according to the temporal level of the current picture.

When a bi-predictive reference picture list exists, the encoding apparatus may add an inter-layer reference picture to reference picture lists in both directions for the entire sequence, and an inter-layer reference picture only to a reference picture list in one direction. can also be added. The encoding apparatus may encode and transmit information (eg, LIST0ILRPflag, LIST1ILRPflag) indicating a reference picture list to which an inter-layer reference picture is added in the SPS. In this case, the decoding apparatus may add the inter-layer reference picture to both reference picture reference picture lists in both directions based on information transmitted from the SPS, or add the inter-layer reference picture only to the reference picture list in one direction. .

Meanwhile, information specifying the temporal level of the current picture may be transmitted in a Network Abstract Layer (NAL) unit header for the current picture. For example, the encoding apparatus may transmit information specifying the temporal level (temporal_id) of the current picture in the NAL unit header to the decoding apparatus, and the decoding apparatus may transmit the temporal level of the current picture according to information about the temporal level transmitted in the NAL unit header. can be specified.

Also, when the inter-layer reference picture (the decoded picture of the reference layer) has a higher temporal level than the temporal level of the current picture, the inter-layer reference picture may not be included in the reference picture list for the current block.

According to another embodiment of the present invention, whenever a reference picture list is generated, a position at which a decoded picture (inter-layer reference picture) of a reference layer is placed in the reference picture list can be determined using prediction structure information.

According to the present embodiment, the position of the inter-layer reference picture in the reference picture list may be determined by using information on the structure of prediction (SOP) of the current encoding/decoding target picture.

7 is a diagram schematically illustrating an example of depth information of a structure of prediction of a current layer.

In the case of applying the prediction structure as in the example of FIG. 7 , the position of the inter-layer reference picture in the reference picture list for the current block may be determined according to depth information of the prediction structure for the current picture.

For example, when considering the example of FIG. 7 , if the depth corresponding to the current picture in the prediction structure is 0, the position in the reference picture list of the inter-layer reference picture is the first position (the value of the reference picture index is 0 ) can be determined.

In addition, when considering the example of FIG. 7 , if the depth corresponding to the current picture in the prediction structure is 1 or more, the position of the inter-layer reference picture in the reference picture list may be determined as the last position.

The encoding apparatus may transmit information specifying a position in a reference picture list of an inter-layer reference picture determined according to depth information in a prediction structure in a picture parameter set (PPS) or a slice header.

The decoding apparatus may determine the position of the inter-layer reference picture in the reference picture list based on information received from the encoding apparatus.

For example, in the example of FIG. 7 , when the depth of the prediction structure of the current picture specified by the received information is 0, the decoding apparatus sets the position in the reference picture list of the inter-layer reference picture to the first position (of the reference picture index). The value can be determined to be 0).

Also, for example, in the example of FIG. 7 , when the depth in the prediction structure of the current picture specified by the received information is 1 or more, the decoding apparatus may determine the position in the reference picture list of the inter-layer reference picture as the last position. .

Meanwhile, in this embodiment, the position of the inter-layer reference picture in the reference picture list may be implicitly determined. In this case, it may be determined whether to add the inter-layer reference picture only to the reference picture list L0 according to the depth information of the prediction structure. In this case, the encoding apparatus encodes information specifying the reference picture list to which the inter-layer reference picture is added so that the inter-layer reference picture can be added at the same position in the reference picture list even in the decoding apparatus to picture parameter sets (PPS) or slice It can be sent in the header. The decoding apparatus may perform prediction on the current block by adding the inter-layer reference picture to the reference picture list based on the received information.

Meanwhile, up to now, it has been described as 'the position of the inter-layer reference picture in the reference picture list', but this is for convenience of description, and the position of the inter-layer reference picture in the reference picture list is 'reference picture index indicating the inter-layer reference picture'. It may be a 'position in the reference picture list of ' or 'order of values of reference picture indexes indicating inter-layer reference pictures in the reference picture list'.

8 is a flowchart schematically illustrating an example of a video encoding method according to the present invention.

Referring to FIG. 8 , the encoding apparatus may decode a picture of a reference layer ( S810 ). The picture to be decoded may be a picture belonging to the same access unit as the encoding target picture (current picture) of the current layer.

The encoding apparatus may derive an inter-layer reference picture (a decoded picture of the reference layer) by sampling the decoded picture of the reference layer ( S820 ). The encoding apparatus may derive an inter-layer reference picture that can be used as a reference picture for the current block (the encoding target block in the current picture of the current layer) from the decoded (reconstructed) picture of the reference layer.

The encoding apparatus may construct a reference picture including the inter-layer reference picture and the reference picture of the current layer (S830).

In this case, in the current layer, the position of the inter-layer reference picture in the reference picture list may be a fixed specific position. For example, the position of the inter-layer reference picture in the reference picture list may be fixed to the position next to the reference pictures preceding the picture to which the current block belongs in the current layer.

The encoding apparatus may configure the reference picture list to include at least one or more inter-layer reference pictures derived from at least one or more reference layers.

Also, the encoding apparatus may construct a reference picture list based on the temporal level of the current picture. For example, when the temporal level of the current picture is equal to or greater than a specific range level, the encoding apparatus may exclude the inter-layer reference picture from the reference picture list. That is, when the temporal level of the current picture is equal to or greater than a specific range level, the encoding apparatus may not refer to the decoded picture of the reference layer for inter prediction of the current block.

The encoding apparatus may perform prediction on the current sample of the current layer based on the reference picture list to perform the prediction sample on the current block (S840). When inter-layer prediction is applied, the encoding apparatus may perform inter prediction on the current block with reference to the inter-layer reference picture.

The encoding apparatus may generate a residual sample for the current block based on the prediction sample ( S850 ). The encoding apparatus may derive a difference value between the original signal and the prediction signal of the current block as a residual sample.

The encoding apparatus may entropy-encode the residual sample and transmit it to the decoding apparatus (S860). The encoding apparatus may transform and quantize the residual sample, then entropy-encode it and transmit it to the decoding apparatus.

9 is a flowchart schematically illustrating an example of a video decoding method according to the present invention.

Referring to FIG. 9 , the decoding apparatus may decode a picture of a reference layer ( S910 ). The picture to be decoded may be a picture belonging to the same access unit as the decoding target picture (current picture) of the current layer.

The decoding apparatus may derive an inter-layer reference picture (a decoded picture of the reference layer) by sampling the decoded picture of the reference layer ( S920 ). The decoding apparatus may derive an inter-layer reference picture that can be used as a reference picture for a current block (a decoding target block in the current picture of the current layer) from the decoded (reconstructed) picture of the reference layer.

The decoding apparatus may construct a reference picture including the inter-layer reference picture and the reference picture of the current layer (S930).

In this case, in the current layer, the position of the inter-layer reference picture in the reference picture list may be a fixed specific position. For example, the position of the inter-layer reference picture in the reference picture list may be fixed to the position next to the reference pictures preceding the picture to which the current block belongs in the current layer.

The decoding apparatus may configure the reference picture list to include at least one or more inter-layer reference pictures derived from at least one or more reference layers.

Also, the decoding apparatus may construct a reference picture list based on the temporal level of the current picture. For example, when the temporal level of the current picture is equal to or greater than a specific range level, the decoding apparatus may exclude the inter-layer reference picture from the reference picture list. That is, when the temporal level of the current picture is equal to or greater than a specific range level, the decoding apparatus may not refer to the decoded picture of the reference layer for inter prediction of the current block.

The decoding apparatus may perform prediction on the current sample of the current layer based on the reference picture list to perform the prediction sample on the current block (S940). When inter-layer prediction is applied, the decoding apparatus may perform inter prediction on the current block with reference to the inter-layer reference picture.

The decoding apparatus may generate a reconstructed sample for the current block based on the prediction sample (S850). The encoding apparatus may derive a reconstructed sample for the current block by adding a prediction sample for the current block and a residual signal (residual sample) received from the encoding apparatus.

The method according to the present invention described above may be produced as a program to be executed by a computer and stored in a computer-readable recording medium. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape. , a floppy disk, an optical data storage device, and the like, and also includes those implemented in the form of a carrier wave (eg, transmission through the Internet).

The computer-readable recording medium is distributed in a network-connected computer system, so that the computer-readable code can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the method can be easily inferred by programmers in the art to which the present invention pertains.

In addition, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention as claimed in the claims Various modifications may be made by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

The above-described embodiments include examples of various aspects. It is not possible to describe every possible combination for representing the various aspects, but one of ordinary skill in the art will recognize that other combinations are possible. Accordingly, it is intended that the present invention cover all other substitutions, modifications and variations falling within the scope of the following claims.

Claims (11)

deriving an inter-layer reference picture from pictures of the reference layer;
constructing one or more reference picture lists including the inter-layer reference picture and the reference picture of the current layer;
generating a prediction sample for the current block by performing prediction on the current block of the current layer based on the one or more reference picture lists;
generating a residual sample for the current block based on the prediction sample; and
encoding the residual sample;
The position of the inter-layer reference picture in the one or more reference picture lists is adaptively determined from encoding information,
The encoding information specifies a reference picture list including the inter-layer reference picture and a position of the inter-layer reference picture in the reference picture list,
The encoding information is a video encoding method, characterized in that included in the slice header. The video encoding method of claim 1, wherein the reference picture list includes at least one or more inter-layer reference pictures derived from at least one or more reference layers. The video encoding method of claim 1, wherein the inter-layer reference picture belongs to the same access unit as the picture to which the current block belongs. The method of claim 1, wherein in the step of constructing the reference picture list including the inter-layer reference picture and the reference picture of the current layer,
When the temporal level of the inter-layer reference picture is equal to or higher than a predetermined level, the inter-layer reference picture is not included in the reference picture list. The method of claim 4, wherein in the step of constructing the reference picture list including the inter-layer reference picture and the reference picture of the current layer,
The video encoding method of claim 1, wherein the reference picture list is configured by including an inter-layer reference picture derived from a picture of a reference layer having a temporal level equal to or lower than a temporal level of the picture to which the current block belongs. deriving an inter-layer reference picture from decoded pictures of the reference layer;
constructing one or more reference picture lists including the inter-layer reference picture and the reference picture of the current layer;
generating a prediction sample for the current block by performing prediction on the current block of the current layer based on the one or more reference picture lists; and
generating a reconstructed sample for the current block based on the prediction sample and a residual sample for the current block;
The position of the inter-layer reference picture in the one or more reference picture lists is adaptively determined from encoding information,
The encoding information specifies a reference picture list including the inter-layer reference picture and a position of the inter-layer reference picture in the reference picture list,
The encoding information is obtained from a slice header. The video decoding method of claim 6 , wherein the reference picture list includes at least one inter-layer reference picture derived from at least one or more reference layers. The video decoding method of claim 6, wherein the inter-layer reference picture belongs to the same access unit as the picture to which the current block belongs. The method of claim 6, wherein in the step of constructing the reference picture list including the inter-layer reference picture and the reference picture of the current layer,
When the temporal level of the inter-layer reference picture is equal to or higher than a predetermined level, the inter-layer reference picture is not included in the reference picture list. The method of claim 9, wherein in the step of constructing the reference picture list including the inter-layer reference picture and the reference picture of the current layer,
The video decoding method of claim 1, wherein the reference picture list is configured by including an inter-layer reference picture derived from a picture of a reference layer having a temporal level equal to or lower than a temporal level of the picture to which the current block belongs. deriving an inter-layer reference picture from pictures of the reference layer;
constructing one or more reference picture lists including the inter-layer reference picture and the reference picture of the current layer;
generating a prediction sample for the current block by performing prediction on the current block of the current layer based on the one or more reference picture lists;
generating a residual sample for the current block based on the prediction sample; and
encoding the residual sample;
Storing the bitstream generated by the video encoding method, characterized in that the position of the inter-layer reference picture in the one or more reference picture lists is adaptively determined from encoding information;
The encoding information specifies a reference picture list including the inter-layer reference picture and a position of the inter-layer reference picture in the reference picture list,
The encoding information is included in a slice header.

Images