KR102219841B1 - Method and Apparatus for Video Encoding and Video Decoding - Google Patents

Abstract

The present invention relates to prediction of video information and video encoding/decoding using the same. In the video encoding according to the present invention, the step of decoding a picture of a reference layer, sampling a decoded picture of the reference layer to derive an inter-layer reference picture The step of: constructing a reference picture list including the inter-layer reference picture and the reference picture of the current layer, prediction for the current block by performing prediction on the current block of the current layer based on the reference picture list Generating a sample, generating a residual sample for the current block based on the predicted sample, and encoding the residual sample.

Description

Video encoding method, video decoding method, and apparatus using the same

The present invention relates to a video encoding and decoding method and an apparatus using the same, and in particular, a method of using information of a reference picture corresponding to a reference layer in a reference layer to generate a prediction sample for a block to be encoded or decoded of a current layer, and It relates to the device.

Recently, as broadcasting services having high definition (HD) resolution are expanded not only domestically but also globally, many users are getting accustomed to high-definition and high-definition images, and accordingly, many organizations are spurring the development of next-generation imaging devices. In addition, as interest in UHD (Ultra High Definition) having a resolution of 4 or more times higher than that of HDTV in addition to HDTV is increasing, a compression technology for higher resolution and high quality images is required.

For image compression, inter prediction technology that temporally predicts pixel values included in the current picture from previous and/or subsequent pictures, and predicts pixel values included in the current picture using pixel information in the current picture. An intra prediction technique, an entropy coding technique in which a short code is assigned to a symbol with a high frequency of appearance, and a long code is assigned to a symbol with a low frequency of occurrence, may be used.

Image compression technology includes a technology that provides a constant network bandwidth under a limited operating environment of hardware without considering a flexible network environment. However, in order to compress image data applied to a network environment in which the bandwidth changes frequently, a new compression technique is required, and for this purpose, a scalable video encoding/decoding method may be used.

Therefore, there is an increasing demand to efficiently transmit data to various transmission environments and various terminals by generating one integrated data that can support various spatial resolutions and various frame rates. .

In this regard, as a video encoding technology for supporting efficient transmission of data, standardization of SHVC (Scalable High Efficiency Video Coding) based on HEVC is in progress.

An object of the present invention is to provide a method and apparatus capable of performing prediction using information of another layer by adding a decoded picture of a lower layer (reference layer) to a reference picture list for an encoding/decoding target within the current layer. do.

The present invention also provides a method and apparatus for generating a prediction signal such that a difference signal between the original signal and the prediction signal is minimized by performing prediction (eg, motion prediction) on a decoded picture of a lower layer (reference layer). It aims to provide.

In the present invention, when a decoded picture of a lower layer (reference layer) is added to a reference picture list for a current encoding/decoding target block, the lower layer adaptively uses encoding information, depth information of a prediction structure, etc. An object of the present invention is to provide a method and apparatus for determining a position of a decoded picture of a reference layer) in a reference picture list.

An embodiment of the present invention is a video encoding method, comprising: decoding a picture of a reference layer, deriving an inter-layer reference picture by sampling a decoded picture of the reference layer, and referencing the inter-layer reference picture and the current layer. Constructing a reference picture list including pictures, generating a prediction sample for the current block by performing prediction on the current block of the current layer based on the reference picture list, and generating a prediction sample for the current block, based on the prediction sample, It may include generating a residual sample for the current block and encoding the residual sample.

Another embodiment of the present invention is a video decoding method, comprising: decoding a picture of a reference layer, deriving an inter-layer reference picture by sampling a decoded picture of the reference layer, and referencing the inter-layer reference picture and the current layer. Constructing a reference picture list including pictures, generating a prediction sample for the current block by performing prediction on the current block of the current layer based on the reference picture list, and generating a prediction sample for the current block, and the prediction sample and the current block It may include generating a reconstructed sample for the current block based on the residual sample for.

Another embodiment of the present invention is an inter prediction method of video information, comprising: decoding a picture of a reference layer, deriving an inter-layer reference picture by sampling a decoded picture of the reference layer, the inter-layer reference picture, and Constructing a reference picture list including a reference picture of a current layer, and generating a prediction sample for the current block by performing prediction on the current block of the current layer based on the reference picture list. have.

According to the present invention, in constructing a reference picture list to perform prediction (eg, motion prediction and motion compensation) for a picture to be encoded/decoded of an upper layer (current layer), decoding of a lower layer (reference layer) A reference picture list may be constructed including the picture.

According to the present invention, in adding the decoded image of the lower layer (reference layer) to the reference picture list, the reference picture list of the decoded image of the lower layer (reference layer) using encoding information, prediction structure depth information, etc. I can adaptively determine my location.

In addition, according to the present invention, the decoded picture of the lower layer (reference layer) is used as a reference picture, but the coding efficiency is improved by adaptively determining the position of the decoded picture of the lower layer (reference layer) within the reference picture. I can.

1 is a block diagram showing a configuration of an image encoding apparatus according to an embodiment.
2 is a block diagram illustrating a configuration of a video decoding apparatus according to an embodiment.
3 is a conceptual diagram schematically showing an embodiment of a scalable video coding structure using multiple layers to which the present invention can be applied.
4 is a flowchart schematically illustrating a method of performing inter-layer prediction in an encoding device/decoding device according to the present invention.
5 is a diagram schematically illustrating an example of a method of constructing a reference picture list according to an embodiment of the present invention.
6 is a diagram schematically illustrating an example of temporal levels for each picture in a current layer.
7 is a diagram schematically illustrating an example of depth information of a structure of prediction of a current layer.
8 is a flowchart schematically illustrating an example of a video encoding method according to the present invention.
9 is a flowchart schematically illustrating an example of a video decoding method according to the present invention.

In describing the embodiments of the present specification, when it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present specification, a detailed description thereof will be omitted.

When a component is referred to as being “connected” or “connected” to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. Should be. In addition, the description of "including" a specific configuration in the present invention does not exclude configurations other than the corresponding configuration, and means that additional configurations may be included in the scope of the implementation of the present invention or the technical idea of the present invention.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. These terms are used only for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element.

In addition, the components shown in the embodiments of the present invention are shown independently to represent different characteristic functions, and it does not mean that each component is formed of separate hardware or a single software component. That is, each constituent part is listed and included as a respective constituent part for convenience of explanation, and at least two of the constituent parts are combined to form one constituent part, or one constituent part is divided into a plurality of constituent parts to perform a function. Integrated embodiments and separate embodiments of the component are also included in the scope of the present invention unless departing from the essence of the present invention.

In addition, some of the components are not essential components that perform essential functions in the present invention, but may be optional components only for improving performance. The present invention can be implemented by including only the components essential to implement the essence of the present invention excluding components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement Also included in the scope of the present invention.

The present invention relates to a technique for encoding and decoding an image having a structure including a plurality of layers, and to a method and apparatus for performing prediction on an upper layer by using information of a lower layer when encoding/decoding an upper layer.

More specifically, according to the present invention, in generating a reference picture list used for motion prediction and/or motion compensation for an encoding/decoding target picture of an upper layer (current layer), a lower layer (reference layer) ), a reference picture list can be generated.

When generating a reference picture list of an upper layer (current layer, hereinafter referred to as'current layer') including the decoded picture of the lower layer (reference layer), it is adaptively used by using encoding information and depth information of the prediction structure. Encoding efficiency can be improved by adding a decoded picture of a lower layer (reference layer, hereinafter referred to as a'reference layer').

Hereinafter, the present invention will be described in detail with reference to the drawings.

1 is a block diagram showing a configuration of an image encoding apparatus according to an embodiment.

A scalable video encoding/decoding apparatus for a multi-layer structure may be implemented by extending a video encoding/decoding apparatus for a single layer structure.

1 shows an embodiment of a video encoding apparatus applicable to a multi-layer structure, that is, providing scalability.

1, the video encoding apparatus 100 includes an inter prediction unit 110, an intra prediction unit 120, a switch 115, a subtractor 125, a transform unit 130, a quantization unit 140, and an entropy. An encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190 are included.

The video encoding apparatus 100 may encode an input image in an intra mode or an inter mode and output a bitstream. Intra prediction means intra prediction, and inter prediction means inter prediction. In the case of the intra mode, the switch 115 is switched to the intra mode, and in the case of the inter mode, the switch 115 is switched to the inter mode. The video encoding apparatus 100 may generate a prediction block for a block (current block) of an input picture and then encode a difference between the current block and the prediction block.

In the case of the intra mode, the intra prediction unit 120 may use a pixel value of an already coded block around the current block as a reference pixel. The intra prediction unit 120 may perform spatial prediction using the reference call and generate prediction samples for the current block.

In the case of the inter mode, the inter prediction unit 110 may obtain a motion vector specifying a reference block having the least difference from an input block (current block) from a reference picture stored in the reference picture buffer 190. The inter prediction unit 110 may generate a prediction block for the current block by performing motion compensation using a motion vector and a reference picture stored in the reference picture buffer 190.

In the case of a multi-layer structure, inter prediction applied in the inter mode may include inter-layer prediction. The inter prediction unit 110 samples a picture of a reference layer to construct an inter-layer reference picture, and performs inter-layer prediction by including the inter-layer reference picture in a reference picture list. The reference relationship between layers may be signaled through information specifying dependency between layers.

Meanwhile, when the current layer picture and the reference layer picture have the same size, sampling applied to the reference layer picture may mean generating a reference sample by copying or interpolating samples from the reference layer picture. When the resolution of the current layer picture and the reference layer picture are different, sampling applied to the reference layer picture may mean upsampling.

For example, when the resolutions between layers are different, an inter-layer reference picture may be configured by up-sampling a reconstructed picture of a reference layer between layers supporting scalability regarding resolution.

Which layer of the picture is used to configure the inter-layer reference picture may be determined in consideration of coding cost and the like. The encoding device may transmit information specifying a layer to which a picture to be used as an inter-layer reference picture belongs to the decoding device.

In addition, a layer referenced in inter-layer prediction, that is, a picture used for prediction of a current block in a reference layer, may be a picture of the same access unit (AU) as the current picture (a prediction target picture in the current layer).

The subtractor 125 may generate a residual block (residual signal) based on a difference between the current block and the prediction block.

The transform unit 130 may transform the residual block and output a transform coefficient. When the transform skip mode is applied, the transform unit 130 may omit the transform of the residual block.

The quantization unit 140 may quantize a transform coefficient according to a quantization parameter and output a quantized coefficient.

The entropy encoder 150 may entropy-encode values calculated by the quantization unit 140 or an encoding parameter value calculated during an encoding process according to a probability distribution to output a bitstream. The entropy encoder 150 may entropy-encode information for video decoding (eg, syntax element, etc.) in addition to pixel information of the video.

The encoding parameter is information necessary for encoding and decoding, and may include information that can be inferred during an encoding or decoding process as well as information encoded by an encoding device and transmitted to a decoding device, such as a syntax element.

Encoding parameters include values such as intra/inter prediction mode, motion/motion vector, reference image index, coded block pattern, presence/absence of a residual signal, transform coefficient, quantized transform coefficient, quantization parameter, block size, block division information, etc. Or it could include statistics.

The residual signal may mean the difference between the original signal and the predicted signal, and a signal in which the difference between the original signal and the predicted signal is transformed, or the difference between the original signal and the predicted signal is transformed and quantized. It can also mean. The residual signal may be referred to as a residual block in block units.

When entropy coding is applied, a small number of bits are allocated to a symbol having a high probability of occurrence, and a symbol is represented by assigning a large number of bits to a symbol having a low probability of occurrence, so that the size of a bit string for encoding target symbols is reduced Can be reduced. Therefore, compression performance of image encoding can be improved through entropy encoding.

For entropy encoding, encoding methods such as exponential golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be used. For example, the entropy encoding unit 150 may perform entropy encoding using a variable length encoding (VLC) table. In addition, the entropy encoder 150 derives a binarization method of a target symbol and a probability model of a target symbol/bin, and then performs entropy encoding using the derived binarization method or a probability model. You may.

The quantized coefficient may be inverse quantized by the inverse quantization unit 160 and may be inversely transformed by the inverse transform unit 170. The inverse quantized and inverse transformed coefficients are added to the prediction block through the adder 175, and a reconstructed block may be generated.

The reconstructed block passes through the filter unit 180, and the filter unit 180 applies at least one of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to a reconstructed block or a reconstructed picture. can do. The reconstructed block that has passed through the filter unit 180 may be stored in the reference image buffer 190.

2 is a block diagram illustrating a configuration of a video decoding apparatus according to an embodiment.

A scalable video encoding/decoding apparatus for a multi-layer structure may be implemented by extending a video encoding/decoding apparatus for a single layer structure.

2 shows an embodiment of a video decoding apparatus applicable to a multi-layer structure, that is, providing scalability.

Referring to FIG. 2, the video decoding apparatus 200 includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra prediction unit 240, an inter prediction unit 250, and a filter unit. 260 and a reference picture buffer 270.

The video decoding apparatus 200 may receive a bitstream output from the encoding apparatus, perform decoding in an intra mode or an inter mode, and output a reconstructed image, that is, a reconstructed image.

In the case of the intra mode, the switch may be switched for intra prediction, and in the case of the inter mode, the switch may be switched for inter prediction.

The video decoding apparatus 200 may generate a reconstructed block, that is, a reconstructed block, by obtaining a reconstructed residual block from an input bitstream, generating a prediction block, and adding the reconstructed residual block and the prediction block. have.

The entropy decoder 210 may entropy-decode the input bitstream according to a probability distribution, and output information such as a quantized coefficient and a syntax element.

The quantized coefficients are inverse quantized by the inverse quantization unit 220 and inversely transformed by the inverse transform unit 230. A reconstructed residual block may be generated by inverse quantization/inverse transform of the quantized coefficient.

In the case of the intra mode, the intra prediction unit 240 may perform spatial prediction using pixel values of an already coded block around the current block and generate a prediction block for the current block.

In the inter mode, the inter prediction unit 250 may generate a prediction block for the current block by performing motion compensation using a motion vector and a reference picture stored in the reference picture buffer 270.

In the case of a multi-layer structure, inter prediction applied in the inter mode may include inter-layer prediction. The inter prediction unit 250 may sample a picture of a reference layer to construct an inter-layer reference picture, and perform inter-layer prediction by including the inter-layer reference picture in a reference picture list. The reference relationship between layers may be signaled through information specifying dependency between layers.

Meanwhile, when the current layer picture and the reference layer picture have the same size, sampling applied to the reference layer picture may mean generating a reference sample by copying or interpolating samples from the reference layer picture. When the resolution of the current layer picture and the reference layer picture are different, sampling applied to the reference layer picture may mean upsampling.

For example, in this case, if inter-layer prediction is applied between layers supporting scalability regarding resolution as a case in which the inter-layer resolution is different, an inter-layer reference picture may be configured by up-sampling the reconstructed picture of the reference layer.

In this case, information specifying a layer to which a picture to be used as an inter-layer reference picture belongs may be transmitted from an encoding device to a decoding device.

In addition, a layer referenced in inter-layer prediction, that is, a picture used for prediction of a current block in a reference layer, may be a picture of the same access unit (AU) as the current picture (a prediction target picture in the current layer).

The reconstructed residual block and the prediction block are added by an adder 255 to generate a reconstructed block. In other words, a reconstructed sample or a reconstructed picture is generated by adding the residual sample and the prediction sample.

The reconstructed picture is filtered by the filter unit 260. The filter unit 260 may apply at least one of a deblocking filter, SAO, and ALF to a reconstructed block or a reconstructed picture. The filter unit 260 outputs a reconstructed or filtered reconstructed picture. The reconstructed image may be stored in the reference picture buffer 270 and used for inter prediction.

Also, the image decoding apparatus 200 may further include a parsing unit (not shown) for parsing information related to an encoded image included in the bitstream. The parsing unit may include an entropy decoding unit 210 or may be included in the entropy decoding unit 210. This parsing unit may also be implemented as a component of the decoding unit.

In FIGS. 1 and 2, it has been described that one encoding device/decoding device processes all encoding/decoding for multi-layers, but this is for convenience of description, and the encoding device/decoding device may be configured for each layer.

In this case, the encoding/decoding device of the upper layer may perform encoding/decoding of the corresponding upper layer by using the information of the upper layer and the information of the lower layer. For example, the prediction unit (inter prediction unit) of the upper layer may perform intra prediction or inter prediction on the current block using pixel information or picture information of the upper layer, or receive picture information reconstructed from the lower layer and By using, inter prediction (inter-layer prediction) on the current block of the upper layer may be performed. Here, only inter-layer prediction has been described as an example, but the encoding/decoding device performs encoding/decoding for the current layer using information of other layers, regardless of whether the encoding device/decoding device is configured for each layer or one device processes multi-layers. can do.

In the present invention, a layer may include a view. In this case, in the case of inter-layer prediction, the prediction of the upper layer is not simply performed using the information of the lower layer, but information of the other layer between the layers specified as having dependence by information specifying the inter-layer dependency. Inter-layer prediction may be performed using.

3 is a conceptual diagram schematically showing an embodiment of a scalable video coding structure using multiple layers to which the present invention can be applied. In FIG. 3, a group of pictures (GOP) represents a picture group, that is, a group of pictures.

Video data is transmitted from a video encoding device to a video decoding device through a transmission medium or a storage medium. Video data may differ for each transmission medium according to a network environment, and may vary for each storage medium. A scalable video coding method may be provided to efficiently support these various video data.

In the scalable video coding method, inter-layer sample information (texture information), motion information, and a residual signal are used. Through this, it is possible to improve encoding/decoding performance by removing redundancy between layers. The scalable video coding method may provide various scalability in terms of spatial, temporal, and image quality according to surrounding conditions such as a transmission bit rate, a transmission error rate, and system resources.

Scalable video coding may be performed using a multiple layer structure. The multi-layer structure may include a base layer that can be independently encoded/decoded without using information of other layers. In addition, the multi-layer structure may include an enhancement layer that is encoded/decoded using information of a base layer and/or another layer.

In the layer structure, a layer with high scalability may be referred to as an upper layer, and a layer with low scalability may be referred to as a lower layer. In addition, a layer referred to for encoding/decoding (eg, prediction) of another layer may be referred to as a reference layer, and a layer encoded/decoded (eg, predicted) using another layer may be referred to as a current layer. The reference layer may be a lower layer than the current layer, and the current layer may be a higher layer than the reference layer.

Here, the layer is an image and bit divided based on spatial (e.g., image size, resolution), time (temporal (e.g., frame rate)), quality, complexity, view, etc. It may be a set of streams.

Also, a plurality of layers may have a dependency between each other. The current layer may be encoded/decoded by referring to the layer in which the dependency exists.

3, for example, the first enhancement layer may be defined as a first resolution higher than the base resolution of the base layer, a first frame rate higher than the frame rate of the base layer, and a first bit rate higher than the bit rate of the base layer. I can. In addition, the second enhancement layer is defined as a second resolution higher than the resolution of the first enhancement layer, a second frame rate higher than the frame rate of the first enhancement layer, and a second bit rate higher than the bit rate of the first enhancement layer. Can be.

For example, if the first enhancement layer is defined as HD (high definition), 30 Hz frame rate, and 3.9 Mbps bit rate, the second enhancement layer is 4K-UHD (ultra high definition), 60 Hz frame rate, and 27.2 Mbps bit rate. Can be defined. This format, frame rate, bit rate, and the like are examples, and may be differently determined as necessary. Also, the number of layers to be used is not limited to this embodiment and may be determined differently according to circumstances.

For example, if the transmission bandwidth is 4 Mbps, the frame rate of the first enhancement layer HD may be reduced to transmit at 15 Hz or less. The scalable video coding method may provide temporal, spatial, quality, and inter-view scalability by the method described above in the embodiment of FIG. 3.

In the case of encoding and decoding video that supports multiple layers in the bitstream, that is, scalable coding, there is a strong correlation between the multiple layers. Redundant elements can be removed and video encoding performance can be improved.

Prediction on a picture of a current layer to be predicted by using information of another layer is called inter-layer prediction.

The scalable video coding has the same meaning as the scalable video coding from the viewpoint of encoding and the scalable video decoding from the viewpoint of decoding.

The plurality of layers may have at least one of a resolution, a frame rate, a color format, and a viewpoint different from each other, and up-sampling or down-sampling of a layer may be performed to adjust the resolution during inter-layer prediction.

In general, inter prediction may use at least one of a previous picture or a subsequent picture of the current picture as a reference picture, and prediction on a current block may be performed based on the reference picture. An image used for prediction of the current block is referred to as a reference picture or a reference frame.

A region referenced for prediction of a current picture within a reference picture may be specified using a reference picture index (refIdx) indicating the reference picture, a motion vector, and the like.

Inter prediction may generate a prediction block for the current block by selecting a reference picture and a reference block corresponding to the current block within the reference picture.

In inter prediction, an encoding apparatus and a decoding apparatus may derive motion information for a current block and then perform inter prediction and/or motion compensation based on the motion information. In this case, the encoding apparatus and the decoding apparatus collate a current block within a collocated picture selected from motion information and/or reference pictures of a neighboring block of the current block. Prediction on a current block may be performed using motion information of a block (collocated block).

As described in FIGS. 1 and 2, in inter prediction (inter picture prediction), at least one of a picture before or after a current picture is determined as a reference picture, and a current picture (current block in the current picture) based on the reference picture Make predictions for

The encoding apparatus may transform and/or quantize a residual signal, which is a difference between the prediction block induced as a result of prediction and the current block (difference between the original signal and the prediction signal), and then apply entropy decoding and transmit it to the decoding apparatus.

The decoding apparatus may reconstruct the current block (original signal) by performing inverse quantization and/or inverse transformation of the residual received from the encoding apparatus through entropy decoding, and adding the derived prediction block (prediction signal) as a result of prediction.

When the PCM mode is applied, transform/quantization and inverse quantization/inverse transform may be omitted, and when the transform skip mode is applied, transform and inverse transform may be omitted. In addition, when a skip mode at a coding unit (CU) level is applied as a method of inter prediction, since there is no residual transmission, a prediction signal may be used as an original signal.

More specifically, inter prediction will be described. In the case of inter prediction, the encoding apparatus may generate a prediction signal to satisfy at least one of minimization of residuals and minimization of motion vectors. For example, the encoding apparatus may generate a prediction signal to minimize a motion vector while minimizing a residual.

The encoding device may transmit information specifying motion information to the decoding device so that the same prediction signal can be derived even in the decoding device. The decoding apparatus may derive motion information based on information received from the encoding apparatus and generate a prediction signal.

In inter prediction, information on neighboring blocks of a current block may be used to specify information on a reference picture. For example, in inter prediction methods such as skip mode, merge mode, and AMVP (Advanced Motion Vector Prediction) mode using mvd (Motion Vector Difference) and mvp (Motion Vector Prediction), motion information of neighboring blocks is Based on this, a prediction block for the current block can be generated.

As described above, in the case of the skip mode, motion information of a neighboring block may be used as motion information of a current block. For example, information specifying information used as a reference picture index and/or a motion vector of the current block among the reference picture index and/or motion vector of a neighboring block is transmitted from the encoding device to the decoding device, but additional information such as residual Not transmitted.

When the merge mode is applied as the inter prediction mode, the encoding device/decoding device may derive a prediction block (prediction signal) for the current block by using motion information of the neighboring block as it is. The encoding apparatus may transmit information indicating whether to apply the merge mode to the current block, information indicating which neighboring block information is to be used, and residual information for the current block to the decoding apparatus.

When the merge mode is applied, the decoding apparatus generates a prediction block (prediction sample) for the current block by using motion information of neighboring blocks indicated by the encoding apparatus, and restores the current block by adding the residual signal (reconstructed sample). Can induce).

Motion information of neighboring blocks used in skip mode and merge mode may include a reference picture index and motion vector of the neighboring block.

When AMVP is applied, the encoding device/decoding device may predict the motion vector of the current block by using the motion vector of the neighboring block. The encoding apparatus transmits information specifying which motion information of a neighboring block is to be used, information specifying a difference (mvd) between the motion vector of the current block and the predicted motion vector, and a reference picture index specifying a reference picture to the decoder. I can.

When AMVP is applied, the decoding apparatus may generate a prediction block (prediction sample) for a current block using motion information (motion vector) of a neighboring block and a reference picture index transmitted from an encoder. For example, the decoding apparatus may generate a motion vector predictor of the current block based on a motion vector of a neighboring block, and may derive a motion vector of the current block by adding a motion vector difference (mvd) received from the encoding apparatus and a motion vector predictor. . The decoding apparatus may derive a prediction block (prediction sample) for the current block by using the derived motion vector and the received reference picture index. The decoding apparatus may derive reconstructed samples for the current picture by reconstructing the current block by adding the prediction sample to the residual.

When performing inter prediction, motion information used by the decoder device may be derived based on information received from the encoder. For example, when the skip mode is applied, motion information of a block indicated by the merge index may be used as motion information of the current block. When the merge mode is applied, motion information of the block indicated by the merge index may be used as motion information of the current block. In this case, some of the motion information of the temporal neighboring block may be fixed to a specific value or may be derived separately. In the case of AMVP, a motion vector of a current block may be predicted from a motion vector of a neighboring block, and a reference picture index may be signaled separately.

The processing unit in which prediction is performed may be different from the unit in which the prediction method is determined. For example, whether intra prediction or inter prediction is applied may be determined in units of a coding unit (CU), and a mode of intra prediction and a mode of inter prediction may be determined in units of a prediction unit (PU). Intra prediction may be performed in units of a transform unit (TU), and inter prediction may be performed in units of a prediction unit.

Meanwhile, in scalable video coding to which a multi-layer structure is applied, prediction on a current layer may be performed using information of another layer. For example, in performing prediction on a current block in a current picture of a current layer that is a decoding/encoding target layer, information of another layer (reference layer) may be used.

In this case, picture information of a reference layer may be used in a manner that uses texture information of another layer. In this case, prediction may be performed using only decoded picture information of a reference layer corresponding to an encoding/decoding target block (current block).

However, when prediction for the current layer is performed using texture information of a certain region corresponding to the reference layer (when inter-layer prediction is performed), signal distortion may occur based on the difference in image size between layers, etc. , The residual signal may increase.

Accordingly, in the present invention, as described in FIGS. 1 and 2, when performing inter-layer prediction, an encoding device/decoding device (eg, an inter-prediction unit in an encoding device/decoding device) is used for inter-layer prediction in a reference layer. An inter-layer reference picture is generated by sampling a picture to be used, and inter prediction on a current block is performed by including the inter-layer reference picture in a reference picture list for the current block.

4 is a flowchart schematically illustrating a method of performing inter-layer prediction in an encoding device/decoding device according to the present invention.

In FIG. 4, for convenience of description, a case in which inter-layer prediction is performed by a prediction unit in an encoding device/decoding device is described as an example. In this case, the prediction unit may include an inter prediction unit.

Referring to FIG. 4, an encoding device/decoding device decodes a picture of a reference layer (S410). As described above, the reference layer refers to a layer referred to by an encoding/decoding target block of the current layer. In this case, the prediction unit may decode a picture of the same AU as the current picture (a picture to which the current block belongs in the current layer) among pictures of the reference layer and use it for prediction of the current block.

The decoding of the picture of the reference layer referenced by the current encoding target block may be performed based on information on the encoding method of the reference layer transmitted through a video parameter set (VPS) or a sequence parameter set (SPS).

For example, when the reference layer is encoded using the MPEG-2 scheme, the encoding device/decoding apparatus may decode the picture of the reference layer according to the MPEG-2 scheme.

In addition, if the reference layer is encoded by the H.264/AVC scheme, the encoding device/decoding apparatus may decode the picture of the reference layer according to the H.264/AVC scheme.

In addition, if the reference layer is encoded by the HEVC scheme, the encoding device/decoding apparatus may decode the picture of the reference layer according to the HEVC scheme.

The encoding/decoding apparatus may construct a reference picture list for inter prediction of the current block (S420). The prediction unit may generate a reference picture list including a decoded picture of a reference layer for inter prediction on a current encoding/decoding target block (current block).

In video encoding/decoding using a multi-layer structure, the prediction unit may perform prediction on an encoding/decoding target block of an upper layer (current layer) by using a decoded picture of a reference layer.

In this case, in using the decoded picture of the reference layer, the decoded picture of the reference layer is included in the reference picture list for the encoding/decoding target block (current block) of the upper layer (current layer), You can make predictions.

The reference picture list can be constructed when decoding for a slice starts. For example, in the case of a P slice, one reference picture list may be configured, and in the case of a B slice, two reference picture lists may be configured. Accordingly, the reference picture list for the current block may be configured for each slice to which the current block belongs.

In this regard, it is a question of where to place the decoded picture of the reference layer in the reference picture list.

According to embodiments of the present invention, (1) when configuring a reference picture list, a decoded picture of a reference layer is always added to a fixed position, or (2) when configuring a reference picture list, within the reference picture list. The position of the decoded picture of the reference layer may be determined using encoding information, or (3) when configuring the reference picture list, the position of the decoded picture of the reference layer within the reference picture list may be determined using encoding structure information. have.

Hereinafter, a method of determining a position of a decoded picture (inter-layer reference picture) of a reference layer in a reference picture list in the present invention will be described in detail.

According to an embodiment of the present invention, Whenever the reference picture list is generated, a decoded picture of the reference layer may be added to a fixed position.

5 is a diagram schematically illustrating an example of a method of constructing a reference picture list according to an embodiment of the present invention.

Referring to FIG. 5, <1> is a reference picture list constructed without considering a decoded picture of a reference layer, and <2> is a reference picture list including decoded pictures of a reference layer.

In the example of FIG. 5, when the reference picture list composed of only pictures of the current layer is equal to <1>, the decoded picture of the reference layer (inter-layer reference picture) may be added to the second position of the reference picture list.

Here, for convenience of description, it has been described that the inter-layer reference picture is located at the second position of the reference picture list, but the present invention is not limited thereto, and the inter-layer reference picture (a decoded picture of the reference layer) is a reference picture list It could also be added to my other (non-second) fixed predetermined location. In this case, the position of the inter-layer reference picture in the reference picture list may be the same position for the entire video.

In FIG. 5, a reference picture list L0 is a reference picture list used for inter prediction of a P slice or a reference picture list used as a first reference picture list in inter prediction of a B slice. The reference picture list L1 is a second reference picture list used for inter prediction of a B slice. In addition, a P slice is a slice in which intra prediction is performed or inter prediction is performed using a maximum of one motion vector per prediction block and a reference picture index. A bi-predictive slice is a slice in which intra prediction is performed or prediction is performed using up to two motion vectors and reference picture indexes per prediction block. In this regard, an I slice is a slice to which only intra prediction is applied.

In the example of FIG. 5, it has been described that the position of the inter-layer reference picture in the reference picture list L0 is the same as the position of the inter-layer reference picture in the reference picture list L1, but the present invention is not limited thereto, and the position of the inter-layer reference picture is referenced. It may be different in the picture list L0 and the reference picture list L1. For example, for L0, the interlayer reference picture can be placed at a position after the reference picture set preceding the current picture in the reference picture list, and for L1, the interlayer reference picture is after the long-term reference picture set in the reference picture list. Can be placed in the position of.

In addition, in the example of FIG. 5, it is said that the inter-layer reference picture is at the second position in the reference picture list. In this case, the second position may mean the second picture, or the second position when the reference picture set is set It may mean that. For example, with respect to L0, the inter-layer reference picture may be placed at a position next to the reference picture set preceding the current picture in the reference picture list.

For example, when the current block (the current encoding/decoding target block) belongs to the B slice, as shown in FIG. 5, bidirectional reference picture lists L0 and L1 may exist. In this case, the decoded picture (inter-layer reference picture) of the reference layer may be added to the reference picture list L0 and L1 in both directions (L0 direction and L1 direction) at a fixed position.

Also, for example, when the current block belongs to the B slice, a bidirectional reference picture list may exist as shown in FIG. 5. In this case, an inter-layer reference picture may be added at a fixed position to only one of the reference picture lists in both directions (ie, L0 or L1).

Also, for example, when the current block is an I slice or a P slice, a decoded picture (inter-layer reference picture) of the reference layer may be added to a fixed position only in the reference picture list L0.

In addition, an inter-layer reference picture is added only to a list specified by information indicating which reference picture list to use for inter-layer prediction of the current block or current picture, for example, transmitted from the SPS (e.g., LIST0ILRPflag, LIST1ILRPflag) You may.

In addition, if the position of the inter-layer reference picture (the decoded picture of the reference layer) in the reference picture list is fixed at the N-th position, another reference picture is placed at the N-th position in the reference picture list configured without considering inter-layer prediction. If there is, the inter-layer reference picture may be placed at the N-th position, and the order of the pictures at the N-th position in the reference picture list configured without considering inter-layer prediction may be pushed back one by one.

In addition, when the position of the inter-layer reference picture (the decoded picture of the reference layer) in the reference picture list is fixedly at the N-th position, the reference picture that was at the N-th position in the reference picture list configured without considering inter-layer prediction May be replaced with an inter-layer reference picture.

According to another embodiment of the present invention, each time a reference picture list is generated, a position of a decoded picture (inter-layer reference picture) of a reference layer in the reference picture list may be determined using encoding information.

In the present embodiment, a position of an inter-layer reference picture in a reference picture list for a current block may be determined by using temporal level information of a current encoding/decoding target picture (current picture).

6 is a diagram schematically illustrating an example of temporal levels for each picture in a current layer. The temporal level of the picture may be transmitted from the encoding apparatus through a syntax element such as nuh_temporal_id.

Considering the case where the temporal level is determined as shown in FIG. 6 as an example, when the temporal level of the current picture is 1, the inter-layer reference picture is located at the first position (the value of the reference picture index is 0) in the reference picture list for the current block. Can be placed.

When determining the position of the inter-layer reference picture (the decoded picture of the reference layer) according to the temporal level, the inter-layer reference picture is used for the current picture (coding/decoding target picture) having the highest temporal level. It may not be included in the reference picture list.

As another example, for example, a picture of a layer having a temporal level equal to or greater than a specific level may not be included in the reference picture list as an inter-layer reference picture.

Meanwhile, the position of the inter-layer reference picture in the reference picture list may be determined according to the range of the temporal level. For example, when the temporal level of the current picture is 2 or more, the inter-layer reference picture can be positioned at the end of the reference picture list, and when the temporal level of the current picture is less than 2, the inter-layer reference picture is referred to as the first position of the reference picture list. You can also put it on.

When adding an inter-layer reference picture to the reference picture list, if there is a bi-predictive reference picture list, it is implicitly inter-layered only in the reference picture list in one direction (reference picture list L0 or reference picture list L1). You can also add a reference picture. For example, if the temporal level of the current picture is 0, an inter-layer reference picture is added to the first position in both direction lists (reference picture list L0 and reference picture list L1), and the temporal level of the current picture is greater than 0. In this case, the inter-layer reference picture list may be added to the first position only in the list in one direction (reference picture list L0 and reference picture list L1). Herein, it has been described that the inter-layer reference picture is added to the first position, but the inter-layer reference picture may be added to a position other than the first according to the temporal level of the current picture.

When a bi-predictive reference picture list exists, the encoding apparatus may add an inter-layer reference picture to reference picture lists in both directions for the entire sequence, or an inter-layer reference picture only to the reference picture list in one direction. You can also add The encoding apparatus may encode and transmit information indicating a reference picture list to which an inter-layer reference picture has been added (eg, LIST0ILRPflag, LIST1ILRPflag) in the SPS. In this case, the decoding apparatus may add an inter-layer reference picture to all reference picture reference picture lists in both directions based on information transmitted from the SPS, or may add an inter-layer reference picture only to the reference picture list in one direction. .

Meanwhile, information specifying the temporal level of the current picture may be transmitted in a Network Abstract Layer (NAL) unit header for the current picture. For example, the encoding device may transmit information specifying the temporal level (temporal_id) of the current picture in the NAL unit header to the decoding device, and the decoding device may transmit the temporal level of the current picture according to information about the temporal level transmitted in the NAL unit header. Can be specified.

In addition, when the inter-layer reference picture (the decoded picture of the reference layer) has a temporal level higher than the temporal level of the current picture, the inter-layer reference picture may not be included in the reference picture list for the current block.

According to another embodiment of the present invention, whenever a reference picture list is generated, a position at which a decoded picture (inter-layer reference picture) of a reference layer is placed in a reference picture list may be determined using prediction structure information.

According to the present embodiment, the position of the inter-layer reference picture in the reference picture list may be determined by using information on the structure of prediction (SOP) of the current encoding/decoding target picture.

7 is a diagram schematically illustrating an example of depth information of a structure of prediction of a current layer.

When the prediction structure as in the example of FIG. 7 is applied, the position of the inter-layer reference picture in the reference picture list for the current block may be determined according to depth information of the prediction structure for the current picture.

For example, taking the example of FIG. 7 into consideration, if the depth corresponding to the current picture in the prediction structure is 0, the position in the reference picture list of the inter-layer reference picture is the first position (the value of the reference picture index is 0). ) Can be determined.

Also, when considering the example of FIG. 7, if the depth corresponding to the current picture in the prediction structure is 1 or more, the position of the inter-layer reference picture in the reference picture list may be determined as the last position.

The encoding apparatus may transmit information specifying a position in a reference picture list of an inter-layer reference picture determined according to depth information in a prediction structure in a picture parameter set (PPS) or a slice header.

The decoding apparatus may determine a position of the inter-layer reference picture in the reference picture list based on information received from the encoding apparatus.

For example, in the example of FIG. 7, when the depth of the prediction structure of the current picture specified by the received information is 0, the decoding apparatus sets the position in the reference picture list of the interlayer reference picture to the first position (the reference picture index). The value can be determined as 0).

Also, for example, in the example of FIG. 7, when the depth of the current picture specified by the received information is 1 or more, the decoding apparatus may determine a position in the reference picture list of the inter-layer reference picture as the last position. .

Meanwhile, in this embodiment, the position of the inter-layer reference picture in the reference picture list may be implicitly determined. In this case, it may be determined whether to add an inter-layer reference picture only to the reference picture list L0 according to depth information of the prediction structure. At this time, so that the decoding device can also add the inter-layer reference picture to the same position in the reference picture list, the encoding device encodes information specifying the reference picture list to which the inter-layer reference picture is added to, and performs a PPS (Picture Parameter Sets) or slice. Can be transmitted in the header. The decoding apparatus may perform prediction on the current block by adding an inter-layer reference picture to the reference picture list based on the received information.

Meanwhile, up to now, it has been described as'the location of the inter-layer reference picture in the reference picture list', but this is for convenience of description, and the location of the inter-layer reference picture in the reference picture list is'reference picture index indicating the inter-layer reference picture. May be a position in the reference picture list of'or'order of values of a reference picture index indicating an inter-layer reference picture in the reference picture list'.

8 is a flowchart schematically illustrating an example of a video encoding method according to the present invention.

Referring to FIG. 8, the encoding apparatus may decode a picture of a reference layer (S810). The picture to be decoded may be a picture belonging to the same access unit as the encoding target picture (current picture) of the current layer.

The encoding apparatus may derive an inter-layer reference picture (a decoded picture of the reference layer) by sampling the decoded picture of the reference layer (S820). The encoding apparatus may derive an inter-layer reference picture that can be used as a reference picture for a current block (a block to be encoded within a current picture of the current layer) from the decoded (restored) picture of the reference layer.

The encoding apparatus may configure a reference picture including an inter-layer reference picture and a reference picture of a current layer (S830).

In this case, in the current layer, the position of the inter-layer reference picture in the reference picture list may be a fixed specific position. For example, a position of the inter-layer reference picture in the reference picture list may be fixed to a position following the reference pictures preceding the picture to which the current block belongs in the current layer.

The encoding apparatus may configure a reference picture list to include at least one inter-layer reference picture derived from at least one or more reference layers.

Also, the encoding apparatus may configure a reference picture list based on the temporal level of the current picture. For example, when the temporal level of the current picture is greater than or equal to a specific range level, the encoding apparatus may exclude the inter-layer reference picture from the reference picture list. That is, when the temporal level of the current picture is greater than or equal to a specific range level, the encoding apparatus may not refer to the decoded picture of the reference layer for inter prediction of the current block.

The encoding apparatus may perform prediction on the current sample of the current layer based on the reference picture list to perform the prediction sample on the current block (S840). When inter-layer prediction is applied, the encoding apparatus may perform inter prediction on the current block by referring to the inter-layer reference picture.

The encoding apparatus may generate a residual sample for the current block based on the prediction sample (S850). The encoding apparatus may derive a difference value between the original signal of the current block and the prediction signal as a residual sample.

The encoding apparatus may entropy-encode the residual sample and transmit it to the decoding apparatus (S860). The encoding apparatus may transform and quantize the residual sample, and then entropy-encode the residual sample and transmit it to the decoding apparatus.

9 is a flowchart schematically illustrating an example of a video decoding method according to the present invention.

Referring to FIG. 9, the decoding apparatus may decode a picture of a reference layer (S910). The picture to be decoded may be a picture belonging to the same access unit as the picture to be decoded (current picture) of the current layer.

The decoding apparatus may derive an inter-layer reference picture (a decoded picture of the reference layer) by sampling the decoded picture of the reference layer (S920). The decoding apparatus may derive an inter-layer reference picture that can be used as a reference picture for a current block (a block to be decoded within a current picture of the current layer) from the decoded (restored) picture of the reference layer.

The decoding apparatus may configure a reference picture including an inter-layer reference picture and a reference picture of a current layer (S930).

In this case, in the current layer, the position of the inter-layer reference picture in the reference picture list may be a fixed specific position. For example, a position of the inter-layer reference picture in the reference picture list may be fixed to a position following the reference pictures preceding the picture to which the current block belongs in the current layer.

The decoding apparatus may configure a reference picture list to include at least one or more inter-layer reference pictures derived from at least one or more reference layers.

Also, the decoding apparatus may configure a reference picture list based on the temporal level of the current picture. For example, when the temporal level of the current picture is greater than or equal to a specific range level, the decoding apparatus may exclude the inter-layer reference picture from the reference picture list. That is, when the temporal level of the current picture is greater than or equal to a specific range level, the decoding apparatus may not refer to the decoded picture of the reference layer for inter prediction of the current block.

The decoding apparatus may perform prediction on the current sample of the current layer based on the reference picture list to perform the prediction sample on the current block (S940). When inter-layer prediction is applied, the decoding apparatus may perform inter prediction on the current block by referring to the inter-layer reference picture.

The decoding apparatus may generate a reconstructed sample for the current block based on the prediction sample (S850). The encoding apparatus may derive a reconstructed sample for the current block by adding a prediction sample for the current block and a residual signal (residual sample) received from the encoding apparatus.

The above-described method according to the present invention may be produced as a program for execution in a computer and stored in a computer-readable recording medium. Examples of computer-readable recording media include ROM, RAM, CD-ROM, and magnetic tape. , Floppy disks, optical data storage devices, and the like, and also include those implemented in the form of carrier waves (for example, transmission through the Internet).

The computer-readable recording medium is distributed over a computer system connected by a network, and computer-readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes and code segments for implementing the method can be easily deduced by programmers in the art to which the present invention belongs.

In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. In addition, various modifications can be implemented by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

The above-described embodiments include examples of various aspects. Although not all possible combinations for representing the various aspects can be described, those of ordinary skill in the art will recognize that other combinations are possible. Accordingly, the present invention will be said to cover all other replacements, modifications and changes falling within the scope of the following claims.

Claims (21)

Sampling a picture of a reference layer to derive an inter-layer reference picture;
Constructing a reference picture list including the inter-layer reference picture and a reference picture of a current layer;
Generating a prediction sample for the current block by performing prediction on the current block of the current layer based on the reference picture list;
Generating a residual sample for the current block based on the prediction sample; And
Including the step of encoding the residual sample,
In the current layer, a position of the inter-layer reference picture in the reference picture list is a fixed specific position.
delete The method of claim 1, wherein a position of the inter-layer reference picture in the reference picture list is
And a position next to reference pictures preceding the picture to which the current block belongs in the current layer. The video encoding method of claim 1, wherein the reference picture list includes at least one inter-layer reference picture derived from at least one reference layer. The video encoding method of claim 1, wherein the inter-layer reference picture belongs to the same access unit as a picture to which the current block belongs. The method of claim 1, wherein in the step of constructing the reference picture list including the inter-layer reference picture and a reference picture of a current layer,
And when the temporal level of the inter-layer reference picture is equal to or greater than a predetermined level, the inter-layer reference picture is not included in the reference picture list. The method of claim 6, wherein in the step of constructing the reference picture list including the inter-layer reference picture and the reference picture of the current layer,
And constructing the reference picture list by including an inter-layer reference picture derived from a picture of a reference layer having a temporal level equal to or lower than the temporal level of a picture to which the current block belongs. Decoding a picture of a reference layer;
Deriving an inter-layer reference picture by sampling the decoded picture of the reference layer;
Constructing a reference picture list including the inter-layer reference picture and a reference picture of a current layer;
Generating a prediction sample for the current block by performing prediction on the current block of the current layer based on the reference picture list; And
Generating a reconstructed sample for the current block based on the prediction sample and a residual sample for the current block,
In the current layer, a position of the inter-layer reference picture in the reference picture list is a fixed specific position.
delete The method of claim 8, wherein a position of the inter-layer reference picture in the reference picture list is
And a position next to reference pictures preceding the picture to which the current block belongs in the current layer. The video decoding method of claim 8, wherein the reference picture list includes at least one inter-layer reference picture derived from at least one reference layer. The video decoding method of claim 8, wherein the inter-layer reference picture belongs to the same access unit as a picture to which the current block belongs. The method of claim 8, wherein in the step of constructing the reference picture list including the inter-layer reference picture and a reference picture of a current layer,
And when the temporal level of the inter-layer reference picture is equal to or greater than a predetermined level, the inter-layer reference picture is not included in the reference picture list. The method of claim 13, wherein in the step of constructing the reference picture list including the inter-layer reference picture and the reference picture of the current layer,
And constructing the reference picture list by including an inter-layer reference picture derived from a picture of a reference layer having a temporal level equal to or lower than the temporal level of the picture to which the current block belongs. Sampling a picture of a reference layer to derive an inter-layer reference picture;
Constructing a reference picture list including the inter-layer reference picture and a reference picture of a current layer;
Generating a prediction sample for the current block by performing prediction on the current block of the current layer based on the reference picture list;
Generating a residual sample for the current block based on the prediction sample; And
Including the step of encoding the residual sample,
A recording medium for storing a bitstream generated by a video encoding method, wherein in the current layer, a position of the inter-layer reference picture in the reference picture list is a fixed specific position. delete delete delete delete delete delete

Images