KR102175542B1 - Video coding/decoding method and apparatus for multi-layers - Google Patents

Abstract

In the method for decoding an image supporting a plurality of layers according to the present invention, when constructing an initial reference picture list of a current picture, the step of receiving flag information indicating whether reference picture set information of a reference layer referred to by the current picture is used. Wow; Generating the initial reference picture list based on the flag information; It may include performing prediction on the current picture based on the initial reference picture list. Accordingly, a method of generating a reference picture list including pictures of different layers for a layer to be currently encoded and decoded and an apparatus using the same are provided.

Description

A video encoding and decoding method supporting multiple layers, and an apparatus using the same {VIDEO CODING/DECODING METHOD AND APPARATUS FOR MULTI-LAYERS}

The present invention relates to a video compression technique, and more particularly, to a method and apparatus for performing video coding supporting multiple layers.

Recently, demand for high-resolution and high-quality images such as high definition (HD) images and ultra high definition (UHD) images is increasing in various application fields. The higher the resolution and quality of the video data, the higher the amount of data is compared to the existing video data. Therefore, if the video data is transmitted using a medium such as a wired/wireless broadband line or stored using an existing storage medium, the transmission cost and The storage cost increases. High-efficiency image compression techniques can be used to solve these problems that occur as image data becomes high-resolution and high-quality.

Inter-screen prediction technology that predicts pixel values included in the current picture from a picture before or after the current picture using image compression technology, intra prediction technology that predicts pixel values included in the current picture using pixel information in the current picture, Various technologies exist, such as an entropy encoding technology that allocates a short code to a value with a high frequency of appearance and a long code to a value with a low frequency of appearance, and it is possible to effectively compress and transmit or store image data using this image compression technology.

An object of the present invention is to provide a method of generating a reference picture list including pictures of different layers for a layer to be currently encoded and decoded and an apparatus using the same.

Another object of the present invention is to provide a method for generating a reference picture list by using information on a reference picture set of a reference layer and an apparatus using the same.

Another object of the present invention is to provide a method of generating a reference picture list and an apparatus using the same by using information on a position at which a reference picture set is added to an initial reference picture list.

In the method of decoding an image supporting a plurality of layers according to an embodiment of the present invention, flag information indicating whether reference picture set information of a reference layer referenced by the current picture is used when configuring an initial reference picture list of a current picture. Receiving a; Generating the initial reference picture list based on the flag information; It may include performing prediction on the current picture based on the initial reference picture list.

The flag information may indicate whether the short-term reference picture set and the long-term reference picture set of the reference layer are derived as the short-term reference picture set and the long-term reference picture set of the current picture.

The flag information may indicate whether the short-term reference picture set of the reference layer is derived as the short-term reference picture set of the current picture.

The flag information may indicate whether the long-term reference picture set of the reference layer is derived as the long-term reference picture set of the current picture.

The reference layer may be a base layer.

The flag information is a position at which the inter-layer reference picture set of the reference layer is added to the initial reference picture list of the reference layer and the inter-layer reference picture set of the current picture is added to the initial reference picture list of the current picture. It can indicate whether to use.

The flag information may indicate whether the inter-layer reference picture set of the reference layer is derived as the inter-layer reference picture set of the current picture.

The flag information may be included in a sequence parameter set and received.

It may further include generating a final reference picture list by modifying the initial reference picture list.

In performing prediction on the current picture, a reference picture included in the inter-layer reference picture set may be used as a reference picture of the current picture.

An image decoding apparatus supporting a plurality of layers according to another embodiment of the present invention includes an entropy decoding unit that decodes information for prediction and decoding of an image received through a beast stream, and an initial reference picture list of the current picture. , The initial reference picture list is generated based on flag information indicating whether reference picture set information of a reference layer referred to by the current picture is used, and prediction for the current picture is performed based on the initial reference picture list. It may include a prediction unit to perform.

According to an embodiment of the present invention, there is provided a method of generating a reference picture list including pictures of different layers for a layer to be currently encoded and decoded, and an apparatus using the same.

According to an embodiment of the present invention, a method of generating a reference picture list using information on a reference picture set of a reference layer and an apparatus using the same are provided.

In addition, according to an embodiment of the present invention, a method of generating a reference picture list using information on a position at which a reference picture set is added to an initial reference picture list and an apparatus using the same are provided.

1 is a block diagram schematically illustrating an encoding apparatus according to an embodiment of the present invention.
2 is a schematic block diagram of a decoding apparatus according to an embodiment of the present invention.
3 is a conceptual diagram illustrating an example of a reference picture set including a short-term reference picture.
4 is a conceptual diagram illustrating a method of inducing a POC of a long-term reference picture.
5 is a diagram illustrating an example of configuring a reference picture list.
6 is a conceptual diagram illustrating an example of configuring an inter-layer reference picture list according to an example of the present invention.
7 is a diagram illustrating a configuration of a reference picture list O according to an example of the present invention.
8 is a diagram illustrating a configuration of a reference picture list O according to another example of the present invention.
9 is a diagram illustrating a configuration of a reference picture list 1 according to an example of the present invention.
10 is a diagram illustrating a configuration of a reference picture list 1 according to another example of the present invention.
11 illustrates a reference picture list constructed using a multiview reference picture according to an example of the present invention.
12 is a diagram illustrating a reference picture list constructed using a multiview reference picture according to another example of the present invention.
13 is a diagram illustrating a reference picture list constructed using an index of a temporal sub-layer according to an example of the present invention.
14 is a diagram illustrating an example of referring to an additional position of an inter-layer reference picture set of a reference layer according to an example of the present invention.
15 is a control flow diagram for explaining a method of decoding an image according to the present invention.

Each of the components disclosed in the embodiments of the present invention and the drawings is disclosed as an independent configuration to represent different characteristic functions of the video encoding apparatus. It does not mean that each component is made up of separate hardware or a single software component. That is, each constituent part is listed and included as a constituent part for convenience of explanation, and at least two of the constituent parts are combined to form a single constituent part, or one constituent part is divided into a plurality of constituent parts to perform a function. The integrated embodiments and separate embodiments of the components are also included in the scope of the present invention unless departing from the essence of the present invention.

In addition, some of the constituent elements disclosed in the present invention are not essential constituent elements that perform essential functions in the present invention, but may be optional constituent elements merely for improving performance. The present invention can be implemented by including only the components essential to implement the essence of the present invention excluding components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement Also included in the scope of the present invention.

In the present invention, various modifications may be made and various embodiments may be provided, and specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments. The terms used in the present specification are only used to describe specific embodiments, and are not intended to limit the technical idea of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present specification, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance the possibility.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. Hereinafter, the same reference numerals are used for the same components in the drawings, and duplicate descriptions for the same components are omitted.

Encoding and decoding of a video supporting a plurality of layers in a bitstream is called scalable coding. Since a strong correlation exists between a plurality of layers, if prediction is performed using this correlation, redundant elements of data can be removed and encoding performance of an image can be improved. Prediction of a current layer to be predicted using information of other layers is hereinafter referred to as inter-layer prediction.

The plurality of layers may have at least one of a resolution, a frame rate, and a color format different from each other, and up-sampling or down-sampling of a layer may be performed to adjust a resolution during inter-layer prediction.

1 is a block diagram schematically illustrating an encoding apparatus according to an embodiment of the present invention.

The encoding apparatus 100 according to the present invention includes an encoding unit 100a for an upper layer and an encoding unit 100b for a lower layer.

The upper layer may be expressed as a current layer or an enhancement layer, and the lower layer may be expressed as a reference layer or a base layer. The upper and lower layers may have at least one of a resolution, a frame rate, and a color format different from each other. When a resolution change is required to perform inter-layer prediction, up-sampling or down-sampling of a layer may be performed.

The encoding unit 100a of the upper layer includes a splitter 110, a prediction unit 100, an intra prediction unit 121, an inter prediction unit 122, an inter-layer prediction unit 123, and a transform unit 130. , A quantization unit 140, a rearrangement unit 150, an entropy encoding unit 160, an inverse quantization unit 170, an inverse transform unit 180, a filter unit 190, and a memory 195 and a MUX 197 can do.

The lower layer encoder 100b includes a splitter 111, a prediction unit 125, an intra prediction unit 126, an inter prediction unit 127, a transform unit 131, a quantization unit 141, and rearrangement. A unit 151, an entropy encoder 161, an inverse quantization unit 171, an inverse transform unit 181, a filter unit 191, and a memory 196 may be included.

The encoding unit may be implemented by the image encoding method described in the following embodiments of the present invention, but operations in some components may not be performed to reduce the complexity of the encoding apparatus or for fast real-time encoding. For example, in performing intra prediction in the prediction unit, a limited number of parts without using a method of selecting the optimal intra prediction mode by using all intra prediction mode methods to perform encoding in real time. A method of selecting one intra-prediction mode among them as the final intra-prediction mode using the intra prediction mode of may be used. As another example, in performing intra prediction or inter prediction, it is possible to restrict the use of the shape of the prediction block used.

A unit of a block processed by the encoding apparatus may be a coding unit that performs encoding, a prediction unit that performs prediction, and a transformation unit that performs transformation. A coding unit may be expressed in terms of a coding unit (CU), a prediction unit (PU), and a transform unit (TU).

The splitters 110 and 111 divide the layer image into a combination of a plurality of coding blocks, prediction blocks, and transform blocks, and use one of the coding blocks, prediction blocks, and transform blocks based on a predetermined criterion (for example, a cost function). You can divide the layer by selecting a combination of. For example, a recursive tree structure such as a quadtree structure may be used to divide a coding unit in a layer image. Hereinafter, in an embodiment of the present invention, the meaning of a coding block may be used not only as a block to be encoded but also to a block to be decoded.

The prediction block may be a unit that performs prediction such as intra prediction or inter prediction. A block that performs intra prediction may be a square block such as 2Nx2N and NxN. As a block that performs inter-screen prediction, the prediction block is divided using 2NxN, Nx2N, or AMP (Asymmetric Motion Partitioning), which is a form obtained by dividing a square shape such as 2Nx2N, NxN or a square prediction block into the same shape. There is a way. A method of performing the transformation in the transform unit 115 may vary according to the shape of the prediction block.

The prediction units 120 and 125 of the encoding units 100a and 100b are the intra prediction units 121 and 126 that perform intra prediction and the inter prediction units that perform inter prediction. (1122, 126) may be included. The prediction unit 120 of the upper layer encoder 100a further includes an inter-layer prediction unit 123 that performs prediction on an upper layer using information on the lower layer.

The prediction units 120 and 125 may determine whether to use inter prediction or intra prediction on the prediction block. A processing unit in which prediction is performed and a unit of a processing block in which a prediction method is determined may be different. For example, in performing intra prediction, a prediction mode may be determined based on a prediction block, and a process of performing prediction may be performed based on a transform block. A residual value (residual block) between the generated prediction block and the original block may be input to the transform units 130 and 131. In addition, prediction mode information, motion vector information, and the like used for prediction may be encoded by the entropy encoder 130 together with a residual value and transmitted to a decoding apparatus.

When using the PCM (Pulse Coded Modulation) encoding mode, it is possible to encode the original block as it is and transmit it to the decoder without performing prediction through the prediction units 120 and 125.

The intra prediction units 121 and 126 may generate an intra-predicted block based on a reference pixel existing around the current block (a block to be predicted). In the intra prediction method, the intra prediction mode may have a directional prediction mode in which reference pixel information is used according to a prediction direction, and a non-directional mode in which the directional information is not used when prediction is performed. A mode for predicting luma information and a mode for predicting color difference information may have different types. In order to predict color difference information, intra prediction mode information for predicting luma information or predicted luma signal information may be used. If the reference pixel is not available, a prediction block may be generated by replacing the non-available reference pixel with another pixel.

The prediction block may include a plurality of transform blocks. If the size of the prediction block and the size of the transform block are the same when performing intra prediction, a pixel on the left side of the prediction block, a pixel on the top left side, and the top In-screen prediction on the prediction block may be performed based on the pixels existing in the screen. However, when performing intra prediction, if the size of the prediction block and the size of the transform block are different and a plurality of transform blocks are included in the prediction block, intra prediction is performed using a reference pixel determined based on the transform block. can do.

In the intra prediction method, a prediction block may be generated after applying a Mode Dependent Intra Smoothing (MDIS) filter to a reference pixel according to an intra prediction mode. The types of MDIS filters applied to the reference pixels may be different. The MDIS filter is an additional filter applied to an intra-predicted block by performing intra prediction, and may be used to reduce a step that exists in an intra-predicted block generated after prediction with a reference pixel. In performing MDIS filtering, different filtering may be performed on a reference pixel and some columns included in the predicted block in the screen according to the direction of the prediction mode in the screen.

The inter prediction units 122 and 127 may perform prediction by referring to information on a block included in at least one picture of a picture before or after a current picture. The inter prediction units 122 and 127 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

The reference picture interpolation unit may receive reference picture information from the memories 195 and 196 and generate pixel information of an integer number of pixels or less from the reference picture. In the case of a luma pixel, a DCT-based interpolation filter with different filter coefficients may be used to generate pixel information of an integer pixel or less in units of 1/4 pixels. In the case of a color difference signal, a DCT-based interpolation filter with different filter coefficients may be used to generate pixel information of an integer pixel or less in units of 1/8 pixels.

The inter prediction units 122 and 127 may perform motion prediction based on the reference picture interpolated by the reference picture interpolation unit. Various methods such as a full search-based block matching algorithm (FBMA), a three step search (TSS), and a new three-step search algorithm (NTS) can be used as a method for calculating a motion vector. The motion vector may have a motion vector value of 1/2 or 1/4 pixel based on the interpolated pixel. The inter prediction units 122 and 127 may perform prediction on the current block by applying one of several inter prediction methods.

Various methods, such as a skip method, a merge method, and a motion vector prediction (MVP) method, may be used as an inter prediction method.

In inter prediction, motion information, that is, information such as an index of a reference picture, a motion vector, and a residual signal, is entropy-coded and transmitted to a decoder. When the skip mode is applied, residuals may not be generated, transformed, quantized, or transmitted.

The inter-layer prediction unit 123 performs inter-layer prediction for predicting an upper layer using information on a lower layer. The inter-layer prediction unit 123 uses inter-layer texture prediction and inter-layer inter prediction using texture of a lower layer, intra prediction mode information, motion information, and syntax information. ), inter-layer syntax prediction, etc. may be performed.

Inter-layer texture prediction means using a texture of a reference block in a lower layer as a prediction sample of a current block of an upper layer, and in this case, the texture of the reference block may be scaled by upsampling.

For inter-layer texture prediction, the intra-BL method and the up-sampled lower layer upsample the reconstructed value of the reference block in the lower layer and use the upsampled reference block as a prediction value for the current block to encode the residual value from the current block. There may be a reference index method that stores in memory and uses the stored lower layer as a reference index.

The intra prediction of the upper layer may be performed by using the intra prediction mode information of the lower layer. In this case, the intra prediction mode of the lower layer may be referred to as a BL intra mode.

Inter-layer motion prediction is also referred to as inter-layer inter prediction, and according to inter-layer motion prediction, prediction of a current block of an upper layer may be performed using motion information of a lower layer. The motion information may include a motion vector and a reference picture index.

In addition, the inter-layer prediction unit 123 may predict the texture of the current block or perform inter-layer syntax prediction by using the syntax information of the lower layer. In this case, the syntax information of the lower layer used for prediction of the current block may be information about an intra prediction mode, motion information, and the like.

As another example of inter-layer prediction, according to the inter-layer difference prediction, prediction for the current block is performed using a difference image generated as a difference value between a reconstructed image of an upper layer and an image obtained by up-sampling a reconstructed image of a lower layer. I can.

As an example of inter-layer prediction, inter-layer texture prediction, inter-layer motion prediction, inter-layer syntax prediction, and inter-layer difference prediction have been described, but inter-layer prediction applicable in the present invention is not limited thereto.

A residual block including residual information that is a difference value from the prediction block generated by the prediction blocks 120 and 125 and the reconstructed block of the prediction block is generated, and the residual block is input to the transform units 130 and 131.

The transform units 130 and 131 may transform the residual block using a transform method such as Discrete Cosine Transform (DST) or Discrete Sine Transform (DST). Whether to apply DCT or DST to transform the residual block may be determined based on intra prediction mode information of the prediction block used to generate the residual block and information on the size of the prediction block. That is, the transform units 130 and 131 may apply different transform methods according to the size of the prediction block and the prediction method.

The quantization units 140 and 141 may quantize values converted into the frequency domain by the transform units 130 and 131. Quantization coefficients may vary depending on the block or the importance of the image. The values calculated by the quantization units 140 and 141 may be provided to the inverse quantization units 170 and 17 and the rearrangement units 150 and 151.

The reordering units 150 and 151 may rearrange coefficient values on the quantized residual values. The reordering units 150 and 151 may change a 2-dimensional block shape coefficient into a 1-dimensional vector shape through a coefficient scanning method. For example, the rearrangement units 150 and 151 may scan from DC coefficients to coefficients in a high frequency region using a Zig-Zag Scan method and change the form to a one-dimensional vector. Depending on the size of the transform block and the prediction mode in the screen, a vertical scan method that scans two-dimensional block shape coefficients in a column direction rather than a zig-zag scan method, and a horizontal scan method that scans two-dimensional block shape coefficients in a row direction. Can be used. That is, it is possible to determine which scan method is to be used among zig-zag scan, vertical direction scan, and horizontal direction scan according to the size of the transform block and the intra prediction mode.

The entropy encoding units 160 and 161 may perform entropy encoding based on values calculated by the reordering units 150 and 151. Entropy coding may use various coding methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

The entropy encoders 160 and 161 transmit residual coefficient information and block type information, prediction mode information, division unit information, prediction block information, and transmission of the coding block from the reordering units 150 and 151 and the prediction units 120 and 125. Entropy encoding may be performed based on a predetermined encoding method by receiving various information such as unit information, motion vector information, reference frame information, block interpolation information, and filtering information. In addition, the entropy encoders 160 and 161 may entropy-encode the coefficient values of the coding units input from the reordering units 150 and 151.

The entropy encoders 160 and 161 may encode intra prediction mode information of the current block by performing binarization on the intra prediction mode information. The entropy encoders 160 and 161 may include a codeword mapping unit for performing such a binarization operation, and may perform different binarization according to the size of a prediction block performing intra prediction. In the codeword mapping unit, the codeword mapping table may be adaptively generated or stored in advance through a binarization operation. In another embodiment, the entropy encoders 160 and 161 may express information on the prediction mode in the current screen by using a codenum mapping unit performing codenum mapping and a codeword mapping unit performing codeword mapping. In the codenum mapping unit and the codeword mapping unit, a codenum mapping table and a codeword mapping table may be generated or stored.

The inverse quantization units 170 and 171 and the inverse transform units 180 and 181 inverse quantize values quantized by the quantization units 140 and 141 and inverse transform the values transformed by the transform units 130 and 131. The residual values generated by the inverse quantization units 170 and 171 and the inverse transform units 180 and 181 are predicted through a motion estimation unit, a motion compensation unit, and an intra prediction unit included in the prediction units 120 and 125. The prediction block may be combined to generate a reconstructed block.

The filter units 190 and 191 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

The deblocking filter can remove block distortion caused by the boundary between blocks in the reconstructed picture. In order to determine whether to perform deblocking, it may be determined whether to apply the deblocking filter to the current block based on the pixels included in several columns or rows included in the block. When applying a deblocking filter to a block, a strong filter or a weak filter may be applied according to the required deblocking filtering strength. In addition, in applying the deblocking filter, horizontal filtering and vertical filtering may be processed in parallel when vertical filtering and horizontal filtering are performed.

The offset correction unit may correct an offset from the original image for the deblocking image in pixel units. In order to perform offset correction for a specific picture, the pixels included in the image are divided into a certain number of areas, and then the area to be offset is determined and the offset is applied to the area, or offset by considering the edge information of each pixel. You can use the method of applying.

ALF (Adaptive Loop Filter) may perform filtering based on a value obtained by comparing the filtered reconstructed image and the original image. After dividing the pixels included in the image into at least one group, one filter to be applied to the corresponding group may be determined, and filtering may be performed differentially for each group.

The filter units 190 and 191 may apply only the deblocking filter, only the deblocking filter and ALF, or only the deblocking filter and the offset correction unit without applying all of the deblocking filter, ALF, and offset correction unit.

The memories 195 and 196 may store the reconstructed blocks or pictures calculated through the filter units 190 and 191, and the stored reconstructed blocks or pictures may be provided to the prediction units 120 and 125 when performing inter prediction. have.

Information output from the entropy encoder 100b of the lower layer and information output from the entropy encoder 100a of the upper layer may be multiplexed by the MUX 197 and output as a bitstream.

The MUX 197 may be included in the encoder 100b of the lower layer, or may be implemented as an independent device or module separate from the encoder 100.

2 is a schematic block diagram of a decoding apparatus according to an embodiment of the present invention.

As shown in FIG. 2, the decoding apparatus 200 includes a decoder 200a of an upper layer and a decoder 200b of a lower layer.

The decoding unit 200a of the upper layer includes an entropy decoding unit 210, a rearrangement unit 220, an inverse quantization unit 230, an inverse transform unit 245, a prediction unit 250, a filter unit 260, and a memory 240. ) Can be included.

The decoding unit 200b of the lower layer includes an entropy decoding unit 211, a rearrangement unit 221, an inverse quantization unit 231, an inverse transform unit 241, a prediction unit 251, a filter unit 261, and a memory 271. ) Can be included.

When a bitstream including a plurality of layers is transmitted from the encoding apparatus, the DEMUX 280 may demultiplex the information for each layer and transmit the demultiplexed information to the decoders 200a and 200b for each layer. The input bitstream may be decoded in a procedure opposite to that of the encoding device.

The entropy decoders 210 and 211 may perform entropy decoding in a procedure opposite to that of entropy encoding performed by the entropy encoder of the encoding apparatus. Among the information decoded by the entropy decoding units 210 and 211, information for generating a prediction block is provided to the prediction units 250 and 251, and the residual value obtained by performing entropy decoding in the entropy decoding unit is reordering units 220 and 221. Can be entered as

Like the entropy encoders 160 and 161, the entropy decoders 210 and 211 may perform inverse transformation using at least one of CABAC and CAVLC.

The entropy decoders 210 and 211 may decode information related to intra prediction and inter prediction performed by the encoding apparatus. The entropy decoding unit may include a codeword mapping unit to include a codeword mapping table for generating a received codeword as an intra prediction mode number. The codeword mapping table may be stored in advance or may be adaptively generated. When using the code-num mapping table, a code-num mapping unit for performing the code-num mapping may be additionally provided.

The reordering units 220 and 221 may perform reordering based on a method of rearranging the bitstream entropy-decoded by the entropy decoders 210 and 211 in the encoder. The coefficients expressed in the form of a one-dimensional vector can be reconstructed into coefficients in the form of a two-dimensional block and rearranged. The reordering unit may perform reordering through a method of receiving information related to coefficient scanning performed by the encoder and performing reverse scanning based on the scanning order performed by the corresponding encoder.

The inverse quantization units 230 and 231 may perform inverse quantization based on a quantization parameter provided by an encoding apparatus and a coefficient value of a rearranged block.

The inverse transform units 240 and 241 may perform inverse DCT and inverse DST with respect to the DCT and DST performed by the transform units 130 and 131 on the quantization result performed by the encoding apparatus. The inverse transformation may be performed based on a transmission unit determined by the encoding device. In the transform unit of the encoding device, DCT and DST may be selectively performed according to a plurality of information such as a prediction method, the size of the current block, and a prediction direction, and in the inverse transform unit 225 of the decoding device, the transform unit of the encoding device Inverse transformation may be performed based on transformation information. When performing transformation, transformation may be performed based on a coding block rather than a transformation block.

The prediction units 250 and 251 may generate the prediction blocks based on the prediction block generation-related information provided from the entropy decoders 210 and 211 and previously decoded block or picture information provided from the memories 270 and 271. .

The prediction units 250 and 251 may include a prediction unit determining unit, an inter prediction unit, and an intra prediction unit.

The prediction unit determining unit receives various information such as prediction unit information input from the entropy decoder, prediction mode information of the intra prediction method, motion prediction related information of the inter prediction method, and so on, and classifies the prediction block from the current coding block. It is possible to determine whether inter prediction is performed or intra prediction is performed.

The inter-prediction unit uses information necessary for inter-prediction of the current prediction block provided by the encoding device to determine the current prediction block based on information included in at least one picture of a previous picture or a subsequent picture of the current picture including the current prediction block. You can perform prediction between screens. Whether the motion prediction method of the prediction block included in the coding block is a skip mode, merge mode, or AMVP mode based on the coding block to perform inter prediction Can judge.

The intra prediction unit may generate a prediction block based on pixel information in the current picture. When the prediction block is a prediction block that has performed intra prediction, intra prediction may be performed based on intra prediction mode information of the prediction block provided by the encoding device. The intra prediction unit is an MDIS filter that performs filtering on the reference pixel of the current block, a reference pixel interpolation unit that interpolates the reference pixel to generate a reference pixel of an integer value or less, and filters when the prediction mode of the current block is the DC mode. A DC filter that generates a prediction block through may be included.

The prediction unit 250 of the upper layer decoder 200a may further include an inter-layer prediction unit that performs inter-layer prediction for predicting an upper layer using information on a lower layer.

The inter-layer prediction unit uses the texture of the lower layer, intra prediction mode information, motion information, and syntax information to predict inter-layer texture prediction, inter-layer inter prediction, and inter-layer. Inter-layer syntax prediction can be performed.

According to inter-layer texture prediction, prediction using a texture of a reference block in a lower layer as a prediction value of a current block of an upper layer may be performed. The texture of the reference block can be scaled by upsampling.

For inter-layer texture prediction, the intra BL method and the upsampled base layer are used to upsample the reconstructed value of the reference block in the lower layer and use the upsampled reference block as a prediction value for the current block to encode the residual value from the current block. There is a reference index method that stores the data in memory and uses the stored base layer as a reference index.

The intra prediction of the upper layer may be performed using the intra prediction mode information of the lower layer, and in this case, the intra prediction mode of the lower layer may be expressed as a BL intra mode.

According to inter-layer motion prediction, prediction of a current block of an upper layer may be performed using motion information of a lower layer.

In addition, the inter-layer prediction unit may predict the texture of the current block or perform inter-layer syntax prediction to generate the texture of the current block by using the syntax information of the lower layer. In this case, the syntax information of the lower layer used for prediction of the current block may be information about an intra prediction mode, motion information, and the like.

In addition, the inter-layer prediction unit may perform inter-layer differential prediction for predicting a current block by using a difference image generated as a difference value between a reconstructed image of an upper layer and an image obtained by up-sampling a reconstructed image of a lower layer.

As an example of inter-layer prediction, inter-layer texture prediction, inter-layer motion prediction, inter-layer syntax prediction, and inter-layer difference prediction have been described, but inter-layer prediction applicable in the present invention is not limited thereto.

The reconstructed block or picture may be provided to the filter units 260 and 261. The filter units 260 and 261 may include a deblocking filter, an offset correction unit, and an ALF.

Information on whether a deblocking filter has been applied to a corresponding block or picture, and when a deblocking filter is applied, information on whether a strong filter or a weak filter is applied may be provided from the encoding apparatus. In the deblocking filter of the decoding apparatus, information related to the deblocking filter provided by the encoding apparatus may be provided, and the decoding apparatus may perform deblocking filtering on a corresponding block.

The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image during encoding and offset value information.

The adaptive loop filter (ALF) may perform filtering based on a value obtained by comparing the reconstructed image and the original image after filtering is performed. The ALF may be applied to a coding unit based on information on whether to apply ALF and information on ALF coefficients provided from the encoding device. Such ALF information may be included in a specific parameter set and provided.

The memories 270 and 271 may store the reconstructed picture or block so that it can be used as a reference picture or a reference block, and output the reconstructed picture.

The encoding device and the decoding device may perform encoding on three or more layers instead of two layers, and in this case, the encoder for the upper layer and the decoder for the upper layer are provided in multiple numbers corresponding to the number of upper layers. Can be.

Hereinafter, in an embodiment of the present invention, for convenience of description, a coding unit is used as a term of a coding block, but it may be a block that performs not only encoding but also decoding. Hereinafter, a method for generating a reference picture list described with reference to the drawings according to an embodiment of the present invention may be implemented according to the functions of each module described above in FIGS. 1 and 2, and such an encoding device and a decoding device are Included in the scope.

As described above, in coding an image supporting a multi-layer structure, a method of encoding/decoding an enhancement layer using motion information and texture information of the base layer, and a coding unit (CU) of the base layer, Alternatively, there is a method of encoding/decoding using an up-sampled base layer as a reference picture without changing a decoding method in units of prediction units (PU). The latter method may be expressed in a reference picture index (RefIdx) mode.

When the reference picture index (RefIdx) mode is used, a picture already decoded in the enhancement layer can be used as a reference picture, or an image of a base layer or a lower layer can be upsampled to be used as a reference picture.

Hereinafter, in the present invention, when an image of a base layer or a lower layer is upsampled and used as a reference picture, the upsampled base layer or lower layer image is expressed as an inter-layer reference picture.

Even in an image that supports a multi-layer structure for multi-views, inter prediction can be performed by using a layer for a view other than the current layer as a reference picture. Layers of other views referenced for encoding and decoding of the corresponding layer are also inter-predictioned. It may be expressed as a layer reference picture.

That is, when constructing a reference picture set of an enhancement layer for coding an image having a multi-layer structure supporting scalability rather than a single layer, an inter-layer reference picture must also be considered.

In general, inter prediction may use at least one of a previous picture or a subsequent picture of the current picture as a reference picture, and prediction on a current block may be performed based on the reference picture. An image used for prediction of the current block is called a reference picture or a reference frame.

The reference picture is specified by a reference picture index (refIdx), and a predetermined region in the reference picture is specified as a reference block through a motion vector.

Inter prediction can generate a prediction block for the current block by selecting a reference picture and a reference block corresponding to the current block from within the reference picture.

In inter prediction, the encoding apparatus and the decoding apparatus may derive motion information of a current block and then perform inter prediction and/or motion compensation based on the derived motion information. In this case, the encoding apparatus and the decoding apparatus move a collocated block corresponding to the current block within the reconstructed neighboring block and/or the already reconstructed collocated picture. By using the information, it is possible to improve the encoding/decoding efficiency.

Here, the reconstructed neighboring block is a block in the current picture that has already been encoded and/or decoded and reconstructed, and may include a block adjacent to the current block and/or a block located at an outer corner of the current block. In addition, the encoding device and the decoding device may determine a predetermined relative position based on a block existing at a position spatially corresponding to the current block in the collocated picture, and the determined relative position (spatially corresponding to the current block The call block may be derived based on a location inside and/or outside of a block existing at a location where the call is made. Here, as an example, the collocated picture may correspond to one picture from among the reference pictures included in the reference picture list.

In the inter prediction, a prediction block may be generated such that a residual signal with a current block is minimized and a motion vector size is also minimized.

Meanwhile, the motion information derivation method may vary according to the prediction mode of the current block. Prediction modes applied for inter prediction may include Advanced Motion Vector Predictor (AMVP), merge, and the like.

For example, when AMVP (Advanced Motion Vector Predictor) is applied, the encoding device and the decoding device may generate a motion vector candidate list using a motion vector of a reconstructed neighboring block and/or a motion vector of a collocated block. That is, a motion vector of a reconstructed neighboring block and/or a motion vector of a collocated block may be used as a motion vector candidate. The encoding apparatus may transmit a predicted motion vector index indicating an optimal motion vector selected from among the motion vector candidates included in the list to the decoding apparatus. In this case, the decoding apparatus may select a predicted motion vector of the current block from among motion vector candidates included in the motion vector candidate list using the motion vector index.

The encoding apparatus may obtain a motion vector difference (MVD) between the motion vector of the current block and the predicted motion vector, encode the motion vector, and transmit it to the decoding apparatus. In this case, the decoding apparatus may decode the received motion vector difference, and may derive a motion vector of the current block through the sum of the decoded motion vector difference and the predicted motion vector.

The encoding apparatus may also transmit a reference picture index indicating a reference picture to the decoding apparatus.

The decoding apparatus may predict a motion vector of a current block using motion information of a neighboring block, and derive a motion vector for the current block using a residual received from the encoding apparatus. The decoding apparatus may generate a prediction block for the current block based on the derived motion vector and reference picture index information received from the encoding apparatus.

As another example, when merge is applied, the encoding apparatus and the decoding apparatus may generate a merge candidate list by using motion information of a reconstructed neighboring block and/or motion information of a collocated block. That is, the encoding apparatus and the decoding apparatus may use the reconstructed neighboring block and/or the collocated block motion information as a merge candidate for the current block when there is motion information.

The encoding apparatus may select a merge candidate capable of providing optimal encoding efficiency among merge candidates included in the merge candidate list as motion information for the current block. In this case, a merge index indicating the selected merge candidate may be included in the bitstream and transmitted to the decoding apparatus. The decoding apparatus may select one of merge candidates included in a merge candidate list by using the transmitted merge index, and determine the selected merge candidate as motion information of the current block. Accordingly, when the merge mode is applied, motion information of a reconstructed neighboring block and/or a collocated block may be used as it is as motion information of the current block. The decoding apparatus may reconstruct the current block by adding the prediction block and the residual transmitted from the encoding apparatus.

In the above-described AMVP and merge modes, motion information of a reconstructed neighboring block and/or motion information of a collocated block may be used to derive motion information of a current block.

In the case of the skip mode, which is one of the other modes used for inter prediction, information of neighboring blocks may be used as it is for the current block. Accordingly, in the case of the skip mode, the encoding apparatus does not transmit syntax information such as residual to the decoding apparatus other than information indicating which block motion information is to be used as motion information of the current block.

The encoding apparatus and the decoding apparatus may generate a prediction block of the current block by performing motion compensation on the current block based on the derived motion information. Here, the prediction block may mean a motion-compensated block generated as a result of performing motion compensation on the current block. In addition, a plurality of motion-compensated blocks may constitute one motion-compensated image.

The decoding apparatus may check the skip flag, merge flag, etc. received from the encoding apparatus for motion information necessary for inter prediction of the current block, such as motion vector and reference picture index, and derive correspondingly thereto.

A processing unit in which prediction is performed and a processing unit in which a prediction method and specific content are determined may be different. For example, a prediction mode may be determined in units of prediction blocks, and prediction may be performed in units of transform blocks, or a prediction mode may be determined in units of prediction blocks, and intra prediction may be performed in units of transform blocks.

Pictures encoded/decoded before the current picture may be stored in a memory (eg, Decoded Picture Buffer: DPB) and used for prediction of a current block (current picture). The list of pictures available for inter prediction of the current block is maintained as a reference picture list.

The P slice is a slice that is decoded through intra prediction or inter prediction using at most one motion vector and one reference picture. The B slice is a slice that is decoded through intra prediction or inter prediction using up to two motion vectors and two reference pictures. In this case, the reference picture includes a short term reference picture and a long term reference picture. A picture may be specified as a POC (Picture Order Count) indicating a display order, short-term reference pictures may be pictures with a small difference in POC from the current picture, and long-term reference pictures are pictures having a large difference in POC from the current picture. I can.

Reference picture list 0 (reference picture list 0, referred to as'L0' for convenience of description below) is a reference picture list used for inter prediction of a P slice or a B slice. Reference picture list 1 (referred to as'L1' for convenience of description below) is used for inter prediction of B slices. Accordingly, L0 is used for inter prediction of a block of P slice performing unidirectional prediction, and L0 and L1 are used for inter prediction of a block of B slice performing bidirectional prediction.

When decoding a P slice and a B slice through inter prediction, the decoding apparatus constructs a reference picture list. A reference picture used for inter prediction is designated through a reference picture list. The reference picture index is an index indicating a reference picture on the reference picture list.

The reference picture list may be configured based on a reference picture set transmitted from an encoding device. The reference picture set may include a POC of a picture used as a reference picture and a flag (used_by_curr_pic_s0_flag) indicating whether the picture is directly referenced. Reference pictures constituting the reference picture list may be stored in a memory (eg, DPB). Pictures stored in the memory (pictures encoded/decoded before the current picture) are managed by an encoding device and a decoding device.

3 is a conceptual diagram illustrating an example of a reference picture set including a short-term reference picture.

As shown, the reference picture set (RPS) is a POC of a picture for a short-term reference picture to be stored in the DPB at the current time point for each picture, and a flag indicating whether or not a specific picture is directly referenced in the current picture. It can be composed of information.

For example, when inter prediction of a picture with a POC of 21, all three pictures (pictures with a POC of 20, 19, and 16) can be referenced as short-term reference pictures, and all three pictures have a used_by_curr_pic_s0_flag value of 1, so the POC They are all directly used for prediction of pictures with 21.

Meanwhile, when inter prediction of a picture with a POC of 26, all three pictures (POCs of 25, 24, and 20) may be referenced as short-term reference pictures, but the used_by_curr_pic_s0_flag value of a picture with a POC of 25 is a picture with a POC of 24 and 20. Unlike the value of used_by_curr_pic_s0_flag, it is 0. In this case, a picture with a POC of 25 is not directly used for inter prediction of a picture with a POC of 26.

A picture not shown in the reference picture set in the current picture may be removed from the DPB after an indication that it is not used as a reference picture (unused for reference).

4 is a conceptual diagram illustrating a method of inducing a POC of a long-term reference picture.

As shown, in the case of a long-term reference picture, since the difference between the current picture and the POC is large, it can be expressed using the least significant bit (LSB) and the most significant bit (MSB) of the POC.

MaxPocLsb means the maximum value that can be expressed in LSB. When MaxPocLsb is 32, a long-term reference picture (LTRF) with a POC of 84 can be expressed as 32*2+20, where LSB is 20 and MSB is 2.

In the case of the current picture with a POC of 338, it can be expressed as 32*10+11 when the maximum value that can be expressed in LSB is 32, so 10 becomes the MSB value and 11 becomes the LSB value.

delta_poc_msb_cycle_lt is a value for determining DeltaPocMsbCycleLt, the MSB of POC, in the long-term reference picture set of the current picture. DeltaPocMsbCycleLt may correspond to a difference between the MSB of the POC of the current picture and the MSB of the POC of the reference picture.

In the case of a long-term reference picture, the POC of the reference picture may be derived using the LSB value of the POC of the reference picture and the MSB of the POC of the current picture and the POC of the reference picture.

For example, a long-term reference picture (LTRP[0]) with a POC of 331 of the current picture and a POC of 308 indexed with 0 has an LSB value of 20, which is the difference between the MSB of the POC of the current picture and the POC of the reference picture. If you use 1, it can be expressed as 331-1*32-11+20.

Similarly, a long-term reference picture (LTRP[1]) with a POC of 170 indexed by 1 uses an LSB value of 10, and the MSB difference between the MSB of the current picture's POC and the reference picture's POC of 5, 331-5*32- It can be expressed as 11+10.

As shown in FIGS. 3 and 4, an initial reference picture list that can be referenced by a current picture may be constructed with a list of pictures existing in the short-term reference picture buffer and the long-term reference picture buffer.

5 is a diagram illustrating an example of configuring a reference picture list.

The reference pictures are a first short-term reference picture set (RefPicSetStCurr0) composed of reference pictures (Ref 1, Ref 2) smaller than the POC of the current picture (Curr) based on the current picture, and reference pictures larger than the POC of the current picture ( It may be classified into a second short-term reference picture set (RefPicSetStCurr1) composed of Ref 3 and Ref 4) and a long-term reference picture set (RefPicSetLtCurr) composed of long-term reference pictures Ref LT1 and Ref LT2.

In this case, the first short-term reference picture set (RefPicSetStCurr0) is composed of pictures with a used_by_curr_pic_s0_flag value of 1 (delta_poc_s0 with used_by_curr_pic_s0_flag=1), and the second short-term reference picture set (RefPicSetStCurr1 with a value of 1 used_by_curr_pics is also used_pics_flag1). Becomes (delta_poc_s1 with used_by_curr_pic_s1_flag=1).

An initial reference picture list may be constructed from a set of reference picture sets having different properties.

Reference picture list 0, i.e., L0, is composed of a first short-term reference picture set (RefPicSetStCurr0), a second short-term reference picture set (RefPicSetStCurr1), and a long-term reference picture set (RefPicSetLtCurr) in this order, as shown in FIG.

On the other hand, reference picture list 1, that is, L1, is configured in the order of a second short-term reference picture set (RefPicSetStCurr1), a first short-term reference picture set (RefPicSetStCurr0), and a long-term reference picture set (RefPicSetLtCurr).

The number of reference pictures that may be included in the reference picture list may be determined based on information transmitted from the encoding device. For example, after configuring a reference picture list, the encoding apparatus determines the number of reference pictures to be used, and sets information on the number of reference pictures to be used (e.g., num_ref_idx_lX_default_active_minus1, X=0 or 1) as a sequence parameter set (SPS). ), which can be transmitted to the decoding device. The decoding apparatus may use the number of reference pictures specified as a value obtained by adding 1 to the received num_ref_idx_lX_default_active_minus1 as a default value in the current sequence.

In addition, in the case of designating the number of reference pictures for each picture or slice, the encoding apparatus uses separate information indicating the number of reference pictures (e.g., num_ref_idx_l1_active_minus1, X=0 or 1) as a picture parameter set (PPS). ) Or through a slice header. The decoding apparatus may apply a value specified as a value obtained by adding 1 to the received num_ref_idx_l1_active_minus1 as the number of reference pictures for the current picture or the current slice.

When performing inter prediction, motion compensation may be performed using a specified reference picture in the reference picture list configured as described above.

In a multi-layer structure providing spatial scalability or multi-view scalability, a reference picture of an enhancement layer may be composed of reference pictures on the same layer and an inter-layer reference picture.

In this case, signaling for the inter-layer reference picture may be performed through information identifying a layer and information identifying a reference picture. For example, the value of nuh_layer_id, which is the j-th layer identifier (i is greater than j), which exists in the same access unit as the current picture of the i-th layer and is transmitted by being included in the NAL (Network Abstraction Layer) unit header, is the current picture. If it is the same as the RefPiclayerId value for, it may be determined that the corresponding picture is used as a reference picture for the current picture. The inter-layer reference picture can be displayed as a long-term reference picture.

RefPicLayerId is a value that can be signaled by the syntax element inter_layer_pred_layer_idc included in the slice header, and refers to a layer that the current layer refers to for inter-layer prediction.

6 is a conceptual diagram illustrating an example of configuring an inter-layer reference picture list according to an example of the present invention.

6, the initial reference picture list in the multi-layered image is a short-term reference picture set (RefPicSetStCurrBefore[i], hereinafter, a first reference picture set) composed of short-term reference pictures having a smaller POC than that of the current picture. A short-term reference picture set (RefPicSetStCurr After[i], hereinafter, a second reference picture set) consisting of short-term reference pictures having a larger POC than the POC, in the order of a long-term reference picture set (RefPicSetLtCurr[i], hereinafter, a third reference picture set), and It may be composed of an inter-layer reference picture set (RefPicSetILCurr[i], hereinafter, a fourth reference picture set).

The reference picture included in the fourth reference picture set may be a picture of a layer supporting spatial scalability, a picture included in a layer supporting multiview scalability, or depth scalability or quality scalability. It may be a picture of a supported layer. In this case, spatial scalability and quality scalability may be coded with the same codec structure.

When a bitstream supports more than one scalability, a reference picture for each scalability may be integrated into one reference picture set. When one reference picture set is configured for a plurality of scalability, information on the order of scalability types arranged in the reference picture set may be transmitted from the encoding device to the decoding device.

Meanwhile, a reference picture set may be individually configured for each of a plurality of scalability. That is, in addition to the first reference picture set, the second reference picture set, and the third reference picture set, for the current picture, a fourth reference picture set composed of inter-layer reference pictures, a fifth reference picture set, and a sixth reference picture set are generated. Can be. The inter-layer reference picture set may be adaptively added to the reference picture list according to the frequency of occurrence. For example, a reference picture set with a high occurrence frequency may be allocated to a low index of the reference picture list.

The initial reference picture list of FIG. 6 is L0 in which the first reference picture set is configured in the lowest order of the list. In this case, the fourth reference picture set may be added in the last order of the initial reference picture list L0.

The initial reference picture list 0 is composed of pictures in which flag information (used_by_curr_pic_flag) indicating whether or not to be used as a reference picture among reference pictures included in each reference picture set is 1. As illustrated, the initial reference picture list 0 may be configured only with first and second pictures of four pictures included in each reference picture set.

The initial reference picture list may be modified for each picture or slice, and reference pictures may be modified according to the change. The initial reference picture list may be changed by syntax elements (eg, ref_pic_list_modification_flag_l0, list_entry_l0, ref_pic_list_modification_flag_l1, list_entry_l1) included in the slice hair.

ref_pic_list_modification_flag_l0 and ref_pic_list_modification_flag_l1 explicitly indicate whether or not the reference picture list 0 and reference picture list 1 are transmitted from the encoding device to the decoding device.If the flag value is 1, the reference picture list is explicitly specified using the transmitted reference picture information. If the flag value is 0, the reference picture list is implicitly derived as an initial reference picture set.

When the flag value is 1, list_entry_l0 indicates specific entry information constituting the reference picture list for L0, that is, the index of the reference picture, and list_entry_l1 indicates specific entry information constituting the reference picture list for L1.

Through this change, the final reference picture list 0 is the first and second reference pictures (0, 1) of the first reference picture set, the first and second reference pictures (0, 1) of the second reference picture set, and the second reference picture set. It may consist of the first reference picture (0) of the 4 reference picture set.

According to another example of the present invention, when configuring a reference picture list, the fourth reference picture set for the inter-layer reference picture may be arranged in an order other than the last of the list of FIG. 6. In a multi-layer structure, since there is a strong correlation between the enhancement layer and the base layer due to the characteristics of an image, a case of referring to an inter-layer reference picture can often occur. If an inter-layer reference picture with a high occurrence frequency is added to a position other than the last of the reference picture list, encoding performance of the reference picture list can be improved.

In consideration of the above, the inter-layer reference picture may be added to various positions of the reference picture list. 7 to 10 are conceptual diagrams illustrating an example of configuring an inter-layer reference picture list according to other examples of the present invention.

7 illustrates a configuration of a reference picture list O according to an example of the present invention. As illustrated, a fourth reference picture set, which is an inter-layer reference picture set, may be arranged in the second order of the reference picture list.

The reference picture list O is composed of a first reference picture set, a fourth reference picture set, a second reference picture set, and a third reference picture set in that order.

Reference picture list 0 is composed of pictures in which flag information (used_by_curr_pic_flag) indicating whether to be used as a reference picture is 1 among reference pictures included in each reference picture set. As illustrated, the initial reference picture list 0 may be configured only with first and second pictures of four pictures included in each reference picture set.

The initial reference picture list configured as described above may be modified for each picture or slice, and entries of reference pictures may be modified according to the change.

Through this change, the final reference picture list 0 is the first and second reference pictures (0, 1) of the first reference picture set, the first and second reference pictures (0, 1) of the fourth reference picture set, It may be composed of the first reference picture (0) of the second reference picture set.

8 is a diagram illustrating a configuration of a reference picture list O according to another example of the present invention.

As shown in <a> of FIG. 8, reference picture list O is a first reference picture set composed of short-term reference pictures having a smaller POC than the current picture's POC, and a second reference picture composed of short-term reference pictures having a larger POC than the current picture A set, a fourth reference picture set composed of an inter-layer reference picture, and a third reference picture set composed of a long-term reference picture may be configured in this order.

Alternatively, in the reference picture list O, as shown in <b> of FIG. 8, a fourth reference picture set consisting of inter-layer reference pictures is arranged first, and a first reference picture set consisting of short-term reference pictures having a smaller POC than the POC of the current picture, They may be arranged in the order of a second reference picture set composed of short-term reference pictures and a third reference picture set composed of long-term reference pictures having a POC larger than that of the current picture.

In the case of <b>, when the frequency of referencing the inter-layer reference picture is high, there is an advantage of improving the encoding and decoding efficiency of the reference picture list.

Alternatively, according to another embodiment, similar to generating a reference picture list of a single layer, a first reference picture set consisting of short-term reference pictures having a smaller POC than the POC of the current picture, a short-term reference picture having a larger POC than the POC of the current picture After configuring in the order of a second reference picture set composed of and a third reference picture set composed of a long-term reference picture, a position to add an inter-layer reference picture to a slice or sequence level, that is, a slice header or a sequence parameter header may be specified.

A reference picture list can be constructed by adding an inter-layer reference picture from the specified position. Such signaling may be included in a slice header, a sequence header, or a video parameter set, encoded, and transmitted to the decoding apparatus.

9 illustrates a configuration of reference picture list 1 according to an example of the present invention. As shown, a fourth reference picture set, which is an inter-layer reference picture set, may be arranged in the last order of the reference picture list.

Reference picture list 1 is composed of a second reference picture set, a first reference picture set, a third reference picture set, and a fourth reference picture set in this order.

Reference picture list 1 is composed of pictures in which flag information (used_by_curr_pic_flag) indicating whether or not to be used as a reference picture among reference pictures included in each reference picture set is 1. As shown, the initial reference picture list 1 may be configured only with first and second pictures of four pictures included in each reference picture set.

The initial reference picture list configured as described above may be modified for each picture or slice, and entries of reference pictures may be modified according to the change.

Through this change, the final reference picture list 1 is the first reference picture (0) of the second reference picture set, the first and second reference pictures (0, 1) of the first reference picture set, and the fourth reference picture set. It may consist of the first and second reference pictures (0, 1).

10 is a diagram illustrating a configuration of a reference picture list 1 according to another example of the present invention.

As shown in <a> of FIG. 10, reference picture list 1 is a second reference picture set composed of short-term reference pictures having a larger POC than the POC of the current picture, and a first reference picture composed of short-term reference pictures having a smaller POC than the POC of the current picture. A set, a fourth reference picture set composed of an inter-layer reference picture, and a third reference picture set composed of a long-term reference picture may be configured in this order.

Alternatively, in the reference picture list 1, as shown in <b> of FIG. 10, a second reference picture set consisting of short-term reference pictures having a larger POC than that of the current picture is allocated in the first order, and a fourth reference picture consisting of inter-layer reference pictures Sets can be assigned to the second order. Next, reference picture list 1 may be configured in the order of a first reference picture set composed of short-term reference pictures having a smaller POC than that of the current picture and a third reference picture set composed of long-term reference pictures.

Alternatively, similar to generating a single-layer reference picture list, a second reference picture set consisting of short-term reference pictures having a larger POC than the POC of the current picture, and a first reference picture consisting of short-term reference pictures having a smaller POC than the POC of the current picture. After the third reference picture set consisting of the set and the long-term reference picture is configured in order, a position to add the inter-layer reference picture to the slice or sequence level, that is, a slice header or a sequence parameter header may be specified.

A reference picture list can be constructed by adding an inter-layer reference picture from the specified position. Such signaling may be included in a slice header, a sequence header, or a video parameter set, encoded, and transmitted to the decoding apparatus.

In an image supporting multi-layers, when the scalability type is a multiview, a layer for a different view may be used as a reference picture of the current layer. When a plurality of viewpoints, that is, a plurality of layers, are used as a reference picture, a fourth reference picture set composed of an inter-layer reference picture may be divided and included in the reference picture list. The reference picture list may be configured in various ways according to the view order of the layers of the current layer and the reference picture or the degree of proximity of the views.

For example, when constructing a reference picture list according to the difference between the current view and the reference view, reference pictures may be separated and sorted, or a view lower than the current view and a view higher than the current view may be separated and sorted based on the current view. May be.

11 is a diagram illustrating a reference picture list constructed by using a multiview reference picture according to an example of the present invention.

As illustrated, when the BEstream supports 5 view layers, or when a reference relationship is established between the 5 view layers, it may be assumed that view 2 is the current layer. View 0 and view 1 having a view order lower than the current view based on the current view may constitute one sub reference picture set (first sub reference picture set) or may be included in the first sub reference picture set, and the current view View 3 and view 4 having a higher view order may constitute one sub reference picture set (a second sub reference picture set) or may be included in a second sub reference picture set.

That is, the first sub-reference layer picture set may be composed of pictures having a view order smaller than the viewpoint of the current picture among multi-view reference layer pictures, and the second sub-reference layer picture set is the current picture among multi-view reference layer pictures. It may be composed of pictures having a view order greater than the view order.

As shown in <a>, reference picture list 0 is composed of a first reference picture set composed of short-term reference pictures having a smaller POC than the current picture's POC, a first sub-reference picture set, and a short-term reference picture having a larger POC than the POC of the current picture. It may be configured in order of a second reference picture set, a second sub-reference picture set, and a third reference picture set composed of a long-term reference picture.

Reference picture list 1 is a second reference picture set consisting of short-term reference pictures having a larger POC than the current picture's POC, a second sub-reference picture set, a first reference picture set consisting of short-term reference pictures having a smaller POC than the POC of the current picture, It may be configured in the order of a first sub-reference picture set and a third reference picture set composed of a long-term reference picture.

Alternatively, reference picture list 0 is a first reference picture set composed of short-term reference pictures having a smaller POC than the current picture's POC as shown in <b>, a first sub-reference picture set, and a short-term reference picture having a larger POC than the POC of the current picture. It may be configured in the order of a configured second reference picture set, a third reference picture set including a long-term reference picture, and a second sub-reference picture set. That is, the second sub-reference picture set may be added to the end of the reference picture list.

Similarly, reference picture list 1 is a first reference picture composed of a second reference picture set composed of short-term reference pictures having a larger POC than the current picture's POC, a second sub-reference picture set, and a short-term reference picture having a smaller POC than that of the current picture. A reference picture list may be configured in the order of a set and a third reference picture set consisting of a long-term reference picture, and the first sub-reference picture set may be added in the last order.

The inter-layer reference picture set of FIG. 11 includes a multiview reference picture, but the reference picture shown in FIG. 11 is even when the type of scalability is spatial scalability and pictures of multiple layers are used as reference pictures. List configuration can be applied. That is, the reference pictures may be divided and added to the reference picture list according to the difference in the order of identifying the current picture and the spatial layer.

12 is a diagram illustrating a reference picture list constructed using a multiview reference picture according to another example of the present invention.

The smaller the difference between the current layer and the viewpoint, the higher the likelihood that the picture of the corresponding layer will be used as an inter-layer reference picture. In other words, the likelihood that the current picture refers to a layer having a larger difference between the current layer and the viewpoint decreases. According to the present embodiment, it is possible to adaptively change the order in which reference pictures are added to the reference picture list according to the proximity order taking into account the frequency of occurrence of such reference pictures, that is, the degree of reference between viewpoints.

That is, a low index of the reference picture list can be allocated to a reference picture of a layer with a small difference between the current layer and a viewpoint, and a high index of the reference picture list can be allocated to a reference picture of a layer with a large difference between the current layer and the viewpoint. have.

For example, as shown in FIG. 12, among reference picture 0, reference picture 1, reference picture 2, and reference picture 3 that may be included in the multi-view reference picture set, reference picture 0 is arranged at the front of the reference picture list 0. Reference picture 1, reference picture 2, and reference picture 3 may be arranged at the end of reference picture list 0, which is the next order of the third reference picture set.

In this case, reference picture 0 may be a reference picture of a layer having a small difference between a current layer and a viewpoint, and reference picture 1, reference picture 2, and reference picture 3 may be pictures having a large difference between the current layer and the viewpoint.

A difference value between a view point of a reference layer and a current layer that can be arranged in the front of the reference picture list may be a value determined by default, or may be variably set in the encoding device and transmitted to the decoding device.

According to another embodiment of the present invention, the sorting order of the inter-layer reference picture set may be set according to the index (tempralId) of a temporal sub-layer of the current picture.

13 is a diagram illustrating a reference picture list constructed using an index of a temporal sub-layer according to an example of the present invention.

In a multi-layer structure, the frequency of using an inter-layer reference picture may be different according to a temporal sub-layer of a current picture belonging to an enhancement layer. As tempralId, which is an index value of the temporal sublayer, is larger, it may be more effective to perform inter prediction by referring to a picture in the same layer rather than referring to a reference picture of another layer.

In other words, if the tempralId is small, it is more effective for coding to refer to a picture of the inter-layer than for the same layer, and as the tempralId is large, it is more effective to use a picture of the same layer as a reference picture than to use the inter-layer as a reference picture. It can be efficient.

In this embodiment, taking into account that the frequency of reference occurrence of an inter-layer picture varies according to the temporal sub-layer, the position of adding an inter-layer reference picture to the initial reference picture list is adaptively changed according to the index tempralId of the temporal sub-layer. I can.

13 is a diagram illustrating a configuration of a reference picture list 0 when the tempralId value is 0 to 2.

When the tempralId value is 0, since inter-layer prediction is highly likely to be performed on the enhancement layer, the fourth reference picture set composed of inter-layer reference pictures may be added to the beginning of the initial reference picture list.

The initial reference picture list consisting of pictures whose flag information (used_by_curr_pic_flag) indicating whether or not to be used as a reference picture among the reference pictures included in each reference picture set is 1 is four pictures included in the fourth reference picture set (0 ,1,2,3), the first reference picture set, the second reference picture set, and the first and second pictures (0,1) included in the third reference picture set.

As shown, after the initial value of the reference picture list is set, the reference picture list may be modified, and then a final reference picture list may be set. Information for modifying the reference picture list may be signaled in the slice header.

The modified final reference picture list 0 is the first and second pictures of the first reference picture set (0,1), the first and second pictures of the fourth reference picture set (0,1), and the second reference picture set. It is organized in order of the first picture (0).

When the tempralId value is 1, the fourth reference picture set composed of the inter-layer reference picture may be added after the first reference picture set in the initial reference picture list 0.

The initial reference picture list consisting of pictures with flag information (used_by_curr_pic_flag) indicating whether to be used as a reference picture is 1 is included in the first reference picture set, the fourth reference picture set, the second reference picture set, and the third reference picture set It consists of the first and second pictures (0,1).

The modified final reference picture list 0 is the first and second pictures of the first reference picture set (0,1), the first and second pictures of the fourth reference picture set (0,1), and the second reference picture set. It is organized in order of the first picture.

When the tempralId value is 2, the possibility of inter-prediction between the same layers is higher than that of the inter-layer prediction of the enhancement layer.Therefore, the fourth reference picture set consisting of inter-layer reference pictures is the third reference picture set in the initial reference picture list. Then it can be added last.

The initial reference picture list 0 consisting of pictures with flag information (used_by_curr_pic_flag) indicating whether to be used as a reference picture is 1 is in the first reference picture set, the second reference picture set, the third reference picture set, and the fourth reference picture set. It consists of the included first and second pictures (0,1).

The modified final reference picture list includes the first and second pictures of the first reference picture set (0,1), the first and second pictures of the second reference picture set (0,1), and the first of the fourth reference picture set. It consists of the first picture (0).

Similar to the above, the position to which the fourth reference picture set is added may be changed in the initial reference list L1 according to the temporalId value. For example, when the temporalId value is 0, the initial reference list L1 may be configured in the order of a fourth reference picture set, a second reference picture set, a first reference picture set, and a third reference picture set, and when the temporalId value is 1 May be configured in the order of a second reference picture set, a fourth reference picture set, a first reference picture set, and a third reference picture set. When the temporalId value is as large as 2, the fourth reference picture set may be added in the last order of the initial reference list L1.

In summary, as illustrated in FIG. 13, the position of adding the inter-layer reference picture to the initial reference picture list can be adaptively changed according to the temporalId value of the temporal sub-layer.

Meanwhile, according to another embodiment of the present invention, the order of configuring a reference picture list may be different according to a difference in a layer index temporalId value of a temporal sub-layer between layers in a multi-layer structure.

In a multi-layer structure, if the temporalId values of the enhancement layer and the base layer are different from each other or the difference between the temporalId values is large, the possibility that the enhancement layer refers to the base layer, that is, the possibility that an inter-layer reference relationship exists is less.

On the other hand, if the temporalId value of the enhancement layer and the temporalId value of the base layer are the same or the difference between the temporalId value is small, the possibility that the enhancement layer refers to the base layer, that is, the possibility that an inter-layer reference relationship exists, increases.

Since the frequency of occurrence of the inter-layer reference varies according to the difference in temporalId values of the enhancement layer and the base layer, the position of adding the inter-layer reference picture to the reference picture list can be adaptively changed according to the difference in temporalId values between layers. .

For example, if the temporalId value of the picture belonging to the enhancement layer and the temporalId value of the picture belonging to the base layer or reference layer are the same, the interlayer reference picture set is added to the very beginning of the reference picture list, and the picture belonging to the two layers When the temporalId value of is different, the inter-layer reference picture set may be added to the end after the long-term reference picture set.

Alternatively, when adding an inter-layer reference picture to the reference picture list, if the difference between the temporalId value of the picture belonging to the enhancement layer and the temporalId value of the picture belonging to the base layer or reference layer is less than a specific value, the first in the reference picture list In addition, when the difference between the temporalId values of pictures belonging to two layers is greater than a specific value, the inter-layer reference picture set may be added to the end after the long-term reference picture set.

Alternatively, when an inter-layer reference picture is added to the front of the reference picture list, it may be added as a second reference picture set other than the first of the reference picture list, or when an inter-layer reference picture is added to the rear of the reference picture list, the reference picture It may be added before the long-term reference picture set other than the last in the list.

For example, when the temporalId value of the current picture of the enhancement layer and the base layer picture used as the inter-layer reference picture are the same as 2, the inter-layer reference picture may be added to the beginning of the reference picture list.

On the other hand, when the temporalId value of the current picture of the enhancement layer is 4 and the temporalId value of the base layer picture used as the inter-layer reference picture is 2, the inter-layer reference pictures are added to the reference picture list in the order following the long-term reference picture set. I can.

According to another embodiment of the present invention, a reference picture list may be adaptively configured according to the number of maximum temporal sublayers allowing inter-layer prediction.

A bitstream supporting multi-layers may include information on a maximum temporal sub-layer allowing inter-layer prediction. This information may be transmitted as a syntax element of max_tid_il_ref_pics_plus1 included in the video parameter set. A value subtracting 1 from max_tid_il_ref_pics_plus1 represents the maximum number of time-axis sub-layers that allow inter-layer prediction.

In this case, when configuring the reference picture list, when the temporalId value of the current layer is greater than the value specified by subtracting 1 from max_tid_il_ref_pics_plus1, it is better to use a picture in the same layer as the upsampled base layer picture as a reference picture. It is advantageous in terms of encoding and decoding efficiency. Therefore, when the temporalId value of the current layer is greater than the value specified by max_tid_il_ref_pics_plus1-1, it may be set not to use the inter-layer reference. In this case, the inter-layer reference picture set may not be added to the reference picture list, or even if the inter-layer reference picture set is added, a separate signal for preventing reference to the inter-layer reference picture set may be signaled.

On the other hand, when the temporalId value of the current layer is less than or equal to the value specified by max_tid_il_ref_pics_plus1, a reference picture list may be configured by combining a short-term reference picture set, a long-term reference picture set, and an inter-layer reference picture set for the current layer. In this case, a reference picture set may be configured according to the embodiment described with reference to FIGS. 6 to 13 and an embodiment that can be modified in the above embodiment.

Meanwhile, among the lower layers of the current layer, a lower layer having a temporalId value greater than a value obtained by subtracting 1 from max_tid_il_ref_pics_plus1 is not referenced when predicting the current layer.

In addition, since inter prediction does not refer to a slice or picture having a termporalId value larger than the current slice or picture, a slice or picture of a base layer having a termporalId value greater than the termporalId value of the current slice or picture will not be referenced. I can.

As described above, when the inter-layer reference picture set is not used according to the temporalId of the temporal sub-layer, the reference picture list may be constructed using only the short-term reference picture set and the long-term reference picture set for the current layer.

Meanwhile, in a bitstream supporting spatial scalability, an image of an enhancement layer and a base layer, or an image of a neighboring layer adjacent to the current layer in a bitstream supporting multi-view scalability have similar characteristics. I can. In this case, similarity of the reference picture set may exist.

Hereinafter, when constructing a reference picture set or a reference picture list of an enhancement layer by using the point that reference picture sets may be similar in different layers, it is possible to use a reference picture set or a reference picture list of a base layer or a lower layer. Let's take a look.

According to an embodiment of the present invention, prediction of an enhancement layer may be performed by using reference picture set information of a base layer as a reference picture set of an enhancement layer.

That is, the encoding device and the decoding device may use a reference picture set of a base layer, another lower layer, or a layer adjacent to the current layer to be encoded or decoded (hereinafter, referred to as a reference layer) as it is in the current layer.

The encoding device and the decoding device can obtain reference picture set information of the reference layer, that is, information about the reference picture constituting the reference picture set, and flag information indicating whether to use or not, from the encoding unit and the decoding unit that encode or decode the reference layer. , This information can be used as it is when predicting the enhancement layer, that is, the current layer.

If the reference layer to which the enhancement layer currently being encoded or decoded is a reference layer is a picture of the base layer, since the base layer does not have an inter-layer reference picture, information on the short-term reference picture set and the long-term reference picture set can be used. . When the reference picture set of the base layer is used for the enhancement layer, such signaling may be performed through a sequence parameter set (SPS).

Table 1 below shows the SPS according to an embodiment of the present invention.

Referring to Table 1, when the value of inter_layer_copy_flag is 1, it indicates that the short-term reference picture set and the long-term reference picture set of the reference layer referenced by the current layer are used as the short-term reference picture set and the long-term reference picture set of the current layer. When the value of inter_layer_copy_flag is 0, the slice header of the current layer other than the reference layer is decoded to derive the short-term reference picture set and the long-term reference picture set of the current layer.

As shown in Table 1, inter_layer_copy_flag information may be included in the SPS and transmitted. Alternatively, such flag information may be transmitted by being included in the picture level or the slice level.

num_short_term_ref_pic_sets represents the number of short-term reference picture sets included in the SPS. As shown in Table 1, num_short_term_ref_pic_sets is received when inter_layer_copy_flag is 0.

When num_short_term_ref_pic_sets is signaled, the index of the short-term reference picture set (short_term_ref_pic_set( i )) is derived by repeating the for statement (i = 0; i <num_short_term_ref_pic_sets; i++)) as many as the number of short-term reference picture sets specified by num_short_term_ref_pic_sets.

If long_term_ref_pics_present_flag is 0, it indicates that there is no long-term reference picture used for prediction of the current layer picture, and if long_term_ref_pics_present_flag is 1, it indicates that at least one long-term reference picture is used for prediction of the current layer picture.

num_long_term_ref_pics_sps represents the number of long-term reference picture sets included in the SPS.

lt_ref_pic_poc_lsb_sps[i] represents MaxPocLsb of the i-th candidate long-term reference picture specified in the SPS.

If used_by_curr_pic_lt_sps_flag[i] is 0, it indicates that the i-th candidate long-term reference picture specified in the SPS is not used as a reference picture of a picture including the i-th candidate long-term reference picture in the corresponding long-term reference picture set.

In summary, according to an embodiment of the present invention, when the reference layer referenced by the current layer is a base layer, when configuring the reference picture list of the current layer, the short-term reference picture set and the long-term reference picture set of the base layer are It can be used as it is. In this case, flag information such as inter_layer_copy_flag may be included in the SPS and signaled.

According to another embodiment of the present invention, the encoding apparatus and the decoding apparatus may use some of the reference picture set of the reference layer as the reference picture set of the current layer. That is, when at least one of the short-term reference picture set or the long-term reference picture set of the reference layer is used as the reference picture set of the current layer, whether to use each reference picture set may be signaled.

Table 2 shows the SPS according to an embodiment of the present invention.

Referring to Table 2, when the value of inter_layer_short_copy_flag is 1, it indicates that the short-term reference picture set of the reference layer referenced by the current layer is used as the short-term reference picture set of the current layer. If the inter_layer_short_copy_flag value is 0, the slice header of the current layer other than the reference layer is decoded to derive the short-term reference picture set of the current layer.

As shown in Table 2, inter_layer_short_copy_flag information may be included in the SPS and transmitted. Alternatively, such flag information may be transmitted by being included in the picture level or the slice level.

num_short_term_ref_pic_sets represents the number of short-term reference picture sets included in the SPS. As shown in Table 2, num_short_term_ref_pic_sets is received when inter_layer_short_copy_flag is 0.

When num_short_term_ref_pic_sets is signaled, the index of the short-term reference picture set (short_term_ref_pic_set( i )) is derived by repeating the for statement (i = 0; i <num_short_term_ref_pic_sets; i++)) as many as the number of short-term reference picture sets specified by num_short_term_ref_pic_sets.

If long_term_ref_pics_present_flag is 0, it indicates that there is no long-term reference picture used for prediction of the current layer picture, and if long_term_ref_pics_present_flag is 1, it indicates that at least one long-term reference picture is used for prediction of the current layer picture.

If the value of inter_layer_long_copy_flag is 1, it indicates that the long-term reference picture set of the reference layer referenced by the current layer is used as the long-term reference picture set of the current layer. If the inter_layer_long_copy_flag value is 0, the slice header of the current layer other than the reference layer is decoded to derive the long-term reference picture set of the current layer.

When inter_layer_long_copy_flag is 0 and long_term_ref_pics_present_flag is 1, num_long_term_ref_pic_sps indicating the number of long-term reference pictures included in the SPS is signaled.

lt_ref_pic_poc_lsb_sps[i] represents MaxPocLsb of the i-th candidate long-term reference picture specified in the SPS.

If used_by_curr_pic_lt_sps_flag[i] is 0, it indicates that the i-th candidate long-term reference picture specified in the SPS is not used as a reference picture of a picture including the i-th candidate long-term reference picture in the corresponding long-term reference picture set.

In summary, according to an embodiment of the present invention, when the reference layer referenced by the current layer is a base layer, when configuring the reference picture list of the current layer, the short-term reference picture set and the long-term reference picture set of the base layer are It can be used as it is. In this case, in order to signal this, the SPS may include flag information such as inter_layer_short_copy_flag or inter_layer_long_copy_flag.

That is, according to the present embodiment, copy information for each of the short-term reference picture set and the long-term reference picture set may be signaled instead of copy information (copy flag) for the entire reference picture set of the current layer.

Alternatively, according to another embodiment, the current layer may use the inter-layer reference picture set of the reference layer as the inter-layer reference picture set of the current layer, and in this case, flag information indicating whether to copy the inter-layer reference picture set is signaled. Can be.

The above-described flag information may be included in a sequence parameter set and signaled as shown in Tables 1 and 2, or may be included in a video parameter set, a picture level, or a slice level and transmitted.

Meanwhile, when the current layer refers to inter-layer reference picture set information of the reference layer, position information to which the inter-layer reference picture is added to the reference picture list may be referred. When adding the reference picture set of the current layer to the reference picture list, the position information of the reference layer may be used. A method of adding an inter-layer reference picture to the reference picture list is the same as described with reference to FIGS. 6 to 13.

14 is a diagram illustrating an example of referring to an additional position of an inter-layer reference picture set of a reference layer according to an example of the present invention.

If the reference layer is not the base layer and the reference layer is also encoded or decoded by referring to another layer, the order of adding the inter-layer reference picture for the reference layer to the initial reference picture list or the specific position to add to the reference picture list is the current layer. It may be used as it is when encoding or decoding of.

As shown in FIG. 14, in the case of a reference layer, when a fourth reference picture set composed of an inter-layer reference picture is added to an initial reference picture list, it may be added after picture 0 and picture 1 constituting the first reference picture set. That is, inter-layer reference pictures may be arranged in the order of the second reference picture set in the initial reference picture list.

After the inter-layer reference picture set, picture 0, picture 1, and picture 0 and picture 1 constituting the second reference picture set having used_by_curr_pic_s0_flag of 1 are aligned.

The finally modified initial reference picture list is composed of picture 0 and picture 1 of the first reference picture set, picture 0 of the fourth reference picture set, picture 1, and picture 0 of the second reference picture set.

As illustrated, even in the initial reference picture list for the current layer, inter-layer reference pictures may be arranged in a second order after the first reference picture set.

When the position information of the reference layer is used when configuring the initial reference picture list of the current layer, such position information may be signaled as flag information indicating whether to merge the position of the inter-layer reference picture set of the reference layer. In addition, the ID of the reference layer to which the location information is merged may be additionally signaled.

After the location to which the inter-layer reference picture is added is specified, there may be a plurality of methods for signaling this. The encoding apparatus may encode position information on a position at which the inter-layer reference picture set is added to the initial reference picture list in a slice or sequence level, such as a slice header or a sequence header. In the case of a bitstream supporting a plurality of view layers such as multi-view scalability, there are many cases in which the characteristics of the image for each layer are different and the correlation with the reconstructed image of the base layer is different. In addition, the temporalId value for each multi-layer may be different, and the frequency at which inter-layer reference occurs may be different according to the temporalId value.

Since the most optimal position to add the inter-layer reference picture to each layer may be different according to these various conditions, the position to add the inter-layer reference picture to each multi-layer may be signaled.

Alternatively, in order to reduce the amount of computation in the encoding step while maintaining the diversity of images for each multilayer, the encoding apparatus limits the positions to which the reference pictures are to be added to a predetermined number, and determines the positions where the inter-layer reference pictures can be added by the number. After setting, the bit distortion rate can be calculated only for the set position. An optimal position to which an inter-layer reference picture can be added may be derived based on the calculated bit distortion rate, and the encoding apparatus may transmit this position to the decoding apparatus.

For example, it may be set that the inter-layer reference picture can be added at three positions as follows: the first of the initial reference picture list, after the short reference picture set, and then the long reference picture set. In this case, the encoding apparatus may encode and signal an identification index of a position to which an inter-layer reference picture is added among three positions.

15 is a control flow diagram illustrating a method of generating a reference picture list according to the present invention.

First, the decoding apparatus receives flag information indicating whether the current picture uses reference picture set information of a reference layer (S1510).

This flag information may be flag information indicating whether to derive the short-term reference picture set and the long-term reference picture set of the reference layer as the short-term reference picture set and the long-term reference picture set of the current picture.

Alternatively, the flag information may indicate whether only the short-term reference picture set of the reference layer is derived as the short-term reference picture set of the current picture, or indicates whether only the long-term reference picture set of the reference layer is derived as the long-term reference picture set of the current picture. It could be information.

As described above, when the flag information indicates whether to use either the short-term reference picture set or the long-term reference picture set of the reference layer, or the short-term reference picture set and the long-term reference picture set as it is in the initial reference picture list of the current picture, reference The layer may be a base layer that does not refer to an inter-layer reference picture.

If the reference layer is not the base layer, the flag information may indicate whether the inter-layer reference picture set of the reference layer is derived as the inter-layer reference picture set of the current picture. That is, the current picture may refer to a picture for another layer referred to by the reference layer.

Alternatively, as the flag information, as described in FIG. 13, whether the position of adding the inter-layer reference picture set of the reference layer to the initial reference picture list is used as the position of adding the inter-layer reference picture set of the current picture to the initial reference picture list? It can also indicate whether or not.

According to another embodiment of the present invention, the information on the position at which the inter-layer reference picture set of the reference layer is added to the initial reference picture list is not flag information, but the position index at which the inter-layer reference picture set is added to the initial reference picture list. It may be signaled as difference value information, that is, a predicted value. For example, if the inter-layer reference picture set of the reference layer is added to the initial reference picture list in the first order, and the inter-layer reference picture set of the current layer is added to the initial reference picture list in the second order, encoding The device may encode and transmit the difference value information for the first position and the second position.

The decoding apparatus may derive a position where the inter-layer reference picture set of the current layer is added to the initial reference picture list using the signaled difference value information.

Such flag information may be included in the video level, sequence level, picture level, or slice level and signaled.

Then, the decoding apparatus generates an initial reference picture list based on the flag information (S1520).

If the above-described flag information indicating whether or not the reference picture set information of the reference layer is used is 1, when configuring the initial reference picture list for the current picture, the reference picture set of the reference layer can be used as it is.

If the flag information is 0, the initial reference picture list of the current picture is independently configured without reference layer information.

The predictor may modify the initial reference picture list based on change information on the additional reference picture list.

The prediction unit predicts the current picture based on the initial reference picture list or the modified final reference picture list (S1530).

When predicting the current picture, a reference picture included in the inter-layer reference picture set is displayed as a long-term reference picture, and the decoding apparatus may use a reference picture included in the inter-layer reference picture set as a reference picture of the current picture.

In the exemplary system described above, the methods are described on the basis of a flowchart as a series of steps or blocks, but the present invention is not limited to the order of the steps, and certain steps may occur in a different order or concurrently with the steps described above. I can. In addition, since the above-described embodiments may include examples of various aspects, a combination of each embodiment should also be understood as an embodiment of the present invention. Accordingly, the present invention will be said to include all other replacements, modifications and changes falling within the scope of the following claims.

100: encoding device 200: decoding device
100a: encoder of an upper layer 100b: encoder of a lower layer
200a: decoding unit of the upper layer 200b: decoding unit of the lower layer

Claims (20)

In a video decoding method supporting a plurality of layers,
Generating an initial reference picture list of the current picture;
Comprising the step of performing prediction on the current picture based on the initial reference picture list,
The initial reference picture list includes an initial reference picture list 0 and an initial reference picture list 1,
Each of the initial reference picture list 0 and the initial reference picture list 1 includes a short-term reference picture set, a long-term reference picture set, and an inter-layer reference picture set, and the inter-layer reference picture set includes a first sub-reference layer picture set and a second Including a sub-reference layer picture set,
In the initial reference picture list 0, the first sub-reference layer picture set includes a first short-term reference picture set consisting of short-term reference pictures having a smaller POC than a picture of count (POC) of the current picture, and a POC of the current picture. POC is aligned between a second short-term reference picture set composed of a large short-term reference picture, the second sub-reference layer picture set is aligned after a long-term reference picture set,
In the initial reference picture list 1, the second sub-reference layer picture set comprises a second short-term reference picture set consisting of short-term reference pictures having a larger POC than a picture of count (POC) of the current picture, and a second short-term reference picture set consisting of a POC of the current picture. POC is arranged between a first short-term reference picture set composed of small short-term reference pictures, the first sub-reference layer picture set is arranged after a long-term reference picture set,
The first short-term reference picture set, the second short-term reference picture set, and the long-term reference picture set correspond to the same layer as a current layer including the current picture,
And the first and second sub-reference layer picture sets correspond to a layer different from the current layer. delete delete delete delete delete delete delete The method of claim 1,
And generating a final reference picture list by modifying the initial reference picture list. The method of claim 1,
The performing of prediction on the current picture comprises using a reference picture included in the inter-layer reference picture set as a reference picture of the current picture. In the video decoding apparatus supporting a plurality of layers,
An entropy decoding unit for decoding information for prediction and decoding of an image received through a bitstream;
A prediction unit that generates an initial reference picture list of the current picture and performs prediction on the current picture based on the initial reference picture list,
The initial reference picture list includes an initial reference picture list 0 and an initial reference picture list 1,
Each of the initial reference picture list 0 and the initial reference picture list 1 includes a short-term reference picture set, a long-term reference picture set, and an inter-layer reference picture set, and the inter-layer reference picture set includes a first sub-reference layer picture set and a second Including a sub-reference layer picture set,
In the initial reference picture list 0, the first sub-reference layer picture set includes a first short-term reference picture set consisting of short-term reference pictures having a smaller POC than a picture of count (POC) of the current picture, and a POC of the current picture. POC is aligned between a second short-term reference picture set composed of a large short-term reference picture, the second sub-reference layer picture set is aligned after a long-term reference picture set,
In the initial reference picture list 1, the second sub-reference layer picture set includes a second short-term reference picture set consisting of short-term reference pictures having a larger POC than a picture of count (POC) of the current picture, and POC is aligned between a first short-term reference picture set composed of small short-term reference pictures, the first sub-reference layer picture set is aligned after a long-term reference picture set,
The first short-term reference picture set, the second short-term reference picture set, and the long-term reference picture set correspond to the same layer as a current layer including the current picture,
The apparatus for decoding an image, wherein the first and second sub-reference layer picture sets correspond to a layer different from the current layer. delete delete delete delete delete delete delete The method of claim 11,
And generating a final reference picture list by modifying the initial reference picture list by the prediction unit. The method of claim 11,
And the prediction unit uses a reference picture included in the inter-layer reference picture set as a reference picture of the current picture.

Images