KR102271878B1 - Video encoding and decoding method and apparatus using the same - Google Patents

Abstract

The present invention relates to a technique for encoding and decoding an image having a structure including at least one layer (quality, spatial, and view), and is a technique for encoding and decoding an upper layer. A technique for predicting a higher layer signal using two or more reference layers. In more detail, a spatial image quality reference layer list composed of spatial and image quality layers to be referenced within the same view as the target layer and a view composed of the same spatial and image quality layers as the target layer in encoding and decoding the encoding and decoding images of the higher layer By separating the reference layer list, inter-layer prediction can be performed in consideration of the characteristics of each layer, thereby improving encoding and decoding efficiency.

Description

Video encoding/decoding method and apparatus using the same

The present invention relates to image encoding and decoding processing, and more particularly, to an inter-layer image encoding/decoding method and apparatus to which multiple reference layers are applied in hierarchical video encoding.

Recently, as broadcasting services having high definition (HD) resolution are expanding not only in Korea but also worldwide, many users are getting used to high-resolution and high-definition images, and accordingly, many organizations are spurring the development of next-generation imaging devices. In addition, as interest in UHD (Ultra High Definition) having a resolution four times higher than that of HDTV increases along with HDTV, compression technology for higher resolution and high-definition images is required.

For image compression, inter prediction technology that predicts pixel values included in the current picture from temporally previous and/or later pictures, predicting pixel values included in the current picture using pixel information in the current picture An intra prediction technique, an entropy encoding technique in which a short code is assigned to a symbol having a high frequency of occurrence and a long code is assigned to a symbol having a low frequency of occurrence, may be used.

In video compression technology, there is a technology that provides a constant network bandwidth under a limited operating environment of hardware without considering a flexible network environment. However, a new compression technique is required to compress image data applied to a network environment in which bandwidth is frequently changed, and for this purpose, a scalable video encoding/decoding method may be used.

The present invention provides a spatial image quality reference layer list composed of spatial and image quality layers to be referenced within the same time point as the target layer in encoding and decoding the encoding and decoding images of the upper layer, and the view point composed of the same spatial and image quality layers as the target layer. Provided are a method for encoding and decoding an image capable of performing inter-layer prediction in consideration of characteristics of each layer by separating a layer list, and an apparatus using the same.

Thereby, the encoding efficiency can be improved.

According to an embodiment of the present invention, a method for decoding an image supporting a plurality of layers includes: generating a reference layer list that can be referenced by an image of a target layer that is a current decoding target; generating a reference picture list including a decoded image of a view reference layer for inter prediction of the image of the target layer; The method may include predicting and decoding the image of the target layer in block units with reference to the reference picture list.

The generating of the reference layer list may include generating a spatial quality reference layer list and the view reference list that can be referenced by the same layer as the target layer in the entire bitstream.

The spatial quality reference layer list may be composed of layers having the same viewpoint as the target layer.

Meanwhile, the view reference layer list may be composed of layers having the same space and image quality as the target layer.

The generating of the reference picture list may include: constructing a first set including the decoded image of the view reference layer; constructing a second set comprising an image of the same layer as the image of the target layer; It may include combining the first set and the second set.

The first set may be represented as a long-term reference picture.

Images included in the first set may be added to any one of the first, second, and last of the reference picture list.

The step of predicting and decoding the image in units of blocks refers to the spatial quality reference layer, and the step of predicting and decoding the image in units of blocks refers to the step of the current decoding object block in the spatial image quality reference layer list used when decoding. determining a reference layer; determining a reference block corresponding to the target block in the determined spatial quality reference layer; The method may include decoding the target block by using at least one of a reconstructed sample of the reference block, a residual of the reference block, and an encoding parameter of the reference block.

The predicting and decoding of the image in block units may include performing inter prediction on the current decoding object block by using a reference picture in the reference picture list.

An apparatus for decoding an image supporting a plurality of layers according to another embodiment of the present invention includes: an entropy decoding unit for decoding information for prediction and decoding of an image received through a non-stream; generating a reference layer list that can be referenced by an image of a target layer currently being decoded, and generating a reference picture list including a decoded image of a view reference layer for inter-screen prediction of an image of the target layer, and , a prediction unit that predicts and decodes the image of the target layer in block units with reference to the reference picture list.

According to an embodiment of the present invention, a list of spatial quality reference layers composed of spatial and picture quality layers to be referenced within the same time point as the target layer and the same spatial and picture quality as the target layer in encoding and decoding the encoded and decoded image of the higher layer A method for encoding and decoding an image capable of performing inter-layer prediction in consideration of characteristics of each layer by separating a view reference layer list consisting of layers, and an apparatus using the same are provided.

Accordingly, an image encoding/decoding method and apparatus capable of improving image encoding/decoding efficiency are provided.

1 is a block diagram illustrating a configuration of an image encoding apparatus according to an embodiment.
2 is a block diagram illustrating a configuration of an image decoding apparatus according to an embodiment.
3 is a conceptual diagram schematically illustrating an embodiment of a scalable video coding structure using a plurality of layers to which the present invention can be applied.
4 is a conceptual diagram schematically illustrating a spatial image quality layer and a view layer according to an embodiment of the present invention.
5 is a control flow diagram illustrating a method of performing encoding of a higher layer in an encoding apparatus according to an embodiment of the present invention.
6 is a control flowchart illustrating a method of constructing a spatial quality reference layer list and a view reference layer list in an encoding apparatus according to an embodiment of the present invention.
7 is a control flowchart illustrating a method of performing upper layer decoding in the decoding apparatus according to an embodiment of the present invention.
8 is a control flowchart illustrating a method of constructing a spatial quality reference layer list and a view reference layer list in a decoding apparatus according to an embodiment of the present invention.

EMBODIMENT OF THE INVENTION Hereinafter, embodiment of this invention is described concretely with reference to drawings. In describing the embodiments of the present specification, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present specification, the detailed description thereof will be omitted.

When it is said that a component is “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be In addition, the content described as “including” a specific configuration in the present invention does not exclude configurations other than the corresponding configuration, and means that additional configurations may be included in the practice of the present invention or the scope of the technical spirit of the present invention.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

In addition, the components shown in the embodiment of the present invention are shown independently to represent different characteristic functions, and it does not mean that each component is composed of separate hardware or a single software component. That is, each component is listed as each component for convenience of description, and at least two components of each component are combined to form one component, or one component can be divided into a plurality of components to perform a function, and each of these components can be divided into a plurality of components. Integrated embodiments and separate embodiments of components are also included in the scope of the present invention without departing from the essence of the present invention.

In addition, some of the components are not essential components for performing essential functions in the present invention, but may be optional components for merely improving performance. The present invention can be implemented by including only essential components to implement the essence of the present invention, except for components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement Also included in the scope of the present invention.

1 is a block diagram illustrating a configuration of an image encoding apparatus according to an embodiment. A scalable video encoding/decoding method or apparatus may be implemented by an extension of a general image encoding/decoding method or apparatus that does not provide scalability, and the block diagram of FIG. 1 is scalable. An embodiment of an image encoding apparatus that can be the basis of a video encoding apparatus is shown.

Referring to FIG. 1 , the image encoding apparatus 100 includes a motion prediction unit 111 , a motion compensation unit 112 , an intra prediction unit 120 , a switch 115 , a subtractor 125 , and a transform unit 130 . , a quantizer 140 , an entropy encoder 150 , an inverse quantizer 160 , an inverse transform unit 170 , an adder 175 , a filter unit 180 , and a reference image buffer 190 .

The image encoding apparatus 100 may perform encoding on an input image in an intra mode or an inter mode and output a bit stream. Intra prediction refers to intra prediction, and inter prediction refers to inter prediction. In the intra mode, the switch 115 is switched to intra, and in the inter mode, the switch 115 is switched to inter. The image encoding apparatus 100 may generate a prediction block for an input block of an input image, and then encode a difference between the input block and the prediction block.

In the intra mode, the intra prediction unit 120 may generate a prediction block by performing spatial prediction using pixel values of an already coded block around the current block.

In the inter mode, the motion predictor 111 may obtain a motion vector by finding a region that best matches the input block in the reference image stored in the reference image buffer 190 during the motion prediction process. The motion compensator 112 may generate a prediction block by performing motion compensation using a motion vector and a reference image stored in the reference image buffer 190 .

The subtractor 125 may generate a residual block by the difference between the input block and the generated prediction block. The transform unit 130 may perform transform on the residual block to output a transform coefficient. In addition, the quantization unit 140 may quantize the input transform coefficient according to the quantization parameter to output a quantized coefficient.

The entropy encoding unit 150 entropy-encodes a symbol according to a probability distribution based on the values calculated by the quantization unit 140 or an encoding parameter value calculated during the encoding process to generate a bit stream. can be printed out. The entropy encoding method is a method of receiving symbols having various values, removing statistical redundancy, and expressing them as a decodable binary number sequence.

Here, the symbol means a syntax element to be encoded/decoded, a coding parameter, and a value of a residual signal. Encoding parameters are parameters necessary for encoding and decoding, and may include information that is encoded in an encoding device and transmitted to a decoding device, such as syntax elements, as well as information that can be inferred during encoding or decoding. information needed to do so. Coding parameters include, for example, intra/inter prediction modes, motion/motion vectors, reference image indexes, coding block patterns, residual signal presence, transform coefficients, quantized transform coefficients, quantization parameters, block size, block partition information, etc. It may include statistics. In addition, the residual signal may mean a difference between the original signal and the predicted signal, and a signal in which the difference between the original signal and the predicted signal is transformed or a signal in which the difference between the original signal and the predicted signal is transformed and quantized. may mean The residual signal may be referred to as a residual block in block units.

When entropy encoding is applied, a small number of bits are allocated to a symbol having a high probability of occurrence and a large number of bits are allocated to a symbol having a low probability of occurrence to express the symbol, so that the size of the bit string for the symbols to be encoded is increased. can be reduced. Accordingly, compression performance of image encoding may be improved through entropy encoding.

For entropy encoding, an encoding method such as exponential golomb, Context-Adaptive Variable Length Coding (CAVLC), or Context-Adaptive Binary Arithmetic Coding (CABAC) may be used. For example, the entropy encoding unit 150 may store a table for performing entropy encoding, such as a variable length coding (VLC) table, and the entropy encoding unit 150 may store the stored variable length encoding. Entropy encoding can be performed using a (VLC) table. In addition, the entropy encoder 150 derives a binarization method of a target symbol and a probability model of a target symbol/bin, and then performs entropy encoding using the derived binarization method or probability model. You may.

The quantized coefficient may be inverse quantized by the inverse quantizer 160 and inversely transformed by the inverse transform unit 170 . The inverse-quantized and inverse-transformed coefficients may be added to the prediction block through the adder 175 and a reconstructed block may be generated.

The reconstructed block passes through the filter unit 180, and the filter unit 180 applies at least one of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the reconstructed block or the reconstructed picture. can do. The reconstructed block passing through the filter unit 180 may be stored in the reference image buffer 190 .

2 is a block diagram illustrating a configuration of an image decoding apparatus according to an embodiment. As described above with reference to FIG. 1 , the scalable video encoding/decoding method or apparatus may be implemented by extension of a general image encoding/decoding method or apparatus that does not provide scalability, and the block diagram of FIG. 2 shows scalable video decoding An embodiment of an image decoding apparatus that can be the basis of the apparatus is shown.

Referring to FIG. 2 , the image decoding apparatus 200 includes an entropy decoding unit 210 , an inverse quantization unit 220 , an inverse transform unit 230 , an intra prediction unit 240 , a motion compensator 250 , and a filter unit. 260 and a reference image buffer 270 .

The image decoding apparatus 200 may receive a bitstream output from the encoding apparatus, perform decoding in an intra mode or an inter mode, and output a reconstructed image, that is, a reconstructed image. In the case of the intra mode, the switch may be switched to the intra mode, and in the inter mode, the switch may be switched to the inter mode. The image decoding apparatus 200 may generate a reconstructed block, that is, a reconstructed block by obtaining a reconstructed residual block from the received bitstream, generating a prediction block, and adding the reconstructed residual block and the prediction block.

The entropy decoding unit 210 may entropy-decode the input bitstream according to a probability distribution to generate symbols including symbols in the form of quantized coefficients. The entropy decoding method is a method of generating each symbol by receiving a binary string as an input. The entropy decoding method is similar to the entropy encoding method described above.

The quantized coefficient is inverse quantized by the inverse quantization unit 220 and inverse transformed by the inverse transformation unit 230 , and as a result of inverse quantization/inverse transformation of the quantized coefficient, a reconstructed residual block may be generated.

In the intra mode, the intra prediction unit 240 may generate a prediction block by performing spatial prediction using pixel values of an already encoded block around the current block. In the inter mode, the motion compensator 250 may generate a prediction block by performing motion compensation using a motion vector and a reference image stored in the reference image buffer 270 .

The reconstructed residual block and the prediction block are added through the adder 255 , and the added block passes through the filter unit 260 . The filter unit 260 may apply at least one of a deblocking filter, SAO, and ALF to a reconstructed block or a reconstructed picture. The filter unit 260 outputs a reconstructed image, that is, a reconstructed image. The restored image may be stored in the reference image buffer 270 and used for inter prediction.

The entropy decoding unit 210 , the inverse quantization unit 220 , the inverse transform unit 230 , the intra prediction unit 240 , the motion compensator 250 , and the filter unit 260 included in the image decoding apparatus 200 . and components directly related to image decoding among the reference image buffer 270 , for example, the entropy decoding unit 210 , the inverse quantization unit 220 , the inverse transform unit 230 , the intra prediction unit 240 , and motion compensation. The unit 250 and the filter unit 260 may be distinguished from other components and expressed as a decoder or a decoder.

Also, the image decoding apparatus 200 may further include a parsing unit (not shown) that parses information related to the encoded image included in the bitstream. The parsing unit may include the entropy decoding unit 210 or may be included in the entropy decoding unit 210 . This parsing unit may also be implemented as one component of the decoding unit.

3 is a conceptual diagram schematically illustrating an embodiment of a scalable video coding structure using a plurality of layers to which the present invention can be applied. In FIG. 3 , a group of pictures (GOP) indicates a picture group, that is, a group of pictures.

In order to transmit image data, a transmission medium is required, and its performance is different for each transmission medium according to various network environments. A scalable video coding method may be provided for application to such various transmission media or network environments.

The scalable video coding method is a coding method that improves encoding/decoding performance by removing inter-layer redundancy by utilizing inter-layer texture information, motion information, residual signals, and the like. The scalable video coding method may provide various scalability in terms of spatial, temporal, and picture quality according to ambient conditions such as a transmission bit rate, a transmission error rate, and a system resource.

Scalable video coding may be performed using a structure of multiple layers so as to provide a bitstream applicable to various network situations. For example, the scalable video coding structure may include a base layer that compresses and processes image data using a general image encoding method, and compresses the image data using both encoding information of the base layer and a general image encoding method. It may include an enhancement layer to process.

Here, a layer is an image and bit classified based on spatial (eg, image size), temporal (eg, encoding order, image output order, frame rate), image quality, complexity, etc. It means a set of streams (bitstream). In addition, the base layer may mean a lower layer, a reference layer or a base layer, and the enhancement layer may mean an upper layer, and an enhancement layer. Also, a plurality of layers may have dependencies on each other.

Referring to FIG. 3 , for example, the base layer may be defined as SD (standard definition), a frame rate of 15 Hz, and a bit rate of 1 Mbps, and the first enhancement layer is HD (high definition), a frame rate of 30 Hz, and a bit rate of 3.9 Mbps. may be defined, and the second enhancement layer may be defined as 4K-UHE (ultra high definition), a frame rate of 60 Hz, and a bit rate of 27.2 Mbps. The format, frame rate, bit rate, etc. are an example and may be differently determined as needed. Also, the number of layers to be used is not limited to this embodiment and may be determined differently depending on circumstances.

For example, if the transmission bandwidth is 4 Mbps, the frame rate of the first enhancement layer HD may be reduced to transmit at 15 Hz or less. The scalable video coding method may provide temporal, spatial, and picture quality scalability by the method described above in the embodiment of FIG. 3 .

In the case of encoding and decoding of video supporting a plurality of layers in a bitstream, that is, scalable coding, there is a strong correlation between a plurality of layers. Duplicate elements can be removed and the encoding performance of an image can be improved. Prediction of a current layer, which is a prediction target, by using information from another layer is hereinafter referred to as inter-layer prediction. Scalable video coding has the same meaning as scalable video coding in terms of encoding and scalable video decoding in terms of decoding.

At least one of a resolution, a frame rate, and a color format of the plurality of layers may be different from each other, and layer upsampling or downsampling may be performed to adjust resolution during inter-layer prediction.

4 is a conceptual diagram schematically illustrating a spatial image quality layer and a view layer according to an embodiment of the present invention.

As shown, the bitstream may include a plurality of layers.

The bitstream may include a plurality of view layers (view 1, view 2, and 3) for different views although spatial and quality are the same.

In addition, the bitstream may be composed of layers having the same view but different spatial and image quality. The spatial image quality layer may be classified into an SD class layer and an HD class layer, and the SD class layer and the HD class layer may be composed of a base layer and an enhancement layer again.

As shown, in order to identify a layer in which space, picture quality, and viewpoint are mixed, each layer may be distinguished from each other by an identifier (layer_id). Information on which layer (eg, view layer, spatial and picture quality layer) each identifier is, and whether it is an upper layer or a lower layer within the layer is a video parameter set (VPS) or a sequence parameter set (SPS), a NAL unit header (nal). unit header) may be included and signaled.

As described above, when inter-layer prediction is performed using inter-layer correlation, the higher layer is predicted with reference to at least one lower layer. Hereinafter, for convenience of description, a layer on which prediction is performed is called a target layer, and a layer used or referenced for prediction of the target layer is expressed as a reference layer.

The present invention is an invention for efficient reference layer list generation and management in consideration of encoding efficiency of spatial, picture quality, and view scalability in encoding using one or more reference layers when encoding blocks within the same slice.

To this end, the spatial quality reference layer list to be referenced within the same view as the target layer and the view reference layer list having the same spatial and quality layer as the target layer are separately generated, and encoding and decoding methods suitable for the characteristics of each layer are applied. It aims to improve encoding efficiency.

In general, inter prediction may use at least one of a picture before or after a picture of the current picture as a reference picture, and prediction of the current block may be performed based on the reference picture. An image used for prediction of the current block is referred to as a reference picture or a reference frame.

The region in the reference picture may be indicated using a reference picture index refIdx indicating the reference picture, a motion vector, and the like.

Inter prediction may generate a prediction block for the current block by selecting a reference picture and a reference block corresponding to the current block in the reference picture.

In inter prediction, the encoding apparatus and the decoding apparatus may derive motion information of the current block and then perform inter prediction and/or motion compensation based on the derived motion information. In this case, the encoding apparatus and the decoding apparatus move a collocated block corresponding to the current block in a reconstructed neighboring block and/or an already reconstructed collocated picture. By using the information, encoding/decoding efficiency can be improved.

Here, the reconstructed neighboring block is a block in the reconstructed current picture that has already been encoded and/or decoded, and may include a block adjacent to the current block and/or a block located at an outer corner of the current block. In addition, the encoding apparatus and the decoding apparatus may determine a predetermined relative position based on a block existing at a position spatially corresponding to the current block in the collocated picture, and the determined relative position (corresponding spatially to the current block). It is possible to derive the collocated block based on the position inside and/or outside the block existing at the position where it is located. Here, as an example, the collocated picture may correspond to one of the reference pictures included in the reference picture list.

In the inter prediction, a prediction block may be generated such that a residual signal with a current block is minimized and a motion vector size is also minimized.

Meanwhile, the method of deriving motion information may vary depending on the prediction mode of the current block. Prediction modes applied for inter prediction may include Advanced Motion Vector Predictor (AMVP), merge, and the like.

For example, when Advanced Motion Vector Predictor (AMVP) is applied, the encoding apparatus and the decoding apparatus may generate a predictive motion vector candidate list by using a motion vector of a reconstructed neighboring block and/or a motion vector of a collocated block. . That is, the motion vector of the reconstructed neighboring block and/or the motion vector of the collocated block may be used as a prediction motion vector candidate. The encoding apparatus may transmit a prediction motion vector index indicating an optimal prediction motion vector selected from among the prediction motion vector candidates included in the list to the decoding apparatus. In this case, the decoding apparatus may select the predicted motion vector of the current block from among the predicted motion vector candidates included in the predicted motion vector candidate list by using the predicted motion vector index.

The encoding apparatus may obtain a motion vector difference (MVD) between the motion vector of the current block and the predicted motion vector, encode it, and transmit it to the decoding apparatus. In this case, the decoding apparatus may decode the received motion vector difference, and may derive the motion vector of the current block through the sum of the decoded motion vector difference and the predicted motion vector.

The encoding apparatus may also transmit a reference picture index indicating a reference picture to the decoding apparatus.

The decoding apparatus may predict the motion vector of the current block by using motion information of the neighboring block, and may derive the motion vector of the current block by using a difference value with respect to the motion vector received from the encoding apparatus. The decoding apparatus may generate a prediction block for the current block based on the derived motion vector and reference picture index information received from the encoding apparatus.

As another example, when merge is applied, the encoding apparatus and the decoding apparatus may generate a merge candidate list by using motion information of a reconstructed neighboring block and/or motion information of a collocated block. That is, when motion information of the reconstructed neighboring block and/or collocated block exists, the encoding apparatus and the decoding apparatus may use it as a merge candidate for the current block.

The encoding apparatus may select a merge candidate capable of providing optimal encoding efficiency from among merge candidates included in the merge candidate list as motion information for the current block. In this case, the merge index indicating the selected merge candidate may be included in the bitstream and transmitted to the decoding apparatus. The decoding apparatus may select one of the merge candidates included in the merge candidate list by using the transmitted merge index, and may determine the selected merge candidate as motion information of the current block. Accordingly, when the merge mode is applied, the motion information of the reconstructed neighboring block and/or the collocated block may be used as the motion information of the current block as it is. The decoding apparatus may reconstruct the current block by adding the prediction block and the residual transmitted from the encoding apparatus.

In the aforementioned AMVP and merge modes, in order to derive the motion information of the current block, motion information of a reconstructed neighboring block and/or motion information of a collocated block may be used.

In the case of the skip mode, which is one of the other modes used for inter prediction, information on neighboring blocks may be used for the current block as it is. Accordingly, in the skip mode, the encoding apparatus does not transmit syntax information such as residuals to the decoding apparatus other than information indicating which block to use as the motion information of the current block.

The encoding apparatus and the decoding apparatus may generate a prediction block of the current block by performing motion compensation on the current block based on the derived motion information. Here, the prediction block may mean a motion-compensated block generated as a result of performing motion compensation on the current block. Also, a plurality of motion-compensated blocks may constitute one motion-compensated image.

The decoding apparatus may check a skip flag, a merge flag, etc. received from the encoding apparatus for motion information necessary for inter prediction of the current block, for example, a motion vector, a reference picture index, and the like, and derive it correspondingly.

A processing unit in which prediction is performed and a processing unit in which a prediction method and specific content are determined may be different from each other. For example, a prediction mode may be determined in units of prediction blocks and prediction may be performed in units of transform blocks, or prediction modes may be determined in units of prediction blocks and intra prediction may be performed in units of transform blocks.

5 is a control flow diagram illustrating a method of performing encoding of a higher layer in an encoding apparatus according to an embodiment of the present invention.

Hereinafter, with reference to FIG. 5 , in a video encoding method that supports at least one scalability (eg, spatial, picture quality, and view scalability) and uses a multi-layered structure, a method of performing higher layer encoding, more specifically Let's look at how to construct a reference layer list that the target layer can refer to.

First, the encoding apparatus constructs a list of layers that an image of the current encoding target layer can refer to ( S510 ).

The encoding apparatus constructs a spatial quality reference layer list including at least one spatial or picture quality layer that the current encoding target layer can refer to within the same view when encoding, among lower layers of the current encoding target layer, and A view reference layer list including view layers that a target layer can refer to may be configured from among the layers having image quality. The reference layer list may be constructed according to at least one of the methods described below.

6 is a control flowchart illustrating a method of constructing a spatial image quality reference layer list and a view reference layer list in an encoding apparatus according to an embodiment of the present invention.

According to the first embodiment shown in FIG. 6 , the encoding apparatus may first construct a spatial image quality reference layer list that can be referred to within the same view by the same layers as the current encoding target layer in the entire bitstream (S610). ).

The encoding apparatus may configure the spatial and picture quality reference layers having the same view as the target layer in an arbitrary order to generate a referenceable spatial picture quality reference layer list having the same view as the current encoding target layer.

Alternatively, the list of referenceable spatial quality reference layers having the same view as the current encoding target layer is a layer having a small difference from the layer_id value of the target layer (that is, a layer close to the layer_id value among the spatial and picture quality reference layers having the same view as the target layer). ) can be configured in a hierarchical order with a large difference.

Alternatively, the referenceable spatial image quality reference layer list having the same view as the current encoding target layer may be configured in the order of a layer having a higher priority to a lower layer among spatial and image quality reference layers having the same view as the target layer. .

Priority related information may be signaled by being included in a NAL unit header (NALU header) or a video parameter set.

Alternatively, the referenceable spatial quality reference layer list having the same view as the current encoding target layer is a layer having a small spatial resolution difference from the current encoding target layer among spatial and quality reference layers having the same view as the current encoding target layer. It can be configured in the order of the largest hierarchy from In this case, the order of the picture quality reference layers in the same spatial resolution may be from a layer having a small difference from a layer_id value of the current encoding target layer (ie, a layer close to it) to a larger layer order.

For example, in the bitstream structure shown in FIG. 4 , the reference layer list of the layer having the layer_id of n may be configured in the order of the layers having the layer_id of n-1, n-2, and n-3.

Alternatively, the referenceable spatial quality reference layer list having the same view as the current encoding target layer is a layer having a small spatial resolution difference from the current encoding target layer among spatial and quality reference layers having the same view as the current encoding target layer. It can be configured in the order of the largest hierarchy from In this case, the order of the picture quality reference layers in the same spatial resolution may be an order of a quantization parameter value to be encoded from a low value to a high value (ie, from a layer having a good quality during decoding to a low layer order).

When the spatial quality reference layer list that can be referenced by the same layers as the target layer is generated, the encoding apparatus applies one of the following methods to the referenceable view reference layer list composed of the same spatial and same picture quality layers as the current encoding target layer. and can be configured (S620).

The encoding apparatus may generate a view reference layer list in which view reference layers formed in the same space and the same image quality layers as the current encoding target layer are arranged in an arbitrary order.

Alternatively, the encoding apparatus may generate a view reference layer list in which view reference layers composed of the same image quality layers and the same space as the current encoding target layer are configured in the order of a view closer to the current encoding target view to a distant view.

The spatial quality reference layer lists and the view reference layer lists configured as described above may be used to encode images belonging to the same layer as the layer to which the current encoding target image belongs.

The spatial quality reference layer list and the view reference layer list in which the same layers as the current encoding target layer (that is, layers having the same layer_id value) can be referenced may be integrated for efficient signaling and described as a single referenceable reference layer list. have.

Tables 1 and 2 show examples in which the reference layer list and the view reference layer list are integrated and signaled.

Referring to Table 1, num_direct_ref_layers[i] indicates the number of reference layers directly referred to by the i-th layer (ie, a layer having a layer_id of nuh_layer_id[i]).

ref_layer_id[i][j] represents the layer_id of the j-th reference layer referenced by the i-th layer.

As shown in Table 1, the spatial quality reference layer list and the view reference layer list may be signaled by description of reference layers (ref_layer_id) of a layer having a value of layer_id_in_nuh[i] in the video parameter set.

Referring to Table 2, direct_dependency_flag[i][j] is the i-th layer (that is, the layer having the layer_id of nuh_layer_id[i]) the j-th reference layer (that is, the layer having the layer_id of nuh_layer_id[j]) Indicates whether the reference layer is referenced, and when direct_dependency_flag[i][j] has a value of “1”, it means that the i-th layer directly refers to the j-th reference layer.

The order of the integrated reference layer list may be signaled in an arbitrary order, or may be signaled from a large value to a small value of the layer_id value, and the view reference layer list may be described after the spatial quality reference layer list, and the view reference layer Following the list, a spatial quality reference layer list may be described.

According to the second embodiment of configuring the list of layers that the image of the current encoding target layer can refer to, the encoding apparatus lists the spatial quality reference layer that the current encoding target layer (or the corresponding slice) of the image to be currently encoded can refer to. and a view reference layer list can be configured.

First, the encoding apparatus may configure a spatial quality reference layer list that can be referenced by a current encoding target layer of an image to be currently encoded using one of the following methods.

The encoding apparatus may configure spatial and picture quality reference layers having the same view as the target layer in an arbitrary order to generate a list of spatial picture quality reference layers that the current encoding target layer can refer to.

Alternatively, the encoding device configures the spatial quality reference layer list in the order of the largest layer from the layer having a smaller layer_id value than the layer_id value of the target layer (ie, the closest layer) among the spatial and picture quality reference layers having the same view as the target layer. can create

Alternatively, the spatial quality reference layer list may be configured in order from a layer having a high priority to a layer having a low priority among spatial and picture quality reference layers having the same view as the target layer.

In this case, the priority-related information may be signaled by being included in a NAL unit header (NALU header) or a video parameter set.

Alternatively, the referenceable spatial quality reference layer list having the same view as the current encoding target layer is a layer having a small spatial resolution difference from the current encoding target layer among spatial and quality reference layers having the same view as the current encoding target layer. It can be configured in the order of the largest hierarchy from In this case, the order of the picture quality reference layers having the same spatial resolution may be from a layer having a small difference from a layer_id value of the current encoding target layer (ie, a layer close to it) to a larger layer order.

For example, in the bitstream structure shown in FIG. 4 , the reference layer list of the layer having the layer_id of n may be configured in the order of the layers having the layer_id of n-1, n-2, and n-3.

Alternatively, the referenceable spatial quality reference layer list having the same view as the current encoding target layer is a layer having a small spatial resolution difference from the current encoding target layer among spatial and quality reference layers having the same view as the current encoding target layer. It can be configured in the order of the largest hierarchy from In this case, the order of the picture quality reference layers in the same spatial resolution may be an order of a quantization parameter value to be encoded from a low value to a high value (ie, from a layer having a good quality during decoding to a low layer order).

When the spatial image quality reference layer list that the target layer can refer to is generated, the encoding apparatus constructs a referenceable view reference layer list comprising the same space and the same picture quality layers as the current encoding target layer by applying one of the following methods. can

The encoding apparatus may generate a view reference layer list in which view reference layers formed in the same space and the same picture quality layers as the current encoding target layer are configured in an arbitrary order.

Alternatively, the encoding apparatus may generate a view reference layer list in which view reference layers composed of the same image quality layers and the same space as the current encoding target layer are configured in the order of a view closer to the current encoding target view to a distant view.

The spatial quality reference layer lists and view reference layer lists configured as described above may be used to encode an encoding target layer or a corresponding slice of the current encoding target image.

The spatial quality reference layer list and the view reference layer list in which the same layers as the current encoding target layer (that is, layers having the same layer_id value) can be referenced may be integrated for efficient signaling and described as a single referenceable reference layer list. have.

Tables 3 to 12 show examples in which the spatial quality reference layer list and the view reference layer list are integrated and signaled.

For example, the encoder may include one of the syntax elements from Tables 3 to 12 in the slice header, and through this, may signal a description of reference layers.

In this case, the layers that the described corresponding layer can refer to during encoding are a subset of reference layers that can be referenced by the same layers as the current encoding target layer in the entire bitstream, that is, the reference layer signaled in the slice header. may be composed of some of reference layers that can be referenced by the same layer as the current encoding target layer in the entire bitstream.

For example, the reference layers signaled in the slice header may be a subset of the reference layer list that can be referenced by the same layers as the current encoding target layer signaled in the video parameter set.

Referring to Table 3, slice_num_direct_ref_layers indicates the number of reference layers directly referenced by a corresponding image. slice_num_direct_ref_layers must be less than or equal to the number of reference layers (ie, NumDirectRefLayers[LayerIdInVps[nuh_layer_id]) referenced by layers having the same layer_id (ie, nuh_layer_id) as the corresponding image signaled in the video parameter set).

ref_layer_id[j] indicates the layer_id of the j-th reference layer directly referenced by the corresponding image.

Referring to Table 4, slice_num_direct_ref_layers indicates the number of reference layers directly referenced by a corresponding image. In this case, slice_num_direct_ref_layers must be equal to or smaller than the number of reference layers (ie, NumDirectRefLayers[LayerIdInVps[nuh_layer_id]) referenced by layers having the same layer_id (ie, nuh_layer_id) as the corresponding image signaled in the video parameter set.

ref_layer_id_delta[j] represents the difference between the layer_id of the j-th reference layer directly referenced by the corresponding image and the layer_id of the j-1th reference layer. In this case, as the index of the layer is closer to “0”, the current image may have a layer_id closer to the corresponding layer. ref_layer_id_delta[0] may indicate a difference between the layer_id of the 0th reference layer and the layer_id of the layer corresponding to the current image.

Referring to Table 5, slice_num_direct_ref_layers indicates the number of reference layers directly referenced by a corresponding image. In this case, slice_num_direct_ref_layers must be equal to or smaller than the number of reference layers (ie, NumDirectRefLayers[LayerIdInVps[nuh_layer_id]) referenced by layers having the same layer_id (ie, nuh_layer_id) as the corresponding image signaled in the video parameter set.

ref_layer_idx_delta[j] represents the difference between the index of the j-th reference layer directly referenced by the image (based on the index described in vps) and the index of the j-1th reference layer (based on the index described in vps), ref_layer_idx_delta[ 0] may indicate the index of the 0th reference layer.

Referring to Table 6, slice_num_direct_ref_layers indicates the number of reference layers directly referenced by a corresponding image. In this case, slice_num_direct_ref_layers must be equal to or smaller than the number of reference layers (ie, NumDirectRefLayers[LayerIdInVps[nuh_layer_id]) referenced by layers having the same layer_id (ie, nuh_layer_id) as the corresponding image signaled in the video parameter set.

ref_layer_idx[j] may indicate the index (based on the index described in vps) of the j-th reference layer directly referenced by the corresponding image.

Referring to Table 7, slice_num_direct_ref_layers indicates the number of reference layers directly referenced by a corresponding image. At this time, slice_num_direct_ref_layers must be equal to or smaller than the number of reference layers referenced by layers having nuh_layer_id (that is, NumDirectRefLayers[LayerIdInVps[nuh_layer_id]) with the same layer_id as the corresponding image signaled in the video parameter set. ”, the reference layer corresponding to the corresponding image signaled in the video parameter set may be used as the reference layer of the current image.

ref_layer_id_delta[j] represents the difference between the layer_id of the j-th reference layer directly referenced by the corresponding image and the layer_id of the j-1th reference layer. In this case, as the layer index is closer to “0”, the current image may have a layer_id closer to the corresponding layer. ref_layer_id_delta[0] may indicate a difference between the layer_id of the 0th reference layer and the layer_id of the layer corresponding to the current image.

Referring to Table 8, slice_num_direct_ref_layers indicates the number of reference layers directly referenced by a corresponding image. In this case, slice_num_direct_ref_layers must be equal to or smaller than the number of reference layers (ie, NumDirectRefLayers[LayerIdInVps[nuh_layer_id]) referenced by layers having the same layer_id (ie, nuh_layer_id) as the corresponding image signaled in the video parameter set. When slice_num_direct_ref_layers is “0”, the reference layer corresponding to the image signaled in the video parameter set can be used as the reference layer of the current image.

ref_layer_idx_delta[j] represents the difference between the index of the j-th reference layer directly referenced by the corresponding image (based on the index described in vps) and the index of the j-1th reference layer (based on the index described in vps). ref_layer_idx_delta[0] may indicate the index of the 0th reference layer.

Referring to Table 9, slice_num_direct_ref_layers indicates the number of reference layers directly referenced by a corresponding image. In this case, slice_num_direct_ref_layers must be equal to or smaller than the number of reference layers (ie, NumDirectRefLayers[LayerIdInVps[nuh_layer_id]) referenced by layers having the same layer_id (ie, nuh_layer_id) as the corresponding image signaled in the video parameter set. When slice_num_direct_ref_layers is “0”, the reference layer corresponding to the image signaled in the video parameter set can be used as the reference layer of the current image.

ref_layer_idx may indicate the index (based on the index described in vps) of the j-th reference layer directly referenced by the corresponding image.

Referring to Table 10, layer_dependency_sps_flag indicates whether reference layer information is signaled in a slice header (slice segment header). Signaled when layer_dependency_sps_flag is “0”.

slice_num_direct_ref_layers indicates the number of reference layers directly referenced by the corresponding image. In this case, slice_num_direct_ref_layers must be equal to or smaller than the number of reference layers (ie, NumDirectRefLayers[LayerIdInVps[nuh_layer_id]) referenced by layers having the same layer_id (ie, nuh_layer_id) as the corresponding image signaled in the video parameter set.

ref_layer_id_delta[j] represents the difference between the layer_id of the j-th reference layer directly referenced by the corresponding image and the layer_id of the j-1th reference layer. In this case, as the index of the layer is closer to “0”, the current image may have a layer_id closer to the corresponding layer. ref_layer_id_delta[0] may indicate a difference between ref_layer_id[0] and layer_id of the current image.

Referring to Table 11, layer_dependency_sps_flag indicates whether reference layer information is signaled in a slice header (slice segment header). Signaled when layer_dependency_sps_flag is “0”.

slice_num_direct_ref_layers indicates the number of reference layers directly referenced by the corresponding image. In this case, slice_num_direct_ref_layers must be equal to or smaller than the number of reference layers (ie, NumDirectRefLayers[LayerIdInVps[nuh_layer_id]) referenced by layers having the same layer_id (ie, nuh_layer_id) as the corresponding image signaled in the video parameter set.

ref_layer_idx_delta[j] represents the difference between the index of the j-th reference layer directly referenced by the corresponding image (based on the index described in vps) and the index of the j-1th reference layer (based on the index described in vps). ref_layer_idx_delta[0] may indicate the index of the 0th reference layer.

Referring to Table 12, layer_dependency_sps_flag indicates whether reference layer information is signaled in a slice header (slice segment header). Signaled when layer_dependency_sps_flag is “0”.

slice_num_direct_ref_layers indicates the number of reference layers directly referenced by the corresponding image. In this case, slice_num_direct_ref_layers must be equal to or smaller than the number of reference layers (ie, NumDirectRefLayers[LayerIdInVps[nuh_layer_id]) referenced by layers having the same layer_id (ie, nuh_layer_id) as the corresponding image signaled in the video parameter set.

ref_layer_idx[j] indicates the index (based on the index described in vps) of the j-th reference layer directly referenced by the corresponding image.

The order of the integrated reference layer list may be signaled in an arbitrary order, or may be signaled from a large value to a small value of the layer_id value, and the view reference layer list may be described after the spatial quality reference layer list, and the view reference layer Following the list, a spatial quality reference layer list may be described.

Returning to FIG. 5 , the encoding apparatus configuring a list of layers that can be referenced by the image of the current encoding target layer performs inter prediction of the current encoding target image including the decoded image of the view reference layer that the target layer can refer to. A reference picture list is generated for ( S520 ).

The encoding apparatus may configure a reference picture set for inter prediction of the current encoding target image including the decoded image of the view reference layer, and perform reference picture marking.

At this time, the encoding device checks whether the image included in the view reference layer list is available as a reconstructed image, and if available, includes the reconstructed image in the reference picture set, and if not available, the reconstructed image is “ No reference picture”.

A reference picture set (first set) composed of images included in the view reference layer list is marked as “used for long term reference” and is used as a long-term reference picture for inter prediction of the current encoding target image. can be treated.

In addition to the first set, that is, a reference picture set composed of images included in the view reference layer list, a reference picture set for inter prediction composed of an image of the same layer as the current encoding target layer may exist in various forms.

The reference picture set for inter prediction, which consists of images of the same layer as the encoding target layer, is used for inter prediction of the current encoding target image, and is a short-term reference picture that is previous to the current encoding target image in the display order. reference picture) (second set), a short-term reference picture that is used for inter prediction of the current encoding target image, and is later based on the current encoding target image in display order (third set), inter prediction of the current encoding target image A long-term reference picture (fourth set) for , a short-term reference picture (fifth set) for an image that can be encoded after the current encoding target image, and a long-term reference picture for an image that can be encoded after the current encoding target image ( 6th set).

Also, the encoding apparatus may generate the reference picture list of the current encoding target image according to the characteristics of the reference picture set and the reference picture type based on the various reference picture sets as described above.

As an example, the encoding apparatus adds a reference picture set including a view reference layer list included in the first set to inter-picture reference picture lists L0 and L1 including reference picture sets including images of the same layer as the current encoding target image. It is possible to create a final reference picture list by adding

In this case, the encoding apparatus may add the decoded image of the view reference layer to a fixed position each time the reference picture list is generated, or may additionally change the position of the decoded image of the view reference layer after generating the reference picture list for efficient encoding. have.

When the decoded image of the view reference layer is added at a fixed position each time the reference picture list is generated, the first set is added from the last or first (ref_idx=0) or second (ref_idx=1) position when the L0 list is generated. can

When the view reference layer is added to the middle position of the L0 list, the index in the list of images after the corresponding position may be increased by the number of the added view reference layers (the number of reference picture sets composed of the reference layer list). .

Alternatively, the encoding apparatus may replace as many reference images as the number of reference picture sets including the view reference layer list from the first (ref_idx=0) or second (ref_idx=1) position as the first set when generating the L0 list.

The encoding apparatus may add the first set from an arbitrary signaled position when generating the L0 list. When the first set is added to the middle position of the list, the index in the list of images at the position and thereafter is increased by the number of added view reference layers (the number of reference picture sets composed of the view reference layer list). can

Alternatively, the encoding apparatus may replace as many reference pictures as the number of reference picture sets composed of the view reference layer list from an arbitrary signaled position as the first set when generating the L0 list.

Alternatively, the encoding apparatus may add each picture included in the view reference layer list of the first set at different positions when generating the L0 list. The corresponding position of the added images and the index in the list of subsequent images may be increased by the number of added view reference layers (the number of reference picture sets composed of the view reference layer list).

Alternatively, when generating the L0 list, the encoding apparatus may replace reference images located at different positions with respective pictures included in the first set of view reference layer lists.

Alternatively, the encoding apparatus may add the first set to the last or first (ref_idx=0) or second (ref_idx=1) position when generating the L1 list.

When the first set is added to the middle position of the L1 list, the index in the list of images at the position and thereafter may be increased by the number of the added reference layers (the number of reference picture sets composed of the view reference layer list). .

Alternatively, the encoding apparatus may replace as many reference images as the number of reference picture sets including the view reference layer list from the first (ref_idx=0) or second (ref_idx=1) position as the first set when generating the L1 list.

The encoding apparatus may add the first set from an arbitrary signaled position when generating the L1 list. When the first set is added to the middle position of the list, the index in the list of images at the position and thereafter may be increased by the number of added view reference layers (the number of reference picture sets composed of the view reference layer list). have.

Alternatively, the encoding apparatus may replace as many reference images as the number of reference picture sets composed of the view reference layer list from an arbitrary signaled position as the first set when generating the L1 list.

Alternatively, when generating the L1 list, the encoding apparatus may add each picture included in the view reference layer list of the first set to arbitrary different positions. Corresponding positions of the added images and indexes in the list of subsequent images may be increased by the number of added view reference layers (the number of reference picture sets composed of the view reference layer list).

Alternatively, when generating the L1 list, the encoding apparatus may replace reference images in arbitrary different positions with each picture included in the view reference layer list of the first set.

On the other hand, when the position of the decoded image of the view reference layer is changed for additional efficient encoding after the reference picture list is generated, the decoded image of the view reference layer is changed using encoding parameters that may be included in the slice header or the picture parameter set. The position can be changed to any position in the reference picture list.

When the reference layer list is generated, the encoding apparatus may encode the image of the current layer in units of blocks ( S530 ).

The encoding apparatus may encode the target block by using inter prediction including the image of the spatial quality reference layer or the view reference layer to which the encoding target block of the current layer can be referenced.

For example, the encoding apparatus may perform encoding by using at least one of a plurality of pieces of information on a reference block of a referenceable spatial image quality reference layer. In this case, the reference block of the reference layer is a block of the reference layer corresponding to the current encoding object block of the current layer, and may mean, for example, a block at the same position as the current encoding object block.

The encoding apparatus may select one reference layer from a list of spatial quality reference layers that can be referenced by the encoding target block of the current layer, and the encoding target block of the current layer is a reference block reconstruction sample and reference among information of the reference block of the reference layer. Encoding of the target block may be performed using any one or at least one of residual of the block and encoding parameters of the reference block, for example, a reference frame, a motion vector prediction mode, and block partitioning information.

When the encoding object block is encoded using information on the reference layer included in the reference layer list, the encoding apparatus may encode an index indicating the reference layer used.

For example, in the encoding apparatus, layer_id of a layer included in the spatial quality reference layer list referenced by a layer having layer_id of n in FIG. 4 is n-1 and n-2, and a layer having layer_id of n-1 is a reference layer list When the layer indexed to 0 and layer_id of n-2 is indexed to 1 of the reference layer list, when the current encoding target block refers to the reference layer having layer_id of n-2, index “1 of the spatial quality reference layer list” ” can be encoded to signal.

In this case, the spatial quality reference layer list used may be configured from the reference layer list referenced by the current encoding target layer signaled in the slice header. If the reference layer list is not signaled in the slice header, the same layers as the current encoding target layer in the entire bitstream signaled in the video parameter set may be configured from reference layers that can be referenced.

According to another example, when the current encoding object block of the current layer performs inter prediction, the encoding apparatus may perform motion prediction and motion compensation on the current encoding object block by using a reference picture in the reference picture list.

According to the present embodiment, the encoding apparatus uses a reference picture in the reference picture list including the decoded image of the view reference layer generated in step S520 to predict motion and motion of the current encoding target image using a conventional inter prediction method. compensation can be performed.

According to the present invention with reference to FIG. 5, when encoding an image of a higher layer, a spatial quality reference layer list composed of spatial and picture quality layers to be referenced within the same view as the target layer and a view composed of the same spatial and picture quality layers as the target layer There is an advantage of improving encoding efficiency by separating the reference layer list and performing inter-layer prediction in consideration of the characteristics of each layer.

7 is a control flow diagram illustrating a method of performing upper layer decoding in a decoding apparatus according to an embodiment of the present invention. The decoding apparatus according to the present invention supports at least one scalability (eg, spatial, picture quality, and view scalability) and performs upper layer decoding in a video structure supporting a multi-layered structure.

Referring to FIG. 7 , first, the decoding apparatus constructs a list of layers that can be referenced by an image of the current decoding target layer ( S710 ). For the list of layers that the image of the current decoding target layer can refer to, the spatial quality reference layer list and the view reference layer list or the image of the current decoding target layer that can be referenced by the same layers as the current decoding target layer in the entire bitstream are referenced. It can be created by deriving a list of the hierarchies it is doing.

According to an embodiment of the present invention, the decoding apparatus may construct a spatial quality reference layer list and a view reference layer list that can be referenced by the same layers as the current decoding target layer in the entire bitstream, and the thus constructed reference layer lists are currently It may be used to decode images belonging to the same layer as the layer to which the decoding target image belongs.

8 is a control flowchart illustrating a method of constructing a spatial quality reference layer list and a view reference layer list in a decoding apparatus according to an embodiment of the present invention.

First, the decoding apparatus may construct a spatial quality reference layer list by using reference layer information of a current decoding target layer that is signaled and included in a video parameter set ( S810 ).

For example, as shown in Table 1, in the decoding apparatus, among reference layers (ref_layer_id) of a layer having a value of layer_id_in_nuh[i], a spatial and quality reference layer having the same view as the current decoding target. A list of spatial image quality reference layers having the same viewpoint as the current decoding target may be configured with the .

According to another example, the decoding apparatus uses spatial and quality reference layers having the same view point as the current decoding target among reference layers of a layer having a value of nuh_layer_id signaled as shown in Table 2 A reference hierarchy list can be constructed.

In constructing the spatial image quality reference layer list, the order of the layers may be variously set.

For example, in the decoding apparatus, among spatial and quality reference layers having the same view as the current decoding target, the layer_id value from the layer having a small difference from the layer_id value of the decoding target layer (that is, the closest layer) to the largest layer The spatial quality reference layer list can be constructed in order.

Alternatively, the decoding apparatus may configure the spatial quality reference layer list in the order of a layer having a high priority to a layer having a low priority among spatial and quality reference layers having the same view as the current decoding target.

In this case, information on priority may be signaled in a NAL unit header or a video parameter set.

Alternatively, the decoding apparatus may configure the spatial quality reference layer list in order from the smallest to the largest spatial resolution difference from the current decoding target layer among spatial and quality reference layers having the same view as the current decoding target. have.

In this case, the image quality reference layer order in the same spatial resolution may be configured from a layer having a small difference from the layer_id value of the current decoding target layer (that is, a layer close to it) to a larger layer order.

For example, in the bitstream structure shown in FIG. 4 , the reference layer list of the layer having the layer_id of n may be configured in the order of the layers having the layer_id of n-1, n-2, and n-3.

Alternatively, the referenceable spatial quality reference layer list having the same view as the current decoding target layer is a layer having a small spatial resolution difference from the current decoding target layer among spatial and quality reference layers having the same view as the current decoding target layer. It can be configured in the order of the largest hierarchy from In this case, the order of the picture quality reference layers in the same spatial resolution may be an order of a quantization parameter to be decoded from a low value to a high value (ie, from a layer having a good picture quality to a low layer during decoding).

When the spatial quality reference layer list that the same layers as the target layer can refer to is generated, the decoding apparatus uses the reference layer information of the current decoding target layer signaled by being included in the video parameter set to have the same space as the current decoding target layer. A referenceable view reference layer list composed of picture quality layers may be configured (S820).

For example, among the reference layers (ref_layer_id) of the layer having the value of layer_id_in_nuh[i] signaled as shown in Table 1, the decoding apparatus is configured among the layers having the same spatial and same quality as the current decoding target. The reference layer list may be composed of layers having different views of the current encoding target layer.

According to another example, the decoding apparatus currently encodes a current encoding target among reference layers of a layer having a value of nuh_layer_id signaled as shown in Table 2, among layers having the same spatial and same quality as a current decoding target. A view reference layer list may be configured with layers having different views of the layer.

The decoding apparatus may configure the view reference layer list in the order in which view reference layers including the same space and the same picture quality layers as the current encoding target layer are signaled.

Alternatively, the decoding apparatus may generate a view reference layer list in which view reference layers composed of the same space and the same picture quality layers as the current decoding target layer are in the order of a view closer to the current decoding target view to a distant view.

According to another embodiment of the present invention, the decoding apparatus may construct a spatial quality reference layer list and a view reference layer list that can be referenced by a current encoding target layer (or a corresponding slice) of an image to be currently encoded, and the configured reference layer lists are It may be used to decode a current decoding target image.

The decoding apparatus may configure the spatial quality reference layer list and the view reference layer list by using reference layer information signaled in the slice header of the current decoding target layer.

The decoding apparatus may have the same reference layer information signaled in the slice header even when the current decoding target image is divided into one or more slices.

First, the decoding apparatus may configure a spatial quality reference layer list that can be referenced by a current encoding target layer of an image to be currently encoded using one of the following methods.

For example, using one of the schemes from Tables 3 to 12, spatial image quality as spatial and quality reference layers having the same view point as the current decoding target among reference layers signaled to the slice header A reference layer list can be constructed.

In this case, the reference layers signaled in the slice header may be a subset of reference layers that can be referenced by the same layers as the current decoding target layer in the entire bitstream.

For example, the reference layers signaled in the slice header may be a subset of the reference layer list that can be referenced by the same layers as the current decoding target layer signaled in the video parameter set.

In this case, the decoding apparatus may configure the order of the layers in various ways in configuring the spatial image quality reference layer list.

For example, in the decoding apparatus, among spatial and quality reference layers having the same view as the current decoding target, the layer_id value is the layer with a small difference from the layer_id value of the decoding target layer (ie, the closest layer) to the largest layer order It is possible to create a spatial quality reference layer list by configuring .

Alternatively, the spatial quality reference layer list may be configured in order from a layer having a high priority to a layer having a low priority among spatial and picture quality reference layers having the same view as the target layer.

In this case, priority-related information may be signaled by being included in a NAL unit header (NALU header) or a video parameter set.

Alternatively, the referenceable spatial quality reference layer list having the same view as the current decoding target layer is a layer having a small spatial resolution difference from the current encoding target layer among spatial and quality reference layers having the same view as the current decoding target layer. It can be configured in the order of the largest hierarchy from In this case, the order of the picture quality reference layers having the same spatial resolution may be from a layer having a small difference from the layer_id value of the current decoding target layer (ie, a layer close to it) to a larger layer order.

For example, in the bitstream structure shown in FIG. 4 , the reference layer list of the layer having the layer_id of n may be configured in the order of the layers having the layer_id of n-1, n-2, and n-3.

Alternatively, the referenceable spatial quality reference layer list having the same view as the current decoding target layer is a layer having a small spatial resolution difference from the current decoding target layer among spatial and quality reference layers having the same view as the current decoding target layer. It can be configured in the order of the largest hierarchy from In this case, the order of the picture quality reference layers in the same spatial resolution may be an order of a quantization parameter to be decoded from a low value to a high value (ie, from a layer having a good picture quality to a low layer during decoding).

When the spatial quality reference layer list that can be referenced by the same layers as the target layer is generated, the decoding apparatus may construct a referenceable view reference layer list including the same spatial and same picture quality layers as the current decoding target layer.

For example, the decoding apparatus uses one of the methods of Tables 3 to 12, among reference layers signaled to the slice header, among the layers having the same spatial and same quality as the current decoding target. The view reference layer list may be composed of layers having different views of the current encoding target layer.

The decoding apparatus may configure the view reference layer list in the order in which view reference layers including the same space and the same picture quality layers as the current encoding target layer are signaled.

Alternatively, the decoding apparatus may generate a view reference layer list in which view reference layers composed of the same space and the same picture quality layers as the current decoding target layer are in the order of a view closer to the current decoding target view to a distant view.

The maximum number of layers that can be referenced may be limited for the entire bitstream, which may be signaled in a video parameter set, a sequence parameter set, or a slice header, or may be restricted according to a profile and a level.

When there is additional signaling (eg, higher-level signaling such as a slice header) with respect to the configured reference layer list, the decoding apparatus may change the order in the list according to content expressed in the signaling.

Next, the decoding apparatus generates a reference picture list for inter prediction of the current decoding target image including the decoded image of the view reference layer (S720).

The decoding apparatus may configure a reference picture set and perform reference picture marking for inter prediction of the current decoding target image including the decoded image of the view reference layer.

That is, the decoding apparatus configures a reference picture set (a first set) including images included in the view reference layer list. At this time, it is checked whether the image included in the viewpoint reference layer list is available as a reconstructed image, and if available, the reconstructed image is included in the reference picture set, and if not available, the reconstructed image is set as “no reference picture”. ” can be indicated.

A reference picture set composed of images included in the view reference layer list may be marked as “used for long term reference” and treated as a long-term reference picture when inter prediction of a current decoding target image is performed.

In addition to the first set, that is, the reference picture set composed of images included in the view reference layer list, the decoding apparatus may configure various reference picture sets as follows for inter prediction composed of images of the same layer as the current decoding target layer. have.

The reference picture sets are used for inter prediction of the current decoding target image, and are used for inter prediction of the short-term reference picture (the second set) that is previous based on the current decoding target image in the display order, and the current decoding target image. Short-term reference pictures (third set) that are later in order based on the current decoding target image, long-term reference pictures for inter-screen prediction of the current decoding target image (fourth set), and images that can be decoded after the current decoding target image It may be at least one of a short-term reference picture (a fifth set) for , and a long-term reference picture (a sixth set) for an image that can be decoded after the current decoding target image.

The decoding apparatus may generate a reference picture list of a current decoding target image according to the reference picture set and the reference picture type. That is, the decoding apparatus may generate a reference picture list by combining the first set and the second to fourth sets.

For example, when the decoding apparatus generates the reference picture list of the current decoding target image, the inter-screen reference picture lists L0 and L1 including reference picture sets including images of the same layer as the current decoding target image, L0 and L1, and the first set A final reference picture list can be generated by adding a reference picture set composed of the included view reference layer list.

In this case, the decoded image of the view reference layer may be added to a fixed position whenever the reference picture list is generated, or the position of the decoded image of the view reference layer may be changed for efficient encoding.

When the decoded image of the view reference layer is added at a fixed position each time the reference picture list is generated, the first set is added from the last or first (ref_idx=0) or second (ref_idx=1) position when the L0 list is generated. can

When the first set is added to the middle position of the L0 list, the index in the list of images at the position and thereafter may be increased by the number of the added view reference layers (the number of reference picture sets composed of the reference layer list). have.

Alternatively, the decoding apparatus may replace as many reference images as the number of reference picture sets including the view reference layer list from the first (ref_idx=0) or second (ref_idx=1) position as the first set when generating the L0 list.

Alternatively, the decoding apparatus may add the first set from an arbitrary signaled position when generating the L0 list. When the first set is added to the middle position of the list, the index in the list of images at the position and thereafter may be increased by the number of added view reference layers (the number of reference picture sets composed of the reference layer list). .

Alternatively, the decoding apparatus may replace as many reference pictures as the number of reference picture sets composed of the view reference layer list from an arbitrary signaled position as the first set when generating the L0 list.

Alternatively, the decoding apparatus may add each picture included in the view reference layer list of the first set to arbitrary different positions when generating the L0 list. Corresponding positions of the added images and indexes in the list of subsequent images may be increased by the number of added view reference layers (the number of reference picture sets composed of the reference layer list).

Alternatively, when the L0 list is generated, the decoding apparatus may replace reference images in arbitrary different positions with each picture included in the view reference layer list of the first set.

Alternatively, the decoding apparatus may add the first set to the last or first (ref_idx=0) or second (ref_idx=1) position when generating the L1 list.

When a view reference layer is added to the middle position of the L1 list, the index in the list of images at the position and thereafter may be increased by the number of the added view reference layers (the number of reference picture sets composed of the reference layer list) have.

Alternatively, the decoding apparatus may replace as many reference pictures as the number of reference picture sets including the view reference layer list from the first (ref_idx=0) or second (ref_idx=1) position as the first set when generating the L1 list.

Alternatively, the decoding apparatus may add the first set from an arbitrary signaled position when generating the L1 list. When the first set is added to the middle position of the list, the index in the list of images at the position and thereafter may be increased by the number of added view reference layers (the number of reference picture sets composed of the reference layer list). .

Alternatively, the decoding apparatus may replace as many reference pictures as the number of reference picture sets composed of the view reference layer list from an arbitrary signaled position as the first set when generating the L1 list.

Alternatively, the decoding apparatus may add each picture included in the view reference layer list of the first set to arbitrary different positions when generating the L1 list. Corresponding positions of the added images and indexes in the list of subsequent images may be increased by the number of added view reference layers (the number of reference picture sets composed of the reference layer list).

Alternatively, when generating the L1 list, the decoding apparatus may replace reference images located at different positions with respective pictures included in the view reference layer list of the first set.

On the other hand, when the position of the decoded image of the view reference layer is changed for additional efficient encoding after the reference picture list is generated, the decoded image of the view reference layer is changed using encoding parameters that may be included in the slice header or the picture parameter set. The position can be changed to any position in the reference picture list.

When the reference layer list is generated, the decoding apparatus may decode the image of the current layer in block units (S730).

When the current decoding target block of the current layer refers to the spatial quality reference layer, decoding can be performed as follows.

For example, the decoding apparatus may determine a reference layer used for decoding by the current decoding object block from the spatial quality reference layer list used in the current decoding object image, and determine the reference block of the reference layer.

In this case, the spatial quality reference layer list used may be configured from the reference layer list referenced by the current decoding target layer signaled in the slice header. If the reference layer list is not signaled in the slice header, the same layers as the current decoding target layer in the entire bitstream signaled in the video parameter set may be configured from reference layers that can be referenced.

The decoding apparatus may determine the spatial quality reference layer according to an index indicating the spatial quality reference layer signaled in units of a decoding object block.

When the spatial quality reference layer is determined, the decoding apparatus may determine a reference block corresponding to the current decoding object block from the determined spatial quality reference layer.

The reference block of the reference layer means a block of the reference layer corresponding to the current decoding object block of the current layer, and may mean, for example, a block existing in the same position as the current decoding object block in the reference layer.

For example, in determining the spatial quality reference layer corresponding to the decoding target block of the layer having the layer_id of n in FIG. 4 , layer_id is n-1 and layer_id in the spatial quality reference layer list of the layer having the layer_id of n at time 1 If an image with n-2 is included and the spatial quality reference layer index of the current decoding target block is “1”, the current decoding target block uses the layer whose layer_id is n-2 as the spatial quality reference layer, and refers to the spatial quality The layer may determine a reference block corresponding to the current decoding object block.

Then, the decoding apparatus determines a reconstructed sample of a reference block, a residual of the reference block, and encoding parameters (eg, reference frame, motion vector, prediction mode, block) of the reference block among the information of the reference block of the selected spatial quality reference layer. Partitioning information, etc.) may be used to decode the target block.

Meanwhile, when the current decoding object block of the current layer performs inter prediction, the decoding apparatus may perform motion compensation on the current decoding object block by using the reference picture in the reference picture list.

In this case, the decoding apparatus may perform motion compensation on the current image to be decoded by using the reference picture in the reference picture list including the decoded image of the view reference layer generated in step S720 by a typical inter prediction method.

In the embodiments described above, the methods are described on the basis of a flowchart as a series of steps or blocks, but the present invention is not limited to the order of steps, and some steps may occur in a different order or concurrently with other steps as described above. have. In addition, those of ordinary skill in the art will recognize that the steps shown in the flowchart are not exclusive, other steps may be included, or that one or more steps in the flowchart may be deleted without affecting the scope of the present invention. You will understand.

The above-described embodiments include examples of various aspects. It is not possible to describe every possible combination for representing the various aspects, but one of ordinary skill in the art will recognize that other combinations are possible. Accordingly, it is intended that the present invention cover all other substitutions, modifications and variations falling within the scope of the following claims.

100: video encoding device 111: motion prediction unit
112: motion compensation unit 120: intra prediction unit
115: switch 125: subtractor
130: transform unit 140: quantization unit
150: entropy encoding unit 160: inverse quantization unit
170: inverse transform unit 180: filter unit

Claims (20)

In the video decoding method supporting a plurality of layers,
generating a reference layer list that can be referenced by an image of a target layer that is a current decoding target;
generating a reference picture list including at least one of a decoded image of a view reference layer and a decoded image of a spatial quality reference layer for inter prediction of the image of the target layer;
Predicting and decoding the image of the target layer in block units with reference to the reference picture list,
The step of generating the reference picture list comprises:
constructing a first set including the decoded image of the view reference layer;
constructing a second set comprising an image of the same layer as the image of the target layer;
combining the first set and the second set;
The picture included in the first set is displayed as a long-term reference picture. According to claim 1,
The step of generating the reference layer list comprises:
and generating a spatial quality reference layer list and a view reference layer list that can be referenced by the same layer as the target layer in the entire bitstream. 3. The method of claim 2,
The image decoding method according to claim 1, wherein the spatial quality reference layer list includes layers having the same viewpoint as the target layer. 3. The method of claim 2,
The view reference layer list is an image decoding method, characterized in that it is composed of a layer having the same space and quality as the target layer. delete delete According to claim 1,
The images included in the first set are added to any one of the first, second, and last of the reference picture list. According to claim 1,
Predicting and decoding the image in block units refers to the spatial quality reference layer,
Predicting and decoding the image in block units includes:
determining a reference layer used for decoding by a current decoding object block from the spatial quality reference layer list;
determining a reference block corresponding to the target block in the determined spatial quality reference layer;
and decoding the target block by using at least one of a reconstructed sample of the reference block, a residual of the reference block, and an encoding parameter of the reference block. According to claim 1,
Predicting and decoding the image in block units includes:
The image decoding method according to claim 1, wherein inter prediction is performed on the current decoding object block by using the reference picture in the reference picture list. delete delete delete delete delete delete delete delete delete In the encoding method of an image supporting a plurality of layers,
generating a reference layer list that can be referenced by an image of a target layer that is a current encoding target;
generating a reference picture list including at least one of an image of a view reference layer and an image of a spatial quality reference layer for inter prediction of the image of the target layer;
Predicting and encoding the image of the target layer in block units with reference to the reference picture list,
The step of generating the reference picture list comprises:
constructing a first set including the decoded image of the view reference layer;
constructing a second set comprising an image of the same layer as the image of the target layer;
combining the first set and the second set;
The picture included in the first set is displayed as a long-term reference picture. A recording medium for storing a bitstream generated by an image encoding method supporting a plurality of layers, the recording medium comprising:
generating a reference layer list that can be referenced by an image of a target layer that is a current encoding target;
generating a reference picture list including at least one of an image of a view reference layer and an image of a spatial quality reference layer for inter prediction of the image of the target layer;
Predicting and encoding the image of the target layer in block units with reference to the reference picture list,
The step of generating the reference picture list comprises:
constructing a first set including the decoded image of the view reference layer;
constructing a second set comprising an image of the same layer as the image of the target layer;
combining the first set and the second set;
A picture included in the first set is displayed as a long-term reference picture.

Images