KR102506469B1 - Method and apparatus for image encoding/decoding - Google Patents

Abstract

An image encoding/decoding method and apparatus supporting a plurality of layers are disclosed. The video decoding method supporting the plurality of layers includes decoding information of a first layer referred to by a picture of a second layer including a current decoding target block, and the information of the first layer using the size of the picture of the second layer. mapping, constructing a reference picture list for a picture of the second layer by adding information of the mapped first layer, and a current decoding target of the second layer based on the reference picture list and generating prediction samples of the current block to be decoded by performing prediction on the block.

Description

Video encoding/decoding method and apparatus {METHOD AND APPARATUS FOR IMAGE ENCODING/DECODING}

The present invention relates to image encoding and decoding, and more particularly, to a method of predicting, encoding, and decoding an image of an upper layer using information of a lower layer when encoding and decoding a multi-layered image.

Recently, as a multimedia environment is established, various terminals and networks are being used, and user demands are also diversifying accordingly.

For example, as the performance and computing capability of terminals diversify, the supported performance also diversifies for each device. In addition, networks through which information is transmitted are also diversifying not only in terms of external structures such as wired and wireless networks, but also in terms of functions such as the type of information to be transmitted, the amount of information, and speed. Users select terminals and networks to use according to desired functions, and the spectrum of terminals and networks provided by companies to users is also diversifying.

In this regard, as broadcasting having a high definition (HD) resolution is recently expanded and serviced not only domestically but also globally, many users are getting used to high-resolution and high-definition images. Accordingly, many video service-related institutions are making great efforts to develop next-generation video devices.

In addition, as interest in UHD (Ultra High Definition) having a resolution four times higher than that of HDTV increases along with HDTV, a demand for a technology for compressing and processing higher resolution and higher quality images is increasing.

In order to compress and process the image, inter prediction technology predicts pixel values included in the current picture from temporally previous and/or subsequent pictures, and other pixels included in the current picture using pixel information in the current picture. An intraprediction technique for predicting a value, an entropy encoding technique for assigning a short code to a symbol with a high frequency of occurrence and a long code to a symbol with a low frequency of occurrence may be used.

As described above, when considering the needs of each terminal and network having different supported functions and diversified users, the quality, size, frame, etc. of the supported video need to be diversified accordingly.

As such, due to heterogeneous communication networks and terminals of various functions and types, scalability, which supports various image quality, resolution, size, frame rate, and the like, has become an important function of a video format.

Therefore, it is necessary to provide a scalability function to enable efficient video encoding and decoding in terms of time, space, image quality, etc. in order to provide services required by users in various environments based on a highly efficient video encoding method.

The present invention provides a method and apparatus for encoding/decoding an upper layer using information of a lower layer in scalable video coding.

The present invention provides a method and apparatus for mapping lower layer images in scalable video coding.

The present invention provides a method and apparatus for constructing a reference picture list of an upper layer image using a lower layer image and performing prediction in scalable video coding.

According to an embodiment of the present invention, an image decoding method supporting a plurality of layers is provided. The video decoding method includes decoding information of a first layer referred to by a picture of a second layer including a current decoding target block, and mapping the information of the first layer to a picture size of the second layer. constructing a reference picture list for a picture of the second layer by adding the mapped information of the first layer, and predicting a current decoding target block of the second layer based on the reference picture list and generating prediction samples of the current decoding object block by doing so.

The information of the first layer may include at least one of a sample value and motion information of the first layer picture.

According to another embodiment of the present invention, an image decoding apparatus supporting a plurality of layers is provided. The video decoding apparatus decodes information of a first layer referred to by a picture of a second layer including a current decoding target block, and maps the information of the first layer to the picture size of the second layer and constructs a reference picture list for a picture of the second layer by adding the mapped information of the first layer, and performs prediction on a current decoding target block of the second layer based on the reference picture list. and a prediction unit generating prediction samples of the current decoding object block.

The information of the first layer may include at least one of a sample value and motion information of the first layer picture.

According to another embodiment of the present invention, an image encoding method supporting a plurality of layers is provided. The video encoding method includes decoding information of a first layer referred to by a picture of a second layer including a current encoding target block, and mapping the information of the first layer to a picture size of the second layer. Constructing a reference picture list for a picture of the second layer by adding the mapped information of the first layer, and predicting a current encoding target block of the second layer based on the reference picture list. and generating prediction samples of the current block to be encoded.

The information of the first layer may include at least one of a sample value and motion information of the first layer picture.

According to another embodiment of the present invention, an image encoding apparatus supporting a plurality of layers is provided. The video encoding apparatus encodes information of a first layer referred to by a picture of a second layer including a current encoding target block, and maps the information of the first layer to the picture size of the second layer. and constructs a reference picture list for a picture of the second layer by adding the mapped information of the first layer, and performs prediction on a current encoding target block of the second layer based on the reference picture list. and a prediction unit generating prediction samples of the current encoding target block.

The information of the first layer may include at least one of a sample value and motion information of the first layer picture.

In the prior art, in a process of mapping motion information of a lower layer, an unnecessary motion information mapping process is performed even when temporal motion vector prediction of an upper layer is not performed. In addition, when the motion information mapping of the lower layer is not performed, since the decoded picture of the lower layer can be indicated as the collocated picture for the temporal motion vector in the upper layer, temporal motion vector prediction of the upper layer is performed. There is a problem in that encoding efficiency is lowered because a case in which it cannot be performed is generated.

According to the present invention, encoding/decoding efficiency is improved by performing mapping of motion information of a lower layer referred to by an upper layer to the image size of the upper layer and then predicting and restoring the upper layer using the mapped image signal of the lower layer. can improve In addition, by modifying the upper level syntax, an unnecessary mapping process for motion information of a lower layer can be omitted and complexity can be reduced. In addition, it is possible to prevent a decrease in coding efficiency by preventing an erroneous corresponding picture from being used.

1 is a block diagram showing a configuration according to an embodiment of an image encoding apparatus to which the present invention is applied.
2 is a block diagram showing a configuration according to an embodiment of a video decoding apparatus to which the present invention is applied.
3 is a conceptual diagram schematically illustrating an embodiment of a scalable video coding structure using a plurality of layers to which the present invention can be applied.
4 is a flowchart schematically illustrating a method of performing inter-layer prediction (inter-layer prediction) in scalable video coding according to an embodiment of the present invention.
5 and 6 are diagrams for explaining a motion information mapping method between layers according to an embodiment of the present invention.

EMBODIMENT OF THE INVENTION Hereinafter, embodiment of this invention is described concretely with reference to drawings. In describing the embodiments of the present specification, when it is determined that a detailed description of a related known configuration or function may obscure the gist of the present specification, the corresponding description may be omitted.

In this specification, when a component is referred to as “connected” or “connected” to another component, it may mean that it is directly connected or connected to the other component, and another component in the middle. It can also mean that an element exists. In addition, the description of "including" a specific configuration in this specification does not exclude configurations other than the corresponding configuration, and means that additional configurations may be included in the practice of the present invention or the scope of the technical spirit of the present invention.

Terms such as first and second may be used to describe various configurations, but the configurations are not limited by the term. The terms are used for the purpose of distinguishing one configuration from another. For example, a first configuration may be termed a second configuration, and similarly, a second configuration may also be termed a first configuration, without departing from the scope of the present invention.

In addition, the components appearing in the embodiments of the present invention are shown independently to indicate different characteristic functions, and it does not mean that each component is made of a separate hardware or a single software component unit. That is, each component is listed and included as each component for convenience of explanation, and at least two components of each component may form one component, or one component may be divided into a plurality of components to perform functions. Integrated embodiments and separate embodiments of each component are included in the scope of the present invention as long as they do not depart from the essence of the present invention.

In addition, some of the components may be optional components for improving performance rather than essential components that perform essential functions in the present invention. The present invention can be implemented by including only components essential to implement the essence of the present invention, excluding components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement. Also included in the scope of the present invention.

1 is a block diagram showing a configuration according to an embodiment of an image encoding apparatus to which the present invention is applied.

A scalable video encoding apparatus supporting a multi-layer structure may be implemented by extending a general video encoding apparatus having a single-layer structure. The block diagram of FIG. 1 shows an embodiment of an image encoding apparatus that can be a basis of a scalable video encoding apparatus applicable to a multi-layer structure.

Referring to FIG. 1 , an image encoding apparatus 100 includes an inter prediction unit 110, an intra prediction unit 120, a switch 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy It includes an encoder 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190.

The image encoding apparatus 100 may encode an input image in an intra mode or an inter mode and output a bitstream.

In the case of the intra mode, the switch 115 may be switched to the intra mode, and in the case of the inter mode, the switch 115 may be switched to the inter mode. Intra prediction means prediction within a picture, and inter prediction means prediction between pictures. After generating a prediction block for an input block of an input image, the image encoding apparatus 100 may encode a residual between the input block and the prediction block. In this case, the input image may mean an original picture.

In the intra mode, the intra predictor 120 may use sample values of previously encoded/decoded blocks adjacent to the current block as reference samples. The intra predictor 120 may perform spatial prediction using reference samples and generate prediction samples for the current block.

In the case of the inter mode, the inter prediction unit 110 generates a motion vector specifying a reference block having the smallest difference from the input block (current block) in the reference picture stored in the reference picture buffer 190 in the motion prediction process. can be saved The inter prediction unit 110 may generate a prediction block for a current block by performing motion compensation using a motion vector and a reference picture stored in the reference picture buffer 190 .

In the case of a multi-layer structure, inter prediction applied in an inter mode may include inter-layer prediction. The inter-prediction unit 110 may perform inter-layer prediction by constructing an inter-layer reference picture by sampling a picture of a reference layer and including the inter-layer reference picture in a reference picture list. The reference relationship between layers may be signaled through information specifying dependencies between layers.

Meanwhile, when the current layer picture and the reference layer picture have the same size, sampling applied to the reference layer picture may mean generation of a reference sample by copying or interpolating a sample from the reference layer picture. When resolutions of the current layer picture and the reference layer picture are different, sampling applied to the reference layer picture may mean upsampling.

For example, when resolutions between layers are different, an inter-layer reference picture may be constructed by upsampling a reconstructed picture of a reference layer between layers supporting resolution-related scalability.

An inter-layer reference picture may be determined using a picture of a layer in consideration of an encoding cost or the like. The encoding device may transmit information specifying a layer to which a picture to be used as an inter-layer reference picture belongs to a decoding device.

In addition, in inter-layer prediction, a picture used for prediction of a current block in a layer referenced, that is, a reference layer, may be a picture of the same access unit (AU) as the current picture (prediction target picture in the current layer).

The subtractor 125 may generate a residual block by a difference between the input block and the generated prediction block.

The transform unit 130 may output a transform coefficient by performing transform on the residual block. Here, the transform coefficient may refer to a coefficient value generated by transforming a residual block and/or a residual signal. Hereinafter, in this specification, a quantized transform coefficient level generated by applying quantization to a transform coefficient may also be referred to as a transform coefficient.

When a transform skip mode is applied, the transform unit 130 may skip transform on the residual block.

The quantization unit 140 may quantize the input transform coefficient according to a quantization parameter and output a quantized coefficient. A quantized coefficient may be referred to as a quantized transform coefficient level. At this time, the quantization unit 140 may quantize the input transform coefficient using a quantization matrix.

The entropy encoding unit 150 may output a bitstream by entropy encoding the values calculated by the quantization unit 140 or the encoding parameter values calculated in the encoding process according to a probability distribution. The entropy encoding unit 150 may entropy encode information for video decoding (eg, syntax elements, etc.) in addition to video pixel information.

The encoding parameter is information necessary for encoding and decoding, and may include information that can be inferred in the encoding or decoding process as well as information encoded in the encoding device and transmitted to the decoding device, such as a syntax element.

For example, encoding parameters include values such as intra/inter prediction mode, motion/motion vector, reference image index, coded block pattern, residual signal presence or absence, transform coefficient, quantized transform coefficient, quantization parameter, block size, block division information, and the like. or statistics.

The residual signal may mean the difference between the original signal and the prediction signal, and also obtain a signal in which the difference between the original signal and the prediction signal is transformed or a signal in which the difference between the original signal and the prediction signal is transformed and quantized. could mean The residual signal may be referred to as a residual block in a block unit.

When entropy encoding is applied, a small number of bits are allocated to symbols with a high probability of occurrence and a large number of bits are allocated to symbols with a low probability of occurrence to represent symbols, thereby increasing the size of bit streams for symbols to be encoded. can be reduced Therefore, compression performance of image encoding can be improved through entropy encoding.

For entropy encoding, the entropy encoding unit 150 may use an encoding method such as exponential golomb, context-adaptive variable length coding (CAVLC), or context-adaptive binary arithmetic coding (CABAC). For example, the entropy encoding unit 150 may perform entropy encoding using a variable length coding/code (VLC) table. In addition, the entropy encoding unit 150 derives a binarization method of the target symbol and a probability model of the target symbol/bin, and then performs entropy encoding using the derived binarization method or probability model. You may.

Since the video encoding apparatus 100 according to the embodiment of FIG. 1 performs inter-prediction encoding, that is, inter-prediction encoding, a currently encoded video needs to be decoded and stored to be used as a reference video. Accordingly, the quantized coefficient may be inversely quantized in the inverse quantization unit 160 and inversely transformed in the inverse transformation unit 170 . The inverse quantized and inverse transformed coefficients are added to the prediction block through an adder 175, and a reconstructed block is generated.

A reconstructed block passes through a filter unit 180, and the filter unit 180 applies at least one of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to a reconstructed block or a reconstructed picture. can do. The filter unit 180 may also be called an adaptive in-loop filter. The deblocking filter may remove block distortion generated at a boundary between blocks. SAO may add an appropriate offset value to a pixel value to compensate for a coding error. ALF may perform filtering based on a value obtained by comparing a reconstructed image with an original image. A reconstructed block that has passed through the filter unit 180 may be stored in the reference picture buffer 190 .

2 is a block diagram showing a configuration according to an embodiment of a video decoding apparatus to which the present invention is applied.

A scalable video decoding apparatus supporting a multi-layer structure may be implemented by extending a general video decoding apparatus having a single-layer structure. The block diagram of FIG. 2 shows an embodiment of an image decoding apparatus that can be a basis of a scalable video decoding apparatus applicable to a multi-layer structure.

Referring to FIG. 2 , the image decoding apparatus 200 includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra prediction unit 240, an inter prediction unit 250, and an adder 255. ), a filter unit 260 and a reference picture buffer 270.

The image decoding apparatus 200 may receive the bitstream output from the encoder, perform decoding in an intra mode or inter mode, and output a reconstructed image, that is, a reconstructed image.

In the intra mode, the switch may be converted to intra, and in the case of the inter mode, the switch may be converted to inter.

The video decoding apparatus 200 may generate a reconstructed block, that is, a reconstructed block, by obtaining a reconstructed residual block from the input bitstream, generating a prediction block, and then adding the reconstructed residual block and the prediction block. .

The entropy decoding unit 210 may entropy-decode the input bitstream according to a probability distribution and output information such as quantized coefficients and syntax elements.

The quantized coefficient is inversely quantized in the inverse quantization unit 220 and inversely transformed in the inverse transformation unit 230 . As a result of inverse quantization/inverse transformation of the quantized coefficients, a reconstructed residual block may be generated. At this time, the inverse quantization unit 220 may apply a quantization matrix to the quantized coefficients.

In the intra mode, the intra predictor 240 may perform spatial prediction using sample values of previously decoded blocks adjacent to the current block and generate prediction samples for the current block.

In the case of the inter mode, the inter prediction unit 250 may generate a prediction block for the current block by performing motion compensation using a motion vector and a reference picture stored in the reference picture buffer 270 .

In the case of a multi-layer structure, inter prediction applied in an inter mode may include inter-layer prediction. The inter-prediction unit 250 may configure an inter-layer reference picture by sampling a picture of a reference layer, and include the inter-layer reference picture in a reference picture list to perform inter-layer prediction. The reference relationship between layers may be signaled through information specifying dependencies between layers.

Meanwhile, when the current layer picture and the reference layer picture have the same size, sampling applied to the reference layer picture may mean generation of a reference sample by copying or interpolating a sample from the reference layer picture. When resolutions of the current layer picture and the reference layer picture are different, sampling applied to the reference layer picture may mean upsampling.

For example, when inter-layer resolution is different and inter-layer prediction is applied between layers supporting resolution-related scalability, an inter-layer reference picture may be constructed by upsampling a reconstructed picture of a reference layer.

In this case, information specifying a layer to which a picture to be used as an inter-layer reference picture belongs may be transmitted from the encoding device to the decoding device.

In addition, in inter-layer prediction, a picture used for prediction of a current block in a layer referenced, that is, a reference layer, may be a picture of the same access unit (AU) as the current picture (prediction target picture in the current layer).

The reconstructed residual block and the prediction block are added in an adder 255 to generate a reconstructed block. In other words, a reconstructed sample or a reconstructed picture is generated by adding residual samples and prediction samples.

The reconstructed picture is filtered in the filter unit 260. The filter unit 260 may apply at least one of a deblocking filter, SAO, and ALF to a reconstructed block or a reconstructed picture. The filter unit 260 outputs a modified or filtered reconstructed picture. The reconstructed image may be stored in the reference picture buffer 270 and used for inter prediction.

In addition, the video decoding apparatus 200 may further include a parsing unit (not shown) that parses information related to an encoded video included in a bitstream. The parsing unit may include the entropy decoding unit 210 or may be included in the entropy decoding unit 210 . This parsing unit may also be implemented as one component of the decoding unit.

In FIGS. 1 and 2, it has been described that one encoding device/decoding device processes all encoding/decoding of multi-layers, but this is for convenience of explanation, and the encoding device/decoding device may be configured for each layer.

In this case, an encoding device/decoding device of an upper layer may perform encoding/decoding of the corresponding upper layer using information of the upper layer and information of the lower layer. For example, the prediction unit (inter prediction unit) of the upper layer may perform intra prediction or inter prediction on the current block by using pixel information or picture information of the upper layer, and may receive reconstructed picture information from the lower layer and perform this Inter-prediction (inter-layer prediction) may be performed on the current block of the upper layer by using. Here, only inter-layer prediction has been described as an example, but encoding/decoding of the current layer is performed using information of another layer regardless of whether the encoding/decoding device is configured per layer or a single device processes multiple layers. can do.

In the present invention, a layer may include a view. In this case, in the case of inter-layer prediction, prediction of an upper layer is not simply performed using information of a lower layer, but information of other layers between layers specified as having dependencies by information specifying inter-layer dependencies. Inter-layer prediction may be performed using

3 is a conceptual diagram schematically illustrating an embodiment of a scalable video coding structure using a plurality of layers to which the present invention can be applied. In FIG. 3, a group of pictures (GOP) represents a picture group, that is, a group of pictures.

In order to transmit image data, a transmission medium is required, and its performance varies depending on the transmission medium according to various network environments. A scalable video coding method may be provided for application to such various transmission media or network environments.

A video coding method that supports scalability (hereinafter referred to as 'scalable coding' or 'scalable video coding') removes redundancy between layers by utilizing texture information, motion information, and residual signals between layers. This is a coding method that improves encoding and decoding performance. The scalable video coding method uses various scalers in terms of spatial, temporal, image quality (or quality), and view according to surrounding conditions such as transmission bit rate, transmission error rate, and system resources. A utility can be provided.

Scalable video coding may be performed using a multiple layer structure to provide a bitstream applicable to various network conditions. For example, the scalable video coding structure may include a base layer that compresses and processes image data using a general image decoding method, and compresses the image data using both decoding information of the base layer and a general image decoding method. An enhancement layer for processing may be included.

The base layer may be referred to as a base layer or may be referred to as a lower layer. An enhancement layer may also be referred to as an enhancement layer or a higher layer. In this case, the lower layer may mean a layer supporting lower scalability than the specific layer, and the upper layer may mean a layer supporting higher scalability than the specific layer. In addition, a layer referred to for encoding/decoding of another layer may be referred to as a reference layer (reference layer), and a layer encoded/decoded using another layer may be referred to as a current layer (current layer). The reference layer may be a layer lower than the current layer, and the current layer may be a layer higher than the reference layer.

Here, the layer is based on spatial (eg, video size), temporal (eg, decoding order, video output order, frame rate), quality, complexity, view, etc. It means a set of differentiated images and bitstreams.

Referring to FIG. 3 , for example, the base layer may be defined as standard definition (SD), frame rate of 15 Hz, and bit rate of 1 Mbps, and the first enhancement layer may be defined as high definition (HD), frame rate of 30 Hz, and bit rate of 3.9 Mbps. , and the second enhancement layer may be defined as 4K-UHD (ultra high definition), a frame rate of 60 Hz, and a bit rate of 27.2 Mbps.

The format, frame rate, bit rate, etc. are one embodiment and may be determined differently as needed. Also, the number of layers used is not limited to the present embodiment and may be determined differently according to circumstances. For example, if the transmission bandwidth is 4 Mbps, the frame rate of the first enhancement layer HD may be reduced and transmitted to 15 Hz or less.

The scalable video coding method may provide temporal, spatial, quality, and viewpoint scalability by the method described above in the embodiment of FIG. 3 . In this specification, scalable video coding has the same meaning as scalable video encoding in terms of encoding and scalable video decoding in terms of decoding.

In general, inter-prediction (hereinafter referred to as inter-prediction) may use at least one of a previous picture or a subsequent picture of a current picture as a reference picture and perform prediction on a current block based on the reference picture.

An image used for prediction of the current block is referred to as a reference picture or reference frame.

A region (reference block) used for prediction of a current block within a reference picture can be expressed using a reference picture index (refIdx) and a motion vector indicating the reference picture.

Inter prediction may generate a prediction block for the current block by selecting a reference picture and a reference block corresponding to the current block within the reference picture. In this case, inter prediction may generate a prediction block such that a residual signal with the current block is minimized and the magnitude of the motion vector is also minimized.

In order to use information of a reference picture in inter prediction, information of neighboring blocks located around the current block may be used. For example, inter prediction may apply a skip mode, a merge mode, Advanced Motion Vector Prediction (AMVP), or the like according to a method of using information of neighboring blocks. These skip mode, merge mode, and AMVP mode can generate a prediction block for the current block based on information on neighboring blocks.

In the skip mode, information of neighboring blocks can be used for the current block as it is. Therefore, when the skip mode is applied, the encoder only needs to transmit information indicating which motion information of a neighboring block is to be used as the motion information of the current block to the decoder, and syntax information such as residual is not transmitted to the decoder. .

In the merge mode, a prediction block for a current block may be generated by using motion information of neighboring blocks as it is. When merge mode is applied, the encoder may transmit information indicating whether merge mode is applied to the current block, information indicating which motion information of a neighboring block is to be used, and residual information about the current block to the decoder. . The decoder may generate a prediction block for the current block using motion information of neighboring blocks indicated by the encoder, and may reconstruct the current block by adding the generated prediction block to the residual transmitted from the encoder.

AMVP can predict a motion vector of a current block using motion information of neighboring blocks. When AMVP is applied, the encoder may transmit information indicating which neighboring block motion information is used, a difference between a motion vector of a current block and a predicted motion vector, a reference picture index indicating a reference picture, and the like to the decoder. . The decoder may predict a motion vector of the current block using motion information (motion vector) of neighboring blocks, and may derive a motion vector of the current block using a motion vector difference received from the encoder. The decoder may generate a prediction block for the current block based on the derived motion vector and reference picture index information received from the encoder.

In the case of inter prediction, the decoder checks the skip flag, merge flag, etc. received from the encoder, and derives motion information (eg, motion vector, information about the reference picture index, etc.) necessary for inter prediction of the current block according to the checked information. can

A processing unit in which prediction is performed may be different from a processing unit in which a prediction method and specific details are determined. For example, inter prediction or intra prediction may be determined in units of coding units (CUs), and prediction modes for intra prediction and prediction modes for inter prediction may be determined in units of prediction units (PUs). there is. Alternatively, a prediction mode may be determined in units of prediction units and prediction may be performed in units of transform units (TUs).

On the other hand, since there is strong correlation between layers in a scalable video coding structure that supports multiple layers, performing prediction using this correlation can remove redundant data elements and improve video encoding performance. there is. Therefore, when predicting a picture (image) of a current layer (upper layer) to be encoded/decoded, not only inter prediction or intra prediction using information of the current layer, but also inter-layer prediction using information of other layers (inter-layer prediction, Or inter-layer prediction) may be performed.

Since the plurality of layers may differ from each other in at least one of resolution, frame rate, color format, and viewpoint (ie, a difference in scalability between layers), signal distortion may occur during inter-layer prediction for the current layer, and residual signal may increase.

Therefore, in the present invention, after performing mapping of the motion information of the reference layer (lower layer) referred to by the current layer (upper layer) to the image size of the current layer, the current layer is added to the reference picture list of the current layer. Provides a method for performing inter prediction for

In addition, the present invention relates to encoding and decoding of an image including a plurality of layers or views, wherein the plurality of layers or views are first, second, third, nth layer or first, It can be expressed as the second, third, and nth viewpoints.

Hereinafter, in the embodiment of the present invention, for convenience of description, an image in which a first layer and a second layer are present is taken as an example, but the same method can be applied to an image in which layers or viewpoints are present. In addition, the first layer may be expressed as a base layer, base layer, or reference layer, and the second layer may be expressed as an enhancement layer, an enhancement layer, or a current layer.

4 is a flowchart schematically illustrating a method of performing inter-layer prediction (inter-layer prediction) in scalable video coding according to an embodiment of the present invention. The method of FIG. 4 may be performed by the above-described video encoding apparatus of FIG. 1 and the video decoding apparatus of FIG. 2 .

Referring to FIG. 4 , the encoding/decoding apparatus decodes first layer information referred to by a second layer image (picture) (S400).

As described above, the second layer means a layer currently performing encoding/decoding, and may be a higher layer providing higher scalability than the first layer. The first layer may be a layer referenced for encoding/decoding of the second layer, and may be referred to as a reference layer or a lower layer.

The encoding/decoding apparatus may decode and use information of the first layer as a reference signal for predicting a block to be encoded/decoded in an image of the second layer.

Information of the first layer to be decoded (decoding object information of the first layer) may include sample values and motion information of the first layer image (picture). Here, the motion information includes a motion vector value, a reference picture index, a prediction direction indicator, a reference picture POC (Picture Order Count), a prediction mode, a reference picture list, a merge flag, a merge index, and a picture type of the reference picture (short-term ) reference picture, long-term reference picture), and the like.

The decoded motion information of the first layer may be compressed into NxN-sized unit blocks and stored. For example, decoded motion information of the first layer may be compressed and stored every 16x16 block.

The encoding/decoding apparatus maps the decoded information of the first layer to the image size of the second layer (S410).

Here, the mapping may be to perform sampling on samples of an image in order to equally adjust the image size or resolution when the size or resolution of the images is different, and the sampling of the sample value of the image and the motion information of the image may include sampling for

In other words, the encoding/decoding apparatus may perform sampling on the decoded first layer image in order to map the decoded first layer image to the size of the second layer image, and sample values of the sampled first layer image. and motion information.

In the scalable video coding structure, since the size of images between layers may be different, when the sizes of images between layers (between the first layer and the second layer) are different, the encoding/decoding apparatus controls the size of the decoded first layer image. Resampling may be performed on sample values of the decoded layer 1 image to be mapped to the size of a layer 2 image.

For example, when the size of the decoded first layer image is 960x540 and the size of the second layer image is 1920x1080, the encoding/decoding device converts the decoded first layer image size to 1920x1080, which is the size of the second layer image. Upsampling of the mapping may be performed. Upsampling can be performed to adjust the image size between layers having different image sizes, and interpolated samples are derived by applying interpolation to images of layers (reference layer or lower layer) having a small image size. and the image size can be adjusted.

The encoding/decoding apparatus may map and use the decoded motion information of the first layer to the image size of the second layer. Inter-layer motion information mapping that maps the motion information of the decoded first layer image to the image size of the second layer divides the second layer image into NxN-sized unit blocks, and each NxN unit block of the second layer Corresponding motion information of the first layer may be used as motion information of a corresponding block of the second layer.

Hereinafter, a motion information mapping method between layers according to an embodiment of the present invention will be described through the embodiments of FIGS. 5 and 6 .

5 and 6 are diagrams for explaining a motion information mapping method between layers according to an embodiment of the present invention.

5 shows a second layer image divided into NxN-sized unit blocks.

For example, as shown in FIG. 5, when the image size of the second layer is 1920x1080 and N = 16, the encoding/decoding device may divide the image of the second layer into a total of 8160 16x16 unit blocks, Motion information of a block of the first layer corresponding to each unit block of the second layer may be used as motion information of a corresponding block of the second layer.

In this case, motion information may be stored for each unit block. Alternatively, motion information may be stored in the form of a look-up table for each picture and used as a reference for each unit block.

FIG. 6 shows an NxN block of the second layer, and for example, a 16x16 block 600 when N=16 is shown. Each square in the 16x16 block 600 means one sample, and it is assumed that the position of the top leftmost sample in the 16x16 block 600 is (xP, yP).

As described above, when inter-layer motion information is mapped, motion information of a first layer block corresponding to each NxN unit block of a second layer is mapped to motion information of a second layer and used. In this case, the motion information of the first layer block corresponding to each NxN block of the second layer may be the motion information of the first layer sample position corresponding to the reference sample location of the second layer NxN block.

For example, as shown in FIG. 6, the encoding/decoding apparatus determines the position (xP+8, yP+8) in the 16x16 block 600 of the second layer as the position of the reference sample, and the position of the reference sample (xP +8, yP+8) may perform motion information mapping between layers using the motion information of the first layer sample position corresponding to yP+8).

Alternatively, the encoding/decoding apparatus determines the (xP, yP) position within the 16x16 block 600 of the second layer as the reference sample position, and motion information of the first layer sample position corresponding to the reference sample position (xP, yP) Motion information mapping between layers may be performed using

Alternatively, the encoding/decoding apparatus determines the location (xP+15, yP) within the 16x16 block 600 of the second layer as the location of the reference sample, and the first layer sample corresponding to the location (xP+15, yP) of the reference sample Motion information mapping between layers may be performed using motion information of a location.

Alternatively, the encoding/decoding apparatus determines the position (xP, yP + 15) in the 16x16 block 600 of the second layer as the reference sample position, and the first layer sample corresponding to the reference sample position (xP, yP + 15) Motion information mapping between layers may be performed using motion information of a location.

Alternatively, the encoding/decoding apparatus determines the position (xP + 15, yP + 15) in the 16x16 block 600 of the second layer as the reference sample position, and corresponds to the reference sample position (xP + 15, yP + 15) Motion information mapping between layers may be performed using the motion information of the sample position of the first layer.

Alternatively, the encoding/decoding apparatus determines the position (xP + 1, yP + 1) in the 16x16 block 600 of the second layer as the reference sample position, and corresponds to the reference sample position (xP + 1, yP + 1) Motion information mapping between layers may be performed using the motion information of the sample position of the first layer.

The encoding/decoding apparatus may perform inter-layer motion information mapping using not only the reference sample positions mentioned above but also motion information of the first hierarchical sample positions corresponding to other reference sample positions.

The position of the sample of the first layer corresponding to the position of the reference sample of the second layer may be calculated as shown in Equation 1 below in consideration of the size of the inter-layer image.

Here, xRef and yRef are sample positions of the first layer, xP and yP are reference sample positions of the second layer, ScaledW and ScaledH are the horizontal and vertical sizes of the scaled picture of the first layer, and picWRL and picHRL are the first The horizontal and vertical dimensions of the layer picture.

For example, when the motion information of the first layer is compressed and stored in units of 16x16, the sample position of the first layer calculated by Equation 1 may use the motion information of 16x16 units through Equation 2 below. can be adjusted so that

As another example, when the motion information of the first layer is compressed and stored in 8x8 units, the position of the sample in the first layer calculated by Equation 1 is such that 8x8 unit motion information can be used through Equation 3 below. can be adjusted

When the prediction mode of the first layer sample location corresponding to the reference sample location of the second layer is an intra prediction mode, the encoding/decoding apparatus may use (0, 0) as a motion vector value of the corresponding block of the second layer, A value of -1 may be assigned to the reference picture index and reference picture POC of the corresponding block of the second layer.

Alternatively, when the prediction mode of the first layer sample position corresponding to the reference sample position of the second layer is an intra prediction mode, the encoding/decoding apparatus may use (0, 0) as the motion vector value of the corresponding block of the second layer. In addition, the reference picture index and reference picture POC of the corresponding block in the second layer have specific values (eg, the reference picture index is assigned as 0, and the reference picture POC is assigned as the POC of the picture indicated by the reference picture index 0). can be assigned

When the prediction mode of the first layer sample position corresponding to the reference sample position of the second layer is the inter prediction mode, the encoding/decoding apparatus uses the motion vector value of the first layer sample location as the motion information of the corresponding block in the second layer, reference Picture index and reference picture POC values can be used.

When using the motion vector of the first layer, the encoding/decoding apparatus may use the motion vector of the first layer by reflecting the size of the inter-layer image as shown in Equation 4 below.

The encoding/decoding apparatus transmits picture type information (picture type information indicating whether the reference picture is a short-term reference picture or a long-term reference picture) of a reference picture referenced by a corresponding block of the first layer corresponding to a block of the second layer to a second layer. It can be used as information of the block corresponding to the layer.

The above-described motion information mapping between layers may be determined at a higher level, and the encoding apparatus may transmit motion information mapping related information between layers at a higher level. The decoding apparatus may obtain inter-layer motion information mapping-related information from a higher level.

The upper level may be a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice segment header, and the like.

Hereinafter, a method of signaling information related to motion information mapping at a higher level according to an embodiment of the present invention will be described.

According to an embodiment of the present invention, as shown in Table 1, information related to motion information mapping of a corresponding layer can be signaled in the VPS.

Referring to Table 1, when inter_layer_mfm_enable_flag is 0, motion information mapping of the i-th layer may not be performed.

When inter_layer_mfm_enable_flag is 1, motion information mapping of the i-th layer may be performed.

According to another embodiment of the present invention, as shown in Table 2, information related to motion information mapping of a corresponding layer may be signaled in an extension of the SPS.

Referring to Table 2, when sps_inter_layer_mfm_enable_flag is 0, motion information mapping of the first layer may not be performed.

When sps_inter_layer_mfm_enable_flag is 1, motion information mapping of the first layer may be performed.

*According to another embodiment of the present invention, as shown in Table 3, according to the information (eg, sps_temporal_mvp_enabled_flag) indicating whether to use temporal motion vector prediction transmitted in the SPS, whether to perform motion information mapping of the corresponding layer Whether to transmit information (eg, sps_inter_layer_mfm_enable_flag) may be determined.

Referring to Table 3, when sps_temporal_mvp_enabled_flag indicating whether to use temporal motion vector prediction is 1, sps_inter_layer_mfm_enable_flag indicating whether motion information mapping of the first layer is to be performed may be transmitted.

When sps_temporal_mvp_enabled_flag is 1 and sps_inter_layer_mfm_enable_flag is 0, motion information of the first layer may not be used in the second layer. That is, the reconstructed image of the first layer may not be used as a temporal motion vector predictor (TMVP) of the second layer.

According to another embodiment of the present invention, whether to perform motion information mapping of a corresponding layer can be determined based on the sps_temporal_mvp_enabled_flag value without transmitting the sps_inter_layer_mfm_enable_flag as shown in Table 3 above. For example, when sps_temporal_mvp_enabled_flag is 1, it is considered that sps_inter_layer_mfm_enable_flag is 1, and motion information mapping of the corresponding layer may be performed. When sps_temporal_mvp_enabled_flag is 0, it is considered that sps_inter_layer_mfm_enable_flag is 0, and motion information mapping of the corresponding layer can be performed.

According to another embodiment of the present invention, as shown in Table 4, information related to motion information mapping of a corresponding layer may be signaled in an extension of the PPS.

Referring to Table 4, when pps_inter_layer_mfm_enable_flag is 0, motion information mapping of the first layer may not be performed.

When pps_inter_layer_mfm_enable_flag is 1, motion information mapping of the first layer may be performed.

According to another embodiment of the present invention, as shown in Table 5, information related to motion information mapping of a corresponding layer may be signaled in an extension of a slice segment header.

Referring to Table 5, when slice_inter_layer_mfm_enable_flag is 0, motion information mapping of the first layer may not be performed.

When slice_inter_layer_mfm_enable_flag is 1, motion information mapping of the first layer may be performed.

According to another embodiment of the present invention, as shown in Table 6, information related to motion information mapping of a corresponding layer may be signaled in an extension of a slice segment header.

Referring to Table 6, when slice_temporal_mvp_enabled_flag indicating whether to use temporal motion vector prediction is 1, slice_inter_layer_mfm_enable_flag indicating whether motion information mapping of the first layer is to be performed may be transmitted. slice_inter_layer_mfm_enable_flag is transmitted only in a layer other than the base layer (ie, an enhancement layer), and in the case of an enhancement layer, if a layer used for motion prediction does not exist, slice_inter_layer_mfm_enable_flag may not be transmitted. If the corresponding flag does not exist, the corresponding flag value can be inferred as 0.

When slice_temporal_mvp_enabled_flag is 1 and slice_inter_layer_mfm_enable_flag is 0, motion information of the first layer may not be used in the second layer. That is, the reconstructed image of the first layer may not be used as a temporal motion vector predictor (TMVP) of the second layer.

When slice_inter_layer_mfm_enable_flag is transmitted in a layer other than the basic layer (ie enhancement layer), slice_inter_layer_mfm_enable_flag may exist above slice_segement_header_extension_present_flag in a slice segment header of a layer other than the basic layer (ie enhancement layer) as shown in Table 7.

Referring to Table 7, when slice_temporal_mvp_enabled_flag indicating whether to use temporal motion vector prediction is 1, slice_inter_layer_mfm_enable_flag indicating whether motion information mapping of the first layer is to be performed may be transmitted.

When slice_temporal_mvp_enabled_flag is 1 and slice_inter_layer_mfm_enable_flag is 0, motion information of the first layer may not be used in the second layer. That is, the reconstructed image of the first layer may not be used as a temporal motion vector predictor (TMVP) of the second layer.

When slice_temporal_mvp_enabled_flag is 1 and slice_inter_layer_mfm_enable_flag is 1, motion information of the first layer can be used in the second layer. That is, after mapping the motion information of the first layer to the size of the second layer, the reconstructed image of the first layer can be used as a collocated picture (ColPic) for TMVP of the second layer.

When slice_inter_layer_mfm_enable_flag is 1, information on a collocated picture (ColPic, collocated picture) of the first layer for TMVP of the second layer may be signaled through additional syntax information as shown in Table 8. For example, information of a layer to be referred to may be known through collocated_ref_layer_idx, and motion information of a picture corresponding to the reference layer may be mapped to an image size of a second layer. In addition, in the case of a B slice, the direction of a reference picture list in which a picture of a reference layer that can be used as a corresponding picture is located can be determined through collocated_ref_layer_from_l0_flag.

Referring to Table 8, collocated_ref_layer_idx is an indicator indicating information on a reference layer used for motion prediction when the number of reference layers used for motion prediction is two or more. When the number of reference layers used for motion prediction is one, collocated_ref_layer_idx may be omitted. In this case, a constraint that all slices in a picture must have the same value may be applied.

collocated_ref_layer_from_l0_flag is transmitted in B slice. When the value of collocated_ref_layer_from_l0_flag is 1, a picture of a reference layer existing in LIST0 is set as a corresponding picture (ColPic). When the collocated_ref_layer_from_l0_flag value is 0, a picture of a reference layer existing in LIST1 is set as a corresponding picture (ColPic). Thereafter, in a process of obtaining a motion vector of the corresponding block (ColPb) in the corresponding picture (ColPic), the collocated_ref_layer_from_l0_flag value may be used with the same meaning instead of collocated_from_l0_flag. In this case, a constraint that all slices in a picture must have the same value may be applied. Alternatively, in the process of obtaining the motion vector of the corresponding block (ColPb) in the corresponding picture (ColPic), if the corresponding block has both LIST0 and LIST1 motion information, the same direction as the current encoding/decoding target picture is pointing. of motion information can be obtained.

When the base layer or slice_inter_layer_mfm_enable_flag is 0, the list direction and corresponding picture in the second layer may be informed using collocated_from_l0_flag and collocated_ref_idx without using motion information of the reference layer.

When slice_inter_layer_mfm_enable_flag is 1, information on a collocated picture (ColPic, collocated picture) of the first layer for TMVP of the second layer and reference picture list direction information can be signaled using additional syntax information as shown in Table 9. . For example, when slice_temporal_mvp_enabled_flag is 1, the direction of the reference picture list of the corresponding picture can be determined from the collocated_from_l0_flag value for the B slice, and the referenced layer can be known from the collocated_ref_layer_idx value.

Referring to Table 9, collocated_from_l0_flag indicates the direction of the reference picture list to which the corresponding picture belongs when slice_temporal_mvp_enabled_flag is 1 and it is a B slice. If collocated_from_l0_flag does not exist, collocated_from_l0_flag can be inferred to be 1. In this case, a constraint that all slices in a picture must have the same value may be applied.

collocated_ref_layer_idx is an indicator indicating information on a reference layer used for motion prediction when the number of reference layers used for motion prediction is two or more. When the number of reference layers used for motion prediction is one, collocated_ref_layer_idx may be omitted. In this case, a constraint that all slices in a picture must have the same value may be applied. A list direction for a corresponding picture can be determined from collocated_from_l0_flag transmitted from the upper level.

When the base layer or slice_inter_layer_mfm_enable_flag is 0, the list direction and corresponding picture in the second layer may be informed using collocated_from_l0_flag and collocated_ref_idx without using motion information of the reference layer.

When slice_inter_layer_mfm_enable_flag is 1, as shown in Table 10, information of a referenced layer can be known through collocated_ref_layer_idx, and motion information of a picture corresponding to the reference layer can be mapped to the image size of the second layer. At this time, according to the slice type, in the case of P slices, the picture of the reference layer existing in LIST0 is used as the corresponding picture, and in the case of B slices, a convention is made in one direction of LIST0 or LIST1, and the picture of the corresponding reference layer is used as the corresponding picture. As a result, the encoding/decoding device can use the same.

When the base layer or slice_inter_layer_mfm_enable_flag is 0, the list direction and corresponding picture in the second layer may be informed using collocated_from_l0_flag and collocated_ref_idx without using motion information of the reference layer.

According to another embodiment of the present invention, when a picture is divided into N independent slices, slice_inter_layer_mfm_enable_flag in each slice segment header must have the same value, and according to the value of slice_inter_layer_mfm_enable_flag in the first independent slice segment header The motion information mapping process may be performed once per picture.

In the above-described embodiments of the present invention, 'sps_inter_layer_mfm_enable_flag' transmitted at a higher level is used as information indicating whether motion information mapping is performed, but this information can be extended and used as information indicating whether inter-layer syntax is predicted. there is. For example, when sps_inter_layer_mfm_enable_flag is 1, inter-layer syntax (motion information) prediction may be performed as well as motion information mapping.

In the above-described embodiments of the present invention, a separate syntax indicating whether motion information mapping is performed is used, but motion information mapping may be performed according to a syntax information value related to performing inter-layer prediction without transmitting a separate syntax.

In the present invention, it is possible to determine whether motion information mapping is performed using syntax information indicating whether or not inter-layer prediction is performed signaled from a higher layer. Syntax information indicating whether or not the inter-layer prediction is performed may be signaled at a higher level.

Here, the upper level may be a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice segment header, and the like.

According to an embodiment of the present invention, as shown in Table 11, syntax indicating whether inter-layer prediction is performed in the VPS may be transmitted.

Referring to Table 11, when syntax indicating whether inter-layer prediction is performed is transmitted in the VPS, if no_inter_layer_pred_flag is 1, motion information mapping may not be performed, and if no_inter_layer_pred_flag is 0, motion information mapping may be performed.

In the present invention, it is possible to determine whether to perform motion information mapping using syntax information indicating whether to perform inter-layer syntax prediction signaled from a higher layer. Syntax information indicating whether the inter-layer syntax prediction is performed may be signaled at a higher level.

Here, the upper level may be a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice segment header, and the like.

According to an embodiment of the present invention, as shown in Table 12, syntax indicating whether inter-layer syntax prediction is performed in the PPS may be transmitted.

Referring to Table 12, when a syntax indicating whether to perform inter-layer syntax prediction is transmitted in the PPS, if no_inter_layer_syntax_pred_flag is 0, motion information mapping may be performed, and if no_inter_layer_syntax_pred_flag is 1, motion information mapping may not be performed. there is.

According to another embodiment of the present invention, as shown in Table 13, syntax indicating whether inter-layer syntax prediction is performed may be transmitted in the slice segment header.

Referring to Table 13, when syntax indicating whether to perform inter-layer syntax prediction is transmitted in the slice segment header, if no_inter_layer_syntax_pred_flag is 0, motion information mapping can be performed, and if no_inter_layer_syntax_pred_flag is 1, motion information mapping is not performed. may not be

Referring back to FIG. 4 , the encoding/decoding apparatus encodes/decodes a second layer image by adding information of the first layer to a reference picture list of the second layer (S420).

That is, the encoding/decoding apparatus transfers decoded sample values and motion information of the first layer mapped to the image size of the second layer through the above steps S400 to S410 into a reference picture list for the current encoding/decoding target image of the second layer. , and a reference picture list of a second layer image can be constructed using the added information of the first layer. Also, the encoding/decoding apparatus may perform inter prediction to generate a prediction signal for a second layer image based on the reference picture list.

When constructing the reference picture list of the second layer, the decoded image of the first layer may be added to the last position of the reference picture list for the current encoding/decoding target image of the second layer.

Alternatively, the decoded image of the first layer may be added to a specific position of the reference picture list for the current encoding/decoding target image of the second layer. In this case, the specific location may be informed at a higher level (eg, VPS, SPS, PPS, slice segment header, etc.), or the specific location may be designated according to a set rule without transmission of additional information.

Alternatively, the decoded image of the first layer may be added to the reference picture lists L0 and L1 for the current encoding/decoding target image of the second layer. In this case, the positions added to the reference picture lists L0 and L1 may be the same or different. For example, the decoded image of the first layer may be added to the first position in the reference picture list L0, and the decoded image of the first layer may be added to the last position in the reference picture list L1.

Alternatively, the decoded image of the first layer may be added to one of directions (L0 and L1) of the reference picture list for the current encoding/decoding target image of the second layer. For example, when the current encoding/decoding target image of the second layer is encoded/decoded into B slices, the decoded image of the first layer may be added only to the reference picture list L0 or the reference picture list L1.

In this case, when the decoded image of the first layer is added to the reference picture list of a specific direction, at any position in the reference picture list of the specific direction at a higher level (eg, VPS, SPS, PPS, slice segment header, etc.) It can notify the location to be added, or a specific location can be designated by a set rule without transmitting additional information.

Alternatively, the decoded image of the first layer may be added to different positions of the reference picture list according to the depth using hierarchical depth information of the prediction structure. In this case, information about the location may be informed at a higher level (eg, VPS, SPS, PPS, slice segment header, etc.). For example, the position of the decoded image of the first layer to be added to the reference picture list may be changed based on the temporal level (temporal_Id) value according to the hierarchical depth. For example, when the temporal level (temporal_Id) is 2 or more, the decoded image of the first layer may be added to the last position of the reference picture list, and when the temporal level (temporal_Id) is less than 2, the reference picture list A decoded image of the first layer may be added to the first position of . In this case, the reference time level value may be informed at a higher level (eg, VPS, SPS, PPS, slice segment header, etc.).

The above-described decoded image of the first layer may include not only decoded sample values of the first layer, but also decoded motion information of the first layer mapped to the image size of the second layer.

After the encoding/decoding apparatus adds the decoded image of the first layer to the reference picture list for the current encoding/decoding target image of the second layer in the above-described method, the second layer of the second layer is Prediction may be performed on an image to be currently encoded/decoded.

When prediction is performed using the decoded image of the first layer as a reference signal of the current encoding target image of the second layer (interlayer prediction), the encoding device predicts within the current encoding target image without a motion prediction process. A first layer sample value at the same location as a block may be used as a prediction signal. In this case, the encoding device may not transmit motion information (eg, a motion vector differential value, a motion prediction candidate flag, etc.) of the corresponding block.

When the image indicated by the reference picture index of the second layer is a decoded image of the first layer in the reference picture list, the decoding apparatus provides motion information (eg, motion information on a prediction block in the current decoding target image of the second layer). A first layer sample value at the same location as a prediction block in an image to be currently decoded may be used as a prediction signal without decoding a vector difference value, a motion prediction candidate flag, etc.).

When the image indicated by the reference picture index of the current decoding target block in the second layer is a decoded image of the first layer in the reference picture list (when interlayer prediction is performed), the decoding apparatus operates at a higher level (e.g., VPS , SPS, PPS, slice segment header, etc.), whether to decode motion information (eg, motion vector difference value, motion prediction candidate flag, etc.) for the current decoding target block of the second layer based on information transmitted in It is possible to perform prediction on the current decoding target block of the second layer by determining whether or not.

Table 14 shows an example of a syntax in which information (sps_inter_layer_mv_zero_flag) indicating whether an inter-layer motion vector has a value of (0, 0) is transmitted in the SPS.

Referring to Table 14, when sps_inter_layer_mv_zero_flag transmitted in the SPS has a value of 1 and the reference picture index of the current decoding target block indicates a decoded image of the first layer in the reference picture list, the decoding apparatus determines the motion vector difference A sample value of a decoded first layer image having the same location as that of the current decoding object block may be used as a prediction signal for the current decoding object block without decoding the value and the motion prediction candidate flag.

When sps_inter_layer_mv_zero_flag transmitted in SPS has a value of 0 and the reference picture index of the current decoding target block indicates a decoded image of the first layer in the reference picture list, the decoding apparatus determines the motion vector difference value and the motion prediction candidate flag. After decoding , a motion vector value may be obtained, and a sample value of the decoded first layer image located at the position of the obtained motion vector value may be used as a prediction signal for the current decoding target block.

Meanwhile, in the encoding apparatus, in order to use the mapped motion information of the first layer as a temporal motion vector candidate for the current block to be encoded in the second layer, the decoded image of the first layer is subjected to temporal motion vector prediction of the second layer. It can be used as a collocated picture for In this case, the encoding device may designate a decoded image of the first layer as a corresponding image through information (collocated_ref_idx) indicating a corresponding picture in the slice segment header.

When information (collocated_ref_idx) indicating a corresponding picture in the slice segment header transmitted from the encoder indicates a decoded image of the first layer, the decoding apparatus converts the mapped motion information of the first layer to a temporal motion vector candidate of the second layer. can be used as

When the mapped motion information of the first layer is used as a temporal motion vector candidate for the second layer, the encoding/decoding apparatus determines the mapped motion information of the first layer, that is, the reference picture POC, the reference picture list, and the picture of the reference picture. One or more types (short-term reference picture, long-term reference picture) information may be used, and a motion vector of a second layer may be scaled and used according to a temporal distance by using one or more mapped motion information of the first layer. there is.

When prediction is performed by adding only the reference sample values of the decoded first layer image to the reference picture list without permitting motion information mapping of the first layer at a higher level, the information indicating the corresponding picture (collocated_ref_idx) is the first layer A restriction may be placed on the standard document so that the decoded image of is not indicated as a corresponding picture.

As another example of using the mapped motion information of the first layer as motion information of the second layer, the mapped motion information of the first layer may be used as an additional candidate mode instead of a temporal motion vector candidate of the second layer. In this case, in order for the temporal motion vector candidate of the second layer to be derived from the reference picture of the second layer, the information (collocated_ref_idx) indicating the corresponding picture does not indicate the decoded image of the first layer as the corresponding picture. Constraints can be placed on standard documents. In addition, in order to use the motion information of the first layer mapped as the additional candidate mode, the encoding device additionally provides an index (eg, base_ref_idx) that can inform the position of the mapped image of the first layer in the reference picture list to the slice segment It can be transmitted in headers, etc. The position of the additional posterior mode in the reference picture list can be equally applied to the coding device and the decoding device.

The method according to the present invention described above may be produced as a program to be executed on a computer and stored in a computer-readable recording medium. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, and magnetic tape. , floppy disks, optical data storage devices, and the like, and also includes those implemented in the form of carrier waves (for example, transmission through the Internet).

The computer-readable recording medium is distributed to computer systems connected through a network, so that computer-readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the method can be easily inferred by programmers in the technical field to which the present invention belongs.

In the foregoing embodiments, the methods are described on the basis of a flowchart as a series of steps or blocks, but the present invention is not limited to the order of steps, and some steps may occur in a different order or concurrently with other steps as described above. can In addition, those skilled in the art will understand that the steps shown in the flow chart are not exclusive, that other steps may be included, or that one or more steps of the flow chart may be deleted without affecting the scope of the present invention. You will understand.

The above description is merely an example of the technical idea of the present invention, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed according to the claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

Claims (5)

In the image decoding method for performing inter-layer prediction,
determining whether inter-layer prediction is performed on a current decoding target block;
decoding a picture of a first layer;
When the picture of the first layer is a picture of a layer referred to by a picture of a second layer including the current decoding target block, adding the picture of the first layer to a reference picture list for the picture of the second layer step; and
Performing prediction on the current decoding target block based on the reference picture list;
The first layer is a base layer, the second layer is an enhancement layer,
When the picture of the first layer and the picture of the second layer have different sizes, the collocated-picture for prediction of the current decoding target block is not determined as the picture of the first layer, and the collocated-picture for prediction of the current decoding target block is not determined. The video decoding method, characterized in that the information indicating the picture does not indicate the picture of the first layer. According to claim 1,
The picture of the first layer is added to the reference picture list when the picture of the first layer and the picture of the second layer have the same Picture Order Count (POC). According to claim 1,
Whether the picture of the first layer is a picture of a layer referred to by a picture of the second layer is determined based on a layer ID of the picture of the second layer. In the video encoding method for performing inter-layer prediction,
determining that inter-layer prediction is to be performed on a current encoding target block;
Deriving a picture of a first layer; and
Adding the picture of the first layer to a reference picture list for a picture of the second layer when the picture of the first layer is a picture of a layer referenced by a picture of a second layer including the current encoding target block step; and
Performing prediction on the current encoding target block based on the reference picture list;
The first layer is a base layer, the second layer is an enhancement layer,
When the picture of the first layer and the picture of the second layer have different sizes, the collocated-picture for prediction of the current encoding target block is not determined as the picture of the first layer, and the collocated-picture for prediction of the current encoding target block is determined. The video encoding method, characterized in that the information indicating the picture does not indicate the picture of the first layer. In a recording medium storing a bitstream decoded by an image decoding method,
The video decoding method,
determining whether inter-layer prediction is performed on a current decoding target block;
decoding a picture of a first layer;
When the picture of the first layer is a picture of a layer referred to by a picture of a second layer including the current decoding target block, adding the picture of the first layer to a reference picture list for the picture of the second layer step; and
Performing prediction on the current decoding target block based on the reference picture list;
The first layer is a base layer, the second layer is an enhancement layer,
When the picture of the first layer and the picture of the second layer have different sizes, the collocated-picture for prediction of the current decoding target block is not determined as the picture of the first layer, and the collocated-picture for prediction of the current decoding target block is not determined. A recording medium storing a bitstream generated by a video decoding method, characterized in that the information indicating a picture does not indicate a picture of the first layer.

Images