KR102150671B1 - Method and apparatus for image encoding/decoding - Google Patents

Abstract

Disclosed are an image encoding/decoding method and apparatus supporting a plurality of layers. The video decoding method includes analyzing a layer dependency on a current layer based on a video parameter set (VPS) extension, analyzing a layer dependency on a current slice based on information encoded for each slice, and a layer dependency on the current layer. And constructing a reference picture list for the current layer based on analysis of at least one of a layer dependency and a layer dependency on the current slice.

Description

Video encoding/decoding method and apparatus {METHOD AND APPARATUS FOR IMAGE ENCODING/DECODING}

The present invention relates to video encoding and decoding, and more particularly, to video encoding and decoding based on video coding.

Recently, as a multimedia environment has been established, various terminals and networks are being used, and user needs are diversifying accordingly.

For example, as the performance and computing capability of the terminal are diversified, the supported performance is also diversifying for each device. In addition, networks to which information is transmitted are diversifying not only by the external structure of wired/wireless networks, but also by functions such as the type of information to be transmitted, the amount and speed of information. Users select terminals and networks to be used according to desired functions, and the spectrum of terminals and networks provided by companies to users is also diversifying.

In this regard, as broadcasts having a high definition (HD) resolution are recently expanded and serviced not only in Korea but also around the world, many users are getting used to high-definition and high-definition images. Accordingly, many video service related organizations are making great efforts to develop next-generation video devices.

In addition, as interest in UHD (Ultra High Definition) having a resolution of 4 or more times higher than HDTV in addition to HDTV is increasing, the demand for a technology for compressing and processing images of higher resolution and high quality is increasing.

In order to compress and process an image, an inter prediction technology that temporally predicts a pixel value included in the current picture from a previous and/or subsequent picture, and another pixel included in the current picture using pixel information in the current picture. An intra prediction technique for predicting a value, an entropy encoding technique in which a short code is assigned to a symbol with a high frequency of appearance and a long code is assigned to a symbol with a low frequency of appearance may be used.

As described above, when considering the requirements of each terminal and network with different supported functions and diversified users, the quality, size, frame, etc. of supported images need to be diversified accordingly.

As described above, due to heterogeneous communication networks and various functions and types of terminals, scalability of variously supporting image quality, resolution, size, frame rate, etc. has become an important function of a video format.

Accordingly, it is necessary to provide a scalability function to enable efficient video encoding and decoding in terms of time, space, quality, etc. in order to provide services requested by users in various environments based on a highly efficient video encoding method.

The present invention provides an image encoding/decoding method and apparatus capable of improving image encoding/decoding efficiency.

The present invention provides a method and apparatus for performing inter-layer switching in scalable video coding capable of improving encoding/decoding efficiency.

The present invention provides a method and apparatus for encoding/decoding scalable reference picture set information capable of improving image encoding/decoding efficiency.

The present invention provides a method and apparatus for managing a reference image in scalable video coding capable of improving image encoding/decoding efficiency.

According to an aspect of the present invention, an image decoding method supporting a plurality of layers is provided. The video decoding method includes analyzing a layer dependency on a current layer based on a video parameter set (VPS) extension, analyzing a layer dependency on a current slice based on information encoded for each slice, and a layer dependency on the current layer. And constructing a reference picture list for the current layer based on analysis of at least one of a layer dependency and a layer dependency on the current slice.

According to another aspect of the present invention, an image encoding method supporting a plurality of layers is provided. The video encoding method includes determining a layer dependency on a current layer and encoding using a video parameter set (VPS) extension, determining a layer dependency on a current slice and encoding for each slice, and layer dependency on the current layer And constructing a reference picture list for the current layer based on at least one of layer dependencies for the current slice.

According to another aspect of the present invention, an image decoding apparatus supporting a plurality of layers is provided. The video decoding apparatus analyzes a layer dependency on a current layer based on a video parameter set (VPS) extension, and a decoder that analyzes a layer dependency on a current slice based on information encoded for each slice, and a decoder for the current layer. And a prediction unit configured to construct a reference picture list for the current layer based on analysis of at least one of a layer dependency and a layer dependency on the current slice, and perform prediction based on the reference picture list.

The present invention can provide a reference image management method in a hierarchical coding scheme capable of providing spatial, temporal, quality, and view scalability. When inter-layer prediction including at least one reference layer is performed, reference pictures including decoded pictures of the reference layer can be efficiently managed. In addition, according to the present invention, efficient reference layer management is possible, and a reference layer can be selected so as to maximize encoding efficiency even in the same layer.

1 is a block diagram showing a configuration according to an embodiment of an image encoding apparatus to which the present invention is applied.
2 is a block diagram showing a configuration according to an embodiment of an image decoding apparatus to which the present invention is applied.
3 is a conceptual diagram schematically showing an example of a scalable video coding structure using multiple layers to which the present invention can be applied.
4 is a diagram showing the configuration of a reference picture list.
5 is a flowchart schematically illustrating a method of managing a reference picture in scalable video coding according to an embodiment of the present invention.
6 is a diagram for describing a method of configuring a scalable reference layer set (SRLS, or layer dependency set) according to the present invention.
7 is a diagram illustrating a method of constructing a reference picture list (reference picture list) according to the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In describing the embodiments of the present specification, if it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present specification, the corresponding description may be omitted.

In the present specification, when a component is referred to as being “connected” or “connected” to another component, it may mean that it is directly connected to or connected to the other component, or another component in the middle. It can also mean that an element is present. In addition, the description of “including” a specific configuration in the present specification does not exclude configurations other than the corresponding configuration, and means that additional configurations may be included in the scope of the implementation of the present invention or the technical idea of the present invention.

Terms such as first and second may be used to describe various configurations, but the configurations are not limited by the terms. These terms are used for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first configuration may be referred to as a second configuration, and similarly, a second configuration may be referred to as a first configuration.

In addition, the components shown in the embodiments of the present invention are shown independently to represent different characteristic functions, and it does not mean that each component is formed of separate hardware or a single software component. That is, each constituent unit is arranged and included as respective constituent units for convenience of description, and at least two constituent units of each constituent unit constitute one constituent unit, or one constituent unit may be divided into a plurality of constituent units to perform a function. Integrated embodiments and separate embodiments of each component are also included in the scope of the present invention unless departing from the essence of the present invention.

In addition, some of the components are not essential components that perform essential functions in the present invention, but may be optional components only for improving performance. The present invention can be implemented by including only the components essential to implement the essence of the present invention excluding components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement Also included in the scope of the present invention.

1 is a block diagram showing a configuration according to an embodiment of an image encoding apparatus to which the present invention is applied.

A scalable video encoding/decoding method or apparatus may be implemented by a general video encoding/decoding method or apparatus that does not provide scalability, and the block diagram of FIG. 1 is Shows an embodiment of an image encoding apparatus that can be the basis of a scalable video encoding apparatus.

Referring to FIG. 1, the image encoding apparatus 100 includes a motion prediction unit 111, a motion compensation unit 112, an intra prediction unit 120, a switch 115, a subtractor 125, a transform unit 130, A quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190 are included.

The image encoding apparatus 100 may encode an input image in an intra mode or an inter mode and output a bitstream. In the intra mode, the switch 115 may be switched to intra, and in the inter mode, the switch 115 may be switched to inter. Intra prediction means intra prediction, and inter prediction means inter prediction. The image encoding apparatus 100 may generate a prediction block for an input block of an input image, and then encode a residual between the input block and the prediction block. In this case, the input image may mean an original picture.

In the case of the intra mode, the intra prediction unit 120 may generate a prediction block by performing spatial prediction using pixel values of an already coded/decoded block surrounding the current block.

In the case of the inter mode, the motion prediction unit 111 may obtain a motion vector by finding a region that best matches the input block in the reference image stored in the reference picture buffer 190 during the motion prediction process. The motion compensation unit 112 may generate a prediction block by performing motion compensation using a motion vector. Here, the motion vector is a 2D vector used for inter prediction, and may represent an offset between a current encoding/decoding target image and a reference image.

The subtractor 125 may generate a residual block based on a difference between the input block and the generated prediction block.

The transform unit 130 may perform transform on the residual block and output a transform coefficient. Here, the transform coefficient may mean a coefficient value generated by transforming the residual block and/or the residual signal. Hereinafter, in the present specification, a quantized transform coefficient level generated by applying quantization to a transform coefficient may also be referred to as a transform coefficient.

The quantization unit 140 may quantize the input transform coefficient according to a quantization parameter (or quantization parameter) and output a quantized coefficient. The quantized coefficient may also be referred to as a quantized transform coefficient level. In this case, the quantization unit 140 may quantize the input transform coefficient using a quantization matrix.

The entropy encoder 150 may perform entropy encoding based on values calculated by the quantization unit 140 or an encoding parameter value calculated during an encoding process to output a bitstream. When entropy coding is applied, a small number of bits are allocated to a symbol having a high probability of occurrence, and a large number of bits are allocated to a symbol having a low probability of occurrence, and the symbol is expressed. The size of the column can be reduced. Therefore, compression performance of image encoding may be improved through entropy encoding. The entropy encoding unit 150 may use an encoding method such as Exponential-Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) for entropy encoding.

Since the image encoding apparatus 100 according to the embodiment of FIG. 1 performs inter prediction encoding, that is, inter prediction encoding, the currently encoded image needs to be decoded and stored in order to be used as a reference image. Accordingly, the quantized coefficients are inverse quantized by the inverse quantization unit 160 and inversely transformed by the inverse transform unit 170. The inverse quantized and inverse transformed coefficients are added to the prediction block through an adder 175, and a reconstructed block is generated.

The reconstructed block passes through the filter unit 180, and the filter unit 180 applies at least one of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to a reconstructed block or a reconstructed picture. can do. The filter unit 180 may also be referred to as an adaptive in-loop filter. The deblocking filter can remove block distortion occurring at the boundary between blocks. SAO may add an appropriate offset value to a pixel value to compensate for a coding error. The ALF may perform filtering based on a value obtained by comparing the reconstructed image and the original image. The reconstructed block that has passed through the filter unit 180 may be stored in the reference picture buffer 190.

2 is a block diagram showing a configuration according to an embodiment of an image decoding apparatus to which the present invention is applied.

As described above in FIG. 1, the scalable video encoding/decoding method or apparatus may be implemented by an extension of a general video encoding/decoding method or apparatus that does not provide scalability, and the block diagram of FIG. 2 is Shows an embodiment of an image decoding apparatus that can be the basis of a video decoding apparatus.

Referring to FIG. 2, the image decoding apparatus 200 includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra prediction unit 240, a motion compensation unit 250, and an adder 255. ), a filter unit 260, and a reference picture buffer 270.

The image decoding apparatus 200 may receive a bitstream output from an encoder, perform decoding in an intra mode or an inter mode, and output a reconstructed image, that is, a reconstructed image. In the intra mode, the switch may be switched to intra, and in the inter mode, the switch may be switched to inter.

The image decoding apparatus 200 may obtain a reconstructed residual block from an input bitstream, generate a prediction block, and then add the reconstructed residual block and the prediction block to generate a reconstructed block, that is, a reconstructed block. .

The entropy decoder 210 may entropy-decode the input bitstream according to a probability distribution to generate symbols including symbols in the form of quantized coefficients.

When the entropy decoding method is applied, a small number of bits are allocated to a symbol having a high probability of occurrence and a large number of bits are allocated to a symbol having a low probability of occurrence, thereby expressing the symbol, so that the size of the bit string for each symbol is reduced. Can be reduced.

The quantized coefficients are inverse quantized by the inverse quantization unit 220 and inversely transformed by the inverse transform unit 230, and as a result of the inverse quantization/inverse transformation of the quantized coefficients, a reconstructed residual block may be generated. In this case, the inverse quantization unit 220 may apply a quantization matrix to the quantized coefficients.

In the case of the intra mode, the intra prediction unit 240 may generate a prediction block by performing spatial prediction using pixel values of an already decoded block around the current block. In the inter mode, the motion compensation unit 250 may generate a prediction block by performing motion compensation using a motion vector and a reference image stored in the reference picture buffer 270.

The residual block and the prediction block are added through an adder 255, and the added block may pass through the filter unit 260. The filter unit 260 may apply at least one of a deblocking filter, SAO, and ALF to a reconstructed block or a reconstructed picture. The filter unit 260 may output a reconstructed image, that is, a reconstructed image. The reconstructed image may be stored in the reference picture buffer 270 and used for inter prediction.

3 is a conceptual diagram schematically showing an example of a scalable video coding structure using multiple layers to which the present invention can be applied. In FIG. 3, a group of pictures (GOP) represents a picture group, that is, a group of pictures.

A transmission medium is required to transmit image data, and its performance varies for each transmission medium according to various network environments. A scalable video coding method may be provided for application to such various transmission media or network environments.

A video coding method that supports scalability (hereinafter referred to as'scalable coding' or'scalable video coding') removes redundancy between layers by using texture information, motion information, and residual signals between layers. This is a coding method that improves encoding and decoding performance. The scalable video coding method includes various scalers in terms of spatial, temporal, image quality (or quality), and view according to surrounding conditions such as transmission bit rate, transmission error rate, and system resources. Can provide utility.

Scalable video coding may be performed using a multiple layers structure so as to provide a bitstream applicable to various network situations. For example, the scalable video coding structure may include a base layer that compresses and processes image data using a general image decoding method, and compresses image data using both decoding information of the base layer and a general image decoding method. It may include an enhancement layer to process.

Here, the layer is based on spatial (for example, image size), time (temporal, for example, decoding order, image output order, frame rate), quality, complexity, view, etc. It refers to a set of divided images and bitstreams.

The base layer may be referred to as a base layer or a lower layer. The enhancement layer may also be referred to as an enhancement layer or a higher layer. In this case, the lower layer may mean a layer supporting lower scalability than a specific layer, and the upper layer may mean a layer supporting higher scalability than a specific layer. A layer that a specific layer refers to during encoding or decoding may be referred to as a reference layer (or reference layer).

Referring to FIG. 3, for example, the base layer may be defined as SD (standard definition), 15Hz frame rate, and 1Mbps bit rate, and the first enhancement layer is HD (high definition), 30Hz frame rate, and 3.9Mbps bit rate. The second enhancement layer may be defined as 4K-UHD (ultra high definition), a frame rate of 60 Hz, and a bit rate of 27.2 Mbps.

The format, frame rate, bit rate, etc. are one embodiment and may be differently determined as necessary. Also, the number of layers to be used is not limited to this embodiment and may be determined differently according to circumstances. For example, if the transmission bandwidth is 4 Mbps, the frame rate of the first enhancement layer HD may be reduced to transmit at 15 Hz or less.

The scalable video coding method may provide temporal, spatial, quality, and viewpoint scalability by the method described above in the embodiment of FIG. 3.

In the present specification, scalable video coding has the same meaning as scalable video encoding from an encoding perspective and scalable video decoding from a decoding perspective.

On the other hand, in the scalable video coding structure, since there is a strong correlation between layers, if prediction is performed using this correlation, redundant elements of data can be removed and encoding performance of an image can be improved. Therefore, when predicting a picture (video) of the current layer (upper layer) to be encoded or decoded, inter-layer prediction using information of other layers as well as inter prediction or intra prediction using information of the current layer Alternatively, inter-layer prediction) may be performed.

Since inter-layer prediction, that is, inter-layer prediction, uses information of other layers, information of other layers used for inter-layer prediction is required. That is, information on another layer (reference layer) used for inter-layer prediction of the current layer, information on a reference picture referenced in the reference layer, and the like are required. When using such inter-layer prediction, management of a reference layer and a reference picture referred to by the current layer is required.

By transmitting information related to the management of the reference picture used for inter-layer prediction (hereinafter, the management of the inter-layer reference picture) from the encoder, it indicates how many layers each layer refers to using the inter-layer dependency set here. Information and information indicating which layer is referred to for each layer can be transmitted. The portion for transmitting the management information for the reference picture may be transmitted in various forms, and one example of them may be a video parameter set (hereinafter, referred to as VPS) extension. For example, the VPS extension may include information indicating how many layers each layer refers to, and information indicating which layer each layer refers to.

For example, the decoder may interpret num_direct_ref_layers[layerID] indicating the number of reference layers referenced by each layer from the VPS extension, and interpret ref_layer_id[i] indicating the reference layer referenced for each layer. In this case, in ref_layer_id[i], i may have a range from '0' to a value specified by num_direct_ref_layers indicating how many layers each layer refers to. In other words, it is possible to know how many layers the current layer refers to from the VPS extension, and also to know the reference layer of the current layer.

When analysis of a layer to be referred to by each layer is completed, a reference picture list (or reference picture list) may be constructed. In this case, not only a reference picture used for inter prediction but also a reference picture used for inter-layer prediction may be added to the reference picture list.

4 is a diagram showing the configuration of a reference picture list.

Referring to FIG. 4, the reference picture list includes a long-term reference picture set referenced by a current picture (RefPicSetLtCurr), a long-term reference picture set not referenced by a current picture (RefPicSetLtFoll), and a forward short-term reference picture set referenced by the current picture. (RefPicSetStCurrBefore), a reverse short-term reference picture set referenced by the current picture (RefPicSetStCurrAfter), a short-term reference picture set not referenced by the current picture (RefPicSetStFoll), and a reference picture set referenced for inter-layer prediction by the current picture (RefPicSetIvCurr ) Can be composed of.

The reference picture set (RefPicSetIvCurr) referenced for inter-layer prediction may include as many reference layers as the number of num_direct_ref_layers signaled in VPS extension.

The reference picture set (RefPicSetIvCurr) referenced for inter-layer prediction includes a picture having the same layer identifier (layer_id) as ref_layer_id[i] signaled by the VPS extension and having the same POC (Picture Order Count) as the current picture of the current layer. I can. The POC may be a value capable of identifying pictures belonging to the same layer, and may be a value indicating an output order.

All pictures constituting a reference picture set (RefPicSetIvCurr) that are referred for inter-layer prediction are indicated as “used for long-term reference”.

Hereinafter, in the present invention, a method of efficiently managing a reference picture when inter-layer prediction is performed by including at least one reference layer in a scalable video coding structure will be described.

5 is a flowchart schematically illustrating a method of managing a reference picture in scalable video coding according to an embodiment of the present invention. The method of FIG. 5 may be performed by the encoding apparatus of FIG. 1 or the decoding apparatus of FIG. 2 described above.

Referring to FIG. 5, an encoder or decoder analyzes a layer dependency at VPS extension (S500).

Layer dependency analysis, as described above, means to identify inter-layer dependencies in order to encode or decode the current layer. When encoding or decoding the current layer, a reference relationship between layers is identified to use an image of another layer as a reference image. Say what to do.

From the encoder's point of view, when encoding a layer dependency using VPS extension, the encoder can (1) use the existing layer dependency expression method, and (2) the existing layer dependency expression method and each layer can refer to it. One of the methods of expressing the layer dependency using a scalable reference layer set (SRLS) for a layer may be used.

Here, the method of expressing layer dependency using SRLS is to specify the number of reference layers (num_direct_ref_layers) and reference layers (ref_layer_id) that the current layer can refer to as a set (SRLS) in advance, and then use the desired set. Tell the way.

When encoding is performed using an existing layer dependency expression method, the encoder may encode the layer dependency according to the conventional method as shown in FIG. 4 described above.

In the case of encoding using the existing layer dependency expression method and the layer dependency expression method using SRLS, the encoder first encodes a flag indicating which method is used (e.g., vps_srls_present_flag), and then encodes the layer dependency according to the flag. I can.

For example, when using the existing layer dependency expression method, vps_srls_present_flag may be encoded as 0, and layer dependency may be encoded through the existing syntax. When using the layer dependency expression method using SRLS, vps_srls_present_flag is encoded as 1, a syntax indicating how many layer dependency sets (SRLS) to use (e.g., num_scalable_ref_layer_sets) and information on each layer dependency set (scalable_ref_layer_set()) Can be encoded.

From the perspective of the decoder, when encoding is performed using the existing layer dependency expression method in the encoder, the decoder can decode the layer dependency according to the conventional method without changing the syntax.

When encoding is performed using either the existing layer dependency expression method or the layer dependency expression method using SRLS in the encoder, the decoder first decodes a flag (e.g., vps_srls_present_flag) indicating by which method the layer dependency is encoded. Then, layer dependency information may be decoded according to the flag.

Table 1 is an example of syntax showing a method of interpreting layer dependency in VPS extension.

Referring to Table 1, vps_srls_present_flag is a flag indicating whether a layer dependency is encoded using which of an existing layer dependency expression method and a layer dependency expression method using SRLS. For example, when vps_srls_present_flag is 0, it is indicated that it has been encoded by the existing layer dependency expression method, and when vps_srls_present_flag is 1, it is possible to indicate that it is encoded by the layer dependency expression method using SRLS.

num_scalable_ref_layer_sets represents the number of layer dependency sets (SRLS).

scalable_ref_layer_set() represents the configuration of each layer dependency set.

The decoder may decode vps_srls_present_flag in order to analyze the layer dependency in the VPS.

When vps_srls_present_flag is 0, the decoder may decode the syntax using an existing layer dependency expression method.

When vps_srls_present_flag is 1, the decoder can decode the syntax using a layer dependency expression method using SRLS. At this time, the decoder may determine the number of layer dependency sets SRLS by decoding num_scalable_ref_layer_sets, and configure each layer dependency set SRLS by decoding scalable_ref_layer_set().

A method of configuring a layer dependency set (SRLS) according to the present invention will be described in detail with reference to FIG. 6.

The encoder or decoder analyzes the layer dependency in the slice (S510).

In the case of analyzing layer dependencies through slices, the encoder or decoder uses (1) a method of using/changing the layer dependency analyzed by the existing method in the VPS extension in the current slice, and (2) a layer expressed as SRLS in the VPS extension. You can use one of the methods to use/change in the current slice using dependencies.

When the VPS extension uses/changes the layer dependency analyzed by the conventional method in the current slice, the encoder distinguishes whether to not use the layer dependency for the current slice or to establish a new layer dependency for the current slice. Information may be encoded for each slice using a flag (eg, slice_srls_present_flag).

If the method of setting a new layer dependency for the current slice is used, the encoder encodes the number of layers that the current slice can refer to as syntax information (e.g., num_scalable_ref_layer). The number of layers to be referred to (layer_id or identifiable information) may be encoded and transmitted with syntax information (eg, scalable_ref_layer[i]). The new layer dependency can be set within the range of inter-layer dependencies set in the VPS extension.

When the VPS extension uses/changes the layer dependency interpreted by the conventional method in the current slice, the decoder does not use the layer dependency for the current slice or a flag indicating whether to set a new layer dependency for the current slice ( For example, slice_srls_present_flag) may be first decoded, and layer dependency information for a current slice may be decoded according to the flag.

Table 2 is an example of syntax showing a method of using/changing a layer dependency interpreted by a conventional method in a VPS extension in a current slice.

Referring to Table 2, slice_srls_present_flag is a flag indicating whether a layer dependency is not used for a current slice or a new layer dependency is set for a current slice.

num_scalable_ref_layer represents the number of layers to be referenced by the current slice.

scalable_ref_layer[i] represents the layer_id of the layer to be referred to or information that can identify the layer to be referred to.

Table 3 is an example of the syntax of Table 2. Referring to Table 3, slice_srls_present_flag is a flag indicating whether to not use a layer dependency for the current slice or to set a new layer dependency for the current slice. For example, if slice_srls_present_flag is 1, it indicates that a new layer dependency is set in the current slice, and if slice_srls_present_flag is 0, it indicates that no layer dependency is used for the current slice. If slice_srls_present_flag is 0, num_scalable_ref_layer is set to 0 and the current slice does not use an inter layer reference.

slice_srls_present_flag applies to all cases where the current layer to be encoded is not the base layer, the inter-layer dependency set in the VPS extension is not always used, and the number of layers that the current layer can reference in the inter-layer dependency set in the VPS extension is 1 or more. If satisfied, it can be interpreted.

num_scalable_ref_layer represents the number of layers to be referenced by the current slice. num_scalable_ref_layer can be interpreted when slice_srls_present_flag is 1 and the number of layers that can be referenced in inter-layer dependency set in VPS extension is 2 or more. num_scalable_ref_layer may be greater than 1 and may have a value smaller than the number of layers that can be referenced by the current layer in the inter-layer dependency set in the VPS extension. If the number of layers that can be referenced by the current layer in the inter-layer dependency set in the VPS extension is 1, num_scalable_ref_layer is set to 1 even without interpretation.

scalable_ref_layer[i] represents the layer_id of the layer to be referred to or information that can identify the layer to be referred to. If the number of layers referenced by the current layer and num_scalable_ref_layer are the same in the inter-layer dependency set in the VPS extension, scalable_ref_layer[i] can identically identify the reference layer specified in the inter-layer dependency set in the VPS extension without interpretation. Set with information.

The decoder may decode slice_srls_present_flag in order to analyze layer dependency in a slice. If slice_srls_present_flag is 1, a new layer dependency is set in the current slice, and if slice_srls_present_flag is 0, the layer dependency is not used.

When using a new layer dependency for the current slice, the decoder finds the number of layers to be referenced by the current slice by decoding the num_scalable_ref_layer, and decodes scalable_ref_layer[i] as many as the number of layers to be referenced by the current slice, The layer_id or information that can identify the layer to be referred to can be identified.

On the other hand, when using/changing in the current slice using the layer dependency expressed as SRLS in the VPS extension, the encoder determines whether to use the layer dependency set in the VPS extension or a new layer dependency in the current slice, and determines the determined information. May be encoded for each slice using a flag (eg, scalable_ref_layer_set_vps_flag).

Here, the encoder may discriminate whether encoding is performed using an existing layer dependency representation method or a layer dependency representation method using SRLS according to flag information (eg, vps_srls_present_flag) set in the VPS extension.

If encoding is performed using the layer dependency expression method using SRLS, the encoder encodes a new layer dependency in the current slice according to the scalable_ref_layer_set_vps_flag, or syntax information indicating which layer dependency set to use in the layer dependency set set in the VPS extension (e.g. For example, scalable_ref_layer_set_Idx) can be encoded.

If the method used in the VPS extension is coded using the existing layer dependency expression method, the layer dependency to be used in the current slice according to scalable_ref_layer_set_vps_flag is used/changed in the current slice using the layer dependency analyzed by the conventional method in the VPS extension described above. Can be set together.

Table 4 is an example of syntax showing how to use/change in the current slice by using the layer dependency expressed by SRLS in the above-described VPS extension.

The encoder or decoder constructs a reference picture list (S520).

The inter-layer reference picture list may be managed by VPS extension or inter-layer dependency set in a slice header. In this case, the layer dependency in the VPS extension and the slice header may be analyzed by the method described in steps S500 to S510 described above.

After the encoder or decoder analyzes the layer to be referred to by each layer, a layer dependency set (ScalableRefLayerSet, hereinafter referred to as SRLS) may be added when constructing a reference picture list. In SRLS, as many layers as the number of reference layers interpreted in the VPS extension or slice header can be designated. In addition, the SRLS can designate a video having the same POC as the current picture in the reference layer with the dependency interpreted in the VPS extension or the slice header. All pictures in SRLS are marked as “used for long-term reference”.

A method of constructing a reference picture list including SRLS according to the present invention will be described in detail with reference to FIG. 7.

6 is a diagram for describing a method of configuring a scalable reference layer set (SRLS, or layer dependency set) according to the present invention.

Referring to FIG. 6, M scalable reference layer sets may exist, and each scalable reference layer set may include a plurality of layer IDs.

The scalable reference layer set 1 may be composed of A layer IDs, the scalable reference layer set 2 may be composed of B layer IDs, and the scalable reference layer set M may be composed of K layer IDs.

The layer ID constituting the scalable reference layer set may be specified as a difference value between the current layer ID and the reference layer ID.

The encoder may encode the above-described scalable_ref_layer_set() in order to configure the scalable reference layer set. scalable_ref_layer_set() is syntax information indicating the number of reference layers (e.g., num_ref_layer), syntax information indicating the sign of the difference value of layer_id between the current layer and the reference layer by the number of reference layers (e.g., delta_layer_id_sign), the Syntax information indicating the absolute value of the difference value (eg, abs_delta_layer_id[i]) may be sequentially encoded and transmitted. At this time, syntax information (eg, num_ref_layer, delta_layer_id_sign, abs_delta_layer_id[i]) encoded by scalable_ref_layer_set() may be specified and encoded for each scalable reference layer set.

Table 5 shows the syntax table of scalable_ref_layer_set() for configuring a scalable reference layer set.

Referring to Table 5, num_scalable_ref_layer_set represents the number of scalable reference layer sets.

A value obtained by adding 1 to delta_srls_idx_mimus1 is a value that specifies the scalable reference layer set, and represents a difference value from the previous scalable reference layer set.

num_ref_layer represents the number of reference rays.

delta_layer_id_sign represents the sign of the difference between the current layer and the reference layer.

abs_delta_layer_id[i] represents the absolute value of the difference between the current layer and the reference layer.

The decoder may determine the number of reference layers constituting the reference layer set through num_ref_layer, and determine the difference value of the layer_id between the current layer and the reference layer through delta_layer_id_sign and abs_delta_layer_id[i] signaled by the number of reference rays.

7 is a diagram illustrating a method of constructing a reference picture list (reference picture list) according to the present invention.

Referring to FIG. 7, before decoding a current image, five lists of POC values and one list used for inter-layer prediction may be configured in order to derive a reference picture set included in a current layer.

Five lists of POC values (Picture order count list) may include PocLtCurr, PocLtFoll, PocStCurrBefore, PocStCurrAfter, and PocStFoll. Each list includes a number of components (ie, POC values) specified by NumPocStCurrBefore, NumPocStCurrAfter, NumPocStFoll, NumPocLtCurr, and NumPocLtFoll, respectively.

PocLtCurr is a list including POCs of long-term reference pictures that are used by the current picture and have a larger POC than that of the current picture, and PocLtFoll includes POCs of long-term reference pictures that are not used by the current picture. It is a list to do. PocLtCurr and PocLtFoll are used to construct a long-term reference picture set.

PocStCurrBefore is a list including the POC of a short-term reference picture that is used by the current picture and has a POC smaller than that of the current picture. PocStCurrAfter is a list including the POC of a short-term reference picture that is used by the current picture and has a POC greater than that of the current picture. PocStFoll is a list containing POCs of short-term reference pictures that are not used by the current picture. PocStCurrBefore, PocStCurrAfter and PocStFoll are used to construct a short-term reference picture set.

One list used for inter-layer prediction may include reference layer candidates for supporting scalability. The reference layer candidate may be composed of a layer ID (LayerIDScalableCurr) for a reference layer candidate for inter-layer prediction. LayerIDScalableCurr is used to construct a scalable reference layer set.

The encoder or decoder can derive 5 reference picture sets from the 5 POC lists by checking the DPB (Decoded Picture Buffer) storing the decoded picture for the current layer, and check the DPB of the other layer to determine the reference layer candidate. A reference layer set (ScalableRefLayerSet) to be used for inter-layer prediction can be configured from (LayerIDScalableCurr).

The scalable reference layer set (ScalableRefLayerSet) may be configured as a reference layer corresponding to the number of reference layers signaled in the VPS extension or slice header.

In the scalable reference layer set (ScalableRefLayerSet), an image having the same POC as the current picture in a reference layer having a dependency signaled by VPS extension or a slice header may be designated.

All images constituting the scalable reference layer set (ScalableRefLayerSet) are indicated as used for long-term reference.

The encoder or decoder may derive a reference picture list based on a reference picture set and a scalable reference layer set, and perform inter prediction or inter-layer prediction using the reference picture list.

According to the present invention, by adding a scalable reference layer set (SRLS) to an existing reference picture set (RPS) to construct a reference image in another layer, it is correlated with the type of scalability. Without it, reference images can be integrated and managed. In addition, DPB management performed in the current layer does not affect DPBs in other layers.

In the above-described embodiments, the methods are described on the basis of a flowchart as a series of steps or blocks, but the present invention is not limited to the order of steps, and certain steps may occur in a different order or concurrently with the steps described above. I can. In addition, those of ordinary skill in the art understand that the steps shown in the flowchart are not exclusive, other steps are included, or one or more steps in the flowchart may be deleted without affecting the scope of the present invention. You can understand.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the art to which the present invention pertains will be able to make various modifications and variations without departing from the essential characteristics of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain the technical idea, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

Claims (7)

In the video decoding method supporting a plurality of layers,
Analyzing a first layer dependency on a current layer based on a video parameter set (VPS);
Analyzing a second layer dependency on a current slice based on information encoded in slice units; And
Constructing a reference picture list for the current slice based on an analysis of at least one of a first layer dependency on the current layer or a second layer dependency on the current slice,
Analyzing the second layer dependency on the current slice based on the information encoded in the slice unit,
Determining whether to use the second layer dependency for the current slice based on information indicating whether to use the second layer dependency for the current slice,
The information indicating whether to use the second layer dependency is a flag. delete The method of claim 1,
Analyzing the second layer dependency on the current slice based on the information encoded in the slice unit,
When it is determined to use the second layer dependency for the current slice,
Analyzing the number of reference layers referenced by the current slice;
Analyzing identification information of a reference layer referenced by the current slice as many as the number of reference layers referenced by the current slice; And configuring an inter-layer reference picture set for the current slice based on identification information of a reference layer referenced by the current slice. In an image encoding method supporting a plurality of layers,
Determining a dependency of a first layer on a current layer and encoding it using a video parameter set (VPS);
Determining a dependency of a second layer on a current slice and encoding it in slice units; And
Constructing a reference picture list for the current slice based on at least one of a first layer dependency on the current layer or a second layer dependency on the current slice,
The step of determining the dependency of the second layer on the current slice and encoding it in units of slices,
Determining whether to set the second layer dependency on the current slice based on information indicating whether to set the second layer dependency on the current slice,
The information indicating whether to set the second layer dependency is a flag. delete The method of claim 4,
The step of determining the dependency of the second layer on the current slice and encoding it in units of slices,
When it is determined to set the second layer dependency on the current slice,
Determining the number of reference layers referenced by the current slice;
Determining identification information of a reference layer referenced by the current slice as many as the number of reference layers referenced by the current slice; And configuring an interlayer reference picture set for the current slice based on identification information of a reference layer referenced by the current slice. A computer-readable recording medium storing a bitstream input to an image decoding apparatus supporting a plurality of layers to restore an image,
The bitstream includes VPS (Video Parameter Set) and information encoded in slice units,
The VPS is used to analyze the first layer dependency on the current layer,
The information encoded in the slice unit is used to analyze the dependency of the second layer on the current slice,
Analysis of at least one of a first layer dependency on the current layer or a second layer dependency on the current slice is used to construct a reference picture list for the current slice,
The bitstream includes information indicating whether to use the second layer dependency for the current slice,
The information indicating whether to use the second layer dependency is used to determine whether to use the second layer dependency for the current slice,
The information indicating whether to use the second layer dependency is a flag.

Images