KR102400485B1 - Video decoding method and apparatus using the same - Google Patents

Abstract

An image decoding method according to an embodiment of the present invention includes: a filter type determining step of determining a type of filter to be applied to a first hierarchical picture that can be referenced by a second hierarchical picture to be decoded; an application target determining step of determining an application target to which the filter is to be applied in the first hierarchical picture; performing filtering on a filter application target based on the determined filter type; It may include adding the filtered first layer picture to the second layer reference picture list. Accordingly, an image decoding method and apparatus using the same are provided for reducing prediction errors in an upper layer and improving encoding efficiency.

Description

Video decoding method and apparatus using the same

The present invention relates to image encoding and decoding processing, and more particularly, to a method and apparatus for encoding and decoding an image supporting a plurality of layers in a bitstream.

Recently, as broadcasting services with high definition (HD) resolution are expanding not only domestically but also worldwide, many users are getting used to high-resolution and high-definition images, and accordingly, many institutions are spurring the development of next-generation imaging devices. Also, along with HDTV, as interest in Ultra High Definition (UHD) having a resolution four times higher than that of HDTV is increasing, compression technology for higher resolution and high-definition images is required.

For image compression, inter prediction technology that predicts pixel values included in the current picture from temporally previous and/or subsequent pictures, predicts pixel values included in the current picture using pixel information in the current picture An intra prediction technique, an entropy encoding technique in which a short code is assigned to a symbol having a high frequency of occurrence and a long code is assigned to a symbol having a low frequency of occurrence, etc. may be used.

In video compression technology, there is a technology that provides a constant network bandwidth under a limited operating environment of hardware without considering a flexible network environment. However, a new compression technique is required to compress image data applied to a network environment in which bandwidth is frequently changed, and for this purpose, a scalable video encoding/decoding method may be used.

The present invention provides a video encoding/decoding method and apparatus using inter-layer filtering.

In addition, in the present invention, when adding a picture of a lower layer to the reference picture list of the upper layer, one or more filters are adaptively applied to the picture of the lower layer and then added, thereby reducing prediction error in the upper layer and improving encoding efficiency. A decryption method and an apparatus using the same are provided.

Accordingly, a method for decoding an image that improves encoding efficiency and does not increase a reference picture list, and an apparatus using the same are provided.

An embodiment of the present invention provides a method for decoding an image supporting a plurality of layers, comprising: a filter type determining step of determining a type of filter to be applied to a first hierarchical picture that can be referenced by a second hierarchical picture to be decoded; an application target determining step of determining an application target to which the filter is to be applied in the first hierarchical picture; performing filtering on a filter application target based on the determined filter type; It may include adding the filtered first layer picture to the second layer reference picture list.

The determining of the filter type may include applying a predetermined fixed filter to the first hierarchical picture.

The fixed filter is a basic filter having a preset filter coefficient set, and when the basic filter is applied, filtering may not be performed on samples at integer positions.

The fixed filter is a replacement filter having a preset filter coefficient set, and when the replacement filter is applied, filtering may be performed on samples at integer positions.

The replacement filter may be applied to the first hierarchical picture to which the basic filter is applied.

The determining of the filter type may further include receiving and decoding a flag signal indicating whether the replacement filter is applied.

In the determining of the filter type, it may be determined that one or more filters are adaptively selected and applied to the corresponding picture of the first layer.

In the filter type determining step, it is determined that a basic filter having a preset filter coefficient set is applied to the first hierarchical picture, and a replacement filter different from the basic filter is applied to samples at integer positions of the first hierarchical picture. and determining whether to apply the replacement filter, wherein the determining whether to apply the replacement filter may receive and decode a flag signal indicating whether to apply the replacement filter.

In the filter type determining step, it is determined that a basic filter having a preset filter coefficient set is applied to the first hierarchical picture, and a replacement filter different from the basic filter is applied to samples at integer positions of the first hierarchical picture. and determining whether to apply the replacement filter, wherein the determining of whether to apply the replacement filter includes: calculating a sample correlation for each block unit of a predetermined size with respect to the first hierarchical picture; and determining whether to apply the replacement filter to the first hierarchical picture based on the correlation.

The step of determining whether to apply the replacement filter may include: horizontal activity in the block unit ?Ч硫痴? When the block activity based on the directional activity exceeds a predetermined threshold, it may be determined to apply the replacement filter.

An apparatus for decoding an image supporting a plurality of layers according to another embodiment of the present invention includes: a first layer decoder for decoding a first layer picture that can be referenced by a second layer picture to be decoded; a filtering unit that determines a type of filter to be applied to the first hierarchical picture, determines an application target to which the filter is to be applied in the first hierarchical picture, and performs filtering on a filter application target based on the determined filter type; It may include a prediction unit that adds the filtered first layer picture to the second layer reference picture list.

According to an embodiment of the present invention, a method and apparatus for encoding/decoding an image using inter-layer filtering are provided.

In addition, according to an embodiment of the present invention, when adding a picture of a lower layer to the reference picture list of the upper layer, one or more filters are adaptively applied to the picture of the lower layer and then added, thereby reducing the prediction error in the upper layer. Provided are a video decoding method and apparatus using the same for reducing and improving encoding efficiency.

Accordingly, a method for decoding an image that improves encoding efficiency and does not increase a reference picture list, and an apparatus using the same are provided.

1 is a block diagram illustrating a configuration of an image encoding apparatus according to an embodiment.
2 is a block diagram illustrating a configuration of an image decoding apparatus according to an embodiment.
3 is a conceptual diagram schematically illustrating an embodiment of a scalable video coding structure using a plurality of layers to which the present invention can be applied.
4 is a diagram illustrating an example of configuring a reference picture list.
5 is a control flowchart illustrating an image processing method according to the present invention.
6 is a diagram illustrating a multi-layered picture according to the present invention.
7 is a diagram illustrating a block sample for calculating horizontal and vertical activity according to the present invention.
8 is a diagram illustrating a POC of a picture of a first layer and a second layer according to the present invention.
9 is a diagram illustrating a reference picture list according to the present invention.
10 is a diagram illustrating a reference picture list for a P slice according to an embodiment of the present invention.
11 is a diagram illustrating a reference picture list for a B slice according to an embodiment of the present invention.

EMBODIMENT OF THE INVENTION Hereinafter, embodiment of this invention is described concretely with reference to drawings. In describing the embodiments of the present specification, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present specification, the detailed description thereof will be omitted.

When it is said that a component is “connected” or “connected to” another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be In addition, the description of “including” a specific configuration in the present invention does not exclude configurations other than the corresponding configuration, and it means that additional configurations may be included in the practice of the present invention or the scope of the technical spirit of the present invention.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

In addition, the components shown in the embodiment of the present invention are shown independently to represent different characteristic functions, and it does not mean that each component is made of separate hardware or a single software component. That is, each component is listed as each component for convenience of description, and at least two components of each component are combined to form one component, or one component can be divided into a plurality of components to perform a function, and each Integrated embodiments and separate embodiments of the components are also included in the scope of the present invention without departing from the essence of the present invention.

In addition, some of the components are not essential components for performing essential functions in the present invention, but may be optional components for merely improving performance. The present invention can be implemented by including only essential components to implement the essence of the present invention, except for components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement Also included in the scope of the present invention.

1 is a block diagram illustrating a configuration of an image encoding apparatus according to an embodiment. A scalable video encoding/decoding method or apparatus may be implemented by extension of a general image encoding/decoding method or apparatus that does not provide scalability, and the block diagram of FIG. 1 is scalable. An embodiment of an image encoding apparatus that can be the basis of a video encoding apparatus is shown.

Referring to FIG. 1 , the image encoding apparatus 100 includes a motion prediction unit 111 , a motion compensation unit 112 , an intra prediction unit 120 , a switch 115 , a subtractor 125 , and a transform unit 130 . , a quantizer 140 , an entropy encoder 150 , an inverse quantizer 160 , an inverse transform unit 170 , an adder 175 , a filter unit 180 , and a reference image buffer 190 .

The image encoding apparatus 100 may encode an input image in an intra mode or an inter mode and output a bitstream. Intra prediction refers to intra prediction, and inter prediction refers to inter prediction. In the intra mode, the switch 115 is switched to intra, and in the inter mode, the switch 115 is switched to inter. The image encoding apparatus 100 may generate a prediction block for an input block of an input image, and then encode a difference between the input block and the prediction block.

In the intra mode, the intra prediction unit 120 may generate a prediction block by performing spatial prediction using pixel values of an already encoded block around the current block.

In the inter mode, the motion prediction unit 111 may obtain a motion vector by finding a region that best matches the input block in the reference image stored in the reference image buffer 190 during the motion prediction process. The motion compensator 112 may generate a prediction block by performing motion compensation using a motion vector and a reference image stored in the reference image buffer 190 .

The subtractor 125 may generate a residual block by the difference between the input block and the generated prediction block. The transform unit 130 may perform transform on the residual block to output a transform coefficient. In addition, the quantization unit 140 may quantize the input transform coefficient according to the quantization parameter to output a quantized coefficient.

The entropy encoding unit 150 entropy-encodes a symbol according to a probability distribution based on the values calculated by the quantization unit 140 or an encoding parameter value calculated during the encoding process, and outputs a bitstream. can do. The entropy encoding method is a method of receiving symbols having various values, removing statistical redundancy, and expressing them as a decodable binary sequence.

Here, the symbol means a syntax element to be encoded/decoded, a coding parameter, and a value of a residual signal. Encoding parameters are parameters necessary for encoding and decoding, and may include information that is encoded in the encoder and transmitted to the decoder, such as syntax elements, as well as information that can be inferred during the encoding or decoding process. means necessary information. Coding parameters include, for example, intra/inter prediction modes, motion/motion vectors, reference image indexes, coding block patterns, presence or absence of residual signals, transform coefficients, quantized transform coefficients, quantization parameters, block size, block partition information, etc. or It may include statistics. In addition, the residual signal may mean a difference between the original signal and the predicted signal, and a signal in which the difference between the original signal and the predicted signal is transformed or a signal in which the difference between the original signal and the predicted signal is transformed and quantized may mean The residual signal may be referred to as a residual block in block units.

When entropy encoding is applied, a small number of bits are allocated to a symbol having a high probability of occurrence and a large number of bits are allocated to a symbol having a low probability of occurrence to express the symbol, so that the size of the bit stream for the symbols to be encoded is increased. can be reduced. Accordingly, compression performance of image encoding may be improved through entropy encoding.

For entropy encoding, an encoding method such as exponential golomb, Context-Adaptive Variable Length Coding (CAVLC), or Context-Adaptive Binary Arithmetic Coding (CABAC) may be used. For example, a table for performing entropy encoding, such as a variable length coding (VLC) table, may be stored in the entropy encoder 150, and the entropy encoder 150 may store the stored variable length encoding. Entropy encoding can be performed using a (VLC) table. In addition, the entropy encoder 150 derives a binarization method of a target symbol and a probability model of a target symbol/bin, and then performs entropy encoding using the derived binarization method or probability model. You may.

The quantized coefficient may be inverse quantized by the inverse quantization unit 160 and inversely transformed by the inverse transform unit 170 . The inverse-quantized and inverse-transformed coefficients may be added to the prediction block through the adder 175 and a reconstructed block may be generated.

The reconstructed block passes through the filter unit 180, and the filter unit 180 applies at least one of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the reconstructed block or the reconstructed picture. can do. The reconstructed block passing through the filter unit 180 may be stored in the reference image buffer 190 .

2 is a block diagram illustrating a configuration of an image decoding apparatus according to an embodiment. As described above with reference to FIG. 1 , the scalable video encoding/decoding method or apparatus may be implemented by extension of a general image encoding/decoding method or apparatus that does not provide scalability, and the block diagram of FIG. 2 shows scalable video decoding An embodiment of an image decoding apparatus that can be the basis of the apparatus is shown.

Referring to FIG. 2 , the image decoding apparatus 200 includes an entropy decoding unit 210 , an inverse quantization unit 220 , an inverse transform unit 230 , an intra prediction unit 240 , a motion compensator 250 , and a filter unit. 260 and a reference image buffer 270 .

The image decoding apparatus 200 may receive a bitstream output from the encoder, perform decoding in an intra mode or an inter mode, and output a reconstructed image, that is, a reconstructed image. In the case of the intra mode, the switch may be switched to the intra mode, and in the case of the inter mode, the switch may be switched to the inter mode. The image decoding apparatus 200 may generate a reconstructed block, that is, a reconstructed block by obtaining a residual block reconstructed from the received bitstream, generating a prediction block, and adding the reconstructed residual block and the prediction block.

The entropy decoding unit 210 may entropy-decode the input bitstream according to a probability distribution to generate symbols including symbols in the form of quantized coefficients. The entropy decoding method is a method of generating each symbol by receiving a binary string as an input. The entropy decoding method is similar to the entropy encoding method described above.

The quantized coefficient is inverse quantized by the inverse quantization unit 220 and inverse transformed by the inverse transformation unit 230 , and as a result of inverse quantization/inverse transformation of the quantized coefficient, a reconstructed residual block may be generated.

In the intra mode, the intra prediction unit 240 may generate a prediction block by performing spatial prediction using pixel values of an already encoded block around the current block. In the inter mode, the motion compensator 250 may generate a prediction block by performing motion compensation using a motion vector and a reference image stored in the reference image buffer 270 .

The reconstructed residual block and the prediction block are added through the adder 255 , and the added block passes through the filter unit 260 . The filter unit 260 may apply at least one of a deblocking filter, SAO, and ALF to a reconstructed block or a reconstructed picture. The filter unit 260 outputs a reconstructed image, that is, a reconstructed image. The restored image may be stored in the reference image buffer 270 and used for inter prediction.

The entropy decoding unit 210 , the inverse quantization unit 220 , the inverse transform unit 230 , the intra prediction unit 240 , the motion compensator 250 , and the filter unit 260 included in the image decoding apparatus 200 . and components directly related to image decoding among the reference image buffer 270 , for example, the entropy decoding unit 210 , the inverse quantization unit 220 , the inverse transform unit 230 , the intra prediction unit 240 , and motion compensation. The unit 250 and the filter unit 260 may be distinguished from other components and expressed as a decoder or a decoder.

Also, the image decoding apparatus 200 may further include a parsing unit (not shown) that parses information related to the encoded image included in the bitstream. The parsing unit may include the entropy decoding unit 210 or may be included in the entropy decoding unit 210 . This parsing unit may also be implemented as one component of the decoding unit.

3 is a conceptual diagram schematically illustrating an embodiment of a scalable video coding structure using a plurality of layers to which the present invention can be applied. In FIG. 3 , a group of pictures (GOP) indicates a picture group, that is, a group of pictures.

A transmission medium is required to transmit image data, and its performance is different for each transmission medium according to various network environments. A scalable video coding method may be provided for application to such various transmission media or network environments.

The scalable video coding method is a coding method that improves encoding/decoding performance by removing inter-layer redundancy by utilizing texture information, motion information, residual signals, etc. between layers. The scalable video coding method may provide various scalability in terms of spatial, temporal, and picture quality according to ambient conditions such as a transmission bit rate, a transmission error rate, and a system resource.

Scalable video coding may be performed using a structure of multiple layers so as to provide a bitstream applicable to various network situations. For example, the scalable video coding structure may include a base layer that compresses and processes image data using a general image encoding method, and compresses image data using both encoding information of the base layer and a general image encoding method. It may include an enhancement layer that handles it.

Here, a layer is an image and bit classified based on spatial (eg, image size), temporal (eg, encoding order, image output order, frame rate), image quality, complexity, etc. It means a set of streams (bitstream). In addition, a base layer may mean a lower layer or a reference layer, and an enhancement layer may mean an upper layer. Also, the plurality of layers may have dependencies on each other.

Referring to FIG. 3 , for example, the base layer may be defined as SD (standard definition), a frame rate of 15 Hz, and a bit rate of 1 Mbps, and the first enhancement layer is HD (high definition), a frame rate of 30 Hz, and a bit rate of 3.9 Mbps. may be defined, and the second enhancement layer may be defined as 4K-UHD (ultra high definition), a frame rate of 60 Hz, and a bit rate of 27.2 Mbps. The format, frame rate, bit rate, etc. are one embodiment and may be differently determined as needed. Also, the number of layers to be used is not limited to the present embodiment and may be determined differently according to circumstances.

For example, if the transmission bandwidth is 4 Mbps, the frame rate of the first enhancement layer HD may be reduced to transmit at 15 Hz or less. The scalable video coding method may provide temporal, spatial, and picture quality scalability by the method described above in the embodiment of FIG. 3 .

Scalable video coding has the same meaning as scalable video coding in terms of encoding and scalable video decoding in terms of decoding.

The present invention relates to a process of encoding and decoding an image including a plurality of layers or views, wherein the plurality of layers or views are first, second, third, It can be expressed as an n-th layer or a viewpoint. In the following description, a picture in which the first layer and the second layer exist is described as an example, and the same method can be applied to layers or viewpoints higher than that. Also, the first layer may be expressed as a base layer and the second layer may be expressed as an upper layer. Also, the first layer may be expressed as a reference layer, and the second layer may be expressed as an enhancement layer.

The picture/block of the first layer corresponding to the picture/block of the second layer may be changed according to the size of the picture/block of the second layer. That is, when the size of the picture/block of the first layer is smaller than the picture/block of the second layer, scaling may be performed using a method such as up-sampling or re-sampling.

Also, the picture of the first layer may be added to the reference picture list of the second layer and used for image encoding/decoding of the second layer. In this case, the second layer may perform prediction and encoding/decoding by using the first layer image in the reference picture list like normal inter prediction.

The size of the block for encoding/decoding may be an NxN-shaped square such as 4x4, 8x8, 16x16, 32x32, 64x64, or an NxM-shaped rectangle such as 4x8, 16x8, 8x32, etc., and the unit of the block is a Coding Block (CB). ), a prediction block (PB), and a transform block (TB), and each may have a different size.

Hereinafter, a prediction block of a block (hereinafter referred to as a current block or a target block) that is a target for encoding and decoding of a higher layer among scalable video, that is, an image encoding and decoding method using a multi-layer structure, that is, a prediction signal is generated Let's see how to do it. The following content (method or apparatus) of the present invention can be equally applied to an encoder and a decoder in general.

In general, inter prediction may use at least one of a picture before or after a picture of the current picture as a reference picture, and may perform prediction on the current block based on the reference picture. A picture used for prediction of the current block is called a reference picture or a reference frame.

A reference picture is specified by a reference picture index refIdx, and a predetermined region in the reference picture is specified as a reference block through a motion vector.

Inter prediction may generate a prediction block for the current block by selecting a reference picture and a reference block corresponding to the current block in the reference picture.

In inter prediction, the encoding apparatus and the decoding apparatus may derive motion information of the current block and then perform inter prediction and/or motion compensation based on the derived motion information. In this case, the encoding apparatus and the decoding apparatus move a collocated block corresponding to the current block in a reconstructed neighboring block and/or an already reconstructed collocated picture. By using the information, encoding/decoding efficiency can be improved.

Here, the reconstructed neighboring block is a block in the reconstructed current picture that has already been encoded and/or decoded, and may include a block adjacent to the current block and/or a block located at an outer corner of the current block. Also, the encoding apparatus and the decoding apparatus may determine a predetermined relative position based on a block existing at a position spatially corresponding to the current block in the collocated picture, and the determined relative position (corresponding spatially to the current block) It is possible to derive the collocated block based on the position inside and/or outside the block existing at the position where the block is located. Here, as an example, the collocated picture may correspond to one picture among the reference pictures included in the reference picture list.

Inter prediction may generate a prediction block such that a residual signal with a current block is minimized and a motion vector size is also minimized.

Meanwhile, the method of deriving motion information may vary depending on the prediction mode of the current block. Prediction modes applied for inter prediction may include Advanced Motion Vector Predictor (AMVP), merge, and the like.

For example, when Advanced Motion Vector Predictor (AMVP) is applied, the encoding apparatus and the decoding apparatus may generate a motion vector candidate list by using a motion vector of a reconstructed neighboring block and/or a motion vector of a collocated block. That is, the motion vector of the reconstructed neighboring block and/or the motion vector of the collocated block may be used as a motion vector candidate. The encoding apparatus may transmit a predicted motion vector index indicating an optimal motion vector selected from among the motion vector candidates included in the list to the decoding apparatus. In this case, the decoding apparatus may select a predicted motion vector of the current block from among motion vector candidates included in the motion vector candidate list by using the motion vector index.

The encoding apparatus may obtain a motion vector difference (MVD) between the motion vector of the current block and the predicted motion vector, encode it, and transmit it to the decoding apparatus. In this case, the decoding apparatus may decode the received motion vector difference, and may derive the motion vector of the current block through the sum of the decoded motion vector difference and the predicted motion vector.

The encoding apparatus may also transmit a reference picture index indicating a reference picture to the decoding apparatus.

The decoding apparatus may predict the motion vector of the current block by using motion information of the neighboring block, and may derive the motion vector of the current block by using the residual received from the encoding apparatus. The decoding apparatus may generate a prediction block for the current block based on the derived motion vector and reference picture index information received from the encoding apparatus.

As another example, when merge is applied, the encoding apparatus and the decoding apparatus may generate a merge candidate list by using motion information of a reconstructed neighboring block and/or motion information of a collocated block. That is, when motion information of the reconstructed neighboring block and/or collocated block exists, the encoding apparatus and the decoding apparatus may use it as a merge candidate for the current block.

The encoding apparatus may select a merge candidate capable of providing optimal encoding efficiency from among merge candidates included in the merge candidate list as motion information for the current block. In this case, the merge index indicating the selected merge candidate may be included in the bitstream and transmitted to the decoding apparatus. The decoding apparatus may select one of the merge candidates included in the merge candidate list by using the transmitted merge index, and may determine the selected merge candidate as motion information of the current block. Accordingly, when the merge mode is applied, the motion information of the reconstructed neighboring block and/or the collocated block may be used as it is as the motion information of the current block. The decoding apparatus may reconstruct the current block by adding the prediction block and the residual transmitted from the encoding apparatus.

In the aforementioned AMVP and merge modes, in order to derive the motion information of the current block, motion information of a reconstructed neighboring block and/or motion information of a collocated block may be used.

In the case of the skip mode, which is one of the other modes used for inter prediction, information on neighboring blocks may be used for the current block as it is. Accordingly, in the skip mode, the encoding apparatus does not transmit syntax information such as residuals to the decoding apparatus other than information indicating which block to use as the motion information of the current block.

The encoding apparatus and the decoding apparatus may generate a prediction block of the current block by performing motion compensation on the current block based on the derived motion information. Here, the prediction block may mean a motion-compensated block generated as a result of performing motion compensation on the current block. Also, a plurality of motion-compensated blocks may constitute one motion-compensated picture.

The decoding apparatus may check a skip flag, a merge flag, etc. received from the encoding apparatus for motion information necessary for inter prediction of the current block, for example, a motion vector, a reference picture index, and the like, and derive it correspondingly.

A processing unit in which prediction is performed and a processing unit in which a prediction method and specific content are determined may be different from each other. For example, a prediction mode may be determined in units of prediction blocks and prediction may be performed in units of transform blocks, or a prediction mode may be determined in units of prediction blocks and intra prediction may be performed in units of transform blocks.

Pictures encoded/decoded before the current picture may be stored in a memory (eg, Decoded Picture Buffer: DPB) and used for prediction of a current block (current picture). A list of pictures available for inter prediction of the current block is maintained as a reference picture list.

A P slice is a slice decoded through intra prediction or inter prediction using at most one motion vector and one reference picture. A B slice is a slice decoded through intra prediction or inter prediction using up to two motion vectors and two reference pictures. In this case, the reference picture includes a short term reference picture and a long term reference picture. A picture may be specified with a picture order count (POC) indicating a display order, short-term reference pictures may be pictures with a small POC difference from the current picture, and long-term reference pictures may be pictures with a large POC difference from the current picture. can

Reference picture list 0 (reference picture list 0, hereinafter referred to as 'L0' for convenience of description) is a reference picture list used for inter prediction of a P slice or a B slice. Reference picture list 1 (reference picture list 1, hereinafter referred to as 'L1' for convenience of description) is used for inter prediction of the B slice. Accordingly, L0 is used for inter prediction of a block of a P slice in which unidirectional prediction is performed, and L0 and L1 are used in inter prediction of a block of a B slice in which bidirectional prediction is performed.

The decoding apparatus constructs a reference picture list when decoding the P slice and the B slice through inter prediction. A reference picture used for inter prediction is designated through a reference picture list. The reference picture index is an index indicating a reference picture on the reference picture list.

The reference picture list may be constructed based on a reference picture set transmitted from the encoding apparatus. The reference picture set may include a POC of a picture used as a reference picture and a flag (used_by_curr_pic_s0_flag) indicating whether the corresponding picture is directly referenced. Reference pictures constituting the reference picture list may be stored in a memory (eg, DPB). Pictures stored in the memory (pictures encoded/decoded before the current picture) are managed by the encoding apparatus and the decoding apparatus.

The reference picture set may include a short-term reference picture set including a short-term reference picture and a long-term reference picture set including a long-term reference picture, and an initial reference picture list is constructed based on the short-term reference picture set and the long-term reference picture set. can be

4 is a diagram illustrating an example of configuring a reference picture list.

The reference pictures are a first short-term reference picture set (RefPicSetStCurr0) consisting of reference pictures (Ref 1, Ref 2) smaller than the POC of the current picture (Curr) based on the current picture and reference pictures larger than the POC of the current picture (RefPicSetStCurr0) ( It may be classified into a second short-term reference picture set (RefPicSetStCurr1) composed of Ref 3 and Ref 4) and a long-term reference picture set (RefPicSetLtCurr) composed of long-term reference pictures (Ref LT1 and Ref LT2).

At this time, the first short-term reference picture set (RefPicSetStCurr0) is composed of pictures having a used_by_curr_pic_s0_flag value of 1 (delta_poc_s0 with used_by_curr_pic_s0_flag=1), and the second short-term reference picture set (RefPicSetStCurr1) is also composed of pictures with a used_by_curr_pic_s0_flag value of 1 (RefPicSetStCurr1). (delta_poc_s1 with used_by_curr_pic_s1_flag=1).

An initial reference picture list may be constituted by a set of reference picture sets having different properties.

As shown in FIG. 4 , the reference picture list 0, that is, L0, consists of a first short-term reference picture set (RefPicSetStCurr0), a second short-term reference picture set (RefPicSetStCurr1), and a long-term reference picture set (RefPicSetLtCurr) in this order.

On the other hand, reference picture list 1, that is, L1, consists of a second short-term reference picture set (RefPicSetStCurr1), a first short-term reference picture set (RefPicSetStCurr0), and a long-term reference picture set (RefPicSetLtCurr) in this order.

The number of reference pictures that may be included in the reference picture list may be determined based on information transmitted from the encoding apparatus. For example, after constructing the reference picture list, the encoding apparatus determines the number of reference pictures to be used, and provides information on the number of reference pictures to be used (eg, num_ref_idx_lX_default_active_minus1, X=0 or 1) as a sequence parameter set (SPS). ) as a syntax element and can be transmitted to the decoding device. The decoding apparatus may use the number of reference pictures specified by adding 1 to the received num_ref_idx_lX_default_active_minus1 as a default value in the current sequence.

Also, when the encoding apparatus intends to designate the number of reference pictures for each picture or slice, separate information indicating the number of reference pictures (eg, num_ref_idx_11_active_minus1, X=0 or 1) is provided as a Picture Parameter Set (PPS). ) or through a slice header. The decoding apparatus may apply a value specified by adding 1 to the received num_ref_idx_11_active_minus1 as the number of reference pictures for the current picture or current slice.

When performing inter prediction, motion compensation may be performed using a reference picture specified in the reference picture list configured as described above.

In a multi-layer structure that provides spatial scalability or multi-view scalability, a reference picture of a higher layer may include reference pictures of the same layer and an inter-layer reference picture.

In this case, signaling of the inter-layer reference picture may be performed through information identifying the layer and information identifying the reference picture. For example, the nuh_layer_id value, which is the j-th layer identifier (i is greater than j), which exists in the same access unit as the current picture of the i-th layer and is transmitted by being included in a Network Abstraction Layer (NAL) unit header, is the current picture. If it is the same as the RefPiclayerId value for , it may be determined that the corresponding picture is used as a reference picture for the current picture. An inter-layer reference picture may be displayed as a long-term reference picture.

RefPicLayerId is a value that can be signaled as a syntax element inter_layer_pred_layer_idc included in the slice header, and means a layer that the current layer refers to for inter-layer prediction.

Meanwhile, when adding a picture of a lower layer to the reference picture list of the upper layer, one fixed filter may be applied to the picture of the lower layer and then added to the reference picture list of the upper layer. However, in this case, there may be a problem in that encoding efficiency is lowered.

In addition, when two filters are applied to a picture of a lower layer, since one or more pictures may be added to the reference picture list, information configuring or signaling the reference picture list may increase.

Accordingly, the present invention improves encoding efficiency by adaptively selecting/applying one or more filters to a predetermined unit unit and adding one lower layer picture to the upper layer reference picture list in applying the filter to the lower layer picture. It is characterized in that the complexity of constructing the reference picture list is not increased.

5 is a control flowchart illustrating an image processing method according to the present invention. More specifically, FIG. 5 is a diagram for explaining a method of encoding and decoding a multi-layered image using a picture of a first layer when performing an encoding and decoding process on a picture of a second layer. Accordingly, the description of FIG. 5 may be applied to both video decoding and encoding methods.

In order to filter the first hierarchical picture, first, the encoder and the decoder determine a filter type for the first hierarchical picture (S510). The first hierarchical picture is decoded before the filter type is determined, and the first hierarchical picture and the second hierarchical picture may be decoded by different configurations or modules, respectively. The encoder and decoder may include a decoder having the configuration of FIGS. 1 and 2 for decoding the first hierarchical picture, and a decoder having the configuration of FIGS. 1 and 2 for decoding the second hierarchical picture, respectively.

6 is a diagram illustrating a multi-layered picture according to the present invention. As shown, the picture of the second layer becomes a target picture to be encoded and decoded, and the picture of the first layer is a corresponding picture corresponding to the target picture and may be a reference picture of the picture of the second layer.

The target picture and the corresponding picture have the same POC, for example, the POC of the target picture and the corresponding picture may be 4. For prediction of a target picture, a corresponding picture having a POC of 4 and a picture of a second layer having a POC of 0 or POC 8 may be used.

The size of the sub/decoding target picture of the second layer and the picture corresponding to the first layer may be the same or different. Even if the size of the target picture of the second layer and the corresponding picture of the first layer are the same, characteristics of signals may be different. Therefore, the prediction efficiency is improved by applying up-sampling or re-sampling to the reconstructed picture of the first layer to make the picture sizes of the two layers the same or to use after changing the signal characteristics. can do it

A predetermined filter may be applied for this sampling, and the filter may be different according to a luminance (Luma) component or a chroma (Chroma) component.

The encoder and decoder may determine the filter type of the first hierarchical picture as one fixed filter, or may adaptively select one or more filters and apply them to the first hierarchical picture.

When the encoder and the decoder apply the fixed filter to the first layer picture, a default interpolation filter may be applied to the corresponding picture of the first layer.

The default filter may have a filter set for the luminance/color difference signal as shown in Tables 1 and 2 below, and the phase and filter coefficients applied according to the picture size ratio between layers may be different. there is.

Table 1 shows the 16-phase resampling filter coefficients for the luminance signal, and Table 2 shows the 16-phase resampling filter coefficients for the chrominance signal.

In this case, the phase '0' may mean a filter coefficient for a sample at an integer position, and may have the same result as not applying a filter.

Alternatively, the encoder and the decoder may apply an alternative filter as a predetermined fixed filter to the corresponding picture of the first layer or the picture to which the basic filter is applied.

An alternative filter may have a filter set comprising one or more filter coefficients, similar to the basic filter described above. In this case, the filter coefficients may be determined by the encoder and signaled to the decoder.

The replacement filter may be applied to a picture to which the basic filter is applied, and may mean a filter for a sample at an integer position corresponding to the phase '0' of the basic filter. In this case, the replacement filter may have a fixed filter coefficient such as [-1, 3, 12, 3, ?1]/16.

In the case of a video supporting picture quality scalability (SNR scalability), filtering may not be performed because the size of the picture of the second layer and the picture of the first layer is the same. However, according to the present invention, a replacement filter may be applied to the picture corresponding to the first layer in order to change the characteristics of the signal of the first layer.

In this case, the flag indicating whether to use the alternative filter is through at least one of a video parameter set, a sequence parameter set, a picture parameter set, a slice header, etc. can be signaled.

According to another embodiment, the encoder and the decoder may adaptively select one or more filters instead of a fixed filter as a filter type and apply them to the corresponding picture of the first layer.

For example, in a video in which spatial scalability is supported, the encoder and the decoder may always apply the replacement filter to the integer position samples after applying the basic filter to the picture of the first layer. In this case, even without additional signaling from the encoder, the decoder can filter the picture by adaptively applying two basic filters and two replacement filters, respectively.

Another, for example, encoder and decoder apply a basic filter to a picture of a first layer in a video in which spatial scalability is supported. Then, the decoder may determine whether to apply the replacement filter according to a flag signaled from the encoder.

After applying the basic filter to the picture of the first layer, the encoder may apply the replacement filter to the integer position sample. After applying the replacement filter to the integer position sample, it is possible to determine whether to apply the replacement filter by calculating one or more of Rate Distortion Optimization (RDO), Sum of Absolute Hadamard Transformed Differences (SATD), and Sum of Absolute Difference (SAD). . According to the result of the operation, if applying the replacement filter is more effective than not applying it, the encoder may decide to apply the replacement filter to integer position samples. A signal indicating whether or not to apply the replacement filter, that is, a flag indicating whether to apply the replacement sample, is a video parameter set, a sequence parameter set, a picture parameter set, and a slice header. (Slice header) may be signaled through at least one of.

According to another embodiment, the encoder and the decoder apply a basic filter to a picture of a first layer included in a video to which spatial scalability is supported, and a first layer included in a video to which picture quality scalability is applied. A replacement filter can be applied to a picture of .

Also, according to another embodiment, the encoder and the decoder may adaptively select and apply a filter based on the correlation between samples. That is, after calculating the correlation between samples in units of specific blocks, such as 4x4, 8x8, 16x16, 32x32, 64x64, the replacement filter can be adaptively applied by comparing it with a specific threshold. In this case, the correlation between the samples can be calculated based on the first layer corresponding picture (reconstructed picture) or the picture to which the basic filter is applied, and the threshold value can be set differently depending on whether the scalability is spatial or image quality. there is.

For example, the encoder and the decoder can obtain the vertical activity (VA) and the horizontal activity (HA) using Equation 1 when the block size is 4x4 as shown in FIG. 7 . there is.

<Formula 1>

In this case, i and j are 0 and 2.

When vertical and horizontal activity is calculated based on Equation 1, block activity may be calculated using Equation 2.

<Formula 2>

Block Activity = (VA + HA) >> 2

The encoder and the decoder may apply the replacement filter to the target block when the block activity is less than the threshold, and may not apply the replacement filter when the block activity is greater than or equal to the threshold.

The process of applying the replacement filter by calculating the vertical vitality, horizontal vitality, and block vitality can be equally applied to blocks of 8x8 or more.

According to another embodiment, the encoder and the decoder apply the replacement filter to the target block when the size of the block is NxN, when samples in the vertical and horizontal directions satisfy Equation 3 below, and do not apply the replacement filter when not satisfied. it may not be

<Formula 3>

Abs(p[?1][N>>1-1] + p[N-1][N>>1-1] ? 2*p[N>>1-1][N>>1-1] ) < (1<<( BitDepthY ? 5))

Abs(p[N>>1-1][?1] + p[N>>1-1][N-1] ? 2*p[N>>1-1][N>>1-1] ) < (1<<( BitDepthY ? 5))

In this case, p[x][y] may indicate a sample value at the (x, y) position, Abs may indicate an absolute value, and BitDepthY may indicate a bit depth of a luminance signal.

As such, according to the present invention, when filtering the first hierarchical picture, either a fixed filter of the basic filter or the replacement filter may be applied, and one or both of the basic filter and the replacement filter may be selectively applied. can

When the filter type for the picture of the first layer is determined, the encoder and the decoder may determine a target to which the filter is applied ( S520 ).

The encoder and the decoder apply the fixed filter or the adaptive filter for each corresponding picture, slice, encoding unit (CU), prediction unit (PU), transform unit (TU), and predetermined unit (NxN) of the first hierarchical image. can do.

For example, the encoder may determine whether to use the replacement filter in units of pictures and may signal a flag indicating whether to use the replacement filter through the PPS. The decoder may determine whether to use an alternative filter according to a signaled flag, and may perform filtering in units of pictures.

According to another example, the encoder may determine whether to use the replacement filter on a slice-by-slice basis, and may signal a flag indicating whether to use the replacement filter through a slice header. The decoder may determine whether to use an alternative filter according to a signaled flag, and may perform filtering in units of slices.

Alternatively, the encoder and the decoder may adaptively apply the basic filter or the replacement filter after analyzing the correlation between samples for each prediction unit. In this case, since the correlation between samples is individually analyzed by the encoder and the decoder to perform adaptive filtering, additional signaling on whether to filter may not be required.

For example, a predetermined unit for adaptive filtering may be determined and a basic filter or a replacement filter may be applied in a predetermined unit unit. In this case, the replacement filter may be a filter that performs filtering on samples in integer positions.

When the predetermined unit is determined to be 8x8, a filter may be adaptively applied after calculating a correlation between samples in an 8x8 unit with respect to the first hierarchical picture.

When the filter type and filter application target are determined through steps S510 and S520, the encoder and the decoder perform filtering on the first layer picture through one or more filters or a combination of application targets (S530). That is, the encoder and the decoder perform filtering on the filter application target based on the determined filter type.

Then, the encoder and the decoder may add the filtered first layer picture to the second layer reference picture list (S540). The first layer picture may be added to the second layer reference picture list by the prediction unit for decoding the second layer picture.

In encoding and decoding the second layer target picture, one or more first layer pictures obtained through filtering may be added and used at an arbitrary position in the second layer reference picture list 0 or reference picture list 1.

In this case, the encoder and the decoder may allocate the first hierarchical picture to an arbitrary index in the reference picture list.

8 is a diagram illustrating a POC of a picture of a first layer and a second layer according to the present invention. Referring to FIG. 8 , a picture to be encoded and decoded may be a second layer picture having a POC of 2, and the target picture is a corresponding picture of the first layer, a picture having a smaller POC than itself (POC 0, POC 1) and Pictures (POC 3, POC 4) having a larger POC than themselves may be referred to as short-term reference pictures. Also, a long-term reference picture (not shown) may be referred to.

9 is a diagram illustrating a reference picture list according to the present invention. When adding the reference picture to the reference picture list, the encoder and the decoder may add the first layer picture F0 to an arbitrary position as shown in FIG. 9 .

For example, as shown in [a] of FIG. 9 , when the encoder and the decoder construct the reference picture list 0 (List 0), the POC of the current target picture is smaller than the POC of the short-term reference picture (POC 0, POC 1), Short-term reference pictures (POC 3, POC 4), the first layer picture (F0), which have a larger POC than the POC of the current target picture, are sequentially added to the reference picture list 0, and finally, a long-term reference picture (lt) can be added. there is.

Similarly, reference picture list 1 (List 1) is a short-term reference picture (POC 3, POC 4) having a larger POC than the POC of the current target picture, and a short-term reference picture having a smaller POC than the POC of the current target picture (POC 0, POC 1) , a first layer picture (F0), and a long-term reference picture (lt) may be configured in the order.

According to another embodiment, the encoder and the decoder may construct a reference picture list as in [b]. Referring to [b] of FIG. 9 , the reference picture list 0 (List 0) is a short-term reference picture (POC 0, POC 1) having a smaller POC than the POC of the current target picture, the first layer picture (F0), the current target picture The short-term reference pictures (POC 3, POC 4) having a larger POC than the POC of may be configured in the order of a long-term reference picture (lt).

Similarly, reference picture list 1 (List 1) is a short-term reference picture (POC 3, POC 4) having a larger POC than the POC of the current target picture, and a short-term reference picture having a smaller POC than the POC of the current target picture (POC 0, POC 1) , a first layer picture (F0), and a long-term reference picture (lt) may be configured in the order.

According to another embodiment, the encoder and the decoder may be added to the beginning or the end of each list. That is, the first layer picture (F0) is added to the reference picture of the second layer corresponding to 1, 0, 3, 4, lt to List 0, and finally the first layer picture (F0) obtained through filtering can be added List 1 may also add a first hierarchical picture F0 following 3, 4, 1, 0, and lt.

Meanwhile, according to another embodiment of the present invention, the number of first layer pictures to be added may be different for each list, and a filter applied to a picture added to each list may be fixed to a specific filter, or may be determined by an encoder and signaled to a decoder You may.

10 is a diagram illustrating a reference picture list for a P slice according to an embodiment of the present invention.

In the case of P slice, one picture to which the filtering method is applied to the first hierarchical picture F0 may be added to the end of the reference picture list 0 as shown in [a] of FIG. 10 . In this case, pictures 1, 0, 3, and 4 may be reference pictures of the second layer, and F0 may represent a picture obtained by filtering the corresponding picture of the first layer using one of the filtering methods.

On the other hand, in the case of a P slice, the encoder and the decoder refer to two pictures (F0, F1) on which filtering is performed on the first layer picture after the second layer reference picture as shown in [b] of FIG. 10 in the case of a P slice reference picture list 0 can be added. In this case, F0 and F1 may represent pictures obtained by filtering the first layer corresponding picture using one of the filtering methods described with reference to FIGS. 5 to 7 , respectively.

11 is a diagram illustrating a reference picture list for a B slice according to an embodiment of the present invention.

In the case of a B slice, the encoder and decoder may add one picture F0 to which filtering is applied to the first hierarchical picture as shown in [a] of FIG. 11 to the reference picture list 0 or the reference picture list 1. In this case, the first layer picture F0 may represent a picture obtained by filtering the first layer corresponding picture using one of the filtering methods described with reference to FIGS. 5 to 7 .

In addition, in the case of a B slice, the encoder and the decoder add the two pictures to which filtering is applied to the first hierarchical picture as shown in [b] of FIG. 11 in the reference picture list 0 (List 0) and the reference picture list 1 (List 1). can be added In this case, F0 and F1 may represent a picture obtained by filtering the first layer-corresponding picture using one of the filtering methods described with reference to FIGS. 5 to 7 , and F0 and F1 may be the same.

For example, when the encoder determines to use the replacement filter through SATD, etc., F0 may be a picture to which the basic filter is applied, and F1 may be a picture to which the replacement filter is applied after applying the basic filter.

In this case, in the case of picture quality scalability, F0 may be a corresponding picture to which no filter is applied, and F1 may be a picture to which a replacement filter is applied.

According to another embodiment, when the encoder and the decoder construct the reference picture list for the B slice, four pictures (F0, F1, F2, F3) to which filtering is applied to the first hierarchical picture as shown in [c] of FIG. 11 . ) may be added to the reference picture list 0 and the reference picture list 1. In this case, F0, F1, F2, and F3 represent a picture obtained by filtering the first layer-corresponding picture using one of the above-described filtering methods, and at least one of F0, F1, F2, and F3 may be the same.

As described above, according to the present invention, when adding a picture of a lower layer to the reference picture list of an upper layer, one or more filters are adaptively applied to the picture of the lower layer and then added, thereby reducing the prediction error in the upper layer and A video decoding method for improving encoding efficiency and an apparatus using the same are provided.

Accordingly, it is possible to provide a method for decoding an image that improves encoding efficiency and does not increase the reference picture list, and an apparatus using the same.

In the above embodiment, the methods are described on the basis of a flowchart as a series of steps or blocks, but the present invention is not limited to the order of the steps, and some steps may occur in a different order or concurrently with other steps as described above. there is. In addition, those of ordinary skill in the art will recognize that the steps shown in the flowchart are not exclusive, other steps may be included, or that one or more steps in the flowchart may be deleted without affecting the scope of the present invention. You will understand.

The above-described embodiments include examples of various aspects. It is not possible to describe every possible combination for representing the various aspects, but one of ordinary skill in the art will recognize that other combinations are possible. Accordingly, it is intended that the present invention cover all other substitutions, modifications and variations falling within the scope of the following claims.

*100: video encoding device 111: motion prediction unit
112: motion compensation unit 120: intra prediction unit
115: switch 125: subtractor
130: transform unit 140: quantization unit
150: entropy encoding unit 160: inverse quantization unit
170: inverse transform unit 180: filter unit

Claims (6)

In the video decoding method supporting a plurality of layers, the method is performed by an image decoding apparatus,
generating a reference picture list of a current slice of the current layer including an inter-layer reference picture of a reference layer of the current layer;
generating a prediction block of a current block of the current slice by referring to at least one of a plurality of reference pictures included in the reference picture list; and
generating a reconstructed block of the current block of the current slice by using the prediction block,
The inter-layer reference picture is marked only as a long-term reference picture, and is added to the reference picture list for inter-layer prediction,
The reference picture list of the current block includes an inter-layer reference picture at an arbitrary position of the reference picture list according to reference picture list information indicating the position of the inter-layer reference picture.
The method of claim 1,
The step of generating a prediction block of the current block comprises:
and generating the prediction block of the current block by using motion information derived using motion information of at least one of neighboring blocks and collocated blocks of the current block.
3. The method of claim 2,
The call block is
The video decoding method, characterized in that the block in the reference picture included in the reference picture list.
3. The method of claim 2,
The call block is
The image decoding method according to claim 1, wherein the external block is located outside a block existing at a position spatially corresponding to the current block in the reference picture included in the reference picture list.
In the video encoding method supporting a plurality of layers, the method is performed by a video encoding apparatus,
generating a reference picture list of a current slice of the current layer including an inter-layer reference picture of a reference layer of the current layer; and
Encoding information of the reference picture list,
The inter-layer reference picture is displayed only as a long-term reference picture, and is added to the reference picture list for inter-layer prediction,
The reference picture list of the current block includes an inter-layer reference picture at an arbitrary position in the reference picture list according to reference picture list information indicating the position of the inter-layer reference picture.
A computer-readable recording medium storing a bitstream generated by a video encoding method, the video encoding method comprising:
generating a reference picture list of a current slice of the current layer including an inter-layer reference picture of a reference layer of the current layer; and
Encoding information of the reference picture list,
The inter-layer reference picture is displayed only as a long-term reference picture, and is added to the reference picture list for inter-layer prediction,
The reference picture list of the current block includes an inter-layer reference picture at an arbitrary position in the reference picture list according to reference picture list information indicating the position of the inter-layer reference picture.

Images