KR102222311B1 - Video decoding method and apparatus using the same - Google Patents

Abstract

A video decoding method according to an embodiment of the present invention includes: determining a filter type of determining a type of a filter to be applied to a first hierarchical picture that can be referenced by a second hierarchical picture to be decoded; An application target determining step of determining an application target to which the filter is to be applied in the first hierarchical picture; Performing filtering on an object to which a filter is to be applied based on the determined filter type; It may include the step of adding the filtered first layer picture to the second layer reference picture list. Accordingly, a method for decoding an image and an apparatus using the same for reducing a prediction error in an upper layer and improving encoding efficiency are provided.

Description

Video decoding method and apparatus using the same {VIDEO DECODING METHOD AND APPARATUS USING THE SAME}

The present invention relates to an image encoding and decoding process, and more particularly, to an image encoding and decoding method and apparatus supporting a plurality of layers in a bitstream.

Recently, as broadcasting services having high definition (HD) resolution are expanded not only domestically but also globally, many users are getting accustomed to high-resolution and high-definition images, and accordingly, many organizations are spurring the development of next-generation video devices. In addition, as interest in UHD (Ultra High Definition) having a resolution of 4 times or more than HDTV in addition to HDTV is increasing, a compression technology for higher resolution and high quality images is required.

For image compression, an inter prediction technology that temporally predicts a pixel value included in a current picture from a previous and/or subsequent picture, and predicts a pixel value included in the current picture using pixel information in the current picture. An intra prediction technique, an entropy encoding technique in which a short code is assigned to a symbol with a high frequency of appearance, and a long code is assigned to a symbol with a low frequency of occurrence, may be used.

Image compression technology includes a technology that provides a constant network bandwidth under a limited operating environment of hardware without considering a flexible network environment. However, a new compression technique is required to compress image data applied to a network environment in which the bandwidth changes frequently, and for this purpose, a scalable video encoding/decoding method may be used.

The present invention provides a method and apparatus for encoding/decoding an image using inter-layer filtering.

In addition, in the present invention, in adding a picture of a lower layer to a reference picture list of an upper layer, by adaptively applying at least one filter to a picture of the lower layer and then adding it, an image that reduces prediction errors in the upper layer and improves encoding efficiency. It provides a method of decoding and an apparatus using the same.

Accordingly, a method of decoding an image and an apparatus using the same, which improves encoding efficiency and does not increase a reference picture list, are provided.

According to an embodiment of the present invention, a method of decoding an image supporting a plurality of layers includes the step of determining a filter type of determining a type of a filter to be applied to a first layer picture that can be referenced by a second layer picture to be decoded; An application target determining step of determining an application target to which the filter is to be applied in the first hierarchical picture; Performing filtering on an object to which a filter is to be applied based on the determined filter type; It may include the step of adding the filtered first layer picture to the second layer reference picture list.

In the determining of the filter type, it may be determined that a predetermined fixed filter is applied to the first hierarchical picture.

The fixed filter is a basic filter having a preset filter coefficient set, and when the basic filter is applied, filtering may not be performed on samples at integer positions.

The fixed filter is a replacement filter having a preset filter coefficient set, and when the replacement filter is applied, filtering may be performed on a sample at an integer position.

The replacement filter may be applied to the first hierarchical picture to which the basic filter is applied.

The determining of the filter type may further include receiving and decoding a flag signal indicating whether to apply the replacement filter.

In the determining of the filter type, it may be determined that one or more filters are adaptively selected and applied to a corresponding picture of the first layer.

In the filter type determination step, it is determined to apply a basic filter having a preset filter coefficient set to the first hierarchical picture, and whether to apply an alternative filter different from the basic filter for samples at integer positions of the first hierarchical picture. Determining whether to apply the replacement filter, and the step of determining whether to apply the replacement filter may receive and decode a flag signal indicating whether to apply the replacement filter.

In the filter type determination step, it is determined to apply a basic filter having a preset filter coefficient set to the first hierarchical picture, and whether to apply an alternative filter different from the basic filter for samples at integer positions of the first hierarchical picture. The step of determining whether to apply the replacement filter may include: calculating a sample correlation for each block unit of a predetermined size with respect to the first hierarchical picture; And determining whether to apply the replacement filter to the first hierarchical picture based on the correlation.

The step of determining whether to apply the replacement filter may include horizontal activity in the block unit ?Ч硫痴? When the block activity based on the directional activity exceeds a predetermined threshold, it may be determined to apply the replacement filter.

According to another embodiment of the present invention, an apparatus for decoding an image supporting a plurality of layers includes: a first layer decoder for decoding a first layer picture that can be referenced by a second layer picture to be decoded; A filtering unit that determines a type of filter to be applied to the first hierarchical picture, determines an application object to which the filter is applied in the first hierarchical picture, and performs filtering on a filter application object based on the determined filter type; It may include a prediction unit that adds the filtered first hierarchical picture to the second hierarchical reference picture list.

According to an embodiment of the present invention, a method and apparatus for encoding/decoding an image using inter-layer filtering are provided.

In addition, according to an embodiment of the present invention, in adding a picture of a lower layer to a reference picture list of an upper layer, by adaptively applying one or more filters to the picture of the lower layer and then adding them, the prediction error in the upper layer is reduced. A video decoding method and an apparatus using the same are provided to reduce and improve coding efficiency.

Accordingly, there is provided a video decoding method and an apparatus using the same, which improves encoding efficiency and does not increase a reference picture list.

1 is a block diagram illustrating a configuration of an image encoding apparatus according to an embodiment.
2 is a block diagram illustrating a configuration of an image decoding apparatus according to an embodiment.
3 is a conceptual diagram schematically showing an embodiment of a scalable video coding structure using multiple layers to which the present invention can be applied.
4 is a diagram illustrating an example of configuring a reference picture list.
5 is a control flowchart illustrating an image processing method according to the present invention.
6 is a diagram showing a multilayer picture according to the present invention.
7 is a diagram illustrating block samples for calculating horizontal and vertical activity according to the present invention.
8 is a diagram illustrating POCs of pictures of a first layer and a second layer according to the present invention.
9 is a diagram illustrating a reference picture list according to the present invention.
10 is a diagram illustrating a reference picture list for a P slice according to an embodiment of the present invention.
11 is a diagram illustrating a reference picture list for a B slice according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In describing the embodiments of the present specification, when it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present specification, a detailed description thereof will be omitted.

When a component is referred to as being “connected” or “connected” to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. It should be. In addition, the description of "including" a specific configuration in the present invention does not exclude configurations other than the corresponding configuration, and means that additional configurations may be included in the scope of the implementation of the present invention or the scope of the technical idea of the present invention.

Terms such as first and second may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element.

In addition, the constituent units shown in the embodiments of the present invention are shown independently to represent different characteristic functions, and does not mean that each constituent unit is formed of separate hardware or a single software constituent unit. That is, each constituent part is listed and included as a respective constituent part for convenience of explanation, and at least two constituent parts of each constituent part are combined to form one constituent part, or one constituent part may be divided into a plurality of constituent parts to perform a function. Integrated embodiments and separate embodiments of the components are also included in the scope of the present invention as long as they do not depart from the essence of the present invention.

In addition, some of the components are not essential components that perform essential functions in the present invention, but may be optional components only for improving performance. The present invention can be implemented by including only components essential to implement the essence of the present invention excluding components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement Also included in the scope of the present invention.

1 is a block diagram illustrating a configuration of an image encoding apparatus according to an embodiment. A scalable video encoding/decoding method or apparatus may be implemented by a general video encoding/decoding method or apparatus that does not provide scalability, and the block diagram of FIG. 1 is scalable. It shows an embodiment of an image encoding apparatus that can be the basis of a video encoding apparatus.

Referring to FIG. 1, the image encoding apparatus 100 includes a motion prediction unit 111, a motion compensation unit 112, an intra prediction unit 120, a switch 115, a subtractor 125, and a transform unit 130. , A quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference image buffer 190.

The image encoding apparatus 100 may encode an input image in an intra mode or an inter mode and output a bitstream. Intra prediction means intra prediction, and inter prediction means inter prediction. In the intra mode, the switch 115 is switched to intra, and in the inter mode, the switch 115 is switched to inter. After generating a prediction block for an input block of an input image, the image encoding apparatus 100 may encode a difference between the input block and the prediction block.

In the case of the intra mode, the intra prediction unit 120 may generate a prediction block by performing spatial prediction using pixel values of an already coded block around the current block.

In the case of the inter mode, the motion prediction unit 111 may find a region in the reference image stored in the reference image buffer 190 that best matches the input block in the motion prediction process to obtain a motion vector. The motion compensation unit 112 may generate a prediction block by performing motion compensation using a motion vector and a reference image stored in the reference image buffer 190.

The subtractor 125 may generate a residual block based on a difference between the input block and the generated prediction block. The transform unit 130 may transform the residual block and output a transform coefficient. In addition, the quantization unit 140 may quantize the input transform coefficient according to a quantization parameter and output a quantized coefficient.

The entropy encoding unit 150 entropy-encodes a symbol according to a probability distribution based on values calculated by the quantization unit 140 or an encoding parameter value calculated during an encoding process, and outputs a bitstream. can do. The entropy encoding method is a method of receiving symbols having various values, removing statistical redundancy, and expressing them as a sequence of decodeable binary numbers.

Here, the symbol means a syntax element to be encoded/decoded, a coding parameter, a value of a residual signal, and the like. An encoding parameter is a parameter necessary for encoding and decoding, and may include information that can be inferred during encoding or decoding as well as information encoded by an encoder and transmitted to the decoder, such as a syntax element. When encoding or decoding an image It means the information you need. Encoding parameters include values such as intra/inter prediction mode, motion/motion vector, reference image index, coded block pattern, residual signal presence, transform coefficient, quantized transform coefficient, quantization parameter, block size, block division information, etc., or Can include statistics. In addition, the residual signal may mean the difference between the original signal and the predicted signal, and a signal in which the difference between the original signal and the predicted signal is transformed, or the difference between the original signal and the predicted signal is transformed and quantized. It can also mean. The residual signal may be referred to as a residual block in block units.

When entropy coding is applied, a small number of bits are allocated to a symbol having a high probability of occurrence and a large number of bits are allocated to a symbol having a low probability of occurrence, thereby expressing the symbol, so that the size of the bit string for the symbols to be encoded is reduced. Can be reduced. Accordingly, compression performance of image encoding may be improved through entropy encoding.

For entropy encoding, encoding methods such as exponential golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be used. For example, the entropy encoding unit 150 may store a table for performing entropy encoding such as a variable length encoding (VLC) table, and the entropy encoding unit 150 may store the stored variable length encoding Entropy coding can be performed using the (VLC) table. In addition, the entropy encoder 150 derives a binarization method of a target symbol and a probability model of a target symbol/bin, and then performs entropy encoding using the derived binarization method or a probability model. You may.

The quantized coefficient may be inverse quantized by the inverse quantization unit 160 and may be inversely transformed by the inverse transform unit 170. The inverse quantized and inverse transformed coefficients are added to the prediction block through the adder 175, and a reconstructed block may be generated.

The restoration block passes through the filter unit 180, and the filter unit 180 applies at least one or more of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to a restoration block or a reconstructed picture. can do. The reconstructed block that has passed through the filter unit 180 may be stored in the reference image buffer 190.

2 is a block diagram illustrating a configuration of an image decoding apparatus according to an embodiment. As described above in FIG. 1, the scalable video encoding/decoding method or apparatus may be implemented by a general video encoding/decoding method or apparatus that does not provide scalability, and the block diagram of FIG. 2 is a scalable video decoding method. It shows an embodiment of an image decoding apparatus that can be the basis of the apparatus.

Referring to FIG. 2, the image decoding apparatus 200 includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra prediction unit 240, a motion compensation unit 250, and a filter unit. 260 and a reference image buffer 270.

The image decoding apparatus 200 may receive a bitstream output from an encoder, perform decoding in an intra mode or an inter mode, and output a reconstructed image, that is, a reconstructed image. In the case of the intra mode, the switch may be switched to intra, and in the case of the inter mode, the switch may be switched to inter. The image decoding apparatus 200 may generate a reconstructed block, that is, a reconstructed block, by obtaining a residual block reconstructed from an input bitstream, generating a prediction block, and adding the reconstructed residual block and the prediction block.

The entropy decoder 210 may entropy-decode the input bitstream according to a probability distribution to generate symbols including symbols in the form of quantized coefficients. The entropy decoding method is a method of generating each symbol by receiving a string of binary numbers. The entropy decoding method is similar to the entropy encoding method described above.

The quantized coefficients are inverse quantized in the inverse quantization unit 220 and inversely transformed in the inverse transform unit 230, and as a result of the inverse quantization/inverse transformation of the quantized coefficients, a reconstructed residual block may be generated.

In the case of the intra mode, the intra prediction unit 240 may generate a prediction block by performing spatial prediction using pixel values of an already coded block around the current block. In the case of the inter mode, the motion compensation unit 250 may generate a prediction block by performing motion compensation using a motion vector and a reference image stored in the reference image buffer 270.

The reconstructed residual block and prediction block are added through an adder 255, and the added block passes through a filter unit 260. The filter unit 260 may apply at least one of a deblocking filter, SAO, and ALF to a reconstructed block or a reconstructed picture. The filter unit 260 outputs a reconstructed image, that is, a reconstructed image. The reconstructed image may be stored in the reference image buffer 270 and used for inter prediction.

An entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra prediction unit 240, a motion compensation unit 250, and a filter unit 260 included in the image decoding apparatus 200 And components of the reference image buffer 270 directly related to decoding of an image, for example, an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra prediction unit 240, and motion compensation. The unit 250, the filter unit 260, and the like may be separated from other components and expressed as a decoding unit or a decoding unit.

In addition, the image decoding apparatus 200 may further include a parsing unit (not shown) for parsing information related to the encoded image included in the bitstream. The parsing unit may include an entropy decoding unit 210 or may be included in the entropy decoding unit 210. This parsing unit may also be implemented as a component of the decoding unit.

3 is a conceptual diagram schematically showing an embodiment of a scalable video coding structure using multiple layers to which the present invention can be applied. In FIG. 3, a group of pictures (GOP) represents a picture group, that is, a group of pictures.

A transmission medium is required to transmit image data, and its performance is different for each transmission medium according to various network environments. A scalable video coding method may be provided for application to such various transmission media or network environments.

The scalable video coding method is a coding method that improves encoding/decoding performance by removing inter-layer redundancy by utilizing texture information, motion information, and residual signals between layers. The scalable video coding method can provide various scalability in terms of spatial, temporal, and image quality according to surrounding conditions such as a transmission bit rate, a transmission error rate, and system resources.

Scalable video coding may be performed using a multiple layers structure to provide a bitstream applicable to various network conditions. For example, the scalable video coding structure may include a base layer that compresses and processes image data using a general image encoding method, and compresses image data by using both encoding information of the base layer and a general image encoding method. It may include an enhancement layer to process.

Here, the layer is an image and a bit divided based on spatial (e.g., image size), temporal (e.g., encoding order, image output order, frame rate), quality, complexity, etc. It means a set of streams (bitstream). In addition, a base layer may refer to a lower layer or a reference layer, and an enhancement layer may refer to an upper layer. Also, multiple layers may have dependencies on each other.

Referring to FIG. 3, for example, the base layer may be defined as SD (standard definition), a frame rate of 15 Hz, and a 1 Mbps bit rate, and the first enhancement layer is a high definition (HD), a frame rate of 30 Hz, and a bit rate of 3.9 Mbps. The second enhancement layer may be defined as 4K-UHD (ultra high definition), a frame rate of 60 Hz, and a bit rate of 27.2 Mbps. The format, frame rate, bit rate, etc. are one embodiment and may be differently determined as necessary. Also, the number of layers to be used is not limited to this embodiment and may be determined differently according to circumstances.

For example, if the transmission bandwidth is 4 Mbps, the frame rate of the first enhancement layer HD may be reduced to transmit at 15 Hz or less. The scalable video coding method may provide temporal, spatial, and quality scalability by the method described above in the embodiment of FIG. 3.

The scalable video coding has the same meaning as the scalable video coding from the viewpoint of encoding and the scalable video decoding from the viewpoint of decoding.

The present invention relates to an en-/de-coding process of an image including a plurality of layers or views, wherein the plurality of layers or views are first, second, third, and It can be expressed as an nth layer or viewpoint. In the following description, a picture in which a first layer and a second layer exist is described as an example, and the same method may be applied to a higher layer or view point. In addition, the first layer may be expressed as a base layer and the second layer may be expressed as an upper layer. In addition, the first layer may be expressed as a reference layer, and the second layer may be expressed as an enhancement layer.

The picture/block of the first layer corresponding to the picture/block of the second layer may be changed according to the size of the picture/block of the second layer. That is, when the size of the picture/block of the first layer is smaller than the picture/block of the second layer, scaling may be performed using a method such as up-sampling and re-sampling.

In addition, the picture of the first layer may be added to the reference picture list of the second layer and used for encoding/decoding the image of the second layer. In this case, the second layer may perform prediction and encoding/decoding by using the first layer image in the reference picture list like normal inter prediction.

The size of a block for encoding/decoding may be an NxN-shaped square such as 4x4, 8x8, 16x16, 32x32, 64x64, or an NxM-shaped rectangle such as 4x8, 16x8, 8x32, and the unit of the block is a coding block (CB: Coding Block). ), a prediction block (PB: Prediction Block), and a transform block (TB: Transform Block), and each may have a different size.

Hereinafter, a prediction block, that is, a prediction signal, of a block (hereinafter referred to as a current block or a target block) to be encoded and decoded of a higher layer among the encoding and decoding methods of scalable video, that is, an image using a multilayer structure Let's look at how to do it. The content (method or apparatus) of the following invention can be applied equally to an encoder and a decoder in general.

In general, inter prediction may use at least one of a previous picture or a subsequent picture of the current picture as a reference picture, and prediction on a current block may be performed based on the reference picture. A picture used for prediction of the current block is called a reference picture or a reference frame.

The reference picture is specified by a reference picture index (refIdx), and a predetermined region in the reference picture is specified as a reference block through a motion vector.

Inter prediction may generate a prediction block for the current block by selecting a reference picture and a reference block corresponding to the current block in the reference picture.

In inter prediction, the encoding apparatus and the decoding apparatus may derive motion information of a current block and then perform inter prediction and/or motion compensation based on the derived motion information. In this case, the encoding apparatus and the decoding apparatus move a collocated block corresponding to the current block in the reconstructed neighboring block and/or the already reconstructed collocated picture. By using information, encoding/decoding efficiency can be improved.

Here, the reconstructed neighboring block is a block in the current picture that has already been encoded and/or decoded and reconstructed, and may include a block adjacent to the current block and/or a block located at an outer corner of the current block. In addition, the encoding device and the decoding device may determine a predetermined relative position based on a block existing at a position spatially corresponding to the current block in the collocated picture, and the determined relative position (spatially correspond to the current block). The call block may be derived based on a location inside and/or outside of a block existing at a location where the call is made. Here, as an example, the collocated picture may correspond to one picture from among the reference pictures included in the reference picture list.

In inter prediction, a prediction block may be generated such that a residual signal with a current block is minimized and a motion vector size is also minimized.

Meanwhile, the motion information derivation method may vary according to the prediction mode of the current block. Prediction modes applied for inter prediction may include Advanced Motion Vector Predictor (AMVP), merge, and the like.

For example, when AMVP (Advanced Motion Vector Predictor) is applied, the encoding device and the decoding device may generate a motion vector candidate list using a motion vector of a reconstructed neighboring block and/or a motion vector of a collocated block. That is, a motion vector of a reconstructed neighboring block and/or a motion vector of a collocated block may be used as a motion vector candidate. The encoding apparatus may transmit a predicted motion vector index indicating an optimal motion vector selected from among motion vector candidates included in the list to the decoding apparatus. In this case, the decoding apparatus may select a predicted motion vector of a current block from among motion vector candidates included in a motion vector candidate list using the motion vector index.

The encoding apparatus may obtain a motion vector difference (MVD) between the motion vector of the current block and the predicted motion vector, encode the motion vector difference, and transmit it to the decoding apparatus. In this case, the decoding apparatus may decode the received motion vector difference, and may derive the motion vector of the current block through the sum of the decoded motion vector difference and the predicted motion vector.

The encoding apparatus may also transmit a reference picture index indicating a reference picture to the decoding apparatus.

The decoding apparatus may predict a motion vector of a current block using motion information of a neighboring block, and derive a motion vector for the current block using a residual received from the encoding apparatus. The decoding apparatus may generate a prediction block for the current block based on the derived motion vector and reference picture index information received from the encoding apparatus.

As another example, when merge is applied, the encoding apparatus and the decoding apparatus may generate a merge candidate list using motion information of a reconstructed neighboring block and/or motion information of a collocated block. That is, when there is motion information of a reconstructed neighboring block and/or a collocated block, the encoding device and the decoding device may use it as a merge candidate for the current block.

The encoding apparatus may select a merge candidate capable of providing optimal encoding efficiency from among merge candidates included in the merge candidate list as motion information for the current block. In this case, a merge index indicating the selected merge candidate may be included in the bitstream and transmitted to the decoding apparatus. The decoding apparatus may select one of merge candidates included in a merge candidate list using the transmitted merge index, and determine the selected merge candidate as motion information of the current block. Therefore, when the merge mode is applied, motion information of a reconstructed neighboring block and/or a collocated block may be used as it is as motion information of the current block. The decoding apparatus may reconstruct the current block by adding the prediction block and the residual transmitted from the encoding apparatus.

In the AMVP and merge modes described above, motion information of a reconstructed neighboring block and/or motion information of a collocated block may be used to derive motion information of a current block.

In the case of the skip mode, which is one of the other modes used for inter prediction, information of neighboring blocks may be used as it is for the current block. Therefore, in the case of the skip mode, the encoding apparatus does not transmit syntax information such as residual to the decoding apparatus other than information indicating which block motion information is to be used as motion information of the current block.

The encoding apparatus and the decoding apparatus may generate a prediction block of the current block by performing motion compensation on the current block based on the derived motion information. Here, the prediction block may mean a motion-compensated block generated as a result of performing motion compensation on the current block. In addition, a plurality of motion-compensated blocks may constitute one motion-compensated picture.

The decoding apparatus may check the skip flag, merge flag, etc. received from the encoding apparatus for motion information necessary for inter prediction of the current block, such as motion vector and reference picture index, and derive correspondingly thereto.

A processing unit in which prediction is performed and a processing unit in which a prediction method and specific content are determined may be different from each other. For example, a prediction mode may be determined in units of prediction blocks, and prediction may be performed in units of transform blocks, or a prediction mode may be determined in units of prediction blocks, and intra prediction may be performed in units of transform blocks.

Pictures encoded/decoded prior to the current picture may be stored in a memory (eg, Decoded Picture Buffer: DPB) and used for prediction of a current block (current picture). The list of pictures available for inter prediction of the current block is maintained as a reference picture list.

The P slice is a slice that is decoded through intra prediction or inter prediction using at most one motion vector and one reference picture. The B slice is a slice that is decoded through intra prediction or inter prediction using up to two motion vectors and two reference pictures. In this case, the reference picture includes a short term reference picture and a long term reference picture. The picture may be specified as a POC (Picture Order Count) indicating the display order, short-term reference pictures may be pictures with a small difference in POC from the current picture, and long-term reference pictures are pictures having a large difference in POC from the current picture. I can.

Reference picture list 0 (reference picture list 0, referred to as'L0' for convenience of description below) is a reference picture list used for inter prediction of a P slice or a B slice. Reference picture list 1 (referred to as'L1' for convenience of description below) is used for inter prediction of B slices. Accordingly, L0 is used for inter prediction of a block of P slice performing unidirectional prediction, and L0 and L1 are used for inter prediction of a block of B slice performing bidirectional prediction.

When decoding a P slice and a B slice through inter prediction, the decoding apparatus constructs a reference picture list. A reference picture used for inter prediction is designated through a reference picture list. The reference picture index is an index indicating a reference picture on the reference picture list.

The reference picture list may be configured based on a reference picture set transmitted from an encoding device. The reference picture set may include a POC of a picture used as a reference picture and a flag (used_by_curr_pic_s0_flag) indicating whether the picture is directly referenced. Reference pictures constituting the reference picture list may be stored in a memory (eg, DPB). Pictures stored in the memory (pictures encoded/decoded before the current picture) are managed by the encoding device and the decoding device.

The reference picture set may include a short-term reference picture set including a short-term reference picture and a long-term reference picture set including a long-term reference picture, and an initial reference picture list is constructed based on the short-term reference picture set and the long-term reference picture set. Can be

4 is a diagram illustrating an example of configuring a reference picture list.

The reference pictures are a first short-term reference picture set (RefPicSetStCurr0) composed of reference pictures (Ref 1, Ref 2) smaller than the POC of the current picture (Curr) based on the current picture, and reference pictures larger than the POC of the current picture ( It may be classified into a second short-term reference picture set (RefPicSetStCurr1) composed of Ref 3 and Ref 4) and a long-term reference picture set (RefPicSetLtCurr) composed of long-term reference pictures Ref LT1 and Ref LT2.

In this case, the first short-term reference picture set (RefPicSetStCurr0) is composed of pictures with a used_by_curr_pic_s0_flag value of 1 (delta_poc_s0 with used_by_curr_pic_s0_flag=1), and the second short-term reference picture set (RefPicSetStCurr1 with a value of 1 used_by_curr_pics is also used_pics_flag1). Becomes (delta_poc_s1 with used_by_curr_pic_s1_flag=1).

In this way, an initial reference picture list may be constructed from a set of reference picture sets having different properties.

Reference picture list 0, i.e., L0, is configured in the order of a first short-term reference picture set (RefPicSetStCurr0), a second short-term reference picture set (RefPicSetStCurr1), and a long-term reference picture set (RefPicSetLtCurr), as shown in FIG.

On the other hand, reference picture list 1, that is, L1, is configured in the order of a second short-term reference picture set (RefPicSetStCurr1), a first short-term reference picture set (RefPicSetStCurr0), and a long-term reference picture set (RefPicSetLtCurr).

The number of reference pictures that may be included in the reference picture list may be determined based on information transmitted from the encoding device. For example, after configuring a reference picture list, the encoding apparatus determines the number of reference pictures to be used, and sets information on the number of reference pictures to be used (e.g., num_ref_idx_lX_default_active_minus1, X=0 or 1) as a sequence parameter set (SPS). It can be transmitted to the decoding device as a syntax element of ). The decoding apparatus may use the number of reference pictures specified as a value obtained by adding 1 to the received num_ref_idx_lX_default_active_minus1 as a default value in the current sequence.

In addition, in the case of designating the number of reference pictures for each picture or slice, the encoding apparatus uses separate information indicating the number of reference pictures (e.g., num_ref_idx_l1_active_minus1, X=0 or 1) as a picture parameter set (PPS). ) Or through a slice header. The decoding apparatus may apply a value specified as a value obtained by adding 1 to the received num_ref_idx_l1_active_minus1 as the number of reference pictures for the current picture or the current slice.

When performing inter prediction, motion compensation may be performed using a reference picture specified in the reference picture list configured as described above.

In a multi-layer structure that provides spatial scalability or multi-view scalability, a reference picture of an upper layer may be composed of reference pictures for the same layer and an inter-layer reference picture.

In this case, signaling for an inter-layer reference picture may be performed through information identifying a layer and information identifying a reference picture. For example, the value of nuh_layer_id, which is the j-th layer identifier (i is greater than j), which exists in the same access unit as the current picture of the i-th layer and is included in the NAL (Network Abstraction Layer) unit header and transmitted is the current picture. If it is the same as the RefPiclayerId value for, it may be determined that the corresponding picture is used as a reference picture for the current picture. The inter-layer reference picture can be displayed as a long-term reference picture.

RefPicLayerId is a value that can be signaled as a syntax element inter_layer_pred_layer_idc included in a slice header, and refers to a layer that the current layer refers to for inter-layer prediction.

Meanwhile, when adding a picture of a lower layer to the reference picture list of an upper layer, one fixed filter may be applied to the picture of the lower layer and then added to the list of reference pictures of the higher layer. However, in this case, there may be a problem that the coding efficiency is deteriorated.

In addition, when two filters are applied to a picture of a lower layer, since one or more pictures may be added to the reference picture list, information configuring or signaling the reference picture list may increase.

Accordingly, in applying a filter to a picture of a lower layer, the present invention adaptively selects/applies one or more filters to a predetermined unit unit, and adds one lower layer picture to the upper layer reference picture list, thereby improving coding efficiency. It is characterized in that it improves and does not increase the complexity of constructing a reference picture list.

5 is a control flowchart illustrating an image processing method according to the present invention. More specifically, FIG. 5 is a diagram for explaining a method of encoding and decoding a multilayer image using a picture of a first layer when sub-decoding a picture of a second layer is performed. Accordingly, the description of FIG. 5 can be applied to both a decoding and encoding method of an image.

In order to filter the first hierarchical picture, first, the encoder and the decoder determine a filter type for the first hierarchical picture (S510). The first layer picture is decoded before the filter type is determined, and the first layer picture and the second layer picture may be decoded by different configurations or modules, respectively. The encoder and the decoder may include a decoder having the configurations of FIGS. 1 and 2 for decoding a first layer picture and a decoder having the configurations of FIGS. 1 and 2 for decoding a second layer picture.

6 is a diagram showing a multilayer picture according to the present invention. As shown, the picture of the second layer becomes a target picture to be encoded and decoded, and the picture of the first layer is a corresponding picture corresponding to the target picture, and may be a reference picture of the second layer picture.

The target picture and the corresponding picture have the same POC, and for example, the POC of the target picture and the corresponding picture may be 4. For prediction of the target picture, a corresponding picture having a POC of 4 and a picture of a second layer having a POC of 0 or POC 8 may be used.

The size of the picture to be sub/decoded in the second layer and the picture corresponding to the first layer may be the same or different. Even if the size of the target picture of the second layer and the corresponding picture of the first layer are the same, characteristics of signals may be different. Therefore, up-sampling or re-sampling is applied to the reconstructed picture of the first layer to increase the prediction efficiency by making the picture sizes of the two layers the same or by changing the characteristics of the signal. I can make it.

A predetermined filter may be applied for such sampling, and the filter may be different according to a luminance component or a chroma component.

The encoder and decoder may determine the filter type of the first layer picture as one fixed filter, or may adaptively select one or more filters and apply them to the first layer picture.

When an encoder and a decoder apply a fixed filter to a first layer picture, a default interpolation filter may be applied to a corresponding picture of the first layer.

The default filter may have a set of filters for luminance/color difference signals as shown in Tables 1 and 2 below, and the phase and filter coefficients applied according to the picture size ratio between layers may be different. have.

Table 1 shows the 16 phase resampling filter coefficients for the luminance signal, and Table 2 shows the 16 phase resampling filter coefficients for the color difference signal.

In this case, the phase '0' may mean a filter coefficient for a sample at an integer position, and the same result as not applying a filter may be obtained.

Alternatively, the encoder and the decoder may apply an alternative filter as a predetermined fixed filter to the corresponding picture of the first layer or the picture to which the basic filter is applied.

The replacement filter may have a filter set that includes one or more filter coefficients, similar to the basic filter described above. In this case, filter coefficients may be determined by the encoder and then signaled by the decoder.

The replacement filter may be applied to a picture to which the basic filter is applied, and may mean a filter for a sample at an integer position corresponding to the phase “0” of the basic filter. In this case, the replacement filter may have a fixed filter coefficient such as [-1, 3, 12, 3, ?1]/16.

In the case of a video supporting SNR scalability, filtering may not be performed because the sizes of the pictures of the second layer and the first layer are the same. However, according to the present invention, a replacement filter may be applied to a picture corresponding to the first layer in order to change the characteristics of the signal of the first layer.

At this time, the flag indicating whether to use the replacement filter is through at least one of a video parameter set, a sequence parameter set, a picture parameter set, and a slice header. Can be signaled.

According to another embodiment, the encoder and the decoder may adaptively select one or more filters other than a fixed filter as a filter type and apply them to a corresponding picture of the first layer.

For example, in a video supporting spatial scalability, an encoder and a decoder may always apply a basic filter to a picture of a first layer and then always apply a replacement filter to an integer position sample. In this case, even if there is no additional signaling from the encoder, the decoder can filter the picture by adaptively applying two of the basic filter and the replacement filter, respectively.

Still another, for example, an encoder and a decoder applies a basic filter to a picture of a first layer in a video supporting spatial scalability. Then, the decoder can determine whether to apply the replacement filter according to the flag signaled from the encoder.

The encoder may apply the basic filter to the picture of the first layer and then apply the replacement filter to the integer position samples. After applying the replacement filter to the integer position sample, it is possible to determine whether to apply the replacement filter by calculating at least one of Rate Distortion Optimization (RDO), Sum of Absolute Hadamard Transformed Differences (SATD), and Sum of Absolute Difference (SAD). . If applying the replacement filter according to the result of the operation has a better effect than when not applying the replacement filter, the encoder may decide to apply the replacement filter to the integer position sample. The signal indicating whether to apply the replacement filter, i.e., a flag indicating whether to apply the replacement sample, is a video parameter set, a sequence parameter set, a picture parameter set, and a slice header. It may be signaled through at least one of (Slice header) and the like.

According to another embodiment, an encoder and a decoder apply a basic filter to a picture of a first layer included in a video supporting spatial scalability, and a first layer included in a video to which quality scalability is applied. An alternative filter can be applied to the pictures of.

Further, according to another embodiment, an encoder and a decoder may adaptively select and apply a filter based on a correlation between samples. That is, after calculating the correlation between samples in a specific block unit such as 4x4, 8x8, 16x16, 32x32, 64x64, etc., the replacement filter can be adaptively applied by comparing it with a specific threshold. In this case, the correlation between the samples may be calculated based on a first layer-corresponding picture (restored picture) or a picture to which a basic filter is applied, and the threshold value may be set differently depending on whether the scalability is spatially perceived quality. have.

For example, when the block size is 4x4 as shown in FIG. 7, the encoder and decoder can obtain vertical activity (VA) and horizontal activity (HA) using Equation 1. have.

<Equation 1>

At this time, i and j are 0 and 2.

When vertical and horizontal activity is calculated based on Equation 1, block activity may be calculated using Equation 2.

<Equation 2>

Block activity = (VA + HA) >> 2

The encoder and decoder may apply the replacement filter to the target block when the block activity is less than the threshold value, and may not apply the replacement filter if the block activity is not less than the threshold value.

The process of applying the replacement filter by calculating vertical activity, horizontal activity, and block activity can be applied equally to blocks of 8x8 or larger.

According to another embodiment, when the size of the block is NxN, the encoder and the decoder apply a replacement filter to the target block when the samples in the vertical direction and the horizontal direction satisfy Equation 3 below, and do not apply the replacement filter when they are not satisfied. May not.

<Equation 3>

Abs(p[?1][N>>1-1] + p[N-1][N>>1-1]? 2*p[N>>1-1][N>>1-1] ) <(1<<( BitDepthY? 5))

Abs(p[N>>1-1][?1] + p[N>>1-1][N-1]? 2*p[N>>1-1][N>>1-1] ) <(1<<( BitDepthY? 5))

In this case, p[x][y] indicates a sample value at the (x, y) position, Abs indicates an absolute value, and BitDepthY may indicate a bit depth of the luminance signal.

As described above, according to the present invention, when filtering a first layer picture, a fixed filter of either the basic filter or the replacement filter may be applied, and one or both of the basic filter and the replacement filter may be selectively applied. I can.

When the filter type for the picture of the first layer is determined, the encoder and the decoder may determine a target to which the filter is applied (S520).

The encoder and decoder apply the fixed filter or the adaptive filter for each corresponding picture, slice, coding unit (CU), prediction unit (PU), transform unit (TU), and predetermined unit (NxN) of the first layer image. can do.

For example, the encoder may determine whether to use a replacement filter in units of pictures, and signal a flag indicating whether to use the replacement filter through PPS. The decoder may determine whether to use an alternative filter according to a signaled flag and perform filtering on a picture basis.

According to another example, the encoder may determine whether to use a replacement filter in units of slices, and signal a flag indicating whether to use it through a slice header. The decoder may determine whether to use an alternative filter according to a signaled flag and perform filtering in slice units.

Alternatively, the encoder and the decoder may adaptively apply a basic filter or a replacement filter after analyzing a correlation between samples for each prediction unit. In this case, since the correlation between the samples is individually analyzed by the encoder and the decoder and filtering may be performed adaptively, additional signaling as to whether or not to filter may not be required.

For example, a predetermined unit for adaptive filtering may be determined and a basic filter or an alternative filter may be applied in units of a predetermined unit. In this case, the replacement filter may be a filter that performs filtering on a sample at an integer position.

When the predetermined unit is determined to be 8x8, a filter may be adaptively applied after a correlation between samples is calculated in units of 8x8 for the first hierarchical picture.

When the filter type and the filter application target are determined through steps S510 and S520, the encoder and decoder perform filtering on the first layer picture through a combination of one or more filters or application targets (S530). That is, the encoder and decoder perform filtering on the filter to be applied based on the determined filter type.

Thereafter, the encoder and decoder may add the filtered first layer picture to the second layer reference picture list (S540). The first hierarchical picture may be added to the second hierarchical reference picture list by the prediction unit for decoding the second hierarchical picture.

In encoding and decoding the second layer target picture, one or more first layer pictures obtained through filtering may be added to an arbitrary position of the second layer reference picture list 0 or the reference picture list 1 to be used.

In this case, the encoder and decoder may allocate the first layer picture to an arbitrary index in the reference picture list.

8 is a diagram illustrating POCs of pictures of a first layer and a second layer according to the present invention. Referring to FIG. 8, a picture to be encoded and decoded may be a second layer picture having a POC of 2, and the target picture is a corresponding picture of the first layer, a picture having a smaller POC than itself (POC 0, POC 1), and Pictures with a larger POC than themselves (POC 3, POC 4) can be referred to as short-term reference pictures. It is also possible to refer to a long-term reference picture (not shown).

9 is a diagram illustrating a reference picture list according to the present invention. When adding such a reference picture to the reference picture list, the encoder and decoder may add the first hierarchical picture F0 at an arbitrary position as shown in FIG. 9.

For example, as shown in [a] of FIG. 9, when the encoder and the decoder configure the reference picture list 0 (List 0), short-term reference pictures (POC 0, POC 1) having a smaller POC than the POC of the current target picture, A short-term reference picture (POC 3, POC 4) and a first-layer picture (F0) having a larger POC than the current target picture can be sequentially added to the reference picture list 0, and finally, a long-term reference picture (lt) can be added. have.

Similarly, reference picture list 1 (List 1) is a short-term reference picture (POC 3, POC 4) having a larger POC than the current target picture's POC, and a short-term reference picture having a smaller POC than the current target picture's POC (POC 0, POC 1). , A first layer picture (F0), and a long-term reference picture (lt).

According to another embodiment, the encoder and decoder may configure a reference picture list as shown in [b]. Referring to [b] of FIG. 9, reference picture list 0 (List 0) is a short-term reference picture (POC 0, POC 1), a first layer picture (F0), and a current target picture having a smaller POC than that of the current target picture. A short-term reference picture (POC 3, POC 4) having a larger POC than the POC of may be configured in the order of a long-term reference picture (lt).

Similarly, the reference picture list 1 (List 1) is a short-term reference picture (POC 3, POC 4) having a larger POC than the current target picture's POC, and a short-term reference picture having a smaller POC than the current target picture's POC (POC 0, POC 1). , A first layer picture (F0), and a long-term reference picture (lt).

According to another embodiment, an encoder and a decoder may be added at the beginning or end of each list. That is, the reference picture of the second layer corresponding to 1, 0, 3, 4, lt is added to the first layer picture (F0) to List 0, and the first layer picture (F0) obtained through filtering is finally added. Can be added. Also in List 1, a first layer picture F0 may be added after 3, 4, 1, 0, and lt.

Meanwhile, according to another embodiment of the present invention, the number of first layer pictures added may be different for each list, and a filter applied to a picture added to each list may be fixed as a specific filter, or determined by an encoder and signaled to a decoder. You may.

10 is a diagram illustrating a reference picture list for a P slice according to an embodiment of the present invention.

In the case of the P slice, one picture to which the filtering method is applied to the first hierarchical picture F0 as shown in FIG. 10A may be added to the end of the reference picture list 0. In this case, pictures 1, 0, 3, and 4 may be reference pictures of the second layer, and F0 may indicate a picture obtained by filtering the first layer-corresponding picture using one of the above filtering methods.

Meanwhile, in the case of a P slice, the encoder and the decoder refer to two pictures (F0, F1) after filtering the first layer picture after the reference picture of the second layer, as shown in FIG. 10B, as a reference picture list. Can be added to zero. In this case, F0 and F1 may represent pictures obtained by filtering the first layer-corresponding picture using one of the filtering methods described with reference to FIGS. 5 to 7, respectively.

11 is a diagram illustrating a reference picture list for a B slice according to an embodiment of the present invention.

In the case of a B slice, the encoder and decoder may add one picture F0 to which filtering is applied to the first hierarchical picture to the reference picture list 0 or the reference picture list 1 as shown in FIG. 11A. In this case, the first hierarchical picture F0 may represent a picture obtained by filtering the first layer-corresponding picture using one of the filtering methods described with reference to FIGS. 5 to 7.

In addition, in the case of the B slice, the encoder and the decoder refer to the reference picture list 0 (List 0) and the reference picture list 1 (List 1) for two pictures to which filtering is applied to the first hierarchical picture as shown in [b] of FIG. Can be added. In this case, F0 and F1 may represent a picture obtained by filtering the first layer-corresponding picture using one of the filtering methods described with reference to FIGS. 5 to 7, and F0 and F1 may be the same.

For example, when it is determined that the encoder uses an alternative filter through SATD or the like, F0 may be a picture to which a basic filter is applied, and F1 may be a picture to which an alternative filter is applied after applying the basic filter.

In this case, in the case of quality scalability, F0 may be a corresponding picture to which no filter is applied, and F1 may be a picture to which an alternative filter is applied.

According to another embodiment, when constructing a reference picture list for a B slice, the encoder and the decoder include four pictures (F0, F1, F2, F3) to which filtering is applied to the first layer picture as shown in [c] of FIG. ) Can be added to reference picture list 0 and reference picture list 1. In this case, F0, F1, F2, and F3 denote pictures obtained by filtering the first layer-corresponding picture using one of the above-described filtering methods, and one or more of F0, F1, F2, and F3 may be the same.

As described above, according to the present invention, in adding a picture of a lower layer to the reference picture list of an upper layer, by adaptively applying one or more filters to the picture of the lower layer and then adding them, the prediction error in the upper layer is reduced. A video decoding method and an apparatus using the same are provided to improve coding efficiency.

Accordingly, a method of decoding an image and an apparatus using the same, which improves encoding efficiency and does not increase a reference picture list, can be provided.

In the above-described embodiment, the methods are described on the basis of a flowchart as a series of steps or blocks, but the present invention is not limited to the order of steps, and certain steps may occur in a different order or concurrently with those described above. have. In addition, those of ordinary skill in the art understand that the steps shown in the flowchart are not exclusive, other steps are included, or one or more steps in the flowchart may be deleted without affecting the scope of the present invention. You can understand.

The above-described embodiments include examples of various aspects. It is not possible to describe all possible combinations for representing the various aspects, but one of ordinary skill in the art will recognize that other combinations are possible. Accordingly, the present invention will be said to include all other replacements, modifications and changes falling within the scope of the following claims.

100: image encoding device 111: motion prediction unit
112: motion compensation unit 120: intra prediction unit
115: switch 125: subtractor
130: transform unit 140: quantization unit
150: entropy encoding unit 160: inverse quantization unit
170: inverse transform unit 180: filter unit

Claims (33)

delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete In a video decoding method supporting a plurality of layers,
Resampling an inter-layer reference picture of a reference layer referenced by the current picture of the current layer, based on the size of the current picture of the current layer;
Adding a resampled inter-layer reference picture of the reference layer to the reference picture list of the current picture; And
And performing inter prediction on a current block of the current picture by referring to at least one of a plurality of reference pictures included in the reference picture list,
The inter-layer reference picture added to the reference picture list is displayed as a long-term reference picture. The method of claim 21,
When the reference picture list includes a short-term reference picture belonging to the current layer, the inter-layer reference picture is added to the reference picture list after the short-term reference picture. The method of claim 21,
The inter-layer reference picture,
A video decoding method, characterized in that it is displayed as a long-term reference picture and is added to the reference picture list. The method of claim 21,
Motion information used for the inter prediction,
An image decoding method, characterized in that the information is derived using motion information of at least one of spatial and temporal neighboring blocks of the current block. The method of claim 24,
The temporal neighboring block,
And a block in a reference picture included in the reference picture list. The method of claim 24,
The temporal neighboring block,
And an outer block located outside a block existing at a position spatially corresponding to the current block within a reference picture included in the reference picture list. In the video encoding method supporting a plurality of layers,
Resampling an inter-layer reference picture of a reference layer referenced by the current picture of the current layer, based on the size of the current picture of the current layer;
Adding a resampled inter-layer reference picture of the reference layer to the reference picture list of the current picture; And
And performing inter prediction on a current block of the current picture by referring to at least one of a plurality of reference pictures included in the reference picture list,
The inter-layer reference picture added to the reference picture list is displayed as a long-term reference picture. The method of claim 27,
When the reference picture list includes a short-term reference picture belonging to the current layer, the inter-layer reference picture is added to the reference picture list after the short-term reference picture. The method of claim 27,
The inter-layer reference picture,
A video encoding method, characterized in that the picture is displayed as a long-term reference picture and is added to the reference picture list. The method of claim 27,
Motion information used for the inter prediction,
An image encoding method, characterized in that it is derived using motion information of at least one of spatial and temporal neighboring blocks of the current block. The method of claim 30,
The temporal neighboring block,
And a block in a reference picture included in the reference picture list. The method of claim 30,
The temporal neighboring block,
And an outer block located outside a block existing at a position spatially corresponding to the current block within a reference picture included in the reference picture list. A non-transitory computer-readable storage medium storing a bitstream generated by an image encoding method supporting a plurality of layers,
The video encoding method,
Resampling an inter-layer reference picture of a reference layer referenced by the current picture of the current layer, based on the size of the current picture of the current layer;
Adding a resampled inter-layer reference picture of the reference layer to the reference picture list of the current picture; And
And performing inter prediction on a current block of the current picture by referring to at least one of a plurality of reference pictures included in the reference picture list,
And the inter-layer reference picture added to the reference picture list is displayed as a long-term reference picture.

Images