KR20220130067A - Video encoding/decoding method using error-resilient in-loop filter and signaling method relating to the same - Google Patents

Abstract

The present invention relates to a video encoding/decoding method and, more specifically, to a method of performing error-resilient in-loop filtering and a signaling method for the same. According to one embodiment of the present invention, the video decoding method comprises: a step of determining whether a constrained intra prediction (CIP) mode is used; a step of determining decoding parameters of blocks positioned on both sides of a boundary to be filtered; and a step of determining whether to apply an in-loop filter to samples positioned on both sides of the boundary to be filtered based on the decoding parameters of the blocks positioned on both sides of the boundary to be filtered and whether the constrained intra prediction mode is used.

Description

A video encoding/decoding method using an error-tolerant in-loop filter and a signaling method related thereto

The present invention relates to an image encoding/decoding method, and more particularly, to a method for performing in-loop filtering that is robust against errors and a signaling method therefor.

Recently, as broadcasting systems supporting HD (High Definition) resolution have been expanded not only in Korea but also worldwide, many users are getting used to high-resolution and high-definition video, and accordingly, many organizations are spurring the development of next-generation video equipment. . In addition, as interest in Ultra High Definition (UHD), which supports a resolution four times higher than that of HDTV along with HDTV, increases, a compression technology for higher resolution and high-definition images is required.

For image compression, an inter prediction technique that predicts a pixel value included in a current picture from a preceding picture and/or a subsequent picture, and intra prediction that predicts a pixel value using pixel information in a picture technology, an entropy encoding technique in which a short code is assigned to a symbol having a high frequency of occurrence and a long code is assigned to a symbol having a low frequency of occurrence, or the like may be used.

An object of the present invention is to provide a method and apparatus for encoding/decoding an image using in-loop filtering that is robust against errors.

Another object of the present invention is to provide a method of performing in-loop filtering based on whether a constrained intra prediction (CIP) mode is used and encoding parameters of blocks located on both sides of a filtering target boundary.

[1] According to an embodiment of the present invention, an image encoding method is provided. The video encoding method includes determining whether a constrained intra prediction (CIP) mode is used, determining encoding parameters of blocks located on both sides of a filtering target boundary, and whether to use and encoding a constrained intra prediction mode and determining whether to apply the in-loop filter to samples located on both sides of a filtering target boundary based on the parameters.

[2] The in-loop filter of [1] may be a deblocking filter.

[3] In [2], when the encoding parameters of blocks located on both sides of the boundary to be filtered indicate that all blocks located on both sides of the boundary to be filtered are blocks coded in the inter prediction mode, on both sides of the boundary to be filtered A deblocking filter may not be applied to located samples.

[4] The filtering target according to [2], when the encoding parameters of blocks located on both sides of the filtering target boundary indicate that blocks located on both sides of the filtering target boundary are blocks coded in the intra prediction mode and the inter prediction mode, respectively A sample belonging to a block encoded in the inter prediction mode among samples located on both sides of the boundary to be filtered without applying a deblocking filter to samples belonging to the block encoded in the intra prediction mode among samples located on both sides of the boundary A deblocking filter can be applied to

[5] The filtering target according to [2], when the encoding parameters of blocks located on both sides of the filtering target boundary indicate that blocks located on both sides of the filtering target boundary are blocks coded in the intra prediction mode and the inter prediction mode, respectively The deblocking filter may not be applied to samples located on both sides of the boundary.

[6] According to another embodiment of the present invention, an image decoding method is provided. The image decoding method includes the steps of determining whether to use a constrained intra prediction (CIP) mode, determining decoding parameters of blocks located on both sides of a filtering target boundary, and whether to use and filtering the constrained intra prediction mode and determining whether to apply the in-loop filter to samples located on both sides of the filtering target boundary based on the decoding parameters of blocks located on both sides of the target boundary.

[7] In [6], the in-loop filter may be a deblocking filter.

* [8] In [7], when the decoding parameters of blocks located on both sides of the boundary to be filtered indicate that all blocks located on both sides of the boundary to be filtered are blocks decoded in the intra prediction mode, both sides of the boundary to be filtered A deblocking filter may be applied to samples located in .

[9] In [7], when the decoding parameters of blocks located on both sides of the boundary to be filtered indicate that all blocks located on both sides of the boundary to be filtered are blocks decoded in the inter prediction mode, on both sides of the boundary to be filtered A deblocking filter may not be applied to located samples.

[10] In [7], when the decoding parameters of blocks located on both sides of the filtering target boundary indicate that all blocks located on both sides of the filtering target boundary are blocks decoded in the inter prediction mode, on both sides of the filtering target boundary A deblocking filter may be applied to located samples.

[11] The filtering target according to [7], when the decoding parameters of blocks located on both sides of the filtering target boundary indicate that blocks located on both sides of the filtering target boundary are blocks decoded in the intra prediction mode and the inter prediction mode, respectively A sample belonging to a block decoded in inter prediction mode among samples located on both sides of a filtering target boundary without applying a deblocking filter to samples belonging to a block decoded in the intra prediction mode among samples located on both sides of the boundary A deblocking filter can be applied to

[12] The filtering target according to [7], when the decoding parameters of blocks located on both sides of the filtering target boundary indicate that blocks located on both sides of the filtering target boundary are blocks decoded in the intra prediction mode and the inter prediction mode, respectively The deblocking filter may not be applied to samples located on both sides of the boundary.

[13] Another embodiment of the present invention provides a video decoding method. The image decoding method includes parsing an indicator indicating whether a restricted deblocking filter is used or not, determining decoding parameters of blocks located on both sides of a filtering target boundary, and an indicator indicating whether to use a restricted deblocking filter and a filtering target and determining whether to apply a deblocking filter to samples located on both sides of a filtering target boundary based on decoding parameters of blocks located on both sides of the boundary.

[14] The filtering target according to [13], when the decoding parameters of blocks located on both sides of the filtering target boundary indicate that blocks located on both sides of the filtering target boundary are blocks decoded in the intra prediction mode and the inter prediction mode, respectively A sample belonging to a block decoded in inter prediction mode among samples located on both sides of a filtering target boundary without applying a deblocking filter to samples belonging to a block decoded in the intra prediction mode among samples located on both sides of the boundary A deblocking filter can be applied to

[15] The filtering target according to [13], when the decoding parameters of blocks located on both sides of the filtering target boundary indicate that blocks located on both sides of the filtering target boundary are blocks decoded in the intra prediction mode and the inter prediction mode, respectively The deblocking filter may not be applied to samples located on both sides of the boundary.

According to the present invention, it is possible to perform error-tolerant in-loop filtering.

According to the present invention, even when a region coded in the inter prediction mode cannot be normally reconstructed, a block coded in the intra prediction mode can be normally decoded.

According to the present invention, it is possible to maintain the same restoration result of the region coded in the intra prediction mode in the encoder and the decoder.

1 is a block diagram illustrating an example of the structure of an image encoder.
2 is a block diagram illustrating an example of the structure of an image decoder.
3 is a flowchart illustrating an in-loop filtering method (coding) according to an embodiment of the present invention.
4 is a conceptual diagram illustrating an example in which limited intra prediction is performed.
5 is a flowchart illustrating a method of performing in-loop filtering on a block coded in a restricted intra prediction mode.
6 shows an example in which blocks located on both sides of a filtering target boundary are encoded in the intra prediction mode.
7 illustrates an example in which blocks located on both sides of a filtering target boundary are encoded in the inter prediction mode.
8 and 9 show an example in which blocks located on both sides of a filtering target boundary are encoded in an intra prediction mode and an inter prediction mode, respectively.
10 is a flowchart illustrating a method of performing deblocking filtering based on whether CIP mode and PCM mode are used and encoding parameters of blocks located on both sides of a filtering target boundary.
11 is a flowchart illustrating an in-loop filtering method (decoding) according to an embodiment of the present invention.
12 is a flowchart illustrating an image encoding method according to an embodiment of the present invention.
13 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. However, in describing the embodiment of the present invention, if it is determined that a detailed description of a well-known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted.

When a component is described as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but another component may exist in the middle. In addition, when it is stated that a specific component is "included" in the present invention, it does not exclude components other than the corresponding component, but additional components may be included in the scope of the embodiment or technical spirit of the present invention. it means.

Terms such as “first” and “second” may be used to describe various elements, but the elements are not limited by the terms. That is, the above terms are used only for the purpose of distinguishing one component from another component. Accordingly, a first component may be termed a second component, and likewise the second component may also be termed a first component.

In addition, the components shown in the embodiment of the present invention are shown independently to indicate that they perform different characteristic functions, and it does not mean that each component cannot be implemented as one piece of hardware or software. That is, each component is divided for convenience of description, and a plurality of components may be combined to operate as one component, or one component may be divided into a plurality of components to operate, which does not deviate from the essence of the present invention. Unless otherwise specified, it is included within the scope of the present invention.

In addition, some components may be optional components for improving performance rather than essential components performing essential functions of the present invention. The present invention may be implemented as a structure including only essential components except for optional components, and a structure including only essential components is also included in the scope of the present invention.

1 is a block diagram illustrating an example of the structure of an image encoder.

The video encoder 100 includes a motion prediction unit 111 , a motion compensation unit 112 , an intra prediction unit 120 , a switch 115 , a subtractor 125 , a transform unit 130 , a quantization unit 140 , and entropy. It includes an encoder 150 , an inverse quantizer 160 , an inverse transform unit 170 , an adder 175 , a filter unit 180 , and a reference picture buffer 190 .

The image encoder 100 encodes an input image in an intra prediction mode or an inter prediction mode and outputs a bitstream. Intra prediction refers to intra prediction, and inter prediction refers to inter prediction. The video encoder 100 transitions between the intra prediction mode and the inter prediction mode by switching the switch 115 . The image encoder 100 generates a prediction block for an input block of an input image, and then encodes a residual between the input block and the prediction block.

In the intra prediction mode, the intra prediction unit 120 generates a prediction block by performing spatial prediction using pixel values of an already encoded block around the encoding object block.

In the inter prediction mode, the motion prediction unit 111 finds a reference block that best matches the input block in the reference picture stored in the reference picture buffer 190 to obtain a motion vector. The motion compensator 112 performs motion compensation using the motion vector and generates a prediction block. Here, the motion vector is a two-dimensional vector used for inter prediction, and represents an offset between a current encoding/decoding target block and a reference block.

The subtractor 125 generates a residual block based on the difference between the input block and the prediction block, and the transform unit 130 transforms the residual block to output a transform coefficient. The quantization unit 140 quantizes the transform coefficient and outputs a quantized coefficient.

The entropy encoding unit 150 outputs a bitstream by performing entropy encoding based on information obtained in the encoding/quantization process. Entropy encoding reduces the size of a bit stream of a symbol to be encoded by expressing frequently occurring symbols with a small number of bits. Therefore, it is expected that the compression performance of the image can be improved through entropy encoding. The entropy encoder 150 may use an encoding method such as exponential golomb or CABAC (Context-Adaptive Binary Arithmetic Coding) for entropy encoding.

Meanwhile, the coded picture needs to be decoded and stored again in order to be used as a reference picture for performing inter prediction. Accordingly, the inverse quantization unit 160 inversely quantizes the quantized coefficient, and the inverse transformation unit 170 inverse transforms the inverse quantized coefficient to output a reconstructed residual block. The adder 175 generates a reconstructed block by adding the reconstructed residual block to the prediction block.

The filter unit 180 is also referred to as an adaptive in-loop filter, and includes at least one of deblocking filtering, sample adaptive offset (SAO) compensation, and adaptive loop filtering (ALF) on the reconstructed block. carry out Deblocking filtering means removing block distortion caused at the boundary between blocks, and SAO compensation means adding an appropriate offset to a pixel value to compensate for a coding error. In addition, ALF means performing filtering based on a comparison value between the reconstructed image and the original image.

2 is a block diagram illustrating an example of the structure of an image decoder.

The image decoder 200 includes an entropy decoding unit 210 , an inverse quantization unit 220 , an inverse transform unit 230 , an intra prediction unit 240 , a motion compensator 250 , an adder 255 , and a filter unit 260 . ) and a reference picture buffer 270 .

The image decoder 200 decodes the bitstream in the intra prediction mode or the inter prediction mode to output a reconstructed image. The image decoder 200 transitions between the intra prediction mode and the inter prediction mode by switching the switch. The image decoder 200 obtains a residual block from a bitstream to generate a prediction block, and then adds the residual block to generate a reconstructed block.

The entropy decoding unit 210 performs entropy decoding based on a probability distribution. The entropy decoding process is the opposite of the entropy encoding process described above. That is, the entropy decoding unit 210 generates a symbol including a quantized coefficient from a bitstream in which frequently generated symbols are expressed by a small number of bits.

The inverse quantization unit 220 inverse quantizes the quantized coefficient, and the inverse transform unit 230 inversely transforms the inverse quantized coefficient to generate a differential block.

In the intra prediction mode, the intra prediction unit 240 generates a prediction block by performing spatial prediction using pixel values of an already decoded block around the decoding object block.

In the inter prediction mode, the motion compensator 250 generates a prediction block by performing motion compensation using a motion vector and a reference picture stored in the reference picture buffer 270 .

The adder 255 adds the prediction block to the residual block, and the filter unit 260 outputs a reconstructed image by performing at least one of deblocking filtering, SAO compensation, and ALF on the block that has been subjected to the adder.

Hereinafter, a block means a unit of encoding/decoding. In the encoding/decoding process, an image is divided into predetermined sizes and encoded/decoded. Accordingly, a block may also be called a macro block (MB), a coding unit (CU), a prediction unit (PU), a transform unit (TU), etc., and one block is It may be divided into smaller sub-blocks.

Here, the prediction unit means a basic unit for performing prediction and/or motion compensation. A prediction unit may be divided into a plurality of partitions, and each partition is called a prediction unit partition. When the prediction unit is divided into a plurality of partitions, the prediction unit partition may be a basic unit for performing prediction and/or motion compensation. Hereinafter, in an embodiment of the present invention, a prediction unit may mean a prediction unit partition.

Also, in the present specification, using a sample may mean using information of the corresponding sample, for example, a pixel value. However, for convenience of description, it should be noted that the expression “using sample information” or “using pixel values” may be simply expressed as “using a sample”.

On the other hand, when the bitstream is transmitted through a network channel that is prone to errors, there is a high probability that an error will occur in the reconstructed picture. Constrained Intra Prediction (CIP) does not refer to neighboring samples belonging to a block not coded in the intra prediction mode, but only the neighboring samples belonging to the block coded in the intra prediction mode. Thus, it is a prediction method for generating a prediction block for the current block. Here, the current block means an encoding object block or a decoding object block. In the case of using the CIP mode, even if a block coded in the inter prediction mode is not normally reconstructed due to loss of a referenced picture, the generation of the intra prediction block is not affected. Accordingly, even when the block coded in the inter prediction mode cannot be normally reconstructed, the block coded in the intra prediction mode can be normally decoded.

However, since in-loop filtering is performed regardless of whether the CIP mode is used, if an error occurs in the compressed image bitstream, the error in the reconstructed image is not generated during the in-loop filtering process. It can spread to other areas.

In order to solve the above-described problem, in the present invention, whether to perform in-loop filtering is determined based on whether the CIP mode is used and the coding parameters of blocks located on both sides of the filtering target boundary.

3 is a flowchart illustrating an in-loop filtering method (coding) according to an embodiment of the present invention.

The video encoder determines whether the CIP mode is used (S310).

The encoder and the decoder may signal information for determining whether to use the CIP mode. For example, information for determining whether to use the CIP mode is signaled through a sequence parameter set (SPS), a picture parameter set (PPS), or a slice header.

In this case, signaling information for determining whether to use the CIP mode means inserting an indicator such as a flag indicating whether or not the CIP mode is used in the encoder into the bitstream, and the decoder (decoder) means parsing. The indicator is inserted into the bitstream through an entropy encoding process such as arithmetic coding in the encoder, and may be extracted in the decoder through a decoding process corresponding to the entropy encoding process.

For example, in order to determine whether the CIP mode is used in the encoding target sequence, constrained_intra_pred_flag may be transmitted as shown in Table 1. When the flag value is '0', it may be determined that the CIP mode is not used, and if the flag value is '1', it may be determined that the CIP mode is used.

For example, in order to determine whether the CIP mode is used in the encoding target picture, constrained_intra_pred_flag may be transmitted as shown in Table 2. As in the case of signaling through SPS, when the flag value is '0', it is determined that the CIP mode is not used, and when the flag value is '1', it can be determined that the CIP mode is used.

On the other hand, a special purpose encoder and decoder used in an error prone network environment may basically use the CIP mode without separately signaling information for determining whether to use the CIP mode.

4 is a conceptual diagram illustrating an example in which limited intra prediction is performed.

Hereinafter, for convenience of description, [x, y] coordinates in which the coordinate values increase in the lower right direction based on the upper left sample of the encoding object block 400 as a reference ([0, 0]) are set. In addition, p[a, b] represents the pixel value of the sample having the position of [a, b]. For example, the pixel value of the upper left sample of the encoding object block 400 may be expressed as p[0, 0].

When the size of the encoding object block 400 is 8x8, reference samples located on the upper and left sides of the encoding object block 400, that is, the upper reference samples (p[-1...15, -1]) and the left reference A prediction block for the encoding object block is generated by using the sample (p[-1, 0...15]).

However, when the CIP mode is used, as described above, since neighboring samples belonging to a block not coded in the intra prediction mode cannot be used for intra prediction (not available for intra prediction), a process of replacing the reference sample (reference sample) substitution process) is required. For example, in Fig. 4, [x = -1, y = 4...11], [x = -1...5, y = -1], [x = 8...15, y = - 1] should be replaced with other samples that can be used for limited intra prediction.

An example of constructing a reference sample by substituting a neighboring sample that cannot be used for intra prediction is as follows.

Example 1) Neighboring samples that cannot be used for intra prediction are replaced with a value of one of neighboring samples that can be used for intra prediction.

For example, in the case of FIG. 4 , if a sample having a position of [x = -1, y = 15] is a sample that cannot be used for intra prediction, [x = -1, y = 15] to [x = -1, y = -1], then [x = 0, y = -1] to [x = 15, y = -1] sequentially. As soon as a sample usable for intra prediction is found, the search ends, and the pixel value of the sample (p[x = -1, y = 15]) is assigned the pixel value of the searched sample.

For example, in the case of FIG. 4 , samples having positions of [x = -1, y = 4...11] and [x = -1, y = -1] cannot be used for intra prediction. , the pixel value (p[x, y+1]) of the sample located at the bottom of the sample is assigned to the pixel value (p[x, y]) of the corresponding sample as in Equations 1 and 2 .

For example, in the case of FIG. 4 , samples having positions of [x = 0...5, y = -1] and [x = 8...15, y = -1] are used for intra prediction. If the sample cannot be, as shown in Equations 3 and 4, the pixel value (p[x-1, y]) of the sample located to the left of the sample in the pixel value (p[x, y]) of the corresponding sample. is assigned

Example 2) The reference sample(s) that cannot be used for intra prediction is replaced with an average value of reference samples that can be used for intra prediction located on both sides of the reference sample(s).

For example, in the case of FIG. 4 , neighboring samples belonging to a block coded in the inter prediction mode are replaced as in Equations 5 to 7.

Example 3) Reference sample(s) that cannot be used for intra prediction are replaced with reference samples that can be used for intra prediction located on both sides of the reference sample(s) with values obtained by linear interpolation.

For example, in the case of FIG. 4 , neighboring samples belonging to a block encoded in the inter prediction mode are replaced as in Equations 8 to 10.

Meanwhile, when the encoding object block is encoded in the intra prediction mode, in order to improve encoding performance, filtering may be performed on reference samples and predicted samples. Here, the prediction sample means a prediction value of a sample belonging to an encoding object block.

For example, a 3-tap low-pass filter having a filter coefficient of [1 2 1] or a 2-tap average filter having a filter coefficient of [1 1] may be applied to the reference sample.

For example, in a specific intra mode such as an Intra_Vertical prediction mode, an Intra_Horizontal prediction mode, or an Intra_DC prediction mode, a prediction value of a sample corresponding to a boundary of an encoding object block may be derived based on a reference sample adjacent to the corresponding sample. can

However, when the CIP mode is used, since neighboring samples not coded in the intra prediction mode are replaced with neighboring samples coded in the intra prediction mode, there is a high probability that pixel values of the reference samples are similar to or identical to each other. Therefore, unlike the case in which the CIP mode is not used, it may be preferable not to perform filtering on reference samples in the restricted intra prediction mode in terms of reduction of computational complexity and improvement of encoding performance. Similarly, since pixel values of prediction samples derived based on reference samples having similar or identical pixel values are highly likely to be similar or identical to each other, it may be preferable not to perform filtering on the prediction samples.

Therefore, based on [1] whether the CIP mode is used, [2] whether the block to which the reference sample belongs is available, [3] the pixel value of the reference sample, [4] the encoding parameters of the block to which the reference sample belongs, the reference sample and / or whether to perform filtering on the prediction sample may be adaptively determined.

Example 1) It is determined whether filtering is performed on a reference sample and/or a prediction sample based on whether the CIP mode is used.

For example, when a filter is applied to a reference sample in an environment in which the CIP mode is used, filtering may not be performed on the reference sample in an inter slice (P slice or B slice).

For example, when a filter is applied to a prediction sample in an environment in which the CIP mode is used, filtering may not be performed on the prediction sample in an inter slice (P slice or B slice).

Example 2) Whether to perform filtering on the reference sample and/or the prediction sample is adaptively determined based on availability of a block to which the reference sample belongs.

For example, when a filter is applied to a reference sample, filtering may not be performed on a reference sample that corresponds to a sample belonging to an unavailable block and is replaced with a sample that can be used for intra prediction.

For example, if a filter is applied to a prediction sample, filtering is not performed on the prediction sample derived based on a reference sample that is replaced by a sample that corresponds to a sample belonging to an unavailable block and can be used for intra prediction. it may not be

Example 3) It is determined whether filtering is performed on the reference sample and/or the prediction sample based on the pixel value of the reference sample. In this case, the similarity of pixel values measured based on the average, variance, and sample values of the reference samples, and whether the sample values are the same may be used.

For example, when a filter is applied to reference samples, filtering may not be performed to reference samples having similar or identical pixel values.

For example, when a filter is applied to prediction samples, filtering may not be performed on prediction samples derived based on reference samples having similar or identical pixel values.

Example 4) Whether to perform filtering on the reference sample and/or the prediction sample is determined based on the encoding parameter of the block to which the reference sample belongs. In this case, an intra prediction mode, a Most Probable Mode (MPM) flag, an inter prediction mode, a motion vector, a reference picture index, a quantization parameter ), a coded block flag, and at least one of coding parameters such as a coding mode indicating whether the encoding object block is encoded in the intra prediction mode or the inter prediction mode, and the like may be used.

For example, when a filter is applied to a reference sample, filtering is not performed on a reference sample that corresponds to a sample belonging to a block that is not encoded in the intra prediction mode and is replaced with a sample that belongs to a block that is encoded in the intra prediction mode. can

For example, when a filter is applied to a prediction sample, a prediction sample derived based on a reference sample corresponding to a sample belonging to a block not coded in the intra prediction mode and replaced with a sample belonging to a block coded in the intra prediction mode filtering may not be performed.

Meanwhile, in the case of a specific intra mode such as the Intra_Vertical prediction mode, the Intra_Horizontal prediction mode, and the Intra_DC prediction mode, as described above, the filter may be applied only to prediction samples corresponding to the boundary of the encoding object block among the prediction samples. Even in the above case, [1] whether the CIP mode is used, [2] whether the block to which the reference sample belongs is available, [3] the pixel value of the reference sample, [4] prediction based on the coding parameters of the block to which the reference sample belongs Whether to perform filtering on prediction samples corresponding to a boundary of an encoding object block among samples may be adaptively determined. Referring back to FIG. 4 , filtering may not be performed on the prediction samples 410 and 420 having positions of [0...5, 0], [0, 4...7].

5 is a flowchart illustrating a method of performing in-loop filtering on a block coded in a restricted intra prediction mode.

Hereinafter, for convenience of description, in-loop filtering will be described as an example of deblocking filtering. However, the present invention is not applicable only to deblocking filtering, but may also be applied to non-deblocking filtering, sample adaptive offset (SAO) compensation, or adaptive loop filter (ALF).

The encoder determines a filtering target boundary (S510). In general, a boundary of a division unit of an image is determined as a boundary to be filtered. For example, a boundary of a coding unit (CU), a boundary of a prediction unit (PU), and a boundary of a transform unit (TU) may be determined as the filtering target boundary. Accordingly, the determination of the filtering target boundary is performed in units of coding units (CUs), largest coding units (LCUs), slices, or pictures.

The encoder determines whether to perform filtering and the type of filter based on at least one of the filtering intensity and neighboring pixel values of the filtering target boundary determined through the filtering boundary determination step ( S510 ) ( S520 ). For example, it is determined whether the filtering target boundary is a blocking artifact due to transform and quantization or an actual edge existing in the picture based on the surrounding pixel values of the filtering target boundary and whether to perform filtering may be determined. For example, the filter strength may indicate a tap size indicating the number of input samples of the low-pass filter, a coefficient of the low-pass filter, and the like.

The encoder performs filtering based on the filtering target boundary and the filter type determined through the filtering target boundary determination step S510, whether filtering is performed, and the filter type determination step S520. In this case, for smooth processing of the boundary between blocks, a low-pass filter may be used according to the amount of change in pixel values around the boundary to be filtered, or a Wiener filter may be applied to minimize distortion from the original image. . In addition, a one-dimensional filter or a two-dimensional or more multi-dimensional filter may be applied according to the boundary of the filtering target. For example, a two-dimensional or more multidimensional filter having a filter shape such as a quadrangle, a circle, a rectangle, and a structure of filter coefficients such as horizontal symmetry, vertical symmetry, and diagonal symmetry may be applied. In addition, various filters may be applied based on the filtering strength determined through the determination of whether filtering is performed and the type of filter ( S520 ).

Referring back to FIG. 3 , the video encoder determines encoding parameters of blocks located on both sides of the filtering target boundary ( S320 ).

The video encoder may determine whether to perform filtering and the type of filter based on encoding parameters of blocks located on both sides of the filtering target boundary. In this case, an intra prediction mode, an inter prediction mode, a motion vector, a reference picture index, a quantization parameter, a coded block flag flag) and at least one of encoding parameters such as a coding mode indicating whether the encoding object block is encoded in the intra prediction mode or the inter prediction mode.

For example, the image encoder may determine whether blocks located on both sides of the filtering target boundary are encoded in the intra prediction mode or the inter prediction mode. In this case, if a block is encoded in the intra prediction mode, the block may be said to be encoded in the intra mode or may be said to be intra-coded. Similarly, if a block is encoded in the inter prediction mode, the block may be said to be encoded in the inter mode or may be said to be inter-coded.

For example, when blocks located on both sides of the boundary to be filtered are encoded in the inter prediction mode, the video encoder uses a residual signal such as a coded block flag (CBF) and a skip mode. It is determined whether there is a transform coefficient for , and a processing method of the deblocking filter can be different based on this.

For example, when blocks located on both sides of the filtering target boundary are encoded in the pulse coded modulation (PCM) mode, the video encoder may determine that the corresponding block is encoded in the intra prediction mode.

In addition, the image encoder may determine whether to perform filtering and the type of filter based on the values of pixels surrounding the boundary of the filtering object together with encoding parameters of blocks located on both sides of the boundary of the filtering object. In this case, a difference, a gradient, a variance, an average, and the like between pixel values surrounding the boundary of the filtering target may be used.

The video encoder determines whether to apply the in-loop filter based on whether the CIP mode is used and the encoding parameters of blocks located on both sides of the filtering target boundary.

When the CIP mode is not used, the video encoder performs filtering on samples located on both sides of the filtering target boundary (S330).

When the CIP mode is used and the encoding parameters of blocks located on both sides of the filtering target boundary indicate that all blocks located on both sides of the filtering target boundary are blocks coded in the intra prediction mode, the video encoder is located on both sides of the filtering target boundary Filtering is performed on the samples to be used (S331).

6 illustrates an example in which blocks located on both sides of a filtering target boundary are encoded in the intra prediction mode. When both blocks are intra-coded, samples belonging to both blocks are not affected by the block coded in the inter prediction mode in which an error may occur. Accordingly, when constrained_intra_pred_flag is 1 and all blocks located on both sides of the filtering target boundary are coded in the intra prediction mode, deblocking filtering may be performed on the filtering target boundary.

When the CIP mode is used and the encoding parameters of blocks located on both sides of the filtering target boundary indicate that all blocks located on both sides of the filtering target boundary are blocks coded in the inter prediction mode, the video encoder is located on both sides of the filtering target boundary The deblocking filter may not be applied to the samples to be used (S333[1]).

7 illustrates an example in which blocks located on both sides of a filtering target boundary are encoded in the inter prediction mode. When both blocks are inter-coded (merge mode, skip mode, PU_2Nx2N/PU_2NxN/PU_Nx2N/PU_NxN mode), errors may occur in samples belonging to both blocks. Accordingly, when constrained_intra_pred_flag is 1 and all encoding parameters of blocks located on both sides of the filtering target boundary are inter prediction modes, the video encoder may not perform filtering on samples located on both sides of the filtering target boundary.

In addition, when constrained_intra_pred_flag is 1 and all encoding parameters of blocks located on both sides of the filtering target boundary are inter prediction modes, the video encoder may determine the filtering strength to be '0' as if filtering is not performed. Equation 11 shows an example of determining the filtering strength as '0'.

Here, bS is the filtering strength, filterDir is the application direction of the one-dimensional filter (vertical/horizontal), and xEk and yEj are the positions of the filtering target boundary.

In addition, when constrained_intra_pred_flag is 1 and coding parameters of blocks located on both sides of the filtering target boundary are all inter prediction modes, errors may occur in samples belonging to both blocks, but by performing deblocking filtering on the samples It may not seem like a problem. Accordingly, the video encoder may perform filtering on samples located on both sides of the filtering target boundary (S333[2]).

When the CIP mode is used and the encoding parameters of blocks located on both sides of the filtering target boundary indicate that blocks located on both sides of the filtering target boundary are blocks coded in the intra prediction mode and the inter prediction mode, respectively, the video encoder performs the filtering target boundary Filtering may be performed only on samples belonging to a block coded in the inter prediction mode among samples located on both sides of (S332[1]).

8 and 9 show an example in which blocks located on both sides of a filtering target boundary are encoded in an intra prediction mode and an inter prediction mode, respectively. When the block coded in the intra prediction mode and the block coded in the inter prediction mode are located on both sides of the boundary to be filtered, an error occurring in the reconstructed sample belonging to the block coded in the inter prediction mode is transmitted to the block coded in the intra prediction mode. It may affect the reconstructed sample to which it belongs. therefore. By performing filtering on only samples belonging to a block encoded in the inter prediction mode among samples on both sides of the filtering target boundary, without performing filtering on samples belonging to the block encoded in the intra prediction mode among samples on both sides of the filtering target boundary , it is possible to prevent the propagation of errors.

When filtering is performed only on samples belonging to blocks encoded in the inter prediction mode among samples on both sides of the filtering target boundary, the following deblocking filtering process is performed.

<Process of performing deblocking filtering on luminance (luma) samples 1>

- input

(1) Sample values: pi, qi (i = 0...3)

(2) Whether to apply a filter to each of p1 and q1 samples: dEp1, dEq1

(3) Threshold for applying the filter: tc

- Print

(1) Number of filtered samples: nDp, nDq

(2) Filtered sample values: pi', qj' (i = 0...nDp -1, j = 0...nDq - 1)

- [1] pi is not a sample of the I_PCM block, pcm_loop_filter_disable_flag is 0, [2] pi belongs to a block coded in inter prediction mode, [3] qi belongs to a block coded in intra prediction mode, and , [4] When dE is 2, nDp is set to 3 and the following strong filter is applied to pi.

Here, Clip3(a, b, c) represents clipping c within the range of a and b.

- [1] pi is not a sample of the I_PCM block, pcm_loop_filter_disable_flag is 0, [2] pi belongs to a block coded in inter prediction mode, [3] qi belongs to a block coded in intra prediction mode, and , [4] When dE is not 2, nDp is set to 1 and a weak filter is applied to pi.

If abs(Δ) is less than tc*10, the following steps are applied.

(1) p0' is obtained as follows.

Here, Clip1Y(x) is defined as in Equation (18).

Here, BitDepthY represents the bit depth of the luminance component.

(2) If dEp1 is 1, pi' is obtained as follows.

(3) nDp becomes dEp1+1.

- [1] qi is not a sample of the I_PCM block, pcm_loop_filter_disable_flag is 0, [2] qi belongs to a block coded in inter prediction mode, [3] pi belongs to a block coded in intra prediction mode, and , [4] When dE is 2, nDq is set to 3 and the following strong filter is applied to qi.

Here, Clip3(a, b, c) represents clipping c within the range of a and b.

- [1] qi is not a sample of the I_PCM block, pcm_loop_filter_disable_flag is 0, [2] qi belongs to a block coded in inter prediction mode, [3] pi belongs to a block coded in intra prediction mode, and , [4] If dE is not 2, nDq is set to 1 and a weak filter is applied to qi.

If abs(Δ) is less than tc*10, the following steps are applied.

(1) q0' is obtained as follows.

Here, Clip1Y(x) is defined as in Equation 27.

Here, BitDepthY represents the bit depth of the luminance component.

(2) If dEq1 is 1, qi' is obtained as follows.

(3) nDq becomes dEq1+1.

Meanwhile, deblocking filtering may be performed on the luma sample as follows.

<Process of performing deblocking filtering on luminance (luma) samples 2>

- input

(1) Sample values: pi, qi (i = 0...3)

(2) Whether to apply a filter to each of p1 and q1 samples: dEp1, dEq1

(3) Threshold for applying the filter: tc

*- Print

(1) Number of filtered samples: nDp, nDq

(2) Filtered sample values: pi', qj' (i = 0...nDp -1, j = 0...nDq - 1)

- If dE is 2, set nDp and nDq to 3, and apply a strong filter.

Here, Clip3(a, b, c) represents clipping c within the range of a and b.

- If dE is not 2, set nDp and nDq to 1 and apply a weak filter.

If abs(Δ) is less than tc*10, the following steps are applied.

(1) p0' and q0' are obtained as follows.

Here, Clip1Y(x) is defined as in Equation (40).

Here, BitDepthY represents the bit depth of the luminance component.

(2) If dEp1 is 1, pi' is obtained as follows.

(3) If dEq1 is 1, qi' is obtained as follows.

(4) nDp becomes dEp1+1, and nDq becomes dEq1+1.

- If at least one of the following two conditions is satisfied, pi' (i = 0...nDp-1) is changed to the input sample pi.

(1) pi is a sample of the I_PCM block, and pcm_loop_filter_disable_flag is 1.

(2) pi belongs to a block coded in the intra prediction mode, and qi belongs to a block coded in the inter prediction mode.

- If at least one of the following two conditions is satisfied, qj' (j = 0...nDq-1) is changed to the input sample qj.

(1) qj is a sample of an I_PCM block, and pcm_loop_filter_disable_flag is 1.

(2) qj belongs to a block coded in the intra prediction mode, and pj belongs to a block coded in the inter prediction mode.

Meanwhile, deblocking filtering may be performed on a chroma sample as follows.

<Process of performing deblocking filtering on chroma samples 1>

- input

(1) Sample values: pi, qi (i = 0, 1)

(2) Threshold for applying the filter: tc

- Print

*(1) Filtered sample values: p0', q0'

- [1] pi is not a sample of the I_PCM block, pcm_loop_filter_disable_flag is 0, [2] pi belongs to a block coded in inter prediction mode, and [3] qi belongs to a block coded in intra prediction mode , apply the following filter.

- [1] When qi is not a sample of the I_PCM block, pcm_loop_filter_disable_flag is 0, [2] qi belongs to a block coded in inter prediction mode, and [3] pi belongs to a block coded in intra prediction mode , apply the following filter.

Meanwhile, deblocking filtering may be performed on a chroma sample as follows.

<Process of performing deblocking filtering on chroma samples 2>

- input

(1) Sample values: pi, qi (i = 0, 1)

(2) Threshold for applying the filter: tc

- Print

(1) Filtered sample values: p0', q0'

- Apply the following filters.

- If at least one of the following two conditions is satisfied, p0' is changed to the input sample p0.

(1) p0 is a sample of the I_PCM block, and pcm_loop_filter_disable_flag is 1.

(2) p0 belongs to a block coded in the intra prediction mode, and q0 belongs to a block coded in the inter prediction mode.

- If at least one of the following two conditions is satisfied, q0' is changed to the input sample q0.

(1) q0 is a sample of an I_PCM block, and pcm_loop_filter_disable_flag is 1.

(2) q0 belongs to a block coded in the intra prediction mode, and p0 belongs to a block coded in the inter prediction mode.

On the other hand, when the CIP mode is used and the encoding parameters of blocks located on both sides of the filtering target boundary indicate that blocks located on both sides of the filtering target boundary are blocks coded in the intra prediction mode and the inter prediction mode, respectively, the video encoder performs filtering Filtering may not be performed on samples located on both sides of the target boundary (S332[2]).

For example, when one of blocks located on both sides of the filtering target boundary is encoded in the intra prediction mode (PU_2Nx2N or PU_NxN), filtering strength may not be determined and filtering may not be performed.

In addition, when the CIP mode is used and the encoding parameters of blocks located on both sides of the filtering target boundary indicate that blocks located on both sides of the filtering target boundary are blocks coded in the intra prediction mode and the inter prediction mode, respectively, the video encoder performs filtering It may be determined that the filtering strength is '0' as in not performing . Equation 52 shows an example of determining the filtering strength as '0'.

Here, bS is the filtering strength, filterDir is the application direction of the one-dimensional filter (vertical/horizontal), and xEk and yEj are the positions of the filtering target boundary.

On the other hand, if the filtering target block is an I_PCM block and pcm_loop_filter_disable_flag is '1', deblocking filtering is not performed. Here, the I_PCM block means that the block to be filtered is a block coded in the PCM mode using the uncompressed original sample, and the pcm_loop_filter_disable_flag is a flag for indicating whether a loop filter process is performed on reconstructed pixels of the I_PCM block. Accordingly, it is possible to determine whether to perform filtering, including when the filtering target block is an I_PCM block.

10 is a flowchart illustrating a method of performing deblocking filtering based on whether CIP mode and PCM mode are used and encoding parameters of blocks located on both sides of a filtering target boundary.

When the CIP mode is used, that is, when constrained_intra_pred_flag is '1', the encoding modes of blocks P and Q located on both sides of the filtering target boundary are determined.

When blocks located on both sides of the boundary to be filtered are coded in the intra prediction mode and the inter prediction mode, respectively, the image encoder performs analysis on samples belonging to the block coded in the intra prediction mode among samples located on both sides of the boundary to be filtered. No filtering is performed.

When all blocks located on both sides of the filtering target boundary are coded in the intra prediction mode, or when all blocks located on both sides of the filtering target boundary are coded in the inter prediction mode, the video encoder determines pcm_loop_filter_disable_flag.

When pcm_loop_filter_disable_flag is '1', it is determined whether blocks P and Q located on both sides of the filtering target boundary are I_PCM blocks, respectively, and filtering is not performed on blocks that are I_PCM blocks.

When the CIP mode is not used, that is, when constrained_intra_pred_flag is '0', the video encoder determines pcm_loop_filter_disable_flag.

When pcm_loop_filter_disable_flag is '1', it is determined whether blocks P and Q located on both sides of the filtering target boundary are I_PCM blocks, respectively, and filtering is not performed on blocks that are I_PCM blocks.

11 is a flowchart illustrating an in-loop filtering method (decoding) according to an embodiment of the present invention.

The decoder may determine whether to perform in-loop filtering in the same way as the encoder. That is, the video decoder determines whether the CIP mode is used (S1110).

When the CIP mode is used, the video decoder determines decoding parameters of blocks located on both sides of the filtering target boundary of the decoding target block (S1120).

The video decoder determines whether to apply the in-loop filter based on whether the CIP mode is used and the decoding parameters of blocks located on both sides of the filtering target boundary.

For example, when the CIP mode is not used, the image decoder performs filtering on samples located on both sides of the filtering target boundary ( S1130 ).

For example, when the CIP mode is used and the decoding parameters of blocks located on both sides of the filtering target boundary indicate that all blocks located on both sides of the filtering target boundary are blocks decoded in the intra prediction mode, the video decoder Filtering is performed on samples located on both sides (S1131).

For example, when the CIP mode is used and decoding parameters of blocks located on both sides of the filtering target boundary indicate that all blocks located on both sides of the filtering target boundary are blocks decoded in the inter prediction mode, the video decoder The deblocking filter may not be applied to the samples located on both sides (S1133[1]).

For example, when the CIP mode is used and decoding parameters of blocks located on both sides of the filtering target boundary indicate that all blocks located on both sides of the filtering target boundary are blocks decoded in the inter prediction mode, the video decoder Filtering may be performed on samples located on both sides (S1133[2]).

For example, when the CIP mode is used and the decoding parameters of blocks located on both sides of the filtering target boundary indicate that blocks located on both sides of the filtering target boundary are blocks decoded in the intra prediction mode and the inter prediction mode, respectively, the video decoder Filtering may be performed only on samples belonging to a block decoded in the inter prediction mode among samples located on both sides of the filtering target boundary (S1132[1]).

For example, when the CIP mode is used and the decoding parameters of blocks located on both sides of the filtering target boundary indicate that blocks located on both sides of the filtering target boundary are blocks decoded in the intra prediction mode and the inter prediction mode, respectively, the video decoder Filtering may not be performed on samples located on both sides of the filtering target boundary (S1132[2]).

12 is a flowchart illustrating an image encoding method according to an embodiment of the present invention.

The video encoder signals information for determining whether to use the limited in-loop filter (S1210). Here, the restricted in-loop filter refers to an in-loop filter applied based on whether the CIP mode is used and the coding parameters of blocks located on both sides of the filtering target boundary, and to determine whether to use the restricted in-loop filter Signaling information for this means inserting an indicator such as a flag indicating whether to use a restricted in-loop filter in the encoder into the bitstream, and parsing it in the decoder. ) means to

When the limited in-loop filter is used, [1] filtering is performed only on samples belonging to blocks encoded in the inter prediction mode among samples located on both sides of the boundary to be filtered, or [2] from both sides of the boundary to be filtered Filtering may not be performed on samples located in [3], or filtering may be performed on samples located on both sides of the filtering target boundary.

For example, when the encoding parameters of blocks located on both sides of the filtering object boundary indicate that blocks located on both sides of the filtering object boundary are blocks encoded in the intra prediction mode and the inter prediction mode, respectively, located on both sides of the filtering object boundary Filtering may be performed only on samples belonging to blocks encoded in the inter prediction mode among the samples to be used.

For example, when the encoding parameters of blocks located on both sides of the boundary to be filtered indicate that all blocks located on both sides of the boundary to be filtered are blocks coded in the inter prediction mode, samples located on both sides of the boundary to be filtered are Filtering may not be performed.

For example, when the encoding parameters of blocks located on both sides of the boundary to be filtered indicate that all blocks located on both sides of the boundary to be filtered are blocks coded in the intra prediction mode, samples located on both sides of the boundary to be filtered are filtering can be performed.

The encoder may signal information indicating that at least one of the above examples is applied, that is, information for determining whether to use a limited in-loop filter. In this case, the signaling is performed in a coding unit (CU), a prediction unit (PU), a transform unit (TU), a largest coding unit (LCU), a slice unit, or a picture unit. can be performed.

For example, when information indicating whether the restricted in-loop filter is used or not is signaled through the PPS, constrained_in_loop_filter_flag may be transmitted as shown in Table 3.

In this case, when constrained_in_loop_filter_flag is '1', the restricted in-loop filter may be used, and when constrained_in_loop_filter_flag is '0', the restricted in-loop filter may not be used.

Meanwhile, the video encoder may determine whether to signal information for determining whether to use the restricted in-loop filter based on whether the CIP mode is used.

For example, when the CIP mode is used (constrained_intra_pred_flag = 1), information for determining whether to use a restricted in-loop filter is signaled, and when the CIP mode is not used (constrained_intra_pred_flag = 0), the restricted in-loop Information for determining whether to use a filter may not be signaled.

Also, the video encoder may determine whether to signal information for determining whether to use the restricted in-loop filter based on the encoding parameters of blocks located on both sides of the filtering target boundary.

For example, when one or more blocks among blocks located on both sides of a filtering target boundary are encoded in the inter prediction mode, information for determining whether to use a restricted in-loop filter may be signaled.

Table 4 shows an example of determining whether to signal information for determining whether to use a restricted in-loop filter based on encoding parameters of blocks located on both sides of a filtering target boundary.

Also, the video encoder may basically signal information for determining whether to use the limited in-loop filter.

The video encoder determines encoding parameters of blocks located on both sides of the filtering target boundary (S1220). At this time, the video encoder may determine the encoding parameters of the blocks located on both sides of the filtering target boundary in the same way as in the “step of determining the encoding parameters of the blocks located on both sides of the filtering target boundary ( S320 )” of FIG. 3 . .

The video encoder determines whether to apply the in-loop filter based on the information for determining whether to use the restricted in-loop filter and the encoding parameters of blocks located on both sides of the filtering target boundary (S1230).

13 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

The decoder may determine whether to use the restricted in-loop filter in a method corresponding to that of the encoder. That is, the image decoder parses information for determining whether to use the limited in-loop filter (S1310).

The image decoder determines decoding parameters of blocks located on both sides of the filtering target boundary (S1320).

The image decoder determines whether to apply the in-loop filter based on the information for determining whether to use the limited in-loop filter and the decoding parameters of blocks located on both sides of the filtering target boundary (S1330).

It is not limited to the order of the steps described above, and some steps may occur with other steps, in a different order, or at the same time. In addition, those of ordinary skill in the art to which the present invention pertains will understand that the steps shown in the flowchart are not exclusive, and other steps may be included or some steps may be deleted.

In addition, the above-described embodiments include examples of various aspects. It is not possible to describe every possible combination in order to represent various aspects, but one of ordinary skill in the art will recognize that other combinations are possible. Accordingly, it is intended that the present invention cover all other substitutions, modifications and variations falling within the scope of the following claims.

Claims (5)

constructing reference samples from neighboring samples of the current block; and
generating a reconstructed block based on prediction of the current block using the reference samples;
The step of constructing the reference samples comprises:
Replacing sample values of unavailable samples that cannot be used for intra prediction among neighboring samples of the current block with sample values of available samples that can be used for intra prediction among neighboring samples of the current block;
When the unavailable sample is a neighboring sample located to the left of the current block, the available sample is a sample located at a lower end of the unavailable sample,
A sample belonging to a block not coded in the intra prediction mode is the unavailable sample.
Flow-determined video decoding method. The method of claim 1,
When the unavailable sample is a neighboring sample located at the top of the current block, the available sample is a sample located to the left of the unavailable sample. constructing reference samples from neighboring samples of the current block; and
generating a reconstructed block based on prediction of the current block using the reference samples;
The step of constructing the reference samples comprises:
Replacing sample values of unavailable samples that cannot be used for intra prediction among neighboring samples of the current block with sample values of available samples that can be used for intra prediction among neighboring samples of the current block;
When the unavailable sample is a neighboring sample located to the left of the current block, the available sample is a sample located at a lower end of the unavailable sample,
An image encoding method in which a sample belonging to a block that is not encoded in the intra prediction mode is determined as the unusable sample. 4. The method of claim 3,
When the unavailable sample is a neighboring sample located at the top of the current block, the available sample is a sample located to the left of the unavailable sample. A computer readable device storing a bitstream generated by an image encoding method
As a rock medium,
The video encoding method comprises:
constructing reference samples from neighboring samples of the current block; and
generating a reconstructed block based on prediction of the current block using the reference samples;
The step of constructing the reference samples comprises:
Replacing sample values of unavailable samples that cannot be used for intra prediction among neighboring samples of the current block with sample values of available samples that can be used for intra prediction among neighboring samples of the current block;
When the unavailable sample is a neighboring sample located to the left of the current block, the available sample is a sample located at a lower end of the unavailable sample,
A recording medium in which a sample belonging to a block not coded in the intra prediction mode is determined as the unusable sample.

Images