JP2020129828A - Method and apparatus for adaptive filtering of sample for video encoding - Google Patents

Abstract

To provide a method and apparatus for adaptive filtering of a sample for video encoding.SOLUTION: A video encoding apparatus comprises: a sequence of filters configurable by one or more primary parameters and one or more secondary parameters; and a filter controller configured to adjust the one or more secondary parameters based on the one or more primary parameters and based on an intensity criterion of the sequence of filters.SELECTED DRAWING: Figure 3

Description

The present invention relates to a video coding device and a method for configuring a sequence of filters for video coding.

The present invention also relates to a computer-readable storage medium having program code stored thereon. The program code includes instructions for performing the method for configuring a sequence of filters for video encoding.

Digital video communication and storage applications are implemented by a wide range of digital devices such as digital cameras, cellular radiotelephones, laptops, broadcast systems, video conferencing systems and the like. One of the most important and difficult tasks of these applications is video compression. The task of video compression is complex and constrained by two conflicting parameters: compression efficiency and computational complexity. Video coding standards such as ITU-T H.264/AVC or ITU-T H.265/HEVC provide a good tradeoff between these parameters.

State of the art video coding standards are mostly based on dividing the source picture into blocks. The processing of these blocks depends on their size, spatial position and the coding mode specified by the encoder. The coding modes can be classified into two groups according to the type of prediction: intra prediction mode and inter prediction mode. Intra-prediction mode uses the pixels of the same picture to generate reference samples to compute the predictions for the pixels of the reconstructed block. Intra prediction can also be referred to as spatial prediction. The inter prediction mode is designed for temporal prediction and uses the reference samples of the previous or next picture to predict the pixels of the block of the current picture.

Due to the different types of redundancy, the prediction process for intra and inter coding is different. Intra prediction typically builds a one-dimensional buffer of reference samples. Inter-prediction typically uses sub-pixel interpolation of a two-dimensional matrix of reference pixels. For both intra and inter coding, additional processing can be used to improve the prediction results (eg smoothing reference samples for intra prediction, reference for inter prediction). Sharpening of sample).

The recently adopted ITU-T H.265/HEVC standard (ISO/IEC23008-2:2013, “Information technology-High efficiency coding and media delivery in heterogeneous environments Part 2: High efficiency video coding”, November 2013) We declare a set of state-of-the-art video coding tools that provide a reasonable trade-off between coding efficiency and computational complexity.

Like the ITU-T H.264/AVC video coding standard, the ITU-T H.265/HEVC video coding standard provides a division of a source picture into blocks, eg, coding units (CUs). .. Each CU can be further divided into smaller CUs or prediction units (PUs). The PU may be intra-predicted or inter-predicted depending on the type of processing applied on the pixels of the PU. In the case of inter prediction, PU represents the region of pixels processed by motion compensation using the motion vector specified for PU. For intra prediction, the PU specifies the prediction mode for a set of transform units (TU). TUs can have different sizes (eg 4x4, 8x8, 16x16 and 32x32 pixels) and can be treated differently. For the TU, transform coding has been performed, that is, the prediction error has been transformed and quantized using the discrete cosine transform (DCT). Thus, the reconstructed pixel contains quantization noise and blocking artifacts, which can affect prediction accuracy.

To reduce this effect on intra prediction, reference pixel filtering is adopted for HEVC/H.265. For inter prediction, reference pixels are calculated using subpixel interpolation. Reference pixel smoothing in the case of motion compensation can be combined with anti-aliasing filtering of the sub-pixel interpolation process.

A mode-adaptive intra prediction smoothing technique has been proposed. Smoothing filtering depends on the intra prediction mode selected and the flags encoded in the video bitstream. Depending on the intra prediction mode defined for a block, the reference samples can be smoothed by the filter or used without modification. If the reference sample is smoothed, the smoothing filter selection can also be based on the intra prediction mode. Furthermore, this selection can be performed according to the value of the flag reconstructed from the bitstream.

The current HEVC/H.265 standard uses this technique in part. Specifically, filter smoothing is turned off for some combinations of intra mode and block size.

A reference sample adaptive filter (RSAF) has been proposed as an extension of the reference sample filter adopted for the HEVC/H.265 standard. The adaptive filter segments the reference samples before smoothing. This is because different filters are applied to different segments. In addition, data hiding procedures have been used to signal the smoothing flags. A simplified version of the adaptive filter for the reference sample was adopted for Joint Exploration Model 1 (JEM1). JEM1 includes some other tools that use smoothing, including:
・Four-tap intra interpolation filter,
Boundary prediction filter and/or Multi-parameter Intra prediction (MPI), which can be replaced by Position Dependent Intra Prediction Combination (PDPC).

Problems with the above method include high signaling effort and video oversmoothing during encoding or decoding.

It is an object of the invention to provide a video coding device and a method for configuring a filter sequence for video coding. The present video encoder and the present method of configuring a filter sequence overcomes one or more of the problems set forth above.

A first aspect of the present invention is a video encoding device:
A sequence of filters configurable by one or more primary parameters and one or more secondary parameters,
A filter controller configured to adjust the one or more secondary parameters based on the one or more primary parameters and based on an intensity criterion of the sequence of filters.
Provide a device.

The video encoder of the first aspect may adjust the one or more secondary parameters such that the strength criterion of the sequence of filters is satisfied. For example, as outlined below, the intensity criterion can relate to the overall smoothness, and the filter controller ensures that the overall smoothness of the sequence of filtering stages is neither too high nor too low. So that the secondary parameters can be set. In other words, the filter controller can be configured to ensure that the strength criteria of the sequence of filters is within a predetermined range.

The video encoder of the first aspect may be configured for video encoding and/or decoding.

The filter controller can be configured to only partially set one or more of the secondary parameters. For example, secondary parameters can be read from the bitstream or user settings and adjusted by the filter controller, for example within a range. In other embodiments, the filter controller may be configured in other ways, for example, to override the value of a secondary parameter derived from the bitstream or user settings.

In the prior art, inconsistent behavior of video coding tools, including RSAF, can lead to over-smoothing. Oversmoothing is
The next filter does not take into account the effects caused by the previous filters, thus reducing the overall coding performance, and/or the overall filter, since all of the above mentioned filters are always on. Increase computational complexity.

This can be avoided with the video encoder of the first aspect. Moreover, signaling effort may be reduced in certain embodiments, since the secondary parameters need not be stored in the bitstream.

In a preferred embodiment, the video encoder of the first aspect can solve the above-mentioned problem of over-smoothing by adjusting the parameters of the filters of a filter sequence that uses smoothing. This adjustment can be reached, for example, by introducing a flag or some condition for a filter that uses smoothing. Flags and/or conditions are
It can be used to turn on and off the smoothing mechanism of the tool and/or to change the smoothing strength of the filter (eg switch from a strong filter to a weak filter).

The first controller of the video encoder of the first aspect may be configured to control multiple filters rather than just a single filter. This can be thought of as a mechanism for reconciling the various filters that influence the results of intra prediction, eg by smoothing. In particular, the filter controller can be configured to make the following adjustments:
Some filter modules can be switched off in response to flag values and/or some conditions, so that the sample processing mechanism can be changed;
A new filter module can be introduced that provides control over all filters that can influence the results of intra prediction by smoothing.

In a first implementation of the video encoder according to the first aspect, the strength criteria are:
・Smoothness standard,
Includes the ratio of the amplification factor for the high frequency region and the amplification factor for the low frequency region and/or the ratio of the contrast value before filtering and the contrast value after filtering.

This allows optimizing the sequence of filters for one or more of the above criteria. As outlined above, preferably the adjustment by the filter controller can be carried out such that the intensity criterion of the sequence of filters is within a certain range, for example within a predetermined range.

In a second implementation of the video encoding device according to the first aspect, the one or more primary parameters are predetermined parameters, in particular an encoded bitstream, a user setting and/or a parameter search loop in the encoding device. Is a parameter that has been determined in advance.

Determining the secondary parameters from the primary parameters, for example only the primary parameters being predetermined, has the advantage that the signaling effort can be reduced. For example, the bit rate can be reduced if the secondary parameters are not stored in the bitstream and can be derived from the primary parameters in the bitstream.

In a third implementation of the video encoding device according to the first aspect, the sequence of filters is one or more primary filters, configurable by one or more primary parameters, and one or more secondary filters. And one or more secondary filters that are configurable by the following parameters: Here, the one or more first-order filters are located before the one or more second-order filters in a sequence of filters.

Adjusting the parameters of the filter at a later stage has the advantage that the effect of the earlier stage filter can possibly be canceled or at least not strengthened further. For example, if an early filter gives some smoothing strength, it can be guaranteed that later filter stages do not increase this smoothing effect. For example, smoothing flags of later filter stages can be switched off.

In other embodiments of the invention, the primary parameter may be associated with a later filter stage and the secondary parameter with an earlier filter stage.

In a fourth implementation of the video encoder according to the first aspect, the sequence of filters is:
A reference sample filter configured to adaptively filter one or more neighboring samples of the current video block to obtain one or more reference samples;
An interpolation filter configured to predict one or more samples of the current video block using interpolation of the one or more reference samples,
The one or more primary parameters include a reference parameter of the reference sample filter, the one or more secondary parameters include a selection parameter of the interpolation filter, and the interpolation filter interpolates according to the selection parameter. Configured to use the method.

The sequence of filters of the video encoder of the fourth implementation may be, for example, a sequence of filters for intra prediction.

Preferably, the filter controller is arranged to determine the selection parameter based on the reference parameter. This has been shown to be an effective way to improve the overall filter strength criterion.

In a fifth implementation of a video encoder according to the first aspect, the one or more first order parameters include a reference sample filter flag of a reference sample filter, and the one or more second order parameters Contains the filter strength parameter of the intra prediction interpolation filter.

Preferably, the filter controller is arranged to determine the filter strength parameter based on the reference sample filter flag. This has been shown to be an effective way to improve the overall filter strength criterion.

In a sixth implementation of the video encoder according to the first aspect, the filter sequence performs boundary smoothing on one or more transform units belonging to one or more prediction units that satisfy a size constraint condition. A boundary smoothing filter configured to

This has the advantage that the boundary smoothing filter obscures the blocking artifacts of the reconstructed block. Prediction units typically include some type of picture area, such as edges, textures, and smooth areas. However, for larger PUs, the probability of smooth regions is higher. For smooth regions, blocking artifacts are more critical, and thus boundary smoothing for large PUs is preferable to smaller PUs. Consequently, it is proposed to constrain the boundary smoothing with PU size (eg by size of 32×32 pixels). By using this constraint, on the one hand, we avoid unwanted blurring for non-smooth regions of smaller PUs, and on the other hand, reduce blocking artifacts for larger PUs. This allows to improve both objective and subjective quality compared to the case where boundary smoothing is predefined for intra prediction.

In a seventh implementation of the video encoder according to the first aspect, the one or more first order parameters include a predictive block filter directional parameter and the one or more second order parameters are boundary smoothed. Includes filter on-off parameters.

Preferably, the filter controller is arranged to determine an on/off parameter of the boundary smoothing filter based on the directional parameter of the prediction block filter. This has been shown to be an effective way to improve the overall filter strength criterion.

In an eighth implementation of the video encoder according to the first aspect, the sequence of filters is:
A sub-pixel interpolation filter configured to adaptively filter the filter samples of the reference block to obtain an interpolated reference block;
A low pass and/or a high pass filter configured to smooth and/or sharpen the interpolated reference block to obtain a filtered reference block,
The one or more primary parameters include interpolation parameters of the interpolation filter, the one or more secondary parameters include selection parameters of the sharpening and/or smoothing filter, and the derivation of the secondary parameters is , Determined by the parameters of the interpolation filter.

The sequence of filters of the video encoder of the eighth implementation may be, for example, a sequence of filters for inter prediction.

Preferably, the filter controller is arranged to determine a selection parameter of the sharpening and/or smoothing filter based on the interpolation parameter. This has been shown to be an effective way to improve the overall smoothness criterion.

Note that the smoothness criterion of the filter sequence can also be position dependent. For example, the filter may introduce strong smoothness in one region and strong sharpness in another region. As such, the filter controller may be configured to set different secondary parameters for different regions of one or more image frames of the video.

In a ninth implementation of video encoding for a video encoder according to the first aspect, the sequence of filters uses a selected codebook to indicate one or more filter coefficients in the bitstream. And an adaptive loop filter configured to: Here, the filter controller is configured to select a codebook from a plurality of codebooks based on the one or more primary parameters.

Adaptive loop filter coefficients encoded by multiple codebooks leverage a priori knowledge of the processing applied to the input signal of the adaptive loop filter. It will be appreciated that if smoothing has already been applied to the signal processed by the adaptive loop filter, then the adaptive loop filter need only introduce high pass filtering. Therefore, some of the adaptive loop filter's coefficient combinations are unavailable. This property is thus duplicated in at least two cases, when the adaptive loop filter is applied to the already smoothed input signal and when no smoothing is applied to the input of the adaptive loop filter. Used to hold the above codebook.

For example, the plurality of codebooks may include first and second codebooks. Here, only the first codebook contains coefficients for both high-pass and low-pass filtering, and the second codebook only contains coefficients for low-pass filtering.

In a preferred embodiment, the codebooks include three or more codebooks, preferably different codebooks of the codebooks correspond to different filtering strengths of filters applied before ALF. ..

In a tenth implementation of the video encoder based on the ninth implementation of the first aspect, the sequence of filters is further:
A deblocking filter configured to process vertical edges based on the vertical filter strength parameter and/or horizontal edges based on the horizontal filter strength parameter;
And a sample adaptive offset (SAO) filter configured to classify the pixels and add an offset value to the pixels according to the SAO class parameter,
The one or more primary parameters include the SAO class parameter and a SAO type parameter of the SAO filter, the one or more secondary parameters include the horizontal filter strength parameter and the vertical strength parameter, The filter controller is configured to derive the secondary parameter based on the SAO class parameter and/or the SAO type parameter of the SAO filter, and/or the filter controller is And is configured to select the codebook from the plurality of codebooks.

The sequence of filters of the video encoder of the tenth implementation may be, for example, a sequence of in-loop filters.

A further implementation of the video encoder of the first aspect relates to a video encoder of one of the above implementations of the first aspect, wherein the secondary parameters of vertical and horizontal edge deblocking filter strength are Differently, the ratio of vertical deblocking filter strength to horizontal deblocking filter strength is adjusted based on the SAO class.

A further implementation of the video encoder of the first aspect relates to a video encoder of one of the above implementations of the first aspect, wherein the sequence of filters does not include an adaptive loop filter, or the filter -The controller does not adjust the parameters of the in-loop filter.

A further implementation of the video encoder of the first aspect relates to a video encoder of one of the above implementations of the first aspect, wherein the sequence of filters does not include a deblocking filter, or the filter The controller does not adjust the deblocking filter parameters.

A second aspect of the invention is a method for constructing a sequence of filters for video coding:
Adjusting one or more secondary parameters based on one or more primary parameters and based on a strength criterion of said sequence of filters,
Configuring the sequence of filters using the primary and secondary parameters,
A method including that.

In a first implementation of the method of the second aspect, the method further comprises the initial step of determining said one or more primary parameters from a bitstream.

The methods according to the second aspect of the invention can be performed by a video coding device according to the first aspect of the invention. Furthermore, further features of the implementation of the method according to the second aspect of the present invention are able to perform the functions of the video encoding device according to the first aspect of the present invention and its various implementations.

A third aspect of the present invention relates to a computer-readable storage medium storing a program code. The program code includes instructions for performing an implementation of the method of the third aspect or one of the implementations of the third aspect.

In order to more clearly illustrate the technical features of the embodiments of the present invention, the accompanying drawings provided for describing the embodiments will be briefly introduced below. The accompanying drawings in the following description depict only some embodiments of the invention. Modifications of these embodiments are possible without departing from the scope of the invention as defined in the claims.
FIG. 3 is a block diagram showing a video encoding device according to an embodiment of the present invention. 6 is a flowchart of a method for configuring a sequence of filters for a video encoder according to a further embodiment of the invention. 6 is a structural scheme of a sequence of filters for intra prediction according to a further embodiment of the invention. 6 is a flowchart of a method for configuring a sequence of filters for controlled reference sample adaptive filtering according to a further embodiment of the invention. 6 is a flow chart flowchart of a method for configuring a sequence of filters for intra prediction according to a further embodiment of the invention. 6 is a flowchart of a method for configuring a sequence of filters for boundary smoothing according to a further embodiment of the invention. 3 is a structural scheme of a filter sequence for intra prediction according to a further embodiment of the invention. 6 is a flowchart of a method for configuring a filter sequence for intra prediction according to a further embodiment of the invention. 6 is a flowchart of a method for intra prediction with a filter control module at the decoder side, according to a further embodiment of the invention. 6 is a structural scheme of a serial-parallel embodiment of inter prediction with a filter control module according to a further embodiment of the present invention. 6 is a flowchart of a method for configuring a sequence of filters for inter prediction with a filter control module at the decoder side, according to a further embodiment of the invention. 3 is a structural scheme of an in-loop filter chain according to a further embodiment of the present invention. 6 is a flowchart of a method for deblocking filtering to configure a sequence of filters that depends on one or more SAO parameters, according to a further embodiment of the invention. 6 is a flowchart of a further method for deblocking filtering depending on SAO parameters, which filtering is performed at the decoder side, according to a further embodiment of the invention. A method for configuring a sequence of filters according to a further embodiment of the invention, wherein the method adjusts one or more ALF parameters according to one or more SAO parameters at a decoder side. It is a flowchart. FIG. 6 is a flow chart of a method of configuring a sequence of filters according to a further embodiment of the invention, wherein the method adjusts one or more ALF parameters according to one or more SAO parameters at the decoder side.

FIG. 1 shows a video encoder 100 having a sequence of filters 110 and a filter controller 120.

The sequence of filters 110 is configurable with one or more primary parameters and one or more secondary parameters. For example, a first set of filters in the sequence of filters may be configured by a primary parameter and a second set of filters in the sequence of filters may be configured by a second set of filters. There may be overlap in the first set of filters and the second set of filters.

The filter controller 120 is configured to adjust the one or more secondary parameters based on the one or more primary parameters and based on an intensity metric of the sequence of filters 110. In particular, the filter controller 120 can be configured to adjust the one or more secondary parameters based on the one or more primary parameters. For example, the value of the secondary parameter may be based in part on the adjustment based on a predetermined value, eg from the bitstream, and in part on the primary parameter.

FIG. 2 shows a method 200 for configuring a sequence of filters for video coding. Method 200 includes a first step 210 of adjusting one or more secondary parameters based on the one or more primary parameters and based on an intensity criterion of the sequence of filters. The method further includes a second step 220 of configuring the sequence of filters with the primary and secondary parameters.

The intra prediction procedure can be part of a hybrid video coding tool chain on the encoder side and/or the decoder side. Similarly, the inter prediction procedure can include a sequence of filters (eg, interpolation filters and so-called prediction filters). These filters can cause over-smoothing or over-sharpening of blocks that are used as references that are actually analogs of intra-predicted blocks for inter prediction.

The sequence of filters can include, for example, one or more of the following filters:
Reference sample smoothing (eg RSAF),
-Interpolation filtering for intra prediction,
Filtering intra-predicted blocks (eg MPI or PDPC) and/or boundary smoothing.
These filters can affect the results of intra prediction by smoothing.

FIG. 3 shows a filter sequence 300 having a filter control module 360 for adjusting filtering parameters at various stages for intra prediction. The filter controller module 360 is a filter controller.

Intra-prediction parameters may include, but are not limited to:
・Size of prediction unit,
The predicted block size,
・Intra prediction mode,
Multi-parameter intra mode index and/or reference sample filtering flags.
One or more of the above parameters can be primary or secondary parameters.

Apart from the filter control module 360, the sequence of filters 300 comprises a reference sample smoothing unit 310, an intra prediction unit 320, a prediction block filter unit 330 and a boundary smoothing unit 340. The reference sample smoothing unit 310 is configured to receive as input one or more neighboring samples 302. Reference sample smoothing unit 310 is further configured to smooth and/or further process said one or more neighboring samples 302 to obtain one or more reference samples 312. The reference sample 312 is provided as an input to the intra prediction unit 320. The intra prediction unit 320 has an interpolation filter 322. Intra prediction unit 320 provides its output 324 as an input to prediction block filter 330.

Predicted block filter 330 is configured to calculate one or more predicted blocks 332. Predicted block 332 is provided as an input to boundary smoothing unit 340. Boundary smoothing unit 340 produces as output 342 one or more intra-predicted blocks 350.

A video encoder having a sequence of filters 300 can be configured to use the implicit or explicit signaling of a reference sample filter selectively, ie only for TUs that meet certain conditions.

The filter control module 360 can be configured to read the intra prediction parameter 362 as a primary parameter. The filter control module 360 can be configured to derive secondary parameters based on these primary parameters.

The quadtree partitioning results can be used as an indication of reference sample filter selection using explicit or implicit signaling. Specifically, the reference sample filter flag is set to 0 if the size of the PU is greater than a threshold (eg 32×32). This assignment overrides the prior art conditions. If a certain condition of PU size is true, only "no filter" and "apply weak filter" can be selected according to PU size and/or intra mode condition.

FIG. 4 is a flowchart of a method 400 for configuring a sequence of filters for controlled reference sample adaptive filtering.

Method 400 includes a first step 402 of evaluating a condition related to the size of a prediction unit. If the evaluation of the condition is true, the method continues to step 404, where a reference sample filter flag is derived. If the condition related to the size of the prediction unit evaluates to false, the method continues to step 406, where the reference sample filter flag is set to false. Step 404 or step 406 is followed by step 408, in which one or more conditions related to intra mode and block size are evaluated.

If the evaluation result of step 408 is false, the method continues to step 410 and the reference sample filter flag is evaluated. If the flag is false, the method continues to step 414 and the reference sample adaptive filter is set to not apply the filter. If at step 410 the flag evaluates to true, then at step 416 a weak filter is applied. Alternatively, if the condition evaluation at step 408 evaluates to true, the method continues to step 412 and the reference sample filter flag is evaluated. If the evaluation is false, then in step 416, the weak filter is applied. If the reference sample filter flag evaluates to true in step 412, the strong filter is applied in step 418.

The "weak filter" and "strong filter" stages map a filter from a predefined set of filters to a selection of a particular filter from the set in intra mode and predicted block size. You can select it immediately. This particular embodiment, which has only three filters, allows for the amount of filters in the filter set to be any amount (e.g., "no filter", "weak filter", "strong filter", and five intermediate filters). It does not mean that it cannot be extended to (state).

In directional intra prediction, the values of pixels of the predicted block and the projections at the left and upper block boundaries are calculated. However, the projection may have odd positions. That is, between the actual positions of the reference samples on the boundaries. A weighted sum of the values of adjacent reference samples is calculated to determine the value of the sample of the intra-predicted block. This process is actually a two-tap interpolation filter, which can be further extended to a four-tap interpolation filter.

A four-tap intra prediction filter can be used to improve directional intra prediction accuracy. In HEVC, a two-tap linear interpolation filter was used to generate intra-predicted blocks in directional prediction modes (ie, without Planar and DC predictors). Alternatively, two types of four-tap interpolation filters can be used: cubic interpolation filters for 4x4 and 8x8 blocks and Gaussian interpolation filters for 16x16 and larger blocks. The filter parameters are fixed according to block size, and the same filter is used for all predicted pixels in all directional modes.

In HEVC, after the intra prediction block is generated for the VER and HOR intra modes, the prediction samples of the leftmost column and the topmost row are further adjusted, respectively. This can be further extended to several diagonal Intra modes, where up to four row or column boundary samples can be further 2-tap (for intra-mode 2 & 34) or 3-tap filter ( Adjusted using Intra Mode 3-6 & 30-33).

4 and 5 show two embodiments for synchronizing interpolation filter type selection using a reference sample filtering process. Both embodiments contemplate that two weak and strong interpolation filter types can be applied. For example, a Gaussian filter is used for 16x16 and larger predicted blocks, and a cubic filter is selected for other block sizes. For either embodiment, the interpolation filter selection can be coordinated with the reference sample filtering process.

FIG. 5 is a flowchart of a method 500 for configuring an interpolation filter for intra prediction.

Method 500 includes a first step 502 of deriving a reference sample filter flag. At step 504, the reference sample filter flag is evaluated. If it evaluates to true, the method continues to step 506 where the condition related to the size of the transform block is evaluated. If the condition evaluates to false, the method continues to step 508, where a weak intra-interpolation filter is applied. Similarly, if the reference sample filter flag evaluates to false at step 504, the method also continues at step 508. If the condition related to the size of the transform block evaluates to true in step 506, the method continues to step 510 and a strong intra interpolation filter is applied.

The embodiments of FIGS. 4 and 5 differ with respect to the reference sample filter flag derivation. In the embodiment of FIG. 4, the reference sample filter flag is true if the predicted block based on the condition has a different reference sample filter option. This filter selection can be signaled explicitly or implicitly (eg by mapping to the prediction mode or by using data hiding in the quantized residual data). For the embodiment of FIG. 4, the actual selected reference filter value is not considered in the reference sample filter flag derivation. However, if the reference sample filter selection is performed at the encoder side for the predicted block and the selection is communicated to the decoder either explicitly or implicitly, the reference sample flag value is true. Otherwise, if the predicted block has a predefined reference filter or has no reference sample filtering, then the reference sample flag value is false.

The embodiment of FIG. 5 uses the reference filter selection value as the reference sample flag value. The reference sample flag value is assigned true if a strong filter is selected for the reference filter (eg, {1 2 1} or a five tap filter). Then, as in the first embodiment, the reference sample flag value is false for the predicted block with no reference filtering or the weak reference sample filter selected.

The beneficial effects of the above-described embodiments are achieved by the harmonization of reference sample filtering and intra prediction interpolation processes. It can be observed that these embodiments prevent the predicted block from being too smooth.

FIG. 6 is a flowchart of a method 600 for configuring a sequence of filters for boundary smoothing.

Method 600 includes a first step 602 of determining whether the predicted block filter is directional. If so, the method continues to step 604, where the block size condition is evaluated. If the block size condition evaluates to false or the predicted block filter directionality evaluates to false, the method continues to step 606, where the intra mode condition is evaluated. If the intra mode condition evaluates to true, the method continues to step 608, where boundary smoothing is applied. Otherwise, and if the block size condition evaluates to true in step 604, no boundary smoothing is applied.

Boundary smoothing can be applied when the intra prediction mode is selected to be DC, horizontal, vertical or skew. The proposed invention synchronizes boundary smoothing with the selection of filters for prediction blocks. In particular, the predictive block filter directionality is used to make the decision whether to apply boundary smoothing. For example, if a two-dimensional filter has the same strength both vertically and horizontally, then the filter is non-directional. In particular, no boundary smoothing is applied for non-directional filters. Multi-parameter intra prediction can be an example of a predictive block filter. When this technique is used as a predictive block filter, the first condition in FIG. 6 can be formulated as “MPI index is greater than 1”.

In the opposite case where the predictive block filter is directional, the invention considers another constraint. If the size of the filtered block is less than 32 pixels, then boundary smoothing is skipped for this block, despite the predictive block filter directionality.

FIG. 7 is a structural scheme of a filter sequence 700 for intra prediction. The filter sequence 700 can be used to encode or decode video.

The filter sequence 700 comprises a filter control module 760 configured to adjust the parameters of some filters. Specifically, the filter sequence 700 receives as input one or more neighborhood samples 702. These neighborhood samples 702 are provided as inputs to a sample adaptive filter SAF 710. Filter 710 represents the first filter of filter sequence 700. Reference sample adaptive filter 710 produces one or more reference samples 712, which are provided as inputs to intra prediction unit 720. The intra prediction unit 720 has a set of four tap interpolation filters 722, which are configurable by one or more interpolation filter parameters.

The output 724 of the intra prediction unit is provided as an input to the boundary prediction filter 730. The output 732 of the boundary prediction filter is provided as an input to the combined multi-parameter intra prediction/position dependent intra prediction unit 740. Unit 740 produces one or more intra-predicted blocks 750 as output 742.

The reference sample adaptive filter 710, the intra prediction unit 720, the boundary prediction filter 730 and the multi-parameter intra prediction/position dependent intra prediction combination unit 740 can each be configured by one or more parameters, and those parameters can be set. Can be set by the filter control module 760.

Multi-parameter intra prediction (MPI) is a post-processing for intra prediction that calls for additional smoothing using the decoded boundaries. this is,

として実装されることができる。ここで、当該ブロックの外では、P_MPI[i,j]は再構成された信号

Can be implemented as. Here, outside the block, P _MPI [i,j] is the reconstructed signal

に等しい。この後処理の強さ（パラメータα＋β＋γ＋δ＝8）は、CUレベルで制御され、高々2ビットを用いて信号伝達されることができる。

be equivalent to. The strength of this post-processing (parameter α+β+γ+δ=8) is controlled at the CU level and can be signaled using at most 2 bits.

Position Dependent Intra Prediction Combination (PDPC), which can replace MPI, is a content-adaptive post-processing for intra prediction that calls the above combination of intra prediction and unfiltered boundaries. This can be implemented as follows.

ここで、(i,j)は左上コーナーに対するサンプル位置を示し、temp(i,j)はi≧0、j≧0については上記のイントラ予測に等しく、i＝−1、j＝−1についてはフィルタリングされない参照に等しい。

Where (i,j) is the sample position for the upper left corner, temp(i,j) is equal to the above intra prediction for i≧0, j≧0, and for i=−1, j=−1 Is equal to the unfiltered reference.

The strength of this post-processing can be controlled by the parameter α+β+γ=8. The different sets of {α, β, γ} make up the dictionary summarized in Table 1. The strength of the post-processing smoothing is different for blocks coded as 2N×2N and N×N. The same post-processing can be applied for both the luminance and chrominance blocks inside the CU.

後処理を決定する組み合わされたイントラ・インデックスは、CUレベルで2ビットを用いて信号伝達される。このシンタックス要素は、CUの左または上の境界がピクチャー境界である場合には信号伝達されない。このインデックスの0の値は、後処理が使われないことを示す。

The combined intra index that determines the post-processing is signaled with 2 bits at the CU level. This syntax element is not signaled if the left or upper boundary of the CU is a picture boundary. A value of 0 for this index indicates that no post processing is used.

If each tool has a flag to turn it on and off and the RDO procedure is run jointly for all tools that use smoothing instead of separately for each tool, the problem of oversmoothing is Can overcome However, this solution has the drawback of redundant signaling, which can reduce the overall coding performance.

FIG. 8 is a flowchart of a method 800 for configuring a filter sequence for intra prediction.

Method 800 includes a first step 810 of determining whether reference sample adaptive filtering (RSAF) is being used. For example, this can be determined by evaluating the RASF flag. If RSAF is determined to be used, then in step 812 reference sample adaptive filtering is applied. Then, in step 814, interpolation using a 4-tap cubic filter is applied. This can be achieved by setting the interpolation mode parameter of the interpolation filter of the filter sequence for intra prediction to a four tap third order filter.

If at step 810 it is determined that RSAF is not used, then at step 816 interpolation using a set of four tap filters is applied. In particular, this can be a predetermined set of four tap filters. Setting the interpolation filter to use the set of four-tap filters can be accomplished, for example, by setting the interpolation parameter of the interpolation filter to "set of four-tap filters".

In a further step 820, the MPI index variable i is set to 0. Then, in step 822, it is determined whether the variable is greater than one. If so, in step 824 it is determined if the current prediction unit PU size is greater than 32. If so, in step 826 border prediction filtering is applied. Boundary prediction filtering is also applied if it is determined in step 822 that i is not greater than one. If in step 824 it is determined that the current PU size is not greater than 32, the method continues to step 828 with a multi-parameter intra prediction/position dependent intra prediction combination.

Then, in step 830, the rate distortion cost, RD cost J _i , for the current configuration setting is calculated. In particular, the current configuration setting can correspond to the current value of MPI index i. In other embodiments, other parameters may be changed and RD costs for different parameter settings may be determined.

In step 832, it is determined whether MPI index i is greater than 3. Otherwise, the MPI index is incremented by 1 in step 834 and the method continues in step 822. If the MPI index is greater than or equal to 3, the method continues to step 836 by selecting the best MPI index. This can be achieved by choosing the MPI index that corresponds to the lowest RD cost J _i .

FIG. 9 is a flowchart of a method 900 for configuring a sequence of filters for intra prediction at the decoder side with a filter control module.

The inter prediction mechanism may include, for example, the following filters:
・Regular filter based on Lagrange interpolation,
A sharpening filter using DCT-based interpolation (usually around relatively sharp edges), and/or a smoothing non-interpolating filter.

Preferably, only one of these filters can be selected. For quarter pixel interpolation, the sharpening filter can be enabled by default. That is, the parameters should be obtained at the decoder side by parsing the bitstream without deriving any other flags or parameters. For half-pixel interpolation, sharpening can be turned off. For integer pixels, both sharpening and smoothing filters are enabled and thus can be switched off when needed. However, if one of them is turned on, its parameters should be obtained from the bitstream before performing the filtering.

Method 900 includes a first step 910 of determining whether reference sample adaptive filtering (RSAF) is being used. For example, this can be determined by evaluating the RASF flag. If it is determined that RSAF is used, then at step 912, reference sample adaptive filtering is applied. Then, in step 914, interpolation using a 4-tap cubic filter is applied. This can be achieved by setting the interpolation mode parameter of the interpolation filter of the filter sequence for intra prediction to a four tap third order filter.

If at step 910 it is determined that RSAF is not used, then at step 916 interpolation using a set of four tap filters is applied. In particular, this can be a predetermined set of four tap filters. Setting the interpolation filter to use the set of four-tap filters can be accomplished, for example, by setting the interpolation parameter of the interpolation filter to "set of four-tap filters".

At step 920, it is determined whether the MPI index is greater than one. The MPI index may have been determined, for example, by parsing or otherwise determining the MPI index value from the bitstream. If the MPI index is greater than 1, the method proceeds to step 924 by determining if the current PU size is greater than 32. If so and if the MPI index is not greater than 1, the method continues to step 926 by applying boundary prediction filtering. However, if the current PU size is not greater than 32, the method continues to step 928 with a multi-parameter intra prediction/position dependent intra prediction combination.

FIG. 10 is a structural scheme of a filter sequence 1000. Sequence 1000 has a serial/parallel structure.

The filter sequence 1000 is configured to process a block of samples 1002 to obtain a reference (predicted) block 1040. Sequence 1000 includes one or more subpixel interpolation filters 1010 configured to interpolate between block samples 1002. The result 1012 of the subpixel interpolation is applied as input to the smoothing filter 1020 and/or the sharpening filter 1030. Preferably, smoothing filter 1020 or sharpening filter 1030 is used.

The output of smoothing filter 1020 and/or sharpening filter 1030 is a reference (predicted) block 1040. The filter sequence 1000 is controlled by the filter control module 1050. The filter control module 1050 is configured to set parameters of the sub-pixel interpolation filter 1010, the smoothing filter 1020 and/or the sharpening filter 1030.

FIG. 11 is a flowchart of a method of configuring a filter sequence for inter prediction using a filter control module on the decoder side.

Method 1100 includes a first step of determining if 1/4 3/4 pixel interpolation is being used. Otherwise, the method proceeds to step 1104 by determining if 1/2 pixel interpolation is used. If so, the method continues to stage 1106 by parsing the bitstream to obtain one or more values of control flags. Then, in step 1112, it is determined whether the sharpening filter is enabled. In particular, this can be determined from the one or more values of the control flag determined in step 1106. Alternatively, if at step 1104 it is determined that 1/2 pixel interpolation is not being used, then the method parses the bitstream to obtain one or more values of one or more control flags. Continue to 1108. Then, in step 1110, it is determined whether the smoothing filter is enabled. In particular, this can be determined from the one or more values of the one or more control flags.

If the smoothing filter is enabled, the method continues to step 1116, where the bitstream is parsed to obtain one or more values of the smoothing filter strength parameter and filtering is performed accordingly. Alternatively, in step 1110, if it is determined, for example, from the bitstream that the smoothing filter is not enabled, the method continues to step 1112 and it is determined whether the sharpening filter is enabled. If the sharpening filter is enabled, the method continues to step 1114 where the bitstream is parsed to obtain one or more values of the sharpening filter strength parameter and filtering is performed accordingly.

State-of-the-art video coding also provides a final filtering stage for encoders and decoders. As soon as the output data of this process is passed to the motion compensation loop, this filtering is called in-loop filtering.

Some sequences of filters can be used in both encoders and decoders. Preferably, the first stage of the sequence of filters is arranged to remove blocking artifacts by using a deblocking filter. The low pass filter is applied to the edges of the TU according to a predefined set of rules. These rules have a parameter called the deblocking parameter that can be specified individually for the entire sequence or for each frame.

Preferably, the second stage is arranged to remove quantization noise by using a sample adaptive offset. The frame can be subdivided into pixel areas, and each of these areas is assigned SAO parameters. SAO parameters can include:
● SAO type that controls the classifier type:
BO (band offset): This SAO type chooses to add an offset to pixels with values within the range specified by the SAO category. o EO (edge offset): This SAO type chooses to add different offsets to pixels depending on the SAO category ● SAO class: Specifies the pattern of pixels that should be used to derive the SAO category ● SAO offset: Offset to each SAO category Lookup table to assign. The corresponding offset should be added according to the pixel category.

Preferably, one or more of these SAO parameters are derived on the encoded side and encoded in the bitstream. This allows the decoder to parse those parameters.

The next stage is to apply an adaptive loop filter (ALF) that is fairly close to the Wiener filter. At the encoder side, after filtering the reconstructed pixels, the filter coefficients that give the smallest mean square error are derived. These coefficients are further quantized and signaled to the decoder in the bitstream.

Preferably, a filter control module for adapting the filter strength in different stages of the in-loop filtering chain adjusts the processing in one filtering stage according to the value of the parameter in the other stage(s). Configured to do.

FIG. 12 is a structural scheme of an in-loop filter chain 1200 that is a sequence of filters.

The in-loop filter chain 1200 is configured to process the reconstructed frame 1202 to obtain the reference picture buffer 1240. The reconstructed frame 1202 is provided as an input to the deblocking filter 1210. The output of the deblocking filter is provided as input 1212 to the sample adaptive offset filter 1220. The output of sample adaptive offset filter 1220 is provided as input 1222 to adaptive loop filter 1230. The output of adaptive loop filter 1230 is provided as input 1232 of reference picture 1240.

Deblocking filter 1210, sample adaptive offset filter 1220 and adaptive loop filter 1230 are configurable with parameters that can be set by filter control module 1250. The filter control module 1250 may determine these parameters based on input parameters including one or more deblocking filter parameters 1252, one or more SAO parameters 1254 and one or more ALF parameters 1256. Composed. For example, these parameters are user defined or derived from the bitstream.

The deblocking filter 1210 may rely on one or more SAO parameters. Preferably, on the encoder side, if the SAO type is selected to be EO (edge offset) and the pixel pattern (SAO class) is assigned horizontally or vertically, the deblocking operation is performed on the selected pixel pattern. Disabled for edges with a direction perpendicular to the direction.

FIG. 13 is a flowchart of a method 1300 for configuring a sequence of filters when deblocking filtering depends on one or more SAO parameters.

Method 1300 includes a first step 1302 of estimating one or more SAO parameters. SAO parameters can include edge offset parameters and SAO classes.

Step 1304 determines if the edge offset parameter is set to true. If so, in step 1306 the SAO class parameters are evaluated. If the SAO class parameter is set to vertical, the method disables horizontal deblocking in step 1308. If the SAO class parameter is set to horizontal, then vertical edge deblocking is disabled at step 1310. If the SAO class is set to another value, or if the edge offset parameter is determined to be false in step 1304, the method applies a deblocking filter according to the configuration settings determined above. Continue to 1312. Thereafter, the method continues with sample adaptive offset filtering stage 1314 and possibly further filtering stages.

FIG. 14 is a flowchart of a further method 1400 of configuring a sequence of filters when deblocking filtering depends on SAO parameters and filtering is done at the decoder side.

In the first step 1402, one or more SAO parameters are retrieved from the bitstream.

At step 1404, it is determined if the edge offset parameter is set to true. If so, in step 1406 the SAO class parameters are evaluated. If the SAO class parameter is set to vertical, the method disables horizontal edge deblocking in step 1408. If the SAO class parameter is set to horizontal, then vertical block deblocking is disabled at step 1410. If the SAO class is set to another value, or if the edge offset parameter is determined to be false in step 1404, the method applies a deblocking filter according to the configuration settings determined above. Continues to 1412. The method then continues with the step 1414 of sample adaptive offset filtering and possibly further filtering steps.

FIG. 15 is a flowchart of a method 1500 of configuring a sequence of filters. Here, method 1500 adjusts one or more adaptive loop filter parameters based on the one or more SAO parameters at the decoder side.

In particular, method 1500 includes a first step 1502 of estimating SAO parameters. In the second step 1504, it is determined whether the edge offset flag is set. If so, then in step 1506 the smoothing filter in the adaptive loop filter ALF is disabled.

Thereafter, in steps 1508 to 1514, deblocking, sample adaptive offset filtering, adaptive loop filter parameter estimation and adaptive loop filtering are applied.

The codebook selections for encoding and decoding ALF parameters are different, especially for the case of edge-offset and band-offset SAOs, as compared to the methods shown in FIGS. 13 and 14, for example.

FIG. 16 is a flowchart of a method 1600 for adjusting one or more ALF parameters in response to one or more SAO parameters at an encoder side.

In the first stage 1602, SAO parameters are derived from the bitstream. In the second step 1604, it is determined if the edge offset flag is set. If not, then at step 1606, the first ALF parameter codebook is selected. If the edge offset flag is not set, then in step 1608 the second ALF parameter codebook is selected.

Then, in steps 1610 to 1616, ALF parameters are obtained from the bitstream, deblocking filters are applied, sample adaptive offset filtering is applied, and adaptive loop filtering is applied.

Embodiments of the invention can relate to the following additional aspects:
[Side 1]
a. Configure the filter parameters in the stages given below according to the intra prediction parameters b. Providing reference samples by adaptively applying filters to neighboring blocks of the predicted block.
c. Computing the predicted value for each sample of the predicted block using the interpolated value of the reference sample d. Applying a filter to the predicted block e. A method of encoding and decoding video data, comprising an intra prediction process consisting of performing boundary smoothing.
[Side 2]
The method of aspect 1, wherein the quadtree splitting result is used as an indication of reference sample filter selection using explicit or implicit signaling.
[Side 3]
The method of aspect 1, wherein calculating the predicted value for each sample of the predicted block is performed with an interpolation filter selected according to a reference sample filtering process.
[Side 4]
The method of aspect 3, wherein interpolated filter selection is performed for predicted blocks calculated without implicit or explicit signaling of reference sample filter selection.
[Side 5]
The method of aspect 3, wherein interpolation filter selection is performed on predicted blocks obtained from reference samples filtered by the weak reference sample filter.
[Side 6]
The method according to aspect 1, wherein boundary smoothing is performed on TUs that belong to PUs that satisfy the size constraint condition.
[Side 7]
The method of aspect 6, wherein the predictive block filter orientation affects the decision to apply boundary smoothing.
[Side 8]
a. Configure the filter parameters in the stages given below according to the intra prediction parameters b. Providing a reference sample by adaptively applying a filter to the search region used to find the block used as a reference after processing by the interpolation and prediction filters c. Applying an interpolation filter to the block currently being processed d. A method of encoding and decoding video data, comprising an inter prediction process consisting of applying a prediction filter to the block currently being processed.
[Side 9]
The method according to aspect 8, wherein step d precedes step c.
[Side 10]
A method of filtering a signal,
a. A number of successive iterative filtering steps, the filter strength of which depends on additional conditions,
b. Instructions relating to some of the filtering steps, the instructions specifying a filtering strength in the relevant steps,
c. A control unit for overriding additional conditions in the filtering stage a depending on the indications associated with the preceding stage.
Method.
[Side 11]
11. The method according to aspect 10, wherein the indication of a strong filter in the filtering step i overrides an additional condition check in the step k>i of selecting a weak filter.

Embodiments of the invention provide one or more of the following advantages.
• Many potential applications in the underlying JEM-compatible hybrid video coding framework for next generation video coding standards;
-Reduced BD rate and subjective quality improvement compared to JEM1.
-Reduced computational complexity of both encoder and decoder compared to JEM1 with integrated RSAF. This makes the present invention potentially attractive for many mobile applications.
-Avoid redundant signal transmission (syntax).

The above description is merely an implementation mode of the present invention, and the scope of the present invention is not limited to this. Any changes and substitutions can be easily made by those skilled in the art. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.

Claims (15)

A sequence of filters, the sequence of filters being configurable by one or more primary parameters and one or more secondary parameters;
A video encoder, the filter controller being configured to set the one or more secondary parameters based on the one or more primary parameters and based on a strength criterion of the sequence of filters. And
The sequence of filters is:
A reference sample filter configured to adaptively filter one or more neighboring samples of the current video block to obtain one or more reference samples;
An interpolation filter configured to predict one or more samples of the current video block using interpolation of the one or more reference samples
Video encoder. The strength criteria are:
・Smoothness standard,
A video coding device according to claim 1, comprising: a ratio of an amplification factor for a high frequency region and an amplification factor for a low frequency region, and/or a ratio of a contrast value before filtering and a contrast value after filtering. The one or more primary parameters are predetermined parameters, in particular those which are predetermined from an encoded bitstream, a user setting and/or a parameter search loop in the encoding device. 2. The video encoding device according to 2. The sequence of filters is one or more primary filters configurable by one or more primary parameters and one or more secondary filters configurable by one or more secondary parameters. 4. A video code according to any one of claims 1 to 3, including: and the one or more first order filters are located before the one or more second order filters in the sequence of filters. Device. The one or more primary parameters include a reference parameter of the reference sample filter, the one or more secondary parameters include a selection parameter of the interpolation filter, and the interpolation filter interpolates according to the selection parameter. 5. A video coding device according to any one of claims 1 to 4, adapted to use the method. 6. The one or more first order parameters include a reference sample filter flag of a reference sample filter and the one or more second order parameters include a filter strength parameter of an intra prediction interpolation filter. The video encoding device according to any one of the above. The sequence of filters comprises a boundary smoothing filter configured to perform boundary smoothing for one or more transform units belonging to one or more prediction units that satisfy a size constraint. 7. The video encoding device according to claim 6. 8. The one of claim 1-7, wherein the one or more first order parameters include a prediction block filter directional parameter and the one or more second order parameters include a boundary smoothing filter on-off parameter. The video encoding device according to any one of claims. The sequence of filters is:
A sub-pixel interpolation filter configured to adaptively filter the filter samples of the reference block to obtain an interpolated reference block;
A low pass and/or high pass filter configured to smooth and/or sharpen the interpolated reference block to obtain a filtered reference block,
The one or more primary parameters include interpolation parameters of the interpolation filter, the one or more secondary parameters include selection parameters of the sharpening and/or smoothing filter, the filter controller A secondary parameter is configured to be derived based on the interpolation parameter of the interpolation filter,
The video encoding device according to claim 1. The sequence of filters includes an adaptive loop filter configured to use a selected codebook to indicate one or more filter coefficients in a bitstream, the filter controller including: The video coding device according to any one of claims 1 to 9, which is configured to select a codebook from a plurality of codebooks based on a plurality of primary parameters. The sequence of filters further:
A deblocking filter configured to process vertical edges based on the vertical filter strength parameter and horizontal edges based on the horizontal filter strength parameter;
A sample adaptive offset, a SAO, a filter configured to classify the pixel and add an offset value to the pixel according to a SAO class parameter,
An adaptive loop filter configured to use more than one codebook to indicate one or more filter coefficients in the bitstream,
The one or more primary parameters include the SAO class parameter and a SAO type parameter of the SAO filter, the one or more secondary parameters include the horizontal filter strength parameter and the vertical strength parameter, The filter controller is configured to derive the secondary parameter based on the SAO class parameter and/or the SAO type parameter of the SAO filter, and/or the filter controller is Configured to select the two or more codebooks based on
A video encoding device according to any one of claims 1 to 10. The video coding device according to any one of claims 1 to 11, wherein the one or more secondary parameters are not included in a bitstream. A method for configuring a sequence of filters for video encoding, the method comprising:
Setting one or more secondary parameters based on one or more primary parameters and based on the strength criteria of said sequence of filters,
Configuring the sequence of filters using the primary and secondary parameters,
Including that,
The sequence of filters is:
A reference sample filter configured to adaptively filter one or more neighboring samples of the current video block to obtain one or more reference samples;
An interpolation filter configured to predict one or more samples of the current video block using interpolation of the one or more reference samples
Method. 14. The method of claim 13, further comprising the initial step of determining the one or more primary parameters from a bitstream. A computer readable storage medium having program code stored thereon, the program code comprising instructions for performing the method of claim 12 or 13.

Images