RU2781233C2 - Device and filtration method in video encoding - Google Patents

Abstract

FIELD: video encoding.

SUBSTANCE: invention relates to the field of video encoding, more specifically to a filter for video encoding and a method for filtration of restored video-frames. A filter for video encoding is proposed, which is made with the possibility of processing of a block for the formation of a filtered block, wherein the block contains a set of pixels. The filter includes one or several processors made with the possibility of: scanning of the current pixel of the block and adjacent pixels of the current pixel in accordance with a preset scanning pattern; obtaining spectrum components by means of conversion to the current pixel and its adjacent pixels; obtaining filtered spectrum components based on a filtration parameter and spectrum components; obtaining filtered pixels by means of reverse conversion for filtered spectrum components; and formation of a filtered block based on filtered pixels.

EFFECT: increase in the efficiency of video encoding/decoding.

33 cl, 26 dwg, 4 tbl

Description

The field of technology to which the invention belongs

[0001] In General, the present invention relates to the field of video coding. More specifically, the present invention relates to a video encoding filter, a reconstructed video frame filtering method, and a video block filtering method, as well as an encoding apparatus and a decoding apparatus comprising such a video encoding filter.

State of the art

[0002] Digital video has been widely used since the advent of DVDs. Before transmission, the video is encoded and transmitted using a transmission medium. The viewer receives the video and uses the viewer to decode and display the video. Video quality has improved over the years, for example with higher resolutions, color depth and frame rates. This has led to an increase in the data flows that are now commonly transmitted over the Internet and mobile networks.

[0003] However, higher resolution videos usually require more bandwidth because they contain more information. To reduce bandwidth requirements, video coding standards have been introduced that involve video compression. When video is encoded, bandwidth requirements (or corresponding memory requirements in the case of storage) are reduced. Often this decline comes at the expense of quality. Thus, video coding standards try to strike a balance between bandwidth requirements and quality.

[0004] Since there is a constant need to improve quality and reduce bandwidth requirements, solutions are constantly being sought that maintain quality with reduced bandwidth requirements or improve quality while maintaining bandwidth requirements. Also, compromises can sometimes be acceptable. For example, it may be acceptable to increase the bandwidth requirements if the quality improvement is significant.

[0005] High Efficiency Video Coding (HEVC) is an example of a video coding standard that is commonly known to those skilled in the art. In HEVC, to separate the coding unit (CU - coding unit) into prediction units (PU - prediction unit) or transformation units (TU - transform unit). The Next Generation Versatile Video Coding (VVC) standard is the latest joint video project of the ITU-T Video Coding Experts Group (VCEG) and the Moving Picture Experts Group (MPEG) Picture Experts Group) ISO/IEC working together in a partnership known as the Joint Video Exploration Team (JVET). VVC is also referred to as ITU-T H.266/Next Generation Video Coding (NGVC). In VVC, it removes the concepts of multiple partition types, i.e. removes the separation of the concepts of CU, PU, and TU, except when necessary for CUs that are too large for the maximum transform length, and maintains more flexibility for the forms of CU separation.

[0006] Image filtering is often used to highlight certain characteristics of an image or to improve the objective or perceived quality of a filtered image. Image filtering encounters various sources of noise. Accordingly, various approaches to quality improvement have been proposed and are being used. For example, in the Adaptive Loop Filter (ALF) method, each reconstructed frame is divided into a set of small blocks (superpixels), and each block is filtered by an adaptive loop filter in the sense that each pixel of the filtered reconstructed frame is a weighted sum of several pixels in the associated pixel area from the reconstructed frame around the position of the generated filtered pixel. The weight coefficients (also called filter coefficients) have the property of central symmetry and are transmitted from the encoder to the decoder side. Edges are often large and therefore the number of weights to be transferred can become too large for efficient processing. A large number of weights requires complex rate-distortion optimization (RDO) on the encoder side to reduce the number of weights for transmission. On the decoder side, ALF requires the implementation of universal multipliers, and these multipliers must be reset for every 2x2 pixel block.

[0007] Thus, there is a need for an improved filter and method that can improve the quality of low complexity prediction and thus improve video coding efficiency.

The essence of the invention

[0008] It is an object of the invention to provide an improved filter and method for improving filtering efficiency with limited complexity, and thus improving video coding efficiency.

[0009] The above and other objects are solved by the subject matter of the independent claims. Additional forms of implementation are apparent from the dependent claims, description and figures.

[0010] According to a first aspect, the invention relates to a filter for video encoding, wherein the filter is configured to process a block to form a filtered block, the block comprising a plurality of pixels. The filter includes a storage device containing instructions; and one or more processors associated with the memory. One or more processors execute instructions to: load the current pixel and its neighboring pixels into a linear buffer in accordance with a predetermined scan pattern; obtaining spectrum components by performing a one-dimensional pixel transformation in a linear buffer; obtaining filtered components of the spectrum by multiplying each component of the spectrum by the gain, while the gain depends on the corresponding component of the spectrum and the filtering parameter; obtaining filtered pixels by performing an inverse one-dimensional transform on the filtered spectrum components; and generating a filtered block based on the filtered pixels.

[0011] As an example, a block (or frame) may be a predicted block, and a filtered block is a filtered predicted block. As another example, a block (or frame) may be a reconstructed block and a filtered block is a filtered reconstructed block.

For example, the gain is a function of the corresponding spectrum component and the filter parameter. The filtering parameter can be obtained from the codec quantization parameter (QP - quantization parameter).

усиления для первого компонента спектра равен единице. Первый компонент спектра соответствует сумме или среднему значению сэмплов в линейном буфере, и первый компонент спектра может соответствовать DC.Another example, the first component of the spectrum is passed without filtering when the coefficient

gain for the first component of the spectrum is equal to unity. The first spectrum component corresponds to the sum or average of the samples in the linear buffer, and the first spectrum component may correspond to DC.

As another example, in which one or more processors execute instructions to remove N bits from LUT table values, N is an integer. N may depend on the value of QP or be a fixed value.

For example, a predetermined scan pattern is defined as a set of spatial or raster offsets relative to the position of the current pixel within the reconstructed block. Point offsets to neighboring pixels are inside the reconstructed block. At least one filtered pixel may be placed in its original position according to a predetermined scan pattern. When all the filtered pixels are added to the accumulation buffer according to a predetermined scan pattern, and the accumulation buffer can be initialized to zero until the filtered spectrum components are obtained. When the finally filtered pixels are obtained as the accumulated values in the accumulation buffer divided by the number of pixels added to the current position of the accumulation buffer; one or more processors are configured to generate a filtered reconstructed block based on the final filtered pixels.

Optionally, the differences between all filtered and corresponding unfiltered pixels are added to the accumulation buffer according to a predefined scan pattern, and the accumulation buffer may be initialized with the unfiltered pixels multiplied by the maximum number of pixel values to be added to the block. The final filtered pixels are obtained as the accumulated values in the accumulation buffer divided by the maximum number of pixel values to be added to the block.

[0012] According to a second aspect, the invention relates to a corresponding filtering method for processing a block to form a filtered block, wherein the block contains a plurality of pixels. Each pixel is associated with a pixel value. The filtering method includes the following steps: loading the current pixel and neighboring pixels into a linear buffer in accordance with a predetermined scan pattern; get the components of the spectrum by performing a one-dimensional transformation of pixels in a linear buffer; obtaining the filtered components of the spectrum by multiplying each component of the spectrum by the gain, while the gain depends on the corresponding component of the spectrum and the filtering parameter; obtaining the filtered pixels by performing an inverse one-dimensional transform on the filtered spectrum components; and generating a filtered block based on the filtered pixels.

[0013] As an example, a block (or frame) may be a predicted block, and a filtered block is a filtered predicted block. As another example, a block (or frame) may be a reconstructed block and the filtered block is a filtered reconstructed block.

For example, the gain is a function of the corresponding spectrum component and the filter parameter. The filtering parameter may be derived from the codec quantization parameter (QP).

усиления для первого компонента спектра равен единице. Первый компонент спектра может соответствовать значению DC.Another example, the first component of the spectrum is passed without filtering when the coefficient

gain for the first component of the spectrum is equal to unity. The first spectrum component may correspond to a DC value.

Another example is filtering spectrum components based on a look up table (LUT). The formation of the LUT may be based on an auxiliary function for at least some of the quantization parameters (QP). The auxiliary function can be a straight line equation combining the points (i, THR) and (a, 0), where a > 0 and a depends on the filtering parameter σ or the value of QP. For example, for the last QP in the QP set, a is 11; or for the penultimate QP in the QP set, a is 9.

As another example, the method further includes the steps of: removing N bits from the values of the LUT table, N is an integer. N may depend on the value of QP or be a fixed value. When N is chosen smaller for a lower QP compared to a higher QP from the QP set, for example, for the first QP in the QP set, N is 2; or for the rest of the QPs in the QP set, N is 3. Alternatively, when N is chosen higher for the higher QP over the lower QP from the QP QP set, for example, for the last QP or for the last two QPs in the QP set, N is four; or for the rest of the QPs in the QP set, N is 3. Alternatively, when N is chosen to be smaller for a lower QP and higher for a higher QP than the rest of the QPs in the QP set, e.g., for the first QP in the QP set, N is 2 ; for the last QP, or for the last two QPs in the QP set, N is 4; or for the remaining QPs from the QP set, N is equal to 3.

For example, a predetermined scan pattern is defined as a set of spatial or raster offsets relative to the position of the current pixel within the reconstructed block. Point offsets to neighboring pixels are inside the reconstructed block. At least one filtered pixel may be placed in its original position according to a predetermined scan pattern. When all the filtered pixels are added to the accumulation buffer according to a predetermined scan pattern, and the accumulation buffer can be initialized to zero until the filtered spectrum components are obtained. When the final filtered pixels are obtained as the accumulated values in the accumulation buffer divided by the number of pixels added to the current position of the accumulation buffer; one or more processors are configured to generate a filtered reconstructed block based on the final filtered pixels.

Optionally, the differences between all filtered and corresponding unfiltered pixels are added to the accumulation buffer according to a predefined scan pattern, and the accumulation buffer may be initialized with the unfiltered pixels multiplied by the maximum number of pixel values to be added to the block. The final filtered pixels are obtained as the accumulated values in the accumulation buffer divided by the maximum number of pixel values to be added to the block.

[0014] According to a third aspect, the invention relates to an encoding device for encoding a current frame from an input video stream, wherein the encoding device comprises a filter according to the first aspect of the invention.

[0015] According to a fourth aspect, the invention relates to a decoding apparatus for decoding a current frame from a received bitstream, wherein the decoding apparatus comprises a filter according to the first aspect of the invention.

[0016] According to a fifth aspect, the invention relates to a computer program containing program code for performing the method according to the second aspect when executed on a computer.

[0017] Thus, a filter is provided to improve the efficiency of video coding. More specifically, the improved filter according to embodiments of the invention estimates the filter parameters from the frame itself without signaling the filter parameters and therefore requires significantly less signaling than conventional filters that signal weights for filtering in the image area.

Brief description of the drawings

[0018] Additional embodiments of the invention will be described with reference to the following figures, in which:

[0019] FIG. 1A shows a schematic diagram illustrating an encoder according to an embodiment, comprising a filter according to an embodiment;

[0020] FIG. 1B shows a schematic diagram illustrating an encoder according to an embodiment, including a filter according to an embodiment;

[0021] FIG. 2A shows a schematic diagram illustrating a decoding apparatus according to an embodiment, comprising a filter according to an embodiment;

[0022] FIG. 2B shows a schematic diagram illustrating a decoding apparatus according to an embodiment, including a filter according to an embodiment;

[0023] FIG. 3A shows a schematic diagram illustrating aspects of a filtering process implemented in a filter according to an embodiment;

[0024] FIG. 3B is a schematic diagram illustrating aspects of a filtering process implemented in a filter according to an embodiment;

[0025] FIG. 3C is a schematic diagram illustrating aspects of a filtering process implemented in a filter according to another embodiment;

[0026] FIG. 4A illustrates patterns for various pixel positions within a square reconstructed block;

[0027] FIG. 4B illustrates the equivalent filter shape for one pixel;

[0028] FIG. 4C gives an example of padding;

[0029] FIG. 4D gives another example of infill;

[0030] FIG. 5A shows a flowchart illustrating the steps of a filtering method according to an embodiment;

[0031] FIG. 5B shows a flowchart illustrating the steps of a filtering method according to an embodiment;

[0032] FIG. 6 is an example of a hardware filter circuit based on SRAM;

[0033] FIG. 7 is an exemplary hardware diagram of 2x2 group filtering based on flip-flops;

[0034] FIG. 8 shows an example of combining the results of four 2×2 group filterings, reusing the results of the same final space group filter;

[0035] FIG. 9 shows the result of LUT optimization as an example;

[0036] FIG. 10 exemplifies an unwanted discontinuity in the filter transfer function;

[0037] FIG. 11 exemplifies the same table with the table entries arranged in turn;

[0038] FIG. 12 illustrates a discontinuity repair method using an auxiliary function as an example;

[0039] FIG. 13 illustrates an example of discontinuity by taking the maximum of two values when generating the LUT;

[0040] FIG. 14 illustrates, by way of example, the transfer function of a filter after applying the method described above;

[0041] FIG. 15 illustrates an example of filter transfer functions versus five QPs in the set;

[0042] FIG. 16 illustrates an example of filter transfer functions for five QPs in a set based on respective tables;

[0043] FIG. 17 is an example of filter transfer functions for five QPs in a set based on respective tables; and

[0044] FIG. 18 is a schematic diagram illustrating an exemplary structure of a device according to an embodiment.

[0045] Throughout the various drawings, identical reference numerals will be used for identical or functionally equivalent elements.

DETAILED DESCRIPTION OF EMBODIMENTS

[0046] In the following description, reference is made to the accompanying drawings, which form part of the disclosure, and which show, by way of illustration, specific aspects into which the present invention may be placed. It is understood that other aspects may be used and structural or logical changes may be made without departing from the scope of the present invention. Therefore, the following detailed description should not be taken in a limiting sense, since the scope of the present invention is defined by the appended claims.

[0047] For example, it is understood that the disclosure in connection with the described method may also be true for a corresponding device or system capable of performing the method, and vice versa. For example, if a particular method step is described, the corresponding device may include a block for performing the described method step, even if such block is not explicitly described or illustrated in the figures. In addition, it is understood that the features of the various illustrative aspects described herein may be combined with each other unless specifically noted otherwise.

[0048] FIG. 1A shows an encoder 100 according to an embodiment, including a filter 120 according to an embodiment. The encoding apparatus 100 is configured to encode a block of a video signal frame containing a plurality of frames (also referred to here as images), wherein each frame may be divided into a plurality of blocks, and each block contains a plurality of pixels. In an embodiment, the blocks may be macroblocks, coding tree units, coding units, prediction units, and/or prediction blocks.

[0049] The term "block" in this disclosure is used for a block of any type or for a block of any depth, for example, the term "block" is included, but not limited to a root block, block, subblock, treetop, etc. Encoded blocks do not necessarily have the same size. One image may include blocks of different sizes, and block rasters of different images of a video sequence may also differ.

[0050] In the exemplary embodiment shown in FIG. 1A, the encoder 100 is implemented in the form of a hybrid video encoder. Typically, the first frame of a video signal is an intra frame that is encoded using only intra prediction. To this end, the embodiment of the encoder 100 shown in FIG. 1A includes an intra prediction block 154 for intra prediction. The intra frame may be decoded without information from other frames. The intra prediction unit 154 may perform intra block prediction based on the information provided by the intra estimator 152 .

[0051] Blocks of subsequent frames following the first intra frame may be encoded using inter or intra prediction as selected by mode selector 160 . To this end, the encoder 100 shown in FIG. 1A further includes an inter prediction block 144. Typically, inter prediction block 144 may be configured to perform block motion compensation based on the motion estimate provided by external estimator 142 .

[0052] In addition, the prediction signal in the hybrid encoder embodiment shown in FIG. 1B obtained from intra or inter prediction is further filtered by filter 145.

[0053] In addition, in the embodiment of the hybrid encoder shown in FIG. 1A and 1B, the residual calculation block 104 determines the difference between the original block and its prediction, that is, the residual block determining the intra/inter prediction error of the image. This residual block is transformed by a transform block 106 (eg using a DCT) and the transform coefficients are quantized by a quantizer 108 . The output signal of the quantizer 108, as well as the coding or side information provided, for example, by the intra predictor 154, the inter prediction 144, and the filter 120, are further encoded by the entropy coding unit 170.

[0054] A hybrid video encoder typically duplicates the processing of a decoder so that both will generate the same predictions. Thus, in the embodiment shown in FIG. 1, the inverse quantizer 110 and the inverse transform block perform the inverse operations of the transform block 106 and the quantizer 108 and duplicate the decoded approximation of the residual block. Then, the decoded data of the residual block is added to the prediction results, that is, to the prediction block, by means of a recovery block 114 . The output of the reconstruction unit 114 may then be provided to a linear buffer 116 to be used for intra prediction and further processed by a filter 120, as will be described in more detail below. The final image is stored in the decoded image buffer 130 and may be used for inter-prediction of subsequent frames.

[0055] FIG. 2A shows a decoder 200 according to an embodiment, including a filter 220 according to an embodiment. The decoding device 200 is configured to decode a frame block of the encoded video signal. In the embodiment shown in FIG. 2A, the decoder 200 is implemented as a hybrid decoder. The entropy decoding block 204 performs entropy decoding of the encoded image data, which may typically contain prediction errors (i.e., residual blocks), motion data, and other additional information that is needed, in particular, for the intra prediction block 254 and the inter prediction block 244, as well as other components of decoder 200 such as filter 220. Typically, intra predictor 254 and inter predictor 244 of decoder 200 shown in FIG. 2A are selected by the mode selector 260 and operate in the same manner as the intra predictor 154 and inter predictor 144 of the encoding device 100 shown in FIG. 1 so that identical predictions can be generated by the encoder 100 and the decoder 200. The recovery unit 214 of the decoding apparatus 200 is configured to recover the block based on the filtered predicted block and the residual block provided by the inverse quantization unit 210 and the inverse transform unit 212 . As with the encoder 100, the reconstructed block may be provided to the linear buffer 216 used for intra prediction, and the filtered block/frame may be provided to the decoded picture buffer 230 via the filter 220 for inter prediction.

[0056] As shown in FIG. 2B, the prediction signal in the hybrid decoder embodiment shown in FIG. 2B obtained from intra or inter prediction is further filtered by filter 264.

[0057] As already described above, filter 120, 220 may be used at the frame level, for example, filter 120, 220 may be configured to process a reconstructed frame from a decoded reconstructed video stream to generate a filtered reconstructed frame, where the reconstructed frame includes a set blocks. Filter 120, 220 may also be used at the block level immediately after block reconstruction without waiting for the entire frame, for example, filter 120, 220 may be configured to process the reconstructed block to form a filtered reconstructed block, where the reconstructed block includes a plurality of pixels.

[0058] Filter 120, 220, 145, or 264 comprises one or more processors (or one or more processing units). As will be explained in more detail below, one or more processors (or one or more processing units) are configured to: load the current pixel and its neighboring pixels into a linear buffer in accordance with a predetermined scan pattern (in other words, scan order or scan model) ; obtaining spectrum components by performing a one-dimensional transform for each pixel in the linear buffer; obtaining filtered components of the spectrum by multiplying each component of the spectrum by the gain, while the gain depends on the corresponding component of the spectrum and the filtering parameter; obtaining filtered pixels by performing an inverse one-dimensional transform on the filtered spectrum; and generating a filtered reconstructed block based on the filtered pixels estimated in previous processing steps. In one example, the gain depends on the respective component of the spectrum and the filter parameter. In another example, the gain depends on one or more filter parameters and one or more corresponding spectrum components. In another example, the appropriate gain may depend on one or more filter parameters and the corresponding spectrum component, as well as adjacent spectrum components to the left and right of the spectrum component.

[0059] The disclosure describes an in-loop filter for a lossy video codec that performs local and/or non-local filtering of a reconstructed block from a reconstructed frame. According to the example, the reconstructed frame is divided into a set of small non-overlapping rectangular blocks (CU blocks). In the next step, each reconstructed block (recovered CU) is filtered in the frequency domain independently of other reconstructed blocks. The filter can also be applied after transformation and reconstruction, and the filtered result is used for both inference and spatial and temporal prediction.

[0060] As another example, the disclosure describes a prediction filter for a lossy video codec that performs local and/or non-local filtering of a reconstructed frame prediction block.

[0061] In the first processing step, all pixels within the reconstructed block may be processed independently of each other. To process the pixel r(0), neighboring pixels are used. For example, as shown in FIG. 3A, pixels r(1) to r(7) are used, and pixels r(0) to r(7) form one processing group.

[0062] FIG. 3A or 3B shows a schematic diagram 300, 300' illustrating aspects of a filtering process implemented in a filter according to an embodiment. At step 302, 302', the current pixel and neighboring pixels from the block are loaded into a linear buffer in accordance with a predetermined scan pattern. By way of example, the block may be a predicted block. As another example, the block may be a reconstructed block.

[0063] In step 304, 304', a one-dimensional transformation is performed on pixel r(0) and its neighboring pixels r(1) through r(7) in the linear buffer to obtain the R components of the spectrum:

[0064] As an example, a one-dimensional transform may be a Hadamard transform.

спектра на соответствующий коэффициент усиления

для получения отфильтрованных компонентов

спектра:[0065] It should be understood that performing a 1D transform on 4 pixels in a row (e.g., pixels A, B, C, D, as in the example in Fig. 3B) or performing a 2D transform on spatially located pixels A, B, C, D depends on the specific implementation. Applying a 2D transform to 4 pixels A, B, C, D located in a 2×2 block can lead to the same result as applying a 1D transform to 4 pixels A, B, C, D taken in a row. In step 306, 306' filtering is performed in the frequency domain based on the multiplication of each component

spectrum to the corresponding gain

to get filtered components

spectrum:

(1)

(one)

[0066] The set of gains for all spectrum components is the impulse response of the filter.

может быть описан следующей формулой как пример:[0067] In the example, the gain depends on the corresponding component of the spectrum and the filtering parameter. In another example, the gain depends on one or more filter parameters and one or more corresponding spectrum components. In another example, the appropriate gain may depend on one or more filter parameters and the corresponding spectrum component, as well as adjacent spectrum components to the left and right of the spectrum component. If each gain is a function of the spectrum component of the reconstructed block and the filter parameter, or is a function of the spectrum component of the predicted block and the filter parameter, the gain is

can be described by the following formula as an example:

(2)

(2)

- коэффициент усиления, соответствующий R(i), σ - параметр фильтрации, m - константа нормализации, равная количеству компонентов спектра. Например, m равно 1, 2, 3, 4…. Различные компоненты спектра могут иметь одинаковый коэффициент усиления или могут иметь разные коэффициенты усиления.where (i) is the index of the spectrum component, R(i) is the spectrum component corresponding to the index (i),

is the gain corresponding to R(i), σ is the filtering parameter, m is the normalization constant equal to the number of spectrum components. For example, m is 1, 2, 3, 4…. Different spectrum components may have the same gain or may have different gains.

[0068] For those transforms that have spectral components corresponding to the average (FFT, DCT, DST, etc.) or sum (Hadamard) of the input samples of the transform block (usually the first component corresponding to the DC value), it can be advantageous to have a coefficient filtering value equal to 1 to avoid changing the average brightness of the filtered block. This means skipping (no filtering) for the first component of the spectrum corresponding to the DC value.

[0069] σ as a filtering parameter can be obtained from the quantization parameter (QP) of the codec at the encoder and decoder sides, for example, using the following formula:

,

,

и

являются константами, имеющими значения, например:

where

and

are constants that have values, for example:

Different spectrum components may have the same filter parameter or may have different filter parameters.

и

могут быть выбраны таким образом, чтобы сигма зависела от размера шага квантования, который удваивается каждый раз, когда значение QP увеличивается на 6 в последних стандартах кодирования видео. В примере с параметрами k = 0,5, n = 1/6 и s = 0 параметры σ получаются следующим образом: [0070] Parameters

and

can be chosen so that the sigma depends on the quantization step size, which doubles every time the QP value is increased by 6 in the latest video coding standards. In the example with parameters k = 0.5, n = 1/6 and s = 0, the parameters σ are obtained as follows:

[0071] Quantization scaling matrices are widely used to improve the quality of video compression. In this method, the quantization step size obtained from the QP is multiplied by a scaling factor transmitted in the bitstream. For such a method, the derivation of the σ parameters can be based on the actual scaled quantization step size used for a particular QP:

и

могут иметь фиксированные значения для вычисления σ или могут иметь разные значения в зависимости от QP, размера и формы блока, типа предсказания (внешнего/внутреннего) для текущего блока. Например, для квадратных блоков с внутренним предсказанием размером 32×32 или более параметр s может вычислен как

. В качестве эквивалентного параметра фильтра σ имеет меньшее значение, это приводит к более мягкому фильтру, который больше подходит для больших квадратных блоков с внутренним предсказанием, обычно соответствующих плоским областям. В качестве другого примера, k может быть изменено на основе глубины цвета пикселей k_mod=k * (1 << (bit_depth - 8)).[0072] Constants

and

may have fixed values for calculating σ or may have different values depending on QP, block size and shape, type of prediction (outer/inner) for the current block. For example, for intra-prediction square blocks of size 32x32 or larger, the parameter s can be computed as

. As an equivalent filter parameter, σ is smaller, resulting in a softer filter that is more suitable for large intra-predictive square blocks, typically corresponding to flat regions. As another example, k may be changed based on the pixel color depth k _mod =k * (1 << (bit_depth - 8)).

[0073] According to the method 300, 300', the gain for each frequency is obtained from the spectrum component of the reconstructed pixels or predicted pixels. Therefore, the method 300, 300' does not require the transmission of filter parameters and can be applied to any recovered block or predicted block without additional signaling.

[0074] The details of the LUT are discussed.

[0075] It can be noted that filtering involves multiplying the spectrum components R(i) by a scaling factor that is always less than 1. It can also be seen that at high values of R(i) the scaling factor is close to 1. Based on these observations, the spectrum is filtered using a lookup table that allows you to exclude multiplication and division from filtering operations:

[0076] The spectrum gain is less than 1, so filtering can be implemented based on reading a short lookup table (LUT) with the following formulas:

. (i) - индекс компонента спектра, R(i) - компонент спектра, соответствующий индексу (i), σ - параметр фильтрации, а THR - пороговое значение, m - константа нормализации, равная количеству компонентов спектра. Например, m равно 1, 2, 3, 4…where

. (i) - spectrum component index, R(i) - spectrum component corresponding to index (i), σ - filtering parameter, and THR - threshold value, m - normalization constant equal to the number of spectrum components. For example, m is 1, 2, 3, 4…

[0077] As an example, THR can be calculated using the following formula:

= C

= C

where C is a value close to 1, such as 0.8 or 0.9. To reduce the LUT size, the THR threshold may depend on the QP value.

, который дополнительно получается как:To further reduce the LUT size, a second threshold may be introduced to replace small filtered values with zero. In this case, the filtered component of the spectrum is

, which is additionally obtained as:

In this case, THR2 defines a threshold value below which the filtered spectrum component is considered to be zero. The second THR2 may also be determined depending on the QP value.

:[0078] After filtering in the frequency domain, at step 308, an inverse one-dimensional transform is performed on the filtered component of the spectrum to obtain filtered pixels

:

[0079] In step 310, 310', the result of the inverse one-dimensional transform is placed in a linear buffer of filtered reconstructed pixels or filtered pixels.

[0080] In step 312, 312' (not shown in FIG. 3A or 3B), a filtered block is generated based on the filtered pixels estimated in previous processing steps. As an example, the filtered block may be a filtered predicted block. As another example, the filtered block may be a filtered reconstructed block.

[0081] As shown in FIG. 3A as an embodiment, after the filtering step 306, the filtered pixel f(0) is placed in its original position according to a predetermined scan pattern. Other filtered samples f(1) - f(7) are not used. In another embodiment, more than one filtered pixel, such as all filtered pixels from a linear filtered sample buffer, are added to the accumulation buffer in accordance with a predefined scan pattern used in step 302 of FIG. 3A. Before the filtering step, the accumulation buffer must be initialized to zero. In the last normalization step, the final filtered pixels are obtained as the accumulated values in the accumulation buffer divided by the number of pixels added to the current position of the accumulation buffer, in other words, by the number of pixel values added to the current position of the accumulation buffer in the previous processing steps. Then, a filtered reconstructed block or a predicted block is generated based on the final filtered pixels.

[0082] In another embodiment, the filter has the same implementation for filtering both inner and outer coding units (CUs).

[0083] The Hadamard transform region filter is always applied to reconstructed luma blocks with non-zero transform coefficients, except for 4x4 blocks and if the slice quantization parameter is greater than 17. The filter parameters are explicitly derived from the encoded information. The proposed filter, if applied, is executed on the decoded samples immediately after the block is reconstructed. The filtered result is used both for output and for spatial and temporal prediction.

[0084] Discusses the filtering process shown schematically in Figure 3C.

[0085] For each pixel from the reconstructed block, pixel processing comprises the following steps:

[0086] Scan 4 neighboring pixels around the processing pixel, including the current one, in accordance with the scan pattern.

[0087] A 4-point Hadamard transform of the read pixels is performed.

[0088] Spectral filtering according to formulas (1) and (2).

[0089] The first component of the spectrum corresponding to the DC value is passed without filtering.

[0090] An inverse 4-point Hadamard transform of the filtered spectrum is performed.

[0091] After the filtering step, the filtered pixels are placed in their original positions in the accumulation buffer.

[0092] After the pixel filtering is completed, the accumulated values are normalized by the number of processing groups used for each pixel filtering. Due to the use of filling one sample around the block, the number of processing groups is 4 for each pixel in the block, and normalization is performed by shifting to the right by 2 bits.

[0093] It can be seen that all pixels in a block can be processed independently if maximum parallelism is desired.

[0094] In these embodiments, the THR threshold is set to a predetermined value, such as 128, which in a simple implementation would require storing 128 (1 << 7) records of 7-bit values for each QP.

[0095] The size of the LUT influences the number of on-chip memory requirements and the cost of the hardware implementation of the filter. To reduce the built-in memory, the LUT is calculated only for a limited set of QPs, starting from QP 20 at a constant interval of 8. A total of five preset LUTs (for five QP groups) are stored. To filter the current QP CU block, it is rounded up to the nearest one from the table.

[0096] To further reduce the LUT size, the N least significant bits are discarded (or ignored) during LUT generation. This allows you to have a sparse table view.

[0097] For an exemplary implementation of A, N is 2, resulting in 7-2=5 table depth bits (32 entries of 7-bit values);

[0098] For an exemplary implementation of B, N is 3, resulting in 7-3=4 table depth bits (16 entries of 7-bit values).

[0099] Thus, the total amount of memory required for the entire LUT store is:

[00100] For exemplary implementation A: 5x32x7 bits=1120 bits=140 bytes;

[00101] For exemplary implementation B: 5x16x7 bits=560 bits=70 bytes;

[00102] Exemplary implementation B targets 16 bytes of LUT size to allow parallel access in software implementation due to the ability to store the entire LUT in a single 16 byte SSE register, so this configuration is suggested.

[00103] If a Hadamard transform is used and a filtered pixel is placed in its original position according to a predefined scan pattern, then the following pseudo-code describes the filtering process of the method 300:

// scan recovered/predicted pixels

const int x0=pIn[p0];

const int x1=pIn[p1];

const int x2=pIn[p2];

const int x3=pIn[p3]; // p0-p3 define the scan pattern

// 1D direct Hadamard transform

const int y0=x0+x2;

const int y1=x1+x3;

const int y2=x0-x2;

const int y3=x1-x3;

const int t0=y0+y1;

const int t1=y0-y1;

const int t2=y2+y3;

const int t3=y2-y3;

// filtering in the frequency domain

const int z0=pTbl[t0];

const int z1=pTbl[t1];

const int z2=pTbl[t2];

const int z3=pTbl[t3];

// inverse Hadamard transform

const int iy0=z0+z2;

const int iy1=z1+z3;

const int iy2=z0-z2;

const int iy3=z1-z3;

// output filtered pixel

pOut[p0_out] = iy0+iy1;

[00104] If a Hadamard transform is used and more than one filtered pixel from the linear filtered sample buffer is added to the accumulation buffer, then the following pseudo-code describes the filtering process for this scenario:

// scan recovered/predicted pixels

const int x0=pIn[p0];

const int x1=pIn[p1];

const int x2=pIn[p2];

const int x3=pIn[p3]; // p0-p3 define the scan pattern

// 1D direct Hadamard transform

const int y0=x0+x2;

const int y1=x1+x3;

const int y2=x0-x2;

const int y3=x1-x3;

const int t0=y0+y1;

const int t1=y0-y1;

const int t2=y2+y3;

const int t3=y2-y3;

// filtering in the frequency domain

const int z0=pTbl[t0];

const int z1=pTbl[t1];

const int z2=pTbl[t2];

const int z3=pTbl[t3];

// inverse Hadamard transform

const int iy0=z0+z2;

const int iy1=z1+z3;

const int iy2=z0-z2;

const int iy3=z1-z3;

[00105] As an alternative implementation, the accumulation buffer should be initialized with unfiltered pixel values multiplied by the maximum number of pixel values to be added to the block. The maximum number of pixel values added to a block is determined based on the scan pattern. The scan template really determines the number of pixel values added for each position. Based on this, the maximum number of all positions in the block can be selected and used during the initialization of the accumulation buffer. Then, at each accumulation step, the unfiltered pixel value is subtracted from the corresponding filtered value and added to the accumulation buffer:

// accumulation of filtered pixels

pOut [p0] += iy0+iy1 // p0-p3 define the scan pattern

pOut[p1] += iy0-iy1

pOut [p2] + = iy2+iy3

pOut[p3] += iy2-iy3

To reduce the color depth of the accumulated pixel values before being placed in the accumulation buffer, the result of the inverse transform can be normalized by the transform size (m):

pOut[p0] += ((iy0+iy1) >> HTDF_BIT_RND4);

pOut[p1] += ((iy0-iy1) >> HTDF_BIT_RND4);

pOut[p2] += ((iy2+iy3) >> HTDF_BIT_RND4);

pOut[p3] += ((iy2-iy3) >> HTDF_BIT_RND4);

where HTDF_BIT_RND4 is 2 for a transform size of 4.

[00106] This embodiment allows the system to avoid saving the number of pixels added to the current position, and allows the replacement division and multiplication to be performed by a shift operation in the last normalization step and the accumulation buffer initialization step, respectively, if the maximum number of added pixel values is a power, for example, 2, 4, 8 etc.

For the accuracy of the normalization step, this can be done like this:

// normalization

pFiltered[p0] = CLIP3(0, (1 << BIT_DEPTH) - 1, (pOut[p0] + HTDF_CNT_SCALE_RND) >> HTDF_CNT_SCALE);

where HTDF_CNT_SCALE is Log2 of the number of pixels placed in the accumulation buffer, for example, for the number 4 HTDF_CNT_SCALE is 2 and HTDF_CNT_SCALE_RND is (1 << (HTDF_CNT_SCALE - 1)). CLIP3 is a clipping function that ensures that the filtered sample is within the allowed range between the sample's minimum and maximum values.

[00107] As mentioned above, in order to avoid changing the average brightness of the filtered block, it may be advantageous to skip the filtering of the first component of the spectrum (corresponding to the DC value). This further simplifies the implementation of the filter. In this case, the filtering step looks like this:

// filtering in the frequency domain

const int z0 = t0;

const int z1 = pTbl[t1];

const int z2 = pTbl[t2];

const int z3 = pTbl[t3];

[00108] For each pixel within the reconstructed block or predicted block, a scan template is selected based on the position of the filter pixel within the reconstructed block or predicted block for steps 302 and 310. The scan template is selected to ensure that all pixels will be within the reconstructed CU or predicted CU, and located close to the processing pixel. Within the template, you can use an arbitrary scan order. For example, a predetermined scan pattern is defined as a set of spatial or raster offsets relative to the position of the current pixel within the reconstructed block or predicted block, where the offsets indicate neighboring pixels within the reconstructed block or predicted block. Below is an example scan pattern: (0.0), (0.1), (1.0), (1.1)

[00109] FIG. 4A illustrates an example of patterns for different pixel positions within a square block (eg, square predicted CU or square reconstructed CU). According to this figure, edge pixels may be filtered based on a 4-point transform, and center pixels may be filtered based on an 8-point transform.

[00110] For rectangular reconstructed blocks or predicted blocks in which the size of one side is larger than the size of the other side, scanning should be performed on the long side. For example, for a horizontal rectangular block, you can use the following scan order

(0,-3), (0,-2), (0,-1), (0.0), (0.1), (0.2), (0.3), (0.4) ,

where in each (y, x) pair, x is a horizontal offset and y is a vertical offset relative to the location of the filter pixel within the filter reconstructed or predicted block.

[00111] The proposed filter can be applied selectively depending on the conditions:

for recovered blocks or predicted blocks with a non-zero residual signal;

depending on the block size, for example, for a small reconstructed block or a predicted block (minimum size is less than a threshold value);

depending on the aspect ratio of the reconstructed block or the predicted block;

depending on the prediction mode (intra or inter) of the reconstructed block or the predicted block, for example, by applying a filter only to the inter-predicted blocks; or

for any combination of the above conditions.

For example, to avoid processing small blocks, your filter can be skipped (not applied) if the block size is less than or equal to 4x4 pixels. This reduces the complexity of the worst case, which usually corresponds to the processing of the smallest block.

As another example, the filter is only applied to blocks that have a non-zero residual signal. This is advantageous if quantization or residuals were used, since the filter is intended to improve the quantization error. If there is no remainder in the block, this may indicate that the prediction is good and no additional filtering is required.

As another example, since intra prediction is generally worse than inter prediction, the filter may be applied to intra predicted blocks regardless of the presence of a non-zero residual, and applied to inter predicted blocks only if the block has a non-zero residual signal.

[00112] The sigma of the filter parameter and the scan pattern may differ depending on the conditions listed above.

[00113] FIG. 4B illustrates the equivalent filter shape given one pixel within the current block for an exemplary scan pattern (0.0), (0.1), (1.0), (1.1). To filter the current pixel, a 3×3 pixel square is used (the current pixel is highlighted in dark gray in the center of the 3×3 square). The filtered pixel is obtained by combining the filtered transform area samples from four 2×2 groups. It can be understood that if the current pixel is located on a block boundary (eg, top boundary), the top left and top right 2x2 groups are not available, and only two 2x2 groups (bottom left and bottom right) can be used for filtering. Also, if the current pixel is located in the corner of the block (for example, in the upper left corner), only one 2×2 group (lower right) can be used for filtering.

[00114] To improve the quality of filtering by using an additional four groups of 2x2 for border and corner pixels, the current block can be filled with additional samples. In FIG. 4C shows an example padding on the left and top. Fill samples can be taken from already restored blocks.

[00115] To further unify the filtering process for all pixels in a block (four 2x2 groups are used to filter all pixels in the current block), in addition to top left padding, the current block can also be extended with bottom right padding, as shown in FIG. 4D. The unification of the filtering is beneficial due to the simplification of the implementation by eliminating the special cases of handling corner pixels and/or boundary pixels.

[00116] The padding samples are preferably taken from corrected neighboring samples from already reconstructed blocks. In modern video codecs, these already restored blocks can be located either to the left or above the current block, or to the right or below, depending on the order in which the blocks are restored. By using additional information from the adjusted samples, the filtering quality is improved and the transition between blocks is smoother.

[00117] Obtaining recovered samples from corrected blocks or previously recovered blocks may require additional memory overhead for hardware or software implementation. To minimize or eliminate additional memory, it is advantageous to use samples intended for intra-prediction of the current block, which are usually taken from one, two or more rows and columns from neighboring blocks, corrected for the boundaries of the current block. These samples are usually stored in fast memory (also known as a "linear" buffer) for easy access for intra-prediction and are referred to as intra-prediction reference samples.

[00118] It should be further noted that in some implementations, prior to performing intra prediction, the reference sample(s) is preprocessed prior to prediction, such as by smoothing, sharpening, ringing removal, or two-way filtering. In this case, it may be advantageous to use pre-rendered samples to populate the current block.

[00119] If some samples in the filled area are not available due to the order in which the adjusted block is restored, the required samples can be filled from the pixels of the current block's expanding border to the filled area, as shown in FIG. 4D.

[00120] FIG. 5A is a flowchart illustrating the steps of a corresponding in-loop filtering method 500 according to an embodiment. The reconstructed block contains many pixels. The method 500 includes the following steps: loading (502) the current pixel and its neighboring pixels into a linear buffer in accordance with a predetermined scan pattern; get (504) components of the spectrum by performing a one-dimensional transformation of pixels in a linear buffer; get (506) filtered components of the spectrum by multiplying each component of the spectrum by the gain, while the gain depends on the corresponding component of the spectrum and the filtering parameter; obtaining (508) filtered pixels by performing an inverse one-dimensional transform on the filtered spectrum components; and generating (510) a filtered reconstructed block based on the filtered pixels estimated in previous processing steps. Method 500 may be performed by the encoder shown in FIG. 1 and the decoding device shown in FIG. 2. Detailed information 300 in FIG. 3A or information 300' in FIG. 3B also apply to method 500 as shown in FIG. 5A.

[00121] Similar to FIG. 5A, FIG. 5B shows a block diagram illustrating the steps of a corresponding in-loop filtering method 500' according to another embodiment. In this example, a block (or frame) is a predicted block and a filtered block is a filtered predicted block. The detailed description of figure 5B is similar to figure 5A.

[00122] A hardware implementation is discussed.

[00123] The Hadamard transform region filter is placed immediately after block recovery and processes samples that can be used in subsequent block recovery, in particular as intra prediction reference samples. Thus, the delay introduced by the filter must be minimized to ensure that the entire recovery process is not heavily affected.

[00124] The Hadamard transform is considered relatively simple to implement in hardware. Its implementation requires only additions, not multiplications. As can be seen from Pseudo-code 1 below, the forward and backward transformation contains 4 additions, which can be performed in parallel or by reusing intermediate results with two consecutive additions.

Pseudocode 1

[00125]

[00126] Forward and inverse Hadamard transforms can be implemented in hardware using combinational logic. The more attention is required for fast and parallel access to the LUT.

[00127] SRAM-based LUTs are discussed.

[00128] In this implementation example, the LUT is stored in the embedded single port static RAM (Figure 6).

[00129] As soon as the data of the previous stage of processing becomes available in the buffer on the rising edge of the clock signal, it is accessed through combinational logic that implements a direct Hadamard transform (containing two subsequent additions). Once completed, the combinational logic address is available for each LUT. Using an inverter and a falling clock edge, data is accessed from SRAM. The second combinational logic, which implements the inverse Hadamard transform and normalization, is launched immediately after receiving data from the LUT. The output filtered samples become available at the end of the current clock cycle and are ready to be processed by the next algorithm at the next clock rising edge.

[00130] Trigger-based LUTs are discussed.

[00131] Considering that one table for the filtering process is limited to 16 entries, it is more efficient to implement a LUT based on triggers. Such a scheme does not require multiple LUTs for parallel processing and a clock edge for data access. Parallel access is provided by a multiplexer as shown in Figure 7 illustrating an exemplary scheme for processing one 2x2 group. The proposed scheme requires 16 flip-flops of 7 bits each to provide parallel access during the filtering process. A QP specific LUT can be loaded into triggers as soon as the QP for the current CU becomes available.

[00132] Combining the filtering results of four 2x2 groups with reuse of the results of the same output of the final space group filter is formed as shown in Figure 8.

[00133] The above analysis leads to the conclusion that the proposed filter can be implemented in hardware in one cycle using either SRAM or a trigger-based LUT implementation.

[00134] Complexity analysis is discussed.

[00135] Measure the impact on the rate/PSNR with respect to the anchor point(s).

[00136] Complexity analysis (eg, measurement of encoding and decoding time, complexity analysis by completing the table below).

Test# filter shape Computational complexity (Number of multiplications/additions/shifts/checks*) per sample Multiplication precision Parallel Oriented (each sample can be filtered independently of other samples) Number of clock cycles need for memory How to get filter coefficients Minimum CU size to apply filters 14.3a 3×3 0/20+4 l-bit add for loop/5/6 no data Yes < 1 cycle 140 bytes (32 7-bit values per group qp) Precalculated in LUT 4×8, 8×4 14.3v 3×3 0/20+4 l-bit add for loop/5/6 no data Yes < 1 cycle 70 bytes (16 7-bit values per group qp) Precalculated in LUT 4×8, 8×4

* max/min/abs count as checks.

Table 1. Summary results of the CE14-3 complexity analysis

[00137] Experimental results are discussed.

[00138] Objective results are discussed.

[00139] Objective indicators are presented in the following tables:

Fully internal Main10 Over BMS-2.0.1 with VTM configuration Y at v. Enct DecT

Class A1 -0.19% 0.29% 0.24% 110% 115% Class A2 -0.35% 0.31% 0.32% 109% 112% Class B -0.42% 0.29% 0.32% 109% 113% Class C -0.83% 0.23% 0.42% 110% 110% Class E -0.54% 0.28% 0.19% 109% 114% Total -0.48% 0.28% 0.31% 109% 112% Class D -0.61% 0.47% 0.01% 109% 108% Class F (optional) -0.82% 0.07% 0.04% 108% 107% Main 10 random access Over BMS-2.0.1 with VTM configuration Y at v. Enct DecT Class A1 -0.43% -0.23% -0.25% 105% 104% Class A2 -0.92% -0.57% -0.43% 105% 103% Class B -0.69% -0.02% -0.27% 106% 104% Class C -0.75% 0.10% 0.14% 105% 104% Class E Total -0.70% -0.14% -0.19% 105% 104% Class D -0.67% -0.14% 0.53% 104% 103% Class F (optional) -0.77% 0.10% 0.13% 104% 102% B Main10 low latency Over BMS-2.0.1 with VTM configuration Y at v. Enct DecT Class A1 Class A2 Class B -0.58% 0.52% 0.54% 104% 104% Class C -0.74% 0.33% 0.52% 104% 104% Class E -0.75% 0.31% 1.09% 102% 101% Total -0.68% 0.40% 0.67% 104% 103% Class D -0.90% 0.75% 0.28% 104% 104% Class F (optional) -0.73% 0.20% -0.09% 103% 102%

Table 2. Encoding performance of test 14-3a

Fully internal Main10 Over BMS-2.0.1 with VTM configuration Y at v. Enct DecT Class A1 -0.17% 0.28% 0.28% 110% 115% Class A2 -0.33% 0.27% 0.29% 109% 112% Class B -0.40% 0.31% 0.39% 109% 113% Class C -0.82% 0.20% 0.32% 109% 110% Class E -0.54% 0.34% 0.25% 109% 113% Total -0.47% 0.28% 0.32% 109% 112% Class D -0.65% 0.43% 0.22% 109% 108% Class F (optional) -0.81% 0.01% 0.12% 108% 107%

Main 10 random access Over BMS-2.0.1 with VTM configuration Y at v. Enct DecT Class A1 -0.41% -0.36% -0.24% 105% 105% Class A2 -0.92% -0.49% -0.37% 105% 103% Class B -0.69% -0.12% 0.12% 106% 104% Class C -0.75% 0.14% 0.10% 105% 104% Class E Total -0.70% -0.17% -0.06% 105% 104% Class D -0.69% -0.30% 0.49% 104% 103% Class F (optional) -0.82% 0.11% 0.11% 104% 102% B Main10 low latency Over BMS-2.0.1 with VTM configuration Y at v. Enct DecT Class A1 Class A2 Class B -0.57% 0.49% 0.69% 104% 104% Class C -0.76% 0.31% 0.42% 104% 106% Class E -0.67% -0.66% -0.48% 102% 102% Total -0.66% 0.14% 0.31% 103% 104% Class D -0.88% 0.92% 0.31% 103% 105% Class F (optional) -0.74% 0.38% -0.15% 103% 102%

Table 3. Encoding performance of test 14-3b

[00140] The proposed LUT is 70 bytes with 16 bytes per QP, which allows HW to be implemented in 1 cycle, etc. It is proposed to adopt the Hadamard transform domain filter in the next version of VTM.

[00141] The following references are incorporated herein by reference as if they were reproduced in their entirety:

[00142] Joint Video Experts Team (JVET) document JVET-K0068.

[00143] The following are examples of LUT optimization.

[00144] As one example 1, a set of quantization parameters (QP) is selected to generate a lookup table (LUT), where the QP set includes a first QP corresponding to (i) index and a second QP corresponding to (i+1) index, and the first QP and the second QP have a constant spacing. For example, the interval could be 8, 10, or 16.

[00145] For example, taking a constant interval of 8 as an example, a sparse table with LUT for qp = {20, 28, 36, 44, 52}. The interval between the first gp 20 and the second gp 28 is 8. Similarly, the interval between the second gp 28 and the third gp 36 is 8. Filtering selects the table with the closest QP.

[00146] As another example, if we take a constant interval of 8, as an example, a sparse table with LUT for qp = {18, 26, 34, 42, 50}. The spacing between the first GP 18 and the second GP 26 is 8. Similarly, the spacing between the second gp 26 and the third gp 34 is 8. Filtering selects the table with the closest QP.

[00147]

LUT size: 5×128=640 bytes

Pseudo-code 2 below reflects which QPs are selected to generate lookup tables (LUTs).

pseudocode 2

[00148] In pseudocode, HTDF_QP_ROUND represents a constant interval. Using the interval as a power of two is advantageous, allowing you to implement the division operation to calculate the index as a shift. It should be noted that different values of the constant interval can be chosen, for example 2, 4, 10, 15 or 16, etc. Moreover, as an alternative implementation, the interval may be arbitrary, and the LUT is calculated for an arbitrary set of QPs.

[00149] During the filtering process, the index corresponding to the LUT for a given QP is calculated as:

int idx = ((qp - HTDF_MIN_QP) + (HTDF_QP_ROUND >> 1)) / HTDF_QP_ROUND;

or alternatively, with less precision:

int idx = (qp - HTDF_MIN_QP) / HTDF_QP_ROUND;

If the constant interval is a power of 2, for example, then the LUT index can advantageously be calculated using a shift operation instead of a division:

int idx = (qp - HTDF_MIN_QP) >> HTDF_QP_ROUND_LOG2 = (qp - HTDF_MIN_QP) >> 3;

[00150] In another example 2, the decoder or encoder discards N bits from the table values, and N is an integer. This allows you to have a sparse LUT representation that only stores selected values from a given range. For example, N is 3. The maximum value of the table is 127 (7 bits), 3 bits are discarded, the result is 4 bits, that is, 16 entries of 7 bit values - approximately 16 bytes.

[00151] The following is a pseudocode 3 describing how the LUT is formed in accordance with a given qp value.

pseudocode 3

In this example, HTDF_TBL_SH specifies the number of bits to discard, which can be 1, 2, 3, 4, and so on.

The following is a pseudo-code illustrating access to a sparse LUT during filtering:

[00152] When combining the above examples 1 and 2 of FIG. 9 shows the result of LUT optimization as an example. As shown in FIG. 9 combining examples 1 and 2: 5 tables × 16 records × 7 bits=560 bits=70 bytes.

[00153] It should be noted that the number of LUT entries is determined by HTDF_SHORT_TBL_THR (threshold from paragraph [0076]) and HTDF_TBL_SH by the number of discarded bits. Given that the threshold is 128 (i.e. 1 << 7) and 3 bits are dropped, the number of entries in the table is 1 << (7-3) = 1 << 4=16. As described above, the LUT threshold is preferably chosen such that the result of the equation in paragraph [0077] is close to 1, and as also described there, the threshold may differ depending on QP. Thus, in order to form a LUT for higher QP values, it may be advantageous to increase the threshold value from 128 (1 << 7) to, for example, 256 (1 << 8). In this case, 32 entries (32=1 << (8-3) = 1 << 5) would be required to maintain the same accuracy of LUT entries (eg, with 3 bits discarded). Alternatively, to keep the same LUT size for higher QPs as for lower QPs, the precision can be further reduced to 4 to keep 16 entries in the table 16=1<<(8-4)=1<< four.

[00154] In some implementations, maintaining a limited LUT size and having a THR to satisfy the equation in paragraph [0077} may be inconsistent. Indeed, at high QP values (resulting in a high σ value), keeping the LUT size limited to, for example, 16 entries can lead to an undesirable discontinuity in the filter transfer function (which is represented by the LUT) around 120, as shown in FIG. 10 (which also includes a method for subsampling the LUT by discarding the 3 least significant bits).

[00155] FIG. Figure 11 shows the same table with table entries built one at a time without taking into account the effect of LUT subsampling, illustrating the gap between the last LUT entry at index 15 and the next point in the filter transfer function at index 16.

[00156] FIG. 12 illustrates how the gap can be bridged by using a helper function passing through the THR value in the argument corresponding to the last LUT+1 entry, for example, using a straight line equation (indicated in green) passing through the THR=128 point in argument 16 which corresponds to the argument of the last LUT entry (which is 15) increased by 1 or, in other words, the size of the LUT (which is 16) and intersects the x-axis at some point (for example, at a value of 11, as in this example). It should be noted that other types of helper functions can be used based on the same principle, such as exponential, logarithmic, parabolic, etc. or their combination.

; а второе - это значение вспомогательной функции (в данном примере прямая линия) при том же аргументе i:

[00157] FIG. 13 illustrates an example of breaking the gap by taking the maximum of two values in LUT generation, where the first value is the LUT entry as described above.

; and the second is the value of the helper function (in this example, a straight line) with the same argument i:

представляет вспомогательную функцию, а вспомогательная функция имеет значение, равное THR в i, соответствующем аргументу последней LUT + 1.where

represents a helper function, and the helper function has a value equal to THR in i corresponding to the argument of the last LUT + 1.

[00158] FIG. 14 illustrates the filter transfer function after applying the above-described method of using an auxiliary straight line equation and subsampling the LUT by discarding the 3 least significant bits.

[00159] As described in paragraph [00144], one LUT may be used for a QP group. To cover the possible range of QPs, a predefined set of QPs is used and one LUT is formed from the set for each QP. 15 shows an example of filter transfer functions versus five QPs in the set and corresponding tables (0 to 4). In this example, table 4 is generated using the method described in paragraphs [00156] - [00157], with the direct helper function crossing the x-axis at a value of 11; and table 3 is formed using the same method with a direct helper function crossing the x-axis at a value of 9. The tables used in this example have values according to:

table0 = {0, 2, 10, 19, 28, 36, 45, 53, 61, 70, 78, 86, 94, 102, 110, 118,},

table 1 = {0, 0, 5, 12, 20, 29, 38, 47, 56, 65, 73, 82, 90, 98, 107, 115,},

table2 = {0, 0, 1, 4, 9, 16, 24, 32, 41, 50, 59, 68, 77, 86, 94, 103,},

table3 = {0, 0, 0, 1, 3, 5, 9, 14, 19, 25, 32, 40, 55, 73, 91, 110,},

table4 = {0, 0, 0, 0, 0, 1, 2, 4, 6, 8, 11, 14, 26, 51, 77, 102,},

относительно ниже, чем для более высоких QP. Таким образом, абсолютное значение THR может быть уменьшено без ущерба для производительности. Кроме того, чтобы сохранить размер таблицы одинаковым для всех QP в наборе (что выгодно для упрощения реализации) и уменьшить частоту субсэмплирования для более низких QP (которые соответствуют меньшему уровню сжатия и лучшему качеству восстановленного изображения), может быть выгодно уменьшить количество отброшенных бит N по сравнению с другими таблицами QP, например, установив N для нижнего QP равным 2 и установив THR равным 64. Фиг. 16 иллюстрирует пример передаточных функций фильтра для пяти QP в наборе на основе соответствующих таблиц (от 0 до 4) с N, установленным на 2 для первой таблицы (соответствует нижнему диапазону QP) и N, установленным на 3 для других таблиц. Этот пример также включает в себя способ использования вспомогательной функции для формирования таблиц 3 и 4, как описано в абзацах [00159] - [00160]. Для таблицы 4 прямолинейная вспомогательная функция пересекает ось абсцисс в значении 11. Для таблицы 4 прямолинейная вспомогательная функция пересекает ось абсцисс в значении 9. Таблицы, используемые в этом примере, имеют значения в соответствии с:[00160] As described above in paragraphs [00150]-[00153], the table subsampling method may discard N bits from table values to reduce the size of the table. As also mentioned in these paragraphs, N can be different depending on the QP used to form a particular table and the selected THR value for that table. For example, for lower QP values, the filter parameter

relatively lower than for higher QPs. Thus, the absolute value of THR can be reduced without sacrificing performance. In addition, to keep the table size the same for all QPs in the set (which is beneficial to simplify implementation) and to reduce the subsampling rate for lower QPs (which correspond to less compression and better quality of the reconstructed image), it may be beneficial to reduce the number of discarded bits N by compared to other QP tables, for example by setting N for lower QP to 2 and setting THR to 64. FIG. 16 illustrates an example of filter transfer functions for five QPs in a set based on respective tables (0 to 4) with N set to 2 for the first table (corresponding to the lower QP range) and N set to 3 for the other tables. This example also includes a method for using a helper function to generate tables 3 and 4, as described in paragraphs [00159] - [00160]. For Table 4, the straight-line helper function crosses the x-axis at value 11. For Table 4, the straight-line helper crosses the x-axis at value 9. The tables used in this example have values according to:

table0 = {0, 0, 2, 6, 10, 14, 19, 23, 28, 32, 36, 41, 45, 49, 53, 57,},

table1 = {0, 0, 5, 12, 20, 29, 38, 47, 56, 65, 73, 82, 90, 98, 107, 115,},

table2 = {0, 0, 1, 4, 9, 16, 24, 32, 41, 50, 59, 68, 77, 86, 94, 103,},

table3 = {0, 0, 0, 1, 3, 5, 9, 14, 19, 25, 32, 40, 55, 73, 91, 110,},

table4 = {0, 0, 0, 0, 0, 1, 2, 4, 6, 8, 11, 14, 26, 51, 77, 102,},

[00161] As described in paragraphs [00150]-[00153] above, the table subsampling method may discard N bits from the table values to reduce the size of the table. As also mentioned in these paragraphs, N can be different depending on the QP used to form a particular table and the selected THR value for that table. For example, for higher QP values, the filtering parameter σ is relatively higher than for lower QPs, which may require increasing the THR value to bring the equation from paragraph [0077] closer to 1. At the same time, in order to keep the LUT size the same for all QP in the set (which is beneficial due to the simplification of implementation), and given that for higher QP values, the reconstructed image has more distortion and an increase in LUT subsampling is acceptable due to a subjectively imperceptible effect for LUT subsampling in the presence of strong compression artifacts, the value of N the least significant bits can be increased to 4, for example, for the last and penultimate table in the set. Fig. 17 illustrates an example of filter transfer functions for five QPs in a set based on respective tables (0 to 4) with N equal to 2 for the first table (table 0 corresponding to the lower QP range), N set to 4 for the last and penultimate tables ( table 3 and table 4), and N set to 3 for other tables. In this example, TRH is set to 64 for the first table, set to 256 for the last and penultimate tables, and set to 128 for the rest of the tables. This example also includes a method for using a helper function to generate tables 3 and 4, as described in paragraphs [00159]-[00160]. For table 4, the straight-line helper function crosses the x-axis at value 6. For table 4, the straight-line helper crosses the x-axis at value 8. The tables used in this example have values according to:

table0 = {0, 0, 2, 6, 10, 14, 19, 23, 28, 32, 36, 41, 45, 49, 53, 57,},

table1 = {0, 0, 5, 12, 20, 29, 38, 47, 56, 65, 73, 82, 90, 98, 107, 115,},

table2 = {0, 0, 1, 4, 9, 16, 24, 32, 41, 50, 59, 68, 77, 86, 94, 103,},

table3 = {0, 0, 3, 9, 19, 32, 47, 64, 81, 99, 117, 135, 154, 179, 205, 230,},

table4 = {0, 0, 0, 2, 6, 11, 18, 27, 38, 51, 64, 96, 128, 160, 192, 224,},

[00162] FIG. 18 is a block diagram of an apparatus 600 that can be used to implement various embodiments. Device 600 may be an encoder as shown in FIG. 1 and a decoding device as shown in FIG. 2. In addition, the device 600 may include one or more of the elements described. In some embodiments, device 600 is equipped with one or more input/output devices such as a speaker, microphone, mouse, touch screen, keypad, keyboard, printer, display, and the like. Device 600 may include one or more central processing units (CPU) 610, memory 620, mass storage device 630, video adapter 640, and I/O interface 660 connected to the bus. A bus is one or more of several bus architectures of any type, including a memory bus or memory controller, a peripheral bus, a video bus, and the like.

[00163] CPU 610 may be any type of electronic data processor. Memory 620 may be or be any type of system memory, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), read only memory (ROM), a combination thereof, or the like. In an embodiment, memory 620 may include ROM for use during bootup and DRAM for program and data storage for use during program execution. In embodiments, memory 620 is non-volatile. The mass storage device 630 includes any type of storage device that stores data, programs, and other information, and for providing access to data, programs, and other information via a bus. The mass storage device 630 includes, for example, one or more of a solid state drive, a hard disk drive, a magnetic disk drive, an optical disk drive, and the like.

[00164] Video adapter 640 and I/O interface 660 provide interfaces for connecting external input and output devices to device 600. For example, device 600 may provide a SQL command interface to clients. As illustrated, examples of input and output devices include a display 690 connected to a video adapter 640 and any mouse/keyboard/printer combination 670 connected to an I/O interface 660. Other devices may be connected to device 600 and additional or fewer interface cards may be used. For example, a serial interface card (not shown) may be used to provide a serial interface for a printer.

[00165] Device 600 also includes one or more network interfaces 650, which include wired links such as an Ethernet cable and the like, and/or wireless links to access points or one or more networks 680. Network interface 650 allows device 600 to communicate with remote devices via networks 680. For example, network interface 650 can provide communication with a database. In an embodiment, the device 600 is connected to a local area network or a wide area network for processing data and communicating with remote devices such as other processing units, the Internet, remote repositories, and the like.

[00166] The proposed in-loop or prediction filter scheme has the following advantages over traditional adaptive filtering methods such as ALF:

The proposed frequency domain filter obtains filter parameters (frequency domain gains) from the reconstructed frame or predicted block at the decoder side, so the filter parameters do not need to be transmitted from the encoder to the decoder side.

ALF requires complex rate and distortion optimization (RDO) on the encoder side to reduce the number of weighted coefficients to transmit. The proposed method does not require a complex RDO on the encoder side (without passing parameters) and is applied to all blocks that meet predefined conditions.

ALF is a linear filter in the pixel domain. The proposed filter is non-linear, since the gain for each component of the one-dimensional spectrum depends on the value of this component of the spectrum. This allows for additional coding gain from non-linear processing.

ALF requires universal multipliers on the decoder side. In the proposed method, filtering can be implemented in the form of a lookup table, since the gain for each spectrum coefficient is less than one. Therefore, the proposed method can be implemented without multiplication.

[00167] Thus, a filter is provided that can improve the efficiency of video coding with low complexity.

[00168] While a particular feature or aspect of the disclosure may only be disclosed in relation to one of several implementations or embodiments, such feature or aspect may be combined with one or more other features or aspects of other implementations or embodiments that may be desirable and beneficial. for any given or specific application. Furthermore, to the extent that the terms "comprise", "have", "with", or other variations thereof are used in either the detailed description or the claims, it is intended that such terms are intended to be included in a manner similar to the term "contain". In addition, the terms "exemplary" and "for example" mean simply as an example, and not the best or optimal. The terms "connected" and "connected" and derivatives may have been used. It should be understood that these terms could be used to mean that two elements co-operate or interact with each other whether they are in direct physical or electrical contact, or they are not in direct contact with each other.

[00169] While specific aspects have been illustrated and described herein, those skilled in the art will appreciate that various alternative and/or equivalent implementations may be substituted for the specific aspects shown and described without departing from the scope of this disclosure. This application is intended to cover any adaptations or variations of the specific aspects discussed here.

[00170] Although the elements in the following claims are set forth in a specific sequence with appropriate markings, unless the statement of the claims otherwise implies a specific sequence for implementing some or all of these elements, these elements need not be limited to implementation in that specific sequence.

[00171] Many alternatives, modifications, and variations will be apparent to those skilled in the art in light of the foregoing. Of course, those skilled in the art will readily recognize that there are many applications of the invention beyond those described herein. Although the present invention has been described with reference to one or more specific embodiments, those skilled in the art will recognize that many changes can be made to it without departing from the scope of the present invention. Thus, it is to be understood that within the scope of the appended claims and their equivalents, the invention may be practiced otherwise than as specifically described herein.

Claims (59)

1. A method for processing a block, wherein the block contains a plurality of pixels, the method comprising the steps of: scanning the current pixel of the block and adjacent pixels of the current pixel according to a predetermined scanning pattern, the block being a reconstructed block or a predicted block; get the components of the spectrum by performing the transformation for the current pixel and its neighboring pixels; determining whether the filter is applied to the spectrum components based on the conditions; when a filter is applied to a spectrum component, obtaining filtered spectrum components based on the filter parameter and spectrum components; when the filter is not applied to the spectrum component, keeping the spectrum component unchanged; obtaining filtered pixels by performing an inverse transform on the filtered spectrum components; and generating a filtered block based on the filtered pixels; wherein the conditions contain at least one of the following: the block has a non-zero residual signal; block size; block aspect ratio; block prediction mode; block prediction mode internal; or external block prediction mode. 2. The method of claim 1, wherein the filter parameter and/or the predefined scan pattern is dependent on conditions. 3. The method of claim 1, wherein the filter is applied if the block size obtained as the block width times the block height is greater than 4x4. 4. The method of claim 1, wherein the filter is applied if the block prediction mode is extrinsic and the block has a non-zero residual signal. 5. The method of claim 1, wherein the filter is applied if the block prediction mode is intra. 6. The method of claim 1, wherein the filter is not applied if the block prediction mode is outer and the block width and block height are greater than or equal to 32. 7. The method of claim 6, wherein the acquisition of the filtered spectrum components is evaluated based on a lookup table (LUT). 8. The method of claim 7, wherein the filtered spectrum component

receive through:

where (i) - spectrum component index, R(i) - spectrum component corresponding to index (i), σ - filtering parameter,

is the array in the lookup table and THR is the threshold value. 9. The method according to claim 8, in which

, m is the normalization constant equal to the number of spectrum components. 10. The method according to any one of paragraphs. 7-9, in which the filtering parameter is obtained based on the codec quantization parameter (QP). 11. The method of claim 10, wherein the filtering parameter σ is defined as

, where QP is the codec quantization parameter,

and

- constants. 12. The method according to claim 11, in which

modified based on the color bit depth of the pixels

* (1 << (bit_depth - 8)). 13. The method of claim 12, wherein the filter parameter is zero if QP <= 17 and the block is skipped without filtering. 14. The method of claim 8, wherein the THR depends on the QP value. 15. The method of claim 14 wherein the THR is obtained as:

= C, where C is a value close to 1. 16. The method according to any one of paragraphs. 1-15, in which a predetermined scan pattern is defined as a set of spatial or raster offsets relative to the position of the current pixel within the block. 17. The method of claim 16, wherein when the predetermined scan pattern is defined as a set of raster offsets relative to the position of the current pixel within the block, the point offsets to neighboring pixels are within the block. 18. The method of claim 17, wherein the predefined scan pattern is defined as follows: scanTmpl[4] (y, x) = {(0.0), (0.1), (1.0), (1, one)}. 19. The method according to any one of paragraphs. 1-18, in which at least one filtered pixel is placed in its original position in accordance with a predetermined scan pattern. 20. The method according to any one of paragraphs. 1-19, in which all filtered pixels are added to the accumulation buffer according to a predetermined scan pattern, and in which the accumulation buffer is initialized to zero before obtaining the filtered spectrum components. 21. The method of claim 20, wherein the final filtered pixels are obtained as the accumulated values in the accumulation buffer divided by the number of pixels added to the current position of the accumulation buffer; and wherein a filtered block is generated based on the final filtered pixels. 22. The method according to any one of paragraphs. 1-21, in which the differences between all filtered and corresponding unfiltered pixels are added to an accumulation buffer according to a predetermined scan pattern, and in which the accumulation buffer is initialized by the unfiltered pixels multiplied by the maximum number of pixel values to be added to the block. 23. The method of claim 22, wherein the final filtered pixels are obtained as the accumulated values in the accumulation buffer divided by the maximum number of pixel values to be added to the block. 24. The method according to any one of paragraphs. 1-23, comprising the steps of: scanning the current pixel of the block and its neighboring pixels of the current pixel in accordance with a predetermined scan pattern into the accumulation buffer; and the spectrum components are obtained by performing a pixel transformation in the accumulation buffer. 25. The method according to any one of paragraphs. 1-24, in which the transformation is a one-dimensional transformation. 26. The method according to any one of paragraphs. 1-25, in which the transform is the Hadamard transform. 27. A decoder containing a processing circuit for performing the method according to any one of paragraphs. 1-26. 28. An encoder containing a processing circuit for performing the method according to any one of paragraphs. 1-26. 29. A decoder containing a filter for performing the method according to any one of paragraphs. 1-26. 30. An encoder containing a filter for performing the method according to any one of paragraphs. 1-26. 31. Decoder, containing: a storage device containing instructions; and one or more processors associated with the memory, the one or more processors executing instructions for performing the method of any one of claims. 1-26. 32. An encoder containing: a storage device containing instructions; and one or more processors associated with the memory, the one or more processors executing instructions for performing the method of any one of claims. 1-26. 33. A computer-readable storage medium, wherein program code is stored in the computer-readable storage medium and, when said program code is executed on a computer or processor, a method according to any one of paragraphs is performed. 1-26.

Images