TWI704800B - Boundary filtering for sub-block - Google Patents

Abstract

Devices, systems and methods for boundary filtering for sub-block based prediction are described. Partitioning a block can result in discontinuities between adjacent sub-blocks that may introduce some undesirable high-frequency energy in the residual signal, which can deteriorate the performance of subsequent transform coding. Implementations of the disclosed technology can reduce the effect of the discontinuities. In a representative aspect, a method for video coding includes partitioning a block of video data into multiple sub-blocks, filtering, using a set of filter taps/coefficients, boundary samples of at least one sub-block of the multiple sub-blocks to generate filtered boundary samples, and perform prediction of the block of video data using the filtered boundary samples.

Description

Sub-block boundary filtering

This patent document generally involves image and video coding technology. [Cross reference of related applications]

In accordance with the applicable patent law and/or in accordance with the rules of the Paris Convention, this application is to timely claim the priority and rights of the international patent application number PCT/CN2018/093634 filed on June 29, 2018. For all purposes of US law, the entire disclosure of International Patent Application No. PCT/CN2018/093634 is incorporated herein by reference as a part of the disclosure of this patent document.

Motion compensation is a technique in video processing that predicts the frame in the video given the previous frame and/or the future frame by considering the motion of the camera and/or the object in the video. Motion compensation can be used for encoding and decoding of video data compressed by video.

Described are devices, systems, and methods for image and video coding related to boundary filtering for sub-block-based prediction.

In a representative aspect, the disclosed technology can be used to provide a method for video encoding, the method includes dividing a current video material block into a plurality of sub-blocks, and generating a prediction sub-block for at least one of the plurality of sub-blocks, A set of filter coefficients is used to filter the boundary samples of the predicted sub-block of at least one sub-block to generate filtered boundary samples, and the filtered boundary samples are used to determine the final prediction block of the current video data block.

In another representative aspect, the disclosed technology may be used to provide a method for video decoding, the method including filtering boundary samples of a prediction sub-block using a set of filter coefficients to generate filtered boundary samples. A prediction sub-block is generated for at least one sub-block of the multiple sub-blocks of the current video block. The method also includes performing prediction of the current video block of the video material using the filtered boundary samples, and reconstructing the current video block using the prediction.

In yet another representative aspect, a device configured or operable to perform the above method is disclosed. The device may include a processor programmed to implement the method.

In yet another representative aspect, the video encoder device may implement the method as described herein. In yet another representative aspect, the video decoder device may implement the method as described herein.

In yet another representative aspect, the above method is implemented in the form of processor executable code and stored in a computer-readable program medium.

The above and other aspects and features of the disclosed technology are described in more detail in the drawings, specification, and scope of patent application.

Due to the increasing demand for higher-resolution video, video coding methods and technologies are ubiquitous in modern technologies. Video codecs typically include electronic circuits or software that compress or decompress digital video, and are constantly being improved to provide higher coding efficiency. Video codecs convert uncompressed video to compressed format and vice versa. Video quality, the amount of data used to represent the video (determined by the bit rate), the complexity of encoding and decoding algorithms, the sensitivity to data loss and errors, ease of editing, random access and end-to-end delay (delay ) There is a complicated relationship. The compression format usually conforms to standard video compression specifications, for example, the High Efficiency Video Coding (HEVC) standard (also known as H.265 or MPEG-H Part 2), the multifunctional video coding standard to be completed, or other Current and/or future video coding standards.

First, the sub-block-based prediction is introduced into the video coding standard through the High Efficiency Video Coding (HEVC) standard. With sub-block-based prediction, a block such as a coding unit (Coding Unit, CU) or a prediction unit (Prediction Unit, PU) is divided into several non-overlapping sub-blocks. Different sub-blocks can be assigned different motion information, such as a reference index or a motion vector (Motion Vector, MV), and motion compensation (Motion Compensation, MC) can be performed separately for each sub-block. Fig. 1 shows an example of sub-block-based prediction.

The embodiments of the disclosed technology can be applied to existing video coding standards (for example, HEVC, H.265) and future standards to improve runtime performance. The chapter titles are used in this document to improve the readability of the description, and the discussion or examples (and/or implementation) are not limited to the corresponding chapters in any way.

1. Example of prediction based on sub-block

Use a reference software called Joint Exploration Model (JEM) to study future video coding technologies. In JEM, sub-block-based prediction is used in several coding tools, such as affine prediction, alternative temporal motion vector prediction (Alternative Temporal Motion Vector Prediction, ATMVP), and spatial-temporal motion vector prediction (Spatial-Temporal Motion Vector). Prediction, STMVP), Bi-Directional Optical Flow (BIO), Frame-Rate Up Conversion (FRUC), Locally Adaptive Motion Vector Resolution (LAMVR), overlap Overlapped Block Motion Compensation (OBMC), Local Illumination Compensation (LIC), and Decoder-side Motion Vector Refinement (DMVR).

1.1 Examples of affine prediction

等式（1）In HEVC, only translation (translation) motion models are applied to motion compensation prediction (Motion Compensation Prediction, MCP). However, the camera and the object may have various movements, for example, zoom in/out, rotation, perspective motion, and/or other irregular motions. On the other hand, JEM applies simplified affine transform motion compensation prediction. FIG. 2 shows an example of the affine motion field of the block 200 described by two control point motion vectors V0 and V1. The Motion Vector Field (MVF) of block 200 can be described by the following equation:

Equation (1)

等式（2）As shown in Figure 2, (v _0x , v _0y ) is the motion vector of the control point in the upper left corner, and (v _1x , v _1y ) is the motion vector of the control point in the upper right corner. In order to simplify the motion compensation prediction, sub-block-based affine transform prediction can be applied. The sub-block size M×N is derived as follows:

Equation (2)

Here, MvPre is the motion vector score accuracy (for example, 1/16 in JEM). (v _2x ,v _2y ) is the motion vector of the lower left control point, calculated according to equation (1). If necessary, you can adjust M and N downwards so that they are divisors of w and h , respectively.

FIG. 3 shows an example of the affine MVF of each sub-block of the block 300. In order to derive the motion vector of each M×N sub-block, the motion vector of the center sample of each sub-block can be calculated according to equation (1) and rounded to the accuracy of the motion vector score (for example, 1/16 in JEM). Then, a motion compensation interpolation filter can be applied to generate a prediction for each sub-block using the derived motion vector. After the MCP, the high-precision motion vector of each sub-block is rounded and saved to the same accuracy as that of the normal motion vector.

1.2 Examples of optional temporal motion vector prediction (ATMVP)

In the ATMVP method, the temporal motion vector prediction (TMVP) method is modified by extracting multiple sets of motion information (including motion vectors and reference indexes) from blocks smaller than the current CU.

FIG. 4 shows an example of the ATMVP motion prediction process for the CU 400. The ATMVP method predicts the motion vector of the sub-CU 401 in the CU 400 in two steps. The first step is to use the time domain vector to identify the corresponding block 451 in the reference picture 450. The reference picture 450 is also called a motion source picture. The second step is to divide the current CU 400 into sub-CUs 401, and obtain the motion vector and reference index of each sub-CU from the block corresponding to each sub-CU.

In the first step, the reference picture 450 and the corresponding block are determined by the motion information of the spatial neighboring blocks of the current CU 400. In order to avoid the repeated scanning process of neighboring blocks, the first Merge candidate in the Merge candidate list of the current CU 400 is used. The first available motion vector and its associated reference index are set as the time domain vector and the index to the motion source picture. In this way, compared with TMVP, the corresponding block can be identified more accurately, where the corresponding block (sometimes called a collocated block) is always in the lower right or center position relative to the current CU.

In the second step, by adding a time domain vector to the coordinates of the current CU, the corresponding block of the sub-CU 451 is identified from the time domain vector in the motion source picture 450. For each sub-CU, the motion information of the corresponding block (for example, the smallest motion grid covering the center sample) is used to derive the motion information of the sub-CU. After the motion information of the corresponding NxN block is identified, it is converted into the motion vector and reference index of the current sub-CU in the same way as the TMVP of HEVC, where motion scaling and other processes are applicable. For example, the decoder checks whether low-latency conditions are met (for example, the POC of all reference pictures of the current picture is less than the POC of the current picture) and may use the motion vector MVx (for example, the motion vector corresponding to the reference picture list X) for Predict the motion vector MVy of each sub-CU (for example, where X is equal to 0 or 1 and Y is equal to 1-X).

1.3 Examples of Space-Time Motion Vector Prediction (STMVP)

In the STMVP method, the motion vector of the sub-CU is derived recursively following the raster scan order. Fig. 5 shows an example of one CU with four sub-blocks and neighboring blocks. Consider an 8×8 CU 500 that includes four 4×4 sub CUs A (501), B (502), C (503), and D (504). The adjacent 4×4 blocks in the current frame are labeled a (511), b (512), c (513), and d (514).

The motion derivation of sub CU A starts by identifying its two spatial neighbors. The first neighbor is the N×N block above the sub-CU A 501 (block c 513). If the block c (513) is not available or is in-frame coding, then check the other N×N blocks above the sub-CU A (501) (from left to right, starting from block c 513). The second neighbor is the block to the left of the sub-CU A 501 (block b 512). If block b (512) is not available or is intra-frame coding, check other blocks on the left side of sub-CU A 501 (from top to bottom, starting from block b 512). The motion information obtained from the neighboring blocks of each list is scaled to the first reference frame of the given list. Next, the temporal motion vector prediction value (TMVP) of the sub-block A 501 is derived by following the same process as the TMVP derivation specified in HEVC. The motion information of the co-located block at block D 504 is extracted and scaled accordingly. Finally, after retrieving and scaling the motion information, all available motion vectors are averaged for each reference list. The average motion vector is designated as the motion vector of the current sub-CU.

1.4 Example of bidirectional optical flow (BIO)

Motion estimation can be performed based on one reference picture (also called unidirectional prediction) or based on two reference pictures (also called bidirectional prediction). The Bidirectional Optical Flow (BIO) method is a sample-wise motion refinement performed on top of block-wise motion compensation for bidirectional prediction. In some embodiments, sample-level motion refinement does not use signaling.

是塊運動補償之後的參考k(k=0,1)的亮度值，並且

分別是

梯度的水平和垂直分量。假設光流有效，運動向量場

由下式給出：

等式（3）Assume

Is the luminance value of reference k (k=0,1) after block motion compensation, and

Respectively are

The horizontal and vertical components of the gradient. Assuming that the optical flow is valid, the motion vector field

It is given by:

Equation (3)

和末端導數

兩者的唯一的三階多項式。該多項式在t=0時的值是BIO預測：

等式（4）For the trajectory of each sample, the optical flow equation is combined with Hermite interpolation, and the result is the matching function value

And terminal derivative

The only third-order polynomial of both. The value of the polynomial at t=0 is the BIO prediction:

Equation (4)

和

表示到參考影格的距離。基於Ref₀ 和Ref₁ 的POC計算距離

和

：

=POC(當前)-POC(Ref₀ )，

=POC(Ref₁ )-POC(當前)。如果兩個預測都來自相同的時間方向（兩者都來自過去或兩者都來自未來），則符號是不同的（例如，

）。在這種情況下，如果預測不是來自相同的時刻（例如，

），則應用BIO。參考區域兩者都具有非零運動（例如，

），並且塊運動向量與時間距離成比例（例如，

）。Figure 6 shows an example optical flow trajectory in the bidirectional optical flow (BIO) method. Here,

with

Indicates the distance to the reference frame. Calculate distance based on POC of Ref ₀ and Ref ₁

with

:

=POC(current)-POC(Ref ₀ ),

=POC(Ref ₁ )-POC(current). If both predictions are from the same time direction (both from the past or both from the future), the signs are different (for example,

). In this case, if the prediction is not from the same moment (for example,

), then apply BIO. Both reference areas have non-zero motion (e.g.,

), and the block motion vector is proportional to the time distance (for example,

).

。圖7A至圖7B示出了運動軌跡和參考影格平面的交叉的示例。對於Δ，模型僅使用局部泰勒展開的第一個線性項：

等式（5）Determine the motion vector field by minimizing the difference Δ between the values in points A and B

. 7A to 7B show examples of the intersection of the motion trajectory and the reference frame plane. For Δ, the model only uses the first linear term of the local Taylor expansion:

Equation (5)

。假設運動在局部周圍區域是一致的，Δ可以在以當前預測點

為中心的(2M+1)x(2M+1)方窗口

內最小化，其中M等於2：

等式（6）All values in the above equation depend on the sample position and are expressed as

. Assuming that the motion is consistent in the local surrounding area, Δ can be used at the current prediction point

Centered (2M+1)x(2M+1) square window

Minimize within, where M is equal to 2:

Equation (6)

等式（7）

等式（8） 其中，

等式（9）For this optimization problem, JEM uses a simplified method, first minimizing in the vertical direction, and then minimizing in the horizontal direction. This causes the following results:

Equation (7)

Equation (8) where,

Equation (9)

等式（10）

等式（11）In order to avoid division by zero or very small values, regularization parameters r and m can be introduced in equation (7) and equation (8). among them:

Equation (10)

Equation (11)

Here, d is the bit depth of the video sample.

，是針對當前塊內的位置計算的。圖7A示出了塊700外部的訪問位置的示例。如圖7A所示，在等式（9）中，以預測塊的邊界上的當前預測點為中心的(2M+1)x(2M+1)方視窗Ω需要訪問塊外部的位置。在JEM中，塊外部的

的值被設置為等於塊內最近的可用的值。例如，這可以被實施為填充區域701，如圖7B所示。In order to keep the memory access of BIO the same as the conventional two-way predictive motion compensation, all predictions and gradient values,

, Is calculated for the position in the current block. FIG. 7A shows an example of an access location outside the block 700. As shown in FIG. 7A, in equation (9), the (2M+1)x(2M+1) square window Ω centered on the current prediction point on the boundary of the prediction block needs to access a location outside the block. In JEM, outside the block

The value of is set equal to the nearest available value in the block. For example, this can be implemented as a filled area 701, as shown in Figure 7B.

等式（12）Using BIO, it is possible to refine the playing field for each sample. In order to reduce computational complexity, BIO's block-based design is used in JEM. The motion refinement can be calculated based on 4×4 blocks. In block-based BIO, the value s _n is 4 × 4 blocks in all samples in equation (9) may be polymerized, and the polymerization is then used to derive values of s _n BIO 4 × 4 motion vector block Offset. More specifically, the following formula can be used for block-based BIO derivation:

Equation (12)

Here, b _k represents the sample set of the k-th 4×4 block belonging to the prediction block. S _{n in} equation (7) and equation (8) is replaced by ((s _n ,b _k )>>4) to derive the associated motion vector offset.

In JEM, BIO is only called for the luminance component.

1.5 Example of Frame Rate Up Conversion (FRUC)

When the Merge flag of the CU is true, the FRUC flag can be notified for CU signaling. When the FRUC flag is false, the Merge index can be signaled and the regular Merge mode is used. When the FRUC flag is true, an additional FRUC mode flag can be signaled to indicate which method (for example, bilateral matching or template matching) will be used to derive the motion information of the block.

On the encoder side, the decision on whether to use the FRUC Merge mode for the CU is based on the RD cost selection made for the normal Merge candidates. For example, by using RD cost selection to check multiple matching modes of CU (for example, bilateral matching and template matching). The matching mode that causes the least cost is further compared with other CU modes. If the FRUC matching mode is the most effective mode, set the FRUC flag to true for the CU and use the relevant matching mode.

Typically, the motion derivation process in the FRUC Merge mode has two steps: first perform a CU-level motion search, and then perform sub-CU-level motion refinement. At the CU level, the initial motion vector is derived for the entire CU based on bilateral matching or template matching. First, a list of MV candidates is generated, and the candidate that causes the smallest matching cost is selected as the starting point for further CU-level refinement. Then, a local search based on bilateral matching or template matching is performed around the starting point. The MV that causes the smallest matching cost is taken as the MV of the entire CU. Subsequently, the derived CU motion vector is used as the starting point to further refine the motion information at the sub-CU level.

CU運動資訊推導執行以下推導過程。在第一階段，推導整個

CU的MV。在第二階段，CU進一步被分成

子CU。如等式（13）中那樣計算M的值，D是預定義的分裂深度，其在JEM中默認設置為3。然後推導每個子CU的MV。

等式（13）For example, for

CU motion information derivation performs the following derivation process. In the first stage, the whole

CU’s MV. In the second stage, CU is further divided into

Child CU. Calculate the value of M as in equation (13). D is the predefined split depth, which is set to 3 by default in JEM. Then derive the MV of each sub-CU.

Equation (13)

Figure 8 shows an example of bilateral matching used in the frame rate up conversion (FRUC) method. Bilateral matching is used to find the closest match between two blocks along the motion trajectory of the current CU (800) in two different reference pictures (810, 811) to derive the motion information of the current CU. Under the assumption of continuous motion trajectories, the time distance between the motion vectors MV0 (801) and MV1 (802) pointing to the two reference blocks and the current picture and two reference pictures (for example, TD0 (803) and TD1 (804) ) Proportional. In some embodiments, when the current picture 800 is temporally between two reference pictures (810, 811) and the temporal distance from the current picture to the two reference pictures is the same, the bilateral matching becomes a mirror-based two-way MV.

Figure 9 shows an example of template matching used in the frame rate up conversion (FRUC) method. Template matching can be used to find the closest match between a template in the current picture (for example, the top and/or left adjacent block of the current CU) and a block in the reference picture 910 (for example, a block of the same size as the template) To derive the current movement information of CU 900. In addition to the aforementioned FRUC Merge mode, template matching can also be applied to AMVP mode. In both JEM and HEVC, AMVP has two candidates. Using template matching methods, new candidates can be derived. If the newly derived candidate matched by the template is different from the first existing AMVP candidate, it is inserted into the very beginning of the AMVP candidate list (for example, by removing the second existing AMVP candidate), and then the list size is set to 2. When applied to AMVP mode, only CU-level search is applied.

1.6 Example of MV derived for MC in chroma components

In one example, HEVC standard defines how the luminance components from the MC for the MV (labeled mv) for deriving the MV MC chrominance component (labeled mvC). Generally speaking, mvC is calculated as mv multiplied by a factor, which depends on the color format, such as 4:2:0 or 4:2:2.

2. Boundary filtering in sub-block-based prediction

In some existing implementations, sub-block-based prediction is used because it is generally more accurate than whole-block prediction because it can divide a block into multiple parts, each part having its own MV. However, segmentation may result in discontinuities between two adjacent sub-blocks along its boundary. The discontinuity may introduce some undesirable high-frequency energy in the residual signal, which may deteriorate the performance of subsequent transform coding.

The examples described below for various embodiments illustrate the use of boundary filtering for sub-block-based prediction to improve video coding efficiency (for example, to reduce discontinuities at sub-block boundaries) and to enhance existing and future video coding standards. In the following example that should not be interpreted as a limitation, the width and height of the current block of the component are denoted as W and H , respectively, and the width and height of the sub-block assigned to the component are denoted as W and h, respectively .

Example 1. The prediction samples along the boundary of the sub-block are filtered before they are used to determine the prediction block of the current block, and the sub-block is divided by the prediction based on the sub-block. (A) In one example, the boundary includes only the inner boundary (for example, the boundary between sub-blocks) as shown in the example in FIG. 10A. The shaded area covers the samples along the boundary. (B) In one example, the boundary includes the inner boundary and the outer boundary as shown in the example in FIG. 10B (for example, the boundary between a sub-block and other blocks that have been encoded or decoded). The shaded area covers the samples along the boundary. (C) In one example, there may be N (N>=0) columns of prediction samples along the vertical boundary and M (M>=0) rows of prediction samples along the horizontal boundary to be filtered. 11A and 11B show examples of prediction samples for M=N=2. (I) In an example, M and/or N depend on the width and height of the sub-block; for example, if the shape of the sub-block is 4×4, then M=N=2; if the shape of the sub-block is 8×8 , Then M=N=4. (Ii) In one example, M and/or N depend on color components. For example, for the luminance component, M=N=2; for the chrominance component, M=N=1. (Iii) In one example, M and/or N depend on the location of the boundary. For example, if the boundary is between a sub-block and the adjacent block to be coded/decoded, then M=N=2; if the boundary is between two sub-blocks, then M=N=1 (Iv) In one example, M and/or N may depend on the position of the sub-block. Alternatively, it may depend on how many neighboring blocks are encoded/decoded, and/or how many prediction blocks of neighboring blocks are available. (V) In one example, M and N are signaled from the encoder to the decoder. For example, video parameter set (Video Parameter Set, VPS), sequence parameter set (Sequence Parameter Set, SPS), picture parameter set (Picture Parameter Set, PPS), strip header, coding tree unit (Coding Tree Unit, CTU) or coding unit (CU) signaling M and N.

Example 2. The filtering of prediction samples along the boundaries of sub-blocks may depend on neighboring prediction samples. If the boundary is the outer boundary, that is, it is between the sub-block and the encoded/decoded neighboring block, the filtering can also depend on the reconstructed samples of the encoded/decoded neighboring block.

{

}/

，其中x(k) 是相鄰樣本，並且a(k) 是濾波抽頭。 （a）在一個示例中，如果邊界是垂直的，則依賴的相鄰樣本在要被濾波的樣本的相同列中。例如，如圖12A所示，依賴的相鄰樣本包括q0、p0和p1，其中p0是要被濾波的樣本。 （b）在一個示例中，如果邊界是水平的，則依賴的相鄰樣本在要被濾波的樣本的相同行中。例如，如圖12B所示，依賴的相鄰樣本包括p1、p0和q0，其中p0是要被濾波的樣本。 （c）在一個示例中，依賴的相鄰樣本可以在2-D區域而不是1-D線內。例如，如圖12C所示，依賴的相鄰樣本包括p0周圍的3×3區域，其中p0是要被濾波的樣本。 （d）在一個示例中，依賴的相鄰樣本的選擇取決於子塊的寬度和高度。 （e）在一個示例中，依賴的相鄰樣本的選擇取決於顏色分量。 （f）在一個示例中，依賴的相鄰樣本的選擇取決於邊界的位置。 （g）在一個示例中，依賴的相鄰樣本的選擇從編碼器向解碼器信令通知。例如，選擇可以在視頻參數集（VPS）、序列參數集（SPS）、圖片參數集（PPS）、條帶標頭、編碼樹單元（CTU）或編碼單元（CU）中信令通知。Example 3. When samples along the boundary of the sub-block are filtered, the filtered samples are calculated as the output of the filter applied to several dependent neighboring samples (in one example, including filtered The sample itself). These "dependent adjacent" samples include adjacent samples and non-adjacent samples covered by the filtering range. In the formula,

{

}/

, Where x(k) is the neighboring sample, and a(k) is the filter tap. (A) In one example, if the boundary is vertical, the dependent neighboring samples are in the same column of the samples to be filtered. For example, as shown in FIG. 12A, the dependent neighboring samples include q0, p0, and p1, where p0 is the sample to be filtered. (B) In one example, if the boundary is horizontal, the dependent neighboring samples are in the same row of the samples to be filtered. For example, as shown in FIG. 12B, dependent neighboring samples include p1, p0, and q0, where p0 is the sample to be filtered. (C) In one example, the dependent neighboring samples can be in the 2-D area instead of the 1-D line. For example, as shown in FIG. 12C, dependent neighboring samples include a 3×3 area around p0, where p0 is the sample to be filtered. (D) In one example, the choice of dependent neighboring samples depends on the width and height of the sub-block. (E) In one example, the choice of dependent neighboring samples depends on the color components. (F) In one example, the choice of dependent neighboring samples depends on the location of the boundary. (G) In one example, the selection of dependent neighboring samples is signaled from the encoder to the decoder. For example, the selection can be signaled in a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice header, a coding tree unit (CTU), or a coding unit (CU).

Example 4. Filter taps/coefficients and/or filter support (filter support) can be predefined. (A) Filter taps/coefficients can be the same for vertical filtering and horizontal filtering. For example, and as shown in Figures 12A and 12B, p0'=(3×p0+q0+2)>>2, where p0' is the filtered sample. (B) Filter taps/coefficients can be the same for symmetrical samples around the boundary. For example, and as shown in FIGS. 12A and 12B, the coefficient of the target sample p0 is 3, and the coefficient of the adjacent sample q0 is 1: p0'=(3×p0+q0+2)>>2. The coefficient of the target sample q0 is 3, and the coefficient of the adjacent sample p0 is 1: q0'=(3×q0+p0+2)>>2. (C) Filter taps/coefficients can depend on the distance between the sample to be filtered and the boundary. For example, and as shown in FIGS. 12A and 12B, p0'=(3×p0+q0+2)>>2 and p1'=(6×p1+p0+q0+4)>>3. (D) In one example, the filter taps/coefficients depend on the width and height of the sub-block. (E) In one example, the filter taps/coefficients depend on the color components. (F) In one example, and as shown in Figure 12C, the filter taps/coefficients depend on the position of the boundary/sub-block.

Example 5. The indication of filter taps/coefficients and/or filter support can be signaled from the encoder to the decoder, such as in video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS) , Slice header, coding tree unit (CTU) or coding unit (CU) signaling.

Example 6. Boundary filtering can be Finite Impulse Response (FIR) filtering or Infinite Impulse Response (IIR) filtering.

Example 7. The filtering process can be done in a given order, such as first all vertical boundaries, then all horizontal boundaries, and vice versa. (A) Alternatively, the filtering process may be applied according to the sub-block scanning order (for example, the raster scanning order of the sub-blocks). For each sub-block, the vertical boundary can be filtered first, and then the horizontal boundary, and vice versa. (B) In one example, all adjacent samples used for filtering samples should be extracted before filtering adjacent samples. In other words, the filtering process is applied to the unfiltered samples. (C) In one example, some or all neighboring samples used for filtering the samples are extracted after filtering the neighboring samples. (I) In one example, all vertical boundaries are filtered first, and then horizontal boundaries are filtered, where neighboring samples have been filtered in vertical boundary filtering. (Ii) In one example, all horizontal boundaries are filtered first, and then vertical boundaries are filtered, where adjacent samples have been filtered in horizontal boundary filtering.

Example 8. Boundary filtering may depend on some coding information of the current coding block or the neighboring blocks of coding/decoding. (A) The coding information includes MV, quantization parameter (QP), intra-frame prediction mode, inter-frame prediction direction, Merge mode or AMVP mode, etc. (B) In one example, filtering is enabled/disabled adaptively for each sub-block. When a sub-block and its horizontal/vertical neighboring blocks have similar motion information (for example, if they have the same reference picture and the absolute motion vector component difference on each reference picture is less than 1 integer pixel), the sub-block Disable the corresponding horizontal/vertical boundary filtering.

Example 9. Assuming that all sub-blocks have non-zero or zero residual values, the de-blocking filtering method can be applied to the boundary filtering of the sub-blocks. In this case, the deblocking filter is applied to the predicted samples instead of reconstructed samples. (A) Alternatively, de-blocking filtering can be applied to both prediction samples and reconstruction samples.

Example 10. It should be noted that it is allowed to filter some samples along the boundary without filtering the remaining samples along the boundary. (A) Filtering or non-filtering can be determined implicitly. (B) In one example, the rule of enabling/disabling the deblocking filter for the PU/TU/CU boundary can be used to determine whether the samples located at the sub-block boundary should be filtered. (C) In one example, the rule for determining whether the samples located at the boundary of the sub-block should be filtered may depend on the block shape/size/coding mode/reconstruction sample/predicted sample value.

Example 11. When there is at least one sub-block encoded with bidirectional prediction, the filtering process can be applied twice to each intermediate prediction block corresponding to the reference picture list. (A) Alternatively, apply filtering to the final prediction block. (B) Similarly, for multi-hypothesis coding, filtering can be performed on each intermediate prediction block or filtering on the final prediction block.

Example 12. The proposed method can be applied to certain color components. (A) In one example, only the luma block can enable the proposed method. (B) The proposed method can be applied to certain block sizes/shapes, and/or certain sub-block sizes. (C) The proposed method can be applied to certain coding tools, such as alternative temporal motion vector prediction (ATMVP) and/or affine prediction.

The above examples can be combined in the context of the methods described below, for example, the method 1300, which can be implemented at a video decoder and/or a video encoder.

Figure 13A shows a flowchart of an example method 1300 for video encoding. The method 1300 includes, at operation 1302, dividing a current video material block into a plurality of sub-blocks. The method 1300 includes, at operation 1304, generating a predicted sub-block for at least one sub-block of a plurality of sub-blocks. The method 1300 includes, at operation 1306, using a set of filter coefficients to filter boundary samples of a prediction sub-block of at least one sub-block to generate filtered boundary samples. The method 1300 includes, at operation 1308, using the filtered boundary samples to determine a final prediction block for the current block of video material.

Figure 13B shows a flowchart of an example method 1350 for video decoding. The method 1350 includes, at operation 1352, using a set of filter coefficients to filter boundary samples of the prediction sub-block to generate filtered boundary samples. A prediction sub-block is generated for at least one sub-block of the multiple sub-blocks of the current video block. Method 1350 includes, at operation 1354, performing prediction of a current video block of video material using the filtered boundary samples. The method 1350 also includes, at operation 1356, reconstructing the current video block using prediction.

In some embodiments of both encoding and decoding methods, the prediction sub-block is generated by performing bidirectional prediction using two references, and filtering includes filtering at least one intermediate prediction block of the bidirectional prediction. In some embodiments, the filtering includes deblocking filtering, and at least one of the sub-blocks includes non-zero or zero residual values.

In some embodiments of both encoding and decoding methods, at least a first subset of boundary samples are located along the inner boundary between two adjacent sub-blocks in a block of video material. The second subset of boundary samples is located along the outer boundary between one of the multiple sub-blocks of the video material block and the second video material block. The boundary samples may include multiple columns or rows of samples along the boundary. The encoding or decoding method may include determining the number of columns or rows of boundary samples based on the size of at least one sub-block, determining the number of columns or rows based on the color component of the at least one sub-block, and determining whether the boundary samples are located along the inner boundary or the outer boundary The number of columns or rows, the number of columns or rows is determined based on the position of the sub-block in the video data block, or based on the video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), and strip header The number of columns or rows of information in the coding tree unit (CTU) or coding unit (CU).

In some embodiments, filtering the boundary samples of the plurality of sub-blocks includes filtering the boundary samples by applying a set of filter coefficients to adjacent samples adjacent to the boundary samples. In some embodiments, adjacent samples and boundary samples are located in the same column or row. In some embodiments, neighboring samples are located in a two-dimensional area around the boundary samples.

In some embodiments, the encoding or decoding method may include selecting neighboring samples based on the size of the corresponding sub-block, selecting neighboring samples based on the color components of the corresponding sub-block, selecting neighboring samples based on the position of the boundary sample, or based on the video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), slice header, coding tree unit (CTU) or coding unit (CU) information in the selection of adjacent samples.

In some embodiments, the set of filter coefficients is the same for horizontal filtering and vertical filtering. In some embodiments, the set of filter coefficients is the same for symmetric samples surrounding the boundary of the sub-block.

In some embodiments, the encoding or decoding method may include determining a set of filter coefficients of boundary samples based on the distance from the boundary sample to the corresponding boundary, determining a set of filter coefficients based on the size of at least one sub-block, and determining a set of filter coefficients based on the size of at least one sub-block. The color components determine a set of filter coefficients, or based on the video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), slice header, coding tree unit (CTU) or coding unit (CU) The information in determines a set of filter coefficients.

In some embodiments, the filtering includes finite impulse response (FIR) filtering or infinite impulse response (IIR) filtering. In some embodiments, the filtering includes: for each sub-block in the plurality of sub-blocks, filtering a set of boundary samples along the vertical boundary; and for each sub-block in the plurality of sub-blocks, After a set of boundary samples are filtered, the remaining boundary samples along the horizontal boundary are filtered. The filtering of multiple sub-blocks is sequentially performed according to the sub-block scanning.

Alternatively, the filtering may include: for each of the plurality of sub-blocks, filtering a set of boundary samples along the horizontal boundary; and for each of the plurality of sub-blocks, filtering a set of boundary samples along the horizontal boundary. After the group boundary samples are filtered, the remaining boundary samples along the vertical boundary are filtered. The filtering of multiple sub-blocks is sequentially performed according to the sub-block scanning.

In some embodiments, filtering includes: filtering a set of boundary samples along the vertical boundary of all the plurality of sub-blocks; and after filtering a set of boundary samples along the vertical boundary, filtering the set of boundary samples along all the plurality of sub-blocks The remaining boundary samples of the horizontal boundary are filtered.

Alternatively, the filtering includes: filtering a set of boundary samples along the horizontal boundary of all the plurality of sub-blocks; and after filtering a set of boundary samples along the horizontal boundary, filtering the vertical The remaining boundary samples of the boundary are filtered.

In some embodiments, filtering is based on one or more attributes of the video data block or adjacent video data blocks, where one or more attributes include motion vector, quantization parameter (QP), intra-frame prediction mode, inter-frame prediction direction, Merge mode or Advanced Motion Vector Prediction (AMVP) mode. The filtering can be adaptively enabled or disabled for each of the multiple sub-blocks.

Figure 14 shows a flowchart of an exemplary method for video encoding. The method 1400 includes, at step 1410, dividing a block of video material into a plurality of sub-blocks.

The method 1400 includes, at step 1420, using a set of filter taps/coefficients to filter boundary samples of at least one of the plurality of sub-blocks to generate filtered boundary samples. In some embodiments, the boundary of at least one sub-block is an inner boundary of a video material block, and wherein the boundary samples include prediction samples from neighboring sub-blocks of multiple sub-blocks, as described in the context of FIG. 10A. In other embodiments, the boundary of at least one sub-block is the outer boundary of the video material block, and wherein the boundary samples include reconstructed samples from adjacent video material blocks, as described in the context of FIG. 10B. In other embodiments, the set of filter taps/coefficients may be different in order to filter different boundaries of the same or different sub-blocks (of multiple sub-blocks). For example, a set of filters (eg, filter bank) may be used, with different predetermined or adaptive filter taps/coefficients for each filter.

In some embodiments, and as described in the context of Example 3, the boundary may be vertical or horizontal. For example, the boundary of at least one sub-block includes a vertical boundary, and the column index of each prediction sample is the same as the column index of each adjacent sample. For example, the boundary of at least one sub-block includes a horizontal boundary, and the row index of each prediction sample is the same as the row index of each adjacent sample. For example, adjacent samples include samples with at least two different column indexes or at least two different row indexes, thereby forming a two-dimensional sample area.

In some embodiments, and as described in the context of Example 3, the selection of neighboring samples may be based on the size of at least one sub-block, the color component of at least one sub-block, or the boundary of at least one sub-block relative to the video The location of the data block. In some embodiments, the video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), slice header, coding tree unit (CTU) or coding unit (CU) is signaled in the video parameter set (VPS), Selection of neighbor samples.

The method 1400 includes, at step 1430, performing prediction of a block of video material using the filtered boundary samples. In some embodiments, the filtered boundary samples may be the result of finite impulse response (FIR) filtering or infinite impulse response (IIR) filtering. In some embodiments, and as described in the context of Example 7, filtering includes filtering the vertical boundary before the horizontal boundary, or vice versa. In some embodiments, filtering may be based on one or more attributes of the video data block or adjacent video data blocks (for example, motion vector, quantization parameter (QP), intra-frame prediction mode, inter-frame prediction direction, Merge mode, or advanced Motion Vector Prediction (AMVP) mode).

In some embodiments, and as described in the context of Example 4, a set of filter taps/coefficients may be based on the distance from the sample to the boundary of at least one sub-block, the size of at least one sub-block, or at least one sub-block. The color component of the block. In some embodiments, it can be signaled in video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), slice header, coding tree unit (CTU) or coding unit (CU) The set of filter taps/coefficients.

In some embodiments, and as described in the context of Example 11, when a sub-block is coded for bidirectional prediction, filtering includes filtering each intermediate prediction block corresponding to the reference picture list.

3. Example implementations of the disclosed technology

15 is a block diagram illustrating an example encoding apparatus 1500 that can be used to implement various parts of the currently disclosed technology, including but not limited to the method 1300 and the method 1400. The encoding device 1500 includes a quantizer 1505 for compressing input data bits. The encoding device 1500 also includes a dequantizer 1515 so that data bits can be fed to the memory 1525 and the predictor 1520 to perform motion estimation. The encoding device 1500 further includes a binary encoder 1530 to generate an encoded binary code.

Figure 16 is a block diagram illustrating an example decoding apparatus 1600 that can be used to implement various parts of the currently disclosed technology, including but not limited to method 1350. The decoding device 1600 includes a binary decoder 1605 for decoding binary codes. The decoding device 1600 further includes a dequantizer 1615 so that the decoded data bits can be fed to the memory 1625 and the predictor 1620 to perform motion estimation on the decoding side.

FIG. 17 is a block diagram illustrating an example of the architecture of a computer system or other control device 1700 that can be used to implement various parts of the currently disclosed technology (including but not limited to the method 1300 and the method 1400). In FIG. 17, a computer system 1700 includes one or more processors 1705 and a memory 1710 connected via an interconnection 1725. The interconnection 1725 may represent any one or more separate physical buses, point-to-point connections, or both, connected by a suitable bridge, adapter, or controller. Therefore, the interconnection 1725 may include, for example, a system bus, a peripheral component interconnect (Peripheral Component Interconnect, PCI) bus, a HyperTransport or an industry standard architecture (Industry Standard Architecture, ISA) bus, and a small computer system interface (Small Computer System Interface, SCSI) bus, Universal Serial Bus (USB), IIC (I2C) bus, or Institute of Electrical and Electronics Engineers (IEEE) standard 674 bus, sometimes also called It is "Firewire".

The processor(s) 1705 may include a central processing unit (CPU) that controls overall operations of, for example, the host. In some embodiments, the processor(s) 1705 executes software or firmware stored in the memory 1710 to achieve this purpose. The processor(s) 1705 may be or may include one or more programmable general-purpose or special-purpose microprocessors, digital signal processors (Digital Signal Processors, DSP), programmable controllers, special-purpose integrated circuits (Application Specific Integrated Circuits (ASIC), Programmable Logic Devices (PLD), etc., or a combination of these devices.

The memory 1710 may be or include the main memory of the computer system. The memory 1710 represents any suitable form of random access memory (Random Access Memory, RAM), read-only memory (Read-Only Memory, ROM), flash memory, etc., or a combination of these devices. In use, the memory 1710 may contain, among other things, a set of machine instructions, which when run by the processor 1705, cause the processor 1705 to perform operations to implement embodiments of the currently disclosed technology.

Also connected to the processor(s) 1705 via the interconnect 1725 is an (optional) network interface card 1715. The network interface card 1715 provides the computer system 1700 with the ability to communicate with remote devices (such as storage clients and/or other storage servers), and may be, for example, an Ethernet interface card or a fiber channel adapter.

FIG. 18 shows a block diagram of an example embodiment of a mobile device 1800 that can be used to implement various parts of the currently disclosed technology, including but not limited to method 1300 and method 1400. The mobile device 1800 may be a laptop computer, a smart phone, a tablet computer, a portable camera, or another type of device capable of processing video. The mobile device 1800 includes a processor or controller 1801 for processing data, and a memory 1802 for communicating with the processor 1801, storing and/or buffering data. For example, the processor 1801 may include a central processing unit (CPU) or a microcontroller unit (Microcontroller Unit, MCU). In some embodiments, the processor 1801 may include a Field-Programmable Gate-Array (FPGA). In some embodiments, the mobile device 1800 includes a graphics processing unit (GPU), a video processing unit (Video Processing Unit, VPU), and a graphics processing unit (GPU) for various visual and/or communication data processing functions of a smart phone device. / Or wireless communication unit or communicate with it. For example, the memory 1802 may include and store a processor executable code that configures the mobile device 1800 to perform various operations when run by the processor 1801, such as receiving information, commands and/or data, processing information and Data, and sending or providing processed information/data to another device, such as an actuator or an external display.

In order to support various functions of the mobile device 1800, the memory 1802 can store information and data, such as commands, software, values, images, and other data processed or referenced by the processor 1801. For example, various types of random access memory (RAM) devices, read-only memory (ROM) devices, flash memory devices, and other suitable storage media may be used to implement the storage function of the memory 1802. In some embodiments, the mobile device 1800 includes an input/output (Input/Output, I/O) unit 1803 to couple the processor 1801 and/or the memory 1802 to other modules, units, or devices. For example, using various types of wireless interfaces compatible with typical data communication standards, the I/O unit 1803 can interface with the processor 1801 and the memory 1802, for example, between one or more computers in the cloud and user equipment. between. In some embodiments, the mobile device 1800 may interface with other devices via the I/O unit 1803 using a wired connection. The mobile device 1800 can also interface with other external interfaces (such as a data storage device and/or a visual or audio display device 1804) for retrieval and transmission, which can be processed by the processor, stored in memory, or in the output unit of the display device 1804 or an external device. Data and information on display. For example, the display device 1804 may display a video frame including a block (CU, PU, or TU) for which in-frame block copy is applied based on whether a motion compensation algorithm is used to encode the block and according to the disclosed technology.

In some embodiments, the video decoder device may implement a sub-block-based prediction method for video decoding as described herein. The various features of the method can be similar to the method 1350 described above.

In some embodiments, the video decoding method may be implemented using a decoding device implemented on a hardware platform as described with respect to FIG. 17 and FIG. 18.

It will be appreciated from the foregoing that, this document has described specific embodiments of the currently disclosed technology for illustrative purposes, but various modifications can be made without departing from the scope of the present invention. Therefore, the currently disclosed technology is not limited except for the scope of the attached patent application.

The subject matter and functional operation described in this patent document can be implemented in various systems, digital electronic circuits, or computer software, firmware, or hardware, including the structure disclosed in this specification and its structural equivalents, or A combination of one or more of them. The implementation of the subject described in this specification can be implemented as one or more computer program products, that is, coded on a tangible and non-transitory computer-readable medium for operation by a data processing device or for controlling the operation of a data processing device One or more modules of computer program instructions. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of substances that affect a machine-readable propagation signal, or a combination of one or more of them. The term "data processing unit" or "data processing device" includes all devices, equipment, and machines used to process data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the device may also include code that creates an operating environment for the computer program in question, for example, constituting processor firmware, protocol stack, database management system, operating system, or a combination of one or more of them Code.

Computer programs (also called programs, software, software applications, scripts, or codes) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as stand-alone programs or as suitable for Modules, components, subroutines, or other units used in the computing environment. Computer programs do not necessarily correspond to documents in the file system. The program can be stored in a part of a document that holds other programs or data (for example, one or more scripts stored in a markup language document), in a single document dedicated to the program in question, or in multiple coordination In a document (for example, a document that stores one or more modules, subprograms, or parts of code). Computer programs can be deployed to run on one computer or on multiple computers located on one website or distributed across multiple websites and interconnected by a communication network.

The processes and logic flows described in this specification can be executed by one or more programmable processors running one or more computer programs to perform functions by operating on input data and generating output. The process and logic flow can also be executed by dedicated logic circuits (such as FPGA (Field Programmable Logic Gate Array) or ASIC (Dedicated Integrated Circuit)), and the device can also be implemented as a dedicated logic circuit.

For example, processors suitable for the operation of computer programs include both general-purpose and special-purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, the processor will receive commands and data from read-only memory or random access memory or both. The basic components of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include or be coupled to one or more large storage devices (such as magnetic disks, magneto-optical disks, or optical disks) used to store data to receive data from or transmit data to it, or both to receive data and transmit data . However, the computer does not have to have such equipment. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices. The processor and memory can be supplemented by or incorporated into a dedicated logic circuit.

The description and drawings are intended to be considered exemplary only, where exemplary means example. As used herein, unless the context clearly dictates otherwise, the singular forms "a", "an" and "the" are intended to also include the plural forms. Additionally, the use of "or" is intended to include "and/or" unless the context clearly dictates otherwise.

Although this patent document contains many details, these details should not be construed as limitations on the scope of any invention or claimable, but as descriptions of features specific to specific embodiments of specific inventions. Certain features described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable subcombination. In addition, although features may be described above as functioning in certain combinations and even initially claimed as such, in some cases one or more features from the claimed combination may be cut out from the combination, and the claimed The combination of protection can be directed to sub-combination or sub-combination changes.

Similarly, although operations are depicted in a specific order in the drawings, this should not be understood as requiring that these operations be performed in the specific order shown or sequentially in order, or that all the operations shown are performed to achieve the desired result . Furthermore, the separation of various system components in the embodiments described in this patent document should not be construed as requiring such separation in all embodiments.

Only some implementations and examples are described, and other implementations, enhancements, and changes can be made based on the content described and shown in this patent document.

100, 200, 300, 511, 512, 513, 514, 700, a, b, c, d‧‧‧ block 400, 500, 900‧‧‧Coding unit 401, 501, 502, 503, 504, C, D‧‧‧sub coding unit 450‧‧‧Reference picture/Motion source picture 451‧‧‧Corresponding block/sub coding unit 701‧‧‧Filled area 800‧‧‧Current coding unit/current picture 801, 802, MV0, MV1‧‧‧Motion vector 803, 804, TD0, TD1‧‧‧Time distance 810, 811, 910, Ref0, Ref1‧‧‧Reference picture 1300, 1350, 1400‧‧‧Method 1302, 1304, 1306, 1308, 1352, 1354, 1356‧‧‧Operation 1410, 1420, 1430‧‧‧Step 1500‧‧‧Encoding device 1505‧‧‧Quantizer 1515, 1615‧‧‧Dequantizer 1520, 1620‧‧‧ predictor 1525, 1625, 1710, 1802‧‧‧Memory 1530‧‧‧Binary encoder 1600‧‧‧Decoding device 1605‧‧‧Binary Decoder 1700‧‧‧Computer system 1705‧‧‧Processor 1715‧‧‧Network Interface Card 1725‧‧‧Interconnection 1800‧‧‧Mobile Device 1801‧‧‧Processor/Controller 1803‧‧‧Input/Output Unit 1804‧‧‧Display equipment A, B‧‧‧Sub coding unit/point p0, p1, q0, q1‧‧‧sample

FIG. 1 shows an example of prediction based on sub-block video processing. Figure 2 shows an example of a simplified affine motion model. Fig. 3 shows an example of the affine motion vector field (MVF) of each sub-block. FIG. 4 shows an example of motion prediction using an optional temporal motion vector prediction (ATMVP) algorithm for coding units (CU). FIG. 5 shows an example of coding units (CU) with sub-blocks and neighboring blocks used by the space-time motion vector prediction (STMVP) algorithm. Figure 6 shows an example of the optical flow trajectory used by the Bidirectional Optical Flow (BIO) algorithm. Figure 7A shows an example snapshot using the Bidirectional Optical Flow (BIO) algorithm without block extension. Figure 7B shows another example snapshot using the BIO algorithm without block extension. Figure 8 shows an example of bilateral matching in the frame rate up conversion (FRUC) algorithm. Figure 9 shows an example of template matching in the FRUC algorithm. FIG. 10A shows an example of the boundaries of sub-blocks that can be filtered according to one or more embodiments of the present technology. FIG. 10B shows another example of the boundaries of sub-blocks that can be filtered according to one or more embodiments of the present technology. FIG. 11A shows an example of prediction samples to be filtered according to one or more embodiments of the present technology. FIG. 11B shows another example of prediction samples to be filtered according to one or more embodiments of the present technology. Figure 12A shows an example of neighboring samples for filtering samples according to one or more embodiments of the present technology. Figure 12B shows another example of neighboring samples for filtering samples according to one or more embodiments of the present technology. Figure 12C shows yet another example of neighboring samples for filtering samples according to one or more embodiments of the present technology. FIG. 13A shows a flowchart of an example method for video encoding according to the disclosed technology. FIG. 13B shows a flowchart of an example method for video decoding according to the disclosed technology. Figure 14 shows a flowchart of an example method for video encoding according to the disclosed technology. 15 is a block diagram illustrating an example encoding device that can be used to implement various parts of the currently disclosed technology. Figure 16 is a block diagram illustrating an example encoding device that can be used to implement various parts of the currently disclosed technology. FIG. 17 is a block diagram illustrating an example of the architecture of a computer system or other control device that can be used to implement various parts of the currently disclosed technology. Figure 18 shows a block diagram of an example embodiment of a mobile device that can be used to implement various parts of the presently disclosed technology.

1300‧‧‧Method

1302, 1304, 1306, 1308‧‧‧Operation

Claims (33)

A video coding method includes: dividing a current video data block into a plurality of sub-blocks; generating a prediction sub-block for at least one sub-block of the plurality of sub-blocks; and predicting the at least one sub-block using a set of filter coefficients The boundary samples of the sub-block are filtered to generate filtered boundary samples; and the filtered boundary samples are used to determine the final prediction block of the current video material block, wherein the prediction is generated by performing bidirectional prediction using two references Sub-block, and wherein the filtering includes filtering at least one intermediate prediction block of the bidirectional prediction; or generating the predicted sub-block by performing multiple hypothesis prediction, and wherein the filtering includes performing multiple hypothesis prediction At least one predicted intermediate prediction block is filtered. A video decoding method includes: using a set of filter coefficients to filter boundary samples of a prediction sub-block to generate filtered boundary samples, wherein the predictor is generated for at least one sub-block of a plurality of sub-blocks of the current video block Block; using the filtered boundary samples to perform prediction of the current video block of the video material; and using the prediction to reconstruct the current video block, wherein the prediction sub-block is generated by performing bidirectional prediction using two references, And wherein, the filtering includes at least one intermediate prediction for the bidirectional prediction The measured block is filtered; or the prediction sub-block is generated by performing multi-hypothesis prediction, and wherein the filtering includes filtering at least one intermediate prediction block of the multi-hypothesis prediction. The method according to claim 1 or 2, wherein the filtering includes deblocking filtering, and wherein the at least one sub-block includes a non-zero or zero residual value. The method according to item 1 or 2 of the scope of patent application, wherein at least a first subset of the boundary samples are located along the inner boundary between two adjacent sub-blocks in the video material block, and wherein the The second subset of boundary samples is located along the outer boundary between one of the multiple sub-blocks of the video material block and the second video material block. The method according to item 1 or 2 of the scope of patent application, wherein the boundary sample includes a plurality of columns or rows of samples along the boundary. The method according to item 5 of the scope of patent application includes: determining the number of columns or rows of the boundary sample based on the size of at least one sub-block. The method described in item 5 of the scope of patent application includes: determining the number of columns or rows based on the color components of at least one sub-block. The method described in item 5 of the scope of patent application includes: determining the number of columns or rows based on whether the boundary sample is along the inner boundary or the outer boundary. The method described in item 5 of the scope of patent application includes: determining the number of columns or rows based on the position of the sub-block in the video data block. The method described in item 5 of the scope of patent application includes: based on video parameter set, sequence parameter set, picture parameter set, strip header, coding The information in the tree unit or coding unit determines the number of columns or rows. The method according to item 1 or 2 of the scope of patent application, wherein, filtering the boundary samples of the plurality of sub-blocks includes: applying the set of filter coefficients to adjacent samples adjacent to the boundary samples to perform the The samples are filtered. The method according to claim 11, wherein the adjacent samples and the boundary samples are located in the same column or the same row. The method according to item 11 of the scope of patent application, wherein the adjacent samples are in a two-dimensional area around the boundary samples. The method described in item 11 of the scope of patent application includes: selecting the neighboring samples based on the size of the corresponding sub-block. The method described in item 11 of the scope of patent application includes: selecting the adjacent samples based on the color components of the corresponding sub-blocks. The method described in item 11 of the scope of patent application includes: selecting the adjacent samples based on the positions of the boundary samples. The method described in item 11 of the scope of patent application includes: selecting the adjacent samples based on information in a video parameter set, a sequence parameter set, a picture parameter set, a slice header, a coding tree unit, or a coding unit. The method described in item 1 or 2 of the scope of the patent application, wherein the set of filter coefficients are the same for horizontal filtering and vertical filtering. The method described in item 1 or 2 of the scope of patent application, wherein the set of filter coefficients are the same for symmetric samples surrounding the boundary of the sub-block. The method described in item 1 or 2 of the scope of the patent application includes: determining the set of filter coefficients of the boundary sample based on the distance from the boundary sample to the corresponding boundary. The method described in item 1 or 2 of the scope of patent application includes: determining the set of filter coefficients based on the size of at least one sub-block. The method described in item 1 or 2 of the scope of patent application includes: determining the set of filter coefficients based on the color components of at least one sub-block. The method described in item 1 or 2 of the scope of patent application includes: determining the set of filter coefficients based on information in a video parameter set, a sequence parameter set, a picture parameter set, a slice header, a coding tree unit, or a coding unit. The method according to item 1 or 2 of the scope of the patent application, wherein the filtering includes finite impulse response filtering or infinite impulse response filtering. The method according to item 1 or 2 of the scope of patent application, wherein the filtering includes: for each of the multiple sub-blocks, filtering a set of boundary samples along the vertical boundary; and for each of the multiple sub-blocks For each sub-block, after filtering a set of boundary samples along the vertical boundary, the remaining boundary samples along the horizontal boundary are filtered, wherein the filtering of multiple sub-blocks is sequentially performed according to the sub-block scanning. The method according to item 1 or 2 of the scope of patent application, wherein the filtering includes: for each sub-block of a plurality of sub-blocks, a set of boundary samples along the horizontal boundary Filtering; and for each of the multiple sub-blocks, after filtering a set of boundary samples along the horizontal boundary, filtering the remaining boundary samples along the vertical boundary, wherein the multiple sub-block scans are sequentially performed Filtering of sub-blocks. The method according to item 1 or 2 of the scope of patent application, wherein the filtering includes: filtering a set of boundary samples along the vertical boundary of all the plurality of sub-blocks; and filtering a set of boundary samples along the vertical boundary After the samples are filtered, the remaining boundary samples along the horizontal boundaries of all multiple sub-blocks are filtered. The method according to item 1 or 2 of the scope of patent application, wherein the filtering includes: filtering a set of boundary samples along the horizontal boundary of all the plurality of sub-blocks; and filtering a set of boundary samples along the horizontal boundary After the samples are filtered, the remaining boundary samples along the vertical boundaries of all multiple sub-blocks are filtered. The method according to item 1 or 2 of the scope of patent application, wherein the filtering is based on one or more attributes of the video data block or adjacent video data blocks, wherein the one or more attributes include a motion vector, a quantization parameter , Intra-frame prediction mode, inter-frame prediction direction, Merge mode or advanced motion vector prediction mode. The method according to claim 1 or 2, wherein the filtering is adaptively enabled or disabled for each of the plurality of sub-blocks. A video encoding device includes a processor configured to implement the method according to any one of items 1 and 3 to 30 in the scope of patent application. A video decoding device includes a processor configured to implement the method according to any one of the 2nd to 30th patent applications. A computer program product stored on a non-transitory computer readable medium. The computer program product includes a program code for executing the method described in any one of items 1 to 30 in the scope of patent application.

Images