TW202013960A - Shape dependent interpolation order - Google Patents

Abstract

The application provides A video processing method, comprising: determining a first prediction mode applied to a first video block; performing a first conversion between the first video block and a coded representation of the first video block by applying a horizontal interpolation and/or a vertical interpolation to the first video block; determining a second prediction mode applied to a second video block; performing a second conversion between the second video block and a coded representation of the second video block by applying a horizontal interpolation and/or a vertical interpolation to the second video block, wherein, based on the determination that the first prediction mode is a multi-hypothesis prediction mode and the second prediction mode is not a multi-hypothesis prediction mode, one or both of the horizontal interpolation and the vertical interpolation for first video block use a shorter tap filter compared to that used for the second video block.

Description

Interpolation order depending on shape

The document of the present invention relates to video coding technology, equipment and system. [Cross reference to related applications] According to the provisions of the applicable Patent Law and/or the Paris Convention, the present invention promptly requires the priority and benefits of the international patent application number PCT/CN2018/095576 filed on July 13, 2018. The entire disclosure of International Patent Application No. PCT/CN2018/095576 is incorporated herein by reference as part of the disclosure of the present invention.

Despite the advances in video compression, digital video still has the largest bandwidth on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the demand for digital video bandwidth will continue to grow.

The disclosed techniques may be used by video decoder or encoder embodiments, where block-shaped interpolation sequential techniques are used to improve interpolation.

In an example aspect, a video bit stream processing method is disclosed. The method includes: determining the shape of the video block; determining an interpolation order based on the shape of the video block, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation; and performing horizontal interpolation and vertical interpolation of the video block in the sequence indicated by the interpolation order, to Reconstruct the decoded representation of the video block.

In another example aspect, a video bit stream processing method includes: determining characteristics of a motion vector related to a video block; determining an interpolation order based on the characteristics of the motion vector, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation; and interpolation by The sequence indicated by the order performs horizontal interpolation and vertical interpolation on the video block to reconstruct the decoded representation of the video block.

In another example aspect, a video bit stream processing method is disclosed. The method includes: determining the shape of the video block; determining an interpolation order based on the shape of the video block, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation; and performing horizontal interpolation and vertical interpolation of the video block in the sequence indicated by the interpolation order, to Construct a coded representation of the video block.

In another example aspect, a video bit stream processing method is disclosed. The method includes: determining characteristics of a motion vector related to a video block; determining an interpolation order based on the characteristics of the motion vector, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation; and performing horizontal interpolation on the video block in the sequence indicated by the interpolation order And vertical interpolation to construct a coded representation of the video block.

In an example aspect, a video processing method is disclosed. The method includes: determining a first prediction mode applied to the first video block; performing a first between the first video block and the encoded representation of the first video block by applying horizontal interpolation and/or vertical interpolation to the first video block Conversion, determining the second prediction mode applied to the second video block; by applying horizontal interpolation and/or vertical interpolation to the second video block, performing a second conversion between the second video block and the encoded representation of the second video block, Among them, based on the determination that the first prediction mode is a multi-hypothesis prediction mode and the second prediction mode is not a multi-hypothesis prediction mode, one or both of horizontal interpolation and vertical interpolation of the first video block are used as A filter with a shorter tap compared to the filter.

In another example aspect, a video decoding device implementing the video processing method described herein is disclosed.

In yet another example aspect, a video encoding device implementing the video processing method described herein is disclosed.

In yet another typical aspect, the various technologies described herein can be implemented as computer program products stored on a non-transitory computer-readable medium. The computer program product includes code for performing the methods described herein.

In yet another example aspect, an apparatus in a video system is disclosed. The device includes a processor and a non-transitory memory having instructions thereon, wherein the instructions executed by the processor cause the processor to implement the above method.

The details of one or more implementations are set forth in the annex, drawings, and the following description. Other features will be apparent from the description and drawings, and the scope of patent application.

This document provides various techniques that can be used by decoders of video bitstreams to improve the quality of decompressed or decoded digital video. In addition, the video encoder can also implement these techniques during the encoding process in order to reconstruct the decoded frames for further encoding.

For ease of understanding, chapter titles are used in this document, and the embodiments and techniques are not limited to the corresponding parts. In this way, embodiments from one chapter can be combined with embodiments from other chapters.

1. Summary

The invention relates to video coding technology. Specifically, it relates to interpolation in video coding. It can be applied to existing video coding standards, such as HEVC, or the standard to be finalized (multifunctional video coding). It may also be suitable for future video coding standards or video encoders.

2. Background

Video coding standards are mainly developed by developing well-known ITU-T and ISO/IEC standards. ITU-T developed H.261 and H.263, ISO/IEC developed MPEG-1 and MPEG-4 vision, and the two organizations jointly developed H.262/MPEG-2 video, H.264/MPEG-4 Advanced Video Coding (AVC) and H.265/HEVC standards. Since H.262, the video coding standard is based on a hybrid video coding structure, in which time domain prediction plus transform coding is used. In order to explore future video coding technologies beyond HEVC, the Joint Video Exploration Team (JVET) was jointly established by VCEG and MPEG in 2015. Since then, JVET has adopted many new methods and introduced it into a reference software called Joint Exploration Model (JEM). In April 2018, the Joint Video Experts Group (JVET) between VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11 (MPEG) was created to study the VVC standard with a goal of 50% reduction compared to HEVC Bit rate.

Figure 18 is a block diagram of an example implementation of a video encoder.

2.1 Quadtree plus Binary Tree (QTBT) block structure with larger CTU

In HEVC, various local characteristics are adapted by dividing the CTU into CUs by using a quadtree structure (represented as a coding tree). It is decided at the CU level whether to use inter (temporal) prediction or intra (spatial) prediction to encode the picture area. Each CU can be further divided into one, two, or four PUs according to the type of PU partitioning. In a PU, the same prediction process is applied, and related information is transmitted to the decoder based on the PU. After the residual block is obtained by applying prediction processing based on the PU partition type, the CU can be partitioned into transform units (TUs) according to another quadtree structure similar to the CU's coding tree. An important feature of the HEVC structure is that it has multiple partitioning concepts, including CU, PU, and TU.

The QTBT structure eliminates the concept of multiple partition types, that is, the QTBT structure eliminates the separation of the concepts of CU, PU, and TU, and supports more flexibility in the shape of the CU partition. In the QTBT block structure, the CU may be square or rectangular. As shown in Figure 1, the coding tree unit (CTU) is first partitioned with a quadtree structure. The quad leaf nodes are further divided by a binary tree structure. There are two types of partitioning in binary tree partitioning: symmetric horizontal partitioning and symmetric vertical partitioning. The binary leaf node is called a coding unit (CU), and this division is used for prediction and conversion processing without further segmentation. This means that CU, PU and TU have the same block size in the QTBT coding block structure. In JEM, the CU is sometimes composed of coding blocks (CB) of different color components. For example, in the P and B slices in the 4:2:0 chroma format, one CU contains one luminance CB and two colors Degree CB, and CU is sometimes composed of a single component CB, for example, in the case of I-slice, one CU contains only one luminance CB or only two chroma CB.

The following parameters are defined for the QTBT segmentation scheme.

– CTU size: The size of the root node of the quadtree, which is the same as the concept in HEVC.

– MiNQTSize : The minimum allowed size of the quad leaf node

– MaxBTSize : the maximum allowed size of the root node of the binary tree

– MaxBTDePTh : maximum allowable depth of binary tree

– MiNBTSize : the minimum allowable binary leaf node size

In an example of the QTBT segmentation structure, the CTU size is set to 128×128 luma samples with two corresponding 64×64 chroma sample blocks, MiNQTSize is set to 16×16, and MaxBTSize is set to 64× 64, MiNBTSize (width and height) is set to 4×4, and MaxBTSize is set to 4. Quadtree segmentation is first applied to CTU to generate quadtree leaf nodes. The size of the quad leaf node may have a size from 16×16 (ie, MiNQTSize ) to 128×128 (ie, CTU size). If the leaf quadtree node is 128×128, it will not be further divided by the binary tree because its size exceeds MaxBTSize (for example, 64×64). Otherwise, the leaf quadtree nodes can be further divided by the binary tree. Therefore, the quad leaf node is also the root node of the binary tree, and the depth of the binary tree is 0. When the depth of the binary tree reaches MaxBTDePTh (ie, 4), no further division is considered. When the width of the binary tree node is equal to MiNBTSize (ie, 4), no further horizontal division is considered. Similarly, when the height of the binary tree node is equal to MiNBTSize , no further vertical division is considered. The leaf nodes of the binary tree are further processed through prediction and transformation processing without further segmentation. In JEM, the maximum CTU size is 256×256 luminance samples.

Fig. 1 (left side) illustrates an example of block division by using QTBT, and Fig. 1 (right side) illustrates a corresponding tree representation. The solid line indicates a quadtree partition, and the broken line indicates a binary tree partition. In each division (ie, non-leaf) node of the binary tree, a flag is signaled to indicate which type of division is used (ie, horizontal or vertical), where 0 represents horizontal division and 1 represents vertical division. For quadtree partitioning, there is no need to specify the partitioning type, because quadtree partitioning always divides a block horizontally and vertically to generate 4 sub-blocks of the same size.

In addition, the QTBT solution supports the ability to have separate QTBT structures for brightness and saturation. At present, for P-band and B-band, the brightness and saturation CTB in a CTU share the same QTBT structure. However, for the I band, the QTBT structure is used to divide the luminance CTB into CUs, and the other QTBT structure is used to divide the chroma CTB into chroma CUs. This means that the CU in the I slice is composed of coding blocks of the luma component or two chroma components, and the CU in the P slice or B slice is composed of coding blocks of all three color components.

In HEVC, in order to reduce memory access for motion compensation, inter prediction of small blocks is restricted so that 4×8 and 8×4 blocks do not support bidirectional prediction, and 4×4 blocks do not support inter prediction. In JEM's QTBT, these restrictions have been removed.

2.2 Inter prediction in HEVC/H.265

Each inter-predicted PU has motion parameters of one or two reference picture lists. Motion parameters include motion vectors and reference picture indexes. The use of one of the two reference picture lists can also be signaled using inter_pred_idc . Motion vectors can be explicitly encoded as increments relative to the predicted value.

When the CU is coded in skip mode, one PU is associated with the CU, and there are no significant residual coefficients, no coded motion vector increments, or reference picture indexes. A Merge mode is specified, through which the motion parameters of the current PU can be obtained from neighboring PUs (including spatial and temporal candidates). Merge mode can be applied to any inter-predicted PU, not just skip mode. Another option for the Merge mode is the explicit transmission of motion parameters, where the motion vector (more precisely, the motion vector difference compared to the motion vector predictor), the reference picture index and reference picture list corresponding to each reference picture list Will be explicitly signaled in each PU. In this document, this mode is called Advanced Motion Vector Prediction (AMVP).

When the signal indicates that one of the two reference picture lists is to be used, the PU is generated from one sample block. This is called "one-way prediction". One-way prediction is available for both P and B bands.

When the signal indicates that two reference picture lists are to be used, the PU is generated from the two sample blocks. This is called "bidirectional prediction". Bidirectional prediction is only available for B bands.

The text below provides details of the inter prediction modes specified in HEVC. The description will start in Merge mode.

2.2.1 Merge mode

2.2.1.1 Derivation of candidate in Merge mode

When using the Merge mode to predict the PU, the index pointing to the entry in the Merge candidate list is analyzed from the bitstream and used to retrieve motion information. The structure of this list is specified in the HEVC standard and can be summarized in the following order of steps:

Step 1: Derivation of initial candidates

Step 1.1: Derivation of airspace candidates

Step 1.2: Redundancy check of airspace candidates

Step 1.3: Time-domain candidate derivation

Step 2: Additional candidate insertion

Step 2.1: Creation of bidirectional prediction candidates

Step 2.2: Insertion of zero motion candidates

These steps are also schematically depicted in Figure 2. For the spatial Merge candidate derivation, a maximum of four Merge candidates are selected among the candidates located in five different positions. For the time-domain Merge candidate derivation, at most one Merge candidate is selected from the two candidates. Since the number of candidates for each PU is assumed to be constant at the decoder, when the number of candidates obtained from step 1 does not reach the maximum number of Merge candidates ( MaxNumMergeCand ) signaled in the stripe header , additional candidates are generated. Since the number of candidates is constant, the index of the best Merge candidate is encoded using truncated unary binarization (TU). If the size of the CU is equal to 8, all PUs of the current CU share a Merge candidate list, which is the same as the Merge candidate list of the 2N×2N prediction unit.

The operations related to the above steps are described in detail below.

2.2.1.2 Derivation of airspace candidates

In the derivation of spatial Merge candidates, up to four Merge candidates are selected among the candidates located in the position shown in FIG. 3. The derivation order is A1, B1, B0, A0 and B2. Position B2 is only considered when any PU at positions A1, B1, B0, A0 is not available (for example, because it belongs to another slice or slice) or is internally coded. After adding candidates at the A1 position, a redundancy check is performed on the addition of the remaining candidates, which ensures that candidates with the same motion information are excluded from the list, thereby improving coding efficiency. In order to reduce the computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check. On the contrary, only the pairs connected with the arrows in FIG. 4 are considered, and the candidates are added to the list only when the corresponding candidates for redundancy check do not have the same motion information. Another source for copying motion information is the "second PU" associated with 2N×2N different partitions. For example, FIG. 5 depicts the second PU in the case of N×2N and 2N×N, respectively. When the current PU is divided into N×2N, candidates for the A1 position are not considered for list construction. In some embodiments, adding this candidate may result in two prediction units with the same motion information, which is redundant for having only one PU in the coding unit. Likewise, when the current PU is divided into 2N×N, the position B1 is not considered.

2.2.1.3 Time domain candidate derivation

In this step, only one candidate is added to the list. In particular, in the derivation of this time-domain Merge candidate, the scaled motion vector is derived based on the juxtaposed PU with the smallest picture order count POC difference from the current picture in the given reference picture list. The reference picture list used to derive the collocated PU is explicitly signaled in the stripe header. The dotted line in FIG. 6 shows the acquisition of the scaled motion vector of the time domain Merge candidate, which uses the POC distances tb and td to scale from the motion vector of the juxtaposed PU, where tb is defined as the reference picture between the current picture and the current picture. POC difference, and td is defined as the POC difference between the reference picture of the juxtaposed picture and the juxtaposed picture. The reference picture index of the time domain Merge candidate is set to zero. The actual implementation of scaling processing is described in the HEVC specification. For the B slice, two motion vectors (one for reference picture list 0 and the other for reference picture list 1) are obtained and combined to make it a bidirectional prediction Merge candidate.

6 is an illustration of motion vector scaling for time-domain Merge candidates.

In the juxtaposed PU (Y) belonging to the reference frame, the position of the time domain candidate is selected between the candidates C ₀ and C ₁ , as shown in FIG. 7. If the PU at position C ₀ is unavailable, internally coded, or outside the current CTU line, then position C ₁ is used. Otherwise, the position C ₀ is used for the derivation of time-domain Merge candidates.

2.2.1.4 Additional candidate insertion

In addition to spatial and temporal Merge candidates, there are two additional types of Merge candidates: combined bidirectional prediction Merge candidates and zero Merge candidates. Combined bidirectional prediction Merge candidates are generated using spatial and temporal Merge candidates. The combined bidirectional prediction Merge candidate is only used for B bands. By combining the initial candidate first reference picture list motion parameter and another candidate's second reference picture list motion parameter, a combined bidirectional prediction candidate is generated. If these two tuples provide different motion hypotheses, they will form new bidirectional prediction candidates. As an example, FIG. 8 shows a case where two candidates in the original list (on the left) are used to create a combined bidirectional prediction Merge candidate added to the final list (on the right), which has MvL0 and refIdxL0 or MvL1 and refIdxL1 Of the two candidates. In the prior art, many rules regarding combinations need to be considered to generate these additional Merge candidates.

Insert zero motion candidates to fill the remaining entries in the Merge candidate list, thus reaching the capacity of MaxNumMergeCand. These candidates have zero spatial displacement and a reference picture index that starts from zero and increases each time a new zero motion candidate is added to the list. The number of reference frames used by these candidates is 1 frame and 2 frames for unidirectional prediction and bidirectional prediction, respectively. Finally, no redundancy check is performed on these candidates.

2.2.1.5 Parallel processing motion estimation area

In order to speed up the encoding process, motion estimation can be performed concurrently, thereby deriving the motion vectors of all prediction units in a given area simultaneously. Derivation of Merge candidates from spatial neighborhoods may interfere with parallel processing because a prediction unit cannot derive motion parameters from neighboring PUs before completing related motion estimation. In order to ease the balance between coding efficiency and processing delay, HEVC defines a motion estimation area (MER). The syntax element "log2_parallel_merge_level_minus2" can be used to signal the size of the MER in the picture parameter set. When MER is defined, Merge candidates that fall into the same area are marked as unavailable, and therefore are not considered in the list construction.

2.2.2 AMVP

AMVP utilizes the space-time correlation of motion vectors with neighboring PUs, which is used for explicit transmission of motion parameters. For each reference picture list, a motion vector candidate list is first constructed by checking the availability of PU positions adjacent to the time domain in the upper left, removing redundant candidate positions, and adding a zero vector to make the candidate list length constant. The encoder can then select the best predicted value from the candidate list and send the corresponding index indicating the selected candidate. Similar to the Merge index signal, the index of the best motion vector candidate is encoded using a truncated unary. In this case, the maximum value to be encoded is 2 (refer to FIG. 9). In the following chapters, the derivation process of motion vector prediction candidates will be introduced in detail.

2.2.2.1 Derivation of AMVP candidates

Fig. 9 summarizes the derivation process of motion vector prediction candidates.

In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and time-domain motion vector candidates. For the derivation of spatial motion vector candidates, two motion vector candidates are finally derived based on the motion vectors of each PU located at five different positions shown in FIG. 3.

For the derivation of time-domain motion vector candidates, one motion vector candidate is selected from the two candidates, which are derived based on two different juxtaposed positions. After making the first space-time candidate list, the repeated motion vector candidates in the list are removed. If the number of potential candidates is greater than two, the motion vector candidates whose reference picture index is greater than 1 in the associated reference picture list are removed from the list. If the number of space-time motion vector candidates is less than two, additional zero motion vector candidates will be added to the list.

2.2.2.2 Spatial motion vector candidates

When deriving spatial motion vector candidates, at most two candidates are considered among the five potential candidates, and these five candidates are from the PU at the positions depicted in FIG. 3, and these positions are the same as the positions of the motion Merge. The derivation order on the left side of the current PU is defined as A ₀ , A ₁ , and scaled A ₀ , scaled A ₁ . The derivation order on the current PU is defined as B ₀ , B ₁ , B ₂ , scaled B ₀ , scaled B ₁ , scaled B ₂ . Therefore, there are four cases on each side that can be used as motion vector candidates, of which two cases do not require the use of spatial scaling, and two cases use spatial scaling. The four different situations are summarized as follows:

- No space zoom

(1) The same reference picture list and the same reference picture index (same POC)

(2) Different reference picture lists, but the same reference picture index (same POC)

--Space zoom

(3) The same reference picture list, but different reference picture indexes (different POC)

(4) Different reference picture lists and different reference picture indexes (different POC)

First check the situation without spatial scaling, then check the spatial scaling. When the POC is different between the reference picture of the neighboring PU and the reference picture of the current PU, spatial scaling will be considered instead of the reference picture list. If all PUs of the left candidate are unavailable or intra-coded, the above motion vectors are allowed to be scaled to help parallel derivation of the left and upper MV candidates. Otherwise, spatial scaling of the above motion vectors is not allowed.

FIG. 10 is an illustration of motion vector scaling of spatial motion vector candidates.

In the spatial scaling process, the motion vectors of neighboring PUs are scaled in a similar manner to time-domain scaling, as shown in FIG. The main difference is that the reference picture list and index of the current PU are given as input, and the actual scaling process is the same as the time domain scaling process.

2.2.2.3 Time domain motion vector candidates

Except for the derivation of the reference picture index, all the derivation processes of the time-domain Merge candidate are the same as the derivation process of the spatial motion vector candidate (see FIG. 7). The decoder is signaled to refer to the picture index.

2.3 New interframe Merge candidate in JEM

2.3.1 Motion vector prediction based on sub-CU

In JEM with QTBT, each CU can have at most one set of motion parameters for each prediction direction. By dividing a large CU into sub-CUs and deriving the motion information of all sub-CUs of the large CU, two sub-CU-level motion vector prediction methods are considered in the encoder. The optional time domain motion vector prediction (ATMVP) method allows each CU to obtain multiple sets of motion information from multiple blocks smaller than the current CU in the collocated reference picture. In the space-time motion vector prediction (STMVP) method, the motion vector of the sub-CU is recursively derived by using the time-domain motion vector prediction value and the spatially adjacent motion vector.

In order to maintain a more accurate motion field for sub-CU motion prediction, motion compression of reference frames is currently disabled.

2.3.1.1 Optional time-domain motion vector prediction

In the optional time domain motion vector prediction (ATMVP) method, motion vector time domain motion vector prediction (TMVP) is modified by extracting multiple sets of motion information (including motion vectors and reference indexes) from a block smaller than the current CU. As shown in FIG. 11, the sub-CU is a square N×N block (the default N is set to 4).

ATMVP predicts the motion vector of the sub-CU in the CU in two steps. The first step is to identify the corresponding blocks in the reference picture with so-called time-domain vectors. The reference picture is called the motion source picture. The second step is to divide the current CU into sub-CUs, and obtain the motion vector and the reference index of each sub-CU from the block corresponding to each sub-CU, as shown in FIG. 11.

In the first step, the reference picture and the corresponding block are determined by the motion information of the spatial neighboring block of the current CU. In order to avoid repeated scanning processing of adjacent blocks, the first Merge candidate in the Merge candidate list of the current CU is used. The first available motion vector and its associated reference index are set to the time domain vector and the index of the motion source picture. In this way, in ATMVP, compared with TMVP, the corresponding block can be more accurately identified, where the corresponding block (sometimes called a juxtaposed block) is always located in the lower right corner or center position relative to the current CU.

In the second step, by adding the time domain vector to the coordinates of the current CU, the corresponding block of the sub-CU is identified by the time domain vector in the motion source picture. For each sub-CU, the motion information of the corresponding block (the smallest motion grid covering the center sample) is used to derive the motion information of the sub-CU. After identifying the motion information corresponding to the N×N block, it is converted into the motion vector and reference index of the current sub-CU, which is the same as the HEVC TMVP method, in which motion scaling and other processing are applied. For example, the decoder checks whether the low-latency condition is met (for example, the POC of all reference pictures of the current picture is less than the POC of the current picture), and may use the motion vector MVx (the motion vector corresponding to the reference picture list X) for each sub The CU predicts the motion vector MVy (X is equal to 0 or 1 and Y is equal to 1-X).

2.3.1.2 Space-time motion vector prediction

In this method, the motion vector of the sub-CU is derived recursively in the order of raster scan. Figure 12 illustrates this concept. Let us consider an 8×8 CU, which contains four 4×4 sub-CUs A, B, C and D. The adjacent 4×4 blocks in the current frame are labeled a, b, c, and d.

The motion derivation of sub-CU A starts by identifying its two spatial neighbors. The first neighbor is the N×N block (block c) above sub-CU A. If this block c is unavailable or intra-coded, check the other N×N blocks above sub-CU A (from left to right, starting at block c). The second neighbor is a block to the left of sub-CU A (block b). If block b is unavailable or internally coded, check the other blocks on the left side of sub-CU A (from top to bottom, starting at block b). The motion information obtained from neighboring blocks in each list is scaled to the first reference frame of the given list. Next, the time domain motion vector prediction (TMVP) of sub-block A is derived according to the same procedure as that of TMVP specified in HEVC. Extract the motion information of the juxtaposed block at position D and zoom accordingly. Finally, after retrieving and scaling motion information, all available motion vectors (up to 3) are averaged for each reference list. The average motion vector is specified as the motion vector of the current sub-CU.

2.3.1.3 Sub-CU motion prediction mode signal

The sub-CU mode is enabled as an additional merge candidate, and no additional syntax elements are required to signal the mode. Two additional merge candidates are added to the merge candidate list of each CU to represent the ATMVP mode and the STMVP mode. If the sequence parameter set indicates that ATMVP and STMVP are enabled, up to seven merge candidates are used. The coding logic of the extra merge candidate is the same as the merge candidate in HM, which means that for each CU in the P or B slice, two more RD checks are required for the two additional merge candidates.

In JEM, CABAC context encodes all binary bits indexed by merge. In HEVC, only the first binary bit is context coded, and the remaining binary bit context is bypass coded.

2.3.2 Non-adjacent Merge candidates

In J0021, Qualcomm proposed to derive additional spatial Merge candidates from non-adjacent adjacent positions marked as 6 to 49 as in FIG. 13. The derived candidate is added after the TMVP candidate in the Merge candidate list.

In J0058, Tencent proposed to have an offset from the current block (-96,- 96) The location in the external reference area derives additional spatial Merge candidates.

As shown in FIG. 14, the position markers are A(i, j), B(i, j), C(i, j), D(i, j), and E(i, j). Compared to its previous B or C candidate, each candidate B(i, j) or C(i, j) has an offset of 16 in the vertical direction. Compared with its previous A or D candidate, each candidate A(i, j) or D(i, j) has an offset of 16 in the horizontal direction. Compared to its previous E candidate, each E(i, j) has an offset of 16 in the horizontal and vertical directions. Check candidates from inside to outside. And the order of candidates is A(i, j), B(i, j), C(i, j), D(i, j) and E(i, j). Further study whether the number of merge candidates can be further reduced. The candidate is added after the TMVP candidate in the merge candidate list.

In J0059, the extended spatial positions from 6 to 27 in FIG. 15 are checked according to their digital order after the time domain candidates. In order to save the MV line buffer, all spatial candidates are limited to two CTU lines.

2.4 Intra prediction in JEM

2.4.1 Intra mode coding with 67 intra prediction modes

For luminance interpolation filtering, an 8-tap separable DCT-based interpolation filter is used for 2/4 precision samples, and a 7-tap separable DCT-based interpolation filter is used for 1/4 precision samples, as shown in Table 1.

table 1 : For 1/4 Luminance interpolated 8 Tap DCT-IF coefficient.

Similarly, a 4-tap separable DCT-based interpolation filter is used for the chroma interpolation filter, as shown in Table 2.

table 2 : For 1/8 Chroma interpolation 4 Tap DCT-IF coefficient.

For 4:2:2 vertical interpolation and 4:4:4 chroma channel horizontal and vertical interpolation, the odd positions in Table 2 are not used, resulting in 1/4 chroma interpolation.

For bidirectional prediction, before averaging the two prediction signals, the bit depth of the output of the interpolation filter is maintained at 14-bit accuracy regardless of the source bit depth. The actual averaging process is done implicitly through the bit depth reduction process:

predSamples[x, y] = predSamplesL0[x, y] + predSamplesL1[x, y] + offset)>>shift

Where shift = (15 – BitDepth) and offset = 1 >> (shift – 1)

If both the horizontal and vertical components of the motion vector point to sub-pixel positions, then horizontal interpolation is always performed first, followed by vertical interpolation. For example, in order to interpolate the sub-pixel j0,0 shown in FIG. 16, first, interpolate b0,k (k = -3,-2,...3) according to equation 2-1, and then according to the equation 2-2 Interpolate j0,0. Here, shift1 = Min(4, BitDepthY-8), and shift2 = 6, where BitDepthY is the bit depth of the video block, and more specifically, the bit depth of the luminance component of the video block.

b0,k = (-A-3,k + 4 * A-2,k-11 * A – 1,k + 40 * A0,k + 40 * A1,k-11 * A2,k + 4 * A3, k-A4,k) >> shift1 (2-1)

j0,0 = (-b0,-3 + 4 * b0,-2-11 * b0,-1 + 40 * b0,0 + 40 * b0,1-11 * b0,2 + 4 * b0,3-b0 ,4) >> shift2 (2-2)

Alternatively, we can perform vertical interpolation first and then horizontal interpolation. In this case, in order to interpolate j0,0, first, interpolate hk,0 (k = -3,-2,...3) according to equation 2-3, and then according to equation 2-4 Interpolate j0,0. When BitDepthY is less than or equal to 8, shift1 is 0, and there is no loss in the first interpolation stage. Therefore, the final interpolation result will not be changed by the interpolation order. However, when BitDepthY is greater than 8, shift1 is greater than 0. In this case, when different interpolation orders are applied, the final interpolation result may be different.

hk,0 = (-Ak,-3 + 4 * Ak,-2-11 * Ak, -1 + 40 * Ak,0 + 40 * Ak,1-11 * Ak,2 + 4 * Ak,3 – Ak , 4)>> shift1 (2-3)

j0,0 = (-h-3,0 + 4 * h-2,0-11 * h-1,0 + 40 * h0,0 + 40 * h1,0-11 * h2,0 + 4 * h3, 0-h4,0 )>> shift2 (2-4)

3. Examples of problems solved by the embodiment

For the luminance block size WxH, if we always perform horizontal interpolation first, the required interpolation (per pixel) is shown in Table 3.

Table 3: HEVC/JEM interpolation for WxH luminance component

On the other hand, if we perform vertical interpolation first, the required interpolation is shown in Table 4. Obviously, the best interpolation order is the one that requires a smaller number of interpolations between Table 3 and Table 4.

Table 4: Interpolation required for WxH luminance component when the interpolation order is reversed

For the chroma component, if we always perform horizontal interpolation first, the desired interpolation value is ((H + 3) x W + W x H) / (W x H) = 2 + 3 / H. If we always perform vertical interpolation first, the required interpolation value is ((W + 3) x H + W x H) / (W x H) = 2 + 3 / W.

As described above, when the bit depth of the input video is greater than 8, different interpolation orders may result in different interpolation results. Therefore, the interpolation order should be implicitly defined in the encoder and decoder.

4. Examples of embodiments

To solve these problems and provide other benefits, we propose shape-dependent interpolation sequences.

The following detailed examples should be considered as examples explaining general concepts. These inventions should not be interpreted in a narrow manner. Furthermore, these inventions can be combined in any way.

1. It is proposed that the interpolation order depends on the current coding block shape (for example, the coding block is a CU).

a. In one example, for blocks with width> height (such as CU, PU, or sub-blocks used in sub-block-based prediction (such as affine, ATMVP, or BIO)), first perform vertical interpolation, and then perform horizontal interpolation For example, first, pixels d _k,0 , h _k,0, and n _k,0 are interpolated, and then e _0,0 to r _0,0 are interpolated. Examples of j _0,0 are shown in equations 2-3 and 2-4.

i. Alternatively, for blocks with width >= height (such as CU, PU, or sub-blocks used in sub-block-based prediction (such as affine, ATMVP, or BIO)), perform vertical interpolation first, and then perform horizontal interpolation .

b. In one example, for blocks with width >= height (such as CU, PU, or sub-blocks used in sub-block-based prediction (such as affine, ATMVP, or BIO)), perform horizontal interpolation first, and then perform vertical Interpolation.

i. Alternatively, for blocks with width>height (such as CU, PU, or sub-blocks used in sub-block-based prediction (such as affine, ATMVP, or BIO)), perform horizontal interpolation first, and then perform vertical interpolation.

c. In one example, both the luma and chroma components follow the same interpolation order.

d. Alternatively, when one chroma encoding block corresponds to multiple luma encoding blocks (for example, for a color format of 4:2:0, one chroma 4×4 block may correspond to two 8×4 or 4 ×8 brightness block), brightness and saturation can use different interpolation order.

e. In one example, when different interpolation orders are utilized, the scaling factors in multiple stages (ie, shift1 and shift2) can be further changed accordingly.

2. Alternatively, in addition, it is proposed that the interpolation order of the luminance components may also depend on the MV.

a. In one example, if the vertical MV component points to a quarter-pixel position and the horizontal MV component points to a half-pixel position, then horizontal interpolation is performed first, followed by vertical interpolation.

b. In one example, if the vertical MV component points to a half-pixel position and the horizontal MV component points to a quarter-pixel position, vertical interpolation is performed first, and then horizontal interpolation is performed.

c. In one example, the proposed method only applies to square coding blocks.

3. The proposed method can be applied to certain modes, block sizes/shapes and/or certain sub-block sizes.

a. The proposed method can be applied to certain modes, such as bidirectional prediction mode.

b. The proposed method can be applied to certain block sizes.

i. In one example, it applies only to blocks with w×h >= T1, where w and h are the width and height of the current block, and T1 is the first threshold, which may be a predefined value depending on design requirements , Such as 16, 32 or 64.

ii. In one example, it applies only to blocks where h >= T2, and T2 is the second threshold, which may be a predefined value, such as 4 or 8, depending on design requirements.

c. The proposed method can be applied to certain color components (such as only the luminance component). 4. It is proposed that when multi-hypothesis prediction is applied to a block, it is possible to apply short taps or different interpolation filters compared to those applied to ordinary prediction modes.

a. In one example, a bilinear filter can be used.

b. The short-tap or second interpolation filter can be applied to a reference picture list involving multiple reference blocks, and for another reference picture having only one reference block, the same filtering as the filter used for the normal prediction mode can be applied Device.

c. The proposed method can be applied under certain conditions, such as certain time domain layer(s) containing the block, block/slice/strip/picture quantization parameters are within the range (such as greater than the threshold) .

FIG. 17 is a block diagram of the video processing device 1700. The apparatus 1700 may be used to implement one or more methods described herein. The device 1700 may be embedded in a smart phone, tablet computer, computer, Internet of Things (IoT) receiver, and so on. The apparatus 1700 may include one or more processors 1702, one or more storages 1704, and video processing hardware 1706. The processor(s) 1702 may be configured to implement one or more methods described in this document. The storage(s) 1704 may be used to store data and codes for implementing the methods and techniques described herein. Video processing hardware 1706 can be used to implement some of the techniques described in this document in a hardware circuit.

19 is a flowchart of a method 1900 of video bitstream processing. Method 1900 includes determining (1905) the shape of a video block, determining (1910) an interpolation order based on the video block, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation, and sequentially performing horizontal interpolation and vertical interpolation according to the interpolation of the video block, to Reconstruct (1915) the decoded representation of the video block.

20 is a flowchart of a method 2000 of video bitstream processing. The method 2000 includes determining (2005) the characteristics of the motion vector related to the video block, determining (2010) the interpolation order of the video block based on the characteristics of the motion vector, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation, and according to the video block's Interpolation performs horizontal interpolation and vertical interpolation in order to reconstruct (2015) the decoded representation of the video block.

With reference to methods 1900 and 2000, some examples of sequences that perform horizontal interpolation and vertical interpolation and their use are described in Section 4 of this document. For example, as described in Chapter 4, under different shapes of video blocks, it may be preferred to first perform one of horizontal interpolation or vertical interpolation. In some embodiments, horizontal interpolation is performed prior to vertical interpolation, and in some embodiments, vertical interpolation is performed prior to horizontal interpolation.

Referring to methods 1900 and 2000, video blocks can be encoded in a video bitstream, where bit efficiency can be achieved by using bitstream generation rules related to the interpolation order, which also depends on the shape of the video block.

It should be understood that the disclosed technology may be embedded in a video encoder or decoder to improve compression efficiency when the compressed coding unit has a shape that is significantly different from a conventional square block or semi-square rectangular block. For example, new coding tools using long or high coding units such as 4×32 or 32×4 size units may benefit from the disclosed technology.

21 is a flowchart of an example of a video processing method 2100. Method 2100 includes: determining (2102) a first prediction mode applied to the first video block; by applying horizontal interpolation and/or vertical interpolation to the first video block, between the first video block and the encoded representation of the first video block Perform (2104) the first conversion; determine (2106) the second prediction mode applied to the second video block; by applying horizontal interpolation and/or vertical interpolation to the second video block, between the second video block and the second video block Perform (2108) a second conversion between encoding representations, where, based on the determination that the first prediction mode is a multi-hypothesis prediction mode and the second prediction mode is not a multi-hypothesis prediction mode, one of horizontal interpolation and vertical interpolation of the first video block Or both use shorter tap filters compared to the filters used for the second video block.

22 is a flowchart of a method 2200 of video bitstream processing. The method includes: determining (2205) the shape of the video block; determining (2210) an interpolation order based on the shape of the video block, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation, and performing horizontal interpolation on the video block in the sequence indicated by the interpolation order And vertical interpolation to construct (2215) the encoded representation of the video block.

23 is a flowchart of a method 2300 of video bitstream processing. The method includes: determining (2305) the characteristics of the motion vector related to the video block; determining (2310) the interpolation order based on the characteristics of the motion vector, the interpolation order indicating the sequence of performing horizontal interpolation and vertical interpolation; and the sequence indicated by the interpolation order Perform horizontal interpolation and vertical interpolation on the video block to construct (2315) the encoded representation of the video block.

Various embodiments and techniques disclosed in this document can be described in the following list of embodiments.

1. A video processing method, comprising: determining a first prediction mode applied to a first video block; by applying horizontal interpolation and/or vertical interpolation to the first video block, the encoded representation in the first video block and the first video block Perform the first conversion between; determine the second prediction mode applied to the second video block; by applying horizontal and/or vertical interpolation to the second video block, between the encoded representation of the second video block and the second video block Perform a second conversion in which, based on the determination that the first prediction mode is a multi-hypothesis prediction mode and the second prediction mode is not a multi-hypothesis prediction mode, one or both of horizontal interpolation and vertical interpolation of the first video block are used A shorter tap filter compared to the filter of the second video block.

2. The method according to example 1, wherein the first video block is converted with more than two reference blocks for bidirectional prediction, and it uses more than two reference blocks for at least one reference picture list.

3. The method according to example 1, wherein the first video block is converted with more than one reference block for unidirectional prediction.

4. The method according to any one of Examples 1-3, wherein the shorter tap filter is a bilinear filter.

5. The method according to any one of Examples 1-3, wherein one or both of horizontal interpolation and vertical interpolation use a shorter tap filter for a reference picture list related to a plurality of reference blocks.

6. The method according to any one of Examples 1-5, wherein, when the reference picture list is related to a single reference block, one or both of horizontal interpolation or vertical interpolation uses the same filter as used for the ordinary prediction mode.

7. The method according to any one of examples 1-6, wherein the method is applied based on the determination of one or more of the following: use of the time domain layer, one or more blocks containing video blocks, slices, slices or The quantization parameters of the picture are within the threshold.

8. The method according to example 7, wherein the quantization parameters within the threshold range include quantization parameters greater than the threshold.

9. The method according to example 6, wherein the general prediction mode includes unidirectional prediction or bidirectional prediction inter prediction mode, unidirectional prediction uses inter prediction with at most one motion vector and one reference index to predict samples of samples in the block Value, the bi-directional prediction inter prediction mode uses inter prediction with up to two motion vectors and reference indexes to predict the sample value of the samples in the block.

10. A video decoding device, including a processor, configured to implement one or more methods of Examples 1 to 9.

11. A video encoding device, including a processor, configured to implement one or more methods of Examples 1 to 9.

12. A computer-readable program medium having code stored thereon, the code including instructions, when the processor executes the instructions, causing the processor to implement the method of one or more of Examples 1 to 9.

13. A video bit stream processing method, comprising: determining a shape of a video block; determining an interpolation order based on the shape of the video block, the interpolation order indicating a sequence in which horizontal interpolation and vertical interpolation are performed; and pairing the video blocks in a sequence indicated by the interpolation order Perform horizontal and vertical interpolation to reconstruct the decoded representation of the video block.

14. The method according to example 13, wherein the shape of the video block is represented by the width and height of the video block, and the step of determining the interpolation order further includes:

When the width of the video block is greater than the height of the video block, it is determined that vertical interpolation is performed as the interpolation order before horizontal interpolation.

15. The method according to example 13, wherein the shape of the video block is represented by the width and height, and the step of determining the interpolation order further includes:

When the width of the video block is greater than or equal to the height of the video block, it is determined that vertical interpolation is performed as the interpolation sequence before horizontal interpolation.

16. The method according to example 13, wherein the shape of the video block is represented by the width and height, and the step of determining the interpolation order further includes:

When the height of the video block is greater than or equal to the width of the video block, it is determined that horizontal interpolation is performed as the interpolation sequence before vertical interpolation.

17. The method according to example 1, wherein the shape of the video block is represented by a width and a height, and the step of determining the interpolation order further includes:

When the height of the video block is greater than the width of the video block, it is determined that horizontal interpolation is performed as the interpolation order before vertical interpolation.

18. The method according to example 1, wherein the luma and chroma components of the video block are interpolated based on the interpolation order or based on different interpolation orders.

19. The method according to example 1, wherein, when each chroma block of the chroma component corresponds to a plurality of luma blocks of the luma component, the luma and chroma components of the video block are interpolated using different interpolation orders.

20. The method according to example 13, wherein the luminance component and the chroma component of the video block are interpolated using different interpolation orders, and wherein the scaling factors used in the horizontal interpolation and the vertical interpolation are different for the luminance component and the chroma component.

21. A video bit stream processing method, comprising: determining a feature of a motion vector related to a video block; determining an interpolation order based on the feature of the motion vector, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation; and in accordance with the order of interpolation The sequence of performs horizontal interpolation and vertical interpolation on the video block to reconstruct the decoded representation of the video block.

22. The method according to example 21, wherein the characteristics of the motion vector are represented by the quarter pixel position and the half pixel position pointed by the motion vector, the motion vector includes a vertical component and a horizontal component, and determining the interpolation order includes: when the vertical component points When the quarter-pixel position and the horizontal component point to the half-pixel position, it is determined that horizontal interpolation is performed as the interpolation order before vertical interpolation.

23. The method according to example 21, wherein the feature of the motion vector is represented by a quarter pixel position and a half pixel position pointed by the motion vector, the motion vector includes a vertical component and a horizontal component, and determining the interpolation order includes: when the vertical component points When the half-pixel position and the horizontal component point to the quarter-pixel position, it is determined that vertical interpolation is performed before horizontal interpolation.

24. The method according to any one of examples 21-23, wherein the shape of the video block is a square.

25. The method according to any one of Examples 21-24, wherein the method is applied to a dual prediction mode.

26. The method according to any one of Examples 21-25, wherein the method is applied when the height of the video block times the width of the video block is less than or equal to T1, T1 being the first threshold.

27. The method according to any one of Examples 21-25, wherein when the video block has a height less than or equal to T2, the method is applied, and T2 is the second threshold.

28. The method according to any one of examples 21-25, wherein the method is applied to the luminance component of the video block.

29. A video bit stream processing method, including: Determine the shape of the video block;

The interpolation order is determined based on the shape of the video block, and the interpolation order indicates a sequence of performing horizontal interpolation and vertical interpolation; and

The horizontal interpolation and vertical interpolation are performed on the video block in the sequence indicated by the interpolation order to construct the encoded representation of the video block.

30. A video bit stream processing method, including:

Determine the characteristics of the motion vector associated with the video block;

The interpolation order is determined based on the characteristics of the motion vector, and the interpolation order indicates a sequence in which horizontal interpolation and vertical interpolation are performed; and

The horizontal interpolation and vertical interpolation are performed on the video block in the sequence indicated by the interpolation order to construct the encoded representation of the video block.

31. A video decoding device including a processor configured to implement one or more of the methods of Examples 21 to 28.

32. A video encoding device comprising a processor configured to implement the method of Example 29 or 30.

33. A computer program product on which computer code is stored, which when executed by a processor causes the processor to implement the method of any of Examples 13 to 30.

34. An apparatus in a video system, including a processor and a non-transitory storage having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any of Examples 13 to 30.

From the above, it should be understood that, for the convenience of explanation, specific embodiments of the technology disclosed in the present invention have been described herein, but various modifications can be made without departing from the scope of the present invention. Therefore, in addition to the above, the technology disclosed in the present invention is not limited to the limitation of the scope of patent application.

The implementation and functional operation of the subject name in this patent document can be implemented in various systems, digital electronic circuits, or computer software, firmware or hardware, including the structures disclosed in this specification and their structural equivalents, or one of them Or a combination of multiple. The implementation of the subject matter described in this specification can be implemented as one or more computer program products, that is, one or more modules of computer program instructions encoded on temporary and non-transitory computer readable media for data processing The device performs or controls the operation of the data processing device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a storage device, a material composition that affects a machine-readable propagation signal, or a combination of one or more of them. The term "data processing unit" or "data processing device" includes all devices, equipment, and machines used to process data, including, for example, programmable processors, computers, or multiple processors or computer groups. In addition to the hardware, the device may also include code to create an execution environment for the computer program, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

Computer programs (also known as programs, software, software applications, scripts, or codes) can be written in any form of programming language (including compiled or interpreted languages) and can be deployed in any form, including as standalone programs or as modules , Components, subprograms, or other units suitable for use in computing environments. Computer programs do not necessarily correspond to files in the file system. Programs can be stored in parts of documents that hold other programs or data (for example, one or more scripts stored in markup language documents), in a single document dedicated to the program, or in multiple coordination documents (for example, storing a Or multiple modules, subprograms, or some code files). Computer programs can be deployed on one or more computers for execution. These computers are located on one website or distributed on multiple websites and are interconnected through a communication network.

The processes and logic flows described in this specification can be performed by one or more programmable processors that execute one or more computer programs and perform functions by operating on input data and generating output. Processing and logic flow can also be performed by special purpose logic circuits, and the device can also be implemented as special purpose logic circuits, such as FPGA (field programmable gate array) or ASIC (dedicated integrated circuit).

For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, and any one or more of any type of digital computer. Typically, the processor will receive commands and data from read-only memory or random access memory or both. The basic components of a computer are a processor that executes instructions and one or more storage devices that store instructions and data. Typically, the computer will also include one or more large storage area devices for storing data, such as magnetic disks, magneto-optical disks, or optical discs, or be operatively coupled to one or more large storage area devices to receive data from or Data is transferred to one or more large storage area equipment, or both. However, computers do not necessarily have such equipment. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile storage, media, and storage devices, including, for example, semiconductor storage devices, such as EPROM, EEPROM, and flash storage devices. The processor and memory can be supplemented by dedicated logic circuits or incorporated into dedicated logic circuits.

The description and drawings are intended to be considered exemplary, where exemplary means exemplary. As used herein, the singular forms "a", "an", and "the" are intended to also include the plural forms unless the context clearly dictates otherwise. In addition, the use of "or" is intended to include "and/or" unless the context clearly dictates otherwise.

Although this patent document contains many details, it should not be construed as a limitation on the scope of any invention or patent application, but as a description of features of specific embodiments of specific inventions. Some features described in this patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various functions that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. In addition, although the above-mentioned features can be described as functioning in some combinations, even if initially required, in some cases, one or more features in the patent-applicable combination can be removed from the combination, and Combinations can refer to sub-combinations or variations of sub-combinations.

Similarly, although the operations are described in a specific order in the drawings, it should not be understood that such operations must be performed in the specific order or order shown, or that all the operations described should be performed in order to obtain the desired results. Furthermore, the separation of various system elements in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

Only some implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

tb, td: distance 1700: device 1702: processor 1704: storage 1706: Video processing hardware 1900, 2000, 2100, 2200, 2300: method 1905, 1910, 1915, 2005, 2010, 2015, 2102, 2104, 2106, 2108, 2205, 2210, 2215, 2305, 2310, 2315: steps

Figure 1 is a diagram of a quadtree binary tree (QTBT) structure. FIG. 2 shows an example derivation process of Merge candidate list construction. Figure 3 shows an example location of spatial Merge candidates. FIG. 4 shows an example of candidate pairs considered for the redundancy check of spatial Merge candidates. FIG. 5 shows an example of the position of the second prediction unit (PU) partitioned by Nx2N and 2NxN. FIG. 6 is an illustration of motion vector scaling of time-domain Merge candidates. FIG. 7 shows example candidate positions of time domain Merge candidates C0 and C1. FIG. 8 shows an example of combined bidirectional prediction Merge candidates. FIG. 9 shows an example of the derivation process of motion vector prediction candidates. 10 is an illustration of motion vector scaling of spatial motion vector candidates. FIG. 11 shows an example of advanced temporal motion vector prediction (ATMVP) motion prediction of a coding unit (CU). FIG. 12 shows an example of one CU having four sub-blocks (A-D) and its neighboring blocks (a-d). Fig. 13 shows the non-adjacent Merge candidates proposed in J0021. Fig. 14 shows the non-adjacent Merge candidates proposed in J0058. Figure 15 shows the non-adjacent Merge candidates proposed in J0059. FIG. 16 shows an example of integer sample and fractional sample positions for quarter sample luminance interpolation. 17 is a block diagram of an example of a video processing device. Figure 18 shows a block diagram of an example implementation of a video encoder. 19 is a flowchart of an example of a video bit stream processing method. 20 is a flowchart of an example of a video bit stream processing method. 21 is a flowchart of an example of a video processing method. 22 is a flowchart of an example of a video bit stream processing method. 23 is a flowchart of an example of a video bit stream processing method.

2100: Method

2102, 2104, 2106, 2108: steps

Claims (12)

A video processing method, including: Determine the first prediction mode applied to the first video block; Performing a first conversion between the first video block and the encoded representation of the first video block by applying horizontal interpolation and/or vertical interpolation to the first video block; Determine the second prediction mode applied to the second video block; Performing a second conversion between the encoded representation of the second video block and the second video block by applying horizontal interpolation and/or vertical interpolation to the second video block, Wherein, based on the determination that the first prediction mode is a multiple hypothesis prediction mode and the second prediction mode is not a multiple hypothesis prediction mode, one or both of the horizontal interpolation and the vertical interpolation of the first video block Uses a shorter tap filter than the filter used for the second video block. The method according to item 1 of the patent application scope, wherein the first video block is converted using more than two reference blocks for bidirectional prediction, and at least for one reference picture list, it uses at least two reference blocks . The method according to item 1 of the patent application scope, wherein more than one reference block is used to convert the first video block for unidirectional prediction. The method according to any one of items 1 to 3 of the patent application range, wherein the shorter tap filter is a bilinear filter. The method according to any one of items 1 to 3 of the patent application range, wherein one or both of the horizontal interpolation and the vertical interpolation use the shorter for a reference picture list related to a plurality of reference blocks Tap filter. The method according to any one of items 1 to 5 of the patent application range, wherein, when the reference picture list is related to a single reference block, one or both of the horizontal interpolation or the vertical interpolation are used and used for The same filter as the normal prediction mode. The method according to any one of items 1 to 6 of the patent application range, wherein the method is applied based on the determination of one or more of the following: use of the time domain layer, one or more including the video block The quantization parameters of multiple blocks, slices, slices, or pictures are within the threshold. The method according to item 7 of the patent application scope, wherein the quantization parameters within the threshold range include quantization parameters greater than the threshold. The method according to item 6 of the patent application scope, wherein the normal prediction mode includes a unidirectional prediction or a bidirectional prediction inter prediction mode, the unidirectional prediction uses inter prediction with at most one motion vector and one reference index To predict the sample values of the samples in the block, the bidirectional prediction inter prediction mode uses inter prediction with at most two motion vectors and reference indexes to predict the sample values of the samples in the block. A video decoding device includes a processor configured to implement one or more of the methods described in items 1 to 9 of the patent application. A video encoding device, including a processor, is configured to implement one or more of the methods described in items 1 to 9 of the patent application. A computer readable program medium having code stored thereon, the code including instructions, when the processor executes the instructions, causing the processor to implement one or more of items 1 to 9 of the patent application Methods.

Images