TW202023276A - Shape dependent interpolation order - Google Patents

Abstract

The application provides a method for video bitstream processing, a video decoding apparatus and a video encoding apparatus, the method comprising: determining a shape of a video block; determining an interpolation order based on the shape of the video block, the interpolation order indicative of a sequence of performing a horizontal interpolation and a vertical interpolation; and performing the horizontal interpolation and the vertical interpolation for the video block in the sequence indicated by the interpolation order, to reconstruct a decoded representation of the video block.

Description

Depends on the interpolation order of the shape

The document of the present invention relates to video coding technology, equipment and system. [Cross references to related applications] According to the applicable "Patent Law" and/or the "Paris Convention", the present invention timely claims the priority and benefits of the international patent application number PCT/CN2018/095576 filed on July 13, 2018. The entire disclosure of International Patent Application No. PCT/CN2018/095576 is incorporated herein by reference as a part of the disclosure of the present invention.

Despite advances in video compression, digital video still uses the largest bandwidth on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the bandwidth demand for digital video usage will continue to grow.

The disclosed technique can be used by video decoder or encoder embodiments, where block-shaped interpolation sequence techniques are used to improve interpolation.

In an example aspect, a video bitstream processing method is disclosed. The method includes: determining the shape of the video block; determining the interpolation order based on the shape of the video block, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation; and performing horizontal interpolation and vertical interpolation on the video block in the sequence indicated by the interpolation order to The decoded representation of the reconstructed video block.

In another example aspect, a video bitstream processing method includes: determining a feature of a motion vector related to a video block; determining an interpolation order based on the feature of the motion vector, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation; and by interpolation The sequence indicated by the order performs horizontal interpolation and vertical interpolation on the video block to reconstruct the decoded representation of the video block.

In another example aspect, a video bitstream processing method is disclosed. The method includes: determining the shape of the video block; determining the interpolation order based on the shape of the video block, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation; and performing horizontal interpolation and vertical interpolation on the video block in the sequence indicated by the interpolation order to Construct an encoded representation of the video block.

In another example aspect, a video bitstream processing method is disclosed. The method includes: determining the characteristics of a motion vector related to the video block; determining an interpolation order based on the characteristics of the motion vector, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation; and performing horizontal interpolation on the video block in the sequence indicated by the interpolation order And vertical interpolation to construct the coded representation of the video block.

In an example aspect, a video processing method is disclosed. The method includes: determining a first prediction mode applied to a first video block; performing a first prediction mode between the first video block and an encoded representation of the first video block by applying horizontal interpolation and/or vertical interpolation to the first video block Convert, determine the second prediction mode applied to the second video block; perform a second conversion between the second video block and the encoded representation of the second video block by applying horizontal interpolation and/or vertical interpolation to the second video block, Wherein, based on the determination that the first prediction mode is a multi-hypothesis prediction mode and the second prediction mode is not a multi-hypothesis prediction mode, one or both of the horizontal interpolation and the vertical interpolation of the first video block are used with the one used for the second video block The filter has a shorter tap compared to the filter.

In another example aspect, a video decoding device implementing the video processing method described herein is disclosed.

In yet another example aspect, a video encoding device implementing the video processing method described herein is disclosed.

In another typical aspect, the various technologies described herein can be implemented as computer program products stored on non-transitory computer-readable media. The computer program product includes program code for executing the method described herein.

In yet another example aspect, an apparatus in a video system is disclosed. The device includes a processor and a non-transitory storage with instructions thereon, wherein the instructions executed by the processor enable the processor to implement the above method.

One or more implementation details are set forth in the appendix, drawings and the following description. Other features will be apparent from the specification and drawings and the scope of patent application.

This document provides various techniques that can be used by decoders of video bitstreams to improve the quality of decompressed or decoded digital video. In addition, video encoders can also implement these techniques during the encoding process in order to reconstruct decoded frames for further encoding.

In order to facilitate understanding, chapter titles are used in this document, and the embodiments and techniques are not limited to the corresponding parts. In this way, embodiments from one chapter can be combined with embodiments from other chapters.

1. Summary

The present invention relates to video coding technology. Specifically, it relates to interpolation in video coding. It can be applied to existing video coding standards, such as HEVC, or standards to be finalized (multifunctional video coding). It may also be applicable to future video coding standards or video encoders.

2. Background Video coding standards are mainly developed through the development of well-known ITU-T and ISO/IEC standards. ITU-T developed H.261 and H.263, ISO/IEC developed MPEG-1 and MPEG-4 vision, and the two organizations jointly developed H.262/MPEG-2 video and H.264/MPEG-4 Advanced Video Coding (AVC) and H.265/HEVC standard. Since H.262, the video coding standard is based on a hybrid video coding structure, which uses time domain prediction plus transform coding. In order to explore future video coding technologies beyond HEVC, the Joint Video Exploration Team (JVET) was jointly established by VCEG and MPEG in 2015. Since then, JVET has adopted many new methods and introduced them into a reference software called Joint Exploration Model (JEM). In April 2018, the Joint Video Expert Group (JVET) between VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11 (MPEG) was created to study the VVC standard, with the goal of reducing it by 50% compared to HEVC The bit rate.

Figure 18 is a block diagram of an example implementation of a video encoder.

2.1 Quadtree plus binary tree (QTBT) block structure with larger CTU In HEVC, the CTU is divided into CUs by using a quad-tree structure (represented as a coding tree) to adapt to various local characteristics. It is decided at the CU level whether to use inter (temporal) prediction or intra (spatial) prediction to encode the picture area. According to the PU partition type, each CU can be further divided into one, two, or four PUs. In a PU, the same prediction process is applied, and relevant information is transmitted to the decoder on the basis of the PU. After the residual block is obtained by applying prediction processing based on the PU partition type, the CU may be partitioned into transformation units (TU) according to another quadtree structure similar to the coding tree of the CU. An important feature of the HEVC structure is that it has multiple partition concepts, including CU, PU, and TU.

The QTBT structure eliminates the concept of multiple segmentation types, that is, the QTBT structure eliminates the separation of the concepts of CU, PU, and TU, and supports more flexibility in the shape of CU segmentation. In the QTBT block structure, the CU can be square or rectangular. As shown in Figure 1, the coding tree unit (CTU) is first segmented with a quadtree structure. The quad leaf node is further divided by the binary tree structure. There are two types of divisions in binary tree division: symmetric horizontal division and symmetric vertical division. The binary leaf node is called a coding unit (CU), and the division is used for prediction and transformation processing without further division. This means that CU, PU, and TU have the same block size in the QTBT coding block structure. In JEM, a CU is sometimes composed of coded blocks (CB) of different color components. For example, in the P-strip and B-strip in the 4:2:0 chroma format, a CU contains one luminance CB and two color components. CU is sometimes composed of a single component of CB, for example, in the case of I stripe, one CU contains only one luma CB or only two chroma CBs.

The following parameters are defined for the QTBT segmentation scheme.

–CTU size: the size of the root node of the quadtree, which is the same as the concept in HEVC.

– MiNQTSize : The minimum allowable quadrilateral leaf node size

– MaxBTSize : the maximum allowable size of the root node of the binary tree

– MaxBTDePTh : the maximum allowable depth of the binary tree

– MiNBTSize : the minimum allowable size of a binary leaf node

In an example of the QTBT segmentation structure, the CTU size is set to 128×128 luma samples with two corresponding 64×64 chroma sample blocks, MiNQTSize is set to 16×16, and MaxBTSize is set to 64× 64. MiNBTSize (width and height) is set to 4×4, and MaxBTSize is set to 4. The quadtree division is first applied to CTU to generate quadtree leaf nodes. The size of the quad leaf node may have a size from 16×16 (ie, MiNQTSize ) to 128×128 (ie, CTU size). If the leaf quadtree node is 128×128, it will not be further divided by the binary tree because its size exceeds MaxBTSize (for example, 64×64). Otherwise, the leaf quadtree node can be further divided by the binary tree. Therefore, the quad leaf node is also the root node of the binary tree, and its binary tree depth is zero. When the depth of the binary tree reaches MaxBTDePTh (ie, 4), no further division is considered. When the width of the binary tree node is equal to MiNBTSize (ie, 4), no further horizontal division is considered. Similarly, when the height of the binary tree node is equal to MiNBTSize , no further vertical division is considered. The leaf nodes of the binary tree are further processed through prediction and transformation processing without further segmentation. In JEM, the maximum CTU size is 256×256 luminance samples.

Figure 1 (left side) illustrates an example of block segmentation by using QTBT, and Figure 1 (right side) illustrates the corresponding tree representation. The solid line represents the quadtree division, and the dashed line represents the binary tree division. In each partition (ie, non-leaf) node of the binary tree, a flag is signaled to indicate which partition type (ie, horizontal or vertical) to use, where 0 means horizontal partition and 1 means vertical partition. For quadtree partitioning, there is no need to specify the partition type, because quadtree partitioning always divides one block horizontally and vertically to generate 4 sub-blocks of the same size. In addition, the QTBT solution supports the ability to have separate QTBT structures for brightness and chroma. At present, for P-strip and B-strip, the luminance and chroma CTB in a CTU share the same QTBT structure. However, for the I-slice, the QTBT structure is used to divide the luminance CTB into CU, and another QTBT structure is used to divide the chroma CTB into chroma CU. This means that the CU in the I slice consists of coding blocks of the luma component or two chroma component coding blocks, and the CU in the P slice or the B slice consists of coding blocks of all three color components.

In HEVC, in order to reduce memory access for motion compensation, inter prediction of small blocks is restricted, so that 4×8 and 8×4 blocks do not support bidirectional prediction, and 4×4 blocks do not support inter prediction. In JEM's QTBT, these restrictions are removed.

2.2 Inter prediction in HEVC/H.265

Each inter-predicted PU has motion parameters of one or two reference picture lists. The motion parameters include motion vectors and reference picture indexes. The use of one of the two reference picture lists can also be signaled using inter_pred_idc . The motion vector can be explicitly coded as increments relative to the predicted value.

When the CU is coded in skip mode, a PU is associated with the CU, and there is no significant residual coefficient, no coded motion vector increment or reference picture index. A Merge mode is specified, through which the motion parameters of the current PU can be obtained from adjacent PUs (including spatial and temporal candidates). The Merge mode can be applied to any inter-predicted PU, not just the skip mode. Another option for Merge mode is the explicit transmission of motion parameters, where the motion vector (more precisely, the motion vector difference compared to the motion vector prediction value), the reference picture index corresponding to each reference picture list, and the reference picture list The use of will be signaled explicitly in each PU. In this document, this mode is called Advanced Motion Vector Prediction (AMVP). When the signal indicates that one of the two reference picture lists is to be used, the PU is generated from one sample block. This is called "one-way prediction". One-way prediction is available for both P-band and B-band.

When the signal indicates that two reference picture lists are to be used, the PU is generated from two sample blocks. This is called "bidirectional prediction". Bi-directional prediction is only available for band B.

The following text provides details of the inter prediction mode specified in HEVC. The description will start in Merge mode.

2.2.1 Merge mode

2.2.1.1 Derivation of candidates for Merge mode

When using the Merge mode to predict the PU, the index pointing to the entry in the Merge candidate list is analyzed from the bit stream and used to retrieve motion information. The structure of this list is specified in the HEVC standard and can be summarized in the following order of steps:

Step 1: Initial candidate derivation

Step 1.1: Airspace candidate derivation

Step 1.2: Redundancy check of airspace candidates

Step 1.3: Time domain candidate derivation

Step 2: Additional candidate insertion

Step 2.1: Creation of bidirectional prediction candidates

Step 2.2: Insertion of zero motion candidates

These steps are also schematically depicted in FIG. 2. For the derivation of spatial Merge candidates, at most four Merge candidates are selected among candidates located at five different positions. For the time-domain Merge candidate derivation, at most one Merge candidate is selected among the two candidates. Since the number of candidates for each PU is assumed to be constant at the decoder, when the number of candidates obtained from step 1 does not reach the maximum number of Merge candidates signaled in the slice header ( MaxNumMergeCand ), additional candidates are generated. Since the number of candidates is constant, the index of the best Merge candidate is encoded using truncated unary binarization (TU). If the size of the CU is equal to 8, all PUs of the current CU share one Merge candidate list, which is the same as the Merge candidate list of the 2N×2N prediction unit.

The following describes the operations related to the above steps in detail.

2.2.1.2 Airspace candidate derivation

In the derivation of spatial Merge candidates, a maximum of four Merge candidates are selected among the candidates located at the positions shown in FIG. 3. The derivation sequence is A1, B1, B0, A0 and B2. Position B2 is only considered when any PU at positions A1, B1, B0, A0 is not available (for example, because it belongs to another slice or slice) or is internally coded. After the candidate at the A1 position is added, redundancy check is performed on the addition of the remaining candidates, which ensures that candidates with the same motion information are excluded from the list, thereby improving coding efficiency. In order to reduce the computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check. On the contrary, only the pair connected with the arrow in Fig. 4 will be considered, and only when the corresponding candidate for redundancy check does not have the same motion information, the candidate is added to the list. Another source of copied motion information is the "second PU" related to the 2N×2N different partition. For example, Fig. 5 depicts the second PU in the case of N×2N and 2N×N, respectively. When the current PU is divided into N×2N, candidates at the A1 position are not considered for list construction. In some embodiments, adding this candidate may result in two prediction units with the same motion information, which is redundant for having only one PU in the coding unit. Likewise, when the current PU is divided into 2N×N, the position B1 is not considered.

2.2.1.3 Time-domain candidate derivation

In this step, only one candidate is added to the list. In particular, in the derivation of this time-domain Merge candidate, the scaling motion vector is derived based on the collocated PU that has the smallest picture order count POC difference with the current picture in the given reference picture list. The reference picture list used to derive the collocated PU is signaled explicitly in the slice header. The dotted line in Figure 6 shows the acquisition of the scaled motion vector of the time-domain Merge candidate, which uses the POC distances tb and td to scale from the motion vector of the collocated PU, where tb is defined as the distance between the reference picture of the current picture and the current picture. POC difference, and td is defined as the POC difference between the reference picture of the collocated picture and the collocated picture. The reference picture index of the time domain Merge candidate is set to zero. The actual implementation of the scaling process is described in the HEVC specification. For the B slice, two motion vectors (one for reference picture list 0 and the other for reference picture list 1) are obtained and combined to make them a bi-directional predictive Merge candidate.

Fig. 6 is an illustration of motion vector scaling for time-domain Merge candidates.

In the collocated PU (Y) belonging to the reference frame, the position of the time domain candidate is selected between the candidates C ₀ and C ₁ , as shown in FIG. 7. If the PU at position C ₀ is not available, internally coded, or outside the current CTU line, position C ₁ is used. Otherwise, the position C ₀ is used for the derivation of time-domain Merge candidates.

2.2.1.4 Additional candidate insertion

In addition to spatial and temporal Merge candidates, there are two additional types of Merge candidates: combined bidirectional prediction Merge candidates and zero Merge candidates. The combined bidirectional prediction Merge candidate is generated using spatial and temporal Merge candidates. The combined bidirectional prediction Merge candidate is only used for B bands. The combined bi-directional prediction candidate is generated by combining the first reference picture list motion parameter of the initial candidate with the second reference picture list motion parameter of another candidate. If these two tuples provide different motion hypotheses, they will form a new bi-prediction candidate. As an example, Figure 8 shows a situation where two candidates in the original list (on the left) are used to create a combined bidirectional prediction Merge candidate added to the final list (on the right), which has MvL0 and refIdxL0 or MvL1 and refIdxL1 Two candidates. In the prior art, there are many rules about combination that need to be considered to generate these additional Merge candidates.

Insert zero motion candidates to fill the remaining entries in the Merge candidate list to reach the capacity of MaxNumMergeCand. These candidates have a zero spatial displacement and a reference picture index that starts from zero and increases every time a new zero motion candidate is added to the list. The number of reference frames used by these candidates is 1 frame and 2 frames for unidirectional prediction and bidirectional prediction, respectively. Finally, no redundancy check is performed on these candidates.

2.2.1.5 Motion estimation area processed in parallel

In order to speed up the encoding process, motion estimation can be performed concurrently, thereby deriving the motion vectors of all prediction units in a given area at the same time. Deriving Merge candidates from the spatial neighborhood may interfere with parallel processing, because a prediction unit cannot derive motion parameters from neighboring PUs before completing related motion estimation. In order to ease the balance between coding efficiency and processing delay, HEVC defines a motion estimation area (MER). The syntax element "log2_parallel_merge_level_minus2" can be used to signal the size of MER in the picture parameter set. When MER is defined, Merge candidates that fall into the same area are marked as unavailable, and therefore are not considered in the list construction.

2.2.2 AMVP AMVP utilizes the space-time correlation between the motion vector and the adjacent PU, which is used for the explicit transmission of motion parameters. For each reference picture list, the motion vector candidate list is first constructed by checking the availability of the temporally adjacent PU positions on the upper left, removing redundant candidate positions and adding a zero vector to make the length of the candidate list constant. Then, the encoder can select the best predicted value from the candidate list and send a corresponding index indicating the selected candidate. Similar to the Merge index signal, the index of the best motion vector candidate is coded using truncated unary. The maximum value to be encoded in this case is 2 (refer to Figure 9). In the following chapters, the derivation process of motion vector prediction candidates will be introduced in detail.

2.2.2.1 Derivation of AMVP candidates

Figure 9 summarizes the derivation process of motion vector prediction candidates.

In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. For the derivation of the spatial motion vector candidates, two motion vector candidates are finally derived based on the motion vector of each PU located in the five different positions shown in FIG. 3.

For the derivation of temporal motion vector candidates, one motion vector candidate is selected from two candidates, which are derived based on two different juxtaposed positions. After making the first empty time selection list, remove the repeated motion vector candidates from the list. If the number of potential candidates is greater than two, the motion vector candidates whose reference picture index is greater than 1 in the associated reference picture list are removed from the list. If the number of space-time motion vector candidates is less than two, additional zero motion vector candidates will be added to the list.

2.2.2.2 Spatial Motion Vector Candidates When deriving spatial motion vector candidates, at most two candidates are considered among the five potential candidates. These five candidates are from the PU at the positions depicted in Figure 3, and these positions are the same as the positions of the motion merge. The derivation order on the left side of the current PU is defined as A ₀ , A ₁ , and scaled A ₀ , scaled A ₁ . The derivation sequence above the current PU is defined as B ₀ , B ₁ , B ₂ , scaled B ₀ , scaled B ₁ , scaled B ₂ . Therefore, there are four cases on each side that can be used as motion vector candidates, two cases do not need to use spatial scaling, and two cases use spatial scaling. The four different situations are summarized as follows:

--Zoom without space

(1) The same reference picture list and the same reference picture index (same POC)

(2) Different reference picture lists, but the same reference picture index (same POC)

--Space zoom

(3) The same reference picture list, but different reference picture indexes (different POC)

(4) Different reference picture lists, and different reference picture indexes (different POCs)

First check the situation without space zoom, and then check the space zoom. When the POC is different between the reference picture of the neighboring PU and the reference picture of the current PU, spatial scaling is considered regardless of the reference picture list. If all the PUs of the left candidate are unavailable or are internally coded, the aforementioned motion vector is allowed to be scaled to help parallel derivation of the left and upper MV candidates. Otherwise, spatial scaling of the aforementioned motion vector is not allowed.

Fig. 10 is an illustration of motion vector scaling of spatial motion vector candidates.

In the spatial scaling process, the motion vectors of adjacent PUs are scaled in a manner similar to temporal scaling, as shown in FIG. 10. The main difference is that the reference picture list and index of the current PU are given as input, and the actual scaling process is the same as the time domain scaling process.

2.2.2.3 Time domain motion vector candidates

Except for the derivation of the reference picture index, all the derivation processes of the temporal Merge candidates are the same as the derivation process of the spatial motion vector candidates (see FIG. 7). Signal the reference picture index to the decoder.

2.3 New Inter-frame Merge Candidates in JEM

2.3.1 Motion vector prediction based on sub-CU

In JEM with QTBT, each CU can have at most one set of motion parameters for each prediction direction. By dividing a large CU into sub-CUs and deriving the motion information of all sub-CUs of the large CU, two sub-CU-level motion vector prediction methods are considered in the encoder. The optional temporal motion vector prediction (ATMVP) method allows each CU to obtain multiple sets of motion information from multiple blocks smaller than the current CU in the collocated reference picture. In the space-time motion vector prediction (STMVP) method, the motion vector of the sub-CU is recursively derived by using the temporal motion vector prediction value and the spatial adjacent motion vector.

In order to maintain a more accurate motion field for sub-CU motion prediction, motion compression of reference frames is currently disabled.

2.3.1.1 Optional temporal motion vector prediction

In the optional temporal motion vector prediction (ATMVP) method, the motion vector temporal motion vector prediction (TMVP) is modified by extracting multiple sets of motion information (including motion vectors and reference indexes) from blocks smaller than the current CU. As shown in Figure 11, the sub-CU is a square N×N block (default N is set to 4).

ATMVP predicts the motion vectors of sub-CUs in the CU in two steps. The first step is to use so-called time-domain vectors to identify corresponding blocks in the reference picture. The reference picture is called the motion source picture. The second step is to divide the current CU into sub-CUs, and obtain the motion vector and the reference index of each sub-CU from the block corresponding to each sub-CU, as shown in FIG. 11.

In the first step, the reference picture and the corresponding block are determined by the motion information of the spatial neighboring blocks of the current CU. In order to avoid repeated scanning processing of adjacent blocks, the first Merge candidate in the Merge candidate list of the current CU is used. The first available motion vector and its associated reference index are set as the index of the time domain vector and the motion source picture. In this way, in ATMVP, compared with TMVP, the corresponding block can be identified more accurately, where the corresponding block (sometimes called a collocated block) is always located at the lower right corner or center position relative to the current CU. In the second step, by adding the time domain vector to the coordinates of the current CU, the corresponding block of the sub-CU is identified by the time domain vector in the motion source picture. For each sub-CU, the motion information of its corresponding block (the smallest motion grid covering the center sample point) is used to derive the motion information of the sub-CU. After identifying the motion information corresponding to the N×N block, it is converted into the motion vector and reference index of the current sub-CU, which is the same as the TMVP method of HEVC, in which motion scaling and other processing are applied. For example, the decoder checks whether the low-delay condition is satisfied (for example, the POC of all reference pictures of the current picture is less than the POC of the current picture), and may use the motion vector MVx (the motion vector corresponding to the reference picture list X) to assign each sub The CU predicts the motion vector MVy (X is equal to 0 or 1 and Y is equal to 1-X).

2.3.1.2 Space-time motion vector prediction

In this method, the motion vector of the sub-CU is derived recursively in the raster scan order. Figure 12 illustrates this concept. Let us consider an 8×8 CU, which contains four 4×4 sub-CUs A, B, C, and D. The adjacent 4×4 blocks in the current frame are labeled a, b, c, and d.

The motion derivation of sub CU A starts by identifying its two spatial neighbors. The first neighbor is the N×N block above the sub CU A (block c). If the block c is not available or internally coded, then check the other N×N blocks above the sub-CU A (from left to right, starting from block c). The second neighbor is a block to the left of the sub CU A (block b). If block b is not available or is internally coded, check other blocks on the left of sub-CU A (from top to bottom, starting from block b). The motion information obtained from adjacent blocks in each list is scaled to the first reference frame of the given list. Next, according to the same formula as TMVP specified in HEVC, the temporal motion vector prediction (TMVP) of sub-block A is derived. Extract the motion information of the collocated block at position D and perform corresponding scaling. Finally, after retrieving and scaling the motion information, average all available motion vectors (up to 3) for each reference list. The average motion vector is designated as the motion vector of the current sub-CU.

2.3.1.3 Sub-CU motion prediction mode signal

The sub-CU mode is enabled as an additional merge candidate, and no additional syntax elements are required to signal the mode. Two additional merge candidates are added to the merge candidate list of each CU to indicate the ATMVP mode and the STMVP mode. If the sequence parameter set indicates that ATMVP and STMVP are enabled, up to seven merge candidates are used. The coding logic of the additional merge candidates is the same as the merge candidates in the HM, which means that for each CU in the P or B slice, two additional RD checks are required for the two additional merge candidates.

In JEM, CABAC performs context encoding on all binary bits of the merge index. In HEVC, only the first binary bit is context-encoded, and the remaining binary bits are context-coded by bypass.

2.3.2 Non-adjacent Merge candidates

In J0021, Qualcomm proposes to derive additional spatial Merge candidates from non-adjacent neighboring positions marked as 6 to 49 in FIG. 13. The derived candidate is added after the TMVP candidate in the Merge candidate list.

In J0058, Tencent proposes to derive additional spatial Merge candidates from the position in the external reference area with offset (-96, -96) from the current block.

As shown in Fig. 14, the positions are marked as A(i, j), B(i, j), C(i, j), D(i, j) and E(i, j). Compared with the previous B or C candidates, each candidate B(i, j) or C(i, j) has an offset of 16 in the vertical direction. Compared to its previous A or D candidates, each candidate A(i, j) or D(i, j) has an offset of 16 in the horizontal direction. Compared to its previous E candidate, each E(i, j) has an offset of 16 in the horizontal and vertical directions. Check candidates from the inside out. And the order of the candidates is A(i, j), B(i, j), C(i, j), D(i, j) and E(i, j). Further study whether the number of merge candidates can be further reduced. The candidate is added after the TMVP candidate in the merge candidate list.

In J0059, according to their digit order after the time domain candidates, the extended space positions from 6 to 27 in Figure 15 are checked. In order to save MV line buffering, all space candidates are limited to two CTU lines.

2.4 Intra prediction in JEM

2.4.1 Intra mode coding with 67 intra prediction modes

For luminance interpolation filtering, an 8-tap separable DCT-based interpolation filter is used for 2/4 precision samples, and a 7-tap separable DCT-based interpolation filter is used for 1/4 precision samples, as shown in Table 1.

table 1 : Used for 1/4 Luminance interpolation 8 Tap DCT-IF coefficient.

Similarly, the 4-tap separable DCT-based interpolation filter is used For the chroma interpolation filter, as shown in Table 2.

table 2 : Used for 1/8 Chromatic interpolation 4 Tap DCT-IF coefficient.

For 4:2:2 vertical interpolation and 4:4:4 chroma channel horizontal and vertical interpolation, the odd positions in Table 2 are not used, resulting in 1/4 chroma interpolation.

For bidirectional prediction, before averaging the two prediction signals, regardless of the source bit depth, the bit depth of the output of the interpolation filter remains at 14-bit precision. The actual averaging process is done implicitly through the bit depth reduction process:

predSamples[x, y] = predSamplesL0[x, y] + predSamplesL1[x, y] + offset)>>shift

Where shift = (15 – BitDepth) and offset = 1 >> (shift – 1)

If both the horizontal component and the vertical component of the motion vector point to the sub-pixel position, the horizontal interpolation is always performed first, and then the vertical interpolation is performed. For example, in order to interpolate the sub-pixel j0,0 shown in FIG. 16, first, b0,k (k = -3, -2,... 3) is interpolated according to equation 2-1, and then according to equation 2-2 Interpolate j0,0. Here, shift1 = Min(4, BitDepthY-8), and shift2 = 6, where BitDepthY is the bit depth of the video block, more specifically, the bit depth of the luminance component of the video block.

b0,k = (-A-3,k + 4 * A-2,k-11 * A – 1,k + 40 * A0,k + 40 * A1,k-11 * A2,k + 4 * A3, k-A4,k) >> shift1 (2-1)

j0,0 = (-b0,-3 + 4 * b0,-2-11 * b0,-1 + 40 * b0,0 + 40 * b0,1-11 * b0,2 + 4 * b0,3-b0 ,4) >> shift2 (2-2)

Alternatively, we can perform vertical interpolation first, and then perform horizontal interpolation. In this case, to interpolate j0,0, first, interpolate hk,0 (k = -3, -2,... 3) according to equation 2-3, and then according to equation 2-4 Interpolate j0,0. When BitDepthY is less than or equal to 8, shift1 is 0, and there is no loss in the first interpolation stage. Therefore, the final interpolation result will not be changed by the interpolation sequence. However, when BitDepthY is greater than 8, shift1 is greater than 0. In this case, when a different interpolation sequence is applied, the final interpolation result may be different.

hk,0 =(-Ak,-3 + 4 * Ak,-2-11 * Ak, -1 + 40 * Ak,0 + 40 * Ak,1-11 * Ak,2 + 4 * Ak,3 – Ak , 4)>> shift1 (2-3)

j0,0 =(-h-3,0 + 4 * h-2,0-11 * h-1,0 + 40 * h0,0 + 40 * h1,0-11 * h2,0 + 4 * h3, 0-h4,0 )>> shift2 (2-4)

3. Examples of problems solved by the embodiments

For the luminance block size WxH, if we always perform horizontal interpolation first, the required interpolation (per pixel) is shown in Table 3.

Table 3: HEVC/JEM interpolation required for WxH luminance component

On the other hand, if we perform vertical interpolation first, the required interpolation is shown in Table 4. Obviously, the best interpolation order is the interpolation order that requires a smaller number of interpolations between Table 3 and Table 4.

Table 4: When the interpolation order is reversed, the required interpolation of the WxH luminance component

For chroma components, if we always perform horizontal interpolation first, the required interpolation is ((H + 3) x W + W x H) / (W x H) = 2 + 3 / H. If we always perform vertical interpolation first, the required interpolation is ((W + 3) x H + W x H) / (W x H) = 2 + 3 / W.

As mentioned above, when the bit depth of the input video is greater than 8, different interpolation orders may lead to different interpolation results. Therefore, the interpolation order should be implicitly defined in the encoder and decoder.

4. Example of embodiment

To solve these problems and provide other benefits, we propose a shape-dependent interpolation sequence.

The following detailed examples should be regarded as examples explaining general concepts. These inventions should not be interpreted in a narrow way. In addition, these inventions can be combined in any manner.

1. It is proposed that the interpolation order depends on the current coding block shape (for example, the coding block is a CU).

a. In one example, for blocks with width> height (such as CU, PU, or sub-blocks used in sub-block-based prediction (such as affine, ATMVP, or BIO)), first perform vertical interpolation, and then perform horizontal interpolation For example, first interpolate the pixels d _k,0 , h _k,0 and n _k,0 , and then interpolate e _0,0 to r _0,0 . Examples of j _0,0 are shown in equations 2-3 and 2-4.

i. Alternatively, for blocks with width >= height (such as CU, PU or sub-blocks used in sub-block-based prediction (such as affine, ATMVP or BIO)), first perform vertical interpolation, and then perform horizontal interpolation .

b. In one example, for blocks with width >= height (such as CU, PU or sub-blocks used in sub-block-based prediction (such as affine, ATMVP or BIO)), first perform horizontal interpolation, and then perform vertical Interpolation.

i. Alternatively, for blocks with width>height (such as CU, PU, or sub-blocks used in sub-block-based prediction (such as affine, ATMVP, or BIO)), first perform horizontal interpolation and then perform vertical interpolation.

c. In one example, both the luma and chroma components follow the same interpolation order.

d. Alternatively, when one chroma coding block corresponds to multiple luma coding blocks (for example, for a 4:2:0 color format, one chroma 4×4 block can correspond to two 8×4 or 4 ×8 luma block), different interpolation sequences can be used for luma and chroma.

e. In an example, when using different interpolation orders, the zoom factors in multiple stages (ie, shift1 and shift2) can be further changed accordingly.

2. Alternatively, in addition, the interpolation order of the proposed luminance components can also depend on the MV.

a. In an example, if the vertical MV component points to a quarter-pixel position and the horizontal MV component points to a half-pixel position, first perform horizontal interpolation and then perform vertical interpolation.

b. In one example, if the vertical MV component points to a half-pixel position and the horizontal MV component points to a quarter-pixel position, first perform vertical interpolation, and then perform horizontal interpolation.

c. In one example, the proposed method is only applied to square coding blocks.

3. The proposed method can be applied to certain modes, block sizes/shapes and/or certain sub-block sizes.

a. The proposed method can be applied to some modes, such as bidirectional prediction mode.

b. The proposed method can be applied to certain block sizes.

i. In an example, it only applies to blocks with w×h >= T1, where w and h are the width and height of the current block, and T1 is the first threshold, which can be a predefined value depending on design requirements , Such as 16, 32, or 64.

ii. In an example, it is only applied to blocks where h >= T2, and T2 is the second threshold, which may be a predefined value depending on design requirements, such as 4 or 8.

c. The proposed method can be applied to certain color components (such as only the luminance component).

4. Propose that when applying multi-hypothesis prediction to a block, short-tap or different interpolation filters can be applied compared to those filters applied to ordinary prediction modes.

a. In one example, a bilinear filter can be used.

b. The short-tap or second interpolation filter can be applied to a reference picture list involving multiple reference blocks, and for another reference picture with only one reference block, the same filtering as the filter used for the normal prediction mode can be applied Device.

c. The proposed method can be applied under certain conditions, such as the quantization parameters of some (one or more) time domain layers, blocks/slices/slices/pictures containing the block are within a range (such as greater than a threshold) .

FIG. 17 is a block diagram of the video processing device 1700. The apparatus 1700 may be used to implement one or more methods described herein. The device 1700 may be embedded in a smart phone, a tablet computer, a computer, an Internet of Things (IoT) receiver, and the like. The apparatus 1700 may include one or more processors 1702, one or more storages 1704, and video processing hardware 1706. The processor(s) 1702 may be configured to implement one or more methods described in this document. The storage(s) 1704 may be used to store data and codes used to implement the methods and techniques described herein. Video processing hardware 1706 can be used to implement some of the techniques described in this document in hardware circuits.

FIG. 19 is a flowchart of a method 1900 of video bitstream processing. The method 1900 includes determining (1905) the shape of a video block, determining (1910) an interpolation order based on the video block, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation, and sequentially performing horizontal interpolation and vertical interpolation according to the interpolation of the video block, to Rebuild (1915) the decoded representation of the video block.

FIG. 20 is a flowchart of a method 2000 of video bitstream processing. The method 2000 includes determining (2005) characteristics of a motion vector related to a video block, determining (2010) an interpolation order of the video block based on the characteristics of the motion vector, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation, and according to the characteristics of the video block Interpolation performs horizontal interpolation and vertical interpolation sequentially to reconstruct the decoded representation of the (2015) video block.

With reference to methods 1900 and 2000, some examples of sequences that perform horizontal interpolation and vertical interpolation and their use are described in section 4 of this document. For example, as described in Chapter 4, under different shapes of video blocks, one of horizontal interpolation or vertical interpolation may be performed first. In some embodiments, horizontal interpolation is performed before vertical interpolation, and in some embodiments, vertical interpolation is performed before horizontal interpolation.

With reference to methods 1900 and 2000, video blocks can be encoded in a video bitstream, where bitstream generation rules related to interpolation order can be used to achieve bit efficiency, which also depends on the shape of the video block.

It should be understood that the disclosed technology can be embedded in a video encoder or decoder to improve compression efficiency when the compressed coding unit has a shape significantly different from a traditional square block or a half-square rectangular block. For example, new coding tools that use long or high coding units such as 4×32 or 32×4 size units can benefit from the disclosed technology.

FIG. 21 is a flowchart of an example of a video processing method 2100. The method 2100 includes: determining (2102) a first prediction mode applied to a first video block; by applying horizontal interpolation and/or vertical interpolation to the first video block, between the first video block and the encoded representation of the first video block Perform (2104) the first conversion; determine (2106) the second prediction mode applied to the second video block; by applying horizontal interpolation and/or vertical interpolation to the second video block, in the second video block and the second video block A second conversion (2108) is performed between encoding representations, where, based on the determination that the first prediction mode is a multi-hypothesis prediction mode and the second prediction mode is not a multi-hypothesis prediction mode, one of the horizontal interpolation and the vertical interpolation of the first video block Or both use a shorter tap filter compared to the filter used for the second video block.

FIG. 22 is a flowchart of a method 2200 of video bitstream processing. The method includes: determining (2205) the shape of the video block; determining (2210) the interpolation order based on the shape of the video block, the interpolation order indicates a sequence of performing horizontal interpolation and vertical interpolation, and performing horizontal interpolation on the video block in the sequence indicated by the interpolation order And vertical interpolation to construct (2215) the coded representation of the video block.

FIG. 23 is a flowchart of a method 2300 of video bitstream processing. The method includes: determining (2305) features of a motion vector related to a video block; determining (2310) an interpolation order based on the features of the motion vector, the interpolation order indicates a sequence of performing horizontal interpolation and vertical interpolation; and in the sequence indicated by the interpolation order Perform horizontal interpolation and vertical interpolation on the video block to construct (2315) an encoded representation of the video block.

The various embodiments and techniques disclosed in this document can be described in the following embodiment list.

1. A video processing method, comprising: determining a first prediction mode applied to a first video block; by applying horizontal interpolation and/or vertical interpolation to the first video block, the first video block and the encoding representation of the first video block Perform the first conversion between; determine the second prediction mode applied to the second video block; by applying horizontal interpolation and/or vertical interpolation to the second video block, between the second video block and the encoded representation of the second video block Perform a second conversion, wherein, based on the determination that the first prediction mode is a multi-hypothesis prediction mode and the second prediction mode is not a multi-hypothesis prediction mode, one or both of the horizontal interpolation and the vertical interpolation of the first video block are used and used for The filter of the second video block is shorter than the tap filter.

2. The method according to example 1, wherein the first video block is converted using more than two reference blocks for bidirectional prediction, and it uses more than two reference blocks for at least one reference picture list.

3. The method according to example 1, wherein the first video block is converted using more than one reference block for unidirectional prediction.

4. The method according to any one of examples 1-3, wherein the shorter tap filter is a bilinear filter.

5. The method according to any one of examples 1-3, wherein one or both of the horizontal interpolation and the vertical interpolation use a shorter tap filter for the reference picture list related to the multiple reference blocks.

6. The method according to any one of examples 1-5, wherein when the reference picture list is related to a single reference block, one or both of the horizontal interpolation or the vertical interpolation uses the same filter as used for the normal prediction mode.

7. The method according to any one of examples 1-6, wherein the method is applied based on the determination of one or more of the following: use of a temporal layer, one or more blocks containing video blocks, slices, slices, or The quantization parameter of the picture is within the threshold range.

8. The method according to example 7, wherein the quantization parameter within the threshold range includes a quantization parameter greater than the threshold.

9. The method according to example 6, wherein the normal prediction mode includes unidirectional prediction or bidirectional prediction inter prediction mode, and unidirectional prediction uses inter prediction with at most one motion vector and one reference index to predict samples of samples in the block Value, the bidirectional prediction inter prediction mode uses inter prediction with at most two motion vectors and a reference index to predict the sample values of the samples in the block.

10. A video decoding device, including a processor, configured to implement the method of one or more of Examples 1 to 9.

11. A video encoding device, including a processor, configured to implement the method of one or more of Examples 1 to 9.

12. A computer-readable program medium on which codes are stored. The codes include instructions that, when the processor executes the instructions, cause the processor to implement one or the methods in Examples 1 to 9.

13. A video bitstream processing method, comprising: determining the shape of a video block; determining an interpolation order based on the shape of the video block, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation; and pairing the video blocks in the sequence indicated by the interpolation order Perform horizontal interpolation and vertical interpolation to reconstruct the decoded representation of the video block.

14. The method according to example 13, wherein the shape of the video block is represented by the width and height of the video block, and the step of determining the interpolation order further includes: When the width of the video block is greater than the height of the video block, it is determined to perform vertical interpolation before horizontal interpolation as the interpolation order.

15. The method according to example 13, wherein the shape of the video block is represented by width and height, and the step of determining the interpolation order further includes: When the width of the video block is greater than or equal to the height of the video block, it is determined to perform vertical interpolation before horizontal interpolation as the interpolation order.

16. The method according to example 13, wherein the shape of the video block is represented by width and height, and the step of determining the interpolation order further includes:

When the height of the video block is greater than or equal to the width of the video block, it is determined to perform horizontal interpolation before vertical interpolation as the interpolation order.

17. The method according to example 1, wherein the shape of the video block is represented by width and height, and the step of determining the interpolation order further includes:

When the height of the video block is greater than the width of the video block, it is determined that horizontal interpolation is performed before vertical interpolation as the interpolation order.

18. The method according to example 1, wherein the luma component and the chroma component of the video block are interpolated based on the interpolation order or based on a different interpolation order.

19. The method according to example 1, wherein when each chroma block of the chroma component corresponds to a plurality of brightness blocks of the brightness component, the brightness component and the chroma component of the video block are interpolated using different interpolation orders.

20. The method according to example 13, wherein the luma component and the chroma component of the video block are interpolated using different interpolation orders, and wherein for the luma component and the chroma component, the scaling factors used in the horizontal interpolation and the vertical interpolation are different.

21. A video bit stream processing method, comprising: determining the characteristics of a motion vector related to a video block; determining an interpolation order based on the characteristics of the motion vector, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation; and instructing by the interpolation order The sequence of performs horizontal interpolation and vertical interpolation on the video block to reconstruct the decoded representation of the video block.

22. The method according to example 21, wherein the feature of the motion vector is represented by a quarter-pixel position and a half-pixel position pointed to by the motion vector, the motion vector includes a vertical component and a horizontal component, and determining the interpolation order includes: when the vertical component points When the quarter pixel position and the horizontal component point to the half pixel position, it is determined that the horizontal interpolation is performed before the vertical interpolation as the interpolation order.

23. The method according to example 21, wherein the feature of the motion vector is represented by a quarter pixel position and a half pixel position pointed to by the motion vector, the motion vector includes a vertical component and a horizontal component, and determining the interpolation order includes: when the vertical component points When the half-pixel position and the horizontal component point to the quarter-pixel position, it is determined to perform vertical interpolation before horizontal interpolation.

24. The method according to any of examples 21-23, wherein the shape of the video block is a square.

25. The method according to any one of examples 21-24, wherein the method is applied in bi-prediction mode.

26. The method according to any one of examples 21-25, wherein the method is applied when the height of the video block multiplied by the width of the video block is less than or equal to T1, and T1 is the first threshold.

27. The method according to any one of examples 21-25, wherein the method is applied when the video block has a height less than or equal to T2, and T2 is the second threshold.

28. The method according to any of examples 21-25, wherein the method is applied to the luminance component of the video block.

29. A video bit stream processing method, comprising:

Determine the shape of the video block;

Determine the interpolation order based on the shape of the video block, the interpolation order indicates the sequence of performing horizontal interpolation and vertical interpolation; and

Horizontal interpolation and vertical interpolation are performed on the video block in the sequence indicated by the interpolation order to construct an encoded representation of the video block.

30. A video bit stream processing method, comprising:

Determine the characteristics of the motion vector related to the video block;

Determine the interpolation order based on the characteristics of the motion vector, the interpolation order indicates the sequence of performing horizontal interpolation and vertical interpolation; and

Horizontal interpolation and vertical interpolation are performed on the video block in the sequence indicated by the interpolation order to construct an encoded representation of the video block.

31. A video decoding device, comprising a processor configured to implement the method of one or more of Examples 21 to 28.

32. A video encoding device, comprising a processor configured to implement the method of example 29 or 30.

33. A computer program product on which computer code is stored, and when the code is executed by a processor, the processor implements the method of any one of Examples 13 to 30.

34. A device in a video system, comprising a processor and a non-transitory storage with instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of Examples 13 to 30.

From the above point of view, it should be understood that, for ease of description, specific embodiments of the technology disclosed in the present invention have been described herein, but various modifications can be made without departing from the scope of the present invention. Therefore, in addition to the above, the technology disclosed in the present invention is not limited to the limitation of the scope of patent application.

The realization and functional operations of the subject names in this patent document can be realized in various systems, digital electronic circuits, or computer software, firmware or hardware, including the structure disclosed in this specification and its structural equivalents, or one of them Or a combination of multiple. The realization of the subject described in this manual can be realized as one or more computer program products, that is, one or more modules of computer program instructions encoded on temporary and non-transitory computer-readable media for data processing The device executes or controls the operation of the data processing device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a storage device, a material composition that affects a machine-readable propagation signal, or a combination of one or more of them. The term "data processing unit" or "data processing device" includes all devices, equipment, and machines for processing data, including, for example, programmable processors, computers, or multi-processors or computer sets. In addition to hardware, the device may also include code for creating an execution environment for computer programs, for example, code that constitutes processor firmware, protocol stack, database management system, operating system, or a combination of one or more of them.

Computer programs (also called programs, software, software applications, scripts or codes) can be written in any form of programming language (including compiled or interpreted language) and can be deployed in any form, including as stand-alone programs or as modules , Components, subprograms or other units suitable for use in a computing environment. Computer programs do not necessarily correspond to documents in the file system. The program can be stored in the part of the document that saves other programs or data (for example, one or more scripts stored in a markup language document), in a single document dedicated to the program, or multiple coordinated documents (for example, storing a Or multiple modules, subprograms or partial code files). A computer program can be deployed on one or more computers to be executed. These computers are located on one website or distributed on multiple websites and are interconnected through a communication network.

The processing and logic flow described in this specification can be executed by one or more programmable processors that execute one or more computer programs that perform functions by operating on input data and generating output. Processing and logic flow can also be performed by special-purpose logic circuits, and the device can also be implemented as special-purpose logic circuits, such as FPGA (Field Programmable Gate Array) or ASIC (dedicated integrated circuit).

For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, and any one or more of any type of digital computer. Generally, the processor will receive commands and data from read-only memory or random access memory or both. The basic components of a computer are a processor that executes instructions and one or more storage devices that store instructions and data. Generally, a computer will also include one or more large storage area devices for storing data, such as floppy disks, magneto-optical disks, or optical discs, or be operatively coupled to one or more large storage area devices to receive data or transfer data from them. Data is transferred to one or more large storage area devices, or both. However, computers do not necessarily have such equipment. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor storage devices such as EPROM, EEPROM, and flash memory devices. The processor and storage can be supplemented by dedicated logic circuits or incorporated into dedicated logic circuits.

The description and drawings are intended to be considered exemplary, where exemplary means example. As used herein, the singular forms "a", "an" and "the" are intended to also include the plural forms, unless the context clearly dictates otherwise. In addition, the use of "or" is intended to include "and/or" unless the context clearly dictates otherwise.

Although this patent document contains many details, it should not be construed as a limitation on the scope of any invention or patent application, but as a description of the features of a particular embodiment of a particular invention. Some features described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various functions described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. In addition, although the above-mentioned features can be described as working in some combinations, even as originally required, in some cases, one or more of the features in the patented combination can be removed from the combination, and the patented Combinations can refer to subcombinations or variants of subcombinations.

Similarly, although the operations are described in a specific order in the drawings, it should not be understood that such operations must be performed in the specific order or sequence shown or all the operations described in order to obtain the desired result. In addition, the separation of various system elements in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

Only some implementations and examples are described, and other implementations, enhancements and variations can be made based on the content described and illustrated in this patent document.

1700: installation 1702: processor 1704: storage 1706: Video processing hardware 1900, 2000, 2100, 2200, 2300: method 1905, 1910, 1915, 2005, 2010, 2015, 2102, 2104, 2106, 2108, 2205, 2210, 2215, 2305, 2310, 2315: steps tb, td: distance

Figure 1 is an illustration of the structure of a quadtree binary tree (QTBT). Figure 2 shows an example derivation process of Merge candidate list construction. Fig. 3 shows example positions of spatial Merge candidates. FIG. 4 shows an example of candidate pairs considered for redundancy check of spatial Merge candidates. FIG. 5 shows an example of the position of the second prediction unit (PU) divided by Nx2N and 2NxN. Fig. 6 is an illustration of the motion vector scaling of the time-domain Merge candidate. Fig. 7 shows example candidate positions of time-domain Merge candidates C0 and C1. Fig. 8 shows an example of combined bidirectional prediction Merge candidates. FIG. 9 shows an example of the derivation process of motion vector prediction candidates. Fig. 10 is an illustration of motion vector scaling of spatial motion vector candidates. FIG. 11 shows an example of advanced temporal motion vector prediction (ATMVP) motion prediction of a coding unit (CU). Fig. 12 shows an example of one CU with four sub-blocks (A-D) and its neighboring blocks (a-d). Figure 13 shows the non-adjacent Merge candidates proposed in J0021. Figure 14 shows the non-adjacent Merge candidates proposed in J0058. Figure 15 shows the non-adjacent Merge candidates proposed in J0059. Figure 16 shows examples of integer sample and fractional sample positions used for quarter-sample luminance interpolation. Fig. 17 is a block diagram of an example of a video processing device. Figure 18 shows a block diagram of an example implementation of a video encoder. Fig. 19 is a flowchart of an example of a video bitstream processing method. Fig. 20 is a flowchart of an example of a video bitstream processing method. Fig. 21 is a flowchart of an example of a video processing method. Fig. 22 is a flowchart of an example of a video bitstream processing method. Fig. 23 is a flowchart of an example of a video bitstream processing method.

2200: method

2205, 2210, 2215: steps

Claims (24)

A video bit stream processing method, including: Determine the shape of the video block; Determining an interpolation order based on the shape of the video block, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation; and The horizontal interpolation and the vertical interpolation are performed on the video block in the sequence indicated by the interpolation order to reconstruct a decoded representation of the video block. The method according to claim 1, wherein the shape of the video block is represented by the width and height of the video block, and the step of determining the interpolation order further includes: When the width of the video block is greater than the height of the video block, it is determined that the vertical interpolation is performed before the horizontal interpolation as the interpolation order. The method according to claim 1, wherein the shape of the video block is represented by the width and height of the video block, and the step of determining the interpolation order further includes: When the width of the video block is greater than or equal to the height of the video block, determining to perform the vertical interpolation before the horizontal interpolation as the interpolation order. The method according to claim 1, wherein the shape of the video block is represented by the width and height of the video block, and the step of determining the interpolation order further includes: When the height of the video block is greater than or equal to the width of the video block, it is determined that the horizontal interpolation is performed before the vertical interpolation as the interpolation order. The method according to claim 1, wherein the shape of the video block is represented by the width and height of the video block, and the step of determining the interpolation order further includes: When the height of the video block is greater than the width of the video block, it is determined that the horizontal interpolation is performed before the vertical interpolation as the interpolation order. The method according to the first item of the scope of patent application, wherein the luminance component and the chroma component of the video block are interpolated based on the interpolation order or based on a different interpolation order. The method according to item 1 of the scope of patent application, wherein, when each chroma block of the chroma component corresponds to a plurality of luma blocks of the luma component, a different interpolation sequence is used for the video block The luma component and the chroma component are interpolated. The method described in item 1 of the scope of patent application, wherein when different interpolation orders are used, the scaling factors used in the horizontal interpolation and the vertical interpolation are different. The method according to item 8 of the scope of patent application, wherein, when the vertical interpolation is performed before the horizontal interpolation, when the bit depth of the video block is less than or equal to 8, all of the vertical interpolation is The scaling factor is equal to 0, and when the bit depth is greater than 8, the scaling factor in the vertical interpolation is greater than zero. The method according to item 8 of the scope of patent application, wherein, when the horizontal interpolation is performed before the vertical interpolation, the scaling factor in the horizontal interpolation is equal to Min(4, BitDepthY-8) and the vertical interpolation The scaling factor in is equal to 6, and where BitDepthY is the bit depth of the video block. A video bit stream processing method, including: Determine the characteristics of the motion vector related to the video block; Determining an interpolation order based on the feature of the motion vector, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation; and The horizontal interpolation and the vertical interpolation are performed on the video block in a sequence indicated by the interpolation order to reconstruct a decoded representation of the video block. The method according to item 11 of the scope of patent application, wherein the feature of the motion vector is represented by a quarter pixel position and a half pixel position pointed to by the motion vector, and the motion vector includes a vertical component and a horizontal component. Components, and determining the interpolation order includes: When the vertical component points to the quarter-pixel position and the horizontal component points to the half-pixel position, it is determined that the horizontal interpolation is performed before the vertical interpolation as the interpolation order. The method according to item 11 of the scope of patent application, wherein the feature of the motion vector is represented by a quarter pixel position and a half pixel position pointed to by the motion vector, and the motion vector includes a vertical component and a horizontal component. Components, and determining the interpolation order includes: When the vertical component points to the half pixel position and the horizontal component points to the quarter pixel position, it is determined to perform the vertical interpolation before the horizontal interpolation. The method according to any one of items 11-13 in the scope of patent application, wherein the shape of the video block is a square. The method according to any one of the claims 1-14, wherein the method is applied to a bi-prediction mode. The method according to any one of items 1-15 of the scope of patent application, wherein, when the height of the video block multiplied by the width of the video block is less than or equal to T1, the method is applied, and T1 is the first Threshold. The method according to any one of items 1-15 in the scope of patent application, wherein, when the video block has a height less than or equal to T2, the method is applied, and T2 is the second threshold. The method according to any one of the claims 1-15, wherein the method is applied to the luminance component of the video block. A video bit stream processing method, including: Determine the shape of the video block; Determining an interpolation order based on the shape of the video block, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation; and The horizontal interpolation and the vertical interpolation are performed on the video block in a sequence indicated by the interpolation order to construct an encoded representation of the video block. A video bit stream processing method, including: Determine the characteristics of the motion vector related to the video block; Determining an interpolation order based on the feature of the motion vector, the interpolation order indicating a sequence of performing horizontal interpolation and vertical interpolation; and The horizontal interpolation and the vertical interpolation are performed on the video block in a sequence indicated by the interpolation order to construct an encoded representation of the video block. A video decoding device includes a processor configured to implement the method described in one or more of items 1 to 18 in the scope of patent application. A video encoding device includes a processor configured to implement the method described in item 19 or 20 of the scope of patent application. A computer program product has computer code stored thereon, and when the code is executed by a processor, the processor realizes the method described in any one of items 1 to 20 of the scope of patent application. A device in a video system, comprising a processor and a non-transitory storage with instructions thereon, wherein the instructions when executed by the processor enable the processor to implement any of the 1 to 20 patent applications The method described in one item.

Images