TW202017370A - Interaction between emm and other tools - Google Patents

Abstract

Methods, devices and systems for using an extended merge mode (EMM) in video coding are described. An exemplary method of video processing includes constructing a list of extended merge mode (EMM) candidates; determining, based on a first set of bits in a bitstream representation of a current block, the motion information inherited by the current block from the list; determining, based on a second set of bits in the bitstream representation of a current block, the signaled motion information of the current block; and performing, based on the list of EMM candidates and the signaled motion information, a conversion between the current block and the bitstream representation, wherein a motion vector difference (MVD) precision of the list of EMM candidates is based on at least one candidate inserted into the list of EMM candidates.

Description

Expand the interaction between the Merge mode and other video encoding tools

This document deals with video encoding technology. [Cross reference to related applications] According to the applicable patent law and/or the rules of the Paris Convention, this application requires the priority and rights of the prior Chinese patent application filed as International Patent Application No. PCT/CN2018/093646 on June 29, 2018. The entire disclosure of this International Patent Application No. PCT/CN2018/093646 is incorporated by reference as part of the disclosure of this application.

Digital video accounts for the largest bandwidth usage on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the demand for bandwidth for digital video usage will continue to grow.

The disclosed technique can be used by a video decoder or encoder embodiment to use the extended Merge mode, where some motion information can be inherited and some motion information can be signaled.

In an example aspect, a video processing method is disclosed. The method includes constructing an extended Merge mode (EMM) candidate list; determining the motion information inherited from the list by the current block based on the first set of bits in the bitstream representation of the current block; based on the bitstream representation of the current block The second set of bits to determine the motion information signaled for the current block; and based on the EMM candidate list and the signaled motion information to perform the conversion between the current block and the bitstream representation, where the EMM candidate list The accuracy of the motion vector difference (MVD) is based on at least one candidate inserted into the EMM candidate list.

In another example aspect, the above method may be implemented by a video decoder device that includes a processor.

In another example aspect, the above method may be implemented by a video transcoder device including a processor for decoding encoded video during a video encoding process.

In yet another example aspect, these methods may be implemented in the form of processor-executable instructions and stored on a computer-readable program medium.

These and other aspects are further described in this document.

This document provides various techniques that can be used by decoders of video bitstreams to improve the quality of decompressed or decoded digital video. In addition, the video transcoder can also implement these techniques during the encoding process in order to reconstruct the decoded frames for further encoding.

For ease of understanding, section titles are used in this file, but the embodiments and techniques are not limited to the corresponding sections. In this way, embodiments from one section can be combined with embodiments from other sections. 2 . Technical framework

Video coding standards are mainly evolved through the development of well-known ITU-T and ISO/IEC standards. ITU-T produced H.261 and H.263 standards, ISO/IEC produced MPEG-1 and MPEG-4 Visual standards, and the two organizations jointly produced H.262/MPEG-2 video standards and H.264/ MPEG-4 Advanced Video Coding (Advanced Video Coding, AVC) standard and H.265/HEVC [00130] standard. Since H.262, the video coding standard is based on a hybrid video coding structure, in which temporal prediction plus transform coding is used. To explore future video coding technologies beyond HEVC, the Joint Video Exploration Team (JVET) was co-founded by VCEG and MPEG in 2015. Since then, JVET has adopted many new methods and incorporated them into a reference software called Joint Exploration Model (JEM) [1][2]. In April 2018, the Joint Video Expert Team (JVET) between VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11 (MPEG) was created to work on the VVC standard with the goal of collaborating with HEVC This reduces the bit rate by 50%. 2.1 interframe prediction in HEVC / H. 265 in

Each inter-predicted PU has motion parameters for one or two reference picture lists. Motion parameters include motion vectors and reference image indexes. You can also use inter_pred_idc to signal the use of one of the two reference image lists. The motion vector can be explicitly encoded as an increment relative to the predictor.

When encoding a CU using a skip mode, one PU is associated with the CU, and there are no significant residual coefficients, and there is no encoded motion vector increment or reference image index. The Merge mode is specified, thereby obtaining motion parameters for the current PU from neighboring PUs, including spatial and temporal candidates. Merge mode can be applied to any inter-predicted PU, not just skip mode. An alternative to the Merge mode is explicit transmission of motion parameters, where the motion vector (more precisely, the motion vector difference compared to the motion vector predictor), the corresponding reference image for each reference image list The index and reference picture list usage are explicitly signaled by each PU. This mode is named Advanced Motion Vector Prediction (AMVP) in this file.

When signaling indicates that one of the two reference picture lists is to be used, the PU is generated from a sample block. This is called "one-way prediction". One-way prediction can be used for P-band and B-band.

When signaling indicates that two reference image lists are to be used, the PU is generated from two sample blocks. This is called "bidirectional prediction". Bidirectional prediction is only available for B bands.

The following text provides details of the inter prediction modes specified in HEVC. The description will start in Merge mode. 2.1.1 Merge mode 2.1.1.1 Derivation of Merge mode candidates

When the PU is predicted using the Merge mode, an index pointing to the entries in the Merge candidates list is parsed from the bit stream, and the index is used to retrieve motion information. The construction of this list is specified in the HEVC standard and can be summarized according to the following sequence of steps: Step 1: Derivation of initial candidates Step 1.1: Spatial candidate derivation Step 1.2: Spatial candidate redundancy check Step 1.3: Time candidate derivation Step 2: Additional candidate insertion Step 2.1: Create bidirectional prediction candidates Step 2.2: Insert zero motion candidates

These steps are also depicted schematically in Figure 1. For the spatial Merge candidate derivation, a maximum of four Merge candidates are selected among the candidates located in five different positions. For temporal Merge candidate derivation, at most one Merge candidate is selected from the two candidates. Since the number of candidates per PU is assumed to be constant at the decoder, when the number of candidates obtained from step 1 does not reach the maximum number of Merge candidates (MaxNumMergeCand) signaled in the slice header, additional candidates are generated . Since the number of candidates is constant, binary unary truncation (TUB) is used to encode the index of the best Merge candidate. If the size of the CU is equal to 8, all PUs of the current CU share a single Merge candidate list, which is the same as the Merge candidate list of the 2N×2N prediction unit.

In the following, the operations associated with the above steps are described in detail. 2.1.1.2 Derivation of spatial candidates

In the derivation of spatial Merge candidates, a maximum of four Merge candidates are selected among the candidates located in the positions depicted in FIG. 2. The order of derivation is A ₁ , B ₁ , B ₀ , A ₀ and B ₂ . Position B is only considered when any PU at position A ₁ , B ₁ , B ₀ , A0 is not available (for example, because the PU belongs to another slice or tile) or is intra-coded ₂ . After the addition of the candidate position at A _1, of the remaining candidate is added redundancy check, CRC to ensure that the candidate has the same motion information is excluded from the list, thereby improving coding efficiency. In order to reduce the computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check. On the contrary, only the pair connected with the arrow in FIG. 3 is considered, and if the corresponding candidate for redundancy check has different motion information, the candidate is only added to the list. Another source of repetitive motion information is the "second PU " associated with a division other than 2Nx2N. As an example, FIG. 4 depicts the second PU for the cases of N×2N and 2N×N, respectively. When the current PU is divided into N×2N, the candidate at position A ₁ is not considered for list construction. In fact, adding this candidate will result in two prediction units with the same motion information, which is redundant for having only one PU in the coding unit. Similarly, when the current PU is divided into 2N×N, the position B _{1 is} not considered. 2.1.1.3 Time candidate derivation

In this step, only one candidate is added to the list. In particular, in the derivation of the Merge candidate at this time, a scaled motion vector is derived based on a co-located PU, which belongs to the current image with respect to a given reference image list The image with the smallest POC difference. The deduced reference picture list for the co-located PU is explicitly signaled in the slice header. As shown by the dotted line in FIG. 5, a scaled motion vector for the temporal Merge candidate is obtained, which is scaled from the motion vector of the co-located PU using the POC distances tb and td, where tb is defined as the reference image of the current image The POC difference from the current image, and td is defined as the POC difference between the reference image of the co-located image and the co-located image. The reference image index of the temporal Merge candidate is set equal to zero. The actual implementation of the scaling process is described in the HEVC specification [00130]. For the B slice, two motion vectors are obtained and combined to generate a bidirectional prediction Merge candidate, one of the two motion vectors is used for reference image list 0 and the other is used for reference image list 1.

As shown in FIG. 6, in the co-located PU(Y) belonging to the reference frame, the position for the temporal candidate is selected between the candidates C ₀ and C ₁ . If the PU at position C ₀ is unavailable, intra-coded, or outside the current CTU line, then position C ₁ is used. Otherwise, the position C ₀ is used in the derivation of the time Merge candidate. 2.1.1.4 Additional candidate insertion

In addition to space and time Merge candidates, there are two additional types of Merge candidates: combined bidirectional prediction Merge candidates and zero Merge candidates. The combined bidirectional prediction Merge candidates are generated by using spatial and temporal Merge candidates. The combined bidirectional prediction Merge candidate is only used for B bands. The combined bidirectional prediction candidate is generated by combining an initial candidate's first reference image list motion parameter and another candidate's second reference image list motion parameter. If these two tuples provide different motion hypotheses, they will form a new bidirectional prediction candidate. As an example, FIG. 7 depicts the case when two candidates with mvL0 and refIdxL0 or mvL1 and refIdxL1 in the original list (on the left) are used to create a combined bidirectional prediction Merge candidate, the combined bidirectional prediction Merge candidate is added To the final list (on the right). There are many rules regarding the combination considered to generate these additional Merge candidates, which are defined in [00130].

Zero motion candidates are inserted to fill the remaining entries in the Merge candidate list, and thus reach the MaxNumMergeCand capacity. These candidates have zero spatial displacement and a reference image index that starts at zero and increases each time a new zero motion candidate is added to the list. The number of reference frames used by these candidates is 1 and 2 for unidirectional and bidirectional prediction, respectively. Finally, no redundancy check is performed on these candidates. 2.1.1.5 Motion estimation area for parallel processing

In order to speed up the encoding process, motion estimation can be performed concurrently, thereby simultaneously deriving motion vectors of all prediction units in a given area. The derivation of Merge candidates from spatial neighborhoods may interfere with parallel processing because a prediction unit cannot export motion parameters from neighboring PUs until its associated motion estimation is complete. In order to reduce the trade-off between coding efficiency and processing latency, HEVC defines a motion estimation area (MER). The size of the motion estimation area is signaled using the "log2_parallel_merge_level_minus2" syntax element [00130] in the image parameter set. When MER is defined, Merge candidates that fall into the same area are marked as unavailable, and therefore are not considered in the list construction. 2.1.2 AMVP

AMVP utilizes the spatiotemporal correlation of motion vectors with neighboring PUs, which is used for explicit transmission of motion parameters. For each reference image list, a motion vector candidate list is first constructed by checking the availability of temporally adjacent PU positions on the left and above, removing redundant candidates, and adding a zero vector to make the candidate list It is a constant length. The encoder can then select the best predictor from the candidate list and transmit the corresponding index indicating the selected candidate. Similar to Merge index signaling, binary unary truncation is used to encode the index of the best motion vector candidate. The maximum value to be encoded in this case is 2 (see Figure 8). In the following section, details about the derivation process of motion vector prediction candidates are provided. 2.1.2.1 Derivation of AMVP candidates

FIG. 8 summarizes the derivation process for motion vector prediction candidates.

In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. As shown in FIG. 2, for the derivation of spatial motion vector candidates, two motion vector candidates are finally derived based on the motion vectors of each PU located at five different positions.

For temporal motion vector candidate derivation, one motion vector candidate is selected from two candidates derived based on two different co-located positions. After making the first list of spatiotemporal candidates, the duplicate motion vector candidates in the list are removed. If the number of potential candidates is greater than 2, the motion vector candidates whose reference image index is greater than 1 in the associated reference image list are removed from the list. If the number of spatiotemporal motion vector candidates is less than 2, additional zero motion vector candidates are added to the list. 2.1.2.2 Spatial motion vector candidates

In the derivation of spatial motion vector candidates, a maximum of two candidates are considered among five potential candidates from PUs located at positions shown in FIG. 2, which are the same as those of motion Merge. The derivation order on the left side of the current PU is defined as A ₀ , A ₁ and scaled A ₀ , scaled A ₁ . The derivation order of the upper side of the current PU is defined as B ₀ , B ₁ , B ₂ , scaled B ₀ , scaled B ₁ , scaled B ₂ . Therefore, for each side, there are four cases that can be used as motion vector candidates, two of which do not need to use spatial scaling, and two use spatial scaling. The four different situations are summarized below. • No spatial scaling (1) Same reference image list, and same reference image index (same POC) (2) Different reference image list, but same reference image index (same POC) • Spatial Zoom (3) the same reference image list, but different reference image index (different POC) (4) different reference image list, and different reference image index (different POC)

First verify that there is no spatial scaling, and then verify the spatial scaling. Regardless of the reference image list, when the POC is different between the reference image of the neighboring PU and the reference image of the current PU, spatial scaling is considered. If all PUs of the left candidate are unavailable or are intra-coded, the upper motion vector is allowed to be scaled to help the parallel derivation of the left and upper MV candidates. Otherwise, spatial scaling is not allowed for the upper motion vector.

As shown in FIG. 9, during spatial scaling, the motion vectors of neighboring PUs are scaled in a similar manner to temporal scaling. The main difference is that the reference picture list and index of the current PU are given as input; the actual scaling process is the same as the time scaling process. 2.1.2.3 Temporal motion vector candidates

Except for the reference image index derivation, all processes used for the derivation of temporal Merge candidates are the same as those used for the derivation of spatial motion vector candidates (see FIG. 6). The reference picture index is signaled to the decoder. 2.2 New inter prediction method in JEM 2.2.1 Self-adjusting motion vector difference resolution

In HEVC, when use_integer_mv_flag in the slice header is equal to 0, the motion vector difference (MVD) between the motion vector of the PU and the predicted motion vector is signaled in units of quarter luminance samples. In JEM, local self-adjusting motion vector resolution (LAMVR) was introduced. In JEM, MVD can be encoded in units of quarter brightness samples, integer brightness samples, or four brightness samples. The MVD resolution is controlled at the coding unit (CU) level, and to each CU having at least one non-zero MVD component, the MVD resolution flag is conditionally signaled.

For a CU with at least one non-zero MVD component, the first flag is signaled to indicate whether to use a quarter luminance sample MV accuracy in the CU. When the first flag (equal to 1) indicates that the quarter brightness sample MV precision is not used, another flag is signaled to indicate whether to use integer brightness sample MV precision or four brightness sample MV precision.

When the CU's first MVD resolution flag is zero, or is not coded for the CU (meaning that all MVDs in the CU are zero), the quarter brightness sample MV resolution is used for the CU. When the CU uses integer luminance sample MV accuracy or four luminance sample MV accuracy, the MVP in the AMVP candidate list of the CU is rounded to the corresponding accuracy.

In the encoder, a CU level RD check is used to determine which MVD resolution will be used for the CU. In other words, for each MVD resolution, a CU level RD check is performed three times. In order to speed up the encoder, the following encoding scheme is applied in JEM. • During the RD check of a CU with a normal quarter-luminance sample MVD resolution, store the current CU motion information (integer luminance sample accuracy). For the same CU with integer brightness samples and MVD resolution of 4 brightness samples, the stored motion information (after rounding) is used as the starting point for further small-scale motion vector refinement during RD verification, making time-consuming motion The estimation process is not repeated three times. • Conditionally call the RD check of the CU with 4 brightness samples MVD resolution. For the CU, when the RD cost of the MVD resolution of the integer brightness samples is much greater than the RD cost of the MVD resolution of the quarter brightness samples, the RD check of the MVD resolution of the 4 brightness samples of the CU is skipped. 2.2.2 Higher motion vector storage accuracy

In HEVC, the accuracy of motion vectors is a quarter of a pixel (pel) (for quarter brightness samples and eighth chrominance samples for 4:2:0 video). In JEM, the accuracy of internal motion vector storage and Merge candidates is increased to 1/16 primitive. Higher motion vector accuracy (1/16 picture element) is used for motion-compensated inter prediction of CUs encoded in skip mode/Merge mode. For CUs encoded in normal AMVP mode, integer primitive or quarter primitive motion is used, as described in section 2.2.1.

The SHVC upsampling interpolation filter with the same filter length and normalization factor as the HEVC motion compensation interpolation filter is used as the motion compensation interpolation filter with additional fractional primitive positions. The accuracy of the chrominance component motion vector in JEM is 1/32 samples. By using the average value of the filters of two adjacent 1/16 primitive fractional positions, additional interpolation filtering of 1/32 primitive fractional positions is derived. Device. 2.2.3 Local brightness compensation

Local brightness compensation (LIC) is based on a linear model for brightness changes, using a scaling factor a and an offset b . And, the coding unit (CU) coded for each inter mode self-adjusts to enable or disable LIC.

When the LIC is applied to the CU, the least square error method is adopted, and the parameters a and b are derived by using the neighboring samples of the current CU and their corresponding reference samples. More specifically, as shown in FIG. 10, the adjacent samples of the sub-sampling (2:1 sub-sampling) of the CU and the corresponding samples in the reference image (which are identified by the motion information of the current CU or sub-CU ). The IC parameters are exported and applied to each prediction direction separately.

When encoding the CU using the Merge mode, the LIC flag is copied from the adjacent block in a manner similar to the motion information copying in the Merge mode; otherwise, the CU flag is signaled to the CU to indicate whether the LIC is applicable.

When LIC is enabled for an image, an additional CU level RD check is required to determine whether to apply LIC to the CU. When LIC is enabled for CU, use integer-memory-removed sum of absolute diffefference (MR-SAD) and mean-removed absolute Hadamard for integer primitive motion search and fractional primitive motion search respectively The mean-removed sum of absolute Hadamard-transformed difference (MR-SATD) is not SAD and SATD.

In order to reduce the coding complexity, the following coding scheme is applied in JEM.

When there is no obvious brightness change between the current image and its reference image, LIC is disabled for the entire image. To identify this situation, at the encoder, a histogram of the current picture and each reference picture of the current picture is calculated. If the histogram difference between the current image and each reference image of the current image is less than a given threshold, LIC is disabled for the current image; otherwise, LIC is enabled for the current image. 2.2.4 Affine motion compensation prediction

In HEVC, only translational motion models are applied to motion compensated prediction (MCP). In the real world, there are many kinds of movement, such as zoom in/out, rotation, perspective movement, and other irregular movements. In JEM, simplified affine transformation motion compensation prediction is applied. As shown in Fig. 11, the affine motion field of the block is described by two control point motion vectors.

(1) 其中（v_0x ,v_0y ）是左頂角控制點的運動向量，（v_1x ,v_1y ）是右頂角控制點的運動向量。The motion vector field (MVF) of the block is described by the following equation:

(1) where ( v _0x , v _0y ) is the motion vector of the left top corner control point, and ( v _1x , v _1y ) is the motion vector of the right top corner control point.

如等式（2）。中匯出，其中MvPre 是運動向量分數精度（在JEM中為1/16），（v_2x ,v_2y ）是左下控制點的運動向量，根據等式（1）計算。

(2)In order to further simplify motion compensation prediction, sub-block-based affine transformation prediction is applied. Subblock size

Such as equation (2). Exported in, where MvPre is the motion vector fractional accuracy (1/16 in JEM), ( v _2x , v _2y ) is the motion vector of the lower left control point, calculated according to equation (1).

(2)

After exporting from equation (2), if necessary, M and N should be adjusted downwards so that they are the divisors of w and h, respectively.

As shown in FIG. 12, in order to derive the motion vector of each M×N sub-block, the motion vector of the center sample of each sub-block is calculated according to equation (1) and rounded to 1/16 fractional accuracy. Then, a motion compensation interpolation filter is applied to generate the prediction of each sub-block using the exported motion vector.

After the MCP, the high-precision motion vector of each sub-block is rounded and saved with the same precision as the normal motion vector.

的候選列表。如圖13所示，從塊A、B或C的運動向量中選擇

。來自相鄰塊的運動向量根據參考清單以及根據相鄰塊的參考的POC、當前CU的參考的POC和當前CU的POC之間的關係來縮放。並且從相鄰塊D和E中選擇

的方法是類似的。如果候選列表的數量小於2，則由通過重複每個AMVP候選而構建的運動向量對來填充該列表。當候選清單大於2時，首先根據相鄰運動向量的一致性（候選對中的兩個運動向量的相似性）對候選進行分類，並且僅保留前兩個候選。RD成本校驗用於確定選擇哪個運動向量對候選作為當前CU的控制點運動向量預測（CPMVP）。並且，在位元流中信令通知指示候選列表中的CPMVP的位置的索引。在確定當前仿射CU的CPMVP之後，應用仿射運動估計，並找到控制點運動向量（CPMV）。然後在位元流中信令通知CPMV與CPMVP的差異。In JEM, there are two affine motion modes: AF_INTER mode and AF_MERGE mode. For CUs with a width and height greater than 8, the AF_INTER mode can be applied. The CU-level affine flag is signaled in the bit stream to indicate whether to use AF_INTER mode. In this mode, use neighboring blocks to construct pairs with motion vectors

Candidate list. As shown in Figure 13, choose from the motion vectors of block A, B or C

. The motion vectors from adjacent blocks are scaled according to the reference list and the relationship between the referenced POC of the adjacent block, the referenced POC of the current CU, and the current CU. And choose from neighboring blocks D and E

The method is similar. If the number of candidate lists is less than 2, the list is populated by the pair of motion vectors constructed by repeating each AMVP candidate. When the candidate list is greater than 2, the candidates are first classified according to the consistency of the adjacent motion vectors (similarity of the two motion vectors in the candidate pair), and only the first two candidates are retained. The RD cost check is used to determine which motion vector pair is selected as the control point motion vector prediction (CPMVP) of the current CU. And, an index indicating the position of CPMVP in the candidate list is signaled in the bit stream. After determining the CPMVP of the current affine CU, apply affine motion estimation and find the control point motion vector (CPMV). The difference between CPMV and CPMVP is then signaled in the bit stream.

、

和

。並且根據

、

和

來計算當前CU的左頂角的運動向量

。其次，計算當前CU的右上方的運動向量

。When the CU is applied in the AF_MERGE mode, it obtains the first block encoded using the affine mode from valid neighboring reconstruction blocks. As shown in FIG. 14A, and the selection order for the candidate blocks is from left, above, right above, below left to above left. As shown in FIG. 14B, if the adjacent lower left block A is encoded in affine mode, the motion vectors of the top left corner, top right corner, and bottom left corner of the CU containing the block A are exported

,

with

. And according to

,

with

To calculate the motion vector of the top left corner of the current CU

. Second, calculate the motion vector at the upper right of the current CU

.

和

之後，根據簡化的仿射運動模型等式（1），生成該當前CU的MVF。為了識別當前CU是否使用AF_MERGE模式編碼，當存在至少一個相鄰塊以仿射模式編碼時，在位元流中信令通知仿射標誌。2.2.5 模式匹配的運動向量推導 Export the CPMV of the current CU

with

After that, according to the simplified affine motion model equation (1), the MVF of the current CU is generated. In order to identify whether the current CU uses the AF_MERGE mode encoding, when there is at least one adjacent block encoded in the affine mode, the affine flag is signaled in the bit stream. 2.2.5 Motion vector derivation of pattern matching

Pattern matched motion vector derivation (PMMVD) mode is a special Merge mode based on Frame-Rate Up Conversion (FRUC) technology. With this mode, the motion information of the block is not signaled, but is exported on the decoder side.

When the CU's Merge flag is true, the CU is signaled to notify the FRUC flag. When the FRUC flag is false, the Merge index is signaled and the normal Merge mode is used. When the FRUC flag is true, the additional FRUC mode flag is signaled to indicate which method (bilateral matching or template matching) will be used to export the motion information of the block.

On the encoder side, the decision about whether to use the FRUC Merge mode for the CU is based on the RD cost selection made as for the normal Merge candidate. In other words, two matching modes (bilateral matching and template matching) of the CU are verified by using RD cost selection. The matching mode that results in the smallest cost is further compared with other CU modes. If the FRUC matching mode is the most effective mode, the FRUC flag is set to true for the CU, and the relevant matching mode is used.

The motion derivation process in FRUC Merge mode has two steps. CU-level motion search is performed first, followed by sub-CU-level motion refinement. At the CU level, the initial motion vector is exported for the entire CU based on bilateral matching or template matching. First, a MV candidate list is generated, and the candidate that results in the smallest matching cost is selected as the starting point for further CU level refinement. Then, a local search based on bilateral matching or template matching is performed around the starting point, and the MV that results in the smallest matching cost is taken as the MV of the entire CU. Subsequently, the motion information is further refined at the sub-CU level, where the exported CU motion vector is used as a starting point.

CU運動資訊推導執行以下推導處理。在第一階段，匯出整體

CU的MV。在第二階段，CU進一步劃分為

子CU。如（3）中計算

的值，

是預定義的劃分深度，其在JEM中默認設置為3。然後匯出每個子CU的MV。

}                        (3)For example, for

The CU motion information derivation performs the following derivation processing. In the first stage, export the whole

CU's MV. In the second stage, CU is further divided into

Sub CU. As calculated in (3)

'S value,

It is a predefined division depth, which is set to 3 by default in JEM. Then export the MV of each sub-CU.

} (3)

As shown in FIG. 15, bilateral matching is used to derive the motion information of the current CU by finding the closest match between the two blocks along the motion trajectory of the current CU in two different reference images. Under the assumption of continuous motion trajectories, the motion vectors MV0 and MV1 pointing to the two reference blocks should be proportional to the time distance between the current picture and the two reference pictures, namely TD0 and TD1. As a special case, when the current image is temporally between two reference images and the time distance from the current image to the two reference images is the same, bilateral matching becomes a mirror-based bidirectional MV.

As shown in Figure 16, template matching is used to find the template in the current image (the top adjacent block and/or the left adjacent block in the current CU) and the block in the reference image (having the same size as the template) The closest match between them to export the current CU sports information. In addition to the above FRUC Merge mode, template matching is also applicable to AMVP mode. In JEM, as in HEVC, AMVP has two candidates. Use the template matching method to export new candidates. If the newly exported candidate matched by the template is different from the first existing AMVP candidate, it is inserted at the very beginning of the AMVP candidate list, and then the list size is set to 2 (this means that the second existing AMVP candidate is removed). When applied to AMVP mode, only CU level search is applied. 2.2.5.1 CU level MV candidate set

The CU level MV candidate set consists of the following: (i) If the current CU is in AMVP mode, it is the original AMVP candidate, (ii) All Merge candidates, (iii) Interpolate several MVs in the MV domain, (iv) The motion vector adjacent to the top and left.

When bilateral matching is used, each valid MV of the Merge candidate is used as an input to generate MV pairs assuming bilateral matching. For example, a valid MV of the Merge candidate is in the reference list A (MVa, refa). Then, the reference image refb of its paired bilateral MV is found in the other reference list B, so that refa and refb are located on different sides of the current picture in time. If such a refb in the reference list B is not available, the refb is determined to be a different reference than the refa, and the time distance of the refb to the current image is the minimum value in the list B. After determining refb, MVb is exported by scaling MVa based on the time distance between the current image and refa, refb.

The four MVs from the interpolation MV domain are also added to the CU level candidate list. More specifically, the interpolation MV at the positions (0, 0), (W/2, 0), (0, H/2), and (W/2, H/2) of the current CU is added.

When FRUC is applied in AMVP mode, the original AMVP candidate is also added to the CU level MV candidate set.

At the CU level, a maximum of 15 MVs for AMVP CU and a maximum of 13 MVs for Merge CU are added to the candidate list. 2.2.5.2 Sub- CU level MV candidate set

The MV candidate set at the sub-CU level consists of the following: (i) Search the determined MV from the CU level, (ii) Adjacent MVs at the top, left, top and right, (iii) A scaled version of the parallel MV from the reference image, (iv) Up to 4 ATMVP candidates, (v) Up to 4 STMVP candidates.

The scaled MV from the reference image is exported as follows. Iterate through all the reference images in the two lists. The MV at the parallel position of the sub-CUs in the reference image is scaled to the reference of the starting CU level MV.

ATMVP and STMVP candidates are limited to the first four.

At the sub-CU level, a maximum of 17 MVs are added to the candidate list. 2.2.5.3 Generation of interpolated MV domain

Before encoding the frame, an interpolated motion domain is generated for the entire image based on the single-sided ME. The motion domain can then be used as a CU level or sub-CU level MV candidate later.

First, the motion domain of each reference image in the two reference lists is traversed at the 4×4 block level. For each 4×4 block, if the motion associated with the block passes through the 4×4 block in the current image (as shown in FIG. 17) and the block has not been assigned any interpolation motion, the motion of the reference block is based on the temporal distance TD0 and TD1 (the same way as the MV scaling of TMVP in HEVC) scale to the current image, and the scaled motion is assigned to the block in the current frame. If the unscaled MV is assigned to a 4x4 block, the motion of the block is marked as unavailable in the interpolated motion domain. 2.2.5.4 Interpolation and matching costs

When the motion vector points to the position of the fractional sample point, motion compensation interpolation is required. To reduce complexity, both bilateral matching and template matching use bilinear interpolation instead of the conventional 8-tap HEVC interpolation.

：

(4) 其中

是一個加權因數，且根據經驗設置為4，

和

分別指示當前MV和起始MV。SAD仍用作子CU級別搜索的範本匹配的匹配成本。The calculation of matching costs is a bit different in different steps. When selecting candidates from the CU-level candidate set, the matching cost is the sum of absolute differences (SAD) of bilateral matching or template matching. After the starting MV is determined, the matching cost of the bilateral matching for sub-CU level search is calculated as follows

:

(4) where

Is a weighting factor and is set to 4 according to experience,

with

Indicate the current MV and the start MV respectively. SAD is still used as the matching cost of template matching for sub-CU level search.

In the FRUC mode, MV is exported by using only luminance samples. The exported motion will be used for the luma and chroma of MC inter prediction. After determining the MV, the final MC is performed using an 8-tap interpolation filter for luma and a 4-tap interpolation filter for chroma. 2.2.5.5 MV refinement

MV refinement is a pattern-based MV search based on the criteria of bilateral matching cost or template matching cost. In JEM, two search modes are supported-unrestricted center-biased diamond search (UCBDS) and adaptive cross search (adaptive cross search) for MV-level and sub-CU-level MV refinement search). For CU-level and sub-CU-level MV refinement, the MV is directly searched with 1/4 brightness sample point MV accuracy, and then VIII refinement sample point MV refinement. The search range for MV refinement for the CU step and the sub-CU step is set equal to 8 luminance samples. 2.2.5.6 Template matching FRUC Merge mode prediction direction selection

In the bilateral matching Merge mode, bidirectional prediction is always applied because the CU motion information is derived based on the closest match between two blocks along the motion trajectory of the current CU in two different reference images. Template matching Merge mode does not have such restrictions. In the template matching Merge mode, the encoder can choose among unidirectional prediction from list 0, unidirectional prediction from list 1, or bidirectional prediction for the CU. Choose the matching cost based on the template as follows: if costBi >= factor * min ( cost 0, cost1 ) use bidirectional prediction; otherwise, if cost 0 >= cost1 use unidirectional prediction from list 0; otherwise, use the list from list 1 Prediction; where cost0 is the SAD matched by the template in list 0, cost1 is the SAD matched by the template in list 1, and costBi is the SAD matched by the bidirectional prediction template. The value of factor is equal to 1.25, which means that the selection process is biased towards two-way prediction.

The inter prediction direction selection is only applied to the CU level template matching process. 2.2.6 Refinement of motion vector on decoder side

In the bidirectional prediction operation, in order to predict one block region, two prediction blocks formed using the motion vector (MV) of list 0 and the MV of list 1 are combined to form a single prediction signal. In the decoder-side motion vector refinement (DMVR) method, the two motion vectors for bidirectional prediction are further refined through a bilateral template matching process. Bilateral template matching is applied in the decoder to perform a distortion-based search between the bilateral template and the reconstructed samples in the reference image in order to obtain a refined MV without transmitting additional motion information.

As shown in FIG. 18, in the DMVR, a bilateral template is generated from the initial MV0 of list 0 and MV1 of list 1, respectively, as a weighted combination (ie, average) of two prediction blocks. The template matching operation includes calculating the cost metric between the generated template and the sample area (around the initial prediction block) in the reference image. For each of the two reference images, the MV that produces the smallest template cost is considered the updated MV of the list to replace the original template. In JEM, for each list, nine MV candidates are searched. The nine MV candidates include the original MV and 8 surrounding MVs, one of which is shifted to the original MV in the horizontal or vertical direction or in both directions. Finally, as shown in Figure 18, two new MVs, MV0' and MV1', are used to generate the final bidirectional prediction result. The sum of absolute differences (SAD) is used as a cost metric. Please note that when calculating the cost of a prediction block generated by a surrounding MV, the rounded MV (to an integer primitive) is actually used instead of the real MV to obtain the prediction block.

DMVR is applied to the Merge mode of bidirectional prediction, using one MV from a reference image in the past and another MV from a reference image in the future without transmitting additional syntax elements. In JEM, when LIC, affine motion, FRUC, or sub-CU Merge candidates are enabled for the CU, DMVR is not applied. 2.3 Non-adjacent Merge candidates

In J0021, Qualcomm recommends that additional spatial Merge candidates are pooled from non-adjacent adjacent locations. These non-adjacent adjacent locations are marked as 6 to 49, as shown in Figure 19. The exported candidates are added after the TMVP candidates in the Merge candidate list.

In J0058, Tencent suggested to export additional spatial Merge candidates from the position in the external reference area, the offset of the external reference area from the current block is (-96, -96).

As shown in FIG. 20, the position markers are A(i,j), B(i,j), C(i,j), D(i,j), and E(i,j). Compared to its previous B or C candidate, each candidate B(i, j) or C(i, j) has an offset of 16 in the vertical direction. Compared with its previous A or D candidate, each candidate A(i, j) or D(i, j) has an offset of 16 in the horizontal direction. Compared to its previous E candidates, each E (i, j) has an offset of 16 in the horizontal and vertical directions. Candidates are checked from the inside out. And the order of candidates is A(i, j), B(i, j), C(i, j), D(i, j), and E(i, j). Further study whether the number of Merge candidates can be further reduced. Add candidates after TMVP candidates in the Merge candidate list.

In J0059, the extended space positions from 6 to 27 in FIG. 21 are checked according to the order of digits after the time candidate. In order to save the MV line buffer, all spatial candidates are limited to two CTU lines. 2.4 Related methods

The ultimate motion vector expression (UMVE) in J0024 can be skip mode or direct (or merge) mode, which uses the proposed motion vector expression method using adjacent motion information. As the skip mode and Merge mode in HEVC, UMVE also makes a candidate list based on adjacent motion information. Among those candidates in the list, the MV candidate is selected and further expanded by the new motion vector expression method.

FIG. 22 shows an example of the UMVE search process, and FIG. 23 shows an example of the UMVE search point.

UMVE provides a new motion vector representation with simplified signaling. The expression method includes a starting point, a movement amplitude and a movement direction.

The base candidate index defines the starting point. The basic candidate index indicates the best candidate among the candidates in the list as follows.

The distance index is motion amplitude information. The distance index indicates a predefined distance from the starting point information. The predefined distance is shown below (pel in the table represents the primitive).

3 ．現有實現方式的缺點的討論 The direction index indicates the direction of the MVD relative to the starting point. The direction index can represent four directions, as shown below.

3 . Discussion of the shortcomings of existing implementations

In the Merge mode, the motion information of the Merge candidate is inherited by the current block, including the motion vector, reference image, prediction direction, and LIC flag. Only the Merge index is signaled, which is efficient in many cases. However, the inherited motion information, especially motion vectors, may not be good enough.

On the other hand, in the AMVP mode, all motion information is signaled, including motion vectors (ie MVP index and MVD), reference pictures (ie reference index), prediction direction, LIC flag and MVD accuracy, etc., which consumes bits .

In the UMVE suggested by JVET-J0024, it is recommended to encode the additional MVD. However, the MVD can only have non-zero components in the horizontal direction or the vertical direction, and cannot have non-zero components in both directions. At the same time, it also signals MVD information, namely distance index or motion amplitude information. 4 . Method for extending Merge mode ( EMM ) based on disclosed technology

Video transcoder and decoder embodiments can use the techniques disclosed in this document to implement the extended Merge mode (EMM), where only very little information is signaled and there are no special restrictions on MVD.

The following detailed inventions should be considered as examples explaining general concepts. These inventions should not be interpreted in a narrow manner. Furthermore, these inventions can be combined in any way.

It is recommended to divide motion information (such as prediction direction, reference index/image, motion vector, LIC flag, affine flag, intra block copy (IBC) flag, MVD accuracy, MVD value) into two parts. The first part is directly inherited, and the second part is explicitly signaled with or without predictive coding.

It is recommended to construct an EMM list and signal the index to indicate which candidate motion information the current block (eg, PU/CU) inherits the first part of. At the same time, additional signaling such as MVD (ie, the second part of sports information) is further signaled. a. The first part of the motion information includes all or some of the following information: prediction direction, reference image, motion vector, LIC flag, MVD accuracy, etc. b. The second part can be coded using predictive coding.

It is recommended to construct a motion information candidate list by inserting motion information of spatially adjacent blocks, temporally adjacent blocks or non-adjacent blocks.

Instead, the predicted direction is not inherited but explicitly signaled. In this case, it is recommended to build two or more candidate lists of sports information.

It is recommended that the MVD accuracy be inherited from adjacent blocks and stored in the Merge mode. a. When inserting the combined bidirectional prediction Merge candidate, the MVD accuracy is set to the highest supported accuracy (for example, 1/4) by default. i. Instead, set the MVD accuracy to the lower or higher accuracy of the two candidates involved. ii. Alternatively, if more than one combined bidirectional prediction Merge candidate is inserted, different MVD accuracy is used for different combined bidirectional prediction candidates. iii. Instead, set the MVD accuracy to the most frequent MVD accuracy in the Merge candidate list. iv. Instead, set the MVD accuracy to any valid MVD accuracy. v. The rounding operation can be applied to achieve this operation. b. When inserting zero Merge candidates, the MVD accuracy is set to the highest supported accuracy (for example, 1/4) by default. i. Alternatively, if more than one zero Merge candidate is inserted, different MVD accuracy is used for different zero candidates. ii. Instead, set the MVD accuracy to the most frequent MVD accuracy in the Merge candidate list. iii. Instead, set the MVD accuracy to any valid MVD accuracy. It is recommended that all zero MVDs can be disabled in EMM mode. a. In one-way prediction, when the horizontal MVD component is zero, the zero flag is not signaled for the vertical MVD component and is implicitly exported as false. b. In bidirectional prediction, if the MVD of L0 and the level L1 MVD component are zero, the zero flag is not signaled for the vertical L1 MVD component and is implicitly exported as false.

EMM mode can work with DMVR or template matching. In this case, some or all candidates are further refined by DMVR or template matching. a. In one example, FRUC is not further refined.

Signal whether the EMM mode is applied from the encoder to the decoder. For example, the selection can be signaled in the following ways: video parameter set (VPS), sequence parameter set (SPS), image parameter set (PPS), slice header, coding tree unit (CTU), coding tree block ( CTB), coding unit (CU) or prediction unit (PU), an area covering multiple CTU/CTB/CU/Pu.

The proposed method can be applied to certain block sizes/shapes and/or certain sub-block sizes. a. When the proposed method is applied only under certain conditions and the required conditions are not met, there is no need to signal the EMM's instructions or call the motion candidate list construction process as described above.

For example, EMM can only be applied to blocks where w×h≥T, where w and h are the width and height of the current block. In another example, EMM can only be applied to blocks w≥T&&h≥T.

The above example can be incorporated in the context of the method described below (eg, method 2400), which can be implemented in a video decoder or video transcoder.

24 is a flowchart of an example method 2400 of processing a video bitstream. The method 2400 includes constructing (2402) an EMM candidate list; determining (2404) the motion information inherited from the list by the current block based on the first set of bits in the bitstream representation of the current block; based on the bits of the current block The second set of bits in the stream representation to determine (2406) the motion information signaled by the current block; and perform (2408) the current block and bit stream representation based on the EMM candidate list and the signaled motion information Conversion between, where the accuracy of the motion vector difference (MVD) of the EMM candidate list is based on at least one candidate inserted into the EMM candidate list.

The examples listed below provide embodiments that can solve the technical problems and other problems described in this document.

1. A video bitstream processing method, including: constructing an extended Merge mode (EMM) candidate list; determining the motion information inherited from the list by the current block based on the first set of bits in the bitstream representation of the current block; based on the current The second set of bits in the bitstream representation of the block to determine the signaled motion information of the current block; and based on the EMM candidate list and the signaled motion information to perform between the current block and the bitstream representation Conversion, where the motion vector difference (MVD) accuracy of the EMM candidate list is based on at least one candidate inserted into the EMM candidate list.

2. The method according to example 1, wherein at least one candidate includes a bidirectional prediction Merge candidate that is merged from the first candidate and the second candidate.

3. The method according to example 2, wherein the MVD accuracy is the lower accuracy of the first candidate or the second candidate.

4. The method according to example 2, wherein the MVD accuracy is the higher accuracy of the first candidate or the second candidate.

5. The method according to example 1, wherein at least one candidate includes a zero Merge candidate.

6. The method according to Example 2 or 5, wherein the MVD accuracy is the highest supported accuracy.

7. The method according to Example 2 or 5, wherein the MVD accuracy is arbitrary accuracy.

8. The method according to example 1, wherein the MVD accuracy is inherited from adjacent blocks.

9. The method according to example 1, wherein at least one candidate includes a plurality of bidirectional prediction Merge candidates.

10. The method according to example 1, wherein at least one candidate includes a plurality of zero Merge candidates.

11. The method according to 9 or 10, wherein the MVD accuracy is different for each of the at least one candidate.

12. The method according to example 9 or 10, wherein the MVD accuracy is set to the most frequent MVD accuracy of at least one candidate.

13. The method according to any one of examples 1 to 12, wherein all zero MVD candidates are excluded from the EMM candidate list and/or video encoding and/or video decoding.

14. The method according to Example 13, further comprising: when the level MVD is zero, inferring the zero mark of the vertical motion component in the unidirectional prediction mode.

15. The method according to Example 13, further comprising: when the MVD of list 0 (L0) and the level MVD of list 1 (L1) are zero, infer the MVD of list 1 (L1) to zero.

16. The method according to example 1, wherein performing the conversion includes a decoder-side motion vector refinement (DMVR) process or template matching.

17. The method according to any one of Examples 1 to 16, wherein the method is selectively used based on the characteristics of the current block, and wherein the characteristics include the size of the current block or the shape of the current block.

18. According to the method of Example 17, the characteristics include that the size of the current block is greater than the threshold.

19. An apparatus in a video system includes a processor and a non-transitory memory having instructions thereon, where the instructions, when executed by the processor, cause the processor to implement the method of any one of Examples 1 to 18.

20. A computer program product stored on a non-transitory computer readable medium. The computer program product includes program code for implementing the method of any one of Examples 1 to 18. 5 . references

ITU-T and ISO/IEC, "High efficiency video coding", Rec. ITU-T H.265|ISO/IEC 23008-2 (valid version).

C. Rosewarne, B. Bross, M. Naccari, K. Sharman, G. Sullivan, "High Efficiency Video Coding (HEVC) Test Model 16 (HM 16) Improved Encoder Description Update 7," JCTVC-Y1002, October 2016 .

J. Chen, E. Alshina, G.J. Sullivan, J.-R. Ohm, J. Boyce, "Algorithm description of Joint Exploration Test Model 7 (JEM7)", JVET-G1001, August 2017.

JEM-7.0: https://jvet.hhi.fraunhofer.de/svn/svn_HMJEMSoftware/tags/ HM-16.6-JEM-7.0.

A. Alshin, E. Alshina, etc., "Description of SDR, HDR and 360° video coding technology proposal by Samsung, Huawei, GoPro, and HiSilicon – mobile application scenario", JVET-J0024, April 2018. 6 . Embodiments of the disclosed technology

25 is a block diagram of the video processing device 2500. The apparatus 2500 may be used to implement one or more methods described herein. The device 2500 may be implemented in smartphones, tablets, computers, Internet of Things (IoT) receivers, and so on. The device 2500 may include one or more processors 2502, one or more memories 2504, and video processing circuits 2506. The processor(s) 2502 may be configured to implement one or more methods (including but not limited to method 2400) described in this document. The memory(s) 2504 may be used to store materials and codes for implementing the methods and techniques described herein. The video processing circuit 2506 can be used to implement some of the techniques described in this document in a hardware circuit.

In some embodiments, the video encoding method may be implemented using a device implemented on a hardware platform as described with respect to FIG. 25.

The disclosure and other solutions, examples, embodiments, modules, and functional operations described in this document can be implemented by digital electronic circuits, or by computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents Thing, or a combination of one or more of them. The disclosed and other embodiments may be implemented as one or more computer program products, that is, one or more computer program instruction modules encoded on a computer-readable medium, for execution by or control of the operation of the data processing device by the data processing device . The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a material combination that affects a machine-readable propagation signal, or a combination of one or more of them. The term "data processing device" encompasses all devices, equipment, and machines used to process data, including, for example, a programmable processor, computer, or multiple processors or computers. In addition to the hardware, the device may also include code to create an execution environment for the computer program in question, for example, the processor firmware, protocol stack, database management system, operating system, or one or more of them Combined code. Propagated signals are artificially generated signals, such as machine-generated electrical signals, optical signals, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device.

Computer programs (also called programs, software, software applications, scripts, or codes) can be written in any form of programming language, including compiled or interpreted languages, and computer programs can be deployed in any form, including as standalone programs or as suitable Modules, components, subroutines, or other units used in computing environments. Computer programs do not necessarily correspond to files in the file system. Programs can be stored in part of a file that holds other programs or data (for example, one or more scripts stored in a markup language file), in a single file dedicated to the program in question, or in multiple coordination Files (for example, files that store one or more modules, subprograms, or code sections). Computer programs can be deployed to execute on one computer or on multiple computers located on one website or distributed across multiple websites and interconnected by a communication network.

The processes and logic flows described in this document can be executed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The process and logic flow can also be performed by dedicated logic circuits, and the device can also be implemented as dedicated logic circuits, such as FPGA (field programmable gate array) or ASIC (dedicated integrated circuit).

For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, the processor will receive commands and data from read-only memory or random access memory or both. The basic components of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, the computer will also include or be operatively coupled to one or more large storage area devices for storing data, such as magnetic disks, magneto-optical disks, or optical discs, to receive data from the one or more large storage area devices, or Transfer data to the one or more large storage area devices, or both receive and transfer data. However, the computer does not need to have such equipment. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices ; Diskettes, such as internal hard drives or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory can be supplemented by or incorporated into dedicated logic circuits.

Although this patent document contains many details, these details should not be construed as limitations on the scope of any inventions or of what may be claimed, but as descriptions of features specific to particular embodiments of particular inventions. In this patent document, certain features described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although the above features may be described as functioning in certain combinations and even claimed as such initially, in some cases, one or more features from the claimed combination may be removed from the claimed combination and the claimed Protected combinations can point to sub-combinations or variations of sub-combinations.

Similarly, although the operations are depicted in a specific order in the drawings, this should not be construed as requiring such operations to be performed in the specific order shown or in order, or to perform all the operations shown to achieve the desired result . Furthermore, the separation of various system elements in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and shown in this patent file.

2400: Methods 2402, 2404, 2406, 2408: Step 2500: Video processing device 2502: Processor 2504: Memory 2506: Video processing circuits A, B, C, D, E: Blocks A ₀ , A ₁ , B ₀ , B ₁ , B ₂ , C ₀ , C ₁ : position L0: list 0 L1: list 1 MV0, MV0', MV1, MV1': motion vector tb, td: POC distance TD0, TD1: time distance

FIG. 1 shows an example of a derivation process for Merge candidate list construction. Figure 2 shows an example location of spatial Merge candidates. FIG. 3 shows an example of candidate pairs considering the redundancy check of spatial Merge candidates. 4A and 4B show example positions of the second PU divided by N×2N and 2N×N. FIG. 5 is an example illustration of motion vector scaling for temporal Merge candidates. FIG. 6 shows an example of candidate positions of time Merge candidates C0 and C1. FIG. 7 shows an example of combined bidirectional prediction Merge candidates. FIG. 8 shows an example derivation process for motion vector prediction candidates. FIG. 9 shows an example illustration of motion vector scaling for spatial motion vector candidates. FIG. 10 shows an example of adjacent samples for exporting IC parameters. FIG. 11 shows an example of a simplified affine motion model. FIG. 12 shows an example of affine MVF for each sub-block. Fig. 13 shows an example of MVP of AF_INTER. 14A and 14B show examples of candidates for AF_MERGE. FIG. 15 shows an example of bilateral matching. Fig. 16 shows an example of template matching. FIG. 17 shows an example of unidirectional ME in FRUC. FIG. 18 shows an example of DMVR based on bilateral template matching. FIG. 19 shows an example of non-adjacent Merge candidates. FIG. 20 shows an example of non-adjacent Merge candidates. FIG. 21 shows an example of non-adjacent Merge candidates. 22 and 23 depict examples of final motion vector expression techniques for video encoding. 24 is a flowchart of an example of a video bit stream processing method. 25 is a block diagram of an example of a video processing device.

2400: Method

2402, 2404, 2406, 2408: steps

Claims (20)

A video bit stream processing method, including: Build an extended Merge mode (EMM) candidate list; Determine the motion information inherited by the current block from the list based on the first set of bits in the bitstream representation of the current block; Determining the signaled motion information of the current block based on the second set of bits in the bitstream representation of the current block; and Perform conversion between the current block and the bitstream representation based on the EMM candidate list and the signaled motion information, Wherein the accuracy of the motion vector difference (MVD) of the EMM candidate list is based on at least one candidate inserted into the EMM candidate list. The method as described in item 1 of the patent application scope, wherein the at least one candidate includes a bidirectional predictive Merge candidate derived from the first candidate and the second candidate. The method according to item 2 of the patent application scope, wherein the MVD accuracy is the lower accuracy of the first candidate or the second candidate. The method according to item 2 of the patent application scope, wherein the MVD accuracy is the higher accuracy of the first candidate or the second candidate. The method according to item 2 of the patent application scope, wherein the at least one candidate includes a zero Merge candidate. The method as described in item 2 or 5 of the patent application scope, wherein the MVD accuracy is the highest supported accuracy. The method according to item 2 or item 5 of the patent application scope, wherein the MVD accuracy is arbitrary accuracy. The method as described in item 1 of the patent application scope, wherein the MVD accuracy is inherited from adjacent blocks. The method according to item 1 of the patent application scope, wherein the at least one candidate includes a plurality of bidirectional prediction Merge candidates. The method according to item 1 of the patent application scope, wherein the at least one candidate includes a plurality of zero Merge candidates. The method according to item 9 or item 10 of the patent application scope, wherein the MVD accuracy is different for each of the at least one candidate. The method according to item 9 or item 10 of the patent application scope, wherein the MVD accuracy is set to the most frequent MVD accuracy of the at least one candidate. The method according to any one of items 1 to 12 of the patent application scope, wherein all zero MVD candidates are excluded from the EMM candidate list and/or video encoding and/or video decoding. The method as described in item 13 of the patent application scope further includes: When the level MVD is zero, the zero mark of the vertical motion component is inferred in the unidirectional prediction mode. The method as described in item 13 of the patent application scope further includes: When the MVD of list 0 (L0) and the level MVD of list 1 (L1) are zero, the MVD of list 1 (L1) is inferred to be zero. The method according to item 1 of the patent application scope, wherein performing the conversion includes a decoder-side motion vector thinning (DMVR) process or template matching. The method according to any one of claims 1 to 16, wherein the method is selectively used based on the characteristics of the current block, and wherein the characteristics include the size of the current block or The shape of the current block. As in the method described in item 17 of the patent application scope, the characteristic includes that the size of the current block is greater than a threshold. An apparatus in a video system, including a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement patent application items 1 to 18 The method as described in any one. A computer program product stored on a non-transitory computer readable medium, the computer program product including program code for implementing the method described in any one of items 1 to 18 of the patent application.

Images