TWI719522B - Symmetric bi-prediction mode for video coding - Google Patents

Abstract

A video bitstream processing method comprising generating, in response to a mirror mode flag in the video bitstream, a second motion vector difference information based on a symmetry rule and a first motion vector difference information; and reconstructing a video block using the first motion vector difference and the second motion vector difference information, wherein the reconstruction is performed bi-predictively.

Description

Symmetrical bidirectional prediction mode for video coding

This document deals with image and video coding technology. [Cross references to related applications] In accordance with applicable patent laws and/or in accordance with the rules of the Paris Convention, this application promptly claims the priority and rights of international patent application No. PCT/CN2018/093897 filed on June 30, 2018. The entire disclosure of International Patent Application No. PCT/CN2018/093897 is incorporated into a part of the disclosure of this application by reference.

Digital video occupies the largest bandwidth usage on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the demand for bandwidth for digital video usage will continue to grow.

The disclosed technology can be used by visual media decoder or encoder embodiments, where the symmetry of motion vectors is used to reduce the bits used for signaling motion information to improve coding efficiency.

In an exemplary aspect, a video bitstream processing method is disclosed. The method includes generating second motion vector difference information based on the symmetry rule and the first motion vector difference information in response to the mirror mode flag in the video bitstream. The method further includes using the first motion vector difference information and the second motion vector difference to reconstruct the video block in the current picture, wherein the reconstruction is performed using bidirectional prediction.

In another exemplary aspect, another video bitstream processing method is disclosed. The method includes receiving motion vector difference information of a first set of motion vectors for a first reference picture list associated with a video block. The method further includes using multiple hypothesis symmetry rules to derive the motion vector difference information associated with the second set of motion vectors of the second reference picture list associated with the video block from the motion vector difference information of the first set of motion vectors, wherein the multiple hypotheses The symmetry rule specifies that the second motion vector difference value is (0, 0), and the corresponding motion vector predictor is set to the mirror motion vector value derived from the first motion vector difference information, and uses the derived result to perform video block summation The bitstream representations of video blocks are converted between.

In another exemplary aspect, another video bitstream processing method is disclosed. The method includes, for a video block, receiving first motion vector difference information associated with a first reference picture list. The method also includes for the video block, receiving second motion vector difference information associated with the second reference picture list, and using multiple hypothesis symmetry rules to derive the difference information from the first motion vector difference information and the second motion vector difference information. The third motion vector difference information associated with the reference picture list and the fourth motion vector difference information associated with the second reference picture list, where the symmetry rule specifies that the second motion vector difference is (0, 0), and The corresponding motion vector predictor is set to the mirror motion vector value derived from the first motion vector difference information.

In another exemplary aspect, another video bitstream processing method is disclosed. The method includes receiving a future frame of the video relative to a reference frame of the video, receiving a motion vector related to the future frame of the video and the past frame of the video, applying a predetermined relationship between the future frame of the video and the past frame of the video, and based on The future frame of the video, the motion vector, and the predetermined relationship between the past frame of the video and the future frame of the video reconstruct the past frame of the video.

In another example aspect, the above-described method may be implemented by a video decoder device including a processor.

In another example aspect, the above-described method may be implemented by a video transcoder device that includes a processor for decoding the encoded video during the video encoding process.

In yet another example aspect, these methods may be embodied in the form of processor-executable instructions and stored on a computer-readable program medium.

These and other aspects are further described in this document.

The chapter titles are used in this document to facilitate understanding, and the embodiments disclosed in the chapters are not limited to this chapter. In this way, embodiments from one chapter can be combined with embodiments from other chapters. In addition, although certain embodiments have been described with reference to specific video transcoders, the disclosed techniques are also applicable to other video encoding techniques. In addition, although some embodiments describe the video encoding step in detail, it should be understood that the corresponding decoding step of de-encoding will be implemented by the decoder. In addition, the term video processing encompasses video encoding or compression, video decoding or decompression, and video transcoding, in which video pixels are expressed from one compression format to another compression format or at different compression bit rates.

This document provides various techniques that can be used by decoders of video bitstreams to improve the quality of decompressed or decoded digital video. In addition, video transcoders can also implement these techniques during the encoding process in order to reconstruct decoded frames for further encoding. Signaling notification of bidirectional prediction in HEVC

In HEVC, inter PU-level signaling can be divided into three different modes. Table 1 and Table 2 show related syntax elements for inter-PU signaling in HEVC. The first mode is the skip mode, in which only a single Merge index (merge_idx) needs to be signaled. The second mode is the Merge mode, in which only the Merge flag (merge_flag) and the Merge index (merge_idx) are signaled. The third mode is the AMVP mode, in which signalling direction index (inter_pred_idc), reference index (ref_idx_l0/ref_idx_l1), mvp index (mvp_l0_flag/mvp_l1_flag) and MVD (mvd_coding).

Among all the three modes, the bidirectional predictive AMVP mode provides a more rate-consuming situation, while it provides the freedom to capture various motion models, including acceleration and other non-linear motion models. The motion vectors of the two lists are separately signaled to provide this degree of freedom. AMVP in HEVC derives motion vector prediction in AMVP mode

表2.HEVC中MVD編碼的語法元素

運動向量預測候選 Motion vector prediction utilizes the spatio-temporal correlation with the motion vectors of neighboring PUs, which is used for explicit transmission of motion parameters. It constructs the motion vector candidate list by first checking the availability of the adjacent PU positions on the left and upper in time, removing redundant candidates and adding zero vectors to make the candidate list a constant length. The encoder can then select the best predictor from the candidate list and send a corresponding index indicating the selected candidate. Similarly, using Merge index signaling, truncated unary is used to encode the index of the best motion vector candidate. The maximum value to be coded in this case is 2. In the following sections, details on the derivation process of motion vector prediction candidates are provided. Table 1. Inter PU Syntax Elements in HEVC

Table 2. Syntax elements of MVD encoding in HEVC

Motion vector prediction candidate

Figure 1 summarizes the derivation process of motion vector prediction candidates.

In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. For the derivation of spatial motion vector candidates, two motion vector candidates are finally derived based on the motion vector of each PU located at five different positions, as shown in FIG. 2.

For the derivation of temporal motion vector candidates, one motion vector candidate is selected from two candidates, which are derived based on two different co-located positions. After the first list of spatiotemporal candidates is made, the repeated motion vector candidates in the list are removed. If the number of potential candidates is greater than two, the motion vector candidates whose reference picture index is greater than 1 in the related reference picture list are removed from the list. If the number of spatiotemporal motion vector candidates is less than two, additional zero motion vector candidates are added to the list. Spatial motion vector candidate

In the derivation of the spatial motion vector candidates, at most two candidates are considered among the five potential candidates, which are derived from the PU located at the positions shown in FIG. 2, and those positions are the same as the positions of the motion merge. The derivation order on the left side of the current PU is defined as A0, A1, scaled A0, and scaled A1. The derivation order on the upper side of the current PU is defined as B0, B1, B2, scale B0, scale B1, and scale B2. Therefore, for each side, there are four cases that can be used as motion vector candidates, of which two cases do not need to use spatial scaling, and two cases use spatial scaling. The four different situations are summarized as follows. No space zoom (1) The same reference picture list and the same reference picture index (same picture order count (POC)) (2) Different reference picture lists, but the same reference picture index (same POC) Space zoom (3) The same reference picture list, but different reference picture indexes (different POC) (4) Different reference picture lists, and different reference picture indexes (different POCs)

First verify the non-spatial scaling, and then verify the spatial scaling. When the POC between the reference picture of the adjacent PU and the reference picture of the current PU is different, spatial scaling is considered regardless of the reference picture list. If all the PUs of the left candidate are unavailable or are intra-coded, the scaling for the above-mentioned motion vector can help the parallel derivation of the left and upper MV candidates. Otherwise, the aforementioned motion vector does not allow spatial scaling.

In the spatial scaling process, the motion vectors of adjacent PUs are scaled in a similar manner to the temporal scaling, as shown in FIG. 3. The main difference is that the reference picture list and index of the current PU are given as input; the actual scaling process is the same as the time scaling process. Temporal motion vector candidate

Except for reference picture index derivation, all processes for deriving temporal Merge candidates are the same as those for deriving spatial motion vector candidates. Signal the reference picture index to the decoder. Merge mode in HEVC Candidates for Merge mode

When using the Merge mode to predict the PU, the index pointing to the entry in the Merge candidate list is parsed from the bit stream and used to retrieve motion information. The construction of this list is stipulated in the HEVC standard, and can be summarized according to the following sequence of steps: Step 1: Initial candidate derivation -Step 1.1: Spatial candidate derivation -Step 1.2: Redundancy check for spatial candidates -Step 1.3: Time candidate derivation Step 2: Additional candidate insertion -Step 2.1: Two-way prediction candidate creation -Step 2.2: Zero motion candidate insertion

These steps are also schematically depicted in FIG. 4. For the derivation of spatial Merge candidates, at most four Merge candidates are selected from candidates located at five different positions. For the temporal Merge candidate derivation, at most one Merge candidate is selected from the two candidates. Since the number of candidates for each PU is assumed to be constant at the decoder, when the number of candidates does not reach the maximum number of Merge candidates (MaxNumMergeCand) signaled in the slice header, additional candidates are generated. Since the number of candidates is constant, truncated unary binarization (TU) is used to encode the index of the best Merge candidate. If the size of the CU is equal to 8, all PUs of the current CU share a single Merge candidate list, which is the same as the Merge candidate list of the 2N×2N prediction unit.

In the following subsections, the detailed operation of each of the above steps is described. Spatial candidate

In the pushing of spatial Merge candidates, at most four Merge candidates are selected from the candidates located at the positions shown in FIG. 2. The order of derivation is A1, B1, B0, A0, and B2. Position B2 is considered only when any PU at positions A1, B1, B0, A0 is not available (for example, because it belongs to another slice or tile) or is intra-coded. After the candidate at position A1 is added, the addition of the remaining candidates undergoes a redundancy check, which ensures that candidates with the same motion information are excluded from the list, thereby improving coding efficiency. In order to reduce the computational complexity, all possible candidate pairs are not considered in the mentioned redundancy check. Instead, only the pair connected with the arrow in FIG. 5 is considered, and only when the corresponding candidate used for redundancy check does not have the same motion information, the candidate is added to the list. Another source of repetitive motion information is the "second PU" associated with a partition other than 2Nx2N. As an example, FIG. 6 depicts the second PU in the case of N×2N and 2N×N, respectively. When the current PU is divided into N×2N, the candidate at position A1 is not considered for list construction. In fact, adding this candidate will cause two prediction units to have the same motion information, which is redundant for having only one prediction unit in the coding unit. Similarly, when the current PU is divided into 2N×N, the position B1 is not considered. Time candidate

In this step, only one candidate is added to the list. Specifically, in the derivation of this temporal Merge candidate, the scaling motion vector is derived based on the collocated PU belonging to the picture with the smallest POC difference from the current picture in the given reference picture list. The reference picture list of the PU to be used to derive the co-bit is explicitly signaled in the slice header. As shown by the dashed line in Figure 7, the scaled motion vector of the temporal Merge candidate is obtained. The motion vector is scaled from the motion vector of the co-located PU using the POC distance, tb, and td, where tb is defined as the reference picture of the current picture and the current The POC difference between pictures, and td is defined as the POC difference between the reference picture of the co-bit picture and the co-bit picture. The reference picture index of the temporal Merge candidate is set equal to zero. The actual implementation of the scaling process is described in the HEVC specification. For the B slice, two motion vectors are obtained and combined, one of the two motion vectors is used for reference picture list 0 (list0), and the other is used for reference picture list 1 (list1) to form a bi-directional predictive Merge candidate.

In the co-located PU (Y) belonging to the reference frame, the position of the temporal candidate is selected between the candidates C0 and C1, as shown in FIG. 8. If the PU at position C0 is not available, is intra-coded, or is outside the current coding tree unit (coding tree unit, CTU), position C1 is used. Otherwise, position C0 is used to derive temporal Merge candidates. Additional candidate insertion

In addition to spatiotemporal Merge candidates, there are two other types of Merge candidates: combined bidirectional predictive Merge candidates and zero Merge candidates. The combined bidirectional predictive Merge candidate is generated by using spatio-temporal Merge candidates. The combined bi-directional prediction Merge candidate is only used for the B band. The combined bi-directional prediction candidate is generated by combining the first reference picture list motion parameter of the initial candidate with another second reference picture list motion parameter. If these two tuples provide different motion hypotheses, they will form a new bi-prediction candidate. As an example, Figure 9 depicts the situation when there are two candidates in the original list (on the left), which have mvL0 and refIdxL0 or mvL1 and refIdxL1, which are used to create the combined bi-directional predictive Merge candidates (on the right) added to the final list. There are many rules regarding the combination that is considered to generate these additional Merge candidates.

Insert zero motion candidates to fill the remaining entries in the Merge candidate list to reach the MaxNumMergeCand capacity. These candidates have a zero spatial displacement and a reference picture index, which starts at zero and increases every time a new zero motion candidate is added to the list. The number of reference frames used by these candidates is one and two for unidirectional and bidirectional prediction, respectively. Finally, no redundancy check is performed on these candidates. Pattern matching motion vector

Pattern matched motion vector derivation (PMMVD) mode is a special Merge mode based on Frame-Rate Up Conversion (FRUC) technology. Using this mode, the motion information of the block is not signaled, but derived on the decoder side.

When the Merge flag of the CU is true, the FRUC flag is notified for the CU signaling. When the FRUC flag is false, the Merge index is signaled and the regular Merge mode is used. When the FRUC flag is true, the additional FRUC mode flag is signaled to indicate which method (bilateral matching or template matching) will be used to derive the motion information of the block.

On the encoder side, the decision of whether to use the FRUC Merge mode for the CU is based on the cost of research and development (cost) selection, just like the normal Merge candidate. That is to say, the two matching modes of CU (bilateral matching and template matching) are checked by using RD cost selection. The one that causes the least cost is further compared with other CU models. If the FRUC matching mode is the most effective mode, the FRUC flag is set to true for the CU, and the relevant matching mode is used.

The motion derivation process in FRUC Merge mode has two steps. First, perform CU-level motion search, and then perform sub-CU-level motion refinement. At the CU level, the initial motion vector is derived for the entire CU based on bilateral matching or template matching. First, a list of MV candidates is generated, and the candidate that leads to the smallest matching cost is selected as the starting point for further CU-level refinement. Then, a local search based on bilateral matching or template matching around the starting point is performed, and the MV that results in the smallest matching cost is taken as the MV of the entire CU. Subsequently, the motion information is further refined at the sub-CU level, and the derived CU motion vector is used as the starting point.

For example, the following export processing is executed for W×H CU motion information export. In the first stage, the MV of the entire W×H CU is derived. In the second stage, the CU is further divided into M×M sub-CUs. For example, the value of M is calculated in (1), D is the predefined division depth, which is set to 3 by default in JEM. Then export the MV of each sub-CU.

As shown in FIG. 10, bilateral matching is used to derive the motion information of the current CU by finding the closest match between two blocks along the motion trajectory of the current CU in two different reference pictures. Under the assumption of continuous motion trajectories, the motion vectors MV0 and MV1 pointing to the two reference blocks should be proportional to the time distance between the current picture and the two reference pictures (ie, TD0 and TD1). As a special case, when the current picture is between two reference pictures in time and the time distance from the current picture to the two reference pictures is the same, the bilateral matching becomes a mirror-based two-way MV.

As shown in Figure 11, template matching is used to derive by finding the closest match between the template in the current picture (the top and/or left adjacent block of the current CU) and the block in the reference picture (the same size as the template) Sports information of the current CU. In addition to FRUC Merge mode, template matching is also applicable to AMVP mode. In JEM, there are two AMVP candidates. Use template matching methods to derive new candidates. If the newly derived candidate matched by the template is different from the first existing AMVP candidate, it is inserted into the very beginning of the AMVP candidate list, and then the list size is set to 2 (meaning the second existing AMVP candidate is removed). When applied to AMVP mode, only CU-level search is applied. CU -level MV candidate set

The MV candidates set at the CU level include: -If the current CU is in AMVP mode, it is the original AMVP candidate -All Merge candidates, -Interpolate several MVs in the MV field. -Top and left adjacent motion vectors

When using bilateral matching, each valid MV of the Merge candidate is used as input to generate MV pairs under the assumption of bilateral matching. For example, a valid MV of the Merge candidate is located at the reference list a (MVa, refa). Then, find the reference picture refb of its paired bilateral MV in another reference list B, so that refa and refb are located on different sides of the current picture in time. If there is no such refb in the reference list B, the refb is determined to be a reference different from refa, and its time distance to the current picture is the smallest in the list B. After refb is determined, MVb is derived by scaling MVa based on the time distance between the current picture and refa and refb.

Four MVs from the interpolated MV field are also added to the CU-level candidate list. More specifically, the interpolation MVs at the positions (0, 0), (W/2, 0), (0, H/2), and (W/2, H/2) of the current CU are added.

When FRUC is applied to the AMVP mode, the original AMVP candidate is also added to the CU-level MV candidate set.

At the CU level, AMVP Cu at most 15 MVs and Merge Cu at most 13 MVs are added to the candidate list. Sub-CU-level MV candidate set

The MV candidate set set at the CU level includes: -MV determined from CU-level search, -Adjacent MVs at the top, left, top left, and top right corners, -A zoomed version of the co-bit MV from the reference picture, -Up to 4 ATMVP candidates, -Up to 4 STMVP candidates

The zoomed MV from the reference picture is derived as follows. Go through all the reference pictures in the two lists. The MV at the co-located position of the sub-CU in the reference picture is scaled to the reference of the starting CU-level MV.

ATMVP and STMVP candidates are limited to the first four.

At the sub-CU level, up to 17 MVs are added to the candidate list. Generation of interpolated MV field

Before encoding the frame, an interpolated motion field is generated for the entire picture based on the single-sided ME. Then, the sports field can be used as a CU-level or sub-CU-level MV candidate later.

First, the motion field of each reference picture in the two reference lists is traversed at a 4×4 block level. For each 4×4 block, if the motion associated with the block passes through a 4×4 block in the current picture (as shown in Figure 12) and the block is not assigned any interpolation motion, the motion of the reference block is based on the time distance TD0 Scaling to the current picture with TD1 (the same way as the MV scaling of TMVP in HEVC), and assigns the scaled motion to the block in the current frame. If the scaled MV is not assigned to a 4×4 block, the motion of the block is marked as unavailable in the interpolated motion field. Interpolation and matching costs

When the motion vector points to the fractional sample position, motion compensation interpolation is required. In order to reduce complexity, bilinear interpolation replaces conventional 8-tap HEVC interpolation for bilateral matching and template matching.

（等式2）The calculation of the matching cost is slightly different in different steps. When selecting candidates from the candidate set at the CU level, the matching cost is the sum of absolute difference (SAD) of bilateral matching or template matching. After the initial MV is determined, the matching cost of the bilateral match in the sub-CU-level search is calculated as follows:

(Equation 2)

是根據經驗設置為4的加權因數，

和

分別表示當前MV和起始MV。SAD仍然用作子CU級搜索的範本匹配的匹配成本。among them,

Is a weighting factor set to 4 based on experience,

with

Represents the current MV and the start MV respectively. SAD is still used as the matching cost for template matching in sub-CU-level searches.

In FRUC mode, MV is derived only by using luminance samples. The derived motion will be used for both luma and chroma for MC inter prediction. After MV is decided, the final MC is performed using an 8-tap interpolation filter for luma and a 4-tap interpolation filter for chroma. MV refinement

MV refinement is a pattern-based MV search, which has a standard of bilateral matching cost or template matching cost. In JEM, two search modes are supported-unrestricted center-biased diamond search (UCBDS) and self-adjusted cross search with MV refinement at the CU level and the sub-CU level respectively. For both CU and sub-CU-level MV refinement, the MV is directly searched with quarter-luminance sample MV accuracy, and then one-eighth luminance sample MV refinement follows. The search range for MV refinement for the CU and sub-CU steps is set equal to 8 luma samples. Select the prediction direction in the template matching FRUC Merge mode

In the bilateral matching Merge mode, bidirectional prediction is always applied because the motion information of the CU is derived based on the closest match between two blocks along the motion trajectory of the current CU in two different reference pictures. There is no such restriction for template matching Merge mode. In the template matching Merge mode, the encoder can choose from the unidirectional prediction of List 0, the unidirectional prediction of List 1, or the bidirectional prediction of CU. The selection is based on the template matching cost, as shown below: If costBi >= factor * min ( cost 0, cost1 ) use bidirectional prediction; otherwise, if cost 0 >= cost1 use unidirectional prediction from list 0; otherwise, use from list 1 one-way forecast;

Where cost0 is the SAD matched by the template in Listing 0, cost1 is the SAD matched by the template in Listing 1, and costBi is the SAD matched by the bidirectional prediction template. The value of factor is equal to 1.25, which means that the selection process is biased towards bidirectional prediction.

Inter-frame prediction direction selection is only applied to CU-level template matching processing. Decoder side motion vector refinement

In the bidirectional prediction operation, for the prediction of a block region, two prediction blocks formed using the motion vector (MV) of List 0 and the MV of List 1 are combined to form a single prediction signal. In the decoder-side motion vector refinement (DMVR) method, the two motion vectors of bidirectional prediction are further refined through a bilateral template matching process. Bilateral template matching is applied in the decoder to perform a distortion-based search between the reconstructed samples in the bilateral template and the reference picture to obtain a refined MV without transmitting additional motion information.

In DMVR, a bilateral template is generated from the initial MV0 of list 0 and MV1 of list 1 as a weighted combination (ie, average) of two prediction blocks, as shown in Figure 10. The template matching operation includes calculating the cost measurement between the generated template and the sample area (around the initial prediction block) in the reference picture. For each of the two reference pictures, the MV that produces the smallest template cost is regarded as the updated MV of the list to replace the original template. In JEM, nine MV candidates are searched for each list. The nine MV candidates include the original MV and 8 surrounding MVs, where one luminance sample is offset from the original MV in the horizontal or vertical direction or both. Finally, two new MVs, MV0' and MV1' shown in Figure 10, are used to generate the final bidirectional prediction result. The sum of absolute differences (SAD) is used as a cost measure.

DMVR is applied to the Merge mode of bidirectional prediction, where one MV is from the past reference picture and the other MV is from the future reference picture, without transmitting additional syntax elements. In JEM, when LIC, affine motion, FRUC, or sub-CU Merge candidates are enabled for CU, DMVR is not applied. Self-adjusting motion vector difference resolution

In HEVC, when the use_integer_mv_flag in the slice header is equal to 0, the motion vector difference (MVD) (between the motion vector of the PU and the predicted motion vector) is signaled in units of quarter luminance samples. . In JEM, locally adaptive motion vector resolution (LAMVR) is introduced. In JEM, MVD can be coded in units of quarter-luminance samples, integer-luminance samples, or four-luminance samples. The MVD resolution is controlled at the coding unit (CU) level, and for each CU with at least one non-zero MVD component, the MVD resolution flag is conditionally signaled.

For a CU with at least one non-zero MVD component, the first flag is signaled to indicate whether to use the quarter luma sample MV accuracy in the CU. When the first flag (equal to 1) indicates that one-quarter luminance sample MV accuracy is not used, another flag is signaled to indicate whether to use integer luminance sample MV accuracy or four luminance sample MV accuracy.

When the first MVD resolution flag of the CU is zero or is not coded for the CU (meaning that all MVDs in the CU are zero), a quarter of the luminance sample MV resolution is used for the CU. When the CU uses the integer luma sample MV precision or the four luma sample MV precision, the MVP in the AMVP candidate list of the CU is rounded to the corresponding precision.

In the encoder, the CU-level RD check is used to determine which MVD resolution to use for the CU. That is, for each MVD resolution, CU-level RD verification is performed three times. In order to speed up the encoder, the following coding scheme is applied in JEM.

During the RD verification period of the CU with the MVD resolution of a normal quarter luminance sample, the motion information of the current CU (integer luminance sample accuracy) is stored. The stored motion information (after rounding) is used as the starting point for further small-range motion vector refinement for the same CU with integer luminance samples and 4-luminance sample MVD resolution during RD verification, making it time-consuming The motion estimation process is not repeated three times.

Conditionally invoke the RD check of the CU with the MVD resolution of 4 luma samples. For CU, when the MVD resolution of the RD cost integer luminance sample is much greater than the MVD resolution of a quarter luminance sample, skip the RD check for the MVD resolution of 4 luminance samples of the CU. Motion vector prediction based on sub-CU

In JEM, each CU may have at most one set of motion parameters for each prediction direction. By dividing the large CU into sub-CUs and deriving the motion information of all sub-CUs of the large CU, two sub-CU-level motion vector prediction methods are considered in the encoder. The alternative temporal motion vector prediction (ATMVP) method allows each CU to obtain multiple sets of motion information from multiple blocks smaller than the current CU in the co-bit reference picture. In the spatial-temporal motion vector prediction (STMVP) method, the motion vector of the sub-CU is recursively derived by using a temporal motion vector predictor and spatial adjacent motion vectors.

In order to retain a more accurate motion field for sub-CU motion prediction, motion compression of reference frames is currently disabled. Alternative temporal motion vector prediction

In the alternative temporal motion vector prediction (ATMVP) method, temporal motion vector prediction (TMVP) is modified by extracting multiple sets of motion information (including motion vectors and reference indexes) from blocks smaller than the current CU. As shown in Figure 11, the sub-CU is a square N×N block (N is set to 4 by default).

Fig. 13 shows an example of a bilateral template matching process. In the first step, a bilateral template is generated from the prediction block. In the second step, two-sided template matching is used to find the most matching block.

ATMVP predicts the motion vectors of sub-CUs in the CU in two steps. The first step is to use the so-called time vector to identify the corresponding block in the reference picture. The reference picture is called the motion source picture. The second step is to divide the current CU into sub-CUs, and obtain the motion vector and the reference index of each sub-CU from the block corresponding to each sub-CU, as shown in FIG. 14.

In the first step, the reference picture and the corresponding block are determined by the motion information of the spatial neighboring blocks of the current CU. In order to avoid the repeated scanning process of adjacent blocks, the first Merge candidate in the Merge candidate list of the current CU is used. The first available motion vector and its related reference index are set as the index of the time vector and the motion source picture. In this way, in ATMVP, compared with TMVP, the corresponding block can be identified more accurately, where the corresponding block (sometimes called a co-located block) is always located in the lower right or center position relative to the current CU. In one example, if the first Merge candidate is from the left neighboring block (ie, A _{1 in} FIG. 15), the associated MV and reference picture are used to identify the source block and the source picture.

In the second step, by adding a time vector to the coordinates of the current CU, the corresponding block of the sub-CU is identified through the time vector in the motion source picture. For each sub-CU, the motion information of the corresponding block (the smallest motion grid covering the center sample) is used to derive the motion information of the sub-CU. After the motion information of the corresponding N×N block is identified, the motion information is converted into the motion vector and reference index of the current sub-CU in the same way as the TMVP of HEVC, where motion scaling and other processes are applied. For example, the decoder checks whether the low-delay condition is satisfied (ie, the POC of all reference pictures of the current picture is less than the POC of the current picture) and may use the motion vector MVx (corresponding to the motion vector of the reference picture list X) to predict each sub-CU The motion vector MVy (where X is equal to 0 or 1 and Y is equal to 1-X). Space-time motion vector prediction

In this method, the motion vector of the sub-CU is derived recursively in the raster scan order. Figure 16 illustrates this concept. Let us consider an 8×8 CU, which contains four 4×4 sub-CUs A, B, C, and D. The adjacent 4×4 blocks in the current frame are labeled a, b, c, and d.

The motion derivation of sub CU A starts by identifying that its two spaces are adjacent. The first neighbor is the N×N block (block c) above the sub CU A. If the block c is not available or is intra-coded, the other N×N blocks above the syndrome CU A (starting from block c, from left to right). The second neighbor is the block to the left of the sub-CU A (block b). If block b is not available or is intra-coded, the other blocks on the left side of syndrome CU A (start from block b, from top to bottom). The motion information obtained from the neighboring blocks of each list is scaled to the first reference frame of a given list. Next, the temporal motion vector predictor (TMVP) of sub-block A is derived by following the same process as the TMVP derivation specified in HEVC. Obtain the motion information of the co-bit block at position D and scale it accordingly. Finally, after retrieving and scaling the motion information, for each reference list, all available motion vectors (up to 3) are averaged individually. The average motion vector is designated as the motion vector of the current sub-CU. Sub-CU motion prediction mode signaling notification

The sub-CU mode is enabled as an additional Merge candidate, and no additional syntax elements are required to signal the mode. Add two additional Merge candidates to the Merge candidate list of each CU to indicate ATMVP mode and STMVP mode. If the sequence parameter set indicates that ATMVP and STMVP are enabled, at most seven Merge candidates are used. The coding logic of the additional Merge candidates is the same as that of the Merge candidates in the HM, which means that for each CU in the P or B slice, two additional Merge candidates require additional two RD checks.

In JEM, all bins indexed by Merge are context-encoded by CABAC. In HEVC, only the first bin is context coded, and the remaining bins are context by-pass coded. Examples of problems solved by the embodiments

Although MVD provides great flexibility to adapt to various movements in the video signal, it constitutes a large part of the bit stream. Especially during the bidirectional prediction period, it is necessary to signal the MVD of L0 and the MVD of L1, and they introduce a large overhead, especially for low-rate visual communication. Some properties about motion symmetry can be used to save the rate spent on encoding motion information.

The current AMVP mode (including both the MVP index and the reference index) is separately signaled for both L0 and L1, and when the motion follows the symmetry model, they can be represented more effectively. Example of embodiment

1. During the bidirectional prediction, the symmetry property of the motion vector can be used to generate the basic MV set for the AMVP mode. Specifically, only for a single direction (list), MVD is signaled, and the mirroring condition is used to set the MV in the other direction. Alternatively, in addition, the MV can be further refined. This mode is called symmetric bi-directional prediction mode (sym-bi-mode). In this paper, bidirectional prediction refers to prediction by using one reference frame from the past and another reference frame from the future in display order. In some example embodiments, universal video coding (VVC) (for example, JVET-N1001-v5 and other versions and standards) includes a symmetric motion vector difference (SMVD) mode, which can skip Signaling through L1 MVD. The skipped L1 MVD can be set as a mirror image of the L0 MVD without scaling. a. In an example, when L( 1-N ) ( N=0 or 1 ) MVD is sent, the MVD value of LN is not sent (that is, inherited as (0, 0)), and the MVP is set from L ( 1-N ) Mirror MV of MV. After that, motion refinement can be applied to the L N motion vector. (i) In one example, the DMVR refinement process can be applied. Alternatively, the FRUC refinement process can be applied to refine the L N motion vector. (ii) In an example, the refined search range can be through SPS (Sequence Parameter Set), PPS (Picture Parameter Set), VPS (Video Parameter Set, video parameter set) or strip header To be pre-defined or signaled. (iii) In one example, motion refinement can be applied to a specific grid. For example, a uniform sampling grid with a grid distance d can be used to define search points. The grid distance d can be predefined or signaled via SPS, PPS, VPS or slice header. The use of the sampling grid can be considered as a sub-sampling search area, and therefore has the benefit of reducing the memory bandwidth required for searching. (iv) In an example, the signaling of the mirroring mode can be performed at the CU level, CTU level, regional level (covering multiple CU/CTU), or stripe level. When it is performed at the CU level, when it is sym-bi-mode, a one-bit flag needs to be signaled. That is, when the flag is signaled as 1, the associated L N MVD and its MVP index can be skipped. When completed at the CTU level, regional level, or stripe level, all sym-bi-modes will not signal the L N MVD value and its MVP index. In some example embodiments, the signaling of the SMVD flag occurs at the CU level. b. In an example, there is a one-bit flag in the slice header/picture parameter set/sequence parameter set for signaling whether the refinement process should be invoked. Alternatively, signaling can also be performed at the CU/CTU/regional level. c. In one example, during bidirectional prediction, it is possible to signal which MVD list to skip. Signaling can occur at the CU level, regional level, CTU level, or stripe level. When signaling at the CU level, a one-bit flag needs to be signaled in sym-bi-mode. When signaling at the regional level, CTU level, or stripe level, all bi-predictive CUs will skip the MVD signaling of the specified list and use the mirror MVP as their starting point to find the final motion vector . d. In one example, only the mirrored MVP needs to be stored in the MV buffer for use in the motion prediction (AMVP, Merge) of subsequent blocks. The refined motion vector does not need to be stored in the MV buffer. e. In an example, the MVP can be placed along with the regular MVP index, and an additional bit (2 in total) is required to signal three MVP indexes. In some embodiments, in the SMVD mode, both MVP indexes are signaled as the regular AMVP mode. f. In one example, add a mirrored MVP candidate to replace the second AMVP candidate. Nevertheless, only one bit is needed to signal the MVP index. g. In an example, when the POC distance between two reference frames is equal, the mirror MVP mode can be applied. In some embodiments, in the SMVD mode, two references are derived as the reference frames closest to the current frame in L0 and L1. h. In one example, the scaling introduced by mirroring can use the relative time distance between the source frame and the target frame. For example, if L (1-N) of the reference frame and the reference frame L N, L N and the MVD decided to skip signaling, L N (N = 0 or 1) of the initial motion vector can be calculated as: MVP N = (τ N /τ( 1-N ))∙MV( 1-N ), where τ0 and τ1 represent the POC distance between the current frame of L0 and the reference frame and the distance between the current frame of L1 and the reference frame, respectively POC distance.

2. Various matching schemes can be used to complete the refinement process. Let the patches from the L0 and L1 pictures be P0 and P1, respectively. Patches are defined as prediction samples generated by the interpolation process of MV. a. The similarity between P0 and P1 is used as a criterion for selecting refined MVs. In one example, the refinement finds MV N (N=0 or 1) , which minimizes the sum of absolute differences (SAD) between P0 and P1. b. Generate temporary patches from P0 and P1, and the standard can be defined as finding the MV with the highest correlation between the predicted patch and the temporary patch. For example, a separate patch P'= (P0+P1)/2 can be created and used to find MV N (N=0 or 1) , which minimizes the SAD between P'and P N. More generally, P'can be generated by the following formula: P'=ω∙P0+(1-ω)∙P1, where ω is a weighting factor between 0 and 1. c. In one example, a template-based matching scheme can be used to define the refinement process. The top template, left template, or both top and left templates can be used to find MV N (N=0 or 1) . The process of finding MV N (N=0 or 1) is similar to the process described in the above two examples. d. In an example, depending on the distance from the search point to the initial mirror MVP position, the interpolation process can be skipped for some of the search points. When searching for points whose distance to MVP N (N=0 or 1) exceeds the threshold T , no interpolation process is involved. Only integer pixel reference samples are used as patches to derive motion vectors. T can be pre-defined or notified via SPS, PPS, VPS, or slice header signaling. e. In an example, the cost metric used to find MV N includes the estimated rate introduced from the search point to the mirror MVP: C = SAD + λ∙R, where λ is a weighting factor used for estimation in the weighted refinement process The importance of speed. The value of λ can be pre-defined and signaled through SPS, PPS, VPS or slice header. Note that the MVDN, MVN, and MVPN defined below are two-dimensional vectors. i. In an example, R = ||MVD N ||, where MVD N = MV N –MVP N. Here, the function ||∙|| represents the L1 specification. ii. In an example, R = round(log ₂ (||MVD N ||)), where the function round indicates the rounding function of the input argument to the nearest integer. iii. In an example, R = mvd_coding(MVD N ), where the function mvd_coding indicates the standard-compliant binarization process of the input MVD value.

3. MVD_L1_ZERO_FLAG is a stripe-level flag, which imposes strong constraints on L1 MVD signaling by removing all L1 MVD values. Mirror MV and refinement can be combined with this design in the following ways. f. In an example, when MVD_L1_ZERO_FLAG is enabled, the MVP index is not signaled, and the mirror MVP constraint and refinement process can still be applied. g. In an example, when MVD_L1_ZERO_FLAG is enabled, the MVP index is still signaled (for example, as in 1.e or 1.f above) and mirroring MVP constraints are not applied. However, the MV refinement process can still be applied. h. In an example, when MVD_L1_ZERO_FLAG is enabled, the mirror MVP is added to the MVP candidate list, followed by the MV refinement process.

4. When signaling of the reference index and MVP index of L N ( N = 0 or 1 ) is involved, a joint MVP list can be created to support the mirror MVD mode. That is, the MVP list is jointly derived for L0 and L1 (a given pair of specific reference indexes), and only a single index needs to be signaled. i. In an example, the signaling of refIdx N can be skipped, and only the reference frame closest to the mirror position of the L(1-N ) reference frame is selected as its reference frame for MVP scaling. In some embodiments, in the SMVD mode, two reference indexes are skipped because they are selected as the reference frame closest to the current frame in the two lists. j. In one example, during the derivation process, MVP candidates that cannot create a Bi predictor should be considered invalid. k. In an example, in addition to when the scaling occurs, the candidate pair that causes the motion vector to be located on the reference frame of L0 and L1 in the decoded picture buffer (Decoded Picture Buffer, DPB) is considered to be a valid candidate, the derivation can be This is done by following the existing process of MVP derivation for L(1-N). l. It can represent the mirror MVD mode, including: If (sym_mvd_flag[ x0 ][ y0]) {MvdL1[ x0 ][ y0 ][ 0] = −MvdL0[ x0 ][ y0 ][ 0] MvdL1[ x0 ][ y0] [1] = −MvdL0[ x0 ][ y0 ][ 1]} otherwise

5. The proposed method can also be applied to the multi-hypothesis model. m. In this case, when there are two sets of MV information for each reference picture list, the MV information can be signaled for one reference picture list. However, the MVD of the MV information set of another reference picture list can be derived. For each group of MV information in a reference picture list, it can be processed in the same way as sym-bi-mode. n. Alternatively, when there are two sets of MV information for each reference picture list, one set of MV information for the two reference picture lists can be signaled. However, you can use sym-bi-mode to export the other two sets of MV information in the two reference picture lists during operation.

Many video coding standards are based on a hybrid video coding structure in which temporal prediction plus transform coding is used. An example of a typical HEVC encoder framework is depicted in FIG. 17.

Figure 18 is a block diagram 1800 of a video processing device. The apparatus 1800 can be used to implement one or more of the methods described herein. The device 1800 may be embodied in a smart phone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The device 1800 may include one or more processors 1802, one or more memories 1804, and video processing hardware 1806. The processor(s) 1802 may be configured to implement one or more methods described in this document. The memory (multiple memories) 1804 can be used to store data and codes used to implement the methods and techniques described herein. Video processing hardware 1806 can be used to implement some of the techniques described in this document in hardware circuits.

Figure 19 is a flowchart of an example method 1900 of video bitstream processing. The method 1900 includes: generating (1902) second motion vector difference information based on the symmetry rule and the first motion vector difference information in response to the mirror mode flag in the video bitstream; using the first motion vector difference and the second motion vector difference Information reconstruction (1904) video block, where the reconstruction is performed bi-predictively.

Figure 20 is a flowchart of an example method 2000 of video bitstream processing. The method 2000 includes: for a first reference picture list associated with a video block, receiving (2002) motion vector difference information of a first set of motion vectors; and using a multiple hypothesis symmetry rule, from the motion vector difference of the first set of motion vectors The information derives (2004) motion vector difference information associated with the second set of motion vectors of the second reference picture list, which is associated with the video block. The information can be generated using the received motion vector difference information of the first set of motion vectors.

In some embodiments, the method for processing a video bit stream may include a variant of the method 2000, in which, under multiple hypotheses, part of the motion vector difference information is signaled in an interleaved manner. This method includes: for a video block, receiving first motion vector difference information associated with a first reference picture list, and for this video block, receiving second motion vector difference information associated with a second reference picture list; Assume that the symmetry rule derives the third motion vector difference information associated with the first reference picture list and the fourth motion vector difference information associated with the second reference picture list from the first motion vector difference information and the second motion vector difference information .

Regarding methods 1900 and 2000, bitstream processing may include generating a bitstream representing the video in compressed form. Alternatively, bitstream processing may include reconstructing the video from its compressed form using the bitstream.

Regarding methods 1900 and 2000, in some embodiments, the symmetry rule and the multi-hypothesis symmetry rule may be the same or different. In particular, the multi-hypothesis symmetry rule can be used only when the video block (or picture) is coded using multi-hypothesis motion prediction.

Regarding methods 1900 and 2000, the symmetry rule can specify that the second motion vector prediction difference will be (0,0), and the corresponding motion vector predictor is set to mirror the motion vector, whose value is from the first motion vector difference information Export. In addition, motion vector refinement can be further performed on the mirror motion vector value. As described in the above example, the mirroring mode can be selectively used based on the indication in the bit stream at the CU/CTU/region level. Similarly, the refinement flag can also be signaled to control the motion vector refinement to be used (or not used). The refinement flag can be used at the slice header or picture parameter set, or sequence parameter set or region level, or coding unit or coding tree unit level.

Regarding methods 1900 and 2000, the use of techniques based on symmetry rules to generate mirrored motion vectors can enable skipping of sending motion vector difference information in the bit stream (because the information can be generated by the decoder). The skip operation can be selectively controlled via flags in the bit stream. In an advantageous aspect, the mirror MVP calculation using the above technique can be used on the decoder side to improve the decoding of subsequent blocks, without being subject to calculation dependence that may occur in the case where the refined motion vector is used for the prediction of subsequent blocks. The adverse effects of sex.

Regarding methods 1900 and 2000, in some embodiments, the symmetry rule may only be used to generate mirror motion vectors when two reference frames have the same distance. Otherwise, the scaling of the motion vector can be performed based on the relative time distance of the reference frame.

Regarding methods 1900 and 2000, in some embodiments, patch-based techniques may be used to calculate the mirror motion vector, and may include using the first motion vector difference from the reference frame list 0 to generate the first patch of the prediction sample, using The second patch of the prediction sample is generated with reference to the first motion vector difference of the frame list 1, and the motion vector refinement is determined as a value that minimizes the error function between the first patch and the second patch. Various optimization criteria (for example, rate distortion, SAD, etc.) can be used to determine the refined motion vector.

It should be understood that techniques for reducing the amount of bits used to represent motion in a compressed video bitstream are disclosed. Using the disclosed technology, only half of the motion information of the conventional technology can be used to signal bidirectional prediction, and the mirror symmetry of the motion of the object in the video can be used to generate the other half of the motion information at the decoder. The symmetry flag and the refinement flag can be used to signal the use (or non-use) of the mode and the further refinement of the motion vector. The symmetry rule can be used to calculate the mirror motion vector. One assumption made in the symmetry rule is that the object maintains its translational motion between the time of the current block and the time of the reference block used for bidirectional prediction. For example, using a symmetry rule, a motion vector pointing to a reference area shifted by delx and dely from the current block in one time direction can be assumed to be changed to a scaled version of delx and dely in another direction (zoom can also include Negative scaling, this may be due to the change in the direction of the motion vector) Scaling can depend on time distance and other considerations and is described in this document.

The disclosed and other solutions, examples, embodiments, modules, and functional operations described in this document can be implemented in digital electronic circuits, or implemented in computer software, firmware, or hardware, including the structures disclosed in this document and Its structural equivalents, or a combination of one or more of them. The disclosed and other embodiments can be implemented as one or more computer program products, that is, one or more modules of computer program instructions encoded on a computer readable medium for executing or controlling data processing by a data processing device Operation of the device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of substances that affect a machine-readable propagated signal, or a combination of one or more of them. The term "data processing device" covers all devices, equipment, and machines used to process data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to the hardware, the device may also include code that creates an execution environment for the computer program in question, for example, the code that constitutes the processor firmware, protocol stack, database management system, operating system, or one or more of them. A combination of. Propagated signals are artificially generated signals, such as electrical, optical or electromagnetic signals generated by a machine, which are generated to encode information for transmission to a suitable receiver device.

Computer programs (also called programs, software, software applications, scripts or codes) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as stand-alone programs or As a module, component, subroutine or other unit suitable for use in a computing environment. Computer programs do not necessarily correspond to files in the file system. The program can be stored in a part of a file that holds other programs or data (for example, one or more scripts stored in a markup language file), in a single file dedicated to the program in question, or in multiple coordination File (for example, a file that stores one or more modules, subprograms, or part of the code). Computer programs can be deployed to be executed on one computer or on multiple computers located on one website or distributed on multiple websites and interconnected by a communication network.

The processes and logic flows described in this document can be executed by one or more programmable processors that execute one or more computer programs to perform functions by operating on input data and generating output. The process and logic flow can also be executed by a dedicated logic circuit, and the device can also be implemented as a dedicated logic circuit, such as FPGA (field programmable gate array) or ASIC (application specific integrated circuit, dedicated integrated circuit) ).

As an example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, and any one or more processors of any type of digital computer. Generally, the processor will receive commands and data from read-only memory or random access memory or both. The basic components of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include one or more large storage area devices for storing data, such as floppy disks, magneto-optical discs, or optical discs, or be operatively coupled to receive data from or transmit data from one or more large storage area devices Or both. However, computers do not need such equipment. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM and flash memory devices; magnetic disks , Such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory can be supplemented by or incorporated into a dedicated logic circuit.

Although this patent document contains many details, these details should not be construed as limitations on the scope of any invention or claimable, but as a description of the features specific to a particular embodiment of a particular invention. Certain features described in this patent document in the context of a single embodiment can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable subcombination. In addition, although the above features can be described as functioning in certain combinations and even initially claimed, in some cases, one or more features from the claimed combination can be deleted from the combination, and the claimed The combination of protection can point to changes in sub-combinations or sub-groups.

Similarly, although operations are depicted in a specific order in the drawings, this should not be understood as requiring that these operations be performed in the specific order shown or in a sequential order, or that all the operations shown are performed to achieve the desired result. In addition, the separation of various system elements in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

Only a few implementations and examples are described, and other implementations, enhancements and changes can be made based on the content described and illustrated in this patent document.

1800: Video processing device 1802: processor 1804: memory 1806: Video processing hardware 1900, 2000: method 1902, 1904, 2002, 2004: steps A0, A1, B0, B1, B2, C0, C1: position A, B, C, D: sub CU a, b, c, d: block mvL0, mvL1, refIdxL0, refIdxL1: candidates MV0, MV1, MV0′, MV1′: motion vector refa, refb: reference pictures tb, td: location TD0, TD1: Time distance

Fig. 1 shows an example of the derivation process for the construction of the Merge candidate list. Fig. 2 shows example positions of spatial Merge candidates. Figure 3 is an illustration of motion vector scaling for spatial motion vector candidates. Figure 4 shows an example derivation process for motion vector prediction candidates. FIG. 5 shows an example of candidate pairs considered for redundancy check of spatial Merge candidates. FIG. 6 shows an example location of the second PU divided by Nx2N and 2NxN. Fig. 7 is an example of motion vector scaling for temporal Merge candidates. FIG. 8 shows an example of candidate positions of temporal Merge candidates marked as C0 and C1. Fig. 9 shows an example of combined bidirectional prediction Merge candidates. Fig. 10 shows an example of a bilateral matching process. Fig. 11 shows an example of a template matching process. FIG. 12 shows an example of unilateral motion estimation (ME) in frame rate up-conversion (FRUC). Fig. 13 shows an example of a bilateral template matching process. FIG. 14 shows an example of an alternative temporal motion vector prediction (ATMVP) method. Fig. 15 shows an example of identifying source blocks and source pictures. Fig. 16 is an example of a coding unit (CU) with four sub-blocks (A-D) and its adjacent sub-blocks (a-d). Fig. 17 shows an example of a block diagram of a video encoding device. Fig. 18 is a block diagram of an example of a video processing device. Fig. 19 is a flowchart of an example of a video bitstream processing method. Fig. 20 is a flowchart of another example of a video bitstream processing method.

1900: method

1902, 1904: steps

Claims (34)

A method for processing a video bit stream includes: in response to a mirror mode flag in the video bit stream, generating second motion vector difference information based on a symmetry rule and first motion vector difference information; and using the first motion vector Difference information and the second motion vector difference information to reconstruct a video block in the current picture, wherein the reconstruction is performed using bidirectional prediction, wherein the symmetry rule specifies not to send the second motion vector difference information, and the first The second motion vector difference is set as a mirror image of the first motion vector difference without scaling. The method described in item 1 of the scope of patent application further includes: setting a second motion vector prediction as a mirror motion vector of the first motion vector; and performing motion vector refinement of the mirror motion vector value to generate a motion vector fine化值。 The value. The method described in item 1 of the scope of patent application, wherein the mirror mode flag exists at the coding unit (CU) level, the coding tree unit (CTU) level, the area level covering multiple CU/CTU, or the slice level. The method according to item 2 of the scope of patent application, wherein the motion vector refinement is selectively performed based on a refinement flag in the video bitstream. The method according to item 4 of the scope of patent application, wherein the refinement flag is at least included in the slice header, picture parameter set, sequence parameter set, region level, coding unit or coding tree unit level. The method according to claim 1, wherein the video bitstream includes skip information indicating a list of motion vector differences signaled by skipping in the video bitstream. The method according to item 6 of the scope of patent application, wherein the skip information is at a coding unit level, a region level, a coding tree unit level, or a slice level. The method according to item 7 of the scope of patent application, wherein, when the skip information is at the region level, the coding tree unit level, or the slice level, the coding unit uses the symmetry rule to generate the second motion vector News. The method described in item 1 of the scope of patent application further includes: storing a motion vector predictor generated using the symmetry rule for processing prediction information of subsequent video blocks. The method according to item 9 of the scope of patent application, wherein the motion vector predictor is used together with a conventional motion vector predictor, and wherein a two-bit field signals the motion vector in the video bitstream Predictor. The method according to claim 9, wherein the motion vector predictor is used to replace one of the conventional motion vector predictors, and a single bit in the video bitstream is used to perform signaling Notice. The method according to any one of items 1 to 11 of the scope of the patent application, wherein the symmetry rule is used only when the picture sequence count distances between two reference frames used for bidirectional prediction are equal . The method according to the first item of the scope of patent application, wherein the video bit stream omits the signaling of the difference of the motion vector used to refer to the picture list 11, and wherein the following is used to perform the bidirectional prediction:( The first picture sequence number (POC) of the first reference picture in the reference picture list 0)-(the second POC of the current picture) = (the POC of the current picture)-(the third of another reference picture in the reference picture list 1 POC). The method according to the first item of the scope of patent application, wherein the two reference pictures used for the bidirectional prediction are the reference pictures that are closest to the current picture derived from the past frame and the future frame. The method according to item 1 of the scope of patent application, wherein the video bitstream uses a single reference index and a single motion vector prediction index for each video block, and jointly signals the references of reference list 0 and reference list 1 Index and motion vector prediction index. A method for processing a video bit stream includes: in response to a mirror mode flag in the video bit stream, generating second motion vector difference information based on a symmetry rule and first motion vector difference information; and using the first motion vector The difference information and the second motion vector difference information reconstruct the video block in the current picture, wherein the reconstruction is performed using bidirectional prediction, wherein the relative time distance between the source frame and the target reference frame of the video block is used as Proportional scaling to determine the mirror motion vector value. The method according to item 2 of the scope of patent application, wherein performing the motion vector refinement includes: using a third motion vector from a reference frame associated with the first reference picture list to generate a first patch of prediction samples; The mirror motion vector value of the reference frame associated with the second reference picture list generates a second patch of prediction samples; and the motion vector refinement value is determined to minimize one of the first patch and the second patch The value of the error function between. The method according to item 17 of the scope of patent application, wherein the error function includes absolute difference and measurement. The method according to claim 17, wherein the error function includes the correlation between the refined value of the motion vector and the weighted linear average of the first patch and the second patch. The method according to item 17 of the scope of patent application, wherein the error function is a rate-distortion function using the refinement value of the motion vector. The method according to item 2 of the scope of patent application, wherein performing the motion vector refinement includes: determining the motion vector refinement value to use the top and Reference or interpolate samples on the left to minimize the value of the error function. The method according to item 2 of the scope of patent application, wherein performing the motion vector refinement includes: When the motion vector refinement value is greater than the threshold, the motion vector refinement value is determined to use integer reference samples between the two reference frames associated with the two reference picture lists to minimize the value of the error function. The method according to claim 1, wherein the symmetry rule responds to a flag including MVD_L1_ZERO_FLAG in the slice-level signaling for the video block. A video bit stream processing method, comprising: for a first reference picture list associated with a video block, receiving motion vector difference information of a first set of motion vectors; and using multiple hypothesis symmetry rules from the first set of motion vectors The motion vector difference information of derives the motion vector difference information associated with the second set of motion vectors of the second reference picture list associated with the video block, wherein the multiple hypothesis symmetry rule specifies the second motion vector difference value Is (0, 0), and the corresponding motion vector predictor is set to the mirror motion vector value derived from the first motion vector difference information without scaling; and using the derived result to execute the video block and all The bit stream representation of the video block is converted between. The method according to claim 24, comprising: using the multiple hypothesis symmetry rule to derive another motion vector difference information associated with the first reference picture list associated with the video block; and using the The multiple hypothesis symmetry rule derives another motion vector difference information associated with the second reference picture list associated with the video block. A method for processing a video bitstream includes: for a video block, receiving first motion vector difference information associated with a first reference picture list; for the video block, receiving a second motion vector difference information associated with a second reference picture list Motion vector difference information; using multiple hypothesis symmetry rules to derive the third motion vector difference information and the third motion vector difference information associated with the first reference picture list from the first motion vector difference information and the second motion vector difference information The fourth motion vector difference information picture list associated with the second reference, wherein the multiple hypothesis symmetry rule specifies that the second motion vector difference value is (0, 0), and the corresponding motion vector predictor is set to The mirror motion vector value derived from the first motion vector difference information. The method according to any one of items 24 to 26 of the scope of the patent application further includes: performing a motion vector refinement of the mirror motion vector value to generate a motion vector refinement value. A video processing method includes: receiving a future frame of the video relative to a reference frame of the video; receiving a motion vector related to the future frame of the video and the past frame of the video; applying the future frame of the video and the past frame of the video A predetermined relationship between; reconstructing the past frame of the video based on the future frame of the video, the motion vector, and the predetermined relationship between the past frame of the video and the future frame of the video, The predetermined relationship is that the future frame of the video and the past frame of the video are related by a mirroring condition, and the mirroring condition does not need to be scaled. The method according to item 28 of the scope of patent application, wherein the mirroring condition means that an object having coordinates (x, y) in the future frame of the video has coordinates (-x) in the past frame of the video , -Y). A video processing method includes: receiving a past frame of a video relative to a reference frame of the video; receiving a motion vector related to a past frame of the video and a future frame of the video; applying the future frame of the video and the past frame of the video The predetermined relationship between the video data; based on the past frame of the video, the motion vector, and the predetermined relationship between the past frame of the video and the future frame of the video to reconstruct the future frame of the video data, wherein The predetermined relationship is that the future frame of the video and the past frame of the video are related by a mirroring condition, and the mirroring condition does not need to be scaled. The method according to item 30 of the scope of patent application, wherein the mirroring condition means that an object having coordinates (x, y) in the past frame of the video has coordinates (-x) in the future frame of the video , -Y). A video decoding device includes a processor configured to implement the method according to any one of items 1 to 31 of the scope of patent application. A video encoding device, comprising: a processor, configured to implement any of items 1 to 31 of the scope of patent application The method described in one item. A computer program product having computer code stored thereon, wherein when the code is executed by a processor, the processor implements the method described in any one of items 1 to 31 of the scope of patent application.

Images