TW202037157A - Motion vector accessing range for affine - Google Patents

Abstract

A method of video processing includes accessing motion vectors(MV) stored in at least one of basic unit blocks adjacent to a current block; and performing a video processing between the current block and a bitstream representation of the current block based on the accessed MVs.

Description

Affine motion vector access range

This patent document relates to video coding technology, devices and systems. [Cross references to related applications] In accordance with applicable patent laws and/or in accordance with the rules of the Paris Convention, this application promptly requests the International Patent Application No. PCT/CN2018/107629 filed on September 26, 2018 and the International Patent Application No. filed on September 27, 2018. Priority and rights of PCT/CN2018/107869. The entire disclosures of International Patent Application No. PCT/CN2018/107629 and International Patent Application No. PCT/CN2018/107869 are incorporated herein by reference as part of the disclosure of this application.

Motion compensation (MC) is a technique in video processing that, given previous and/or future frames, predicts frames in a video by taking into account the motion of the camera and/or objects in the video. Motion compensation can be used in the encoding of video data for video compression.

This document discloses methods, systems and devices related to the use of affine motion compensation in video encoding and decoding.

In an exemplary aspect, a method of video processing is disclosed. The method includes: accessing a motion vector (MV) stored in at least one of the basic unit blocks adjacent to the current block; and performing a bit stream of the current block and the current block based on the accessed MV Represents the video processing between.

In an exemplary aspect, a video processing device is disclosed that includes a processor configured to implement the method described herein.

In yet another representative aspect, the various technologies described herein can be implemented as computer program products stored on non-transitory computer-readable media. Computer program products contain program code to perform the methods described in this article.

In yet another representative aspect, the video decoder device may implement the method as described herein.

The details of one or more implementations are proposed in the attached appendices, drawings and the following description. Other features will become apparent from the description and drawings as well as the scope of the patent application.

This document provides several technologies that can be implemented as digital video encoders and decoders. The chapter titles are used in this document to facilitate understanding, and the scope of the technology and embodiments disclosed in each chapter is not limited to only that chapter.

In this document, the term "video processing" can refer to video encoding, video decoding, video compression, or video decompression. For example, during the conversion of the pixel representation of the video to the corresponding bitstream representation, a video compression algorithm can be applied, or vice versa.

1 Overview

The present invention relates to video/image coding technology. Specifically, it relates to affine prediction in video/image coding. It can be applied to existing video coding standards, such as HEVC, or a standard yet to be finalized (multifunctional video coding). It can also be applied to future video/image coding standards or video/image codecs.

2. Introduction

Sub-block-based prediction was first introduced into the video coding standard by HEVC Annex I (3D-HEVC). With sub-block-based prediction, a block such as a coding unit (CU) or prediction unit (PU) is divided into several non-overlapping sub-blocks. Different motion information, such as reference index or motion vector (MV), may be allocated to different sub-blocks, and motion compensation (MC) may be performed separately for each sub-block. Figure 1 shows the concept of sub-block-based prediction.

In order to explore future video coding technologies beyond HEVC, the Joint Video Exploration Team (JVET) was co-founded by VCEG and MPEG in 2015. Since then, JVET has adopted many new methods and incorporated them into a reference software called Joint Exploration Model (JEM).

In JEM, sub-block-based prediction is used in several coding tools, such as affine prediction, optional temporal motion vector prediction (ATMVP), space-time motion vector prediction (STMVP), bidirectional optical flow (BIO) and frame rate Upconversion (FRUC). Affine prediction is also adopted in VVC.

2.1 Affine prediction

In HEVC, only the translational motion model is applied to motion compensation prediction (MCP). In the real world, there are multiple movements, such as zoom in/out, rotation, perspective movement, and other irregular movements. In VVC, a simplified affine transform motion compensation prediction is applied. As shown in Figures 2A-2B, the affine motion field of the block is described by two (in the 4-parameter affine model) or three (in the 6-parameter affine model) control point motion vectors.

Figures 2A-2B show simplified affine motion models (a) 4-parameter affine model; (b) 6-parameter affine model.

The motion vector field (MVF) of the block is described by the following equation with a 4-parameter affine model:

（1）

(1)

And the 6-parameter affine model:

（2）

(2)

Where ( mv ^h ₀ , mv ^v ₀ ) is the motion vector of the upper left corner control point, and ( mv ^h ₁ , mv ^v ₁ ) is the motion vector of the upper right control point, and ( mv ^h ₂ , mv ^v ₂ ) is the lower left The motion vector of the corner control point. The point (x, y) represents the coordinates of a representative point relative to the upper left sample in the current block. The CP motion vector can be signaled (as in the affine AMVP mode) or derived on-the-fly (as in the affine merge mode). w and h are the width and height of the current block. In practice, division is implemented by shifting right with rounding operations. In VTM, the representative point is defined as the center position of the sub-block. For example, when the coordinates of the upper left corner of the sub-block relative to the upper left sample in the current block are (xs, ys), the coordinates of the representative point are defined as (Xs+2, ys+2).

In the design without division, (1) and (2) are implemented as:

（3）

(3)

For the 4-parameter affine model shown in (1):

（4）

(4)

For the 6-parameter affine model shown in (2):

（5）

(5)

finally,

（6）

(6)

（7）

(7)

Where S represents calculation accuracy, for example, in VVC, S=7. In VVC, the MV used in the MC of the sub-block of the upper left sample at (xs, ys) is calculated by (6), where x=xs+2 and y=ys+2.

In order to derive the motion vector of each 4×4 sub-block, the motion vector of the center sample of each sub-block is calculated according to equation (1) or (2), as shown in Figure 3, and rounded to 1/16 fractional precision. Then, a motion compensation interpolation filter is applied to generate a prediction of each sub-block with the derived motion vector.

The affine model can be inherited from spatially adjacent affine coding blocks, such as left, top, top right, bottom left, and top left neighboring blocks, as shown in FIG. 4A. For example, if the adjacent left block A in Fig. 4A is coded in affine mode, as indicated by A0 in Fig. 4B, then the control points of the upper left corner, upper right corner and lower left corner of the adjacent CU/PU containing block A are obtained ( CP) Motion vectors mv ₀ ^N , mv ₁ ^N and mv ₂ ^N. And based on mv ₀ ^N , mv ₁ ^N and mv ₂ ^{N to} calculate the current upper left corner / upper right / lower left motion vectors mv ₀ ^C , mv ₁ ^C and mv ₂ ^C on the current CU/PU (it is only used for the 6-parameter affine model ). It should be noted that in VTM-2.0, if the current block is affine-coded, the sub-block (for example, a 4×4 block in VTM) stores mv0 in LT and mv1 in RT. If the current block is coded with a 6-parameter affine model, LB stores mv2; otherwise (in the case of a 4-parameter affine model), LB stores mv2'. Other sub-blocks store MV for MC.

It should be noted that when the CU is encoded in the affine merge mode, that is, in the AF_MERGE mode, it obtains the first block encoded in the affine mode from the effective neighboring reconstructed blocks. And the selection order of candidate blocks is from left, top, top right, bottom left to top left, as shown in FIG. 4A.

The derived CP MV mv ₀ ^C , mv ₁ ^C and mv ₂ ^{C of the} current block can be used as the CP MV in the affine merge mode. Or they can be used as MVP for affine inter mode in VVC. It should be noted that for the merge mode, if the current block is coded in affine mode, after deriving the CP MV of the current block, the current block can be further divided into multiple sub-blocks, and each block will be based on the derived CP MV of the current block Derive its movement information.

2.2 JVET-K0186

Unlike VTM, where only one affine spatial neighboring block can be used to derive the affine motion of the block, in JVET-K0186, it is proposed to construct a separate list of affine candidates for the AF_MERGE mode. Follow the steps below.

1) Insert the inherited affine candidate into the candidate list

FIG. 5 shows an example of candidate positions of the affine merge mode.

The inherited affine candidate refers to a candidate derived from an effective neighboring reconstructed block coded in an affine mode.

As shown in Fig. 5, the scan order of the candidate blocks is A ₁ , B ₁ , B ₀ , A ₀ and B ₂ . When selecting a block (for example, A1), a two-step process is applied:

a) First, use the three corner motion vectors of the CU covering the block to derive the two/three control points of the current block.

b) Based on the control point of the current block to derive the sub-block motion of each sub-block in the current block.

2) Insert the constructed affine candidate

If the number of candidates in the affine merge candidate list is less than MaxNumAffineCand , the constructed affine candidate is inserted into the candidate list.

The constructed affine candidate refers to a candidate constructed by combining the neighboring motion information of each control point.

First, the motion information of the control point is derived from the designated spatial neighborhood and time domain, as shown in Figure 5. CPk (k=1, 2, 3, 4) represents the kth control point. A ₀ , A ₁ , A ₂ , B ₀ , B ₁ , B ₂ and B ₃ are the spatial positions of predicted CPk (k=1, 2, 3); T is the temporal position of predicted CP4.

The coordinates of CP1, CP2, CP3 and CP4 are (0, 0), (W, 0), (H, 0) and (W, H) respectively, where W and H are the width and height of the current block.

The movement information of each control point is obtained according to the following priority order:

-For CP1, the check priority is B ₂ ->B ₃ ->A ₂ . If B _{2 is} available, use B ₂ . Otherwise, if B _{2 is} not available, B ₃ is used. If neither B ₂ nor B ₃ is available, A ₂ is used. If all three candidates are unavailable, the motion information of CP1 cannot be obtained.

-For CP2, the check priority is B1->B0;

-For CP3, the check priority is A1->A0;

-For CP4, use T.

Second, the combination of control points is used to construct the motion model.

The motion vectors of the three control points are needed to calculate the transformation parameters in the 6-parameter affine model. Three control points can be selected from one of the following four combinations: ({CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}). For example, using CP1, CP2 and CP3 control points to construct a 6-parameter affine motion model, referred to as affine (CP1, CP2, CP3).

The motion vectors of the two control points are needed to calculate the transformation parameters in the 4-parameter affine model. Two control points can be selected from one of the following six combinations ({CP1, CP4}, {CP2, CP3}, {CP1, CP2}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4}) . For example, using CP1 and CP2 control points to construct a 4-parameter affine motion model, referred to as affine (CP1, CP2).

The constructed combination of affine candidates is inserted into the candidate list in the following order:

{CP1, CP2, CP3}, {CP1, CP2, CP4}, {CP1, CP3, CP4}, {CP2, CP3, CP4}, {CP1, CP2}, {CP1, CP3}, {CP2, CP3}, {CP1, CP4}, {CP2, CP4}, {CP3, CP4}.

3) Insert zero motion vector

If the number of candidates in the affine merge candidate list is less than MaxNumAffineCand, the zero motion vector is inserted into the candidate list until the list is full.

2.3 ATMVP (Advanced Time Domain Motion Vector Prediction)

At the 10th JVET meeting, Advanced Temporal Motion Vector Prediction (ATMVP) was included in the benchmark set (BMS)-1.0 reference software, which derives one based on the motion information from collocated blocks of neighboring pictures in the time domain Multiple motions of sub-blocks of coding unit (CU). Although it improves the efficiency of time-domain motion vector prediction, the following complexity issues are identified for the existing ATMVP design:

If multiple reference pictures are used, the co-located pictures of different ATMVP CUs may be different. This means that multiple reference pictures need to be retrieved in the sports field.

The motion information of each ATMVP CU is always derived based on 4×4 units, resulting in multiple calls for motion derivation and motion compensation for each 4×4 sub-block in an ATMVP CU.

Some further simplifications of ATMVP are proposed and have been adopted in VTM2.0.

2.3.1 Simplified co-location block derivation using a fixed co-location picture

In this method, a simplified design is proposed to use the same co-located picture as in HEVC, which is signaled at the slice header as the co-located picture for ATMVP derivation. At the block level, if the reference picture of the neighboring block is different from the co-located picture, the HEVC time-domain MV scaling method is used to scale the MV of the block, and the scaled MV is used in ATMVP.

Refers to the motion vector used to retrieve the field in the _collocated picture R _col as MV _col . In order to minimize the impact due to MV scaling, the MV in the spatial candidate list used to derive the MV _col is selected in the following manner: if the reference picture of the candidate MV is a co-located picture, the MV is selected and used as the MV _col , Without using any zoom. Otherwise, the MV with the reference picture closest to the co-located picture is selected to use scaling to derive MV _col .

2.3.2 Adaptive ATMVP sub-block size

In this method, it is proposed to support the strip-level matching of the sub-block size for ATMVP motion derivation. Specifically, a preset sub-block size for ATMVP motion derivation is signaled at the sequence level. In addition, a flag is signaled at the slice level to indicate whether the preset sub-block size is used for the current slice. If the flag is false, the corresponding ATMVP sub-block size is further signaled in the strip header of the strip.

2.4 STMVP (Space-Time Motion Vector Prediction)

STMVP was proposed and adopted in JEM, but has not yet been adopted in VVC. In STMVP, the motion vector of the sub-CU is derived recursively in the raster scan order. Figure 6 illustrates this concept. Let us consider an 8×8 CU with four 4×4 sub-CUs A, B, C, and D. The adjacent 4×4 blocks in the current frame are labeled a, b, c, and d.

The motion derivation of sub CU A starts by identifying its two spatial neighbors. The first neighborhood is the N×N block (block c) above the sub CU A. If the block c is not available or is intra-coded, check the other N×N blocks above the sub-CU A (from left to right, starting with block c). The second neighborhood is the block to the left of the sub CU A (block b). If block b is not available or is intra-coded, check other blocks on the left of sub-CU A (from top to bottom, starting with block b). The motion information obtained from the neighboring blocks of each list is scaled to the first reference frame of a given list. Next, the temporal motion vector predictor (TMVP) of sub-block A is derived by following the same process as the TMVP derivation specified in HEVC. The motion information of the co-located block at position D is retrieved and scaled accordingly. Finally, after retrieving and scaling motion information, all available motion vectors (up to 3) are averaged separately for each reference list. The average motion vector is designated as the motion vector of the current sub-CU.

Fig. 6 shows an example of a CU with four sub-blocks (A-D) and its neighboring blocks (a-d).

2.5 Exemplary affine inheritance method

In order to reduce the storage requirement for affine model inheritance, some embodiments may implement a solution in which the affine model is inherited by only accessing the MVs in the column to the left of the current block and the MVs in the column above the current block.

Affine model inheritance can be performed by deriving the CP MV of the current block from the affine-encoded lower left MV and lower right MV of the adjacent block, as shown in FIG. 7.

a. In an example, mv ₀ ^C = ( mv ₀ ^Ch , mv ₀ ^Cv ) and mv ₁ ^C = ( mv ₁ ^Ch , mv ₁ ^Cv ) are derived from mv ₀ ^N = ( mv ₀ ^Nh , mv ₀ ^Nv ) and mv ₁ ^N = ( mv ₁ ^Nh , mv ₁ ^Nv ) is derived as follows:

（8）

(8)

Where w and w'are the width of the current block and the width of adjacent blocks, respectively. (X _0, y ₀₎ are the coordinates of the upper left corner of the current block, and _{(x '0, y' 0} ) are the coordinates of the lower left corner of the neighboring blocks.

Alternatively, in addition, the division operation in the calculation of a and b can be replaced by a shift with or without addition.

b. For example, the affine model inheritance is performed by deriving the CP MV of the current block from the lower left MV and the lower right MV of the adjacent blocks of affine encoding from "B" and "C" in FIG. 4A.

i. Alternatively, the affine model inheritance is performed by deriving the CP MV of the current block from the lower left MV and the lower right MV of the affine-coded neighboring blocks from "B", "C" and "E" in FIG. 4A.

c. For example, only when the neighboring block is in the M×N area above (or upper right, or upper left) of the M×N area containing the current block, can it be derived from the lower left MV and the lower right MV of the neighboring block encoded by affine The CP MV of the current block is used for affine model inheritance.

d. For example, y ₀ = y _'0.

i. Alternatively, y ₀ = 1+y ₀ '.

e. For example, if the current block inherits the affine model by deriving the CP MV of the current block from the affine-encoded lower left MV and lower right MV of the neighboring block, the current block is regarded as using a 4-parameter affine model.

FIG. 7 shows an example of affine inheritance by derivation from two bottom CPs of neighboring blocks.

The affine model inheritance can be performed by deriving the CP MV of the current block from the affine-encoded upper right MV and lower right MV of the adjacent block, as shown in FIG.

a. For example, mv ₀ ^C = ( mv ₀ ^Ch , mv ₀ ^Cv ) and mv ₁ ^C = ( mv ₁ ^Ch , mv ₁ ^Cv ) can be determined by mv ₀ ^N = ( mv ₀ ^Nh , mv ₀ ^Nv ) and mv ₁ ^N =( mv ₁ ^Nh , mv ₁ ^Nv ) is derived as follows:

（9）

(9)

Where h'is the height of the adjacent block. w is the width of the current block. (X _0, y ₀₎ are the coordinates of the upper left corner of the current block, and _{(x '0, y' 0} ) are the coordinates of the upper right corner of the neighboring blocks.

Alternatively, in addition, the division operation in the calculation of a and b can be replaced by a shift with or without addition.

b. For example, affine model inheritance is performed by deriving the CP MV of the current block from the upper-right MV and lower-right MV of the adjacent blocks of affine encoding from the middle "A" and "D" in FIG. 4A.

i. Alternatively, affine model inheritance is performed by deriving the CP MV of the current block from the upper right MV and the lower right MV of the adjacent blocks of the affine encoding from "A", "D" and "E" in FIG. 4A.

c. For example, only when the neighboring block is in the M×N area on the left (or upper left, or lower left) of the M×N area containing the current block, can it be derived from the upper right MV and the lower right MV of the adjacent block that are affine-encoded The CP MV of the current block performs affine model inheritance.

d. For example, x ₀ = x _'0.

i. _{Alternatively, x 0 = 1+ x '0} .

e. For example, if the current block inherits the affine model by deriving the CP MV of the current block from the upper right MV and the lower right MV of the adjacent block that are affine-encoded, the current block is regarded as using a 4-parameter affine model.

If the current block does not use the 6-parameter affine model, more CP MVs can be derived and stored for use in the motion vector prediction and/or filtering process.

a. In an example, the stored lower left MV can be used for motion prediction, including the affine model inheritance of the PU/CU to be coded later.

b. In an example, the stored lower left MV can be used for motion prediction of subsequent pictures.

c. In an example, the stored lower left MV can be used for the deblocking filtering process.

d. If the affine-coded block does not use the 6-parameter affine model, the CP MV in the lower left corner is derived for the affine-coded block and stored in the lower left MV unit, which is 4×4 in VVC.

i. The CP MV in the lower left corner (denoted as mv ₂ = ( mv ₂ ^h , mv ₂ ^v )) is derived as follows for the 4-parameter affine model

（10）

(10)

e. Derive the CP MV in the lower right corner of the affine coding block and store it in the lower right MV unit, which is 4×4 in the VVC. The stored lower right MV can be used for motion prediction, including affine model inheritance of PU/CU for subsequent encoding, or motion prediction or deblocking filtering process of subsequent pictures.

i. The CP MV in the lower right corner (denoted as mv ₃ = ( mv ₃ ^h , mv ₃ ^v )) is derived as follows for the 4-parameter affine model:

（11）

(11)

ii. The CP MV in the lower right corner is derived as follows for the 6-parameter affine model:

（12）

(12)

iii. The CP MV in the lower right corner is derived as follows for both the 4-parameter affine model and the 6-parameter affine model:

（13）a.

(13)

b. If mv ₂ = ( mv ₂ ^h , mv ₂ ^v ) in the 4-parameter model, calculate as (10). FIG. 8 shows an example of affine inheritance by derivation from two right CPs of neighboring blocks.

3. Problems solved by the embodiment

Some previous designs can only support the inheritance of the 4-parameter affine model, which may cause coding performance loss when the 6-parameter affine model is enabled.

4. Exemplary embodiment

We propose several methods to inherit the 6-parameter affine model while reducing storage requirements.

The following detailed description of the invention should be considered as an example to explain the general concept. These inventions should not be interpreted narrowly. In addition, these inventions can be combined in any manner. Combinations between the present invention and other inventions are also applicable.

In the following discussion, it is assumed that the coordinates of the upper left corner/upper right corner/lower left corner/lower right corner of the adjacent CU above or on the left side of the affine encoding are (LTNx, LTNy)/(RTNx, RTNy)/(LBNx, LBNy)/ (RBNx, RBNy); The coordinates of the upper left corner/upper right corner/lower left corner/lower right corner of the current CU are (LTCx, LTCy)/(RTCx, RTCy)/(LBCx, LBCy)/(RBCx, RBCy); Affine The width and height of the adjacent CU above or on the left side of the encoding are w'and h', respectively; the width and height of the current CU of affine encoding are w and h, respectively.

1. Carry out affine model inheritance by using MVs in the column directly above the current block (for example, MVs associated with blocks adjacent to the current block) in different ways, depending on the upper proximity of affine coding Whether the block adopts a 4-parameter affine model or a 6-parameter affine model.

a. In an example, when the upper adjacent CU of the affine coding adopts a 4-parameter affine model, the inheritance method is applied.

b. In an example, if the upper neighboring block of affine encoding adopts a 6-parameter affine model, the lower left MV and the lower right MV of the upper neighboring block of affine encoding as shown in FIG. 9 and an auxiliary MV (ie, mv ₀ ^N , mv ₁ ^N and the auxiliary MV associated with the auxiliary point) derives the CPMV of the current block and performs 6-parameter affine model inheritance.

i. In one example, the proposed 6-parameter affine model is inherited only when the upper adjacent block of the affine encoding adopts the 6-parameter affine model and w'> Th0 (for example, Th0 is 8).

ii. Alternatively, when the upper square is coded in affine mode, whether it is a 4- or 6-parameter affine mode, it can be inherited by calling the proposed 6-parameter affine model.

c. In one example, the auxiliary position is used to derive the auxiliary MV through the affine model of the adjacent block above the affine encoding.

i. The auxiliary position is predetermined;

ii. Alternatively, the auxiliary position is adaptive. For example, the auxiliary position depends on the size of the adjacent block above.

iii. Alternatively, the auxiliary location is signaled from the encoder to the decoder in VPS/SPS/PPS/slice header/CTU/CU.

iv. Alternatively, the auxiliary position is (LTNx + (w'>>1), LTNy + h'+ Offset). Offset is an integer. For example, Offset=2 ^K. In another example, Offset=-2 ^K. In some examples, K can be 1, 2, 3, 4, or 5. Specifically, the auxiliary position is (LTNx + (w'>>1), LTNy + h'+ 8).

v. The auxiliary MV is stored in one of the bottom columns of the basic unit block (for example, the 4x4 block in VVC) of the upper adjacent block of the affine encoding, as shown in FIG. 10. The basic unit block where the auxiliary MV is to be stored is named auxiliary block.

(A) In one example, the auxiliary MV cannot be stored in the lower left and lower right basic unit blocks of the upper adjacent block of the affine encoding.

(B) The bottom column of the basic unit block refers to B(0), B(1),...,B(M-1) from left to right. In one example, the auxiliary MV is stored in the basic unit block B (M/2).

a. Alternatively, the auxiliary MV is stored in the basic unit block B (M/2+1);

b. Alternatively, the auxiliary MV is stored in the basic unit block B (M/2-1);

(C) In an example, the stored auxiliary MV can be used for PU/CU motion prediction or merge to be coded later.

(D) In an example, the stored auxiliary MV can be used for motion prediction or merge of subsequent pictures.

(E) In an example, the stored auxiliary MV can be used in the filtering process (for example, deblocking filtering).

(F) Alternatively, additional buffers can be used to store auxiliary MVs instead of storing them in the basic unit block. In this case, the stored auxiliary MV may only be used for affine motion inheritance, and not used for encoding the current slice/slice or a block to be encoded later in a different picture, and not used for the filtering process (for example, deblocking filtering).

vi. After decoding the upper adjacent block of the affine encoding, the coordinates of the auxiliary position are used as input (x, y), and the auxiliary MV is calculated by Eq. (2). And then the auxiliary MV is stored in the auxiliary block.

(A) In an example, the MV stored in the auxiliary block is not used for MC for the auxiliary block.

d. In an example shown in Figure 9, the three CPMVs of the current block are referred to as mv ₀ ^C = (mv ₀ ^Ch , mv ₀ ^Cv ), mv ₁ ^C = (mv ₁ ^Ch , mv ₁ ^Cv ) and mv ₂ ^C = (mv ₂ ^Ch , mv ₂ ^Cv ), which is affine-coded from mv ₀ ^N = (mv ₀ ^Nh , mv ₀ ^Nv ) of the MV at the lower left corner of the upper adjacent block, which is affine-coded The mv ₁ ^N = (mv ₁ ^Nh , mv ₁ ^Nv ) of the MV at the lower right corner of the upper adjacent block and mv _A = (mv _A ^h , mv _A ^v ) as the auxiliary MV are derived as follows:

（14）

(14)

Where (x0, y0) are the coordinates of the upper left corner of the current block, and (x’0, y’0) are the coordinates of the lower left corner of the adjacent block.

Alternatively, in addition, the division operation in (14) can be replaced by a right shift, and an offset can be added or not added before the shift i. For example, the number K in equation (14) depends on how the auxiliary position is defined to obtain Support MV. In the example disclosed in 1.c.iv, the auxiliary position is (LTNx+(w'>>1), LTNy+h'+Offset), where Offset= ^2K . In the example, K = 3.

e. For example, only when the neighboring block is in the M×N area above (or top right, or top left) of the M×N area containing the current block, can the current block be derived from the bottom left MV and bottom right MV of the affine-encoded neighboring block Block CP MV for affine model inheritance.

i. For example, M×N area is CTU, such as 128×128 area;

ii. For example, the M×N area is the size of the pipeline, such as a 64×64 area.

f. For example, y ₀ = y _'0.

i. Alternatively, y ₀ = 1+y' ₀ .

Fig. 9 shows an example of 6-parameter affine inheritance derived from the MV stored in the bottom column of the upper adjacent block of affine encoding.

Fig. 10 shows an example of the bottom row (shaded) of the basic unit block where the auxiliary MV can be stored.

2. Carry out affine model inheritance in different ways by using the MV in the column on the left side of the current block, depending on whether the left neighboring CU of affine coding adopts a 4-parameter affine model or a 6-parameter affine model.

a. In an example, when the left-side neighboring CU of the affine coding adopts a 4-parameter affine model, the previously disclosed method is applied.

b. In one example, if the left neighboring block of affine coding adopts a 6-parameter affine model, the affine coding of the left neighboring block from the upper right MV and the lower right MV and an auxiliary MV Derive the CPMV of the current block and carry out 6-parameter affine model inheritance.

i. In one example, the 6-parameter affine model can only be inherited when the left neighboring block of the affine encoding adopts the 6-parameter affine model and h'> Th1 (for example, Th1 is 8).

ii. Alternatively, when the left block is coded in affine mode, regardless of whether it is a 4- or 6-parameter affine mode, the proposed 6-parameter affine model can be invoked to inherit.

c. In one example, the auxiliary position is used to derive the auxiliary MV through the affine model of the affine-coded left neighboring block.

i. In one example, the auxiliary position is predetermined;

ii. In one example, the auxiliary position is adaptive. For example, the auxiliary position depends on the size of the adjacent block on the left.

iii. In an example, the auxiliary location is signaled from the encoder to the decoder in VPS/SPS/PPS/slice header/CTU/CU.

iv. In an example, the auxiliary position is (LTNx+w'+Offset), LTNy+(h'>>1)). Offset is an integer. For example, Offset=2 ^K. In another example, Offset = -2 ^K. In some examples, K can be 1, 2, 3, 4, or 5. In particular, the auxiliary position is (LTNx+w'+8, LTNy+(h'>> 1)).

v. The auxiliary MV is stored in one of the right rows of the basic unit block (for example, the 4x4 block in VVC) of the left neighboring block of affine coding, as shown in Figure 12. The basic unit block to store the auxiliary MV is named auxiliary block.

(A) In an example, the auxiliary MV cannot be stored in the upper right and lower right basic unit blocks of the left neighboring block of affine encoding.

(B) The right row of the basic unit block refers to B(0), B(1),..., B(M-1) from top to bottom. In one example, the auxiliary MV is stored in the basic unit block B (M/2).

a. Alternatively, the auxiliary MV is stored in the basic unit block B (M/2+1);

b. Alternatively, the auxiliary MV is stored in the basic unit block B (M/2-1);

(C) In an example, the stored auxiliary MV can be used for PU/CU motion prediction or merge to be coded later.

(D) In an example, the stored auxiliary MV can be used for motion prediction or merge of subsequent pictures.

(E) In one example, the stored auxiliary MV can be used in the filtering process (for example, deblocking filtering).

(F) Alternatively, additional buffers can be used to store auxiliary MVs instead of storing them in the basic unit block. In this case, the stored auxiliary MV may only be used for affine motion inheritance, and not used for encoding the current slice/slice or a block to be encoded later in a different picture, and not used for the filtering process (for example, deblocking filtering).

vi. After decoding the left adjacent block of affine encoding, the coordinates of the auxiliary position are used as input (x, y), and the auxiliary MV is calculated by equation (2). And then the auxiliary MV is stored in the auxiliary block.

(A) In an example, the MV stored in the auxiliary block is not used to perform MC for the auxiliary block.

d. In an example shown in Figure 11, the three CPMVs of the current block are referred to as mv ₀ ^C = (mv ₀ ^Ch , mv ₀ ^Cv ), mv ₁ ^C = (mv ₁ ^Ch , mv ₁ ^Cv ) And mv ₂ ^C = (mv ₂ ^Ch , mv ₂ ^Cv ), which is derived from mv ₀ ^N = (mv ₀ ^Nh , mv ₀ ^Nv ) of the MV at the upper right corner of the left adjacent block as affine coding, as affine coding The mv ₁ ^N = (mv ₁ ^Nh , mv ₁ ^Nv ) and mv _A = (mv _A ^h , mv _A ^v ) of the MV as the auxiliary MV at the lower right corner of the adjacent block on the left of, are derived as follows:

（15）

(15)

Where (x _0, y ₀₎ are the coordinates of the upper left corner of the current _{block, (x '0, y'} 0) are the coordinates of the upper right corner of the neighboring blocks.

Alternatively, in addition, the division operation in (15) can be replaced by a shift with or without addition operation.

i. For example, the number K in equation (15) depends on how the auxiliary position is defined to obtain the auxiliary MV. In the example disclosed in 2.c.iv, the auxiliary position is (LTNx+w'+Offset), LTNy+(h'>>1)) where Offset=2 ^K. In the example, K = 3.

e. For example, only when the neighboring block is in the M×N area to the left (or upper left, or lower left) of the M×N area containing the current block, can the current block be derived from the upper right MV and lower right MV of the affine-encoded adjacent block Block CP MV for affine model inheritance.

i. For example, M×N area is CTU, such as 128×128 area;

ii. For example, the M×N area is the size of the pipeline, such as a 64×64 area.

f. For example, x ₀ = x _'0.

i. Alternatively, x ₀ = 1+x' ₀ .

3. In an example, the current block only needs to access the MV in the basic unit (for example, the 4×4 block in VVC) stored in the column above the current block and in the row to the left of the current block, as shown in Figure 13. Shown. The coordinates of the upper left position of the current block are referred to as (x0, y0). The current block can access the basic unit column above the current block starting with the basic unit with upper left coordinates (xRS, yRS) and ending with the basic unit with upper left coordinates (xRE, yRE) (refer to the required upper column) In the MV. Similarly, the current block can access the basic unit row on the left side of the current block starting with the basic unit with upper left coordinates (xCS, yCS) and ending with the basic unit with upper left coordinates (xCE, yCE) (referred to as the desired left Line) in the MV. The width and height of the current block are referred to as W and H, respectively. Assuming that the basic unit size is B×B (for example, 4×4 in VVC), then yRS=yRE=y0-B and xCS=xCE=x0-B.

a. The required upper column and the required left row range can be limited.

i. In an example, xRS = x0-n×W-m. n and m are integers, such as n=0, m=0, n=0, m=1, n=1, m=1, or n=2, m=1;

ii. In an example, xRE = x0+ n×W+m. n and m are integers, such as n=1, m=-B, n=1, m=B, n=2, m=-B, or n=3, m=-B;

iii. In an example, yCS = y0-n×H-m. n and m are integers, such as n=0, m=0, n=0, m=1, n=1, m=1, or n=2, m=1;

iv. In an example, yCE = y0+ n×H+m. n and m are integers, such as n=1, m=-B, n=1, m=B, n=2, m=-B, or n=3, m=-B;

v. In an example, the current block does not need the required upper column;

vi. In an example, the required left row is not needed for the current block;

vii. In one example, the selection of xRS, xRE, yCS and yCE may depend on the location of the auxiliary block.

(A) In an example, the auxiliary block is always covered by the selected upper column or left row.

(B) Alternatively, in addition, (xRS, yRS), (xRE, yRE), (xCS, yCS) and (xCE, yCE) should not overlap with auxiliary blocks.

viii. In one example, the required upper column and the required left row range depend on the current block position.

(A) If the current block is at the top boundary of the P×Q area, that is, when y0%Q==0, Q can be 128 (CTU area) or 64 (pipeline area).

a. In an example, the current block does not need the required upper column;

b. In an example, xRS = x0;

(B) If the current block is at the left boundary of the P×Q area, that is, when x0%P==0, where P can be 128 (CTU area) or 64 (pipeline area).

a. In an example, the current block does not need the required left row;

b. In an example, yCS = y0;

4. As an alternative to example 2.d, the three CPMVs of the current block are referred to as mv ₀ ^C = (mv ₀ ^Ch , mv ₀ ^Cv ), mv ₁ ^C = (mv ₁ ^Ch , mv ₁ ^Cv ) and mv ₂ ^C = (mv ₂ ^Ch , mv ₂ ^Cv ), mv ₀ ^N = (mv ₀ ^Nh , mv ₀ ^Nv ), which is the left side neighbor of affine encoding, at the upper right corner of the left neighboring block The mv ₁ ^N = (mv ₁ ^Nh , mv ₁ ^Nv ) of the MV at the lower right corner of the block and mv _A = (mv _A h, mv _A v) as the auxiliary MV are derived as follows:

（16）

(16)

Where (x0, y0) are the coordinates of the upper left corner of the current block, and (x’0, y’0) are the coordinates of the upper right corner of the adjacent block.

Alternatively, in addition, the division operation in (16) can be replaced by a shift with or without addition operation.

5. In example 1.cv, only when the upper neighboring block of affine encoding is coded with a 6-parameter affine model, the auxiliary MV is stored in the base unit block of the upper neighboring block of affine encoding (such as VVC). One of the 4x4 blocks (such as the middle one).

a. Alternatively, the auxiliary MV is stored in one of the basic unit blocks (such as the middle one) of the upper neighboring block of affine encoding (for example, the 4x4 block in VVC), regardless of whether the upper neighboring block of affine encoding is Use 4-parameter affine model or 6-parameter affine model to encode.

6. In example 2.cv, only when the left neighboring block of affine encoding is coded with a 6-parameter affine model, the auxiliary MV is stored in the right row basic unit block of the left neighboring block of affine encoding (for example, VVC One of the 4x4 blocks (such as the middle one).

a. Alternatively, store the auxiliary MV in one (such as the middle one) of the right-row basic unit blocks of the left neighboring block of the affine encoding (for example, the 4x4 block in VVC), regardless of the left neighboring block of the affine encoding Whether to use 4-parameter affine model or 6-parameter affine model to encode

7. In an example, the lower left basic unit block of the current block always stores the CPMV at the lower left corner, regardless of whether the current block uses a 4-parameter model or a 6-parameter model. For example, in Figure 3, the block LB always stores mv2.

8. In an example, the lower right basic unit block of the current block always stores the CPMV at the lower right corner, regardless of whether the current block uses a 4-parameter model or a 6-parameter model. For example, in Figure 3, the block RB always stores mv3.

9. Regardless of whether the adjacent blocks of affine coding adopt a 4-parameter model or a 6-parameter model, affine inheritance is performed in the same way.

a. Derive the CPMV of the current block in the upper left corner, upper right corner and lower left corner from the MVs in the upper left basic block, upper right basic block, and lower left basic unit stored in the adjacent blocks of affine coding in a 6-parameter affine model inheritance method (For example, mv ₀ ^C , mv ₁ ^C and mv ₂ ^{C in} Figure 4(b)).

i. For example, the MVs stored in the upper left basic block, the upper right basic block, and the lower left basic unit of the affine-coded neighboring blocks are the CPMV at the upper left, upper right, and lower right corners of the affine-coded neighboring blocks.

b. Derive the current block in the upper left corner and upper right corner from the MVs in the lower left basic unit, lower right basic unit, and additional basic unit stored in the bottom column of the basic unit of the adjacent block above the affine encoding in the manner defined in Example 1. And the CPMV in the lower left corner (for example, mv ₀ ^C , mv ₁ ^C and mv ₂ ^{C in} Figure 4(b)).

i. For example, the MVs in the lower left basic unit, lower right basic unit, and extra basic unit stored in the bottom column of the basic unit of the upper adjacent block of affine encoding are the CPMV at the lower left corner of the adjacent block of affine encoding, CPMV and auxiliary MV in the lower right corner.

c. Derive the current block in the upper left corner, upper right corner from the upper right basic unit, lower right basic unit, and the MV in the additional basic unit stored in the right row of the basic unit of the left adjacent block of the affine encoding in the manner defined in Example 2. And the CPMV at the lower left corner (for example, mv ₀ ^C , mv ₁ ^C and mv ₂ ^{C in} Figure 4(b)).

i. For example, the MVs in the upper right basic unit, lower right basic unit, and extra basic unit stored in the right row of the basic unit of the left adjacent block of affine encoding are the CPMV at the upper right corner of the adjacent block of affine encoding, CPMV and auxiliary MV in the lower right corner.

d. The inherited affine merge block is always marked as "using 6 parameters".

e. When the affine model is inherited from the upper adjacent block and the width of the upper adjacent block is not greater than 8, the auxiliary MV is calculated as the average of the MVs stored in the lower left basic unit and the lower right basic unit.

i. For example, the MVs stored in the lower left basic unit and the lower right basic unit of the upper adjacent block of affine encoding are the CPMV at the lower left corner and the CPMV at the lower right corner of the upper adjacent block of affine encoding.

f. When the affine model is inherited from the left adjacent block, and the height of the left adjacent block is not greater than 8, the auxiliary MV is calculated as the average of the MVs stored in the upper right basic unit and the lower right basic unit.

i. For example, the MVs stored in the upper right basic unit and the lower right basic unit of the left adjacent block of affine encoding are the CPMV at the upper right corner and the CPMV at the lower right corner of the left adjacent block of affine encoding.

10. In an example, the MC for the sub-block is performed by the MV stored in the sub-block.

a. For example, the stored MV is CPMV;

b. For example, the stored MV is an auxiliary MV.

11. Regardless of whether the adjacent blocks of affine coding adopt a 4-parameter model or a 6-parameter model, perform affine inheritance in the same way.

12. The auxiliary MV to be stored in each coding block can be more than 1.

a. The auxiliary MV can be used in the same way as above.

Fig. 11 shows an example of 6-parameter affine inheritance derived from the MV stored in the right row of the left neighboring block of affine encoding.

Fig. 12 shows an example of the right row (shaded) of the basic unit block where the auxiliary MV can be stored.

Fig. 13 shows an example of MV storage.

5. Other embodiments

This section discloses examples of embodiments of the proposed invention. It should be noted that it is only one of all possible embodiments of the proposed method and should not be understood in a narrow way.

The normalization (a, b) is as defined in equation (7).

enter:

The coordinates of the current block to the upper left corner are recorded as (posCurX, posCurY);

The coordinates of the upper left corner of the adjacent block are recorded as (posLTX, posLTY);

The coordinates of the upper right corner of the adjacent block are recorded as (posRTX, posRTY);

The coordinates of the lower left corner of the adjacent block are recorded as (posLBX, posLBY);

The coordinates of the lower right corner of the adjacent block are recorded as (posRBX, posRBY);

The width and height of the current block are recorded as W and H;

The width and height of adjacent blocks are marked as W’ and H’;

The MV at the upper left corner of the adjacent block is marked as (mvLTX, mvLTY);

The MV at the upper right corner of the adjacent block is marked as (mvRTX, mvRTY);

The MV at the lower left corner of the adjacent block is marked as (mvLBX, mvLBY);

The MV at the lower right corner of the adjacent block is marked as (mvRBX, mvRBY);

Constant: shift, which can be any positive integer, such as 7 or 8.

Output:

The MV at the upper left corner of the current block is marked as (MV0X, MV0Y);

The MV at the upper right corner of the current block is marked as (MV1X, MV1Y);

The MV at the lower right corner of the current block is marked as (MV2X, MV2Y);

Routines inherited by affine model: If posRBY is equal to (posCurY-1) {// Above neighbouring block iDMvHorX = (mvRBX – mvLBX) >> (shift – log2(W’)) iDMvHorY = (mvRBY – mvLBY) >> (shift – log2(W’)) iDMvVerX = -iDMvHorY; iDMvVerY = iDMvHorX; if the neighbouring block applies the 6-parameter affine model and W’> 8{ AuxiliaryX = posLBx + (W’ >> 1); AuxiliaryY = posLBy; (mvAuxiliaryX, mvAuxiliaryY) is set to be the MV stored in the basic unit block containing (AuxiliaryX, AuxiliaryY); mvMidX = Normalize( mvRBX + mvLBX, 1); mvMidY = Normalize( mvRBY + mvLBY, 1); iDMvVerX = (mvAuxiliaryX-mvMidX) >> (shift-3); iDMvVerY = (mvAuxiliaryY-mvMidY) >> (shift-3); } iMvScaleHor = mvLBX >> shift; iMvScaleVer = mvLBY >> shift; posNeiX = posLBX; posNeiY = posLBY + 1; } else if posRBX is equal to (posCurX -1) {//Left neighbouring block iDMvHorX = (mvRBY– mvRTY) >> (shift – log2(H’)) iDMvHorY = -(mvRBX – mvRTX) >> (shift – log2(H’)) iDMvVerX = -iDMvHorY; iDMvVerY = iDMvHorX; if the neighbouring block applies the 6-parameter affine model and H’> 8 { AuxiliaryX = posRTX; AuxiliaryY = posRTY + (H’>>1); (mvAuxiliaryX, mvAuxiliaryY) is set to be the MV stored in the basic unit block containing (AuxiliaryX, AuxiliaryY); mvMidX = Normalize( mvRBX + mvRTX, 1); mvMidY = Normalize( mvRBY + mvRTY, 1); iDMvHorX = (mvAuxiliaryX-mvMidX) >> (shift-3); iDMvHorY = (mvAuxiliaryY-mvMidY) >> (shift-3); } iMvScaleHor = mvRTX >> shift; iMvScaleVer = mvRTY >> shift; posNeiX = posRTX+1; posNeiY = posRTY; } else{ Return "Input is invalid!" } } horTmp0 = iMvScaleHor + iDMvHorX * (posCurX-posNeiX) + iDMvVerX * (posCurY-posNeiY); verTmp0 = iMvScaleVer + iDMvHorY * (posCurX-posNeiX) + iDMvVerY * (posCurY-posNeiY); MV0X = Normalize (horTmp0, shift ); MV0Y = Normalize (verTmp0, shift ); horTmp1 = iMvScaleHor + iDMvHorX * (posCurX + W-posNeiX) + iDMvVerX * (posCurY-posNeiY); verTmp1 = iMvScaleVer + iDMvHorY * (posCurX + W-posNeiX) + iDMvVerY * (posCurY-posNeiY); MV1X = Normalize (horTmp1, shift ); MV1Y = Normalize (verTmp1, shift ); if the neighbouring block applies the 6-parameter affine model{ The current block also applies the 6-parameter affine model; horTmp2 = iMvScaleHor + iDMvHorX * (posCurX-posNeiX) + iDMvVerX * (posCurY+H-posNeiY); verTmp2 = iMvScaleVer + iDMvHorY * (posCurX-posNeiX) + iDMvVerY * (posCurY+H- posNeiY); MV2X = Normalize (horTmp2, shift ); MV2Y = Normalize (verTmp2, shift ); }

FIG. 14 is a block diagram showing an example of the architecture of a computer system or other control device 2600 that can be used to implement various parts of the disclosed technology. In FIG. 14, a computer system 2600 includes one or more processors 2605 and memory 2610 connected via an interconnection 2625. The interconnection 2625 may represent any one or more separate physical buses, point-to-point connections, or both connected by a suitable bridge, adapter, or controller. Therefore, the interconnection 2625 may include, for example, system bus, peripheral component interconnect (PCI) bus, HyperTransport or industry standard architecture (ISA) bus, small computer system interface (SCSI) bus, universal serial bus (USB) ), IIC (I2C) bus or Institute of Electrical and Electronics Engineers (IEEE) standard 674 bus, sometimes called "Firewire".

The processor(s) 2605 may include a central processing unit (CPU) to control, for example, the overall operation of the host computer. In some embodiments, the processor(s) 2605 executes software or firmware stored in the memory 2610 to achieve this purpose. The processor(s) 2605 may be or may include one or more programmable general-purpose or special-purpose microprocessors, digital signal processors (DSP), programmable controllers, application-specific integrated circuits (ASIC), programmable logic Device (PLD), etc., or a combination of these devices.

The memory 2610 may be or include the main memory of the computer system. The memory 2610 represents any suitable form of random access memory (RAM), read-only memory (ROM), flash memory, etc., or a combination of these devices. In use, among other things, the memory 2610 may contain a set of machine instructions, which when executed by the processor 2605, cause the processor 2605 to perform operations to implement the embodiments of the disclosed technology.

Also connected to the processor(s) 2605 via interconnection 2625 is an (optional) network adapter 2615. The network adapter 2615 provides the computer system 2600 with the ability to communicate with remote devices (such as storage clients) and/or other storage servers, and may be, for example, an Ethernet adapter or a fiber channel adapter.

FIG. 15 shows a block diagram of an example embodiment of an apparatus 2700 that can be used to implement various parts of the disclosed technology. The mobile device 2700 may be a laptop computer, a smart phone, a tablet computer, a camcorder, or other types of equipment capable of processing video. The mobile device 2700 includes a processor or controller 2701 for processing data, and a memory 2702 that communicates with the processor 2701 to store and/or buffer data. For example, the processor 2701 may include a central processing unit (CPU) or a microcontroller unit (MCU). In some implementations, the processor 2701 may include a field programmable gate array (FPGA). In some implementations, the mobile device 2700 includes or communicates with a graphics processing unit (GPU), a video processing unit (VPU), and/or a wireless communication unit for various visual and/or communication data processing functions of the smartphone device. For example, the memory 2702 may contain and store processor executable code. When executed by the processor 2701, the mobile device 2700 is configured to perform various operations, such as receiving information, commands and/or data, processing information and data, and The processed information/data is sent or provided to another device, such as an actuator or external display. In order to support various functions of the mobile device 2700, the memory 2702 can store information and data, such as instructions, software, values, images, and other data processed or referenced by the processor 2701. For example, various types of random access memory (RAM) devices, read-only memory (ROM) devices, flash memory devices, and other suitable storage media can be used to implement the storage function of the memory 2702. In some implementations, the mobile device 2700 includes an input/output (I/O) unit 2703 for connecting the processor 2701 and/or the memory 2702 to other modules, units, or devices. For example, the I/O unit 2703 can be connected to the processor 2701 and the memory 2702 to utilize various types of wireless interfaces compatible with typical data communication standards, for example, between one or more computers in the cloud and user equipment . In some implementations, the mobile device 2700 can interface with other devices via the I/O unit 2703 using wired connections. The mobile device 2700 can also be connected to other external interfaces (such as data memory) and/or visual or audio display devices 2704 for retrieval and transmission, which can be processed by the processor, stored in the memory, or in the output unit of the display device 2704 or Data and information displayed on external devices. For example, the display device 2704 may display a video frame modified based on the MVP according to the disclosed technology.

FIG. 16 is a flowchart of a method 1600 for video processing. The method 1600 includes: accessing (1602) a motion vector (MV) stored in at least one of the basic unit blocks adjacent to the current block; and, based on the accessed MV, in the current block and the current block. (1604) video processing between stream representations.

The various embodiments and techniques disclosed in this document can be described in the list of examples below.

1. Video processing methods, including:

Access the motion vector (MV) of at least one of the basic unit blocks adjacent to the current block; and

Perform video processing between the current block and the bitstream representation of the current block based on the accessed MV.

2. The method according to example 1, wherein the basic unit block forms a column above the current block, and the column contains the leftmost basic unit block with upper-left corner coordinates (xRS, yRS) and the upper-left corner coordinate At least one of the rightmost basic unit blocks of (xRE, yRE).

3. The method according to example 1, wherein the basic unit block forms a row located to the left of the current block, and the row includes a top basic unit block with upper-left corner coordinates (xCS, yCS) and a top-left corner coordinate ( xCE, yCE) at least one of the bottom basic unit blocks.

4. The method according to example 2, wherein yRS=yRE=y0-B, (x0, y0) represents the coordinates of the upper left corner of the current block, and the basic unit block has a size of B×B.

5. The method according to example 3, wherein xCS=xCE=x0-B, (x0, y0) represents the coordinates of the upper left corner of the current block, and the basic unit block has a size of B×B.

6. The method as described in Example 4 or 5, where B=4.

7. The method as described in Example 4, where xRS = x0-n×W-m, n and m are integers, and W represents the width of the current block.

8. The method as described in Example 5, where yCS = y0-n×H-m, n and m are integers, and H represents the height of the current block.

9. The method as described in Example 7 or 8, where n = 0, m = 0; n = 0, m = 1; n = 1, m = 1; or n = 2, m = 1.

10. The method as described in Example 4, where xRE = x0+n×W+m, n and m are integers, and W represents the width of the current block.

11. The method as described in Example 5, where yCE = y0+n×H+m, n and m are integers, and H represents the height of the current block.

12. The method as described in Example 10 or 11, where n = 1, m = -B; n = 1, m = B; n = 2, m =-B; or n = 3, m = -B.

13. The method according to example 2, wherein the column above the current block is not required to predict the current block.

14. The method according to example 3, wherein the row to the left of the current block is not required to predict the current block.

15. The method according to example 4, wherein the value of at least one of xRS and xRE depends on the location of the auxiliary block storing the auxiliary MV for predicting the current block.

16. The method according to example 5, wherein the value of at least one of yCS and yCE depends on the location of the auxiliary block storing the auxiliary MV for predicting the current block.

17. The method according to example 15, wherein the auxiliary block is covered by the column located above the current block.

18. The method according to example 16, wherein the auxiliary block is covered by the row located to the left of the current block.

19. The method according to example 15, wherein the auxiliary block does not overlap with any one of the upper left corner coordinates (xRS, yRS) and the upper left corner coordinates (xRE, yRE).

20. The method according to example 16, wherein the auxiliary block does not overlap with any one of the upper left corner coordinates (xCS, yCS) and the upper left corner coordinates (xCE, yCE).

21. The method according to example 2, wherein the range of the column located above the current block depends on the position of the current block.

22. The method according to example 21, wherein if the current block is determined to be at the top boundary of the P×Q area, no column above the current block is required to predict the current block.

23. The method according to example 22, wherein if y0%Q == 0, the current block is determined to be at the top boundary of the P×Q area, and the coordinates of the upper left corner of the current block are expressed as ( x0, y0).

24. The method as described in Example 21, wherein XRS == x0, and the coordinates of the upper left corner of the current block are expressed as (x0, y0).

25. The method according to example 3, wherein the range of the row located to the left of the current block depends on the position of the current block.

26. The method according to example 25, wherein if the current block is determined to be at the left boundary of the P×Q area, the row on the left of the current block is not required to predict the current block.

27. The method of example 26, wherein if x0%P == 0, the current block is determined to be at the left boundary of the P×Q area, and the upper left corner coordinate of the current block is expressed as ( x0, y0).

28. The method as described in example 25, where yCS == y0, and the coordinates of the upper left corner of the current block are expressed as (x0, y0).

29. The method according to example 26, wherein Q=128 in the coding tree unit area and Q=64 in the pipeline area.

30. The method according to example 27, wherein P=128 in the coding tree unit area and P=64 in the pipeline area.

31. The method according to example 1, wherein the current block is divided into multiple basic unit blocks, and the method further includes:

Store the control point (CP) MV at the lower left corner of the current block in the lower left basic unit block of the current block; and/or

The control point (CP) MV at the lower right corner of the current block is stored in the lower right basic unit block of the current block.

32. The method according to example 31, wherein the current block is coded with a 4-parameter affine model or a 6-parameter affine model.

33. The method as described in Example 1, further including:

By using the MV stored in the sub-block, motion compensation is performed for the sub-block divided from the current block.

34. The method as described in Example 33, wherein

The stored MV is the control point (CP) MV of the current block or the auxiliary MV used to predict the current block.

35. The method of any one of examples 1-34, wherein the video processing includes encoding the video block into a bitstream representation of the video block and decoding from the bitstream representation of the video block At least one of the video blocks.

36. A video processing device, including a processor, configured to implement the method in any one of Examples 1 to 35.

37. A computer program product stored in a non-transitory computer-readable medium, the computer program product containing program code for executing the method in any one of Examples 1 to 35.

The disclosed and other embodiments, modules, and functional operations described in this document can be implemented as digital electronic circuits, or as computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or among them A combination of one or more. The disclosed and other embodiments can be implemented as one or more computer program products, that is, one or more modules of computer program instructions encoded on a computer readable medium, used to be executed or controlled by a data processing device Operation. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of substances that realizes a machine-readable propagated signal, or a combination of one or more of them. The term "data processing equipment" covers all equipment, devices, and machines used to process data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the device can also include code that creates an execution environment for the computer program in question, for example, code that constitutes processor firmware, protocol stack, database management system, operating system, or one or more of them. A combination of. A propagated signal is an artificially generated signal, for example, a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.

Computer programs (also called programs, software, software applications, scripts or codes) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including, for example, stand-alone programs or suitable for use in a computing environment Modules, components, subroutines, or other units. Computer programs do not necessarily correspond to documents in the file system. The program can be stored in part of a document containing other programs or data (for example, one or more scripts stored in a markup language document), in a single document dedicated to the program in question, or in multiple coordination In a document (for example, a document that stores one or more modules, subprograms, or code parts). A computer program can be deployed to be executed on one computer or on multiple computers located on one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this document can be executed by one or more programmable processors that execute one or more computer programs to perform functions by operating on input data and generating output. Process and logic flow can also be executed by dedicated logic circuits (such as FPGA (field programmable gate array) or ASIC (application-specific integrated circuit)), and the device can also be implemented as dedicated logic circuits (such as FPGA (field programmable gate array) ) Or ASIC (application-specific integrated circuit)).

As examples, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more processors of any type of digital computer. Generally, the processor will receive commands and data from read-only memory or random access memory or both. The basic components of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include one or more mass storage devices or be operatively coupled to one or more mass storage devices to receive data from or transmit data to one or more mass storage devices. A mass storage device or both, such as floppy disks, magneto-optical disks, or optical disks. However, computers do not necessarily need such equipment. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM and flash memory devices; magnetic disks, For example, internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory can be supplemented by or incorporated into a dedicated logic circuit.

Although this patent document contains many details, these details should not be construed as limitations on the scope of any invention or claimable, but as descriptions of features specific to specific embodiments of specific inventions. Certain features described in the context of separate embodiments in this patent document can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable subcombination. In addition, although the above features can be described as functioning in certain combinations and even initially claimed as such, in some cases one or more features from the claimed combination can be removed from the combination, and the claimed The combination of protection can be directed to sub-combination or sub-combination changes.

Similarly, although operations are depicted in a specific order in the drawings, this should not be understood as achieving the desired result requires performing such operations in the specific order shown or in order, or performing all the operations shown. In addition, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

Only a few implementations and examples are described, and other implementations, enhancements and changes can be made based on the content described and shown in this patent document.

1600: Method 1602~1604: Step 2600: Computer System 2605: (Multiple) Processors 2610, 2702: Memory 2615: Network Adapter 2625: Interconnect 2700: Mobile Device 2701: Processor/Controller 2703: I/O unit 2704: display devices A, B, C, D, E, LT, RT, LB, RB: sub-blocks a, b, c, d: adjacent blocks A0, A1, A2, B0, B1, B2 B3: Candidate blocks CP ₀ , CP ₁ , CP ₂ : Control points Cur: current block mv ₀ , mv0', mv ₀ ^C , mv ₀ ^N , mv ₁ , mv1', mv ₁ ^C , mv ₁ ^N , mv ₂ , mv2', mv ₂ ^C , mv ₂ ^N , mv ₃ , mv3': motion vector T: time domain position (x ₀ ,y ₀ ), (x ₀ ',y ₀ '), (x ₀ +w,y ₀ ), (x ₀ '+w',y ₀ '), (x ₀ ',y ₀ '+h'), (x ₁ ,y ₁ ), (x ₁ ',y ₁ '), (x ₂ , y ₂ ), (xCS,yCS), (xRS,yRS), (xRE,yRE), (xCE,yCE): coordinates

Fig. 1 shows an example of prediction calculation based on sub-blocks. 2A-2B show examples of simplified affine motion models (a) 4-parameter affine model; (b) 6-parameter affine model. Fig. 3 shows an example of the affine motion vector field (MVF) of each sub-block. Figures 4A-4B show candidates for the AF_MERGE mode. FIG. 5 shows exemplary candidate positions of the affine merge mode. Figure 6 shows an example of a coding unit (CU) with four sub-blocks (A-D) and its neighboring blocks (a-d). Fig. 7 shows an example of affine inheritance derived from the two right CPs of adjacent blocks. Figure 8 is through affine inheritance derived from the two right CPs of adjacent blocks. FIG. 9 shows an example of 6-parameter affine inheritance derived from the MV stored in the bottom column of the upper adjacent block of affine coding. Fig. 10 shows an example of the bottom row (shaded) of the basic unit block where the auxiliary MV can be stored. FIG. 11 shows an example of 6-parameter affine inheritance derived from the MV stored in the right row of the left neighboring block of affine encoding. Fig. 12 shows an example of the right row (shaded) of the basic unit block where the auxiliary MV can be stored. Figure 13 shows an example of the MV storage used. FIG. 14 is a block diagram illustrating an example of an architecture that can be used to form a computer system or other control device that implements various parts of the disclosed technology. FIG. 15 shows a block diagram of an exemplary embodiment of various parts of a mobile device that can be used to implement the technology of the present disclosure. Figure 16 is a flowchart of an exemplary method of visual media processing.

1600: method

1602~1604: steps

Claims (37)

A method for video processing, including: Access the motion vector (MV) of at least one of the basic unit blocks adjacent to the current block; and Perform video processing between the current block and the bitstream representation of the current block based on the accessed MV. The method described in item 1 of the scope of patent application, wherein the basic unit block forms a column above the current block, and the column includes the leftmost basic unit block with the upper left corner coordinates (xRS, yRS) and the upper left At least one of the rightmost basic unit blocks of angular coordinates (xRE, yRE). The method described in item 1 of the scope of patent application, wherein the basic unit block forms a row located on the left side of the current block, and the row includes the top basic unit block with the upper left corner coordinates (xCS, yCS) and the upper left corner At least one of the bottom basic unit blocks of coordinates (xCE, yCE). The method described in item 2 of the scope of patent application, wherein yRS=yRE=y0-B, (x0, y0) represents the coordinates of the upper left corner of the current block, and the basic unit block has a size of B×B. The method described in item 3 of the scope of patent application, wherein xCS=xCE=x0-B, (x0, y0) represents the coordinates of the upper left corner of the current block, and the basic unit block has a size of B×B. Such as the method described in item 4 or 5 of the scope of patent application, wherein B=4. The method described in item 4 of the scope of patent application, wherein xRS = x0-n×W-m, n and m are integers, and W represents the width of the current block. The method described in item 5 of the scope of patent application, wherein yCS = y0-n×H-m, n and m are integers, and H represents the height of the current block. Such as the method described in item 7 or 8 of the scope of patent application, wherein n=0, m=0; n=0, m=1; n=1, m=1; or n=2, m=1. The method described in item 4 of the scope of patent application, wherein xRE = x0+n×W+m, n and m are integers, and W represents the width of the current block. The method described in item 5 of the scope of patent application, wherein yCE = y0+n×H+m, n and m are integers, and H represents the height of the current block. Such as the method described in item 10 or 11 of the scope of patent application, where n = 1, m = -B; n = 1, m = B; n = 2, m =-B; or n = 3, m = -B . The method according to the second item of the scope of patent application, wherein the column above the current block is not required to predict the current block. The method according to the third item of the scope of patent application, wherein the row on the left side of the current block is not required to predict the current block. The method according to claim 4, wherein the value of at least one of xRS and xRE depends on the location of the auxiliary block storing the auxiliary MV for predicting the current block. The method described in item 5 of the scope of patent application, wherein the value of at least one of yCS and yCE depends on the location of the auxiliary block storing the auxiliary MV for predicting the current block. The method according to claim 15, wherein the auxiliary block is covered by the column above the current block. The method according to item 16 of the scope of patent application, wherein the auxiliary block is covered by the row located to the left of the current block. The method according to item 15 of the scope of patent application, wherein the auxiliary block does not overlap with any one of the upper left corner coordinates (xRS, yRS) and the upper left corner coordinates (xRE, yRE). The method according to item 16 of the scope of patent application, wherein the auxiliary block does not overlap with any one of the upper left corner coordinates (xCS, yCS) and the upper left corner coordinates (xCE, yCE). The method according to the second item of the scope of patent application, wherein the range of the column above the current block depends on the position of the current block. The method described in item 21 of the scope of patent application, wherein if the current block is determined to be at the top boundary of the P×Q area, the column above the current block is not required to predict the current block. The method described in item 22 of the scope of patent application, wherein if y0%Q == 0, the current block is determined to be at the top boundary of the P×Q area, and the upper left corner coordinate of the current block represents Is (x0, y0). As for the method described in item 21 of the scope of patent application, where XRS == x0, the coordinates of the upper left corner of the current block are expressed as (x0, y0). The method according to the third item of the scope of patent application, wherein the range of the row located on the left side of the current block depends on the position of the current block. The method according to item 25 of the scope of patent application, wherein if the current block is determined to be at the left boundary of the P×Q area, the row on the left of the current block is not required to predict the current block. The method described in item 26 of the scope of patent application, wherein if x0%P == 0, the current block is determined to be at the left boundary of the P×Q area, and the upper left corner coordinate of the current block represents Is (x0, y0). For the method described in item 25 of the scope of patent application, wherein yCS == y0, the coordinates of the upper left corner of the current block are expressed as (x0, y0). The method described in item 26 of the scope of patent application, wherein Q=128 in the coding tree unit area and Q=64 in the pipeline area. The method described in item 27 of the scope of patent application, wherein P=128 in the coding tree unit area and P=64 in the pipeline area. The method according to the first item of the scope of patent application, wherein the current block is divided into a plurality of basic unit blocks, and the method further includes: Store the control point (CP) MV at the lower left corner of the current block in the lower left basic unit block of the current block; and/or The control point (CP) MV at the lower right corner of the current block is stored in the lower right basic unit block of the current block. The method described in item 31 of the scope of patent application, wherein the current block is encoded with a 4-parameter affine model or a 6-parameter affine model. The method described in item 1 of the scope of patent application also includes: By using the MV stored in the sub-block, motion compensation is performed for the sub-block divided from the current block. The method described in item 33 of the scope of patent application, wherein the stored MV is a control point (CP) MV of the current block or an auxiliary MV used to predict the current block. The method according to any one of claims 1 to 34, wherein the video processing includes encoding the video block into a bitstream representation of the video block and a bitstream from the video block Indicates that at least one of the video blocks is decoded. A video processing device includes a processor and is configured to implement the method described in any one of items 1 to 35 in the scope of patent application. A computer program product is stored in a non-transitory computer readable medium, and the computer program product contains program code for executing the method in any one of the scope of the patent application from 1 to 35.

Images