TWI617185B - Method and apparatus of video coding with affine motion compensation - Google Patents

Abstract

A method and device for video encoding and decoding with affine motion compensation are disclosed. An embodiment of this method receives an input data associated with a current block encoded in or to be encoded in an affine mode. Two or more motion vectors of neighboring blocks are obtained from the buffer, wherein at least one of the acquired motion vectors does not correspond to a sub-block in a first neighboring block column or a first neighboring block row. This method uses the obtained two or more motion vectors to derive an affine candidate including multiple affine motion vectors, and uses one or more motion-compensated blocks derived from the derived affine candidate to encode or predict the current block or Decode the current block. The buffer used for affine candidate derivation stores selected motion vectors that are less than all motion vectors of the coded block in the current picture.

Description

Video coding method and device with affine motion compensation [Cross-reference to related applications]

The present invention claims that the PCT patent application with serial number PCT / CN2016 / 076360 and titled "Affine Prediction for Video Coding" was filed on March 15, 2016, and was filed on December 22, 2016 with serial number US62 / 437,757 , Priority to a US provisional patent application entitled "Affine Prediction Buffer Management for Video Coding". This PCT patent application and this US provisional patent application are incorporated herein by reference.

The present invention relates to affine motion compensation. cormpensation). In particular, the present invention relates to a buffer management of a video encoding system for implementing affine motion compensation and a technique for enabling an adaptive motion vector resolution for affine motion compensation.

During the encoding and decoding process, video data needs a lot of storage space to buffer intermediate data. With increasing high resolution and higher frame rates, and more powerful encoding technologies being developed to obtain better encoding performance, the storage requirements of video encoders and decoders have increased significantly. One of the newly developed coding technologies is affine motion prediction and compensation, which can effectively track more complex Motion, such as: rotation, scaling, and deformation of a moving object. Applied to recently developed coding standards, such as the High-Efficiency Video Coding (HEVC) inter-motion prediction method that only considers two-dimensional (2D) translational motion, where all pixels in the interest area follow the same Direction of movement and amplitude. Affine motion prediction can describe 2D block rotation and scaling according to a four-parameter affine model. Affine motion prediction can also capture 2D deformations based on a six-parameter affine model that transforms squares or rectangles into parallelograms. There are two main modes of affine motion prediction proposed in the literature, including affine merge mode and affine inter mode. The affine merging mode allows affine motion information to be inherited from spatially neighboring blocks, whereas the affine inter-frame mode constructs several of the most likely candidates by combining the motion information of spatially neighboring blocks. The affine inter mode is also called an affine advance motion vector prediction (AMVP) mode.

The motion across the picture along the time axis can be described by a four-parameter affine motion model as shown in equation (1). Suppose A (x, y) is the original pixel considered at position (x, y), and A '(x', y ') is the position (x of the reference picture for the original pixel A (x, y)) ', Y') corresponding reference pixels.

x '= a0 * x + a1 * y + a2, and y' =-a1 * x + a0 * y + a3. (1) Among them, a0, a1, a2, and a3 are four parameters in the four-parameter affine motion model.

The motion vector (v _x , v _y ) between this original pixel A (x, y) and its corresponding reference pixel A '(x', y ') in a block of the affine mode can be described as: v _x = (1-a0) * x-a1 * y-a2, and v _y = (1-a0) * y + a1 * x-a3. (2)

Figure 1A shows an exemplary four-parameter affine motion model. The two corner pixels 110 and 112 are located at the upper left and upper right corners of the current block 102. In the four-parameter affine motion model, these two corner pixels are also referred to as control points of the current block 102. The motion vectors Mv0 and Mv1 of the two control points 110 and 112 map the current block 102 to the reference block 104 in the reference picture. According to formula (3), based on the motion vectors Mv0 and Mv1 of the control points 110 and 112, the motion vector field of each pixel A (x, y) in the current block 102 can be derived.

Among them, (v _0x , v _0y ) represents the motion vector Mv0 of the upper left corner 110, (v _1x , v _1y ) represents the motion vector Mv1 of the upper right corner 112, and w represents the width of the current block. For block-based affine motion compensation, when the motion vectors Mv0 and Mv1 of the two control points are decoded, the motion vector of each 4x4 block of the current block 102 can be determined according to equation (3). In other words, the four-parameter affine motion model for the current block 102 can be specified by two motion vectors Mv0 and Mv1 of two control points. In addition, when the upper left corner and upper right corner of the block are used as two control points, the other two control points can also be used.

The six-parameter affine motion model can be described by equation (4). In this model, a total of six parameters a0, a1, a2, b0, b1, and b2 and three control points are used. For each pixel A (x, y), the motion vector (v _x , v _y ) between this pixel A (x, y) and its corresponding reference pixel A '(x', y ') is as follows: (5 ).

x '= a0 + a1 * x + a2 * y, and y' = b0 + b1 * x + b2 * y. (4)

v _x = (a1-1) * x + a2 * y + a0, and v _y = (b2-1) * y + b1 * x + b0. (5)

The motion vector of each pixel predicted by the six-parameter affine motion model is also position-dependent. FIG. 1B shows an example of affine motion compensation according to a six-parameter affine motion model, in which a current block 122 is mapped to a reference block 124 in a reference picture. The corresponding relationship between the three corner pixels 130, 132, and 134 of the current block 122 and the three corner pixels of the reference block 124 can be determined by the three arrows shown in FIG. 1B. The six parameters for the affine motion model may be derived based on three known motion vectors Mv0, Mv1, Mv2 of the upper left, upper right, and lower left control points of the current block 122. Parameter derivation for affine motion models is well known in the art, and details thereof are omitted here.

Various implementations of the affine inter mode and the affine merge mode have been discussed. For example, the affine flag is used to indicate whether to apply the affine inter mode, and when the CU is equal to or greater than 16x16, this affine flag is Identifies a coding unit (coding unit, CU) used for each inter-frame coding. If the current CU is coded or will be coded in affine inter mode, a list of candidate motion vector predictor (MVP) pairs is constructed for the current CU using valid neighboring coded blocks. FIG. 2 shows an example of deriving the candidate MVP pair of the current block 20 encoded in the affine inter mode or the affine merge mode. As shown in FIG. 2, the MVP of the motion vector Mv0 for the upper-left control point of the current block 20 is selected from the motion vectors of the upper-left neighboring coded block A0, A1, or A2; The MVP of the control point's motion vector Mv1 is from Select from the motion vectors of the upper right adjacent coded blocks B0 and B1. The MVP index for the candidate MVP pair list is identified in the video bitstream, and the motion vector difference (MVD) of the two control points is encoded in the video bitstream.

For the current block 20 encoded in the merge mode, the five adjacent coded sub-blocks C0 (called the bottom left block), B0 (called the top right block), and B1 (called the top right corner) in Figure 2 Block), C1 (referred to as the lower left corner block), and A0 (referred to as the upper left corner block) are sequentially checked to determine whether any of the adjacent encoded sub-blocks is encoded in the affine inter-mode or affine Merge mode. In this example, the current block 20 is a prediction unit (PU). Only when any one of the adjacent coded sub-blocks is encoded in the affine inter mode or the affine merge mode, the affine flag is identified to indicate whether the current block 20 is encoded in the affine merge mode. When encoding or decoding the current block 20 according to the affine merge mode, the first available affine-coded neighboring block is selected from 5 neighboring coded sub-blocks. The first available affine-coded neighboring block including the selected neighboring coded sub-blocks is used to derive the affine-combined selection. The affine merge is used to select the predictor used to derive the reference picture of the current block. As shown in Figure 2, the selection order for selecting one of the adjacent coded sub-blocks is from the bottom-left block, top-right block, top-right block, bottom-left block, and top-left block (C0 → B0 → B1 → C1 → A0). The affine merge selection for the current block 20 is derived from the MV of the control point of the first available affine coding neighboring block. For example, if a four-parameter affine motion model is applied, the first available affine coding phase is The MVs of the upper left NxN sub-blocks and the upper right NxN sub-blocks of the neighboring blocks are used to derive the affine merge selection. When a third control point is included for a six-parameter affine motion model, the first available affine coding The MVs of the NxN sub-blocks at the bottom left of neighboring blocks are also used to derive the affine merged selection.

A method and device for video encoding and decoding with affine motion compensation in a video encoding system are disclosed. An embodiment of the video encoder according to the present invention receives input data associated with the current block in the current picture, and an embodiment of the video decoder according to the present invention receives video bits corresponding to the compressed data including the current block in the current picture Streaming. According to the affine motion model, the current block is encoded or will be encoded in affine mode. Embodiments of the present invention reduce the buffer requirements for the time buffer used for affine candidate derivation. The time buffer stores selected motion vectors that are less than all motion vectors of the previous coded block in the current picture. An embodiment of this method obtains two or more motion vectors of neighboring blocks from the time buffer, wherein at least one of the acquired motion vectors does not correspond to the first neighboring NxN closest to the upper boundary of the current block A block row or a sub-block of the first adjacent NxN block column closest to the left boundary of the current block, and NxN is the block size in the time buffer used to store one motion vector. This method further uses two or more acquired motion vectors of neighboring blocks to derive an affine candidate including multiple affine motion vectors, and by using one or more motion compensations derived from the derived affine candidate Block to predict the current block to encode or decode the current block. Each affine motion vector predicts motion between points of the current block and corresponding points of one or more motion-compensated blocks.

In some embodiments, the time buffer stores the MVs of the first adjacent NxN block column and the second adjacent NxN block column that are closest to the upper boundary of the current block, and the time buffer stores the first Adjacent The MV of the NxN block row and the second adjacent NxN block row.

In one embodiment, the acquired MV includes the first and second MVs for the four-parameter affine motion model. For example, if an adjacent block is adjacent or located above the upper left corner of the current block, the first and second MVs are The replacement MV (replacing MV) of the original MV in the upper left corner and the upper right corner of the neighboring blocks, respectively. If the neighboring block is located on the left side of the current block, the first MV is a substitute MV replacing the original MV in the upper left corner of the neighboring block, and the second MV is the original MV in the upper right corner of the neighboring block. In an embodiment, the first and second MVs correspond to the first and second sub-blocks in adjacent blocks, and the first affine MV in the affine MV is derived using the first and second MVs, and the current block The pixel position of is related to the current picture, and the pixel position of the first sub-block is related to the width of the current picture and neighboring blocks. The second affine MV of the affine MV is derived using the first and second MVs, the width of adjacent blocks, and the width of the current block.

In another embodiment, the acquired MV includes the first, second, and third MVs for the six-parameter affine motion model. For example, the three MVs are the first, second, and third MVs in adjacent blocks. Three sub-blocks, and the first affine MV is derived using at least two of the three MVs. The pixel position of the current block is related to the current picture, and the pixel position of the first sub-block is related to the width of the current picture and neighboring blocks. . The second affine MV is derived using the first and second motion vectors, the width of the neighboring block, and the width of the current block, and the third affine MV is derived using the first and third motion vectors and the height of the neighboring block. And at least one of the height of the current block. In this embodiment, if the neighboring block is located above the current block, the first and second MVs are substitute MVs that replace the upper-left corner of the neighboring block and the original MV at the upper-right corner, and the third MV is the neighboring MV Lower-left corner Original MV. The first, second, and third MVs are replacement MVs that replace the upper-left corner, upper-right corner, and lower-left corner of the original MV if the neighboring block is adjacent to the upper-left corner of the current block. If the neighboring block is located on the left side of the current block, the first and third MVs are substitute MVs that replace the upper left corner and the lower left corner of the original MV, and the second MV is the upper right corner original MV of the neighboring block.

According to a four-parameter affine motion model with two control points or a six-parameter affine motion model with three control points, the current block is predicted by the motion compensation block through the affine motion vector of the affine candidate. The time buffer stores the selected motion vector. For example, the time buffer stores the MVs of M NxN block columns above the current block, and M is less than the maximum coding unit height divided by N (CTU_height / N). In another embodiment, the time buffer stores the MVs of K NxN block rows to the left of the current block, and K is less than the maximum coding unit width divided by N (CTU_width / N).

In some embodiments of the method, the acquired MV is an original MV of two or more control points of a neighboring block. The control point includes at least two of the upper left corner, the upper right corner, the lower right corner, and the lower left corner of the adjacent block. In an embodiment, the time buffer stores two NxN block columns and two NxN block rows of MVs, including: first adjacent NxN block columns, first adjacent NxN block rows, and top NxN block column set (top NxN block row set) and the original MV of the left-most NxN block column set. The first adjacent NxN block column is the last column of the upper neighboring block closest to the upper boundary of the current block, and the first adjacent NxN block row is the left adjacent block closest to the left boundary of the current block. In the last row, the top NxN block column set is the first column in the upper neighboring block, and the leftmost NxN block row set is the first row in the left neighboring block.

In yet another embodiment, the method includes receiving input data associated with a current block encoded or to be encoded in an affine mode, calculating and storing affine parameters for a plurality of encoded blocks in the current picture, buffered from time The encoder obtains affine parameters of one or more encoded blocks corresponding to neighboring blocks of the current block to derive an affine candidate including a plurality of affine MVs. The current block is encoded or decoded by predicting the current block using one or more motion-compensated blocks derived from the derived affine candidate. Each affine motion vector predicts motion between points of the current block and corresponding points of one or more motion-compensated blocks.

In an embodiment, when the affine motion model is a four-parameter affine motion model using two of the upper left corner, the upper right corner, the lower left corner, and the lower right corner as control points, the affine parameters include the levels of adjacent blocks. Directional motion vector offset and one motion vector, or when the affine motion model is a four-parameter affine motion model using two of the upper left, upper right, lower left, and lower right corners as control points, the affine parameters include Adjacent blocks have vertical motion vector offsets and one motion vector. In another embodiment, when the affine motion model is a six-parameter affine motion model, the affine parameters include a horizontal MV offset, a vertical MV offset, and a motion vector in neighboring blocks. Examples of motion vectors in adjacent blocks are the motion vectors of the upper left, upper right, lower right, or lower left corners of neighboring blocks. In another embodiment, the affine parameter includes a scaled MV offset for the coded block. In yet another embodiment, the affine parameter includes two or three affine motion vectors representing motion vectors of two or three control points, and a time buffer stores two or three for the encoded block. Affine motion vectors.

Another embodiment of this method includes receiving input data associated with a current block encoded or to be encoded in affine mode, and obtaining from the time buffer Two or more MVs of a valid neighboring block are used for the current block, and two or more MVs of the acquired valid neighboring block are used to derive an affine candidate including an affine MV, and by using One or more compensation blocks derived by the derived affine candidate predict the current block to encode or decode the current block. A valid neighboring block does not include neighboring blocks that are adjacent to the upper left corner of the current block. The time buffer stores the MV of an adjacent NxN block column and an adjacent NxN block row of the current block, where NxN is the block size used to store one MV in the time buffer. The affine MV predicts motion between points of the current block and corresponding points of one or more motion-compensated blocks.

Aspects of the present disclosure further provide an electronic circuit including one or more video coding methods configured to perform affine motion compensation. Other aspects and features of the invention will become apparent to those skilled in the art from a review of the following description of specific embodiments.

110, 112, 130, 132, 134‧‧‧ corner pixels

102, 122, 20, 30, 40, 44, 60, 80‧‧‧ current block

104, 124‧‧‧ reference blocks

32, 34‧‧‧adjacent blocks

322‧‧‧ Upper left NxN block

324‧‧‧ Upper right NxN block

326‧‧‧Left bottom NxN block

342, 344, 346 ‧‧‧ subblocks

41, 42, 43‧‧‧ Neighboring coded blocks

45, 46, 47‧‧‧‧adjacent affine coded blocks

422, 424, 423, 425, 432, 433, 434, 435, 437, 436, 412, 413, 414, 415, 416, 462, 464, 465, 466, 467, 452, 453, 454, 455, 456, 457, 472, 473, 474, 475, 477‧‧‧ subblocks

S50, S52, S54, S56, S58‧‧‧ steps

61, 62, 63‧‧‧adjacent blocks

616, 612, 636, 632, 634, 622, 624, 615, 613, 637, 633, 635, 623, 625, 614, 626‧‧‧ subblocks

S70, S72, S74, S76 ‧‧‧ steps

81, 82, 83‧‧‧ neighbouring blocks

812, 814, 816, 818, 824, 822, 826, 828, 832, 834, 836‧‧‧ subblocks

900‧‧‧ Video Encoder

910, 1012‧‧‧ Intra prediction

912, 1014 ‧ ‧ affine prediction

9122, 10142‧Affine inter prediction

9124, 10144 ‧ ‧ affine combined prediction

914, 1016‧‧‧ Switch

916‧‧‧Adder

918‧‧‧ transformation

920‧‧‧Quantitative

922, 1020‧‧‧ inverse quantization

924, 1022‧‧‧ inverse transform

926, 1018 ‧ ‧ ‧ Reconstruction

928, 1024‧‧‧ Deblocking filtering

930, 1026‧‧‧sample adaptive bias

932, 1028‧‧‧ reference picture buffer

934‧‧‧entropy encoder

1000‧‧‧Video decoder

1010‧‧‧ Entropy Decoder

Figure 1A shows a four-parameter affine prediction that maps the current block to one or more compensation blocks based on two control points.

Figure 1B shows a six-parameter affine prediction that maps the current block to one or more compensation blocks based on three control points.

Figure 2 shows an example of affine candidate derivation based on neighboring coded blocks for affine inter-mode or affine merge mode.

Figure 3 shows an example of encoding or decoding the current block using information from neighboring blocks according to the affine merge mode.

4A and 4B are diagrams showing a storage device according to an embodiment of the present invention. Examples of two adjacent block columns and two adjacent block rows derived from affine candidates.

FIG. 5 is an exemplary flowchart of a video coding system with affine motion compensation combined with an embodiment of the present invention.

FIG. 6 shows an example of storing an original MV of control points of neighboring blocks used for affine candidate derivation according to an embodiment of the present invention.

FIG. 7 is an exemplary flowchart of a video coding system with affine motion compensation combined with an embodiment of the present invention.

FIG. 8 shows an example of storing the MVs of the nearest neighboring block column and the nearest neighboring block row for affine candidate derivation according to an embodiment of the present invention.

FIG. 9 shows an exemplary system block diagram of a video encoder for implementing affine motion prediction and compensation according to various embodiments of the present invention.

FIG. 10 shows an exemplary system block diagram of a video decoder for implementing affine motion compensation according to various embodiments of the present invention.

It can be easily understood that the components of the present invention as shown in the drawings and described herein can be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of embodiments of the system and method of the present invention as shown in the accompanying drawings is not intended to limit the scope of the present invention as claimed, but merely to represent alternative embodiments of the present invention .

Reference throughout the specification to "one embodiment", "some embodiments", or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. Thus, the appearance of the phrases "in one embodiment" or "in some embodiments" throughout this specification does not They must all refer to the same embodiment, which can be implemented individually or in combination with one or more other embodiments. Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Those skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, and the like. In other instances, well-known structures or operations have not been shown or described in detail to avoid obscuring aspects of the present invention.

The HEVC-compliant decoder down-sample consists of the decoded motion vector of each PU including inter prediction encoding using AMVP mode and merge mode with a 16: 1 ratio, and the decoder will downsample the motion The vector is stored in the buffer for subsequent blocks in the current picture and for MVP export of subsequent pictures. The top-left 4x4 motion vectors of each 16x16 block are stored in the buffer, and the stored motion vectors represent the motion vectors of the entire 16x16 block. The motion vector accuracy can be 1/64 pixel accuracy and the predictor is generated by applying a high-precision Discrete Cosine Transform Interpolation Filter (DCTIF). Then, the high-precision motion vector field is clipped to 1/8 pixel accuracy before being stored in the buffer.

In order to process the blocks coded in the affine merge mode, the first available affine-coded neighboring block is determined according to the selection order of the adjacent coded sub-blocks A, B, C, D, and E shown in FIG. 3 . FIG. 3 shows an example of encoding or decoding the current block 30 using the information of neighboring blocks according to the affine merge mode. In this example, two adjacent encoded sub-blocks B and E of the current block 30 are encoded in an affine mode. The neighboring block 32 including the neighboring coded sub-block B may be an affine inter-coded block or the affine merged coded block. Similarly, the neighboring block 34 including the neighboring coded sub-block E may be affine. Inter-coded blocks or affine merged coded blocks. In this example, the first available affine-coded neighboring block is the neighboring block 32. If two control points are used in the affine mode, the motion vector V _B0 and the upper right NxN block of the upper left NxN block 322 and the upper right NxN block of the first available affine encoding neighboring block 32 can be selected after affine merge for predicting the current block 30 The motion vector V _{B1 of} 324 is derived. If three control points are used in the affine mode, the affine merge selection is further derived from the motion vector V _B2 of the left bottom NxN block 326 of the first available affine-coded neighboring block 32. NxN is the minimum block size for storing MVs in the time MV buffer. For example, N is equal to 4. In the HEVC system, only motion vectors of adjacent 4x4 block columns and adjacent 4x4 block rows of the current coding unit (coding unit, CU) or coding tree unit (CTU), and motion vectors of the current CTU are stored. CTU is also a term that defines the largest coding unit (LCU) allowed in a video coding system. All other motion vectors are discarded or down-sampled at a 16: 1 ratio and stored in the buffer. Video coding systems with affine motion compensation require additional MV buffers to store motion vectors of neighboring coded blocks for affine candidate derivation.

The following describes an embodiment of buffer management for a video coding system that implements affine motion compensation while reducing buffer requirements.

First embodiment. In the first embodiment, the current block of the current picture is encoded or decoded by affine motion compensation according to the affine candidate including the affine motion vector. Each affine motion vector predicts a motion vector of a control point of the current block as an affine motion vector used to predict motion between a point of the current block and a corresponding point of the affine motion compensation block. The temporal MV buffer in the first embodiment stores a single adjacent NxN block column of a current block and a motion vector of a single adjacent NxN block row. NxN represents the minimum block size in the temporal MV buffer for storing motion vectors, for example, N is equal to 4. In this embodiment, the number of MVs stored in the temporal MV buffer is the same as the number of conventional HEVC temporal MV buffers used for MVP derivation. When derivation of affine candidates for the current block requires affine-coding motion vectors of control points of adjacent blocks, and the control points are not in the current CTU nor in the adjacent NxN block column or adjacent NxN block row of the current block , The substitute motion vector is obtained from the buffer to derive the corresponding affine motion vector instead of the original motion vector of the control point of the affine-coded neighboring block. The substitute motion vector is a down-sampled motion vector stored in the buffer, and this substitute motion vector represents the motion of the entire 16x16 block. The alternative motion vector may be a motion vector used only by a sub-block near the control point. For example, the substitute motion vector corresponds to a subblock belonging to the same 16x16 block as the control point. The affine motion vector in the affine candidate derived according to the first embodiment is usually not an accurate motion vector predictor, because the down-sampled motion vector sometimes does not reflect the true motion of the control points of adjacent blocks.

The second embodiment. The second embodiment stores more motion vectors by storing MVs of M adjacent NxN block columns and MVs of K adjacent NxN block rows in the time MV buffer. In this embodiment, M and K are integers greater than or equal to 2, and N is an integer greater than 1. Instead of storing all the motion vectors of the coded blocks in the current picture in the temporal MV buffer, a selected motion vector that is less than all the motion vectors of the coded blocks in the current picture is stored. The motion vector is selected as the MV of M block columns and K block rows, where the number M of block columns is less than the height of the largest coding tree unit divided by N (1 <M <CTU_height / N). The NxN block is the minimum block size for storing MVs in the temporal MV buffer. In the example of the second embodiment, as shown in FIGS. 4A and 4B, M and K are both equal to two. In this example, the time buffer stores the motion vectors of the first and second adjacent NxN block columns closest to the upper boundary of the current block, and the time buffer also stores the first and second phases closest to the left boundary of the current block. Motion vectors for adjacent NxN block rows. 4A and 4B show two examples of affine candidate derivation for the current block encoded in the affine mode using two adjacent block columns and two adjacent block rows. If the motion vectors of the control points of adjacent blocks are not stored in the time MV buffer because the corresponding subblocks are not in two adjacent block columns or in two adjacent block rows, some alternative motion vectors are obtained to Derive affine motion vectors from affine candidates.

In the example shown in FIG. 4A, the adjacent coded blocks 41, 42, and 43 of the current block 40 are all encoded in the affine mode. The alternative motion vectors V _B0 and V _B1 of the sub-blocks 423 and 425 of the second neighboring block column above the current block 40 are used to derive the affine motion vectors in the affine candidate to replace the first of the neighboring coded block 42. The original motion vectors V _B0 and V _B1 of the sub-blocks 422 and 424 in the column. The alternative motion vectors V _{E0 ′} , V _{E1 ′} , and V _{E2 ′} of the sub-blocks 433, 435, and 437 in the second neighboring block column above the current block 40 or the second neighboring block row to the left of the current block 40 are Used to derive affine motion vectors to replace the original motion vectors V _E0 , V _E1, and V _{E2 of} sub-blocks 432, 434, and 436. The alternative motion vectors V _{A0 ′} and V _A2 of the sub-blocks 413 and 415 in the second neighboring block row on the left side of the current block 40 are used to derive an affine motion vector to replace the first row in the neighboring encoded block 41 The original motion vectors V _A0 and V _{A2 of the} sub-blocks 412 and 416. The original motion vectors V _B2 and V _{A1 of the} sub-blocks 426 and 414 are used to derive the affine motion vector of the current block 40 because these two sub-blocks 426 and 414 are located in the first and second neighboring block columns of the current block 40. Or in the first and second adjacent block rows. Using the affine motion model, the derived affine motion vector in the affine candidate is used to predict the motion between the points of the current block 40 and the corresponding points of the one or more compensation blocks.

Figure 4B is an alternative solution to the affine candidate derivation method shown in Figure 4A. In FIG. 4B, when affine candidates for encoding or decoding the current block 44 are derived, all original motion vectors of the control points of adjacent affine coded blocks are determined by the first and second neighbors of the current block 44. The motion vectors of the block columns and other sub-blocks of the first and second neighboring block rows are replaced. The control points of the affine coded block predicted by the six-parameter affine motion model include the upper left corner, the upper right corner, and the lower left corner of the affine coded block. Substitute motion vectors V _{B0 ′} , V _{B1 ′,} and V _{B2 ′} in sub-blocks 463, 465, and 467 of neighboring affine coded block 46 are used to derive affine motion vectors to replace sub-blocks 462, 464, and 466. The original motion vectors V _B0 , V _B1 and V _B2 . Substitute motion vectors V _{E0 ′} , V _{E1 ′,} and V _{E2 ′} in sub-blocks 473, 475, and 477 in neighboring affine coded block 47 are used to derive affine motion vectors to replace sub-blocks 472, 474, and 476 The original motion vectors V _E0 , V _E1 and V _E2 . Substitute motion vectors V _{A0 ′} , V _{A1 ′,} and V _{A2 ′ of} subblocks 453, 455, and 457 in neighboring affine coded block 45 are used to derive affine motion vectors to replace subblocks 452, 454, and 456. The original motion vectors V _A0 , V _A0 and V _A2 . Generally speaking, in this embodiment, M adjacent block columns and other positions of K block rows can be used for affine candidate derivation.

Without loss of generality, only the affine candidate derivation method shown in FIG. 4A is further described as follows. In the first example, an affine candidate including three affine motion vectors Mv0, Mv1, and Mv2 is derived to use six The parametric affine motion model predicts the current block 40. In the first example, the affine motion vector of the affine candidate is derived from the adjacent affine coded block 42. For a, the affine motion vector Mv0 = (V0_x, V0_y) for the first control point in the upper left corner is derived from equation (6).

V _{0_x} = V _{B0'_x} + (V _{B2_x} -V _{B0'_x} ) * (posCurPU_Y-posB0'_Y) / (2 * N) + (V _{B1'_x} -V _{B0'_x} ) * (posCurPU_X-posB0'_X ) / RefPU_width, V _{0_y} = V _{B0'_y} + (V _{B2_y} -V _{B0'_y} ) * (posCurPU_Y-posB0'_Y) / (2 * N) + (V _{B1'_y} -V _{B0'_y} ) * (posCurPU_X -posB0'_X) / RefPU_width; (6) where the motion vector V _{B0 '} = (V _{B0'_x} , V _{B0'_y} ), V _B1' = (V _{B1'_x} , V _{B1'_y} ), and V _B2 = (V _{B2_x} , V _{B2_y} ) are three motion vectors obtained from M adjacent NxN block columns and K adjacent NxN block rows. It should also be understood that these motion vectors may be replaced by corresponding motion vectors of the M neighboring NxN block columns of the current block and any other selected sub-blocks in the K neighboring NxN block rows. The coordinates (posCurPU_X, posCurPU_Y) represent the pixel position of the upper-left sample of the current block 40 related to the upper-left sample of the current picture. The coordinates (posB0'_X, posB0'_Y) represent the pixel positions of the upper-left samples of the sub-block 422 related to the upper-left samples of the current picture. RefPU_width indicates the width of the adjacent block 42. The affine motion vectors Mv1 = (V _{1_x} , V _{1_y} ) and Mv2 = (V _{2_x} , V _{2_y} ) at the second and third control points located at the upper right and lower left _corners are derived from equation (7), respectively.

V _{1_x} = V _{0_x} + (V _{B1'_x} -V _{B0'_x} ) * PU_width / RefPU_width, V _{1_y} = V _{0_y} + (V _{B1'_y} -V _{B0'_y} ) * PU_width / RefPU_width; V _{2_x} = V _{0_x} + (V _{B2_x} -V _{B0'_x} ) * PU_height / (2 * N), V _{2_y} = V _{0_y} + (V _{B2_y} -V _{B0'_y} ) * PU_height / (2 * N); (7) Among them, PU_width and PU_height Represents the width and height of the current block 40.

In the second example, an affine candidate including two affine motion vectors Mv0 and Mv1 is derived to predict a current block 40 using a four-parameter affine motion model. The second example also uses motion vectors from neighboring affine coded blocks 42 to derive affine candidates. The affine motion vectors Mv0 = (V0_x, V0_y) and Mv1 = (V1_x, V1_y) for the first and second control points located in the upper left and upper right corners are respectively derived from equation (8).

V _{0_x} = V _{B0'_x-} (V _{B1'_y} -V _{B0'_y} ) * (posCurPU_Y-posB0'_Y) / RefPU_width + (V _{B1'_x} -V _{B0'_x} ) * (posCurPU_X-posB0'_X) / RefPU_width , V _{0_y} = V _{B0'_y} + (V _{B1'_x} -V _{B0'_x} ) * (posCurPU_Y-posB0'_Y) / RefPU_width + (V _{B1'_y} -V _{B0'_y} ) * (posCurPU_X-posB0'_X) / RefPU_width; V _{1_x} = V _{0_x} + (V _{B1'_x} -V _{B0'_x} ) * PU_width / RefPU_width, V _{1_y} = V _{0_y} + (V _{B1'_y} -V _{B0'_y} ) * PU_width / RefPU_width. (8)

Considering that the line buffer storing the motion vector of the top CTU is much larger than the line buffer storing the motion vector of the left CTU, in one example, there is no need to limit the value of K in this second embodiment. By setting K to Equal to the width of the largest coding unit divided by N (K = CTU_width / N), all motion vectors of the left CTU are stored.

The third embodiment. In the third embodiment, the affine parameters or control points for each fixed-size block or each CU are intentionally stored. In equation (3), the motion vector Mv0 = (V _0x , V _0y ) of the upper left NxN sub-block and the motion vector Mv1 = (V _1x , V _1y ) of the upper right NxN sub-block are used to derive all NxN in the current block. Subblock motion vector. The current block is a CU or a prediction unit (PU). The derived motion vector can be represented by the motion vector Mv0 plus the position-dependent MV offset. From Equation (3), in order to derive the motion vector of the NxN sub-block located at position (x, y), the horizontal MV offset H_MV_offset and the vertical MV offset V_MV_offset are shown in Equation (9).

H_MV_offset = (V _1x -V _0x ) * N / w, (V _1y -V _0y ) * N / w;

V_MV_offset =-(V _1y -V _0y ) * N / w, (V _1x -V _0x ) * N / w. (9)

For the six-parameter affine motion model, the motion vector Mv0 of the upper left NxN sub-block Mv0 = (V _0x , V _0y ), the motion vector of the upper right NxN sub-block Mv1 = (V _1x , V _1y ), and the motion of the bottom left NxN sub-block The vector Mv2 = (V _2x , V _2y ) is used to derive the motion vectors of all NxN sub-blocks in the current block. The motion vector field of each pixel A (x, y) in the current block can be derived based on three motion vectors Mv0, Mv1, and Mv2 according to equation (10).

Similarly, in order to derive the motion vector (V _x , V _y ) of the NxN sub-block located at the position (x, y) according to the six-parameter affine motion model, the horizontal MV offset H_MV_ offset and the vertical MV offset H_MV _ Bias is shown in equation (11).

H_MV_offset = (v _1x -v _0x ) * N / w, (v _1y -v _0y ) * N / w; V_MV_offset = (v _2x -v _0x ) * N / h, (v _2y -v _0y ) * N / h; (11), where w and h in the expressions (9) and (11) are the width and height of the current block encoded in the affine mode.

When the motion vector of the center pixel of the NxN sub-block is assigned as the motion vector of the control point, the denominators in equations (6) and (8) are reduced by N. For example, equation (6) can be rewritten as follows.

V _{0_x} = V _{B0'_x} + (V _{B2_x} -V _{B0'_x} ) * (posCurPU_Y-posB0'_Y) / (N) + (V _{B1'_x} -V _{B0'_x} ) * (posCurPU_X-posB0'_X) / (RefPU_width-N)

V _{0_y} = VB _{0'_y} + (V _{B2_y} -V _{B0'_y} ) * (posCurPU_Y-posB0'_Y) / (N) + (V _{B1'_y} -V _{B0'_y} ) * (posCurPU_X-posB0'_X) / (RefPU_width-N) (12)

The third embodiment stores affine parameters, for example, MV offsets for the horizontal and vertical directions of the encoded block. The coded block can be a fixed-size MxM block or a CU. The size of the fixed-size MxM block may depend on the minimum size that allows affine motion prediction to be applied. In one example, if the minimum affine inter mode or affine merge mode has a block size of 8 × 8, then M is equal to 8. For each MxM block or for each CU, include a horizontal direction MV offset (V _1x -V _0x ) * N / w, (V _1y -V _0y ) * N / w and a motion vector of NxN sub-blocks such as Mv0 The affine parameters of (V _0x , V _0y ) are stored for a four-parameter affine motion model using upper left and upper right control points. Affine parameters including a vertical MV offset (V _2x -V _0x ) * N / h, (V _2y -V _0y ) * N / h and a motion vector of the NxN sub-block such as Mv0 (V _0x , V _0y ) It is stored for a four-parameter affine motion model using the upper left and lower left control points. If a six-parameter affine motion model using upper left, upper right, and lower left control points is applied, it includes horizontal MV offset (V _1x -V _0x ) * N / w, (V _1y -V _0y ) * N / w And the vertical MV offset (V _2x -V _0x ) * N / h, (V _2y -V _0y ) * N / h, and an affine of a motion vector of the NxN sub-block such as Mv0 (V _0x , V _0y ) The parameters are stored. The affine motion vector in the affine candidate may be derived from stored affine parameters of one or more MxM blocks or CUs corresponding to neighboring blocks.

To maintain accuracy, the horizontal or vertical MV offset is multiplied by a scale number, where the scale number can be a predefined number or the scale number can be set equal to the maximum coding unit or CTU size. For example, the scaled horizontal MV offset ((V _1x -V _0x ) * S / w, (V _1y -V _0y ) * S / w) and the scaled vertical MV offset ((V _2x -V _0x ) * S / h, (V _2y -V _0y ) * S / h) are stored. Some examples of the scaling number S are set equal to the CTU size or a quarter of the CTU size.

In another example, the motion vectors of two or three control points per MxM block or each CU are stored. Motion vectors can be stored in a line buffer. The affine motion vector in the affine candidate for predicting the current block is derived from a stored motion vector corresponding to a control point of an adjacent block.

FIG. 5 shows an exemplary flowchart of a video coding system with affine motion compensation combined with a third embodiment of the present invention. In step S50, the video encoding system receives input data associated with the current block encoded or to be encoded in the affine mode. In step S52, the video encoding system calculates and stores On the affine parameters of the neighboring block, and in step S54, obtain the affine parameters of the neighboring block corresponding to the current block. In step S56, an affine candidate including an affine motion vector is derived according to the obtained affine parameters. Next, in step S58, the video encoding system uses the affine candidate to encode or decode the current block in the affine mode.

Fourth embodiment. In the fourth embodiment, compared with the existing temporal MV buffer used for HEVC, motion vectors in another NxN block column and another NxN block row are stored in the temporal MV buffer used for affine motion compensation. . The concept of the fourth embodiment is similar to that of the second embodiment, but the original motion vectors of the sub-blocks located at the top column and the leftmost row of one or more adjacent coded blocks are stored instead of the Alternative motion vector. In this embodiment, the time buffer stores the original motion vectors of the first adjacent NxN block column, the first adjacent NxN block row, the top NxN block column set, and the leftmost NxN block row set. The first neighboring NxN block column is the last column in one or more upper neighboring blocks that is closest to the upper boundary of the current block, and the first neighboring NxN block row is one or more that is closest to the left boundary of the current block. The last row of the left adjacent block. The top NxN block column set includes the first column of one or more upper neighboring blocks, and the leftmost NxN block row set includes the first rows of one or more left neighboring blocks.

FIG. 6 shows an example of affine candidate derivation by storing motion vectors of two NxN block columns and two NxN block rows. As shown in FIG. 6, the original motion vectors V _A1 and V _B2 of the first NxN block row adjacent to the left block boundary of the current block 60 and the sub-blocks 614 and 626 of the first NxN block column above the current block 60 are stored in the buffer. Device. The first NxN block row and the first NxN block column in the adjacent blocks 61, 62, and 63 are the rows and columns closest to the current block 60. Original motion vectors V _A2 , V _A0 of the leftmost NxN block rows of neighboring blocks 61 and 62 or the sub-blocks in top NxN block columns of neighboring blocks 62 and 63 616, 612, 636, 632, 634, 622, and 624 , V _E2 , V _E0 , V _E1 , V _B0 , and V _B1 are also stored in the buffer. For example, these original motion vectors V _A2 , V _A0 , V _E2 , V _E0 , V _E1 , V _B0 , and V _B1 of the leftmost NxN block row or top NxN block column of the neighboring block cover the second NxN for storing Buffer space for motion vectors of the block columns and subblocks 615, 613, 637, 633, 635, 623, and 625 in the second NxN block row. In this embodiment, the original motion vectors of the control points of the neighboring blocks are stored in a time MV buffer with the overhead of only one additional MV column and one additional MV row.

FIG. 7 shows an exemplary flowchart for a video encoding system with affine motion compensation in combination with the second or fourth embodiment of the present invention. In step S70, input data associated with the current block in the current picture is received on the video encoder side or a video bit stream corresponding to the compressed data including the current block is received on the video decoder side. Step S72: Obtain two or more motion vectors of neighboring blocks from the time buffer, wherein at least one of the acquired motion vectors does not correspond to the first neighboring NxN block column or the first neighboring NxN block row of the current block. Subblock. In step S74, the video encoding system uses the obtained motion vectors to derive affine candidates, and in step S76, one or more motion compensation blocks derived from the derived affine candidates are used to encode or decode the current block by predicting the current block. Piece. The affine motion vector predicts motion between points of the current block and corresponding points of one or more motion-compensated blocks. The time buffer stores selected motion vectors that are less than all motion vectors of the coded block in the current picture. According to the second embodiment, at least one of the acquired motion vectors is a substitute motion vector, and according to the fourth embodiment, all the acquired motion vectors are original motion vectors of control points of neighboring blocks.

Fifth embodiment. In the fifth embodiment, the video coding system with affine motion compensation reuses the existing temporal MV buffers required by the HEVC standard, so no additional buffers are needed. In other words, in this embodiment, the affine motion prediction only requires a motion vector of one adjacent NxN block column and one adjacent NxN block row. FIG. 8 shows an example of affine candidate derivation using motion vectors of one adjacent NxN block column and one adjacent NxN block row. As shown in FIG. 8, two motion vectors in the closest NxN block column or the closest NxN block row among the neighboring blocks of the current block 80 are used to derive an affine candidate for the current block. For example, when the neighboring block 82 located above the current block 80 is selected to derive the affine candidate, the motion vectors V _B2 and V _B3 in the sub-blocks 826 and 828 are obtained to derive the affine according to the four-parameter affine motion model Affine motion vectors among the candidates; when the neighboring block 81 on the left side of the current block 80 is selected to derive the affine candidate, the motion vectors V _A1 and V _A3 in the sub-blocks 814 and 818 are obtained to simulate according to the four parameters The affine motion model derives the affine motion vector among the affine candidates. In this embodiment, the neighboring block 83 adjacent to the upper left corner of the current block 80 is not a valid neighboring block for affine candidate derivation.

Equation (13) demonstrates an example of modifying equation (8) according to the four-parameter affine motion model, which is used to derive the affine motion vector Mv0 among the affine candidates of motion vectors V _B2 and V _B3 from neighboring blocks 82 And Mv1.

V _{0_x} = V _{B2_x-} (V _{B3_y} -V _{B2_y} ) * (posCurPU_Y-posB2_Y) / RefPU _B _width + (V _{B3_x} -V _{B2_x} ) * (posCurPU_X-posB2_X) / RefPU _B _width,

_{V 0_y = V B2_y + (V} B3_x -V B2_x) * (posCurPU_Y-posB2_Y) / RefPU B _width + (V B3_y -V B2_y) * (posCurPU_X- posB2_X) / RefPU B _width; V 1_x = V 0_x + (V B3_x -V _{B2_x} ) * PU_width / RefPU _B _width, V _{1_y} = V _{0_y} + (V _{B3_y} -V _{B2_y} ) * PU_width / RefPU _B _width; (13) where (V _{0_x} , V _{0_y} ) represents the upper left corner of the current block 80 , And (V _{1_x} , V _{1_y} ) represents the motion vector Mv1 in the upper right corner of the current block 80. The coordinates (posCurPU_X, posCurPU_Y) represent the pixel position of the upper-left sample of the current block 80 related to the upper-left sample of the current picture. The coordinates (posB2_X, posB2_Y) represent the pixel position of the upper-left sample of the sub-block 826 related to the upper-left sample of the current picture. RefPU _B _width denotes the width of the adjacent blocks 82 and the width of the current PU_width block 80 indicates.

The sixth to ninth embodiments described below are related to implementing an adaptive motion vector resolution (AMVR) with affine motion compensation. AMVR provides a flexible solution to reduce motion vector difference (MVD) transmission overhead by adaptively limiting MVD to integer pixel resolution. For CU or PU, the AMVR flag is identified to indicate whether to use pixel resolution or use fractional pixel resolution. The implementation suggestion of affine motion compensation disclosed in the literature does not identify the AMVR flags used for CU or PU encoding in affine mode, so AMVR is always disabled on affine coding blocks. By default, the affine motion vectors used for affine all control points in an encoded block are fractional pixel resolution.

The sixth embodiment. In a sixth embodiment, affine motion compensation is used to enable adaptive motion vector resolution, and the AMVR flag is identified for each affine coded block. In one example, the AMVR flag only controls whether the resolution of the MVD used to affine the encoded block is an integer pixel resolution or a fractional pixel resolution. In another example, the AMVR flag controls the resolution of the MVD and the resolution of the motion vector predictor (MVP) used to affine the encoded block. Therefore, if the AMVR flag indicates that the integer pixel resolution is When used, the final motion vector is of integer resolution. For the current block encoded in the affine mode (for example, affine inter mode), there are M MVDs calculated from M corresponding MVPs, where M represents the number of control points for the current block. In this example, M is selected from 0, 1, 2, 3, and 4. The M corresponding MVPs are affine motion vectors among the affine candidates, and each affine motion vector is a predictor for affine a motion vector of a control point of a coded block. If the current block is encoded in the affine inter mode and the AMVR flag is true, it indicates that an integer pixel resolution is used for the current block, and the MVD of the control point is an integer pixel resolution. In the case where the AMVR flag is also used to adjust the resolution of the MVP, if the AMVR flag is true, all MVPs associated with the MVD of the control point are also rounded to the integer pixel resolution. In the case where the AMVR flag is only used to adjust the resolution of the MVD, when the AMVR flag is true, the MVP associated with the MVD of the control point may be a fractional pixel resolution. For those control points that are not associated with MVD, when the MVD is presumed to be zero, the MVP for control points is still fractional pixel resolution.

The seventh embodiment. In the seventh embodiment, in the syntax design, the MVD is identified before the corresponding AMVR flag and is used to affine the coded block. If there is at least one non-zero MVD of the control point of the affine coded block, The pixel resolution of a non-zero MVD or at least one decoded motion vector is determined according to the AMVR flag. The decoded motion vector is derived by selecting the MVD and the corresponding MVP in the affine candidate for affine the encoded block. If the MVDs of all control points used to affine the encoded block are zero, the MVP of the control points can be maintained at a fractional pixel resolution, and the AMVR flag need not be identified in the video bitstream.

Eighth embodiment. In this embodiment, the bidirectional prediction is disabled to limit the blocks encoded in the affine inter mode to one-way prediction to reduce system complexity and MVD overhead. For example, if the affine flag indicates that the current block is encoded or will be encoded in affine inter mode, the inter prediction direction interDir for the current block is set to 0 or 1, where 0 indicates the list 0 unidirectional prediction And 1 indicates a unidirectional prediction of list 1. In an example of the eighth embodiment, bidirectional prediction is allowed for affine inter-coded blocks only when the MVD of the affine inter-coded blocks is an integer pixel resolution. In other words, enabling or disabling bidirectional prediction for affine inter-coded blocks depends on the value of the AMVR flag for affine inter-coded blocks. When the AMVR flag indicates that the MVD of the affine inter-encoded block is an integer pixel resolution, the MVD overhead is relatively small, so bidirectional prediction is allowed for affine inter-encoded blocks.

The ninth embodiment. The video encoding method or video encoding system implementing the ninth embodiment determines the motion vector resolution for the current block according to the inter prediction direction interDir of the current block and whether the current block is encoded in the affine inter mode. In this embodiment, in the CU syntax structure, the inter prediction direction interDir and the affine flag for the current block are identified before the AMVR flag for the current block. Therefore, when interDir is equal to 2, it indicates bidirectional The prediction is applied to the current block, and the affine flag is true, indicating that the affine inter mode is used, the AMVR flag is presumed to be true and does not need to be identified in the current block.

FIG. 9 shows an exemplary system block diagram of a video encoder 900 based on HEVC with affine motion compensation according to an embodiment of the present invention. Intra prediction 910 provides intra predictors based on the reconstructed video data of the current picture. However, affine prediction 912 performs motion estimation (ME) and motion compensation (MC) based on the affine motion model. Video data for images provides predictors. Each block selection of the current picture processed by affine prediction 912 will be coded by affine inter prediction 9122 in affine inter mode or will be coded by affine merge prediction 9124 in affine merge mode. For blocks encoded in affine inter mode or affine merge mode, the final affine candidate is selected from one or more affine candidates using an affine motion model derived from the final affine candidate to derive one or more compensations Block, and one or more compensation blocks are used to predict the block. Affine candidates can be derived using one embodiment with a temporal MV buffer, which stores selected motion vectors that are less than all motion vectors of the coded block in the current picture. The affine merge prediction 9124 constructs one or more affine merged candidate and inserts one or more affine merged candidate into the merge candidate list according to the motion vectors of one or more neighboring encoded blocks. The affine merge mode allows inheritance of affine motion vectors located at the control points of neighboring coded blocks; therefore, motion information is identified only by the merge index. The merge index used to select the final affine candidate is then identified in the encoded video bitstream. The affine merge selection can be derived using an embodiment of the present invention. For blocks encoded in affine inter-mode, motion information (e.g., MVD) between the affine motion vector in the final affine candidate and the motion vector at the control point of the block is encoded in the encoded video bitstream . According to an embodiment of the present invention, the MVD of the block encoded in the affine inter-mode The resolution is integer pixel resolution or fractional pixel resolution. The switch 914 selects one of the outputs of the intra prediction 910 and the affine prediction 912, and applies the selected predictor to the adder 916 to form a prediction error, which is also referred to as a residual signal.

The prediction residual signal is further processed by Transformation (T) 918, and then Quantization (Q) 920 is performed. The transformed and quantized residual signal is then encoded by an entropy encoder 934 to form an encoded video bit stream. The encoded video bitstream and auxiliary information (eg, merge index and MVD) are encapsulated. The data associated with the auxiliary information is also provided to the entropy encoder 934. When motion-compensated prediction mode is used, the reference picture or picture must also be reconstructed on the encoder side. The transformed and quantized residual signals are processed by Inverse Quantization (IQ) 922 and Inverse Transformation (IT) 924 to recover the reference picture or the predicted residual signal of the picture. As shown in FIG. 9, the prediction residual signal is restored by adding back the selected predictor to Reconstruction (REC) 926 to generate reconstructed video data. The reconstructed video data may be stored in a reference picture buffer (Ref. Pict. Buffer) 932 and used for prediction of other pictures. Due to the encoding process, the reconstructed video data from REC 926 may be damaged in various ways. Therefore, before being stored in the reference picture buffer 932, the loop processing deblocking filter (DF) 928 and the sample adaptive offset (Sample Adaptive Offset (SAO) 930 is applied to reconstruct video data to further enhance picture quality. DF information from DF 928 and SAO information from SAO 930 are also provided to the entropy encoder 934 for merging into the encoded video bitstream. The temporal MV buffer used to store the motion vector derived from the affine candidate can be obtained from the reference picture The chip buffer 932 or any other memory coupled to the affine prediction 912 is implemented.

FIG. 10 shows a corresponding video decoder 1000 of the video encoder 900 of FIG. 9. The encoded video bit stream is input to video decoder 1000 and decoded by entropy decoder 1010 to recover transformed and quantized residual signals, DF and SAO information, and other system information. The decoding process of the decoder 1000 is similar to the reconstruction loop of the encoder 900, except that the decoder 1000 only needs motion compensation (MC) 1014 in the affine prediction 1014. Affine prediction 1014 includes affine inter prediction 10142 and affine merge prediction 10144. The block encoded in the affine inter mode is decoded by the affine inter prediction 10142, and the block encoded in the affine merge mode is decoded by the affine merge prediction 10144. The final affine candidate is selected for a block encoded in the affine inter mode or the affine merge mode, and one or more compensation blocks are derived from the final affine candidate. The final affine candidate may be derived according to one of the embodiments of the present invention with a temporal MV buffer that stores the selected MVs of all MVs that are less than the encoded block of the current picture. The switch 1016 selects an intra predictor from the intra prediction 10512 or an affine predictor from the affine prediction 1014 according to the decoding mode information. The transformed and quantized residual signals are recovered through IQ 1020 and IT 1022. The restored transform and the quantized residual signal are reconstructed by adding the predictors back to REC 1018 to produce a reconstructed video. The reconstructed video is further processed by DF 1024 and SAO 1026 to produce the final decoded video. If the currently decoded picture is a reference picture, the reconstructed video of the currently decoded picture is also stored in the reference picture buffer 1028. A temporal MV buffer for storing motion vectors derived from affine candidates may be reference picture buffer 1028 or coupled to affine prediction 1014 Any other memory.

Various components of the video encoder 900 and the video decoder 1000 in FIG. 9 and FIG. 10, and the various video encoding processes described in the described embodiments can be configured by hardware components to execute program instructions stored in the memory. One or more processors, or a combination of hardware and processors. For example, the processor executes program instructions to control the reception of input data associated with the current block. The processor is equipped with a single or multiple processing cores. In some examples, the processor executes program instructions to perform functions of certain components in the encoder 900 and the decoder 1000, and the memory electrically coupled to the processor is used to store program instructions and information corresponding to the affine mode , Reconstructed images of blocks, and / or intermediate data during encoding or decoding. In some embodiments, the memory includes non-transitory computer-readable media, such as: semiconductor or solid-state memory, random access memory, read-only memory, hard disk, optical disk, or other suitable storage medium. The memory may also be a combination of two or more of the non-transitory computer-readable media listed above. As shown in FIG. 9 and FIG. 10, the encoder 900 and the decoder 1000 can be implemented by the same electronic device. Therefore, if implemented in the same electronic device, components of various functions of the encoder 900 and the decoder 1000 can be implemented. Be shared or reused. For example, one or more of the reconstruction 926, transform 918, quantization 920, deblocking filtering 928, sample adaptive offset 930, and reference picture buffer 932 in FIG. 9 may be used in FIG. 10, respectively. Functions of reconstruction 1018, transform 1022, quantization 1020, deblocking filtering 1024, sample adaptive offset 1026, and reference picture buffer 1028. In some examples, a portion of intra prediction 910 and affine prediction 912 in FIG. 9 may share or reuse a portion of intra prediction 1012 and affine prediction 1014 in FIG. 10.

Although the first to ninth embodiments of the video encoding method with affine motion compensation are described, the present invention is not limited to these embodiments. In each embodiment, the selection of the video coding method with affine motion compensation is an example showing each embodiment and should not be construed as a limitation or requirement for any embodiment of the present invention. The above description is presented to enable one of ordinary skill in the art to implement the invention in the context of a particular application and its requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the principles defined by this disclosure may be applied to other embodiments. Therefore, the present invention is not limited to the specific embodiments shown and described, but is intended to conform to the widest scope consistent with the principles and novel features of the present disclosure. In the above detailed description, various specific details are shown to provide a thorough understanding of the present invention. However, those skilled in the art will understand that the present invention can be implemented.

The invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The described examples are to be considered in all respects only as illustrative and not restrictive. Therefore, the scope of the invention is indicated by the appended claims rather than the above description. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.

60‧‧‧current block

61, 62, 63‧‧‧adjacent blocks

616, 612, 636, 632, 634, 622, 624, 615, 613, 637, 633, 635, 623, 625, 614, 626‧‧‧ subblocks

Claims (23)

A method for video coding with affine motion compensation in a video coding system includes: receiving input data associated with a current block in a current picture at a video encoder or receiving corresponding information including the current picture at a video decoder. A video bit stream of compressed data of the current block, wherein, according to the affine motion model, the current block is encoded or will be encoded in the affine mode; two or more adjacent blocks are obtained from the time buffer A motion vector for the current block, wherein at least one of the acquired motion vectors does not correspond to a first adjacent NxN block column closest to an upper boundary of the current block or a left boundary closest to the current block Sub-blocks of the first adjacent NxN block row, wherein the time buffer stores selected motion vectors that are less than all motion vectors of previously coded blocks in the current picture, and NxN is used in the time buffer A block size for storing one motion vector; using the acquired two or more motion vectors of the neighboring block to derive an affine candidate including a plurality of affine motion vectors; and Predicting the current block using one or more motion-compensated blocks derived from the derived affine candidate to encode or decode the current block, wherein the affine motion vector predicts the points of the current block and the The motion between corresponding points of one or more motion compensation blocks is described. The method according to item 1 of the patent application scope, wherein the time buffer stores motion vectors of 2 NxN block columns and 2 NxN block rows, and the 2 NxN block columns include the closest to the current block The first adjacent NxN block column and the second adjacent NxN block column of the upper boundary, and the 2 NxN The block line includes the first adjacent NxN block line and the second adjacent NxN block line that are closest to the left boundary of the current block. The method according to item 1 of the scope of patent application, wherein if the adjacent block is located above or adjacent to the upper left corner of the current block, the two or more obtained motion vectors include the first And a second motion vector, the first motion vector is a replacement motion vector replacing the original motion vector of the upper left corner of the adjacent block, and the second motion vector is the original motion replacing the upper right corner of the adjacent block A replacement motion vector of a vector; wherein, if the adjacent block is located on the left side of the current block, the first motion vector is a replacement motion vector that replaces the original motion vector of the upper left corner of the adjacent block, And the second motion vector is an original motion vector in an upper right corner of the neighboring block. The method according to item 1 of the scope of patent application, wherein the obtained two or more motion vectors include first and second motion vectors of first and second sub-blocks in the adjacent block, so The deriving the affine candidate further includes: using the first and second motion vectors, a pixel position of the current block is related to the current picture, and a pixel position of the first sub-block is related to the current picture and the A first affine motion vector of the affine motion vector is derived by correlating the widths of neighboring blocks; and using the first and second motion vectors, the width of the neighboring block, and the current block Width to derive the second affine motion vector. The method according to item 1 of the patent application scope, wherein the two or more obtained motion vectors include first, second, and first and second sub-blocks of the neighboring block And a third motion vector, where the guide The affine candidate further includes: using at least two motion vectors of the first, second and third motion vectors, the pixel position of the current block is related to the current picture, and the first sub-block The pixel position of is related to the width of the current picture and the neighboring block to derive a first affine motion vector of the affine motion vector; using the first and second motion vectors, the neighboring block's Deriving a second affine motion vector using the width and the width of the current block; and using the first and third motion vectors and at least one of the height of the neighboring block and the height of the current block To derive a third affine motion vector. The method according to item 1 of the patent application scope, wherein the two or more obtained motion vectors include first, second, and third motion vectors, and if the adjacent block is located above the current block, Then the first motion vector is a replacement motion vector replacing the original motion vector of the upper left corner of the adjacent block, and the second motion vector is a replacement motion vector replacing the original motion vector of the upper right corner of the adjacent block And the third motion vector is an original motion vector of a lower left corner of the neighboring block; wherein if the neighboring block is adjacent to an upper left corner of the current block, the first motion vector is used instead of the phase An alternative motion vector of the original motion vector of the upper left corner of the neighboring block, the second motion vector is an alternative motion vector of the original motion vector of the upper right corner of the adjacent block, and the third motion vector is an alternative to the A replacement motion vector of the original motion vector of the lower left corner of the neighboring block; and wherein if the neighboring block is located on the left side of the current block, the first motion The vector is a replacement motion vector replacing the original motion vector of the upper left corner of the adjacent block, the second motion vector is the original motion vector of the upper right corner of the adjacent block, and the third motion vector is the replacement The replacement motion vector of the original motion vector of the lower left corner of the neighboring block is described. The method according to item 1 of the scope of patent application, wherein, according to a four-parameter affine motion model having two control points or a six-parameter affine motion model having three control points, the current block is determined by the One or more motion compensation blocks are predicted by the affine motion vector of the affine candidate. The method according to item 1 of the scope of patent application, wherein the time buffer stores M NxN block columns above the current block, and M is less than the maximum coding unit height divided by N. The method according to item 1 of the scope of patent application, wherein the time buffer stores K NxN block rows to the left of the current block, and K is less than the maximum coding unit width divided by N. The method according to item 1 of the scope of patent application, wherein the obtained two or more motion vectors are original motion vectors of two or more control points of the adjacent block. The method of claim 10, wherein the control points of the adjacent blocks include at least two of an upper left corner, an upper right corner, a lower right corner, and a lower left corner of the adjacent block. The method of claim 10, wherein the time buffer stores the first adjacent NxN block column, the first adjacent NxN block row, the top NxN block column set, and the leftmost NxN block row set. The original motion vector, wherein the first adjacent NxN block column is the closest to the upper boundary of the current block The last column of at least one upper adjacent block, the first adjacent NxN block row is the last row of at least one left adjacent block closest to the left boundary of the current block, and the top NxN block column set includes A first column in the at least one upper adjacent block, and the leftmost NxN block row set includes a first row of the at least one left adjacent block. A method for video coding with affine motion compensation in a video coding system includes: receiving input data associated with a current block in a current picture at a video encoder or receiving corresponding information including the current picture at a video decoder. A video bit stream of compressed data of the current block, wherein the current block is encoded or will be encoded in an affine mode according to an affine motion model; calculation and storage of multiple encoded blocks in the current picture are performed and stored Affine parameters; acquiring the affine parameters of one or more coded blocks corresponding to neighboring blocks of the current block from the time buffer; using the acquired affine of the neighboring blocks The parameter derivation includes an affine candidate including a plurality of affine motion vectors; and encoding or decoding the current block by predicting the current block using one or more motion-compensated blocks derived from the derived affine candidate, wherein , The affine motion vector predicts motion between points of the current block and corresponding points of the one or more motion compensation blocks. The method according to item 13 of the application, wherein the affine parameters include horizontal motion vector offsets, vertical motion vector offsets, and two or more of a motion vector of the neighboring blocks. many. The method according to item 14 of the scope of patent application, wherein the motion vector in the adjacent block is the upper left corner, the upper right corner, the left bottom, or the neighboring block Motion vector at bottom right. The method of claim 13, wherein the affine parameter stored in the time buffer includes a scaled motion vector offset for the coded block. The method according to item 13 of the application, wherein the affine parameters include two or three affine motion vectors representing motion vectors of two or three control points, and the time buffer storage The two or three affine motion vectors for the coded block. A method for video coding with affine motion compensation in a video coding system includes: receiving input data associated with a current block in a current picture at a video encoder or receiving a video decoder corresponding to the current picture included in the current picture. A video bitstream of compressed data of a block, wherein the current block is encoded or will be encoded in affine mode according to the affine motion model; two or more motion vectors of valid neighboring blocks are received from the time buffer For the current block, wherein the effective neighboring block does not include a neighboring block adjacent to the upper left corner of the current block, and wherein the time buffer stores a column of neighboring NxN blocks of the current block And a motion vector of a row of adjacent NxN blocks, where NxN is a block size used to store a motion vector in the time buffer; using the acquired two or more motion vectors of the effective neighboring block to Deriving affine candidates including multiple affine motion vectors; and Encoding or decoding the current block by predicting the current block using one or more motion compensated blocks derived from the derived affine candidate, wherein the affine motion vector predicts a point of the current block and Motion between corresponding points of the one or more motion compensation blocks. The method according to item 18 of the scope of patent application, wherein the obtained two or more motion vectors correspond to two motion vectors, and if the effective neighboring block is located above the current block, the One of the two motion vectors is an original motion vector located at a lower left corner of the effective neighboring block, and the other of the two motion vectors is an original motion vector located at a lower right corner of the valid neighboring block; wherein If the effective neighboring block is located on the left side of the current block, one of the two motion vectors is the original motion vector in the upper right corner of the effective neighboring block, and the The other is the original motion vector in the lower right corner of the effective neighboring block. The method as described in claim 18, wherein according to a four-parameter affine motion model having two control points, the current block is determined by the one through the affine motion vector in the affine candidate. Or multiple motion-compensated blocks to predict. A video encoding device with affine motion compensation in a video encoding system, the device includes one or more electronic circuits configured to receive input data associated with a current block in a current picture at a video encoder or at a video. The decoder receives a video bit stream corresponding to the compressed data including the current block in the current picture, wherein the current block is encoded or to be encoded in an affine mode according to an affine motion model; Obtain two or more motion vectors of neighboring blocks from the time buffer for the current block, wherein at least one of the acquired motion vectors does not correspond to the first closest to the upper boundary of the current block A neighboring NxN block column or a sub-block of a first neighboring NxN block row closest to the left boundary of the current block, wherein the time buffer stores less than A motion vector has been selected, and NxN is the block size used to store one motion vector in the time buffer; using the acquired two or more motion vectors of the neighboring block to derive a plurality of affine motion vectors An affine candidate; and encoding or decoding the current block by predicting the current block using one or more motion-compensated blocks derived from the derived affine candidate, wherein the affine motion vector predicts Describe the motion between points of the current block and corresponding points of the one or more motion compensation blocks. A video encoding device with affine motion compensation in a video encoding system, the device includes one or more electronic circuits configured to receive input data associated with a current block in a current picture at a video encoder or at a video. The decoder receives a video bit stream corresponding to the compressed data including the current block in the current picture, wherein the current block is encoded or will be encoded in an affine mode according to an affine motion model; calculation and storage Affine parameters for a plurality of coded blocks in the current picture; obtaining the affine parameters of one or more coded blocks corresponding to neighboring blocks of the current block from the time buffer; Using the obtained affine parameters of the neighboring block to derive a plurality of affine An affine candidate for a motion vector; and encoding or decoding the current block by predicting the current block using one or more motion-compensated blocks derived from the derived affine candidate, wherein the affine motion vector Predicting motion between points of the current block and corresponding points of the one or more motion-compensated blocks. A video encoding device with affine motion compensation in a video encoding system, the device includes one or more electronic circuits configured to receive input data associated with a current block in a current picture at a video encoder or at a video. The decoder receives a video bit stream corresponding to the compressed data including the current block in the current picture, wherein the current block is encoded or to be encoded in an affine mode according to an affine motion model; from time buffering The processor receives two or more motion vectors of a valid neighboring block for the current block, wherein the valid neighboring block does not include a neighboring block adjacent to an upper left corner of the current block, wherein the time The buffer stores a motion vector of an adjacent NxN block column and an adjacent NxN block row of the current block, and NxN is a block size for storing a motion vector in the time buffer; using the effective adjacent Deriving an affine candidate including a plurality of affine motion vectors by said acquired two or more motion vectors of a block; and by using said derived affine candidate One or more motion-compensated blocks predict the current block to encode or decode the current block, wherein the affine motion vector predicts one of the points of the current block and the corresponding points of the one or more motion-compensated blocks. Between sports.