KR20210001768A - Video encoding and decoding method and apparatus using complex motion information model - Google Patents

Abstract

The present invention provides a method and device for encoding and decoding a video using a complex motion information model to improve coding efficiency of a video signal. The present invention suggests a method for deriving motion information used for generating a MERGE/AMVP candidate list.

Description

Video encoding and decoding method and apparatus using complex motion information model{.}

The present invention relates to a method and apparatus for encoding/decoding an image, and more particularly, to a method and apparatus for encoding and decoding for efficiently determining motion information in a plurality of triangulated subblocks in a current block in a triangle merge candidate list; The present invention relates to an encoding and decoding method and apparatus for efficiently determining the size of a subblock in a current block when performing Affine prediction.

Recently, the demand for multimedia data such as video is increasing rapidly on the Internet. However, it is difficult to keep up with the rapidly increasing amount of multimedia data at the rate of development of the bandwidth of the channel. Accordingly, ITU-T's VCEG (Video Coding Expert Group) and ISO/IEC's Moving Picture Expert Group (MPEG), an international standardization organization, revised version 1 of HEVC (High Efficiency Video Coding), a video compression standard, in February 2014. Established.

In HEVC, technologies such as intra prediction (or intra prediction), inter prediction (or inter prediction), transformation, quantization, entropy coding, and in-loop filter are defined.

The present invention is to improve the coding efficiency of a video signal.

The present invention proposes a motion information derivation method used when generating a MERGE/AMVP candidate list.

The video signal processing method and apparatus according to the present invention can improve video signal coding efficiency.

1 is a schematic flowchart of an image encoding apparatus.
2 is a flowchart schematically showing an apparatus for decoding an image.
3 is a diagram for describing in detail a prediction unit of an image encoding apparatus.
4 is a flowchart illustrating a method of encoding prediction information of an image encoding apparatus.
5 is a diagram illustrating in detail a prediction unit of an image decoding apparatus.
6 is a flowchart illustrating a method of decoding prediction information of an image decoding apparatus.
7 is a diagram for describing a method of constructing a merge candidate list.
8 is a flowchart illustrating a method of encoding/decoding merge information in an image encoding device and an image decoding apparatus.
9 is a view for explaining a method of triangulating a current block for a triangle merge mode according to the present embodiment.
FIG. 10 is a diagram illustrating a method of determining candidate motion information applied to triangular blocks divided when performing a triangle merge mode in a merge candidate list according to the present embodiment.
11 is a diagram for describing a method of encoding triangle merge mode information according to the present embodiment.
12 is a diagram for explaining a method of determining a size and shape of a sub prediction block when Affine prediction is performed in a current block.

1 is a block flow diagram schematically showing the configuration of an image encoding apparatus. An image encoding apparatus is a device that encodes an image and includes a block division unit, a prediction unit, a transform unit, a quantization unit, an entropy encoding unit, an inverse quantization unit, an inverse transform unit, an addition unit, an in-loop filter unit, a memory unit, and a subtraction unit. can do.

The block dividing unit 101 divides a block to be encoded with a maximum size (hereinafter, referred to as a maximum coding block) to a block to be encoded with a minimum size (hereinafter, referred to as a minimum coding block). There are various methods of block partitioning. The quad-tree division (hereinafter referred to as QT (Quad-Tree) division) is a division that accurately divides the current coding block. Binary-tree division (hereinafter referred to as BT (Binary-Tree) division) is a division in which a coded block is accurately divided in a horizontal direction or a vertical direction. The ternary-tree division (hereinafter referred to as TT (Ternary-Tree) division) is a division in which a coded block is divided in a ratio of 1:2:1 in a horizontal direction or a vertical direction. BT segmentation and TT segmentation may be collectively referred to as MTT (Multi-Type Tree) segmentation. In addition to this, there may be various ways of dividing. In addition, it is possible to divide by considering several division methods at the same time.

The prediction unit 102 generates a prediction block using pixels adjacent to a block to be predicted (hereinafter, referred to as a prediction block) from the current original block or pixels in a reference picture that has been previously encoded/decoded. In the prediction block, one or more prediction blocks may be generated within the coding block. When there is one prediction block in the coding block, the prediction block has the same shape as the coding block. The video signal prediction technology is largely composed of intra prediction and inter prediction. The intra prediction is a method of generating a prediction block using neighboring pixels of the current block, and inter prediction has already been encoded/decoded. This is a method of generating a prediction block by finding a block most similar to the current block from the ended reference picture. Thereafter, an optimal prediction mode of the prediction block is determined using various techniques such as rate-distortion optimization (RDO) for the residual block obtained by subtracting the original block and the prediction block. The RDO cost calculation formula is shown in Equation 1.

D, R, J are the deterioration due to quantization, the rate of the compressed stream, and the RD cost, respectively, Φ is the encoding mode, and λ is a Lagrangian multiplier, which is a scale correction coefficient to match the unit between the amount of error and the amount of bits. use. In order to select the optimal encoding mode during the encoding process, J when the mode is applied, that is, the RD-cost value must be smaller than when the other modes are applied.In the equation for calculating the RD-cost value, both the bit rate and the error are considered. And calculate.

3 is a diagram illustrating an embodiment of a prediction unit of an image encoding apparatus.

The intra prediction unit 301 may perform intra prediction using original information and restoration information. For example, selection of at least one reference pixel line among a plurality of reference pixel lines, generation of a reference pixel, filtering of a reference pixel, generation of a prediction block using a reference pixel, and/or filtering of the generated prediction block may be performed. . The intra prediction mode inducing unit 302 may determine, for example, an optimal intra prediction mode by using the RD-cost value calculated for each prediction mode, and may generate or select a prediction block according to the prediction mode, and finally output it.

The inter prediction unit 303 may perform inter prediction using original information and restoration information. The inter prediction unit 303 may calculate an RD-cost value for each of a plurality of inter prediction modes including a Skip mode, a Merge mode, and an Advanced Motion Vector Prediction (AMVP) mode.

The merge candidate search unit 304 configures a set of candidate motion information for the skip mode and the merge mode. There may be various merge modes in the skip and merge modes. A weighted sum with one of the motion information of the candidate motion list derived from the regular merge mode, the intra prediction mode and the candidate motion list derived from the merge candidate searcher is performed using the motion information of the candidate motion list derived from the merge candidate search unit as the motion information of the current block. CIIP (Combined Inter-Intra Prediction) merge mode that determines prediction information through a triangle that divides the current block into two triangular blocks and applies different motion information from the candidate motion list derived from the merge candidate search unit to each triangular block. Merge mode, Merge with Motion Vector Difference (MMVD) merge mode, which induces motion information by transmitting additional MVD (Motion Vector Difference) information to motion information in the candidate motion list derived from the merge candidate search unit, each corner point of the current block The CPMV (Control Point Motion Vector) of is derived from the Affine candidate motion list, and the motion vector is derived by applying Equation 2 using the two CPMVs of the upper left and upper right points in units of subblocks of arbitrary size of the current block, or There is an Affine merge mode in which motion information is derived by deriving a motion vector by applying Equation 3 using three CPMVs at the upper right and lower left points.

In Equations 2 and 3, (MVx, MVy) is the motion vector of each sub-block, (MV0x, MV0y) is the CPMV of the upper left point, (MV1x, MV1y) is the CPMV of the upper right point, and (MV2x, MV2y) is the lower left It means the CPMV of the point, W and H are the horizontal and vertical lengths of the current block, and x and y are the center coordinates of each sub-block. When the current merge mode is an Affine merge mode, the merge candidate search unit 304 may configure an Affine merge candidate motion list, and in other merge modes, a Normal merge candidate motion list (also referred to as a'candidate motion list') may be configured. have. The optimal motion information of each merge mode is derived (305, 306) by calculating the RD-cost value for each candidate in the Regular and CIIP merge modes through the candidate motion list derived from the merge candidate search unit, and the optimal motion information is bidirectional motion. In the case of information, DMVR (Decoder-side Motion Vector Refinement) and BDOF (Bi-Directional Optical Flow) processes are performed (310, 311). However, in the case of the Triangle, MMVD, and Affine merge modes through the candidate motion list derived from the merge candidate search unit, the RD-cost value for each candidate is calculated to derive optimal motion information for each merge mode (305, 306), DMVR, BDOF process is not performed. Thereafter, the optimal merge mode is determined by comparing the optimal RD-cost values of each merge mode, and the motion compensation unit 315 generates a prediction block using motion information of the optimal merge mode. Here, the DMVD and BDOF processes may be performed only when the conditions described below are met.

DMVR execution conditions (if all the following conditions are met)

-When the motion information transmitted from the upper header is true

-When the merge mode is Regular merge mode or CIIP merge mode

-When motion information is predicted in both directions

-The distance between the current picture and the reference picture in the List0 direction while the reference picture in the Lsit0 direction (or in the past) is temporally earlier than the current picture, and the reference picture in the List 1 direction (or in the future) is temporally later than the current picture And the current picture and the reference picture in the List1 direction are the same

-When the bidirectional prediction blocks to generate the prediction block are weighted, the weight applied to the prediction block in the List0 direction (or in the past direction) and the prediction block in the List1 direction (or in the future direction) is the same.

-When the current block's width is 8 or more

-If the current block's height is 8 or more

-When the current block's width x height is 128 or more

BDOF execution conditions (Executed when all of the following conditions are met)

-When the motion information transmitted from the upper header is true

-When motion information is predicted in both directions

-When the reference picture in the Lsit0 direction (or in the past) is temporally earlier than the current picture, and the reference picture in the List 1 direction (or in the future) is temporally later than the current picture

-When the prediction mode is Regular merge mode, CIIP merge mode or Regular AMVP mode

-When the bidirectional prediction blocks to generate the prediction block are weighted, the weight applied to the prediction block in the List0 direction (or in the past direction) and the prediction block in the List1 direction (or in the future direction) is the same.

-If the current block's height is 8 or more

-When generating a prediction block of the Luma component (Chroma component X)

Detailed Description of the DMVR and BDOF Processes Since they are different from the description purpose of the present patent, detailed descriptions are omitted.

The AMVP candidate search unit 312 configures a candidate motion information set for the AMVP mode. The optimal motion information estimating unit 313 in the regular AMVP mode performs motion estimation by using the candidate motion information list of the current block, and determines, for example, optimal motion information having a minimum RD-cost value. The optimal motion information estimating unit 314 of the Affine AMVP mode performs motion estimation using the candidate motion information list for the optimal CPMV of each corner point of the current block, and for example, the optimal CPMV information with the minimum RD-cost value. Decide. The number of CPMVs may be two or three, and in case of two, it may be the CPMV at the upper left, upper right corner of the current block, and if three, the CPMV at the upper left, upper right, and lower left corners. Using the determined CPMV, optimal motion information is determined in units of sub-blocks within the current block. The motion compensation unit 208 generates a prediction block by performing motion compensation using optimal motion information.

The above-described inter prediction mode may largely include three modes (Skip mode, Merge mode, and AMVP mode). Each prediction mode may generate a prediction block of the current block using motion information (prediction direction information, reference picture information, and motion vector). In addition, there may be an additional prediction mode using motion information.

In the case of the Skip mode, prediction information (eg, optimal prediction information) of a current block may be determined by using motion information of an area that has already been reconstructed for each mode. A motion information candidate group may be configured within the reconstructed region, and a candidate having a minimum RD-cost value among the candidate groups may be used as prediction information of the current block. The method of constructing the motion information candidate group in the skip mode is the same as the method of constructing the motion information candidate group in the merge mode to be described later.

In the merge mode, it is the same as the Skip mode in that prediction information (eg, optimal prediction information) of a current block is determined by using motion information of an area that has already been reconstructed for each mode. However, the Skip mode is different in that the motion information that causes the prediction error to be 0 is searched in the motion information candidate group, and the Merge mode is in the motion information candidate group for the motion information whose prediction error is not 0. Similar to the skip mode, a motion information candidate group may be configured within a reconstructed region, and a prediction block may be generated by using, for example, a candidate having a minimum RD-cost value among the candidate groups as prediction information of the current block.

701 of FIG. 7 is a diagram for explaining a method of generating a motion information candidate group in Skip mode or Merge mode, Merge702 is a diagram for explaining the location of a spatial candidate block and a temporal candidate block, and 703 is a temporal candidate A diagram for describing a method of determining motion information of 704 is a diagram for describing a method of deriving average bidirectional candidate motion information.

The maximum number of motion information candidate groups may be equally determined by the video encoding apparatus and the video decoding apparatus, and the upper header of the video encoding apparatus (the upper header is, a video parameter set, a sequence parameter set, a picture parameter set, a tile header, a slice header, etc.) (Meaning parameters transmitted at a higher level of a block) may transmit the corresponding number information.

Only when the spatial candidate block and the temporal candidate block are encoded in the inter prediction mode in steps S705 and S706, motion information derived using the corresponding motion information may be included in the motion information candidate group.

In step S705, a spatial candidate may be selected from around the current block within the same picture. For example, 4 out of 5 spatial candidate blocks at a predetermined location may be selected as spatial candidates. The positions of the spatial candidate blocks may be, for example, positions A1 to A5 shown in FIG. 4. However, the number and location of spatial candidate blocks are not limited thereto, and can be changed to any block within the reconstructed area. Spatial candidates are considered in the order of A1, A2, A3, A4, A5, and motion information of an available spatial candidate block may be first determined as a spatial candidate. However, the order of consideration among the plurality of candidates is not limited to the above order. If motion information of a plurality of spatial candidates is overlapped, only motion information of a candidate having a high priority may be considered.

In step S706, a temporal candidate may be selected from a picture encoded/decoded before the current picture. For example, one of two temporal candidate blocks in a collocated picture may be selected as a temporal candidate. The locations of the temporal candidate blocks may be B1 and B2 shown in Figure 702. The positions of each candidate are determined based on a block at the same position as the current block position of the current picture in the picture. Here, the collocated picture can be set under the same conditions in the video encoding apparatus and the video decoding apparatus. For example, a reference picture corresponding to a predetermined reference picture index may be selected as a collocated picture. Alternatively, index information indicating a collocated picture may be signaled. The temporal candidate is considered in the order of B1 and B2 blocks, and motion information of an available candidate block may be first determined as a temporal candidate. However, the number, location, and order of consideration of temporal candidate blocks are not limited to the above-described embodiment.

As shown in 702 and 703, for example, motion information of a candidate block (B1 or B2) in a collocated picture may indicate a prediction block in a reference picture B. Reference pictures of each candidate block may be different from each other, but in this description, all of them are referred to as reference pictures B for convenience. The corresponding motion vector is scaled using a ratio of the distance (TD) between the collocated picture and the reference picture B, and the distance (TB) between the current picture and the reference picture A, and thus may be determined as a motion vector of a temporal candidate. For example, the scaling may be performed using Equation 4 below.

MV is a motion vector of motion information of a temporal candidate block, MVscale is a scaled motion vector, TB is a temporal distance between a collocated picture and a reference picture B, and TD is a temporal distance between a current picture and a reference picture A. In this case, the reference picture A and the reference picture B may be the same reference picture. The scaled motion vector may be determined as a temporal candidate motion vector. Also, the reference picture information of the temporal candidate motion information may be determined as a reference picture of the current picture, and motion information of the temporal candidate may be derived.

In step S707, a history-based candidate may be selected. History-based candidates (hereinafter referred to as'restored information-based candidates') include a restoration information-based motion candidate storage buffer (hereinafter referred to as'H buffer') for storing motion information coded in units of sequence, picture, or slice. exist. The corresponding buffer manages the coded motion information while updating the buffer in a first-in first-out (FIFO) method. The H buffer may be initialized in units of CTU, CTU row, slice, or picture, and when the current block is predicted and coded by motion information, corresponding motion information is updated in the H buffer. Motion information stored in the H buffer can be used as Skip, Merge, and AMVP candidates. When adding the H buffer candidate motion information to the candidate list, the H buffer may be added in the order of the most recently updated motion information, and vice versa. When motion information stored in the H buffer is added to the candidate list, the number of motion information that can be added may be limited. For example, the maximum number of candidate lists may be filled using motion information in the H buffer, but only up to (maximum number of candidate lists-1) may be filled. The number of candidate motion information stored in the H buffer may be determined under the same condition in the image encoding apparatus and the image decoding apparatus, or may be transmitted from the upper header to the image decoding apparatus.

In step S708, an average bidirectional candidate may be selected. When there is less than one motion information filled in the candidate list through steps S705 to S907, this step is omitted. When there are two or more motion information filled in the candidate list, the candidate list may be filled by generating average candidate motion information between each candidate. The motion vector of the average candidate motion information is derived for each prediction direction of the motion information (List0 or List1), and refers to a motion vector generated by averaging motion vectors in the same direction stored in the candidate list. When deriving the average candidate motion information, if reference picture index information of the motion information used for averaging is different, reference picture information of the motion information having a higher priority may be determined as the reference picture index information of the average candidate motion information. Alternatively, reference picture information of motion information having a low priority may be determined as reference picture index information of average candidate motion information. Figure 704 shows an example of the description of this step.

If the maximum number of motion information candidates is not satisfied even when the average bidirectional motion information candidate is used, step S709 is performed. In step S709, the motion vector of the motion information candidate is fixed as a zero motion vector, and the maximum number of motion information candidates may be filled by different reference pictures according to the prediction direction.

The transform unit 103 generates a transform block by transforming a residual block that is a difference between the original block and the prediction block. The transform block is the smallest unit used for transform and quantization processes. The transform unit transforms the residual signal into the frequency domain to generate a transform block having transform coefficients. Here, various transformation techniques such as Discrete Cosine Transform (DCT) based transformation, Discrete Sine Transform (DST), and Karhunen Loeve Transform (KLT) can be used to transform the residual signal into the frequency domain. , Using this, the residual signal is transformed into the frequency domain to generate a transform coefficient. In order to conveniently use the transformation method, matrix operations are performed using a basis vector. Depending on which prediction mode the prediction block is coded in, various transformation methods can be mixed and used during matrix operation. For example, during intra prediction, a discrete cosine transform may be used in the horizontal direction and a discrete sine transform may be used in the vertical direction according to the prediction mode.

The quantization unit 104 quantizes the transform block to generate a quantized transform block. That is, the quantization unit quantizes the transform coefficients of the transform block generated by the transform unit 103 to generate a quantized transform block having quantized transform coefficients. As a quantization method, Dead Zone Uniform Threshold Quantization (DZUTQ) or a quantization weighted matrix may be used, but various quantization methods such as quantization improved therefrom may be used.

Meanwhile, although it has been described above that the image encoding apparatus includes a transform unit and a quantization unit, the transform unit and the quantization unit may be selectively included. That is, the apparatus for encoding an image generates a transform block by transforming the residual block, and may not perform a quantization process, and can perform only a quantization process without converting the residual block into a frequency coefficient, and even a transform and quantization process. May not do all of them. In the image encoding apparatus, a block entering the input of the entropy encoding unit even if some of the transformation unit and the quantization unit are not performed or all of the processes are not performed is generally referred to as a'quantized transformation block'.

The entropy encoding unit 105 outputs a bitstream by encoding the quantized transform block. That is, the entropy encoding unit encodes the coefficients of the quantized transform block output from the quantization unit using various encoding techniques such as entropy encoding, and additional information necessary for decoding the block in an image decoding apparatus described later ( For example, a bitstream including information on a prediction mode (information on a prediction mode may include motion information or intra prediction mode information determined by a prediction unit), a quantization coefficient, etc.) is generated and output.

4 is a flowchart illustrating a method of encoding inter prediction information in an entropy encoder. In step S401, skip mode operation information is encoded. In step S402, it is determined whether or not the skip mode operation information is true, and if not, the merge information is encoded in step S407, and the flow chart ends. Detailed description of merge information will be described later. If the skip mode operation information is false, prediction mode information is encoded in step S403. In step S404, it is determined whether the prediction mode is inter prediction or intra prediction. If the prediction mode is the intra prediction mode, the intra prediction mode information is encoded in step S425, and this flowchart ends. If the prediction mode is the inter prediction mode, the merge mode operation information is encoded in step S405. Whether or not the merge mode operation information is true is determined in step S406, and if true, the merge information is encoded in step S407 and the flow chart ends. If the merge mode operation information is false, Affine AMVP mode operation information is encoded in step S408. Thereafter, in step S409, prediction direction information is encoded. In step S410, it is determined whether the Affine AMVP mode is true. If false, the current AMVP mode is Normal AMVP mode. If the current AMVP mode is the Affine AMVP mode, model information of the Affine AMVP mode is encoded in step S411. The model information indicates whether the current Affine model is a 4-parameter model or a 6-parameter model. If the current Affine model is a 4-parameter model, the number of MVD information is set to 2 in step S413, and if it is a 6-parameter model, the number of MVD information is set to 3 in step S414. Thereafter, in step S415, it is determined whether the prediction direction is the List1 direction. If the prediction direction is not the List1 direction in step S415, the index information of the reference picture in the List0 direction is encoded in step S416. In step S417, MVD (Motion Vector Difference) information in the direction of List 0 is encoded. At this time, if the current AMVP mode is the Affine AMVP mode, the number of MVDs determined through steps S413 and S414 is encoded. In step S418, motion vector predictor (MVP) information in the direction of List0 is encoded. In step S415, it is determined whether the prediction direction is the List1 direction or both directions, or when the S418 step is finished, whether the prediction direction is the List0 direction in step S419. If the prediction direction is not the List0 direction in step S419, the index information of the List1 direction reference picture is encoded in step S420. In step S421, MVD information in the List1 direction is encoded. At this time, if the current AMVP mode is the Affine AMVP mode, the number of MVDs determined through steps S413 and S414 is encoded. In step S422, MVP information in the direction of List1 is encoded. If the prediction direction was the List0 direction in step S419, this flowchart ends. In step S423, it is determined whether the prediction direction is bidirectional. If the prediction direction is not bidirectional, the flow chart is terminated. The weight information applied to the prediction block in the List0 direction and the prediction block in the List1 direction is (-1/8: 9/8), (1/8: 7/8), (3/8: 5/8), (4 Various weight information such as /8: 4/8), (5/8: 3/8), (7/8: 1/8), and (9/8: -1/8) can exist. Thereafter, this flowchart ends.

The inverse quantization unit 106 restores the inverse quantization transform block by inversely performing the quantization technique used for quantization on the quantized transform block.

The inverse transform unit 107 restores the residual block by inverse transforming the inverse quantization transform block using the same method as the method used in the transform, and performs inverse transform by inversely performing the transform method used in the transform unit.

Meanwhile, in the above description, the inverse quantization unit and the inverse transform unit may perform inverse quantization and inverse transformation by inversely using the quantization method and the transform method used in the quantization unit and the transform unit. In addition, when the transform unit and the quantization unit perform only quantization and not perform the transformation, only inverse quantization may be performed and no inverse transformation may be performed. If both the transform and quantization are not performed, the inverse quantization unit and the inverse transform unit may not perform both inverse transformation and inverse quantization, or may be omitted without being included in the image encoding apparatus.

The addition unit 108 restores the current block by adding the residual signal generated by the inverse transform unit and the prediction block generated through prediction.

After all blocks in the current picture are reconstructed, the filter unit 109 additionally filters the entire picture, such as deblocking filtering and sample adaptive offset (SAO). Deblocking filtering refers to the operation of reducing block distortion that occurs while encoding an image in block units, and SAO (Sample Adaptive Offset) is a method that minimizes the difference between the reconstructed image and the original image by subtracting or adding a specific value to the reconstructed pixel. Says work.

The memory 110 adds the residual signal generated by the inverse transform unit and the prediction block generated through prediction, and then stores the restored current block, which has undergone additional filtering by the in-loop filter unit, and stores the next block or the next picture. Can be used to predict

The subtraction unit 111 generates a residual block by subtracting the prediction block from the current original block.

2 is a block flow diagram schematically showing the configuration of an image decoding apparatus. The image decoding apparatus is an apparatus for decoding an image, and may largely include a block entropy decoding unit, an inverse quantization unit, an inverse transform unit, a prediction unit, an addition unit, an in-loop filter unit, and a memory unit. In an image encoding apparatus, a coded block is referred to as a decoding block in an image decoding apparatus.

The entropy decoding unit 201 interprets the bitstream transmitted from the image encoding apparatus and reads various pieces of information necessary for decoding a corresponding block and quantized transform coefficients.

6 is a flowchart illustrating a method of decoding inter prediction information in an entropy decoder. In step S601, the skip mode operation information is decoded. In step S602, it is determined whether or not the skip mode operation information is true, and if not, the merge information is decoded in step S607, and the flowchart ends. Detailed description of merge information will be described later. If the skip mode operation information is false, prediction mode information is decoded in step S603. In step S604, it is determined whether the prediction mode is inter prediction or intra prediction. If the prediction mode is the intra prediction mode, the intra prediction mode information is decoded in step S625, and this flowchart is terminated. When the prediction mode is an inter prediction mode, merge mode operation information is decoded in step S605. Whether or not the merge mode operation information is true is determined in step S606, and if true, the merge information is decoded in step S607, and the flowchart ends. If the merge mode operation information is false, Affine AMVP mode operation information is decoded in step S608. Thereafter, in step S609, the prediction direction information is decoded. In step S610, it is determined whether the Affine AMVP mode is true. If false, the current AMVP mode is Normal AMVP mode. If the current AMVP mode is the Affine AMVP mode, model information of the Affine AMVP mode is decoded in step S611. If the current Affine model is a 4-parameter model, the number of MVD information is set to 2 in step S613, and if the current Affine model is a 6-parameter model, the number of MVD information is set to 3 in step S614. Thereafter, in step S615, it is determined whether the prediction direction is the List1 direction. If the prediction direction is not the List1 direction in step S615, the reference picture index information in the List0 direction is decoded in step S616. In step S617, MVD (Motion Vector Difference) information in the direction of List 0 is decoded. At this time, if the current AMVP mode is the Affine AMVP mode, decoding is performed by the number of MVDs determined through steps S613 and S614. In step S618, motion vector predictor (MVP) information in the direction of List0 is decoded. In step S615, it is determined whether the prediction direction is the List1 direction or both directions, or if the step S618 is over, whether the prediction direction is the List0 direction in step S619. If the prediction direction is not the List0 direction in step S619, the index information of the List1 direction reference picture is decoded in step S620. In step S621, MVD information in the direction of List1 is decoded. At this time, if the current AMVP mode is the Affine AMVP mode, decoding is performed by the number of MVDs determined through steps S613 and S614. In step S622, MVP information in the direction of List1 is decoded. If the prediction direction was the List0 direction in step S619, this flowchart ends. In step S623, it is determined whether the prediction direction is in both directions, and if the prediction direction is not in both directions, the flowchart is terminated. Thereafter, this flowchart ends.

The inverse quantization unit 202 restores an inverse quantized block having inverse quantized coefficients by inversely performing a quantization technique used for quantization on the quantized coefficients decoded by the entropy decoder.

The inverse transform unit 203 restores a residual block having a difference signal by inverse transforming the inverse quantization transform block using the same method as the method used in the transform, and performs inverse transform by performing the transform method used in the transform unit inversely. .

The prediction unit 204 generates a prediction block by using the prediction mode information decoded by the entropy decoder, which uses the same method as the prediction method performed by the prediction unit of the image encoding apparatus.

The addition unit 205 restores the current block by adding the residual signal restored by the inverse transform unit and the prediction block generated through prediction.

After restoring all blocks in the current picture, the filter unit 206 performs additional filtering over the entire picture, including deblocking filtering, sample adaptive offset (SAO), and the like. For details, refer to the above-described video encoding apparatus. It is the same as described in the in-loop filter unit.

The memory 207 adds the residual signal generated by the inverse transform unit and the prediction block generated through prediction, and then stores the restored current block, which has undergone additional filtering by the in-loop filter unit, and stores the next block or the next picture. Can be used to predict

8 is a diagram illustrating the description of steps S407 and S607 as a number code. In step S407 of FIG. 4, coding information in italic type of FIG. 8 is encoded, and in step S607 of FIG. 6, coding information in italic type of FIG. 8 is decoded. The input values cbWidth and cbHeight to the Merge_data function mean the width and height of the current block. First, if sps_mmvd_enabled_flag information, which is MMVD merge mode operation information encoded/decoded in the upper header, is true, regular_merge_flag is encoded/decoded. This means motion information of the above-described regular merge mode, and when the corresponding information is true, it indicates which candidate motion information in the candidate motion information list is to be determined as motion information of the current block by encoding/decoding merge_idx information. If regular_merge_flag is false, first, when sps_mmvd_enabled_flag information is true, mmvd_merge_flag information is encoded/decoded. This means operation information of the aforementioned MMVD merge mode. When the operation information of the MMVd merge mode is true and the maximum number of merge candidate lists is more than one, mmvd_cand_flag is encoded/decoded to indicate which candidate motion vector in the merge candidate list to transmit MVD based on, mmvd_distance_idx, MVD information is encoded/decoded through mmvd_direction_idx information. When mmvd_merge_flag information is false and the maximum number of affine candidate motion information lists exceeds 0, the affine_merge_flag information is encoded/decoded to inform the operation information of the Affine merge mode described above. If Affine_merge_flag information is true and the maximum number of affine candidate motion information lists is more than one, affine_merge_idx information is encoded/decoded to indicate which candidate motion information is to be merged from the Affine candidate motion information list. When the affine_merge_flag information is false, sps_ciip_enabled_flag information, which is CIIP merge mode operation information encoded/decoded in the upper header, is true, and when it is not in Skip mode, ciip_flag, which is CIIP merge mode operation information, is encoded/decoded. When ciip_flag information is true and the maximum number of merge candidate lists is more than one, merge_idx information indicating which candidate motion information is to be weighted with a prediction block generated by intra prediction is encoded/decoded. The MergeTriangleFlag information is implied to be true in the video encoding apparatus and the video decoding apparatus when all the conditions described below are true.

-When sps_triangle_enabled_flag information, which is the operation information of Triangle merge mode encoded/decoded in the upper header, is true

-If the slide type currently being coded is B slice

-If Skip or Merge mode is true

-When the number of maximum candidate motion information in the candidate motion information list in Triangle merge mode is 2 or more

-When the length of the current block is 64 or more

-If ciip_flag information is false

When MergeTriangleFlag information is true, the triangle_split_dir information is encoded/decoded to indicate which direction the current block is to be divided into triangle blocks, and the same candidate motion information list for each of the two triangle blocks divided by encoding/decoding triangle_idx0 and triangle_idx1 information Informs which candidate motion information to select. However, motion information applied to each of the two triangular blocks must be different from each other.

(Example 1-1)

In this embodiment, after dividing the current block into two triangular blocks in the triangle merge mode, a method of inducing motion information for each divided triangular block will be described in detail.

9 shows two examples in which the current prediction block is divided into two triangular blocks (referred to as sub-prediction blocks in this figure). Segmentation example A is an example in which the prediction dividing line is virtually divided from the upper left corner of the prediction block to the lower right corner, and the segmentation example B is divided by imagining the prediction dividing line from the upper right corner of the prediction block to the lower left corner. This is an example. The sub prediction block A and the sub prediction block B may be set as shown in the figure according to the division type. The position of the sub prediction block may be changed.

10 is a diagram illustrating a method of determining motion information of sub-prediction blocks A and B from a candidate motion information list. The present embodiment will be described on the assumption that motion information of sub-prediction blocks A and B considers only unidirectional motion information. Of course, the bidirectional motion information may be determined as motion information of the sub prediction block. In addition, in this example, it is assumed that the motion information of the sub-prediction block A is the same as the coded/decoded triangle_idx0 information in FIG. 8, and the motion information of the sub-prediction block B is the same as the coded/decoded triangle_idx1 information in FIG. 8. In addition, in the selection method examples illustrated in FIG. 10, the left table denotes a method of selecting motion information of sub-prediction block A, and the right table denotes a method of selecting motion information of sub-prediction block B. The motion information of the sub-prediction block A preferentially considers the candidate motion information in the List0 direction when triangle_idx0 corresponding to the even index information is transmitted in the candidate motion information list, and List1 when triangle_idx0 corresponding to the odd index information is transmitted. Candidate motion information of a direction can be considered preferentially. If the candidate motion information in the direction considered preferentially does not exist in the candidate motion information list, the candidate motion information in the opposite direction is considered as the next priority. In the selection methods A-1, A-2, B-1, and B-2 of FIG. 10, candidate motion information having a dark contrast in the left table is candidate motion information that is preferentially considered. At this time, if the candidate motion information with the contrast is not valid, the candidate motion information in the opposite prediction direction is selected. A method of selecting motion information of the sub prediction block B for each selection method will be described below. At this time, it is assumed that the motion information of the sub prediction block A is underlined candidate motion information for each method.

The selection method A-1 assumes that the sub-prediction block A is determined as motion information corresponding to index information 0. When the index information corresponds to 0, the sub prediction block A selects candidate motion information in the List0 direction as motion information. In this case, the sub-prediction block B preferentially considers the candidate motion information in the List1 direction if triangle_idx1 information corresponds to even index information in the candidate motion information list, and prioritizes the candidate motion information in the List0 direction if it corresponds to odd index information. Can be considered. When motion information in the corresponding direction does not exist, candidate motion information in the opposite direction may be considered as a next priority.

In the selection method A-2, it is assumed that the sub prediction block A is determined as motion information corresponding to index information 4. When the index information corresponds to 4, the sub prediction block A selects the candidate motion information in the List1 direction as motion information. In this case, as in the original method, sub-prediction block B preferentially considers the candidate motion information in the List0 direction if triangle_idx1 information corresponds to even index information in the candidate motion information list, and if it corresponds to odd index information, the candidate motion in the List1 direction. Information can be prioritized. When motion information in the corresponding direction does not exist, candidate motion information in the opposite direction may be considered as a next priority.

The selection method B-1 assumes that the sub-prediction block A is determined as motion information corresponding to index information 3. When the index information corresponds to 3, the sub prediction block A selects candidate motion information in the List0 direction as motion information. In this case, the sub-prediction block B considers only the list1 direction candidate motion information of the transmitted triangle_idx1 information regardless of the even/odd number of triangle_idx1 information in the candidate motion information list.

The selection method B-2 assumes that the sub-prediction block A is determined as motion information corresponding to index information 4. When the index information corresponds to 4, the sub prediction block A selects the candidate motion information in the List1 direction as motion information. In this case, the sub prediction block B considers only the list1 direction candidate motion information of the triangle_idx1 information regardless of the even/odd number of triangle_idx1 information in the candidate motion information list.

(Example 1-2)

In this embodiment, a method of restrictive encoding/decoding of triangle_idx1 information will be described. The upper header may transmit the maximum number of candidate motion information in the candidate motion information list of the triangle merge mode. When the corresponding information is transmitted, when the triangle_idx1 information is encoded/decoded in FIG. 1100, limited encoding/decoding using the corresponding information must be performed.

Referring to FIG. 1100, when triangle_idx1 is transmitted, encoding/decoding is performed only when the number of maximum candidate motion information exceeds 2. This means that the motion information of the sub-prediction blocks A and B should be different. If the number of candidate motion information is only two, the remaining motion information in addition to the motion information of the sub-prediction block A selected by triangle_idx0 is naturally the motion of the sub-prediction block B. Since it is information, it is unnecessary to encode/decode triangle_idx1 information. In order to prevent this case, a restrictive encoding/decoding method as shown in FIG. 11 may be implemented.

(Example 2)

In this embodiment, in the Affine merge mode or Affine AMVP mode, the size of the sub-block is used to induce a motion vector in each sub-block unit in the current block using two CPMVs or three CPMVs at the edge of the current block and perform motion compensation. And a method of adaptively determining the shape will be described in detail with reference to sub-block generation examples A and B in FIG. 12.

Generation example A of FIG. 12 is a description of a case in which the current prediction block is a square. When the prediction block is a square, the shape of the sub-block for applying the Affine model may be determined as a square. The size of the sub-block may be determined in proportion to or inversely proportional to the size of the prediction block.

Generation example B of FIG. 12 is a description of a case where the current prediction block is a horizontally long rectangle. When the prediction block is a horizontally long rectangle, the shape of a subblock for applying the Affine model may be determined as a horizontally long rectangle. Alternatively, you can decide to use a vertically long rectangle. Likewise, the size of the sub-block may be determined in proportion to or inversely proportional to the size of the prediction block. Of course, a case in which the current prediction block is a vertically long rectangle can be described as an example opposite to that of the generation example B.

The size information of the sub-block may be transmitted in units of an upper header or block, and may be set in advance in the same manner in the image encoding apparatus and the image decoding apparatus.

Claims (1)

Video encoding and decoding method using complex motion information model.

Images