KR20200095982A - A method and an apparatus for processing a video signal using merge with motion vector difference - Google Patents

Abstract

The present invention relates to a processing method of a video signal and to a device thereof. More particularly, provided are a video signal processing method encoding or decoding a video signal and the device thereof. According to one embodiment of the present invention, an image decoding method includes a step of efficiently receiving a prediction mode selected by an encoder in an intra prediction method.

Description

Video signal processing method and apparatus using motion prediction based on merge-based motion vector difference {A METHOD AND AN APPARATUS FOR PROCESSING A VIDEO SIGNAL USING MERGE WITH MOTION VECTOR DIFFERENCE}

The present invention relates to a video signal processing method and apparatus, and more particularly, to a video signal processing method and apparatus for encoding or decoding a video signal.

Compression coding refers to a series of signal processing techniques for transmitting digitized information through a communication line or storing it in a form suitable for a storage medium. Objects of compression encoding include audio, video, and text, and in particular, a technique for performing compression encoding on an image is called video image compression. Compression coding of a video signal is performed by removing redundant information in consideration of spatial correlation, temporal correlation, and probability correlation. However, due to the recent development of various media and data transmission media, a more efficient video signal processing method and apparatus is required.

An object of the present invention is to increase the coding efficiency of a video signal.

In order to solve the above problems, an image decoding method according to an embodiment of the present invention includes the step of efficiently receiving a prediction mode selected by an encoder in an intra prediction method.

According to an embodiment of the present invention, coding efficiency of a video signal can be improved.

1 is a schematic block diagram of a video signal encoder apparatus according to an embodiment of the present invention.
2 is a schematic block diagram of a video signal decoder device according to an embodiment of the present invention.
3 is a diagram showing an embodiment of the present invention for dividing a coding unit.
FIG. 4 is a diagram illustrating an embodiment of a method for hierarchically representing the division structure of FIG. 3.
5 shows a further embodiment of the present invention for dividing a coding unit.
6 is a diagram illustrating a method of obtaining a reference pixel for intra prediction.
7 is a diagram illustrating an embodiment of prediction modes used for intra prediction.
8 is a diagram showing inter prediction according to an embodiment of the present invention.
9 is a diagram showing a motion vector signaling method according to an embodiment of the present invention.
10 is a diagram showing a motion vector difference syntax according to an embodiment of the present invention.
11 is a diagram showing adaptive motion vector resolution signaling according to an embodiment of the present invention.
12 is a diagram showing syntax related to inter prediction according to an embodiment of the present invention.
13 is a view showing merge with MVD (MMVD) according to an embodiment of the present invention.
14 is a diagram showing MMVD syntax according to an embodiment of the present invention.
15 is a diagram showing MMVD syntax according to an embodiment of the present invention.
16 is a diagram showing syntax related to MMVD according to an embodiment of the present invention.
17 is a diagram showing syntax related to MMVD according to an embodiment of the present invention.
18 is a diagram showing MMVD syntax according to an embodiment of the present invention.
19 is a diagram showing MMVD syntax according to an embodiment of the present invention.
20 is a diagram showing MMVD syntax according to an embodiment of the present invention.
21 is a diagram showing MMVD syntax according to an embodiment of the present invention.
22 is a diagram showing MMVD syntax according to an embodiment of the present invention.
23 is a diagram showing a coding unit syntax according to an embodiment of the present invention.
24 is a diagram showing a merge data syntax according to an embodiment of the present invention.
25 is a diagram showing merge data syntax according to an embodiment of the present invention.
26 is a diagram showing merge data syntax according to an embodiment of the present invention.
27 is a diagram showing a merge data syntax according to an embodiment of the present invention.
28 is a diagram showing a coding unit syntax according to an embodiment of the present invention.
29 is a diagram showing a coding unit syntax according to an embodiment of the present invention.
30 is a diagram showing merge mode signaling according to an embodiment of the present invention.
31 is a diagram showing merge data syntax according to an embodiment of the present invention.
32 is a diagram showing merge data syntax according to an embodiment of the present invention.
33 is a diagram illustrating a syntax structure described in FIGS. 30 to 32.
34 is a diagram showing a syntax structure according to an embodiment of the present invention.
35 is a diagram showing a syntax structure according to an embodiment of the present invention.
36 is a view showing MVD derivation of MMVD according to an embodiment of the present invention.
37 is a diagram showing MVD derivation of MMVD according to an embodiment of the present invention.
38 is a view showing MVD derivation of MMVD according to an embodiment of the present invention.
39 is a view showing motion information derivation according to an embodiment of the present invention.
40 is a diagram showing MVD derivation of MMVD according to an embodiment of the present invention.
41 is a view showing motion information derivation according to an embodiment of the present invention.
42 is a view showing MVD derivation of MMVD according to an embodiment of the present invention.

The terms used in the present specification have been selected as currently widely used general terms as possible while taking functions of the present invention into consideration, but this may vary according to the intention, custom, or the emergence of new technologies of the skilled person in the art. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in the description of the corresponding invention. Therefore, it is to be noted that terms used in the present specification should be interpreted based on the actual meaning of the term and the entire contents of the present specification, not a simple name of the term.

In the present invention, the following terms may be interpreted according to the following criteria, and even terms that are not described may be interpreted according to the following purpose. Coding may be interpreted as encoding or decoding in some cases, and information is a term that includes all values, parameters, coefficients, elements, etc., meaning in some cases As can be interpreted differently, the present invention is not limited thereto. 'Unit' is used to refer to a basic unit of image (picture) processing or a specific position of a picture, and in some cases, it may be used interchangeably with terms such as'block','partition' or'area'. . In addition, in the present specification, a unit may be used as a concept including all of a coding unit, a prediction unit, and a transform unit.

1 is a schematic block diagram of an apparatus for encoding a video signal according to an embodiment of the present invention. Referring to FIG. 1, the encoding apparatus 100 of the present invention is largely a transform unit 110, a quantization unit 115, an inverse quantization unit 120, an inverse transform unit 125, a filtering unit 130, and a prediction unit ( 150) and an entropy coding unit 160.

The conversion unit 110 obtains a transform coefficient value by converting a pixel value of an input video signal. For example, Discrete Cosine Transform (DCT) or Wavelet Transform may be used. Particularly, the discrete cosine transform is performed by dividing the input picture signal into blocks having a predetermined size. In transformation, coding efficiency may vary depending on the distribution and characteristics of values in the transformation region.

The quantization unit 115 quantizes a transform coefficient value output from the transform unit 110. The inverse quantization unit 120 inverse quantizes the transform coefficient value, and the inverse transform unit 125 restores the original pixel value by using the inverse quantization transform coefficient value.

The filtering unit 130 performs a filtering operation to improve the quality of the reconstructed picture. For example, a deblocking filter and an adaptive loop filter may be included. The filtered picture is output or stored in a decoded picture buffer 156 to be used as a reference picture.

In order to improve coding efficiency, the picture signal is not coded as it is, but a picture is predicted by using a region already coded through the prediction unit 150, and a residual value between the original picture and the predicted picture is added to the predicted picture. The method of obtaining is used. The intra prediction unit 152 performs intra prediction within the current picture, and the inter prediction unit 154 predicts the current picture using a reference picture stored in the decoded picture buffer 156. The intra prediction unit 152 performs intra prediction from reconstructed regions in the current picture and transmits the intra prediction information to the entropy coding unit 160. The inter prediction unit 154 may again include a motion estimation unit 154a and a motion compensation unit 154b. The motion estimation unit 154a obtains a motion vector value of the current region by referring to the restored specific region. The motion estimating unit 154a transfers position information (reference frame, motion vector, etc.) of the reference region to the entropy coding unit 160 to be included in the bitstream. The motion compensation unit 154b performs inter-screen motion compensation using the motion vector value transmitted from the motion estimation unit 154a.

The entropy coding unit 160 entropy-codes quantized transform coefficients, inter-picture encoding information, intra-picture encoding information, and reference region information input from the inter prediction unit 154 to generate a video signal bitstream. Here, the entropy coding unit 160 may use a variable length coding (VLC) scheme and arithmetic coding. The variable length coding (VLC) method converts input symbols into consecutive codewords, and the length of the codeword may be variable. For example, frequently occurring symbols are represented by a short codeword, and infrequently occurring symbols are represented by a long codeword. As a variable length coding scheme, a context-based adaptive variable length coding (CAVLC) scheme may be used. Arithmetic coding converts consecutive data symbols into a single prime number, and arithmetic coding can obtain an optimal decimal bit necessary to represent each symbol. Context-based Adaptive Binary Arithmetic Code (CABAC) may be used as arithmetic coding.

The generated bitstream is encapsulated in a basic unit of a Network Abstraction Layer (NAL) unit. The NAL unit includes a coded slice segment, and the slice segment consists of an integer number of coding tree units. In order to decode a bitstream in a video decoder, the bitstream must first be separated into NAL units, and then each separated NAL unit must be decoded.

2 is a schematic block diagram of a video signal decoding apparatus 200 according to an embodiment of the present invention. Referring to FIG. 2, the decoding apparatus 200 of the present invention largely includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 225, a filtering unit 230, and a prediction unit 250.

The entropy decoding unit 210 entropy-decodes the video signal bitstream, and extracts transform coefficients and motion information for each region. The inverse quantization unit 220 inverse quantizes the entropy-decoded transform coefficient, and the inverse transform unit 225 restores the original pixel value by using the inverse quantization transform coefficient.

Meanwhile, the filtering unit 230 improves image quality by performing filtering on a picture. This may include a deblocking filter for reducing block distortion and/or an adaptive loop filter for removing distortion of an entire picture. The filtered picture is output or stored in a decoded picture buffer 256 to be used as a reference picture for the next frame.

In addition, the prediction unit 250 of the present invention includes an intra prediction unit 252 and an inter prediction unit 254, and the encoding type decoded through the entropy decoding unit 210 described above, transform coefficients for each region, The predicted picture is reconstructed using motion information and the like.

In this regard, the intra prediction unit 252 performs intra prediction from decoded samples in the current picture. The inter prediction unit 254 generates a prediction picture using the reference picture and motion information stored in the decoded picture buffer 256. The inter prediction unit 254 may again include a motion estimation unit 254a and a motion compensation unit 254b. The motion estimation unit 254a obtains a motion vector indicating a positional relationship between the current block and a reference block of a reference picture used for coding, and transmits the obtained motion vector to the motion compensation unit 254b.

A reconstructed video frame is generated by adding the prediction value output from the intra prediction unit 252 or the inter prediction unit 254 and the pixel value output from the inverse transform unit 225.

Hereinafter, in the operation of the encoding device 100 and the decoding device 200, a method of dividing a coding unit, a prediction unit, and the like will be described with reference to FIGS. 3 to 5.

The coding unit is a process such as intra/inter prediction, transform, quantization, and/or entropy coding in the process of processing the video signal described above. It means the basic unit for processing pictures in. The size of a coding unit used to code one picture may not be constant. The coding unit may have a square shape, and one coding unit may be divided into several coding units.

3 shows an embodiment of the present invention for dividing a coding unit. For example, one coding unit having a size of 2N X 2N may be divided into four coding units having a size of N X N. The division of the coding unit may be performed recursively, and all coding units need not be divided into the same form. However, for convenience in coding and processing, there may be restrictions on the size of the maximum coding unit and/or the size of the minimum coding unit.

For one coding unit, information indicating whether the corresponding coding unit is divided may be stored. FIG. 4 illustrates an embodiment of a method of hierarchically representing the partition structure of the coding unit shown in FIG. 3 using flag values. Information indicating whether the coding unit is divided may be assigned a value of '1' when the corresponding unit is divided and '0' when the corresponding unit is not divided. As shown in FIG. 4, if the flag value indicating whether to divide is 1, the coding unit corresponding to the node is divided into 4 coding units again, and if it is 0, the coding unit is no longer divided and a processing process for the coding unit is performed. I can.

The structure of the coding unit described above can be represented by using a recursive tree structure. That is, with one picture or a maximum size coding unit as a root, a coding unit divided into other coding units has as many child nodes as the number of divided coding units. Thus, a coding unit that is no longer divided becomes a leaf node. Assuming that only square partitioning is possible for one coding unit, since one coding unit can be divided into up to four different coding units, the tree representing the coding unit may be in the form of a quad tree.

In the encoder, the optimal coding unit size is selected according to a characteristic (eg, resolution) of a video picture or in consideration of coding efficiency, and information about this or information for inducing it may be included in the bitstream. For example, the size of the maximum coding unit and the maximum depth of the tree can be defined. In the case of square division, since the height and width of the coding unit is half the height and width of the coding unit of the parent node, the minimum coding unit size can be obtained by using the above information. Or, conversely, the minimum coding unit size and the maximum depth of the tree can be defined and used in advance, and the size of the maximum coding unit can be derived and used by using them. In square division, since the size of the unit is changed in a multiple of 2, the size of the actual coding unit is represented by a log value of 2 to increase transmission efficiency.

The decoder may obtain information indicating whether the current coding unit is divided. If such information is acquired (transmitted) only under certain conditions, efficiency can be improved. For example, the condition in which the current coding unit can be divided is when the size of the current coding unit added at the current position is smaller than the size of the picture and the current unit size is larger than the preset minimum coding unit size, so that the current coding unit is divided only in this case. It is possible to obtain information indicating whether or not.

If the information indicates that the coding unit is divided, the size of the coding unit to be divided becomes half of the current coding unit, and is divided into four square coding units based on the current processing position. The above-described processing may be repeated for each of the divided coding units.

5 shows a further embodiment of the present invention for dividing the coding unit. According to a further embodiment of the present invention, the coding unit in the form of a quad tree may be further divided into a binary tree structure of horizontal division or vertical division. That is, quadtree division of a square is first applied to the root coding unit, and rectangular binary tree division may be additionally applied at a leaf node of the quad tree. According to an embodiment, the binary tree division may be a symmetric horizontal division or a symmetric vertical division, but the present invention is not limited thereto.

In each split node of the binary tree, a flag indicating a split type (ie, horizontal split or vertical split) may be additionally signaled. According to an embodiment, when the value of the flag is “0”, horizontal division may be indicated, and when the value of the flag is “1”, vertical division may be indicated.

However, in the embodiment of the present invention, a method of dividing a coding unit is not limited to the above-described methods, and asymmetric horizontal/vertical division, a triple tree divided into three rectangular coding units, and the like may be applied.

Picture prediction (motion compensation) for coding is performed for coding units that are no longer divided (ie, leaf nodes of the coding unit tree). The basic unit that performs such prediction is hereinafter referred to as a prediction unit or a prediction block.

Hereinafter, the term unit used in the present specification may be used as a term to replace the prediction unit, which is a basic unit for performing prediction. However, the present invention is not limited thereto, and more broadly, it may be understood as a concept including the coding unit.

The current picture including the current unit or a decoded portion of other pictures may be used to restore the current unit on which decoding is performed. An intra picture or an I picture (slice) using only the current picture for restoration, i.e., a picture (slice) performing only intra prediction, and a picture (slice) capable of performing both intra prediction and inter prediction are inter-picture (slice). ). A picture (slice) using at most one motion vector and a reference index to predict each unit among inter-pictures (slices) is called a predictive picture or a P picture (slice), and up to two motion vectors and a reference index A picture (slice) using is referred to as a bi-predictive picture or a B picture (slice).

The intra prediction unit performs intra prediction for predicting pixel values of the target unit from reconstructed regions in the current picture. For example, the pixel value of the current unit may be predicted from reconstructed pixels of units located on the left and/or upper side, centering on the current unit. In this case, units located to the left of the current unit may include a left unit, an upper left unit, and a lower left unit adjacent to the current unit. Also, units located at the top of the current unit may include an upper unit, an upper left unit, and an upper right unit adjacent to the current unit.

Meanwhile, the inter prediction unit performs inter prediction to predict a pixel value of a target unit by using information of reconstructed pictures other than the current picture. In this case, a picture used for prediction is referred to as a reference picture. In the inter prediction process, which reference region is used to predict the current unit may be indicated using an index indicating a reference picture including the reference region and motion vector information.

Inter prediction may include L0 prediction, L1 prediction, and Bi-prediction. L0 prediction means prediction using one reference picture included in the L0 picture list, and L1 prediction means prediction using one reference picture included in the L1 filter list. For this, one set of motion information (eg, a motion vector and a reference picture index) may be required. In the bi-prediction scheme, up to two reference regions may be used, and the two reference regions may exist in the same reference picture or may exist in different pictures. That is, in the bi-prediction method, up to two sets of motion information (for example, a motion vector and a reference picture index) may be used, and two motion vectors may correspond to the same reference picture index or to different reference picture indexes. May correspond. In this case, reference pictures may be displayed (or output) temporally before or after the current picture.

The reference unit of the current unit may be obtained using the motion vector and the reference picture index. The reference unit exists in a reference picture having the reference picture index. Further, a pixel value or an interpolated value of a unit specified by the motion vector may be used as a predictor of the current unit. For motion prediction with pixel accuracy in sub-pel units, for example, an 8-tap interpolation filter may be used for a luminance signal and a 4-tap interpolation filter may be used for a color difference signal. However, the interpolation filter for motion prediction in units of subpels is not limited thereto. In this way, motion compensation for predicting the texture of the current unit from the previously decoded picture is performed using the motion information.

Hereinafter, an intra prediction method according to an embodiment of the present invention will be described in more detail with reference to FIGS. 6 and 7. As described above, the intra prediction unit predicts the pixel value of the current unit by using the adjacent pixels located on the left and/or top of the current unit as reference pixels.

As illustrated in FIG. 6, when the size of the current unit is NXN, reference pixels may be set using up to 4N+1 adjacent pixels located on the left and/or top of the current unit. When at least some adjacent pixels to be used as reference pixels have not yet been reconstructed, the intra predictor may obtain a reference pixel by performing a reference sample padding process according to a preset rule. Also, the intra prediction unit may perform a reference sample filtering process to reduce an error in intra prediction. That is, reference pixels may be obtained by filtering adjacent pixels and/or pixels obtained by the reference sample padding process. The intra prediction unit predicts the pixels of the current unit by using the obtained reference pixels.

7 illustrates an embodiment of prediction modes used for intra prediction. For intra prediction, intra prediction mode information indicating an intra prediction direction may be signaled. When the current unit is an intra prediction unit, the video signal decoding apparatus extracts intra prediction mode information of the current unit from the bitstream. The intra prediction unit of the video signal decoding apparatus performs intra prediction on the current unit based on the extracted intra prediction mode information.

According to an embodiment of the present invention, the intra prediction mode may include a total of 67 modes. Each intra prediction mode may be indicated through a preset index (ie, an intra mode index). For example, as shown in FIG. 7, intra mode index 0 indicates planar mode, intra mode index 1 indicates DC mode, and intra mode indexes 2 to 66 indicate different direction modes (i.e. , Angle modes) can be indicated respectively. The intra prediction unit determines reference pixels and/or interpolated reference pixels to be used for intra prediction of the current unit based on the intra prediction mode information of the current unit. When the intra mode index indicates a specific direction mode, a reference pixel or an interpolated reference pixel corresponding to the specific direction from the current pixel of the current unit is used for prediction of the current pixel. Accordingly, different sets of reference pixels and/or interpolated reference pixels may be used for intra prediction according to the intra prediction mode.

After intra prediction of the current unit is performed using reference pixels and intra prediction mode information, the video signal decoding apparatus adds the residual signal of the current unit obtained from the inverse transform unit with the intra prediction value of the current unit. Restore pixel values.

8 is a diagram showing inter prediction according to an embodiment of the present invention.

As described above, when encoding or decoding the current picture or block, prediction can be made from another picture or block. In other words, it is possible to encode and decode based on similarity with other pictures or blocks. A portion similar to another picture or block may be encoded and decoded by signaling omitted in the current picture or block, which will be described further below. It is possible to make predictions in blocks.

Referring to FIG. 8, there is a reference picture (reference picture) on the left and a current picture (current picture) on the right, and a current picture or a part of the current picture can be predicted using similarity with the reference picture or a part of the reference picture. have. Assuming that a rectangle indicated by a solid line in the current picture of FIG. 8 is a block currently encoding and decoding, the current block may be predicted from a rectangle indicated by a dotted line of the reference picture. At this time, information indicating a block (reference block) to be referred to by the current block may exist, which may be signaled directly or may be generated by a certain promise to reduce signaling overhead. Information indicating a block to be referred to by the current block may include a motion vector. This may be a vector indicating a relative position in the picture between the current block and the reference block. Referring to FIG. 8, there is a portion indicated by a dotted line of a reference picture, and a vector indicating how a current block can move to a block to be referred to of a reference picture may be a motion vector. That is, the block that appears when the current block is moved according to the motion vector may be a portion indicated by a dotted line in the current picture of FIG. 8, and the portion indicated by the dotted line may have a position in the picture equal to the reference block position of the reference picture.

In addition, information indicating a block to be referred to by the current block may include information indicating a reference picture. Information indicating the reference picture may include a reference picture list and a reference picture index. The reference picture list is a list representing reference pictures, and it is possible to use a reference block in a reference picture included in the reference picture list. That is, it is possible to predict the current block from the reference picture included in the reference picture list. Also, the reference picture index may be an index for indicating a reference picture to be used.

9 is a diagram illustrating a motion vector signaling method according to an embodiment of the present invention.

According to an embodiment of the present invention, a motion vector (MV) can be generated based on a motion vector predictor (MVP). For example, the motion vector predictor can be a motion vector as shown below.

MV = MVP

As another example, the motion vector can be based on the motion vector difference (MVD) as follows. Motion vector difference (MVD) can be added to the motion vector predictor to represent the correct motion vector.

MV = MVP + MVD

In addition, in video coding, motion vector information determined by an encoder is transmitted to a decoder, and the decoder can generate a motion vector from the received motion vector information and determine a prediction block. For example, the motion vector information may include information on a motion vector predictor and a motion vector difference. In this case, a component of the motion vector information may vary depending on the mode. For example, in the merge mode, the motion vector information may include information on a motion vector predictor and may not include a motion vector difference. As another example, in the advanced motion vector prediction (AMVP) mode, the motion vector information may include information on a motion vector predictor and may include a motion vector difference.

In order to determine, transmit, and receive information about the motion vector predictor, the encoder and decoder can generate MVP candidates in the same way. For example, the encoder and the decoder can generate the same MVP candidate in the same order. In addition, the encoder transmits an index representing the determined MVP among the generated MVP candidates to the decoder, and the decoder can find out the MVP and MV determined based on this index.

The MVP candidate and MVP candidate generation method may include a spatial candidate and a temporal candidate. Spatial candidate may be a motion vector for a block located at a certain position from the current block. For example, it may be a motion vector corresponding to a block or position adjacent to or not adjacent to the current block. The temporal candidate may be a motion vector corresponding to a block in a picture different from the current picture. Alternatively, the MVP candidate may include an affine motion vector, ATMVP, STMVP, a combination of motion vectors described above, an average vector of motion vectors described above, a zero motion vector, and the like.

In addition, information indicating the reference picture described above may also be transmitted from the encoder to the decoder. Also, motion vector scaling can be performed when the reference picture corresponding to the MVP candidate does not correspond to information indicating the reference picture. Motion vector scaling may be a calculation based on a picture order count (POC) of a current picture, a POC of a reference picture of a current block, a POC of a reference picture of an MVP candidate, and an MVP candidate.

10 is a diagram showing a motion vector difference syntax according to an embodiment of the present invention.

The motion vector difference can be coded by dividing the sign and absolute value of the motion vector difference. That is, the sign and absolute value of the motion vector difference may have different syntax. In addition, the absolute value of the motion vector difference may be directly coded, but may be coded by including a flag indicating whether the absolute value is greater than N as shown in FIG. 10. If the absolute value is greater than N, the value of (absolute value-N) can be signaled together. In the example of FIG. 10, abs_mvd_greater0_flag may be transmitted, and this flag may be a flag indicating whether the absolute value is greater than 0. If abs_mvd_greater0_flag indicates that the absolute value is not greater than 0, it can be determined that the absolute value is 0. Also, if abs_mvd_greater0_flag is indicated when the absolute value is greater than 0, additional syntax may exist. For example, abs_mvd_greater1_flag may exist, and this flag may be a flag indicating whether the absolute value is greater than 1. If abs_mvd_greater1_flag is indicated that the absolute value is not greater than 1, it can be determined that the absolute value is 1. If abs_mvd_greater1_flag is indicated when the absolute value is greater than 1, additional syntax may exist. For example, abs_mvd_minus2 may exist, which may be a value of (absolute value-2). It is determined that the absolute value is greater than 1 (more than 2) through abs_mvd_greater0_flag and abs_mvd_greater1_flag described above, indicating (absolute value-2). When abs_mvd_minus2 is binarized to a variable length, this is for signaling with fewer bits. For example, there are variable length binarization methods such as Exp-Golomb, truncated unary, and truncated Rice. Also, mvd_sign_flag may be a flag indicating a sign of a motion vector difference.

In this embodiment, the coding method was described through motion vector difference, but information other than motion vector difference is also divided about sign and absolute value, and the absolute value is a flag indicating whether the absolute value is greater than a certain value and the value of the absolute value. It is possible to code by subtracting.

In addition, in FIG. 10, [0] and [1] may represent component indexes. For example, it can represent x-component and y-component.

11 is a diagram showing adaptive motion vector resolution signaling according to an embodiment of the present invention.

According to an embodiment of the present invention, resolution representing a motion vector or a motion vector difference may vary. In other words, the resolution at which the motion vector or motion vector difference is coded may vary. For example, resolution can be expressed based on pixel (pel). For example, a motion vector or a motion vector difference may be signaled in units such as 1/4 (quarter), 1/2 (half), 1 (integer), 2, and 4 pixels. For example, if you want to represent 16, if you use a 1/4 unit, you code 64 (1/4 * 64 = 16), and if you use 1 unit, you code 16 (1 * 16 = 16), and if you use 4 units, it is 4 Can code (4 *. 4 = 16). That is, the value can be determined as follows.

valueDetermined = resolution*valuePerResolution

Here, valueDetermined may be a value to be transferred, and in this embodiment, a motion vector or a motion vector difference. In addition, valuePerResolution may be a value expressed in units of valueDetermined in [/resolution].

At this time, if the value signaled by the motion vector or the motion vector difference is not divided by the resolution, an incorrect value other than the motion vector or the motion vector difference with the best prediction performance may be transmitted through rounding or the like. If high resolution is used, inaccuracy may be reduced, but many bits can be used because the coded value is large, and if low resolution is used, inaccuracy may be increased, but because the coded value is small, fewer bits can be used.

Also, it is possible to set the resolution differently in units of block, CU, slice, etc. Therefore, resolution can be adaptively applied to fit the unit.

The resolution can be signaled from the encoder to the decoder. At this time, the signaling for resolution may be the signaling binarized with the variable length described above. In this case, when signaling with an index corresponding to the smallest value (the value in front of it), the signaling overhead is reduced.

In one embodiment, the signaling index may be matched in order from high resolution (detail signaling) to low resolution.

11 shows signaling for three resolutions. In this case, the three signaling can be 0, 10, 11, and each of the three signaling can correspond to resolution 1, resolution 2, and resolution 3. Signaling overhead is small when signaling resolution 1 because 1 bit is required to signal resolution 1 and 2 bits are required to signal the remaining resolution. In the example of FIG. 11, resolution 1, resolution 2, and resolution 3 are 1/4, 1, and 4 pel, respectively.

In the following inventions, motion vector resolution may mean resolution of motion vector difference.

12 is a diagram showing syntax related to inter prediction according to an embodiment of the present invention.

According to an embodiment of the present invention, the inter prediction method may include a skip mode, a merge mode, and an inter mode. According to an embodiment, a residual signal may not be transmitted in the skip mode. Also, in skip mode, the same MV determination method as merge mode can be used. Whether to use the skip mode may be determined according to the skip flag. Referring to FIG. 12, whether to use the skip mode may be determined according to a cu_skip_flag value.

According to an embodiment, the motion vector difference may not be used in the merge mode. A motion vector can be determined based on the motion candidate index. Whether to use the merge mode may be determined according to the merge flag. Referring to FIG. 12, whether to use a merge mode may be determined according to a merge_flag value. Also, it is possible to use merge mode when skip mode is not used.

It is possible to selectively use one or more candidate list types in skip mode or merge mode. For example, it is possible to use a merge candidate or a subblock merge candidate. In addition, the merge candidate may include a spatial neighboring candidate and a temporal candidate. Also, the merge candidate may include a candidate that uses a motion vector for the entire current block (CU). That is, the motion vector of each subblock belonging to the current block may contain the same candidate. In addition, the subblock merge candidate may include subblock-based temporal MV, affine merge candidate, etc. In addition, the subblock merge candidate may include a candidate capable of using a different motion vector for each subblock of the current block (CU). Affine merge candidate may be a method created by determining a control point motion vector of affine motion prediction without using a motion vector difference. In addition, the subblock merge candidate may include methods of determining a motion vector in units of subblocks in the current block. For example, the subblock merge candidate may include planar MV, regression based MV, STMVP, etc. in addition to the subblock-based temporal MV and affine merge candidate mentioned above.

According to an embodiment, a motion vector difference may be used in inter mode. A motion vector predictor may be determined based on a motion candidate index, and a motion vector may be determined based on the motion vector predictor and a motion vector difference. Whether to use the inter mode may be determined according to whether other modes are used. In another embodiment, whether to use the inter mode may be determined by a flag. 12 shows an example of using the inter mode when other modes such as skip mode and merge mode are not used.

Inter mode may include AMVP mode, affine inter mode, and the like. Inter mode may be a mode for determining a motion vector based on a motion vector predictor and a motion vector difference. Affine inter mode may be a method of using a motion vector difference when determining a control point motion vector of affine motion prediction.

Referring to FIG. 12, it is possible to determine whether to use a subblock merge candidate or a merge candidate after determining as a skip mode or a merge mode. For example, when a specific condition is satisfied, a merge_subblock_flag indicating whether to use a subblock merge candidate can be parsed. Also, the specific condition may be a condition related to block size. For example, it can be conditions related to width, height, area, etc. You can also use them in combination. Referring to FIG. 12, for example, it may be a condition when the width and height of the current block (CU) are greater than or equal to a specific value. When parsing the merge_subblock_flag, the value can be inferred to 0. If merge_subblock_flag is 1, the subblock merge candidate may be used, and if it is 0, the merge candidate may be used. When using the subblock merge candidate, the candidate index merge_subblock_idx can be parsed, and when the merge candidate is used, the candidate index merge_idx can be parsed. In this case, if the maximum number of candidate list is 1, parsing may not be performed. If merge_subblock_idx or merge_idx is not parsed, it can be inferred to 0.

12 shows a coding_unit function, and content related to intra prediction may be omitted, and FIG. 12 may indicate a case where inter prediction is determined.

13 is a diagram illustrating merge with MVD (MMVD) according to an embodiment of the present invention.

According to an embodiment of the present invention, it is possible to determine a motion vector (MV) based on a motion vector predictor (MVP) and a motion vector difference (MVD). At this time, the MVP may be called a base MV (baseMV). Referring to FIG. 13, it is possible to determine the MV by adding the base MV and the MVD. The MVD may be a value that refines MVP or may be refineMV.

According to an embodiment of the present invention, in MMVD, MV can be determined by base MV, distance, and direction.

In addition, according to an embodiment of the present invention, the base MV may be determined from another candidate list. For example, the base MV can be determined from a merge candidate list. It is also possible to determine the base MV from some of the other candidate lists. In addition, a part of the other candidate list may be a part of the front part of the other candidate list (the side with a smaller index). More specifically, it is possible to determine the base MV using the first two candidates among merge candidates. Also, for this, it is possible to signal the candidate index from the encoder to the decoder. Referring to FIG. 13, Base candidate IDX, which is an index signaling base MV, is shown. According to this index, it is possible to determine which of the candidates in the candidate list to use as the base MV.

According to an embodiment of the present invention, an MVD different from the MVD described in FIGS. 9 to 12 may exist. For example, the MVD in MMVD may be different from the MVD described in FIGS. 9 to 12. The different MVD may mean a simplified MVD, an MVD having a small resolution, an MVD having a small number of possible cases (a small number of MVDs), an MVD having a different signaling method, and the like.

For example, the MVD used in the existing AMVP, affine inter mode, etc. described in FIGS. 9 to 12 is based on all regions, e.g., pictures, in the x and y axes for a certain signaling unit (e.g. Although one area (eg, a picture area or an area including a picture and a surrounding area) can be expressed at uniform intervals, the MVD in MMVD may have a limited number of units that can be expressed for a certain signaling unit. In addition, the visible portion may not be uniform in intervals. In addition, MVD in MMVD may indicate only a specific direction for a certain signaling unit.

Also, MVD in MMVD can be based on distance and direction. Referring to FIG. 13, distance and direction according to Distance IDX, which is distance signaling in MMVD, and Direction IDX, which is direction signaling, may be preset. Distance may represent the size of the MVD (may be an absolute value) in units of x-pel, and direction may represent the direction of the MVD. Also, in distance signaling, it is possible to signal a small distance with a small index. That is, in the case of non-fixed length signaling, it is possible to signal a small distance with fewer bits.

It is also possible for the MVD to use a signaled MV or an MV based on the signaled MV. For example, the MV based on the signaled MV may be a sign of the signaled MV inverted. For example, MVD signaling is performed based on a value corresponding to a reference list, and the value corresponding to the reference list and other reference list is written as it is, or the value corresponding to the reference list (that is, signaled MVD) It can be written by changing the sign. Whether to write as it is or by changing the code may be determined by the POC relationship between the current picture and the reference picture of a reference list, and the POC relationship between the current picture and the reference picture of the reference list and another reference list. More specifically, even when both the reference lists L0 and L1 are used, only one MVD can be signaled. For example, an MVD corresponding to L0 may be signaled. In addition, the MVD corresponding to L1 may be determined based on the MVD corresponding to L0. For example, the MVD corresponding to L1 may be the same as the MVD corresponding to L0 or a value obtained by changing the sign of the MVD corresponding to L0. Also, this may be determined by the POC relationship between the current picture and the L0 reference picture and the POC relationship between the current picture and the L1 reference pictrue. For example, depending on whether the value of DiffPicOrderCnt( RefPicList0[ refIdxLN0 ], currPic) * DiffPicOrderCnt( currPic, RefPicList1[ refIdxLN1]) is greater than or less than 0, it is possible to determine whether to write the MVD corresponding to L0 in L1 as it is or transform it. In addition, the fact that DiffPicOrderCnt( RefPicList0[ refIdxLN0 ], currPic) * DiffPicOrderCnt( currPic, RefPicList1[ refIdxLN1]) is greater than 0 means that both the L0 reference picture and the L1 reference picture are temporally ahead of the current picture, or both are temporally behind the current picture. I can. Therefore, in this case, it is possible that the codes of L0 MVD and L1 MVD are the same. In addition, when DiffPicOrderCnt( RefPicList0[ refIdxLN0 ], currPic) * DiffPicOrderCnt( currPic, RefPicList1[ refIdxLN1]) is less than 0, one of the L0 reference picture and the L1 reference picture is temporally ahead of the current picture, and the other is temporally ahead of the current picture. May be the one behind. Therefore, in this case, it is possible that the codes of L0 MVD and L1 MVD are different. Also, those that are in front in time may have a small POC and those that are behind in time may have a large POC. POC may mean picture order count.

Also, the MV scaling process can be added in the preceding description. That is, a process of MV scaling a signaled MV or an MV transformed from the signaled MV (for example, an MV whose sign is reversed) may be added.

14 is a diagram showing MMVD syntax according to an embodiment of the present invention.

According to an embodiment of the present invention, signaling indicating whether to use the MMVD described in FIG. 13 may exist. Referring to FIG. 14, the signaling indicating whether the MMVD is used may be mmvd_flag.

In addition, it is possible to apply MMVD to skip mode or merge mode.

According to an embodiment of the present invention, when using MMVD, it is possible to parse MMVD related syntax. Also, referring to FIG. 14, MMVD-related syntax can be parsed in mmvd_idx_coding.

In an embodiment, when MMVD is not used, it is possible to pars the merge index. In the existing skip mode and merge mode, a merge index may be required. If MMVD can be applied to skip mode or merge mode, when MMVD is not used, the merge index can be parsed. Referring to FIG. 14, the merge index may be merge_idx. Also, in the merge mode other than the skip mode, after parsing the merge_flag, the mmvd_flag can be parsed when the merge_flag is 1. merge_flag may indicate that a merge mode, a sunblock merge mode, etc. are used. In addition, merge_flag may indicate that AMVP, inter mode, affine inter mode, etc. are not used. In this embodiment, merge_idx may be an index different from the base candidate IDX described in FIG. 13.

According to an embodiment of the present invention, signaling indicating the maximum number of merge candidates may exist. In addition, the signaling indicating the maximum number of merge candidates may be signaled in a unit larger than the CU and prediction unit. For example, there may be signaling indicating the maximum number of merge candidates in a slice or tile unit. In addition, it is possible to pars the merge index when the condition based on signaling indicating the maximum number of merge candidates is satisfied. In an embodiment, when signaling indicating the maximum number of merge candidates indicates that the maximum number of merge candidates is 1, the merge index may be inferred to 0. That is, when the signaling indicating the maximum number of merge candidates indicates that the maximum number of merge candidates is 1, the candidate can be automatically determined without parsing the index. Referring to FIG. 14, there may be MaxNumMergeCand, which is a value based on signaling indicating the maximum number of merge candidates, and MaxNumMergeCand may mean the maximum number of merge candidates. MaxNumMergeCand can be a number of 1 or more. In addition, it is possible to determine the maximum number of merge candidates for a signaling unit indicating the maximum number of merge candidates based on signaling indicating the maximum number of merge candidates supported by the standard and the maximum number of merge candidates. For example, it is possible to determine the maximum number of merge candidates for a signaling unit representing the maximum number of merge candidates by subtracting a signaling value representing the maximum number of merge candidates from the maximum number of merge candidates supported by the standard. In addition, although the maximum number of merge candidates has been described, signaling of the maximum number and maximum number of subblock merge candidates of the same concept may exist.

In addition, merge_idx can be parsed when the skip mode or merge mode is used, and the merge index can be parsed when the condition of using the skip mode or merge mode is satisfied. For example, it is possible to use skip mode or merge mode when subblock merge mode is not used.

In the embodiment of FIG. 14, merge_idx after mmvd_flag is parsed, which may be to prioritize MMVD over the existing skip mode or merge mode.

In addition, there may be signaling indicating whether to use the subblock merge mode, and it may be subblock_merge_flag.

15 is a diagram showing MMVD syntax according to an embodiment of the present invention.

The parts described in FIG. 14 may also be applied to FIG. In FIG. 15, content about subblock merge is added compared to FIG. 14. In addition, in FIGS. 15 and hereinafter, parts related to intra prediction and parts related to inter mode, AMVP mode, affine inter mode, and the like among inter prediction may be omitted.

According to an embodiment of the present invention, subblock_merge_flag may be parsed before mmvd_flag and merge_idx. This may prioritize subblock merge mode over skip mode, merge mode, MMVD, etc. When the subblock merge mode is not used, it is possible to pars the mmvd_flag and merge_idx.

16 is a diagram showing syntax related to MMVD according to an embodiment of the present invention.

The function shown in FIG. 16 may be a part for parsing the MMVD related syntax shown in FIGS. 14 to 15 and a diagram showing the MMVD syntax below. As described in FIG. 13, MMVD may be determined based on a base MV, distance, and direction, and there may be indexes signaling this. Referring to FIG. 16, indexes of base MV, distance, and direction may be base_mv_idx, distance_idx, and direction_idx.

In the case of signaling that MMVD is used as described with reference to FIGS. 14 to 15, the MMVD related syntax can always be signaled as shown in FIGS. 14, 15 and 16.

17 is a diagram illustrating syntax related to MMVD according to an embodiment of the present invention.

The function shown in FIG. 17 may be a part for parsing the syntax related to MMVD shown in FIGS. 14 to 15 and a diagram showing the MMVD syntax below.

According to an embodiment of the present invention, it is possible to pars some of MMVD related syntax when a specific condition is satisfied. For example, after it is determined to use MMVD, it is possible to pars some of the MMVD related syntax when certain conditions are satisfied.

According to an embodiment, some of the MMVD-related syntax may be base MV-related signaling.

It may be determined to use MMVD according to the mmvd_flag value. When mmvd_flag is parsed or mmvd_flag is inferred to indicate a specific value (eg, 1), it may be determined to use MMVD.

According to an embodiment, the specific condition may be regarding the maximum number of possible base MV candidates. For example, when the maximum number of possible base MV candidates is 2 or more, the syntax related to base MV may be parsed, and when the maximum number of possible base MV candidates is 1, the syntax related to base MV may not be parsed. It is possible to infer when there is no base MV related syntax. When infering, you can infer to 0. If the base MV candidates are merge candidates, it is possible to determine whether to parse the base MV related syntax according to a condition based on the maximum number of merge candidates possible.

The maximum number possible as Base MV candidates may be a value indicated by signaling indicating the maximum number of merge candidates described in FIG. 14, and the value may be MaxNumMergeCand.

Referring to FIG. 17, base_mv_idx is parsed only when MaxNumMergeCand indicating the maximum number of merge candiates is greater than 1. If base_mv_idx does not exist, for example, if it does not exist because it is not parsed, the value can be inferred to 0. This is because when the maximum number of candiates possible is 1, it is possible to determine the index without signaling.

18 is a diagram showing MMVD syntax according to an embodiment of the present invention.

In the case of the syntax structure described in FIG. 15, mmvd_flag must always be parsed even when the skip mode or merge mode is used without using MMVD. In addition, in the case of using and not using MMVD, the base MV index or merge index may need to be parsed.

In the embodiment of FIG. 18, there may be a case in which mmvd_flag is not parsed even when the MMVD is not used and the skip mode or the merge mode is used.

According to an embodiment of the present invention, the base MV related syntax and merge_idx may be the same. For example, the base MV index and merge_index may be the same. To be identical can mean that they are expressed in the same syntax.

According to an embodiment of the present invention, it is possible to perform syntax parsing related to base MV and to pars mmvd_flag according to conditions. And when mmvd_flag instructs to use MMVD, syntax related to MMVD other than syntax related to base MV can be parsed.

For example, after parsing the merge index, it is possible to distinguish whether the corresponding candidate is a candidate that can be used for MMVD. In addition, in the case of a candidate that can be used for MMVD, mmvd_flag may be parsed, and in the case of a candidate that cannot be used in MMVD, mmvd_flag may not be parsed. In addition, if mmvd_flag does not exist, it can be inferred to not use MMVD.

More specifically, when the base MV of MMVD can be up to num_mmvd_baseCand in front of merge candidate list (the side with smaller index), mmvd_flag is parsed when the parsed index is smaller than num_mmvd_baseCand, otherwise, mmvd_flag is not parsed and MMVD You can decide not to use.

In this case, when using a candidate index greater than num_mmvd_baseCand, there is an advantage of saving bits for mmvd_flag.

Referring to FIG. 18, after parsing merge_idx, when merge_idx is smaller than num_mmvd_baseCand, mmvd_flag is parsed. num_mmvd_baseCand may be the number of candidates possible as the base MV of MMVD. Also, the base MV index of MMVD may be based on merge_idx.

19 is a diagram showing MMVD syntax according to an embodiment of the present invention.

Referring to FIG. 19, mmvd_flag may be higher than merge_flag. For example, it is possible to determine whether to pars the merge_flag based on mmvd_flag. Also, it is possible to apply this in cases other than skip mode. The mmvd_flag and the merge_flag may have the meaning described in the previous drawings.

For example, when mmvd_flag indicates that MMVD is used, merge_flag may not be parsed. For example, when the base MV candidate of MMVD is determined among merge candidates, when mmvd_flag indicates that MMVD is used, the merge_flag may not be parsed. When using MMVD (when mmvd_flag is 1), merge_flag can be inferred by using merge mode. Referring to FIG. 19, when a skip mode is used, that is, when cu_skip_flag is 1, merge_flag may be inferred to 1 regardless of whether MMVD is used. Also, if merge_flag does not exist and skip mode is not used, merge_flag can be inferred to 1 when MMVD is used, and merge_flag can be infered to 0 when MMVD is not used. Alternatively, if merge_flag does not exist and skip mode is not used, it is possible to infer merge_flag to 1.

In addition, referring to FIG. 19, when mmvd_flag is 1 when not in skip mode, MMVD related syntax is parsed, and when mmvd_flag is 0, merge_flag is parsed.

According to an embodiment of the present invention, when subblock_merge_flag is higher than merge_flag, when subblock_merge_flag is 1, merge_flag may not be parsed, and merge_flag may be inferred to 1. In addition, when subblock_merge_flag is higher than mmvd_flag, when subblock_merge_flag is 1, mmvd_flag may not be parsed, and mmvd_flag may be inferred to 0.

In addition, in the embodiments of the present invention, when modeX_flag is 1, it means that modeX is used, and when modeX_flag is 0, it may mean that modeX is not used.

20 is a diagram showing MMVD syntax according to an embodiment of the present invention.

According to an embodiment of the present invention, it is possible to always use MMVD for some candidates in a candidate list used as a base MV candidate for MMVD. For example, when the base MV candidate of MMVD is determined from the merge candidate list, it is always possible to use MMVD for a part of the merge candidate list. For example, it is possible to always use MMVD for a given candidate index. For example, it is always possible to use MMVD when the candidate index is smaller than a preset value.

In this case, it is possible to determine whether to use MMVD from the candidate index. Also, mmvd_flag parsing may not exist.

For example, if the number merge_idx parsed by merge_idx corresponds to a value determined as using MMVD, it may be determined to use MMVD. Also, in this case, MMVD related syntax (eg, distance, direction signaling, etc.) can be parsed.

Referring to FIG. 20, MMVD can always be used until the number of num_mmvd_baseCand in front of the merge candidate list is reached. In this case, if the candidate index is smaller than num_mmvd_baseCand, it can be determined to use MMVD, and MMVD-related syntax can be parsed.

21 is a diagram showing MMVD syntax according to an embodiment of the present invention.

As described in FIG. 20, according to an embodiment of the present invention, it is possible to always use MMVD for some candidates in a candidate list used as a base MV candidate for MMVD. For example, when the base MV candidate of MMVD is determined from the merge candidate list, it is always possible to use MMVD for a part of the merge candidate list. For example, it is possible to always use MMVD for a given candidate index. For example, it is always possible to use MMVD when the candidate index is smaller than a preset value.

According to an embodiment of the present invention, mmvd_flag may be higher than merge_flag. In this case, if mmvd_flag is 1, MMVD related syntax can be parsed. In addition, when mmvd_flag is 1, merge_flag may be inferred to 1. In addition, when mmvd_flag is 0, merge_flag can be parsed. When merge_flag is 1, merge_idx can be parsed. In this case, there may be an additional condition for parsing merge_idx. In this case, according to an embodiment, the merge_idx to be actually used may be determined by transforming it based on the parsed merge_idx. For example, it is possible to determine that the actual merge_idx to be used is the sum of the number of candidates determined to always use MMVD among values smaller than the parsed merge_idx to the parsed merge_idx. For example, if MMVD is always used for num_mmvd_baseCand in front of the candidate list, the value obtained by adding num_mmvd_baseCand to the parsed merge_idx can be used as merge_idx. This is because the candidate using MMVD can be excluded from the candidate list when mmvd_flag is before merge_flag and mmvd_flag is 0.

Referring to FIG. 21, mmvd_flag exists before merge_flag. Also, if mmvd_flag is 1, MMVD related syntax can be parsed. In addition, when mmvd_flag is 0, merge_flag can be parsed. Also, when mmvd_flag is 0 and merge_flag is 1, merge_idx can be parsed. In this case, there may be an additional condition for parsing merge_idx. In addition, merge_idx to be actually used can be determined by adding num_mmvd_baseCand, which is the number of candidates that can be used as the base MV of MMVD, to the parsed merge_idx.

22 is a diagram showing MMVD syntax according to an embodiment of the present invention.

According to an embodiment of the present invention, a candidate index may exist before a flag indicating whether to use a mode. For example, the candidate index may exist before mmvd_flag or merge_flag. In this case, after parsing the candidate index, it is possible to determine whether to pars the mmvd_flag according to whether the parsed index is a candidate that can use MMVD. For example, if the parsed candidate index is a candidate that can use MMVD, mmvd_flag can be parsed, and if the candidate index cannot use MMVD, mmvd_flag can be inferred to 0 without parsing. In addition, when mmvd_flag is 0, merge_flag can be parsed. Through merge_flag, it is possible to determine whether merge mode or subblock merge mode is used, inter mode, AMVP mode, or affine inter mode is used.

For example, if the number of candidates possible in mode 1 is num1, the number of candidates possible in mode 2 is num2, and num1 <num2, when the candidate index is parsed and the parsed index is num 1 or more, it is determined between mode 1 and mode 2 You can not pars the syntax (for example, the syntax indicating whether or not to use mode 1), and you can infer to not using mode 1. In addition, when the parsed index is less than num 1, the syntax that determines between mode 1 and mode 2 can be parsed.

Referring to FIG. 22, after parsing merge_idx, mmvd_flag may be parsed when merge_idx is smaller than num_mmvd_baseCand, which is the number of candiates that can use MMVD. Also, when merge_idx is greater than or equal to num_mmvd_baseCand, mmvd_flag may not be parsed, and in this case, mmvd_flag may be inferred to 0. In addition, when mmvd_flag is 1, MMVD related syntax distance_idx, direction_idx, etc. are parsed. In addition, when mmvd_flag is 0, merge_flag is parsed, and accordingly, whether it is merge mode or subblock merge mode, inter mode, AMVP mode, or affine mode can be distinguished.

23 is a diagram showing a coding unit syntax according to an embodiment of the present invention.

As shown in FIG. 23, when merge_flag is 1, merge_data can be performed. According to an embodiment, merge_data may include a part for parsing merge-related syntax. Also, merge_data may be called a merge data syntax. For example, merge_data may include a syntax parsing part performed when merge_flag is 1 in FIGS. 12 to 22. Also, when the merge_flag is 1, it may mean that the merge mode is used. In addition, when the merge_flag is 1, it may indicate that the inter prediction that does not use mvd_coding shown in FIG. 10 or 12 is used.

24 is a diagram illustrating a merge data syntax according to an embodiment of the present invention.

As described in FIG. 23, when merge_flag is 1, the merge data syntax can be parsed. Also, as described above, when the merge mode or skip mode is used, merge_flag may be set to 1. In addition, referring to FIG. 24, mmvd_flag may be parsed first in the merge data syntax. Also, mmvd_flag may be a syntax that can be parsed first after confirming that merge_flag is 1. In addition, mmvd_flag may be parsed prior to signaling indicating whether to use another prediction mode in which merge_flag is set to 1. Also, mmvd_flag may be a signaling indicating whether MMVD is used. This may be the same as the embodiment shown in Fig. 14 and the like. 24 includes syntax for subblock merge mode, MH intra (multi-hypothesis prediction, intra and inter combined prediction), triangle prediction, and the like.

According to an embodiment of the present invention, MH intra may be a method of generating and combining two or more prediction blocks when generating a prediction block. Alternatively, MH intra may be a method of using both inter prediction and intra prediction when generating a prediction block. In addition, inter prediction and intra prediction may be a method of using a picture different from the picture including the current block, or the same picture when performing prediction, respectively. Referring to FIG. 24, mh_intra_flag may be signaling indicating whether MH intra is used.

According to an embodiment of the present invention, the subblock merge mode may be a method of performing motion compensation in subblock units (motion vector is determined) when predicting a current block (coding unit or prediction unit, etc.). The subblock merge mode may include methods such as subblock-based temporal motion vector prediction (ATMVP) and affine motion prediction. Referring to FIG. 24, merge_subblock_flag may be a signaling indicating whether a subblock merge mode is used.

According to an embodiment of the present invention, triangle prediction may be a method of performing motion compensation for a non-rectangular area within the current block. That is, in triangle prediction, the unit of the same motion vector within the current block may not be a square. Referring to FIG. 24, merge_triangle_flag may be signaling indicating whether triangle prediction is used.

According to an embodiment of the present invention, mmvd_flag may be parsed before signaling indicating that a merge mode other than MMVD is used. Signaling indicating the use of a merge mode other than the MMVD may include mh_intra_flag, merge_subblock_flag, merge_triangle_flag, and the like.

Referring to FIG. 24, mmvd_merge_idx may be a signaling indicating which one to use as the base MV of MMVD. If MMVD is used, merge_idx can be inferred to mmvd_merge_flag.

25 is a diagram showing a merge data syntax according to an embodiment of the present invention.

The embodiment of FIG. 25 may be a merge_data part performed when merge_flag is 1 in FIG. 23.

According to an embodiment of the present invention, mmvd_flag may be parsed after merge_idx. For example, mmvd_flag may be parsed immediately after merge_idx.

For example, mmvd_flag may not be parsed at a position other than the earliest in the merge data syntax. That is, after confirming that merge_flag is 1, after parsing syntax other than mmvd_flag, mmvd_flag can be parsed.

In addition, according to an embodiment of the present invention, as described in FIG. 13, the base MV of MMVD may be determined from another candidate list, and in an embodiment, the base MV may be determined from a part of the other candidate list. Therefore, as described in FIG. 18, according to the present embodiment, whether to pars the mmvd_flag may be determined based on an index related to another candidate list that can be used as the base MV of MMVD. For example, an index related to another candidate list that can be used as the base MV of MMVD can be parsed before mmvd_flag. In addition, when the index related to the other candidate list indicates that the base MV of MMVD can be used, it is possible to pars mmvd_flag, otherwise it is possible not to pars the mmvd_flag. Referring to FIG. 25, the base MV of MMVD may be determined from a merge candidate. Therefore, in an embodiment, when merge_idx indicates that it can be used as the base MV of MMVD, mmvd_flag may be parsed, otherwise mmvd_flag may not be parsed. Or, when the base MV of MMVD can be selected from among the n front parts of the merge candidate list, when merge_idx is less than n (merge_idx can start from 0), parsing mmvd_flag and merge_idx is not less than n The mmvd_flag may not be parsed. More specifically, the base MV of MMVD may be the first or second candidate of the merge candidate list, and referring to FIG. 25, when merge_idx is less than 2, that is, 0 or 1, mmvd_flag may be parsed. merge_idx may represent a merge candidate index. Accordingly, in a merge mode, but when merge_idx indicates that MMVD cannot be used, mmvd_flag may not be parsed, and thus coding efficiency may be improved.

Also, the base MV of MMVD can be determined from the candidate list of any mode. Therefore, according to an embodiment of the present invention, when there are multiple signaling indicating whether various modes are used, it is possible to pars the mmvd_flag after determining which mode to use. For example, when mode 1, mode 2, and mode 3 exist and MMVD is determined based on mode 3, or when the base MV of MMVD is determined from the candidate of mode 3, mmvd_flag after it is determined to use mode 3 It is possible to parsing. For example, it may be determined to use mode 3 through signaling indicating whether mode 3 is used. Or, for example, when it is determined that possible modes other than mode 3, for example, mode 1 and mode 2 are not used, it may be determined to use mode 3. For example, when there is a subblock merge mode, MH intra, triangle prediction, conventional merge mode, and if MMVD can be applied to the conventional merge mode, after it is determined to use the conventional merge mode or MMVD among the listed modes You can pars the mmvd_flag.

Also, when parsing mmvd_flag based on an index related to another candidate list that can be used as the base MV of MMVD, there is no need for a separate signaling indicating the base MV of MMVD. For example, in FIG. 16 or 17, signaling such as base_mv_idx existed separately from merge_idx. Also, referring to FIG. 24, mmvd_merge_flag may be a signaling indicating the base MV of MMVD. According to an embodiment of the present invention, mmvd_merge_flag may not exist as shown in FIG. 25. For example, the MMVD related syntax may include only mmvd_flag, a signaling indicating the MMVD distance (mmvd_distance_idx in FIGS. 24 to 25), and a signaling indicating the MMVD direction (mmvd_direction_idx in FIGS. 24 to 25). Also, the base MV of MMVD can be determined by merge_idx. Accordingly, the signaling and context model of the base MV of the MMVD that existed in FIG. 24 may not exist in the embodiment of FIG. 25.

In addition, according to an embodiment of the present invention, mmvd_flag may be parsed after signaling indicating that a merge mode other than MMVD described above is used. Referring to FIG. 25, mmvd_flag may be parsed after merge_subblock_flag, merge_triangle_flag, mh_intra_flag, and the like.

Also, some merge modes may not be used with MMVD. In such a case, it is possible to pars the mmvd_flag when the flag indicating which merge mode is used indicates not being used. Referring to FIG. 25, it is possible that MMVD cannot be used with triangle prediction, and when merge_triangle_flag is 0, mmvd_flag may be parsed. Alternatively, MMVD may not be used with MH intra, and mmvd_flag may be parsed when mh_intra_flag is 0. Alternatively, MMVD may not be used together with the subblock merge mode, and mmvd_flag may be parsed when merge_subblock_flag is 0.

26 is a diagram showing merge data syntax according to an embodiment of the present invention.

The embodiment of FIG. 26 may be a merge_data part performed when merge_flag is 1 in FIG. 23.

According to an embodiment of the present invention, mmvd_flag may be parsed after signaling indicating whether another mode is used. Alternatively, mmvd_flag may be parsed after signaling indicating whether a mode other than MMVD in which merge_flag is set to 1 is used. For example, mmvd_flag may be parsed after merge_subblock_flag, mh_intra_flag, and merge_triangle_flag.

Also, mmvd_flag may be parsed when a mode other than MMVD is not used. For example, when merge_subblock_flag is 0, mmvd_flag may be parsed. In addition, when mh_intra_flag is 0, mmvd_flag may be parsed. Also, when merge_triangle_flag is 0, mmvd_flag may be parsed. If the flag indicating which mode is used is 0, it may indicate that the mode is not used.

In addition, when MMVD can be applied to the conventional merge mode, it is possible to pars mmvd_flag after it is determined that the conventional merge mode or MMVD is used.

Also, in this embodiment, when MMVD is used, MMVD-related syntax may be parsed, and when MMVD is not used, merge_idx may be parsed. Alternatively, when MMVD is used, mmvd_merge_flag, mmvd_distance_idx, mmvd_direction_idx, etc. can be parsed, and when MMVD is not used, merge_idx can be parsed.

In addition, when parsing mmvd_flag behind a flag indicating whether to use a different mode as shown in FIG. 26, signaling indicating the base MV of the MMVD may exist separately from the merge_idx. Referring to FIG. 26, the signaling indicating the base MV of MMVD may be mmvd_merge_flag. For example, mmvd_flag can be parsed after it is determined to use conventional merge mode or MMVD.

According to an embodiment of the present invention, the conventional merge mode may mean a merge mode used in HEVC.

According to an embodiment of the present invention, the base MV of MMVD may be determined from a candidate list, and a maximum number of candidates of the candidate list may be variable. For example, the maximum number of candidates can be determined from the high level syntax. The high level syntax may be a syntax of a higher level than the current coding unit. For example, the high level syntax may be a level syntax such as sequence, slice, or tile. According to an embodiment, in this case, the maximum number of candidates of the base MV of MMVD may follow the maximum number of candidates of the candidate list that can be the base MV of MMVD. Therefore, when the maximum number of candidates of the candidate list that can be the base MV of MMVD decreases, the maximum number of candidates of the candidate that can be the base MV of MMVD can also be reduced.

For example, the base MV of MMVD may be determined from the merge candidate list, and the maximum number of merge candidates may be MaxNumMergeCand. Also, MaxNumMergeCand can be determined from high level syntax. In this case, the maximum number of candidates that can be the base MV of MMVD may be less than MaxNumMergeCand. Therefore, as shown in FIG. 17, it may be determined whether to pars the signaling indicating the base MV of MMVD according to MaxNumMergeCand. For example, if MaxNumMergeCand is 1, the signaling indicating the base MV of MMVD may not be parsed, and if there is no signaling indicating the base MV of MMVD, it may be inferred to 0.

In another embodiment, when MaxNumMergeCand is 1, MMVD may not be used. This may be to reduce the overhead for MMVD related syntax. Therefore, when MaxNumMergeCand is 1, mmvd_flag may not be parsed. Also, if mmvd_flag does not exist, it can infer to 0.

27 is a diagram illustrating a merge data syntax according to an embodiment of the present invention.

The embodiment of FIG. 27 may be a merge_data part performed when merge_flag is 1 in FIG. 23.

Referring to FIG. 27, merge_triangle_flag can be parsed only when mh_intra_flag is not used. If merge_triangle_flag does not exist, it is possible to infer to 0. In addition, the embodiment of FIG. 27 may include the embodiments described with reference to FIG. 25.

According to an embodiment of the present invention, there may be a plurality of modes that cannot be used together. For example, subblock merge mode, triangle prediction, MH intra, and MMVD may not be used together. In addition, signaling indicating whether to use each of a plurality of modes that cannot be used together may be parsed according to a preset order. In this case, the signaling indicating the use of any mode among the plurality of modes is parsed only when it is determined that all the signaling indicating the use of any other mode among the plurality of modes parsed before it is determined not to be used. It is possible to do.

28 is a diagram showing a coding unit syntax according to an embodiment of the present invention.

According to an embodiment of the present invention, mmvd_flag may be signaled when merge_flag is 0 (when the merge mode is not used). In addition, MMVD-related syntax elements may be signaled when merge_flag is 0 (when merge mode is not used).

As described above, MMVD may be a method of signaling a motion vector difference from a base candidate. In this respect, it may have similarity to modes such as AMVP and affine AMVP (affine inter) signaling MVD. Accordingly, when merge_flag is 0, signaling may be possible.

In addition, according to an embodiment of the present invention, CuPredMode may be a value indicating a prediction mode of a current block. Alternatively, CuPredMode may be a value indicating whether the current block is intra prediction or inter prediction. Alternatively, CuPredMode may be a value determined by pred_mode_flag. If pred_mode_flag is 0, CuPredMode may be set to a value indicating inter prediction is used. A value indicating the use of inter prediction may be MODE_INTER. If pred_mode_flag is 1, CuPredMode may be set to a value indicating that intra prediction is used. A value indicating the use of intra prediction may be MODE_INTRA. If pred_mode_flag does not exist, it is possible to set CuPredMode to a preset value. Also, the preset value may be MODE_INTRA.

Also, cu_cbf may be a value indicating whether syntax related to transform exists. The syntax related to the transform may be a transform tree syntax structure. In addition, the syntax related to the transform may be a syntax signaled through the transform_tree of FIG. 28. In addition, when cu_cbf is 0, the syntax related to transform may not exist. Also, when cu_cbf is 1, there may be syntax related to transform. Referring to FIG. 28, when cu_cbf is 1, transform_tree may be performed. If cu_cbf does not exist, it is possible to determine the cu_cbf value based on cu_skip_flag. For example, when cu_skip_flag is 1, cu_cbf may be 0. Also, when cu_skip_flag is 0, cu_cbf may be 1. As described above, cu_skip_flag may indicate whether the skip mode is used, and the use of the skip mode may indicate that a residual signal is not used. In other words, it may be a mode that restores the prediction signal without adding residuals. Therefore, when cu_skip_flag is 1, it may mean that the syntax related to transform does not exist.

According to an embodiment of the present invention, cu_cbf can be parsed when intra prediction is not used. Also, when cu_skip_flag is 0, cu_cbf can be parsed. Also, when merge_flag is 0, cu_cbf can be parsed. Also, these conditions can be performed in combination. For example, if CuPredMode is not MODE_INTRA and merge_flag is 0, cu_cbf can be parsed. Alternatively, when CuPredMode is MODE_INTER and merge_flag is 0, cu_cbf can be parsed. This may be because the skip mode may or may not be used in the case of inter prediction other than the merge mode.

29 is a diagram showing a coding unit syntax according to an embodiment of the present invention.

FIG. 29 is an example in which syntax related to cu_cbf and transform is changed based on the MMVD signaling of FIG. 28.

According to an embodiment of the present invention, when a specific mode is used, whether to use the skip mode may be determined. For example, when using MMVD, whether to use skip mode can be determined. It is possible to determine whether to use the skip mode or not to pars the cu_cbf accordingly. That is, if it is clear whether to use skip mode, cu_cbf may not be parsed.

According to an embodiment, when using MMVD, skip mode may not be used. Since MMVD cannot accurately represent MVD like AMVP, but can only be expressed in a limited range as described above, it is possible to more accurately restore through residuals. Therefore, it is possible to determine whether to pars the cu_cbf based on whether or not MMVD is used. For example, in the case of using MMVD, it is possible not to parsing cu_cbf. Also, it is possible to pars cu_cbf when MMVD is not used. Referring to FIG. 29, when mmvd_flag is 0, cu_cbf can be parsed, and when mmvd_flag is 1, cu_cbf is not parsed.

Also, if cu_cbf does not exist, a method of inferring cu_cbf value may be required. According to the method described with reference to FIG. 28, cu_cbf may be inferred based on the cu_skip_flag value. According to an embodiment of the present invention, it is possible to infer a cu_cbf value based on the merge_flag. If merge_flag is 0, cu_cbf can be inferred to 1. If merge mode is not used, it may indicate that there is syntax related to transform. Accordingly, in the case of using the MMVD in the embodiments of FIGS. 28 to 29, cu_cbf may be inferred to 1. When combined with the cu_cbf infer method described in FIG. 28, 1) when merge_flag is 1 and cu_skip_flag is 1, cu_cbf is inferred to 0, 2) merge_flag is 1, when cu_skip_flag is 0, cu_cbf is inferred to 1, and 3) When merge_flag is 0, cu_cbf can be inferred to 1. Alternatively, as shown in FIG. 28, 1) cu_cbf may be inferred to 0 when cu_skip_flag is 1, and 2) cu_cbf may be inferred to 1 when cu_skip_flag is 0.

According to another embodiment of the present invention, it is possible to infer a cu_cbf value based on mmvd_flag. When mmvd_flag is 1, it is possible to infer cu_cbf to 1. Also, when mmvd_flag is 0, it is possible to infer cu_cbf to 0 or 1. When combined with the infer method described in FIG. 28, 1) when mmvd_flag is 1, cu_cbf is inferred to 1, 2) when mmvd_flag is 0, cu_skip_flag is 1, cu_cbf is inferred to 0, and 3) mmvd_flag is 0 , If cu_skip_flag is 0, cu_cbf can be inferred to 1.

In addition, in the embodiments of FIGS. 28 to 29, it is possible that mmvd_flag does not exist in merge_data. merge_data may be merge_data shown in FIGS. 28 to 29, merge_data shown in FIGS. 24 to 27, and the like.

30 is a diagram illustrating merge mode signaling according to an embodiment of the present invention.

Referring to FIG. 30, it is possible to perform merge mode signaling through a regular flag, an MMVD flag, a subblock flag, a CIIP flag, or the like. 30(a) and 30(b) may correspond to a merge mode that is not a skip and a merge mode that is a skip, respectively. In the embodiment of FIG. 30, different from the merge data syntax of the previous figures, a regular flag may exist. Also, the triangle flag may not exist. The regular flag may be signaling indicating to use an existing merge mode. The existing merge mode may be the same merge mode as used in HEVC. Also, the existing merge mode may be a merge mode that uses a candidate indicated by merge_idx and does not use a motion vector difference. The regular flag, MMVD flag, subblock flag, and CIIP flag may be signaled in a preset order. The MMVD flag may be signaling indicating whether to use MMVD, the subblock flag may be merge_subblock_flag of the previous drawings, and the CIIP flag may be mh_intra_flag of the previous drawings. Among the regular flags, MMVD flags, subblock flags, and CIIP flags, signaling indicating that a corresponding mode is used may be 1 at most. Therefore, when a value of 1 among the Regular flag, MMVD flag, subblock flag, and CIIP flag occurs, the flags behind it may be determined to be 0. In addition, when the Regular flag, MMVD flag, subblock flag, and CIIP flag are all 0, a mode not indicated by the Regular flag, MMVD flag, subblock flag, and CIIP flag may be used. A mode not indicated by the Regular flag, MMVD flag, subblock flag, and CIIP flag may be triangle prediction. The signaling indicating whether to use a mode not indicated by the Regular flag, MMVD flag, subblock flag, and CIIP flag may be merge_triangle_flag in the preceding drawings.

31 is a diagram illustrating a merge data syntax according to an embodiment of the present invention.

FIG. 31 may show the syntax structure described in FIG. 30. The regular flag described in FIG. 30 may be a regular_merge_flag in FIG. 31. According to an embodiment of the present invention, regular_merge_flag may be in front of the merge data syntax. That is, the first syntax parsed after confirming that merge_flag is 1 may be regular_merge_flag. Also, when the regular_merge flag is 0, it is possible to pars the mmvd_flag. In addition, when regular_merge_flag is 0, it is possible to pars merge_subblock_flag or mh_intra_flag or merge_triangle_flag. Referring to FIG. 31, it is possible to pars mmvd_flag when a certain block size condition is satisfied. Also, the merge_triangle_flag may be determined as !regular_merge_flag && !mmvd_flag && !merge_subblock_flag && !mh_intra_flag. That is, when regular_merge_flag, mmvd_flag, merge_subblock_flag, and mh_intra_flag are all 0, merge_triangle_flag is 1, and when any one of regular_merge_flag, mmvd_flag, merge_subblock_flag, mh_intra_flag is 1, merge_triangle_flag may be 0.

32 is a diagram illustrating a merge data syntax according to an embodiment of the present invention.

32 may show the syntax structure described in FIG. 30. Referring to FIG. 32, when regular_merge_flag is 1, merge_idx can be parsed. Also, when MaxNumMergeCand is greater than 1, merge_idx can be parsed. In addition, when regular_merge_flag is 0, it may be possible to parsing mmvd_flag, merge_subblock_flag, mh_intra_flag, and the like. Also, the value of merge_triangle_flag may be determined by the method described with reference to FIGS. 30 to 31. That is, merge_triangle_flag may be determined based on another flag value. If merge_triangle_flag is 1, it is possible to pars the syntax related to triangle prediction. For example, it is possible to pars the merge_triangle_idx.

In the case of the embodiment of FIG. 31, the merge_idx required for the regular merge mode exists at the back of the merge data syntax, and signals such as mmvd_flag, merge_subblock_flag, and mh_intra_flag exist between the regular_merge_flag and the merge_idx. If used, signaling can be inefficient. However, in the embodiment of FIG. 32, when regular_merge_flag is 1, since merge_idx can be parsed immediately after regular_merge_flag, it may not be necessary to pars other signals irrelevant to the regular merge mode, and thus compression efficiency can be improved. .

33 is a diagram illustrating a syntax structure described in FIGS. 30 to 32.

Referring to FIG. 33, there may be Mode A, Mode B, Mode C, and the like. Also, it is possible to use only one of Mode A, Mode B, and Mode C. There may also be conditions for using Mode A. Conditions for using Mode A may be A1, A2, A3, etc., and it is possible to use Mode A when all of these are satisfied. Also, the conditions for using Mode B may be B1, B2, B3, etc., and it is possible to use Mode B when all of these are satisfied. In addition, conditions for using Mode C may be C1, C2, C3, etc., and it is possible to use Mode C when all of these are satisfied. Signaling indicating whether to use Mode X may be mode_X_flag. Also, referring to FIG. 33, modes are determined in the order of mode A, mode B, and mode C, and related syntax can be parsed. Alternatively, it is possible to signal in the order of mode_A_flag, mode_B_flag, and mode_C_flag. If the condition for using Mode A is satisfied, mode_A_flag can be parsed. If mode_A_flag is 1, syntax related to mode A may be parsed, and mode_X_flag related to the remaining modes and related syntax may not be parsed. If mode_A_flag is 0, there may be a possibility to use mode B or mode C. Therefore, if the condition for using mode B is satisfied, mode_B_flag can be parsed. If mode_B_flag is 1, syntax related to mode B may be parsed, and mode_X_flag related to the remaining mode (mode C) and related syntax may not be parsed. If mode_B_flag is 0, it may be determined that mode C is used. That is, if all of the mode_X_flags that do not correspond to mode C are 0, it may be determined to use mode C. And you can parsing syntax related to mode C.

34 is a diagram showing a syntax structure according to an embodiment of the present invention.

As described with reference to FIG. 33, Mode A, Mode B, Mode C, etc. may exist, and accordingly, mode_X_flag indicating whether to use a mode, related syntax, etc. may exist. In addition, there may be conditions for using mode X, such as X1, X2, X3, etc. Also, mode is determined in the order of mode A, mode B, and mode C, and related syntax can be parsed.

In this case, according to an embodiment of the present invention, when all modes determined after mode X are not available, it may be determined to use mode X. Also, at this time, mode_X_flag may not be parsed. Also, whether the mode is unavailable may be determined according to whether or not the conditions for using the above-mentioned mode are satisfied. For example, when mode A, mode B, and mode C exist and mode B and mode C are both unavailable, mode_A_flag may not be parsed, and it may be determined that mode A is used. Even if there are more modes other than Mode A, Mode B, and Mode C, it is possible to determine the mode using this method. For example, when mode A, mode B, mode C, and mode D exist, it can be determined that mode A is used when mode B, mode C, and mode D are all unavailable. In addition, after determining that mode A is not used, it can be determined that mode B is used when both mode C and mode D are unavailable.

Referring to FIG. 34, a condition in which mode X cannot be used may be when any one of X1, X2, and X3 is not satisfied. I.e. !X1 || !X2 || If !X3, mode X may not be available. Therefore, referring to FIG. 34, when both mode B and mode C cannot be used, ((!B1 || !B2 || !B3) && (!C1 || !C2 || !C3)) is satisfied. In this case, mode_A_flag may not be parsed, and its value may be inferred to 1. That is, it can be determined that mode A is used. Also, if ((!B1 || !B2 || !B3) && (!C1 || !C2 || !C3)) is not satisfied, mode_A_flag can be parsed. At this time, if the conditions for using mode A are also considered, !((!B1 || !B2 || !B3) && (!C1 || !C2 || !C3)) is satisfied, and (A1 && A2 && A3) If it is, mode_A_flag can be parsed. In other words, mode_A_flag can be parsed if either the condition for using mode B or the condition for using mode C is satisfied. That is, in the case of (B1 && B2 && B3) or (C1 && C2 && C3), mode_A_flag can be parsed.

In addition, if mode_A_flag does not exist, it can infer to 0 when it is (B1 && B2 && B3) or (C1 && C2 && C3), and infer it to 1 otherwise. In other words, when both mode B and mode C are unavailable, if mode_A_flag does not exist, it can be inferred to 1 (the one that is used).

35 is a diagram showing a syntax structure according to an embodiment of the present invention.

FIG. 35 may be based on the method described in FIG. 34. Also, as described in FIGS. 30 to 31, it may be an embodiment in which regular_merge_flag exists.

As described above, according to an embodiment of the present invention, when all modes in which the use is determined later than the one in which the mode is used are not available, the signaling indicating the use of the mode is not parsed, and the It can be determined that some mode is used. For example, if all the modes that are used later than the subblock merge mode are determined are not available, it is determined that the subblock merge mode is used without parsing the signaling indicating whether the subblock merge mode is used. can do. The modes in which the use or not is determined late may include mh_intra and triangle prediction.

For example, if all the modes in which the use of MMVD is determined later than that of the use of MMVD are not available, it may be determined that the MMVD is used without parsing the signaling indicating whether to use the MMVD. The modes in which the use or not is determined late may include a subblock merge mode, mh_intra, and triangle prediction.

In addition, the conditions for using mh_intra in the above embodiments (mh_intra_conditions in FIG. 35) may include 1) sps_mh_intra_enabled_flag, 2) cu_skip_flag[x0][y0] == 0, 3) block size conditions, and the like. In addition, the block size condition may be (( cbWidth * cbHeight) >= 64 && cbWidth <128 && cbHeight <128).

In addition, in the above embodiments, conditions for using triangle prediction (merge_triangle_conditions in FIG. 35) may include 1) sps_triangle_enabled_flag, 2) tile_group_type == B, 3) block size conditions, and the like. Also, the block size condition may be (cbWidth * cbHeight >= 64).

In addition, in the above embodiments, conditions for using subblock merge (merge_subblock_conditions in FIG. 35) may include 1) MaxNumSubblockMergeCand> 0, 2) block size conditions, and the like. Also, the block size condition may be (cbWidth >= 8 && cbHeight >= 8).

Therefore, in the case of (!mh_intra_conditions && !merge_triangle_conditions), the merge_subblock_flag may not be parsed. In addition, if merge_subblock_flag does not exist (!mh_intra_conditions && !merge_triangle_conditions), merge_subblock_flag may be inferred to 1, otherwise, inferred to 0.

Also, in the case of (!merge_subblock_conditions && !mh_intra_conditions && !merge_triangle_conditions), mmvd_flag may not be parsed. In addition, if mmvd_flag does not exist (!merge_subblock_conditions && !mh_intra_conditions && !merge_triangle_conditions), mmvd_flag can be inferred to 1, otherwise it can infer to 0.

For example, in the case of (!sps_mh_intra_enabled_flag && !sps_triangle_enabled_flag), merge_subblock_flag may not be parsed and may be inferred to 1. Alternatively, when cu_skip_flag is 1 and tile_group_type is not B, merge_subblock_flag may not be parsed and may be inferred to 1. Alternatively, when width and height are 128 and 128, respectively, and tile_group_type is not B, merge_subblock_flag may not be parsed, and may be inferred to 1.

36 is a diagram showing MVD derivation of MMVD according to an embodiment of the present invention.

According to an embodiment of the present invention, the MVD may be determined from the signaled MVD. This shows a method of using a signaled MVD or an MV modified from the signaled MV as described in FIG. 13. In addition, it may represent a modification based on the POC described in FIG. 13.

Referring to FIG. 36, MmvdOffset may be a signaled MV. For example, as previously described with MMVD, MmvdOffset may be determined based on MMVD distance and MMVD direction. For example, MmvdDistance may be determined based on mmvd_distance_idx. For example, when there are multiple MmvdDistance candidates, MmvdDistance may be determined based on mmvd_distance_idx. For example, MmvdDistance may be selected from {1, 2, 4, 8, 16, 32, 64, 128}. In addition, MmvdSign may be determined based on mmvd_direction_idx. In addition, MmvdSign[0] and/or MmvdSign[1] may be determined based on mmvd_direction_idx. MmvdSign[0] may be a value corresponding to the x-axis, and MmvdSign[1] may be a value corresponding to the y-axis. The MMVD distance may be MmvdDistance, and the MMVD direction may be MmvdSign. Also, MmvdOffset may be determined based on MmvdDistance and MmvdSign. For example, MmvdOffset may be determined as follows.

MmvdOffset[ x0 ][ y0 ][ 0] = (MmvdDistance[ x0 ][ y0] << 2) * MmvdSign[ x0 ][ y0 ][0]

MmvdOffset[ x0 ][ y0 ][ 1] = (MmvdDistance[ x0 ][ y0] << 2) * MmvdSign[ x0 ][ y0 ][1]

MmvdOffset[x0][y0][0] may be a value corresponding to the x-axis, and MmvdOffset[x0][y0][0] may be a value corresponding to the y-axis. In addition, MmvdDistance was bit shifted in the formula for calculating MmvdOffset, and the number of bit shifts may vary according to embodiments.

As shown in Fig. 36, mMvdLX can be derived based on MmvdOffset. LX can represent a reference list. LX may be L0 or L1.

In addition, according to an embodiment of the present invention, when there is a motion vector mvLX, mvLX may be determined and derived through mvLX += mMvdLX. For example, mvLX may be a merge motion vector. In addition, this mvLX determination operation can be performed when using MMVD. That is, mvLX can be determined from the merge candidate, and mvLX can be modified using mMvdLX. For example, mvLX can be calculated as follows.

mvLX[ 0 ][ 0 ][ 0] += mMvdLX[ 0]

mvLX[ 0 ][ 0 ][ 1] += mMvdLX[ 1]

In the above formula, using mMvdLX[0] corresponds to the x-axis, and using mMvdLX[1] may correspond to the y-axis.

Referring to FIG. 36, the operation may be performed by dividing the case where predFlagL0 and predFlagL1 are both 1 and not. Also, a case in which both predFlagL0 and predFlagL1 are 1 may be a case of bi-prediction. Also, otherwise, it may be a case of uni-prediction.

In the case of Bi-prediction, MmvdOffset or -MmvdOffset can be substituted into mMvdLX. Then you can perform MV scaling. Also, you can refer to the POC at this time. For example, the POC of the current picture (currPic in Fig. 36), a POC corresponding to reference picture list 0 (RefPicList0[refIdxL0] in Fig.36; refIdxL0 is a reference index for L0), a POC corresponding to reference picture list 1 (Fig. In 36, RefPicList1[refIdxL1]; refIdxL1 may refer to a reference index for L1). In addition, DiffPicOrderCnt(picA, picB) may be a difference in POC between picture picA and picture picB. For example, it may be DiffPicOrderCnt(picA, picB) = PicOrderCnt(picA)-PicOrderCnt(picB). PicOrderCnt may be a POC. Referring to FIG. 36, as shown in (8-297) and (8-298), currPocDiffL0 and currPocDiffL1 can be defined. If -currPocDiffL0 * currPocDiffL1 is greater than 0, MmvdOffset may be substituted for one of mMvdL0 and mMvdL1, and -MmvdOffset may be substituted for the other. For example, it can be substituted as shown in (8-299), (8-300), (8-301), and (8-302). If -currPocDiffL0 * currPocDiffL1 is greater than 0, it may be assumed that the reference picture corresponding to L0 and the reference picture corresponding to L1 from the current picture are in opposite directions in the POC order, so MVD will be in the opposite direction. That is, symmetric MVD may be assumed. In the embodiment of FIG. 36, an example in which MmvdOffset, which is a signaled MVD, corresponds to L0 is shown.

If -currPocDiffL0 * currPocDiffL1 is not greater than 0, that is, if -currPocDiffL0 * currPocDiffL1 is less than 0 (0, it may be the case when the current picture is a reference picture because currPocDiffL0 or currPocDiffL1 is 0). MmvdOffset can be substituted for both and mMvdL1. This may be because the reference pictures corresponding to L0 and L1 are in the same direction on the POC from the current picture.

Also, as described above, MV scaling can be performed after MmvdOffset is substituted for mMvdLX. For example, you can scale one of mMvdL0 and mMvdL1. For example, one of mMvdL0 and mMvdL1 can be scaled according to a preset rule. For example, it is possible to decide to scale MV based on Abs(currPocDiffL0) or Abs(currPocDiffL1). For example, you can scale mMvdLX, which is the smaller of Abs(currPocDiffL0) and Abs(currPocDiffL1). For example, when scaling mMvdL1, it is possible to do what is shown in (8-307) to (8-312). Or, for example, when scaling mMvdL0, it is possible to do what is shown in (8-313) to (8-318). In addition, it is possible to use a clipping range different from that of the embodiment of FIG. 36 in MV scaling. For example, in (8-311), (8-312), (8-317), (8-318), etc., clipping was set to -2^15, 2^15-1, and these values are the groups that MV wants to represent. It may be a different value depending on the set range. For example, clipping can be set to -131072 and 131071. In addition, in the embodiment of FIG. 36, MV scaling may be performed only when -currPocDiffL0 * currPocDiffL1 is greater than 0. In another embodiment, MV scaling can be performed both when -currPocDiffL0 * currPocDiffL1 is greater than 0 and not.

In the case of Uni-prediction, it may be a value corresponding to the reference list used by MmvdOffset. Therefore, as shown in (8-319) and (8-320) of FIG. 36, MmvdOffset may be mMVDLX for LX in which predFlagLX is 1.

The symbols, promises, equations, MV scaling equations, and the like described in FIG. 36 can also be applied to the following inventions.

37 is a diagram showing MVD derivation of MMVD according to an embodiment of the present invention.

As described in FIG. 36, mMvdLX can be calculated using MmvdOffset based on the signaled value. The overlapping description of FIG. 36 is omitted. For example, the description of FIG. 36 and the like may be referred to for currPocDiffLX and MV scaling in the following embodiments. In FIG. 36, there may be a case where -MmvdOffset is substituted into mMvdLX based on the difference in POC, and there may be a case of performing MV scaling after that. MV scaling may be a calculation method in consideration of the POC location. Therefore, in the embodiment of FIG. 36, when the codes of currPocDiffL0 and currPocDiffL1 are different, a case of changing the code once from MmvdOffset may be changed twice using the symmetric MVD characteristic. 37 may be an embodiment for solving this problem.

Referring to FIG. 37, in the case of bi-prediction, 1) when currPocDiffL0 and currPocDiffL1 are the same, 2) otherwise, and when Abs (currPocDiffL0) is greater than Abs (currPocDiffL1), 3) the remaining cases may be divided to perform the mMvdLX derivation operation. .

First, when currPocDiffL0 and currPocDiffL1 are the same, both mMvdL0 and mMvdL1 may be set to MmvdOffset. Also, at this time, the MV scaling process may not be performed.

In addition, in cases 2) and 3), one of mMvdL0 and mMvdL1 may be set as MmvdOffset according to a preset rule, and the other may be determined by scaling mMvdLX set as MmvdOffset. For example, when Abs(currPocDiffL0) is greater than Abs(currPocDiffL1), mMvdL0 may be set as MmvdOffset, and mMvdL1 may be set as a value obtained by scaling mMvdL0. In the remaining cases, mMvdL1 may be set to MmvdOffset, and mMvdL0 may be set to a value obtained by scaling mMvdL1. In this case, in the remaining cases, Abs(currPocDiffL0) is less than Abs(currPocDiffL1) and Abs(currPocDiffL0) and Abs(currPocDiffL1) are the same, but currPocDiffL0 and currPocDiffL1 have different signs (that is, currPocDiffL1 = -currPocDiffL1). have.

In another embodiment, 1) when currPocDiffL0 and currPocDiffL1 are the same, 2) otherwise, and when Abs (currPocDiffL0) is smaller than Abs (currPocDiffL1), 3) the remaining cases can be divided into the mMvdLX derivation operation. In this case, in the remaining cases, Abs(currPocDiffL0) is greater than Abs(currPocDiffL1) and Abs(currPocDiffL0) and Abs(currPocDiffL1) are the same, but currPocDiffL0 and currPocDiffL1 have different signs (ie, currPocDiffL1 = -currPocDiffL1). have.

38 is a diagram showing MVD derivation of MMVD according to an embodiment of the present invention.

As described with reference to FIGS. 36 to 37, mMvdLX may be calculated using MmvdOffset based on the signaled value. The overlapping description of FIGS. 36 to 37 has been omitted. For example, descriptions of FIGS. 36 to 37 may be referred to for currPocDiffLX and MV scaling in the following embodiments.

In the embodiment of FIG. 37, when Abs (currPocDiffL0) and Abs (currPocDiffL1) are the same, but the signs of currPocDiffL0 and currPocDiffL1 are different (ie, currPocDiffL0 = -currPocDiffL1), MV scaling was also performed. But this can be redundant.

Therefore, according to an embodiment of the present invention, when Abs (currPocDiffL0) and Abs (currPocDiffL1) are the same, but the signs of currPocDiffL0 and currPocDiffL1 are different (that is, currPocDiffL0 = -currPocDiffL1), mMvdL0 and mMvdL1 are substituted for one of the remaining ones and MmvdOffset -MmvdOffset may be substituted for, and MV scaling may not be performed for both mMvdL0 and mMvdL1. At this time, determining one and the other may follow a preset rule. For example, L0 can be fixed by substituting MmvdOffset (FIG. 38). As another example, among L0 and L1, the reference picture may be fixed to the one in front of (or behind) the current picture in the POC order.

Referring to FIG. 38, 1) if currPocDiffL0 is equal to currPocDiffL1, 2) otherwise Abs( currPocDiffL0) is greater than Abs( currPocDiffL1 ), 3) otherwise Abs( currPocDiffL0) is less than Abs( currPocDiffL1) , 4) mMvdLX derivation can be performed by distinguishing the remaining cases. That is, in the remaining cases, Abs(currPocDiffL0) and Abs(currPocDiffL1) are the same, but the signs of currPocDiffL0 and currPocDiffL1 are different (currPocDiffL0 = -currPocDiffL1). For example, in the case of 1), both mMvdL0 and mMvdL1 may be set to MmvdOffset and MV scaling may not be performed. In the case of 2), mMvdL0 can be set as MmvdOffset, and mMvdL1 can be set as “a value obtained by scaling mMvdL0”. In the case of 3), it is possible to set mMvdL1 to MmvdOffset, and to set mMvdL0 to “mMvdL1 scaling value”. In the case of 4), one of mMvdL0 and mMvdL1 may be set as MmvdOffset, and the other may be set as -MmvdOffset. In FIG. 38, mMvdL0 is set to MmvdOffset, and mMvdL1 is set to -MmvdOffset.

The four cases of FIG. 38 can be performed by changing the order. For example, 1) if currPocDiffL0 is equal to currPocDiffL1, 2) otherwise, if Abs( currPocDiffL0) and Abs( currPocDiffL1) are the same, 3) Abs( currPocDiffL0) is greater than Abs( currPocDiffL1 ), 4) remaining cases It can be performed by dividing it into. 2) of this embodiment may be the same as 4) of the previous reference embodiment of FIG. 38. 4) of this embodiment may be the same as 3) of the previous reference embodiment of FIG. 38.

When comparing the embodiment of FIG. 36 with the embodiment of FIG. 38, for example, the following differences may exist. In the example of FIG. 36, when -currPocDiffL0 * currPocDiffL1 is greater than 0, mMvdL0 is set to MmvdOffset, and mMvdL1 is set to -MmvdOffset. However, in the example of FIG. 38, when currPocDiffL0 = -currPocDiffL1, mMvdL0 is set to MmvdOffset, and mMvdL1 is set to -MmvdOffset. Therefore, the setting conditions are different. In addition, in the embodiment of FIG. 36, a process of setting MmvdOffset or -MmvdOffset is always included, but in the embodiment of FIG. 38, there may be a case where there is no process of setting MmvdOffset or -MmvdOffset. In this case, the mMvdLX can be set directly by MV scaling without a process of setting MmvdOffset or -MmvdOffset.

39 is a diagram illustrating motion information derivation according to an embodiment of the present invention.

According to an embodiment of the present invention, there may be a case where bi-prediction is not allowed. For example, there may be a case where bi-prediction is not allowed based on the block size. For example, bi-prediction may not be allowed for a block size smaller than the threshold. For example, bi-prediction may not be allowed for a 4x4 block. Also, if bi-prediction is not allowed, bi-prediction can be converted to uni-prediction. This may be to reduce the memory bandwidth. For example, since bi-prediction requires a large number of reference blocks, it may require more memory than uni-prediction.

Referring to FIG. 39, when both predFlagL0 and predFlagL1 are 1, it means that both L0 and L1 are used, and thus may mean bi-prediction. Therefore, in the case of bi-prediction, there may be a case where bi-prediction is not allowed based on cbWidth and cbHeight. 39 shows an example in which bi-prediction is not allowed for a 4x4 block. If bi-prediction is not allowed, predFlagL0 or predFlagL1 can be set to 0. Also, refIdxLX corresponding to predFlagLX set to 0 may be set to -1, and gbiIdx (a value related to the weight of bi-prediction) may be set to 0. 39 shows an example in which L1 is not used.

Disallowing bi-prediction as in the described embodiment can be applied to merge mode, inter mode (AMVP; mode with MVD), and the like. For example, when bi-prediction is not allowed, after constructing a candidate list, a bi-prediction candidate may be converted into a uni-prediction candidate through the same process as described in FIG. 39. For example, in the case of applying to merge mode, after constructing a merge candidate list, a bi-prediction candidate can be converted into a uni-prediction candidate.

40 is a diagram showing MVD derivation of MMVD according to an embodiment of the present invention.

In the examples of FIGS. 37 to 38, one of mMvdL0 and mMvdL1 is set as a signaled value, MmvdOffset, but the other is set as a modified value such as MV scaling. In addition, when determining one of mMvdL0 and mMvdL1, Abs (currPocDiffL0) or Abs (currPocDiffL1) was referred to. However, there may be a case where bi-prediction is not allowed as described in FIG. 39. Also at that time, either L0 or L1 can be converted to unused (converted to uni-prediction). Therefore, although mMvdLX is set to MmvdOffset, there may be a case where the mMvdLX is not used. At this time, mMvdLY other than mMvdLX may be used as MV scaling. Therefore, MmvdOffset could be directly signaled with a more accurate or high prediction performance value, but there may be cases where it cannot be utilized. 40 to 41 may be a method for solving this.

For example, if bi-prediction is not allowed, it can be converted to uni-prediction using “LX”. According to an embodiment of the present invention, when bi-prediction is not allowed, mMvdLX may be set to MmvdOffset. That is, when bi-prediction is not allowed, mMvdLX can be set as MmvdOffset regardless of the comparison between Abs (currPocDiffL0) and Abs (currPocDiffL1).

As described in FIG. 39, when bi-prediction is not allowed, it may be based on the block size.

40 shows an embodiment of converting to uni-prediction using “L0” when bi-prediction is not allowed. If bi-prediction is not allowed, mMvdL0 can be set as MmvdOffset ((8-307), (8-308)). In addition, mMvdL1 can be determined by MV scaling of mMvdL0. 40 shows a case where bi-prediction is not allowed for a 4x4 block. Also, this may be what is done if bi-prediction is not allowed, regardless of the comparison between Abs(currPocDiffL0) and Abs(currPocDiffL1). That is, even if Abs (currPocDiffL0) is smaller than Abs (currPocDiffL1) in the embodiment of FIG. 40, if bi-prediction is not allowed, mMvdL0 may be set as MmvdOffset.

40 shows an example of using L0 uni-prediction when bi-prediction is not allowed. If bi-prediction is not allowed in the example of using L1 uni-prediction, (8-311) to (8- Operation 318) may be performed. For this, the condition of 2) in the 1) if, 2) Otherwise if, 3) Otherwise structure of FIG. 40 may have to be satisfied until bi-prediction is allowed.

41 is a diagram illustrating motion information derivation according to an embodiment of the present invention.

The embodiment of FIG. 41 may be a method for solving the problem described with reference to FIGS. 39 to 40.

According to an embodiment of the present invention, when bi-prediction is not allowed, a reference list to be used when converting to uni-prediction may not be fixed. According to an embodiment, a reference list to be used may be determined based on the POC. For example, the reference list to be used can be determined based on currPocDiffL0 or currPocDiffL1. For example, it is possible to determine a reference list to be used based on Abs(currPocDiffL0) or Abs(currPocDiffL1). For example, it is possible to determine a reference list to be used based on the sizes of Abs (currPocDiffL0) and Abs (currPocDiffL1).

In an embodiment, a reference picture that is close to the current picture may have a large similarity of blocks, and thus a reference picture that is close to it may be used. For example, if Abs( currPocDiffL0)> Abs( currPocDiffL1) (or Abs( currPocDiffL0) >= Abs( currPocDiffL1 )) LX can be used, otherwise, LY instead of LX can be used. In an embodiment, LX may be L1, and LY may be L0.

Referring to FIG. 41, when bi-prediction is not allowed, it may be determined to use a reference list far from the current picture. If Abs(currPocDiffL0)> Abs(currPocDiffL1), predFlagL1 can be set to 0, otherwise, predFlagL0 can be set to 0. In this case, using MmvdOffset of mMvdL0 and mMvdL1 of FIGS. 37 to 38 as it is may correspond to a reference list used when changing bi-prediction to uni-prediction.

42 is a diagram showing MVD derivation of MMVD according to an embodiment of the present invention.

As described with reference to FIGS. 36 to 39, when the mMvdLX is determined, the MV scaling process may be performed, and there may be a case where bi-prediction is not allowed and thus converted into uni-prediction. Therefore, there may be a case of performing MV scaling that will not be used later.

For example, if bi-prediction is not allowed, it can be converted to uni-prediction using “LX”. According to an embodiment of the present invention, the process of setting “mMVDLX” by MV scaling may be performed only when bi-prediction is allowed in the process of deriveing mMvdL0 or mMVDL1.

For example, when bi-prediction is not allowed, when converting to uni-prediction using “L0”, the process of setting mMVDL1 by MV scaling can be performed only when bi-prediction is allowed. Or, for example, when bi-prediction is not allowed, when converting to uni-prediction using “L0”, the process of setting mMVDL1 by MV scaling of mMvdL0 only when bi-prediction is allowed can be performed.

42 shows a case of converting to uni-prediction using "L0" when bi-prediction is not allowed. In this case, setting mMVDL1 by MV scaling can be redundant. Therefore, only when bi-prediction is allowed, the MV scaling process for setting mMVDL1 such as (8-309) and (8-310) can be performed. (8-303) to (8-306) Also, since it is a process for MV scaling, it may be performed only when bi-prediction is allowed. That is, (8-303) to (8-306) can be put within the conditions indicated in the dark letters of FIG.

As described in FIG. 39, when bi-prediction is not allowed, it may be based on the block size. In FIG. 42, a condition in which cbWidth or cbHeight is not 4 is shown, but when the possible cbWidth or cbHeight is 4 or more, cbWidth or cbHeight may be expressed as a condition greater than 4.

Unlike FIG. 42, when bi-prediction is not allowed, the same method as described in the example using “L1” can be applied. For example, in this case, if bi-prediction is not allowed, MV scaling to set mMvdL0, that is, (8-317), (8-318), (8-311) to (8-314) may not be performed. have.

It is also possible to perform the embodiments described with reference to FIGS. 38, 40, 41, and 42 together.

In the above, the present invention has been described through specific embodiments, but those skilled in the art can make modifications and changes without departing from the spirit and scope of the present invention. Therefore, what can be easily inferred by a person belonging to the technical field to which the present invention pertains from the detailed description and examples of the present invention is interpreted as belonging to the scope of the present invention.

100: encoding device 200: decoding device

Claims (1)

Video signal processing apparatus and method.

Images