KR20200110235A - Method and apparatus for deriving delta motion vector - Google Patents

Abstract

Disclosed are a method for deriving a delta motion vector and an image decoding device. According to one embodiment of the present invention, provided is a method for deriving a delta motion vector used for decoder-side motion vector refinement (DMVR). The method comprises the steps of: acquiring, from a bitstream, an application flag related to whether the DMVR is applied, a motion vector of a current sub-block, and a reference picture of the current sub-block; deriving the delta motion vector in integer units from a position indicated by the motion vector using an integer sample offset indicating an integer sample having a minimum sum of absolute differences (SAD) of candidate integer samples corresponding to the current sub-block; and refining, in fractional units, the delta motion vector derived in integer units by using a fractional sample offset derived from the candidate integer samples wherein whether the refining step is performed is determined according to a value of the application flag. According to the present invention, complexity of a decoding process can be reduced.

Description

A method for deriving a delta motion vector and an image decoding apparatus {METHOD AND APPARATUS FOR DERIVING DELTA MOTION VECTOR}

The present invention relates to encoding and decoding of an image, and more particularly, to a delta motion vector derivation method and an image decoding apparatus in which the efficiency of encoding and decoding is improved by organically controlling the complexity of the process of deriving the delta motion vector. .

Since moving picture data has a large amount of data compared to audio data or still image data, it requires a lot of hardware resources including memory in order to store or transmit itself without processing for compression.

Accordingly, when moving or transmitting moving picture data, the moving picture data is compressed and stored or transmitted using an encoder, and the decoder receives the compressed moving picture data, decompresses and reproduces the compressed moving picture data. As such video compression technologies, there are H.264/AVC and HEVC (High Efficiency Video Coding), which improves coding efficiency by about 40% compared to H.264/AVC.

However, the size, resolution, and frame rate of an image are gradually increasing, and accordingly, the amount of data to be encoded is also increasing. Accordingly, a new compression technique with higher encoding efficiency and higher quality improvement effect than the existing compression technique is required.

In order to meet these demands, the present invention aims to provide an improved video encoding and decoding technology, and in particular, one aspect of the present invention is to perform the entire process or partial process of the DMVR as indicated by the application flag. It relates to a technique for improving the efficiency of encoding and decoding by selectively determining whether or not.

An aspect of the present invention is a method of deriving a delta motion vector used for decoder-side motion vector refinement (DMVR), comprising, from a bitstream, an application flag related to whether or not the DMVR is applied, and a current subblock motion Obtaining a vector and a reference picture of the current subblock; From the position indicated by the motion vector, the delta motion vector is calculated using an integer sample offset indicating an integer sample having a minimum sum of absolute differences (SAD) among candidate integer samples corresponding to the current subblock. Deriving in integer units; And refinement of the delta motion vector derived in integer units in decimal units by using the decimal sample offset derived from the candidate integer samples, wherein the improving comprises a value of the applied flag A method of deriving a delta motion vector is provided, in which the performance is determined according to the method.

Another aspect of the present invention is an image decoding apparatus that induces a delta motion vector used for decoder-side motion vector refinement (DMVR), from a bitstream, an application flag related to whether or not the DMVR is applied, a current sub An acquisition unit acquiring a motion vector of a block and a reference picture of the current subblock; From the position indicated by the motion vector, the delta motion vector is calculated using an integer sample offset indicating an integer sample having a minimum sum of absolute differences (SAD) among candidate integer samples corresponding to the current subblock. An induction unit for guiding in integer units; And an improvement unit for refinement of the delta motion vector derived in the integer unit by the decimal unit by using the decimal sample offset derived from the candidate integer samples, wherein the improvement unit includes: It provides an apparatus for decoding an image, which determines whether to perform the fractional improvement.

As described above, according to an embodiment of the present invention, since it is possible to selectively determine whether to perform all or part of the DMVR process, it is possible to improve efficiency by reducing the complexity of the decoding process.

1 is an exemplary block diagram of an image encoding apparatus capable of implementing the techniques of the present disclosure.
2 is a diagram for explaining a method of dividing a block using a QTBTTT structure.
3A is a diagram illustrating a plurality of intra prediction modes.
3B is a diagram illustrating a plurality of intra prediction modes including wide-angle intra prediction modes.
4 is an exemplary block diagram of an image decoding apparatus capable of implementing the techniques of the present disclosure.
5 is a diagram for describing RDPCM.
6 is a diagram for describing a DMVR.
7 is an exemplary block diagram of an inter prediction unit capable of implementing the techniques of this disclosure.
8 is a flowchart illustrating an example of a method of inducing a delta motion vector.
9 is a diagram for explaining candidate integer samples.
10 is a flowchart illustrating an example of a method of determining whether to perform integer unit improvement.
11 is a flowchart illustrating an example of a method of determining whether to interpolate candidate prime samples.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding identification codes to elements of each drawing, it should be noted that the same elements have the same symbols as possible even if they are indicated on different drawings. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

1 is an exemplary block diagram of an image encoding apparatus capable of implementing the techniques of the present disclosure. Hereinafter, an image encoding apparatus and sub-elements of the apparatus will be described with reference to FIG. 1.

The image encoding apparatus includes a picture segmentation unit 110, a prediction unit 120, a subtractor 130, a transform unit 140, a quantization unit 145, a rearrangement unit 150, an entropy encoding unit 155, an inverse quantization unit. (160), an inverse transform unit 165, an adder 170, a filter unit 180, and a memory 190 may be included.

Each component of the image encoding apparatus may be implemented by hardware or software, or by a combination of hardware and software. In addition, functions of each component may be implemented as software, and a microprocessor may be implemented to execute a function of software corresponding to each component.

One image (video) is composed of a plurality of pictures. Each picture is divided into a plurality of regions, and encoding is performed for each region. For example, one picture is divided into one or more tiles or/and slices. Here, one or more tiles may be defined as a tile group. Each tile or/slice is divided into one or more Coding Tree Units (CTUs). And each CTU is divided into one or more CUs (Coding Units) by a tree structure. Information applied to each CU is encoded as the syntax of the CU, and information commonly applied to CUs included in one CTU is encoded as the syntax of the CTU. In addition, information commonly applied to all blocks in one slice is encoded as the syntax of the slice header, and information applied to all blocks constituting one picture is a picture parameter set (PPS) or picture. It is coded in the header. Further, information commonly referred to by a plurality of pictures is encoded in a sequence parameter set (SPS). In addition, information commonly referred to by one or more SPSs is encoded in a video parameter set (VPS). Also, information commonly applied to one tile or tile group may be encoded as syntax of a tile or tile group header.

The picture dividing unit 110 determines the size of a coding tree unit (CTU). Information on the size of the CTU (CTU size) is encoded as the syntax of the SPS or PPS and transmitted to the video decoding apparatus.

After dividing each picture constituting an image into a plurality of CTUs (Coding Tree Units) having a predetermined size, the picture dividing unit 110 repetitively divides the CTU using a tree structure. (recursively) split. A leaf node in the tree structure becomes a coding unit (CU), which is a basic unit of coding.

As a tree structure, a quad tree (QuadTree, QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of the same size, or a binary tree (BinaryTree) in which an upper node is divided into two lower nodes. , BT), or a ternary tree (TT) in which an upper node is divided into three lower nodes in a 1:2:1 ratio, or a structure in which two or more of these QT structures, BT structures, and TT structures are mixed. have. For example, a QTBT (QuadTree plus BinaryTree) structure may be used, or a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure may be used. Here, by combining BTTT, it may be referred to as MTT (Multiple-Type Tree).

2 shows a QTBTTT split tree structure. As shown in FIG. 2, the CTU may be first divided into a QT structure. The quadtree division may be repeated until the size of a splitting block reaches the minimum block size (MinQTSize) of a leaf node allowed in QT. A first flag (QT_split_flag) indicating whether each node of the QT structure is divided into four nodes of a lower layer is encoded by the entropy encoder 155 and signaled to the image decoding apparatus. If the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in BT, it may be further divided into one or more of a BT structure or a TT structure. In the BT structure and/or the TT structure, a plurality of division directions may exist. For example, there may be two directions in which a block of a corresponding node is divided horizontally and a direction vertically divided. As shown in FIG. 2, when MTT splitting starts, a second flag (mtt_split_flag) indicating whether nodes are split, and if split, a flag indicating a split direction (vertical or horizontal) and/or a split type (Binary or Ternary). A flag indicating) is encoded by the entropy encoder 155 and signaled to the image decoding apparatus. Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is divided into four nodes of a lower layer, a CU split flag (split_cu_flag) indicating whether the node is divided is encoded. It could be. When it is indicated that the value of the CU split flag (split_cu_flag) is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a coding unit (CU), which is a basic unit of encoding. When indicating that the value of the CU split flag (split_cu_flag) is to be split, the video encoding apparatus starts encoding from the first flag in the above-described manner.

When QTBT is used as another example of the tree structure, two types of horizontally splitting a block of the corresponding node into two blocks of the same size (i.e., symmetric horizontal splitting) and a type splitting vertically (i.e., symmetric vertical splitting) Branches can exist. A split flag indicating whether each node of the BT structure is divided into blocks of a lower layer and split type information indicating a type to be divided are encoded by the entropy encoder 155 and transmitted to the image decoding apparatus. Meanwhile, a type of dividing the block of the corresponding node into two blocks having an asymmetric shape may further exist. The asymmetric form may include a form of dividing a block of a corresponding node into two rectangular blocks having a size ratio of 1:3, or a form of dividing a block of a corresponding node in a diagonal direction.

The CU can have various sizes according to the QTBT or QTBTTT split from the CTU. Hereinafter, a block corresponding to a CU to be encoded or decoded (ie, a leaf node of QTBTTT) is referred to as a'current block'. According to the adoption of the QTBTTT division, the shape of the current block may be not only square but also rectangular.

The prediction unit 120 predicts the current block and generates a prediction block. The prediction unit 120 includes an intra prediction unit 122 and an inter prediction unit 124.

In general, each of the current blocks in a picture can be predictively coded. In general, prediction of the current block is performed using an intra prediction technique (using data from a picture containing the current block) or an inter prediction technique (using data from a picture coded before a picture containing the current block). Can be done. Inter prediction includes both one-way prediction and two-way prediction.

The intra prediction unit 122 predicts pixels in the current block by using pixels (reference pixels) located around the current block in the current picture including the current block. There are a plurality of intra prediction modes according to the prediction direction. For example, as shown in FIG. 3A, the plurality of intra prediction modes may include two non-directional modes including a planar mode and a DC mode, and 65 directional modes. Depending on each prediction mode, the surrounding pixels to be used and the calculation expression are defined differently.

For efficient directional prediction for the rectangular-shaped current block, directional modes (67 to 80, intra prediction modes -1 to -14) shown by dotted arrows in FIG. 3B may be additionally used. These may be referred to as "wide angle intra-prediction modes". Arrows in FIG. 3B indicate corresponding reference samples used for prediction, and do not indicate a prediction direction. The prediction direction is opposite to the direction indicated by the arrow. In the wide-angle intra prediction modes, when the current block is a rectangular shape, a specific directional mode is predicted in the opposite direction without additional bit transmission. In this case, among the wide-angle intra prediction modes, some wide-angle intra prediction modes available for the current block may be determined based on a ratio of the width and height of the rectangular current block. For example, wide-angle intra prediction modes with an angle less than 45 degrees (intra prediction modes 67 to 80) can be used when the current block has a rectangular shape with a height smaller than the width, and wide-angle with an angle greater than -135 degrees. The intra prediction modes (intra prediction modes -1 to -14) can be used when the current block has a rectangular shape whose height is greater than the width.

The intra prediction unit 122 may determine an intra prediction mode to be used to encode the current block. In some examples, the intra prediction unit 122 may encode the current block using several intra prediction modes and select an appropriate intra prediction mode to use from the tested modes. For example, the intra prediction unit 122 calculates rate distortion values using rate-distortion analysis for several tested intra prediction modes, and has the best rate distortion characteristics among the tested modes. It is also possible to select an intra prediction mode.

The intra prediction unit 122 selects one intra prediction mode from among a plurality of intra prediction modes, and predicts the current block using a neighboring pixel (reference pixel) determined according to the selected intra prediction mode and an equation. Information on the selected intra prediction mode is encoded by the entropy encoder 155 and transmitted to the image decoding apparatus.

The inter prediction unit 124 generates a prediction block for the current block through a motion compensation process. The inter prediction unit 124 searches for a block most similar to the current block in the coded and decoded reference picture prior to the current picture, and generates a prediction block for the current block using the searched block. Then, a motion vector corresponding to a displacement between the current block in the current picture and the prediction block in the reference picture is generated. In general, motion estimation is performed on a luma component, and a motion vector calculated based on the luma component is used for both the luma component and the chroma component. Motion information including information on a reference picture used to predict the current block and information on a motion vector is encoded by the entropy encoder 155 and transmitted to an image decoding apparatus.

The subtractor 130 generates a residual block by subtracting the prediction block generated by the intra prediction unit 122 or the inter prediction unit 124 from the current block.

The transform unit 140 divides the residual block into one or more transform blocks, applies the transform to one or more transform blocks, and transforms residual values of the transform blocks from the pixel domain to the frequency domain. In the frequency domain, transformed blocks are referred to as coefficient blocks comprising one or more transform coefficient values. A 2D transformation kernel may be used for transformation, and a 1D transformation kernel may be used for horizontal and vertical transformation respectively. The transform kernel may be based on discrete cosine transform (DCT), discrete sine transform (DST), or the like.

The transform unit 140 may transform residual signals in the residual block by using the entire size of the residual block as a transform unit. In addition, the transform unit 140 may divide the residual block into two sub-blocks in a horizontal or vertical direction, and may perform transformation on only one of the two sub-blocks. Accordingly, the size of the transform block may be different from the size of the residual block (and thus the size of the prediction block). Non-zero residual sample values may not exist or may be very sparse in a subblock on which transformation is not performed. The residual samples of the subblock on which the transformation is not performed are not signaled, and may be regarded as "0" by the image decoding apparatus. There can be several partition types depending on the partitioning direction and partitioning ratio. The transform unit 140 includes information on the coding mode (or transform mode) of the residual block (e.g., information indicating whether the residual block is transformed or the residual subblock is transformed, and a partition type selected to divide the residual block into subblocks) The entropy encoding unit 155 may be provided with information indicating information and information identifying a subblock on which transformation is performed. The entropy encoder 155 may encode information about a coding mode (or transform mode) of the residual block.

The quantization unit 145 quantizes the transform coefficients output from the transform unit 140 and outputs the quantized transform coefficients to the entropy encoding unit 155. The quantization unit 145 may immediately quantize a related residual block for a certain block or frame without transformation.

The rearrangement unit 150 may rearrange coefficient values on the quantized residual values. The rearrangement unit 150 may change a two-dimensional coefficient array into a one-dimensional coefficient sequence through coefficient scanning. For example, the rearrangement unit 150 may scan from a DC coefficient to a coefficient in a high frequency region using a zig-zag scan or a diagonal scan to output a one-dimensional coefficient sequence. . Depending on the size of the transform unit and the intra prediction mode, instead of zig-zag scan, a vertical scan that scans a two-dimensional coefficient array in a column direction or a horizontal scan that scans a two-dimensional block shape coefficient in a row direction may be used. That is, a scan method to be used may be determined from among zig-zag scan, diagonal scan, vertical scan, and horizontal scan according to the size of the transform unit and the intra prediction mode.

The entropy encoding unit 155 uses various encoding methods such as Context-based Adaptive Binary Arithmetic Code (CABAC), Exponential Golomb, and the like, and the quantized transform coefficients of 1D output from the reordering unit 150 are A bitstream is generated by encoding the sequence.

In addition, the entropy encoder 155 encodes information such as a CTU size related to block division, a CU division flag, a QT division flag, an MTT division type, and an MTT division direction, so that the video decoding apparatus performs the same block as the video encoding apparatus. Make it possible to divide. In addition, the entropy encoder 155 encodes information on a prediction type indicating whether the current block is encoded by intra prediction or inter prediction, and intra prediction information (ie, intra prediction) according to the prediction type. Mode information) or inter prediction information (reference picture and motion vector information) is encoded.

The inverse quantization unit 160 inverse quantizes the quantized transform coefficients output from the quantization unit 145 to generate transform coefficients. The inverse transform unit 165 converts transform coefficients output from the inverse quantization unit 160 from the frequency domain to the spatial domain to restore the residual block.

The addition unit 170 restores the current block by adding the restored residual block and the prediction block generated by the prediction unit 120. The pixels in the reconstructed current block are used as reference pixels when intra-predicting the next block.

The filter unit 180 filters reconstructed pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transformation/quantization. Perform. The filter unit 180 may include a deblocking filter 182 and a sample adaptive offset (SAO) filter 184.

The deblocking filter 180 filters the boundary between reconstructed blocks to remove blocking artifacts caused by block-based encoding/decoding, and the SAO filter 184 adds additional information to the deblocking-filtered image. Filtering is performed. The SAO filter 184 is a filter used to compensate for a difference between a reconstructed pixel and an original pixel caused by lossy coding.

The reconstructed block filtered through the deblocking filter 182 and the SAO filter 184 is stored in the memory 190. When all blocks in one picture are reconstructed, the reconstructed picture may be used as a reference picture for inter prediction of a block in a picture to be encoded later.

4 is an exemplary block diagram of an image decoding apparatus capable of implementing the techniques of the present disclosure. Hereinafter, an image decoding apparatus and sub-components of the apparatus will be described with reference to FIG. 4.

The image decoding apparatus includes an entropy decoding unit 410, a rearrangement unit 415, an inverse quantization unit 420, an inverse transform unit 430, a prediction unit 440, an adder 450, a filter unit 460, and a memory 470. ) Can be included.

Like the image encoding apparatus of FIG. 1, each component of the image decoding apparatus may be implemented as hardware or software, or may be implemented as a combination of hardware and software. In addition, functions of each component may be implemented as software, and a microprocessor may be implemented to execute a function of software corresponding to each component.

The entropy decoding unit 410 determines the current block to be decoded by decoding the bitstream generated by the image encoding apparatus and extracting information related to block division, and predicting information and residual signals necessary to restore the current block. Extract information, etc.

The entropy decoding unit 410 determines the size of the CTU by extracting information on the CTU size from a sequence parameter set (SPS) or a picture parameter set (PPS), and divides the picture into CTUs of the determined size. Then, the CTU is determined as the uppermost layer of the tree structure, that is, the root node, and the CTU is divided using the tree structure by extracting partition information for the CTU.

For example, in the case of splitting the CTU using the QTBTTT structure, first, a first flag (QT_split_flag) related to the splitting of the QT is extracted and each node is split into four nodes of a lower layer. And, for the node corresponding to the leaf node of QT, the second flag (MTT_split_flag) related to the splitting of the MTT and the splitting direction (vertical / horizontal) and/or split type (binary / ternary) information are extracted and the corresponding leaf node is MTT. Divide into structure. Through this, each node below the leaf node of the QT is recursively divided into a BT or TT structure.

As another example, when splitting the CTU using the QTBTTT structure, first extract the CU split flag (split_cu_flag) indicating whether to split the CU, and if the corresponding block is split, the first flag (QT_split_flag) is extracted. May be. In the segmentation process, each node may have 0 or more repetitive MTT segmentation after 0 or more repetitive QT segmentation. For example, in the CTU, MTT division may occur immediately, or, conversely, only multiple QT divisions may occur.

As another example, when the CTU is divided using the QTBT structure, each node is divided into four nodes of a lower layer by extracting the first flag (QT_split_flag) related to the division of the QT. In addition, a split flag indicating whether or not the node corresponding to the leaf node of the QT is further split into BT and split direction information are extracted.

Meanwhile, when determining the current block to be decoded through the division of the tree structure, the entropy decoder 410 extracts information on a prediction type indicating whether the current block is intra prediction or inter prediction. When the prediction type information indicates intra prediction, the entropy decoder 410 extracts a syntax element for intra prediction information (intra prediction mode) of the current block. When the prediction type information indicates inter prediction, the entropy decoder 410 extracts a syntax element for the inter prediction information, that is, information indicating a motion vector and a reference picture referenced by the motion vector.

On the other hand, the entropy decoder 410 includes information on the coding mode of the residual block (e.g., information on whether the residual block is encoded or only the subblocks of the residual block are encoded, and is selected to divide the residual block into subblocks). Information indicating the partition type, information identifying the encoded residual subblock, quantization parameters, etc.) are extracted from the bitstream. In addition, the entropy decoder 410 extracts information on quantized transform coefficients of the current block as information on the residual signal.

The rearrangement unit 415, in the reverse order of the coefficient scanning order performed by the image encoding apparatus, reconverts the sequence of one-dimensional quantized transform coefficients entropy-decoded by the entropy decoder 410 into a two-dimensional coefficient array (i.e., Block).

The inverse quantization unit 420 inverse quantizes the quantized transform coefficients, and the inverse transform unit 430 inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain based on information on the coding mode of the residual block By reconstructing the signals, a reconstructed residual block for the current block is generated.

When the information on the coding mode of the residual block indicates that the residual block of the current block is coded in the image encoding apparatus, the inverse transform unit 430 determines the size of the current block (and thus, to be reconstructed) with respect to the inverse quantized transformation coefficients. A reconstructed residual block for the current block is generated by performing inverse transformation using the residual block size) as a transformation unit.

In addition, when the information on the coding mode of the residual block indicates that only one subblock of the residual block is coded in the image encoding apparatus, the inverse transform unit 430 performs the transformed sub-blocks on the inverse quantized transform coefficients. By using the size of the block as a transformation unit, performing inverse transformation to restore residual signals for the transformed subblock, and filling the residual signals for untransformed subblocks with a value of "0", the reconstructed current block Create a residual block.

The prediction unit 440 may include an intra prediction unit 442 and an inter prediction unit 444. The intra prediction unit 442 is activated when the prediction type of the current block is intra prediction, and the inter prediction unit 444 is activated when the prediction type of the current block is inter prediction.

The intra prediction unit 442 determines an intra prediction mode of the current block among a plurality of intra prediction modes from the syntax element for the intra prediction mode extracted from the entropy decoding unit 410, and references around the current block according to the intra prediction mode. Predict the current block using pixels.

The inter prediction unit 444 determines a motion vector of the current block and a reference picture referenced by the motion vector using the syntax element for the intra prediction mode extracted from the entropy decoding unit 410, and determines the motion vector and the reference picture. Is used to predict the current block.

The adder 450 adds the residual block output from the inverse transform unit 430 and the prediction block output from the inter prediction unit 444 or the intra prediction unit 442 to restore the current block. The pixels in the reconstructed current block are used as reference pixels for intra prediction of a block to be decoded later.

The filter unit 460 may include a deblocking filter 462 and an SAO filter 464. The deblocking filter 462 performs deblocking filtering on the boundary between reconstructed blocks in order to remove blocking artifacts caused by decoding in units of blocks. The SAO filter 464 performs additional filtering on the reconstructed block after deblocking filtering in order to compensate for the difference between the reconstructed pixel and the original pixel caused by lossy coding. The reconstructed block filtered through the deblocking filter 462 and the SAO filter 464 is stored in the memory 470. When all blocks in one picture are reconstructed, the reconstructed picture is used as a reference picture for inter prediction of a block in a picture to be encoded later.

In the discussion of the next-generation video coding standard (VVC; Versatile Video Coding), several new coding tools have been introduced that enable better coding performance compared to the High Efficiency Video Coding (HEVC) standard.

MTS (multiple transform set or multiple transform selection) is a method of performing transformation on residual samples by applying a transformation kernel (transform type) in each of the horizontal and vertical directions. The TS mode (transform skip mode) is a mode in which quantization and entropy coding are performed at the pixel level without applying transformation to residual samples.

MTS and TS modes have a relationship with each other in the conversion process of residual samples, and new ideas are proposed for this relationship in the process of VVC standardization discussion. Table 1 shows a comparison between one of the new ideas for the MTS and TS mode with a conventional VVC draft.

The conventional VVC draft defines syntax elements for MTS in transform_unit() syntax. The maximum size of a transform block to which MTS can be applied is 32×32, and when this condition is satisfied, a syntax element tu_mts_flag indicating whether MTS is applied to the corresponding transform block is signaled. Syntax elements for the TS mode are defined in the residual_coding() syntax. The maximum size of a transform block to which the TS mode can be applied is 4×4, and when this condition is satisfied, transform_skip_flag, which is a syntax element indicating whether transform is skipped, is signaled in the corresponding transform block. If MTS is applied to the corresponding transform block (tu_mts_flag = 1) and the corresponding transform block is a luma transform block (cIdx = 0), mts_idx is signaled. mts_idx is an index indicating a transform kernel applied to residual samples along the horizontal and vertical directions of the corresponding transform block.

The new idea is to unify the MTS and TS modes in the transform_unit() syntax using tu_mts_idx. In addition, the new idea defines that the TS mode, which was applied to a transform block having a maximum 4x4 size, can be extended to a transform block having a maximum of 32x32 and applied. This is to prevent duplication of syntax coding with MTS.

Table 2 shows an example of selecting the MTS and TS modes using tu_mts_idx.

As shown in Table 2, tu_mts_idx can have a value of 0-5, and values 0-5 are binarized using truncated unary binarization. Any one of the MTS and TS modes may be selected by each value of tu_mts_idx, and transform kernels to be applied to the transform block may be designated.

The new idea was reflected in the revised VVC draft, an example of which is shown in Table 3.

As shown in Table 3, it can be seen that tu_mts_idx representing MTS enabled in Table 2 and transform_skip_flag representing TS mode are reflected in the VVC draft.

Meanwhile, RDPCM (residual differential pulse-code modulation) is a method of improving lossless compression performance by additionally predicting residual samples derived after intra prediction. RDPCM additionally predicts residual samples in a vertical direction or a horizontal direction from the nearest pixel (sample).

In the HEVC extension, RDPCM is performed on residual samples after intra-prediction and inter-prediction, when lossy compression is performed in TS mode. In the TS mode, since transformation is not performed on the residual samples and entropy encoding is directly applied, it can be said that the encoding performance of RDPCM is not excellent compared to DCT. However, for specific images (eg, screen content), RDPCM has a high encoding performance, so it can be usefully used for compression. This is because there are a lot of residual samples in the high frequency range that occur at the boundary of graphic elements with high color contrast in this specific image. Accordingly, RDPCM can provide superior compression performance by reducing the total amount of energy of residual samples for entropy encoding in the TS mode.

There are two types of RDPCM: an implicit RDPCM method in which prediction is performed in a horizontal direction and a vertical direction after intra prediction, and an explicit method in which prediction is performed in a horizontal direction or a vertical direction after inter prediction. In the explicit method, information on the prediction direction of the RDPCM is signaled to the image decoding apparatus through a bitstream.

In the case of lossy compression, an example of an RDPCM for a 4×4 residual block is shown in FIG. 5. FIG. 5(a) shows the horizontal direction prediction of RDPCM, and FIG. 5(b) shows the vertical direction prediction of RDPCM.

가 수학식 1과 같이 표현될 수 있다.As shown in FIG. 5, RDPCM is performed using the residual components of the nearest left column or upper row according to the prediction direction among residual samples. When vertical prediction is performed for the residual signal r _i,j in the block of size N×N, the second residual signal is the result after prediction

May be expressed as in Equation 1.

In Equation 1, Q(r) is a reconstructed residual signal including quantization noise. In the case of RDPCM in the vertical direction, the image encoding apparatus entropy-encodes the second residual signal, then signals it to the image decoding apparatus, and restores (stores) the second residual signal to predict the residual signal of the next row. RDPCM in the vertical direction sequentially proceeds for all rows in the residual block, and the image decoding apparatus restores the residual signal of the i-th row by sequentially summing the reconstructed secondary residual signals as shown in Equation (2).

In the case of implicit RDPCM, the prediction direction information of RDPCM is estimated from intra prediction information decoded in advance, but in the case of explicit RDPCM, RDPCM prediction mode information is decoded from the bitstream and determined.

Meanwhile, a decoder-side motion vector derivation (DMVR) is a method of improving the accuracy of a motion vector used in a merge mode by refinement of a motion vector (MV) in an image decoding apparatus.

An example of DMVR is shown in FIG. 6. First, the video decoding apparatus decodes motion information from a bitstream, and derives an initial MV (MV ₀ , MV ₁ ) for a current subblock (currSb) in a current picture (currPic) based on the decoded motion information.

The video decoding apparatus configures samples (candidate samples) in a prediction block corresponding to a current subblock based on a location within a reference picture (refPicL0, refPicL1) indicated by the initial MV. Candidate samples include integer samples (candidate integer samples) and decimal samples (candidate decimal samples).

When the configuration of the candidate samples is completed, the video decoding apparatus determines a candidate sample having a minimum sum of absolute differences (SAD) by searching around the initial MV. When the candidate sample with the minimum SAD is determined, a delta motion vector (MV _diff ) representing the displacement between the position indicated by the initial MV and the position of the searched candidate sample is derived, and'improvements' using the initial MV and the delta MV The MV (MV ₀ ', MV ₁ ')' is finally derived.

To explain in more detail the process of determining the candidate sample having the minimum SAD, a list (sadList[i]) of candidate integer samples of a preset number and position is set, and the delta MV is calculated using this list. After the corresponding dMvLx is derived in integer units, dMvLx derived in integer units is improved in decimal units. A specific condition in which the fractional unit improvement is performed is when the candidate sample (bestIdx) having the smallest SAD in the derivation of the integer unit corresponds to a position indicated by the initial MV (sadList[4]).

However, even in the case of an image that does not generally use sub-pels, such as screen content, if the fractional improvement is unconditionally performed, decoding complexity may increase, which may act as a major cause of a bottleneck in the image decoding apparatus.

The present invention proposes a new method for selectively performing the fractional improvement according to the characteristics of an image based on the recognition of a problem in which the improvement of the fractional unit is unconditionally performed in the DMVR.

An exemplary block diagram of an inter prediction unit 444 capable of implementing the techniques of the present disclosure is shown in FIG. 7. As shown in FIG. 7, the inter prediction unit 444 includes an acquisition unit 710, a configuration unit 720, a derivation unit 730, an improvement unit 740, a derivation unit 750, and a prediction execution unit 760. It can be configured to include. In addition, the configuration unit 720 may be configured to include an integer sample configuration unit 722 and an interpolation unit 724, and the derivation unit 730 includes a determination unit 732 and an integer unit derivation unit 734. Can be. Hereinafter, functions of each component will be described with reference to FIGS. 8 to 11.

The apparatus for encoding an image may perform inter prediction on a current subblock, encode motion information on a prediction block of the current subblock, and signal. In addition, the image encoding apparatus may determine the necessity of improvement in a fractional unit, and set the determination result as a value of an application flag to signal.

The entropy decoder 410 may decode the applied flag and motion information of the current subblock from the bitstream. The acquisition unit 710 may acquire a motion vector of the current subblock and a reference picture of the current subblock based on the motion information decoded from the bitstream (S810). As a result, the acquisition unit 710 may acquire the applied flag, the motion vector of the current subblock, and the reference picture of the current subblock from the bitstream. The motion vector acquired by the acquisition unit 710 may correspond to an initial MV used for the DMVR, and the application flag may be a syntax element indicating information related to whether or not the DMVR is applied.

When the initial MV is obtained, a process of deriving the delta MV into an integer unit (S820 to S840) and a process of improving the delta MV derived by an integer unit into a decimal unit (S860) may be performed. The process of deriving the delta MV in an integer unit (S820 to S840) is a process of constructing prediction samples corresponding to the current subblock (S820), a process of determining an integer sample having a minimum SAD from among the prediction samples (S830), and , A process of deriving the delta MV in integer units by using the integer sample offset derived from the determined integer sample (S840).

First, the configuration unit 720 may configure samples (prediction samples corresponding to the current subblock) for the prediction block of the current subblock, centering on the position indicated by the initial MV in the reference picture. (S820). The prediction samples may include integer samples (candidate integer samples) and/or decimal samples (candidate decimal samples).

When the configuration of the prediction samples is completed, the determiner 732 may search for candidate integer samples located at a preset position among the prediction samples and determine an integer sample having the minimum SAD (S830). The determiner 732 may search all candidate integer samples or search only some of the candidate integer samples in order to derive an integer sample having a minimum SAD.

When the integer sample having the minimum SAD is determined, the integer unit derivation unit 734 may derive the delta MV in integer units by using the integer sample offset (S840). The integer sample offset may be a displacement between the position in the reference picture indicated by the initial MV and the position of the integer sample having the minimum SAD. For example, the integer sample offset may indicate an integer sample having a minimum SAD from a position in a reference picture indicated by an initial MV. When the integer sample offset (after resolution adjustment) is added to the position indicated by the initial MV, the delta MV can be derived in integer units.

When the derivation of the integer unit is completed, a process of improving the delta MV to a decimal unit (S860) may be performed. However, as described above, since the present invention is configured to selectively determine whether to perform fractional improvement, the process of determining whether to perform fractional improvement may be performed first.

The enhancement unit 740 may determine whether to perform fractional enhancement based on the application flag decoded from the bitstream (S850).

If it is determined that the fractional unit improvement is to be performed, the improvement unit 740 may improve the delta MV derived in the integer unit in the decimal unit by using the decimal sample offset derived from the candidate integer samples (S860). In contrast, if it is determined that the improvement in decimal units is not performed, the delta MV itself derived in integer units may be used in the process of deriving the improved MV (S870).

The derivation unit 750 may derive the improved MV from the initial MV and the delta MV (S870). For example, the improved MV can be derived by summing the initial MV and the delta MV. Here, the delta MV may be a delta MV to which both integer unit derivation and decimal unit improvement are applied, or delta MV to which only integer unit derivation is applied.

When the improved MV is derived, the prediction execution unit 760 may predict the current subblock based on the improved MV (S880). That is, the prediction execution unit 760 may generate or induce a prediction block for the current subblock.

Candidate integer samples

As described above, candidate integer samples used for the DMVR may be placed at a preset position in the reference picture.

The positions where candidate integer samples are located may be preset based on the position indicated by the initial MV. For example, as shown in FIG. 9, when the initial MV indicates a position indicated by C (center) in the reference picture, candidate integer samples are at position C, left, right, upper and lower, centering on position C. Can be seated.

In addition, a position where candidate integer samples are located may be included within a preset search range. For example, if the search range is set to 2, a total of 25 integer samples including 2 columns to the left, 2 columns to the right, 2 rows to the top, and 2 rows to the bottom centered at the C position are included in the candidate integer samples. May be applicable.

Integer Sample Search

As described above, the determiner 732 may search for candidate integer samples to determine an integer sample having a minimum SAD among candidate integer samples. In addition, the determiner 732 may search all of the candidate integer samples or only a part of the candidate integer samples in order to derive an integer sample having a minimum SAD.

In the case of searching all candidate integer samples, the determination unit 732 calculates the SAD of the C position, and if the calculated SAD is less than the threshold value, determines the C position as an integer sample having the minimum SAD, and searches for integer samples. You can end the step. Alternatively, if the SAD of the C position is greater than or equal to the threshold value, the determiner 732 may calculate the SAD of the remaining 24 candidate integer samples and derive an integer sample having the smallest SAD among them.

In the case of searching only some of the candidate integer samples, the determination unit 732 calculates the SAD of the C position, the P1 position, the P2 position, the P3 position and the P4 position, and when the SAD of the C position is the minimum, the C position is It is determined as an integer sample having a minimum SAD, and the integer sample search step can be finished (step 1). In contrast, when the SAD of the C position is not the minimum, the determination unit 732 sets the position having the smallest SAD among the P1 position, P2 position, P3 position, P4 position, and P5 position as the new C position. The first step may be performed once more (step 2).

Whether to perform the improvement process in decimal units-applied flag

As shown in FIG. 10, the improvement unit 740 may determine whether to perform the improvement in decimal units based on the application flag.

The application flag directly indicates whether or not fractional improvement is applied or performed, or the current subblock or the upper region (block, tile, slice, picture, sequence, etc.) containing the current subblock is generally used as a sub-pel. It is also possible to indicate whether it is included in a specific image that is not included. Here, the screen content may be included in the specific image.

When the application flag indicates'improvement in a fractional unit of delta MV is applied' or indicates'the current subblock or an upper region including the current subblock is included in a specific image', the Unit improvement can be performed. In contrast, the enhancement unit 740 indicates that the application flag indicates'the fractional improvement of the delta MV is not applied', or'the current subblock or the upper region including the current subblock is not included in a specific image' In the case of indicating, it is possible not to perform fractional improvement (skip).

The application flag may be decoded from a sequence parameter set (SPS) of the bitstream or may be decoded from a tile group header of the bitstream. Table 4 shows an example in which the application flag (tile_group_scc_subpel_disabled_flag) is defined in the tile group header of the bitstream and is signaled from the video encoding apparatus to the video decoding apparatus.

The tile_group_scc_subpel_disabled_flag is an example in which the application flag is implemented as a syntax element indicating'whether a current subblock or an upper region including the current subblock is included in the screen content'. When tile_group_scc_subpel_disabled_flag=0, improvement in decimal units may be performed, and when tile_group_scc_subpel_disabled_flag=1, improvement in decimal units may not be performed.

Decimal unit improvement

As described above, the improvement unit 740 may improve the delta MV derived in the integer unit in the decimal unit by using the decimal sample offset derived from the candidate integer samples.

The fractional improvement may include a process of deriving a fractional sample offset from candidate integer samples, and a process of improving or adjusting a delta MV derived in an integer unit by using a fractional sample offset in a fractional unit. Instead of comparing SADs between candidates in order to reduce the complexity of the improvement process in decimal units, the refiner 740 may perform fractional improvement using a parametric error surface equation.

Whether to perform integer unit derivation process

The example described above corresponds to an embodiment in which only whether or not to perform the improvement process in a fractional unit is determined according to the value of the application flag. Depending on the embodiment, the present invention may determine whether to perform an integer unit derivation process depending on the value of the application flag. That is, the present invention may selectively determine whether to perform DMVR itself (integer unit improvement and decimal unit improvement) according to the value of the applied flag.

The acquisition unit 710 may acquire the applied flag, the initial MV, and the reference picture of the current subblock from the bitstream (S1010).

When the initial MV is obtained, the induction unit 730 may determine whether to induce the delta MV in integer units according to the value of the application flag (S1020). If it is determined that the derivation of the integer unit is to be performed, the process of constructing the prediction samples (S1030), the process of determining the integer sample having the smallest SAD among the prediction samples (S1040), and the delta MV by using the integer sample The inducing process (S1050) may be performed. In contrast, if it is determined that the derivation of the integer unit is not performed, the processes S1030 to S1050 may not be performed.

As described above, the fractional unit improvement of the delta MV is performed using a fractional sample offset, and the fractional sample offset may be performed using candidate integer samples derived from the processes S1030 and S1040. Therefore, when the derivation of the integer unit is not performed by the value of the application flag, it is preferable to set so that the fractional unit improvement is not performed. Accordingly, when the derivation of the integer unit is not performed, the process of improving the delta MV to the decimal unit (S1060) and the process of deriving the improved MV (S1070) may not be performed.

After all, determining that the derivation of the integer unit is not performed may mean that the DMVR itself is not performed. If this is expanded, the process of determining whether to induce delta MV in integer units according to the value of the application flag (S1020) may be understood as a process of determining whether to apply the DMVR to the current subblock.

The prediction execution unit 760 may predict the current subblock based on the improved MV when the integer unit derivation is performed, and may predict the current subblock based on the initial MV when the integer unit derivation is not performed ( S1080).

Composition of candidate samples

Meanwhile, the process of configuring candidate samples may include a process of configuring candidate integer samples (S1110) and a process of configuring candidate decimal samples through interpolation using values of the candidate integer samples (S1130).

However, when the improvement in decimal units is not performed by the application flag, a'process of configuring candidate decimal samples' for improvement in the decimal unit may be unnecessary. That is, when the fractional improvement is not performed by the application flag, the process of configuring the candidate samples may consist of only the process of configuring the candidate integer samples.

In order to determine this, the configuration unit 720 may determine whether to perform a process of interpolating candidate prime samples based on the application flag decoded from the bitstream (S1120).

When it is determined that the process of interpolating the candidate prime samples is determined, the constructing unit 720 may construct candidate prime samples using values of the candidate integer samples (S1130). In contrast, if it is determined that the process of interpolating the candidate prime samples is not performed, the constructing unit 720 may not perform the interpolation process for constructing the candidate prime samples. In this case, since only candidate integer samples are available, DMVR can only perform integer unit derivation for delta MV.

The above description is merely illustrative of the technical idea of the present embodiment, and those of ordinary skill in the technical field to which the present embodiment belongs will be able to make various modifications and variations without departing from the essential characteristics of the present embodiment. Accordingly, the present exemplary embodiments are not intended to limit the technical idea of the present exemplary embodiment, but are illustrative, and the scope of the technical idea of the present exemplary embodiment is not limited by these exemplary embodiments. The scope of protection of this embodiment should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present embodiment.

120, 440: prediction unit 130: subtractor
170, 450: adder 180, 460: filter unit
720: configuration unit 730: guidance unit

Claims (12)

As a method of deriving a delta motion vector used in a decoder-side motion vector refinement (DMVR),
Obtaining an application flag related to whether or not the DMVR is applied, a motion vector of a current subblock, and a reference picture of the current subblock from a bitstream;
From the position indicated by the motion vector, the delta motion vector is calculated using an integer sample offset indicating an integer sample having a minimum sum of absolute differences (SAD) among candidate integer samples corresponding to the current subblock. Deriving in integer units; And
Using a fractional sample offset derived from the candidate integer samples, refinement of the delta motion vector derived in an integer unit in a decimal unit,
The improving step,
A method of deriving a delta motion vector in which whether to perform is determined according to a value of the applied flag. The method of claim 1,
The applied flag is,
Indicate whether the current subblock corresponds to screen content,
The improving step,
If the application flag indicates that the current subblock corresponds to screen content, the delta motion vector derivation method is not performed. The method of claim 1,
The applied flag is,
Indicating whether or not the improvement of the delta motion vector derived in the integer unit is applied,
The improving step,
If the application flag indicates that the improvement of the delta motion vector derived in the integer unit is not applied, the delta motion vector derivation method is not performed. The method of claim 1,
The inducing step,
A method of deriving a delta motion vector in which whether to perform is determined according to a value of the applied flag. The method of claim 1,
The applied flag is,
A method of deriving a delta motion vector obtained from a sequence parameter set (SPS) of the bitstream or a tile group header. The method of claim 1,
Configuring prediction samples located at a preset position among prediction samples corresponding to the current subblock as the candidate integer samples; And
Further comprising the step of interpolating the candidate prime samples using the values of the candidate integer samples,
The interpolating step,
A method of deriving a delta motion vector in which whether to perform is determined according to a value of the applied flag. An image decoding apparatus for inducing a delta motion vector used in a decoder-side motion vector refinement (DMVR), comprising:
An acquisition unit that obtains an application flag related to whether or not the DMVR is applied, a motion vector of the current subblock, and a reference picture of the current subblock from the bitstream;
From the position indicated by the motion vector, the delta motion vector is calculated using an integer sample offset indicating an integer sample having a minimum sum of absolute differences (SAD) among candidate integer samples corresponding to the current subblock. An induction unit for guiding in integer units; And
An improvement unit that refines the delta motion vector derived in the integer unit to the decimal unit by using the decimal sample offset derived from the candidate integer samples,
The improvement unit,
The image decoding apparatus, which determines whether to perform the fractional improvement according to the value of the application flag. The method of claim 7,
The applied flag is,
Indicate whether the current subblock corresponds to screen content,
The improvement unit,
When the application flag indicates that the current sub-block corresponds to the screen content, the image decoding apparatus does not perform the fractional improvement. The method of claim 7,
The applied flag is,
Indicating whether or not the improvement of the delta motion vector derived in the integer unit is applied,
The improvement unit,
When the application flag indicates that the improvement of the delta motion vector derived in the integer unit is not applied, the improvement in the fractional unit is not performed. The method of claim 7,
The induction part,
The video decoding apparatus for determining whether to perform the derivation of the integer unit according to the value of the application flag. The method of claim 7,
The applied flag is,
A video decoding apparatus obtained from a sequence parameter set (SPS) of the bitstream or a tile group header. The method of claim 7,
An integer sample construction unit configured to configure prediction samples located at a predetermined position among prediction samples corresponding to the current subblock as the candidate integer samples; And
Further comprising an interpolation unit for interpolating the candidate prime samples using values of the candidate integer samples,
The interpolation unit,
The video decoding apparatus, determining whether to interpolate the candidate prime samples according to the value of the applied flag.

Images