WO2023027564A1

WO2023027564A1 - Method for correcting motion information and device therefor

Info

Publication number: WO2023027564A1
Application number: PCT/KR2022/012897
Authority: WO
Inventors: 김경용; 김동철; 손주형; 곽진삼
Original assignee: 주식회사 윌러스표준기술연구소
Priority date: 2021-08-27
Filing date: 2022-08-29
Publication date: 2023-03-02

Abstract

A video signal decoding device comprises a processor that: acquires a first motion information list including one or more pieces of first motion information related to the current block; corrects the one or more pieces of first motion information to acquire one or more pieces of first corrected motion information; acquires one or more first cost values for the one or more pieces of first corrected motion information, respectively; rearranges the one or more pieces of first corrected motion information on the basis of the first cost values to acquire a second motion information list; acquires one or more pieces of second motion information on the basis of the first motion information included in the second motion information list; corrects the one or more pieces of second motion information to acquire one or more pieces of second corrected motion information; acquires one or more second cost values for the one or more pieces of second corrected motion information, respectively; and predicts the current block on the basis of the second motion information determined on the basis of the second cost values.

Description

Method for correcting motion information and apparatus therefor

The present invention relates to a method and apparatus for processing a video signal, and more particularly, to a method and apparatus for processing a video signal for encoding or decoding a video signal.

Compression coding refers to a series of signal processing techniques for transmitting digitized information through a communication line or storing it in a form suitable for a storage medium. Targets of compression coding include voice, video, text, and the like, and in particular, a technique of performing compression coding for video is called video image compression. Compression encoding of a video signal is performed by removing redundant information in consideration of spatial correlation, temporal correlation, and stochastic correlation. However, due to the recent development of various media and data transmission media, a more highly efficient video signal processing method and apparatus are required.

An object of the present specification is to increase coding efficiency of a video signal by providing a video signal processing method and an apparatus therefor.

The present specification provides a video signal processing method and apparatus therefor.

In this specification, a video signal decoding apparatus includes a processor, and the processor acquires a first motion information list including one or more pieces of first motion information related to a current block, and corrects the one or more pieces of first motion information. to obtain one or more pieces of first corrected motion information, obtain one or more first cost values for each of the pieces of one or more pieces of first corrected motion information, and set the one or more pieces of first corrected motion information to the first cost A second motion information list is obtained by rearranging based on values, one or more pieces of second motion information are obtained based on first motion information included in the second motion information list, and the one or more pieces of second motion information are corrected. to obtain one or more pieces of second corrected motion information, obtain one or more second cost values for each of the pieces of second corrected motion information, and based on the second motion information determined based on the second cost values to predict the current block.

In this specification, a video signal encoding device includes a processor, and the processor obtains a bitstream decoded by a decoding method, and the decoding method includes a first motion information related to a current block. acquiring a motion information list; obtaining one or more pieces of first corrected motion information by correcting the one or more pieces of first motion information; obtaining one or more first cost values for each of the one or more pieces of first corrected motion information; acquiring a second motion information list by rearranging the one or more pieces of first corrected motion information based on the first cost values; obtaining one or more pieces of second motion information based on first motion information included in the second motion information list; correcting the one or more pieces of second motion information to obtain one or more pieces of second corrected motion information; obtaining one or more second cost values for each of the second corrected motion information pieces; and predicting the current block based on second motion information determined based on the second cost values.

In the present specification, in a computer-readable non-transitory storage medium storing a bitstream, the bitstream is decoded by a decoding method, wherein the decoding method includes one or more pieces of first motion information related to a current block. 1 obtaining a motion information list; obtaining one or more pieces of first corrected motion information by correcting the one or more pieces of first motion information; obtaining one or more first cost values for each of the one or more pieces of first corrected motion information; acquiring a second motion information list by rearranging the one or more pieces of first corrected motion information based on the first cost values; obtaining one or more pieces of second motion information based on first motion information included in the second motion information list; correcting the one or more pieces of second motion information to obtain one or more pieces of second corrected motion information; obtaining one or more second cost values for each of the second corrected motion information pieces; and predicting the current block based on second motion information determined based on the second cost values.

The one or more first cost values are values related to the similarity between the current block and a reference block corresponding to each of the one or more pieces of first corrected motion information, and the one or more second cost values are values related to a similarity between the current block and the one or more pieces of first corrected motion information. It is characterized in that it is a value related to the degree of similarity between reference blocks corresponding to each of the second corrected motion information pieces.

The second motion information list is characterized in that cost values respectively corresponding to the one or more pieces of first corrected motion information are arranged in ascending order.

The first motion information may be corrected motion information corresponding to the smallest cost value among the first cost values.

The second motion information may be corrected motion information corresponding to the smallest cost value among the second cost values.

The one or more pieces of first motion information are included in different pictures, and the one or more pieces of second motion information are included in different pictures.

The one or more pieces of first corrected motion information and the one or more pieces of second corrected motion information are Motion Vector Difference (MVD), Template Matching (TM), Bilateral Matching (BM), Merge mode with MVD (MMVD), MMVD (Merge mode with MVD)-based TM, optical flow-based TM, and multi-pass decoder-side motion vector refinement (DMVR).

When the encoding mode of the current block is a merge mode, the one or more pieces of first motion information and the one or more pieces of second motion information are motion information derived by merge mode with MVD (MMVD). .

Blocks corresponding to each of the one or more pieces of first motion information are blocks located in a first search area, blocks corresponding to each of the one or more pieces of second motion information are blocks located in a second search area, and The first search area and the second search area are different from each other.

The first cost values are sequentially calculated for each of the one or more pieces of first corrected motion information, and the second cost values are sequentially calculated for each of one or more pieces of second corrected motion information. do.

When a cost value smaller than the preset first value is calculated while the first cost values are sequentially calculated, cost values for the remaining corrected motion information are not calculated, and the second cost values are sequentially calculated. Among them, when a cost value smaller than the preset second value is calculated, cost values for the remaining corrected motion information are not calculated.

The present specification provides a method for efficiently processing a video signal.

The effects obtainable in the present specification are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

1 is a schematic block diagram of a video signal encoding apparatus according to an embodiment of the present invention.

2 is a schematic block diagram of a video signal decoding apparatus according to an embodiment of the present invention.

3 shows an embodiment in which a coding tree unit within a picture is divided into coding units.

4 illustrates one embodiment of a method for signaling splitting of quad trees and multi-type trees.

5 and 6 show an intra prediction method according to an embodiment of the present invention in more detail.

7 is a diagram illustrating positions of neighboring blocks used to construct a motion candidate list in inter prediction.

8 is a diagram illustrating a method of correcting motion information according to an embodiment of the present invention.

9 is a diagram illustrating a method of correcting motion information for a current block by recursively performing a motion compensation method according to an embodiment of the present invention.

10 is a diagram showing a sequence in which a TM method according to an embodiment of the present invention is performed.

11 is a diagram illustrating a method of setting a search area for a TM method based on initial motion information according to an embodiment of the present invention.

12 is a diagram illustrating positions of motion candidates searched within a search area according to an embodiment of the present invention.

13 is a diagram illustrating a process of searching for a position of a motion candidate according to an embodiment of the present invention.

14 is a diagram illustrating a process of evaluating search candidates according to an embodiment of the present invention.

15 is a diagram illustrating motion characteristics of a boundary portion of an object in a current picture according to an embodiment of the present invention.

16 is a diagram illustrating a method of determining a reference picture using template matching according to an embodiment of the present invention.

17 and 18 are diagrams illustrating a process of performing template matching according to an embodiment of the present invention.

19 is a diagram illustrating a case in which initial motion information derived from neighboring blocks of a current block are attached to each other according to an embodiment of the present invention.

20 and 21 are diagrams illustrating a method of changing initial motion information according to an embodiment of the present invention.

22 is a diagram illustrating a position where a motion candidate derived through initial motion information is searched according to an embodiment of the present invention.

23 is a diagram illustrating a method of changing a search position where a motion candidate is searched for according to an embodiment of the present invention.

24 is a diagram illustrating a TM based on a cost value of a motion vector according to an embodiment of the present invention.

25 and 26 are diagrams illustrating a method of correcting motion information using a DMVR according to an embodiment of the present invention.

27 is a diagram illustrating a process of performing multiple DMVRs according to an embodiment of the present invention.

28 is a diagram illustrating a search method for obtaining a cost value related to corrected motion information of a coding block according to an embodiment of the present invention.

29 is a diagram illustrating a 3x3 Square search method according to an embodiment of the present invention.

30 is a diagram showing that a search area is divided into zones in an integer-unit global search process according to an embodiment of the present invention.

31 and 32 are diagrams illustrating a global search process in integer units according to an embodiment of the present invention.

33 and 34 are diagrams illustrating a method of performing correction of motion information based on BDOF according to an embodiment of the present invention.

35 and 36 are diagrams illustrating a method of generating a prediction block for a current block based on BDOF according to an embodiment of the present invention.

37 is a diagram illustrating subblocks to which DMVR according to an embodiment of the present invention is applied.

38 is a diagram illustrating a method of generating a prediction block for a current block while performing multi-DMVR according to an embodiment of the present invention.

The terminology used in this specification has been selected as a general term that is currently widely used as much as possible while considering the function in the present invention, but it may vary according to the intention of a person skilled in the art, custom, or the emergence of new technology. In addition, in certain cases, there are also terms arbitrarily selected by the applicant, and in this case, the meaning will be described in the description of the invention. Therefore, it should be noted that the terms used in this specification should be interpreted based on the actual meaning of the term and the overall content of this specification, not the simple name of the term.

In the present specification, 'A and/or B' may be interpreted as meaning 'including at least one of A or B'.

Some terms in this specification can be interpreted as follows. Coding can be interpreted as either encoding or decoding, as the case may be. In this specification, a device that performs encoding (encoding) of a video signal to generate a video signal bitstream is referred to as an encoding device or an encoder, and a device that performs decoding (decoding) of a video signal bitstream to restore a video signal is referred to as a decoding device. referred to as a device or decoder. Also, in this specification, a video signal processing apparatus is used as a conceptual term including both an encoder and a decoder. Information is a term that includes all of values, parameters, coefficients, elements, etc., and the meaning can be interpreted differently depending on the case, so the present invention is not limited thereto. A 'unit' is used to indicate a basic unit of image processing or a specific location of a picture, and refers to an image area including at least one of a luma component and a chroma component. Also, a 'block' refers to an image area including a specific component among luminance components and chrominance components (ie, Cb and Cr). However, terms such as 'unit', 'block', 'partition', 'signal' and 'region' may be used interchangeably depending on embodiments. Also, in this specification, a 'current block' means a block currently scheduled to be encoded, and a 'reference block' means a block that has already been coded or decoded and is used as a reference in the current block. Also, in the present specification, terms such as 'luma', 'luma', 'luminance', and 'Y' may be used interchangeably. In addition, in this specification, terms such as 'chroma', 'chroma', 'color difference', and 'Cb or Cr' may be used interchangeably. can Also, in this specification, a unit may be used as a concept including all of a coding unit, a prediction unit, and a transform unit. A picture refers to a field or a frame, and the terms may be used interchangeably depending on embodiments. Specifically, when a photographed image is an interlace image, one frame is divided into an odd (or odd, top) field and an even (or even, bottom) field, and each field is composed of one picture unit. and can be encoded or decoded. If the photographed image is a progressive image, one frame may be configured as a picture and encoded or decoded. Also, in this specification, terms such as 'error signal', 'residual signal', 'residual signal', 'residual signal', and 'difference signal' may be used interchangeably. Also, terms such as 'intra-prediction mode', 'intra-prediction directional mode', 'intra-prediction mode', and 'intra-prediction directional mode' may be used interchangeably in this specification. Also, in this specification, terms such as 'motion' and 'movement' may be used interchangeably. In addition, in the present specification, 'left', 'upper left', 'upper', 'upper right', 'right', 'lower right', 'lower', 'lower left' means 'left', 'upper left', ' Top', 'top right', 'right end', 'bottom right', 'bottom', 'bottom left' may be used interchangeably. Also, elements and members may be used interchangeably. POC (Picture Order Count) represents temporal location information of a picture (or frame), may be a playback order displayed on a screen, and may have a unique POC for each picture.

1 is a schematic block diagram of a video signal encoding apparatus 100 according to an embodiment of the present invention. Referring to FIG. 1, the encoding apparatus 100 of the present invention includes a transform unit 110, a quantization unit 115, an inverse quantization unit 120, an inverse transform unit 125, a filtering unit 130, and a prediction unit 150. ) and an entropy coding unit 160.

The transform unit 110 transforms the residual signal, which is the difference between the received video signal and the prediction signal generated by the predictor 150, to obtain a transform coefficient value. For example, a discrete cosine transform (DCT), a discrete sine transform (DST), or a wavelet transform may be used. Discrete cosine transform and discrete sine transform perform conversion by dividing an input picture signal into blocks. In transformation, coding efficiency may vary according to the distribution and characteristics of values within a transformation domain. A transform kernel used for transforming a residual block may be a transform kernel having separable characteristics of vertical transform and horizontal transform. In this case, transformation of the residual block may be performed by dividing the vertical transformation and the horizontal transformation. For example, the encoder may perform vertical transform by applying a transform kernel in the vertical direction of the residual block. Also, the encoder may perform horizontal transformation by applying a transformation kernel in the horizontal direction of the residual block. In the present disclosure, a transform kernel may be used as a term referring to a set of parameters used for transforming a residual signal, such as a transform matrix, a transform array, a transform function, and a transform. For example, the conversion kernel may be any one of a plurality of available kernels. Also, transform kernels based on different transform types may be used for each of the vertical transform and the horizontal transform.

As for conversion coefficients, higher coefficients are distributed toward the upper left of the block, and coefficients close to '0' are distributed toward the lower right of the block. As the size of the current block increases, there is a possibility that more coefficients '0' exist in the lower right area. In order to reduce the complexity of transforming a block having a large size, only an arbitrary upper left region may be left and the remaining regions may be reset to '0'.

Also, an error signal may exist only in a partial region in a coding block. In this case, the conversion process may be performed only on an arbitrary partial area. As an example, an error signal may exist only in the first 2NxN block in a block having a size of 2Nx2N, and a conversion process is performed only in the first 2NxN block, but the conversion process is not performed on the second 2NxN block and may not be encoded or decoded. Where N can be any positive integer.

The encoder may perform additional transforms before the transform coefficients are quantized. The transform method described above is referred to as a primary transform, and an additional transform may be referred to as a secondary transform. Secondary transformation may be selective for each residual block. According to an embodiment, the encoder may improve coding efficiency by performing secondary transform on a region in which it is difficult to concentrate energy in a low frequency region with only the primary transform. For example, secondary transformation may be additionally performed on a block having large residual values in a direction other than the horizontal or vertical direction of the residual block. Unlike the first conversion, the secondary conversion may not be performed separately into vertical conversion and horizontal conversion. This secondary transform may be referred to as a Low Frequency Non-Separable Transform (LFNST).

The quantization unit 115 quantizes the transform coefficient value output from the transform unit 110 .

In order to increase coding efficiency, a picture signal is not coded as it is, but a picture is predicted using an area already coded through the prediction unit 150, and a residual value between the original picture and the predicted picture is added to the predicted picture to obtain a reconstructed picture. A method for obtaining is used. In order to avoid mismatches between the encoder and decoder, when the encoder performs prediction, the decoder must also use available information. To this end, the encoder performs a process of restoring the encoded current block again. The inverse quantization unit 120 inversely quantizes the transform coefficient value, and the inverse transform unit 125 restores the residual value using the inverse quantized transform coefficient value. Meanwhile, the filtering unit 130 performs a filtering operation to improve quality and coding efficiency of a reconstructed picture. For example, a deblocking filter, a Sample Adaptive Offset (SAO), and an adaptive loop filter may be included. A picture that has undergone filtering is stored in a decoded picture buffer (DPB, 156) to be output or used as a reference picture.

A deblocking filter is a filter for removing distortion within a block generated at a boundary between blocks in a reconstructed picture. The encoder may determine whether to apply a deblocking filter to a corresponding edge through a distribution of pixels included in several columns or rows based on an arbitrary edge in a block. When a deblocking filter is applied to a block, the encoder may apply a long filter, a strong filter, or a weak filter according to the strength of the deblocking filtering. Also, horizontal direction filtering and vertical direction filtering can be processed in parallel. The sample adaptive offset (SAO) may be used to correct an offset from an original image in units of pixels for a residual block to which a deblocking filter is applied. In order to correct the offset for a specific picture, the encoder divides the pixels included in the image into a certain number of areas, determines the area to perform offset correction, and uses a method (Band Offset) to apply the offset to the area. can Alternatively, the encoder may use a method (Edge Offset) of applying an offset in consideration of edge information of each pixel. An adaptive loop filter (ALF) is a method of dividing pixels included in an image into predetermined groups, determining one filter to be applied to the group, and performing filtering differentially for each group. Information related to whether to apply ALF may be signaled in units of coding units, and the shape and filter coefficients of an ALF filter to be applied may vary according to each block. In addition, the ALF filter of the same form (fixed form) may be applied regardless of the characteristics of the target block to be applied.

The prediction unit 150 includes an intra prediction unit 152 and an inter prediction unit 154. The intra prediction unit 152 performs intra prediction within the current picture, and the inter prediction unit 154 predicts the current picture using the reference picture stored in the decoded picture buffer 156. Do it. The intra prediction unit 152 performs intra prediction on reconstructed regions in the current picture and transfers intra-encoding information to the entropy coding unit 160 . The intra encoding information may include at least one of an intra prediction mode, a most probable mode (MPM) flag, an MPM index, and information about a reference sample. The inter prediction unit 154 may again include a motion estimation unit 154a and a motion compensation unit 154b. The motion estimation unit 154a refers to a specific region of the reconstructed reference picture to find a part most similar to the current region and obtains a motion vector value that is a distance between the regions. Motion information (reference direction indication information (L0 prediction, L1 prediction, bi-directional prediction), reference picture index, motion vector information, etc.) for the reference region acquired by the motion estimation unit 154a is transferred to the entropy coding unit 160. so that it can be included in the bitstream. The motion compensation unit 154b performs inter-motion compensation using the motion information transmitted from the motion estimation unit 154a to generate a prediction block for the current block. The inter prediction unit 154 transfers inter encoding information including motion information on the reference region to the entropy coding unit 160 .

According to an additional embodiment, the predictor 150 may include an intra block copy (IBC) predictor (not shown). The IBC prediction unit performs IBC prediction from reconstructed samples in the current picture and transfers IBC encoding information to the entropy coding unit 160 . The IBC prediction unit refers to a specific region in the current picture and obtains a block vector value indicating a reference region used for prediction of the current region. The IBC prediction unit may perform IBC prediction using the obtained block vector value. The IBC prediction unit transfers the IBC encoding information to the entropy coding unit 160 . The IBC encoding information may include at least one of size information of a reference region and block vector information (index information for predicting a block vector of a current block in a motion candidate list and block vector difference information).

When the above picture prediction is performed, the transform unit 110 obtains a transform coefficient value by transforming a residual value between an original picture and a predicted picture. In this case, transformation may be performed in units of a specific block within a picture, and the size of a specific block may vary within a preset range. The quantization unit 115 quantizes the transform coefficient values generated by the transform unit 110 and transfers the quantized transform coefficients to the entropy coding unit 160 .

The quantized transform coefficients in the form of a two-dimensional array may be rearranged into a form of a one-dimensional array for entropy coding. A scanning method for quantized transform coefficients may be determined according to a size of a transform block and an intra-prediction mode. As an embodiment, diagonal, vertical, and horizontal scans may be applied. Such scan information may be signaled in units of blocks and may be derived according to pre-determined rules.

The entropy coding unit 160 generates a video signal bitstream by entropy coding information representing quantized transform coefficients, intra-encoding information, and inter-encoding information. In the entropy coding unit 160, a variable length coding (VLC) method and an arithmetic coding method may be used. A variable length coding (VLC) method converts input symbols into continuous codewords, the length of which can be variable. For example, frequently occurring symbols are represented by short codewords, and infrequently occurring symbols are represented by long codewords. As a variable length coding scheme, a context-based adaptive variable length coding (CAVLC) scheme may be used. Arithmetic coding converts successive data symbols into a single prime number using a probability distribution of each data symbol. Arithmetic coding can obtain an optimal number of decimal bits required to represent each symbol. As arithmetic coding, context-based adaptive binary arithmetic code (CABAC) may be used.

CABAC is a method of encoding binary arithmetic through several context models generated based on probabilities obtained through experiments. First, if the symbol is not binary, the encoder binarizes each symbol using exp-Golomb or the like. A binarized 0 or 1 can be described as a bin. The CABAC initialization process is divided into context initialization and arithmetic coding initialization. Context initialization is a process of initializing the occurrence probability of each symbol, and is determined according to the symbol type, quantization parameter (QP), and slice type (whether I, P, or B). A context model having such initialization information may use a probability-based value obtained through an experiment. The context model provides information (valMPS) about the probability of occurrence of a least probable symbol (LPS) or most probable symbol (MPS) for a symbol to be currently coded and which bin value among 0 and 1 corresponds to the MPS. One of several context models is selected through a context index (ctxIdx), and the context index can be derived through information of a block to be currently encoded or information of neighboring blocks. Initialization for binary arithmetic coding is performed based on the probability model selected in the context model. Binary arithmetic encoding is performed by dividing into probability intervals through the occurrence probabilities of 0 and 1, and then the probability interval corresponding to the bin to be processed becomes the entire probability interval for the next bin to be processed. Position information within the probability interval where the last bin was processed is output. However, since the probability interval cannot be divided indefinitely, when it is reduced to within a certain size, a renormalization process is performed to widen the probability interval and corresponding location information is output. In addition, after each bin is processed, a probability update process may be performed in which a probability of a next bin to be processed is newly set based on information of the processed bin.

The generated bitstream is encapsulated in a network abstraction layer (NAL) unit as a basic unit. The NAL unit is divided into a VCL (Video Coding Layer) NAL unit including video data and a non-VCL NAL unit including parameter information for decoding video data. There are various types of VCL or non-VCL NAL units. . The NAL unit is composed of NAL header information and data, RBSP (Raw Byte Sequence Payload), and the NAL header information includes summary information about the RBSP. The RBSP of the VCL NAL unit includes a coded integer number of coding tree units. In order to decode a bitstream in a video decoder, the bitstream must first be divided into NAL unit units and then each separated NAL unit must be decoded. Meanwhile, information necessary for decoding a video signal bitstream is included in a Picture Parameter Set (PPS), a Sequence Parameter Set (SPS), a Video Parameter Set (VPS), etc. and transmitted. can

Meanwhile, the block diagram of FIG. 1 shows the encoding apparatus 100 according to an embodiment of the present invention, and the separately displayed blocks logically distinguish elements of the encoding apparatus 100. Accordingly, the elements of the encoding apparatus 100 described above may be mounted as one chip or as a plurality of chips according to the design of the device. According to one embodiment, the operation of each element of the above-described encoding device 100 may be performed by a processor (not shown).

2 is a schematic block diagram of a video signal decoding apparatus 200 according to an embodiment of the present invention. Referring to FIG. 2 , the decoding apparatus 200 of the present invention includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 225, a filtering unit 230, and a prediction unit 250.

The entropy decoding unit 210 entropy-decodes the video signal bitstream and extracts transform coefficient information, intra-encoding information, and inter-encoding information for each region. For example, the entropy decoding unit 210 may obtain a binarization code for transform coefficient information of a specific region from a video signal bitstream. Also, the entropy decoding unit 210 inversely binarizes the binary code to obtain quantized transform coefficients. The inverse quantization unit 220 inversely quantizes the quantized transform coefficient, and the inverse transform unit 225 restores a residual value using the inverse quantized transform coefficient. The video signal processing apparatus 200 restores an original pixel value by adding the residual value obtained from the inverse transform unit 225 to the prediction value obtained from the predictor 250.

Meanwhile, the filtering unit 230 improves picture quality by performing filtering on pictures. This may include a deblocking filter to reduce block distortion and/or an adaptive loop filter to remove distortion of the entire picture. The filtered picture is output or stored in the decoded picture buffer (DPB) 256 to be used as a reference picture for the next picture.

The prediction unit 250 includes an intra prediction unit 252 and an inter prediction unit 254 . The prediction unit 250 generates a predicted picture by utilizing the coding type decoded through the above-described entropy decoding unit 210, transform coefficients for each region, intra/inter coding information, and the like. In order to reconstruct a current block on which decoding is performed, a current picture including the current block or a decoded area of other pictures may be used. A picture (or tile/slice) that uses only the current picture for reconstruction, that is, performs intra prediction or intra BC prediction, is converted into an intra picture or I picture (or tile/slice), intra prediction, inter prediction, and intra BC prediction. A picture (or tile/slice) that can be performed is called an inter picture (or tile/slice). A picture (or tile/slice) using up to one motion vector and reference picture index to predict sample values of each block among inter-pictures (or tiles/slices) is called a predictive picture or a P picture (or , tile/slice), and a picture (or tile/slice) using up to two motion vectors and a reference picture index is called a bi-predictive picture or B picture (or tile/slice). In other words, a P picture (or tile/slice) uses at most one set of motion information to predict each block, and a B picture (or tile/slice) uses at most two sets of motion information to predict each block. use a set Here, the motion information set includes one or more motion vectors and one reference picture index.

The intra prediction unit 252 generates a prediction block using intra encoding information and reconstructed samples in a current picture. As described above, the intra encoding information may include at least one of an intra prediction mode, a most probable mode (MPM) flag, and an MPM index. The intra predictor 252 predicts sample values of the current block by using reconstructed samples located on the left side and/or above the current block as reference samples. In this disclosure, reconstructed samples, reference samples, and samples of a current block may represent pixels. Also, sample values may represent pixel values.

According to an embodiment, reference samples may be samples included in neighboring blocks of the current block. For example, the reference samples may be samples adjacent to the left boundary of the current block and/or samples adjacent to the upper boundary of the current block. In addition, the reference samples are samples located on a line within a preset distance from the left boundary of the current block among samples of neighboring blocks of the current block and/or located on a line within a preset distance from the upper boundary of the current block. may be samples of At this time, the neighboring blocks of the current block may be a left (L) block, an upper (A) block, a below left (BL) block, an above right (AR) block, or an above left (Above Left) block adjacent to the current block. AL) may include at least one of the blocks.

The inter prediction unit 254 generates a prediction block using a reference picture stored in the decoded picture buffer 256 and inter encoding information. The inter-encoding information may include a motion information set (reference picture index, motion vector information, etc.) of a current block with respect to a reference block. Inter prediction may include L0 prediction, L1 prediction, and bi-prediction. L0 prediction refers to prediction using one reference picture included in the L0 picture list, and L1 prediction refers to prediction using one reference picture included in the L1 picture list. To this end, one set of motion information (eg, a motion vector and a reference picture index) may be required. In the bi-prediction method, up to two reference regions can be used, and these two reference regions may exist in the same reference picture or in different pictures. That is, in the bi-prediction method, up to two sets of motion information (eg, a motion vector and a reference picture index) can be used, and the two motion vectors may correspond to the same reference picture index or to different reference picture indices. may correspond. In this case, the reference pictures are pictures positioned before or after the current picture in terms of time, and may be pictures that have already been reconstructed. According to an embodiment, two reference regions used in the bi-prediction method may be regions selected from each of the L0 picture list and the L1 picture list.

The inter prediction unit 254 may obtain a reference block of the current block by using the motion vector and the reference picture index. The reference block exists in a reference picture corresponding to a reference picture index. Also, a sample value of a block specified by a motion vector or an interpolated value thereof may be used as a predictor of a current block. For motion prediction with sub-pel unit pixel accuracy, for example, an 8-tap interpolation filter for a luminance signal and a 4-tap interpolation filter for a chrominance signal may be used. However, an interpolation filter for motion prediction in units of subpels is not limited thereto. In this way, the inter prediction unit 254 performs motion compensation for predicting the texture of the current unit from the previously reconstructed picture. In this case, the inter prediction unit may use the motion information set.

According to a further embodiment, the prediction unit 250 may include an IBC prediction unit (not shown). The IBC prediction unit may reconstruct the current region by referring to a specific region including reconstructed samples in the current picture. The IBC prediction unit may perform IBC prediction using the IBC encoding information obtained from the entropy decoding unit 210 . IBC encoding information may include block vector information.

A reconstructed video picture is generated by adding the prediction value output from the intra prediction unit 252 or the inter prediction unit 254 and the residual value output from the inverse transform unit 225. That is, the video signal decoding apparatus 200 reconstructs the current block by using the prediction block generated by the prediction unit 250 and the residual obtained from the inverse transform unit 225.

Meanwhile, the block diagram of FIG. 2 shows the decoding apparatus 200 according to an embodiment of the present invention, and the separately displayed blocks logically distinguish elements of the decoding apparatus 200. Accordingly, elements of the decoding apparatus 200 described above may be mounted as one chip or as a plurality of chips according to the design of the device. According to one embodiment, the operation of each element of the decoding apparatus 200 described above may be performed by a processor (not shown).

Meanwhile, the technology proposed in this specification is a technology applicable to both encoder and decoder methods and devices, and parts described as signaling and parsing may be described for convenience of explanation. In general, it can be described that signaling is for encoding each syntax from an encoder point of view, and parsing is for interpreting each syntax from a decoder point of view. That is, each syntax may be included in a bitstream from the encoder and signaled, and the decoder may parse the syntax and use it in the restoration process. In this case, a sequence of bits for each syntax arranged according to a defined hierarchical configuration may be referred to as a bitstream.

One picture may be coded after being divided into sub-pictures, slices, tiles, and the like. A subpicture may contain one or more slices or tiles. When one picture is divided into several slices or tiles and encoded, all slices or tiles in the picture must be decoded before being displayed on the screen. On the other hand, when one picture is coded with several subpictures, only a certain subpicture can be decoded and displayed on the screen. A slice may contain multiple tiles or subpictures. Alternatively, a tile may include multiple subpictures or slices. Since subpictures, slices, and tiles can be encoded or decoded independently of each other, it is effective in improving parallel processing and processing speed. However, since coded information of other adjacent subpictures, other slices, and other tiles cannot be used, the amount of bits increases. Subpictures, slices, and tiles may be coded after being divided into several Coding Tree Units (CTUs).

3 illustrates an embodiment in which a Coding Tree Unit (CTU) within a picture is divided into Coding Units (CUs). In the process of coding a video signal, a picture may be divided into a sequence of coding tree units (CTUs). A coding tree unit may include a luma coding tree block (CTB), two chroma coding tree blocks, and encoded syntax information thereof. One coding tree unit may be composed of one coding unit, or one coding tree unit may be divided into several coding units. One coding unit may include a luminance coding block (CB), two color difference coding blocks, and their encoded syntax information. One coding block may be divided into several sub coding blocks. One coding unit may be composed of one transform unit (TU), or one coding unit may be divided into several transform units. One transform unit may include a luminance transform block (TB), two color difference transform blocks, and encoded syntax information thereof. A coding tree unit may be divided into a plurality of coding units. A coding tree unit may be a leaf node without being split. In this case, the coding tree unit itself may be a coding unit.

A coding unit refers to a basic unit for processing a picture in the process of processing a video signal described above, that is, intra/inter prediction, transformation, quantization, and/or entropy coding. The size and shape of a coding unit within one picture may not be constant. A coding unit may have a square or rectangular shape. A rectangular coding unit (or rectangular block) includes a vertical coding unit (or vertical block) and a horizontal coding unit (or horizontal block). In this specification, a vertical block is a block whose height is greater than its width, and a horizontal block is a block whose width is greater than its height. Also, in this specification, a non-square block may refer to a rectangular block, but the present invention is not limited thereto.

Referring to FIG. 3, the coding tree unit is first divided into a quad tree (QT) structure. That is, in the quad tree structure, one node having a size of 2NX2N may be divided into four nodes having a size of NXN. In this specification, a quad tree may also be referred to as a quaternary tree. Quad tree splitting can be done recursively, and not all nodes need to be split to the same depth.

Meanwhile, the leaf node of the aforementioned quad tree may be further divided into a multi-type tree (MTT) structure. According to an embodiment of the present invention, in a multi-type tree structure, one node may be split into a binary (binary) or ternary (ternary) tree structure of horizontal or vertical split. That is, there are four partition structures of vertical binary partitioning, horizontal binary partitioning, vertical ternary partitioning, and horizontal ternary partitioning in the multi-type tree structure. According to an embodiment of the present invention, both the width and height of a node in each tree structure may have a power of 2 value. For example, in a binary tree (Binary Tree, BT) structure, a node having a size of 2NX2N is divided into two NX2N nodes by vertical binary partitioning and divided into two 2NXN nodes by horizontal binary partitioning. In addition, in the Ternary Tree (TT) structure, a node of size 2NX2N is divided into nodes of (N/2)X2N, NX2N and (N/2)X2N by vertical ternary division, and horizontal ternary division It can be divided into 2NX(N/2), 2NXN and 2NX(N/2) nodes by partitioning. This multi-type tree partitioning can be performed recursively.

A leaf node of a multi-type tree can be a coding unit. If the coding unit is not large compared to the maximum transform length, the coding unit may be used as a unit of prediction and/or transformation without further division. As an embodiment, when the width or height of the current coding unit is greater than the maximum transform length, the current coding unit may be divided into a plurality of transform units without explicit signaling regarding division. Meanwhile, in the aforementioned quad tree and multi-type tree, at least one of the following parameters may be defined in advance or may be transmitted through a higher level set of RBSPs such as PPS, SPS, and VPS. 1) CTU size: root node size of the quad tree, 2) Minimum QT size (MinQtSize): Minimum allowed QT leaf node size, 3) Maximum BT size (MaxBtSize): Maximum allowed BT root node size, 4) Maximum TT size (MaxTtSize): Maximum allowed TT root node size, 5) Maximum MTT depth (MaxMttDepth): Maximum permitted depth of MTT splits from QT's leaf nodes, 6) Minimum BT size (MinBtSize): Allowed Minimum BT leaf node size, 7) Minimum TT size (MinTtSize): Minimum TT leaf node size allowed.

4 illustrates one embodiment of a method for signaling splitting of quad trees and multi-type trees. Preset flags may be used to signal splitting of the aforementioned quad tree and multi-type tree. Referring to FIG. 4, a flag 'split_cu_flag' indicating whether a node is split, a flag 'split_qt_flag' indicating whether a quad tree node is split, a flag 'mtt_split_cu_vertical_flag' indicating a split direction of a multi-type tree node, or a multi-type tree node At least one of flags 'mtt_split_cu_binary_flag' indicating a split shape of a type tree node may be used.

According to an embodiment of the present invention, 'split_cu_flag', which is a flag indicating whether to split a current node, may be signaled first. If the value of 'split_cu_flag' is 0, it indicates that the current node is not split, and the current node becomes a coding unit. When the current node is a coating tree unit, the coding tree unit includes one undivided coding unit. When the current node is a quad tree node 'QT node', the current node is a leaf node 'QT leaf node' of the quad tree and becomes a coding unit. When the current node is a multi-type tree node 'MTT node', the current node is a leaf node 'MTT leaf node' of the multi-type tree and becomes a coding unit.

When the value of 'split_cu_flag' is 1, the current node may be split into quad tree or multi-type tree nodes according to the value of 'split_qt_flag'. A coding tree unit is a root node of a quad tree, and can be first partitioned into a quad tree structure. In the quad tree structure, 'split_qt_flag' is signaled for each node 'QT node'. If the value of 'split_qt_flag' is 1, the corresponding node is split into 4 square nodes, and if the value of 'split_qt_flag' is 0, the corresponding node becomes the 'QT leaf node' of the quad tree, and the corresponding node is a multi-square node. -It is split into type nodes. According to an embodiment of the present invention, quad tree partitioning may be limited according to the type of current node. Quad tree splitting may be allowed if the current node is a coding tree unit (root node of a quad tree) or a quad tree node, and quad tree splitting may not be allowed if the current node is a multi-type tree node. Each quad tree leaf node 'QT leaf node' can be further partitioned into a multi-type tree structure. As described above, when 'split_qt_flag' is 0, the current node may be split into multi-type nodes. In order to indicate the split direction and split shape, 'mtt_split_cu_vertical_flag' and 'mtt_split_cu_binary_flag' may be signaled. When the value of 'mtt_split_cu_vertical_flag' is 1, vertical splitting of the node 'MTT node' is indicated, and when the value of 'mtt_split_cu_vertical_flag' is 0, horizontal splitting of the node 'MTT node' is indicated. In addition, when the value of 'mtt_split_cu_binary_flag' is 1, the node 'MTT node' is split into two rectangular nodes, and when the value of 'mtt_split_cu_binary_flag' is 0, the node 'MTT node' is split into three rectangular nodes.

In the tree division structure, a luminance block and a chrominance block may be equally divided. That is, the chrominance block may be divided by referring to the division form of the luminance block. If the size of the current chrominance block is smaller than a predetermined size, the chrominance block may not be divided even if the luminance block is divided.

In the tree partitioning structure, the luminance block and the chrominance block may have different shapes. In this case, partition information for the luminance block and partition information for the chrominance block may be signaled respectively. In addition, encoding information of the luminance block and the chrominance block as well as partition information may be different. As an example of an embodiment, at least one intra encoding mode of the luminance block and the chrominance block, encoding information about motion information, and the like may be different.

Nodes to be divided into the smallest units can be processed as one coding block. When the current block is a coding block, the coding block may be divided into several sub-blocks (sub-coding blocks), and the prediction information of each sub-block may be the same or different. As an embodiment, when a coding unit is an intra mode, the intra prediction modes of each sub-block may be the same or different. Also, when a coding unit is in an inter mode, motion information of each sub-block may be identical to or different from each other. Also, each sub-block may be independently encoded or decoded. Each sub-block may be identified through a sub-block index (sbIdx). Also, when a coding unit is divided into sub-blocks, it may be divided in a horizontal or vertical direction or diagonally. In the intra mode, a mode in which the current coding unit is divided horizontally or vertically into 2 or 4 sub-blocks is referred to as Intra Sub Partitions (ISP). A mode in which the current coding block is divided into oblique lines in the inter mode is called a geometric partitioning mode (GPM). In the GPM mode, the position and direction of the oblique line are derived using a predetermined angle table, and index information of the angle table is signaled.

Picture prediction (motion compensation) for coding is performed for a coding unit (that is, a leaf node of a coding tree unit) that is not further divided. A basic unit that performs such prediction is hereinafter referred to as a prediction unit or a prediction block.

Hereinafter, the term unit used in this specification may be used as a substitute for the prediction unit, which is a basic unit for performing prediction. However, the present invention is not limited thereto, and may be understood as a concept including the coding unit in a more broad sense.

5 and 6 show an intra prediction method according to an embodiment of the present invention in more detail. As described above, the intra prediction unit predicts sample values of the current block by using reconstructed samples located on the left side and/or above the current block as reference samples.

First, FIG. 5 shows an example of reference samples used for prediction of a current block in intra prediction mode. According to an embodiment, the reference samples may be samples adjacent to a left boundary and/or an upper boundary of the current block. As shown in FIG. 5, when the size of the current block is WXH and samples of a single reference line adjacent to the current block are used for intra prediction, up to 2W+2H+1 located on the left and/or upper side of the current block Reference samples may be set using the number of neighboring samples.

Meanwhile, pixels of multiple reference lines may be used for intra prediction of the current block. Multiple reference lines may be composed of n lines located within a predetermined range from the current block. According to an embodiment, when pixels of multiple reference lines are used for intra prediction, separate index information indicating lines to be set as reference pixels may be signaled, and this may be referred to as a reference line index.

Also, when at least some samples to be used as reference samples have not yet been reconstructed, the intra prediction unit may obtain reference samples by performing a reference sample padding process. Also, the intra prediction unit may perform a reference sample filtering process to reduce intra prediction errors. That is, filtered reference samples may be obtained by filtering the neighboring samples and/or the reference samples obtained through the reference sample padding process. The intra predictor predicts samples of the current block using the reference samples obtained in this way. The intra predictor predicts samples of the current block using unfiltered reference samples or filtered reference samples. In the present disclosure, neighboring samples may include samples on at least one reference line. For example, the neighboring samples may include neighboring samples on a line adjacent to the boundary of the current block.

Next, FIG. 6 shows an embodiment of prediction modes used for intra prediction. For intra prediction, intra prediction mode information indicating an intra prediction direction may be signaled. The intra prediction mode information indicates one of a plurality of intra prediction modes constituting an intra prediction mode set. If the current block is an intra prediction block, the decoder receives intra prediction mode information of the current block from the bitstream. The intra prediction unit of the decoder performs intra prediction on the current block based on the extracted intra prediction mode information.

According to an embodiment of the present invention, the intra prediction mode set may include all intra prediction modes used for intra prediction (eg, a total of 67 intra prediction modes). More specifically, the intra prediction mode set may include a planar mode, a DC mode, and multiple (eg, 65) angular modes (ie, directional modes). Each intra prediction mode may be indicated through a preset index (ie, an intra prediction mode index). For example, as shown in FIG. 6 , an intra prediction mode index 0 indicates a planar mode, and an intra prediction mode index 1 indicates a DC mode. In addition, intra prediction mode indices 2 to 66 may indicate different angular modes, respectively. The angle modes each indicate different angles within a preset angle range. For example, the angle mode may indicate an angle within an angle range between 45 degrees and -135 degrees in a clockwise direction (ie, the first angle range). The angle mode may be defined based on the 12 o'clock direction. At this time, the intra prediction mode index 2 indicates a horizontal diagonal (HDIA) mode, the intra prediction mode index 18 indicates a horizontal (HOR) mode, and the intra prediction mode index 34 indicates a diagonal (DIA) mode. mode, an intra prediction mode index of 50 indicates a vertical (VER) mode, and an intra prediction mode index of 66 indicates a vertical diagonal (VDIA) mode.

Meanwhile, the preset angle range may be set differently according to the shape of the current block. For example, when the current block is a rectangular block, a wide-angle mode indicating an angle exceeding 45 degrees or less than -135 degrees in a clockwise direction may be additionally used. If the current block is a horizontal block, the angle mode may indicate an angle within an angular range (ie, a second angle range) between (45+offset1) degrees and (-135+offset1) degrees clockwise. At this time, angle modes 67 to 76 outside the first angle range may be additionally used. In addition, when the current block is a vertical block, the angle mode may indicate an angle within an angular range (ie, a third angle range) between (45-offset2) and (-135-offset2) degrees clockwise. . At this time, angle modes -10 to -1 outside the first angle range may be additionally used. According to an embodiment of the present invention, the values of offset1 and offset2 may be determined differently according to the ratio between the width and height of the rectangular block. Also, offset1 and offset2 may be positive numbers.

According to a further embodiment of the present invention, the plurality of angular modes constituting the intra prediction mode set may include a basic angular mode and an extended angular mode. In this case, the extended angle mode may be determined based on the basic angle mode.

According to an embodiment, the basic angle mode is a mode corresponding to an angle used in intra prediction of an existing High Efficiency Video Coding (HEVC) standard, and the extended angle mode corresponds to an angle newly added in intra prediction of a next-generation video codec standard. It may be a mode that More specifically, the default angular mode is the intra prediction mode {2, 4, 6, ... , 66}, and the extended angle mode is an intra prediction mode {3, 5, 7, . . . , 65}. That is, the extended angular mode may be an angular mode between basic angular modes within the first angular range. Accordingly, an angle indicated by the extended angle mode may be determined based on an angle indicated by the basic angle mode.

According to another embodiment, the basic angle mode may be a mode corresponding to an angle within a preset first angle range, and the extended angle mode may be a wide angle mode outside the first angle range. That is, the default angle mode is the intra prediction mode {2, 3, 4, ... , 66}, and the extended angle mode is an intra prediction mode {-14, -13, -12, . . . , -1} and {67, 68, ... , 80}. An angle indicated by the extended angle mode may be determined as an angle opposite to an angle indicated by the corresponding basic angle mode. Accordingly, an angle indicated by the extended angle mode may be determined based on an angle indicated by the basic angle mode. Meanwhile, the number of expansion angle modes is not limited thereto, and additional expansion angles may be defined according to the size and/or shape of the current block. Meanwhile, the total number of intra prediction modes included in the intra prediction mode set may vary according to the configuration of the basic angular mode and the extended angular mode.

In the above embodiment, the interval between the extended angle modes may be set based on the interval between the corresponding basic angle modes. For example, extended angle modes {3, 5, 7, ... , 65} corresponds to the corresponding basic angle modes {2, 4, 6, ... , 66}. Also, the extended angle modes {-14, -13, . . . , -1} the corresponding opposite fundamental angle modes {53, 53, ... , 66}, and the expansion angle modes {67, 68, . . . , 80} corresponds to the opposite fundamental angle modes {2, 3, 4, ... , 15}. An angular interval between extended angular modes may be set to be the same as an angular interval between corresponding basic angular modes. Also, the number of extended angular modes in the intra prediction mode set may be set to be less than or equal to the number of basic angular modes.

According to an embodiment of the present invention, the extended angle mode may be signaled based on the basic angle mode. For example, the wide-angle mode (ie, the extended angle mode) may replace at least one angle mode (ie, the basic angle mode) within the first angle range. The default angular mode that is replaced may be an angular mode that corresponds to the opposite side of the wide-angle mode. That is, the replaced basic angle mode is an angle mode corresponding to an angle in a direction opposite to the angle indicated by the wide angle mode or an angle different from the angle in the opposite direction by a preset offset index. According to an embodiment of the present invention, the preset offset index is 1. The intra prediction mode index corresponding to the replaced basic angle mode may be mapped back to the wide-angle mode to signal the corresponding wide-angle mode. For example, wide-angle mode {-14, -13, ... , -1} is the intra prediction mode index {52, 53, ... , 66}, and the wide-angle mode {67, 68, . . . , 80} is the intra prediction mode index {2, 3, ... , 15}, respectively. As such, the intra prediction mode index for the basic angular mode signals the extended angular mode, so even if the configurations of the angular modes used for intra prediction of each block are different, the same set of intra prediction mode indexes are used for intra prediction mode signaling. can be used Accordingly, signaling overhead according to a change in intra prediction mode configuration can be minimized.

Meanwhile, whether to use the extended angle mode may be determined based on at least one of the shape and size of the current block. According to an embodiment, if the size of the current block is larger than a preset size, the extended angle mode is used for intra prediction of the current block, otherwise only the basic angle mode is used for intra prediction of the current block. According to another embodiment, when the current block is a non-square block, the extended angle mode is used for intra prediction of the current block, and when the current block is a square block, only the basic angle mode is used for intra prediction of the current block.

The intra prediction unit determines reference samples to be used for intra prediction of the current block and/or interpolated reference samples based on intra prediction mode information of the current block. When the intra prediction mode index indicates a specific angle mode, a reference sample corresponding to the specific angle from the current sample of the current block or an interpolated reference sample is used to predict the current pixel. Accordingly, different sets of reference samples and/or interpolated reference samples may be used for intra prediction according to the intra prediction mode. After intra prediction of the current block is performed using the reference samples and the intra prediction mode information, the decoder restores sample values of the current block by adding the residual signal of the current block obtained from the inverse transform unit to the intra prediction value of the current block. .

Motion (motion) information used for inter prediction may include reference direction indication information (inter_pred_idc), reference picture indices (ref_idx_l0, ref_idx_l1), and motion (motion) vectors (mvL0, mvL1). Reference picture list utilization information (predFlagL0, predFlagL1) may be set according to the reference direction indication information. As an embodiment, in the case of unidirectional prediction using the L0 reference picture, predFlagL0 = 1 and predFlagL1 = 0 may be set. In the case of unidirectional prediction using the L1 reference picture, predFlagL0 = 0 and predFlagL1 = 1 may be set. In the case of bidirectional prediction using both L0 and L1 reference pictures, predFlagL0 = 1 and predFlagL1 = 1 may be set.

When the current block is a coding unit, the coding unit may be divided into several sub-blocks, and prediction information of each sub-block may be the same or different. As an embodiment, when a coding unit is an intra mode, the intra prediction modes of each sub-block may be the same or different. Also, when a coding unit is in an inter mode, motion information of each sub-block may be identical to or different from each other. Also, each sub-block may be independently encoded or decoded. Each sub-block may be identified through a sub-block index (sbIdx).

The motion vector of the current block is highly likely to be similar to the motion vectors of neighboring blocks. Accordingly, motion vectors of neighboring blocks may be used as motion vector predictors (mvp), and motion vectors of the current block may be derived using motion vectors of neighboring blocks. In addition, in order to increase the accuracy of the motion vector, a motion vector difference (mvd) between an optimal motion vector of the current block found as an original image and a motion prediction value may be signaled by the encoder.

The motion vector may have various resolutions, and the resolution of the motion vector may vary on a block-by-block basis. The motion vector resolution may be expressed in integer units, half-pixel units, 1/4 pixel units, 1/16 pixel units, 4 integer pixel units, and the like. Since an image such as screen content is in the form of a simple graphic such as text, an interpolation filter does not need to be applied, and thus an integer unit and an integer pixel unit of 4 may be selectively applied in block units. Blocks encoded in affine mode capable of expressing rotation and scale vary greatly in shape, so integer units, 1/4 pixel units, and 1/16 pixel units can be selectively applied on a block basis. Information on whether to selectively apply motion vector resolution in block units is signaled as amvr_flag. If applied, which motion vector resolution to apply to the current block is signaled by amvr_precision_idx.

In the case of a block to which bi-directional prediction is applied, the same or different weights between two prediction blocks may be applied when weight average is applied, and information about weights is signaled through bcw_idx.

In order to increase the accuracy of the motion prediction value, a merge or advanced motion vector prediction (AMVP) method may be selectively used in units of blocks. The merge method is a method of configuring the motion information of the current block to be the same as the motion information of neighboring blocks adjacent to the current block, and has the advantage of increasing the encoding efficiency of motion information by propagating motion information spatially without change in a homogeneous motion domain. there is On the other hand, the AMVP method is a method of predicting motion information in L0 and L1 prediction directions, respectively, and signaling the most optimal motion information in order to express accurate motion information. After deriving motion information for the current block through AMVP or Merge method, the decoder uses a reference block located in motion information derived from a reference picture as a prediction block for the current block.

A method of deriving motion information in Merge or AMVP may be a method in which a motion candidate list is constructed using motion prediction values derived from neighboring blocks of the current block, and then index information on an optimal motion candidate is signaled. In the case of AMVP, since motion candidate lists for L0 and L1 are derived, optimal motion candidate indices (mvp_l0_flag and mvp_l1_flag) for L0 and L1 are signaled. In the case of merge, since one motion candidate list is derived, one merge index (merge_idx) is signaled. Motion candidate lists derived from one coding unit may vary, and a motion candidate index or merge index may be signaled for each motion candidate list. In this case, a mode in which there is no information about a residual block in a block encoded in the Merge mode may be referred to as a MergeSkip mode.

Symmetric MVD (SMVD) is a method of reducing the amount of bits of transmitted motion information by making Motion Vector Difference (MVD) values of L0 and L1 directions symmetrical in the case of bi-directional prediction. MVD information in the L1 direction that is symmetrical with the L0 direction is not transmitted, and reference picture information in the L0 and L1 directions is not transmitted and is derived in the decoding process.

OBMC (Overlapped Block Motion Compensation) generates prediction blocks for the current block using the motion information of neighboring blocks when the motion information between blocks is different, and then performs weight averaging of the prediction blocks to obtain a final prediction block for the current block. way to create This has an effect of reducing a blocking phenomenon occurring at a block boundary of a motion compensated image.

In general, merge motion candidates have low motion accuracy. In order to increase the accuracy of such a merge motion candidate, a Merge mode with MVD (MMVD) method may be used. The MMVD method is a method of correcting motion information using one candidate selected from several motion difference value candidates. Information on a compensation value of motion information obtained through the MMVD method (eg, an index indicating one selected from among motion differential value candidates) may be included in a bitstream and transmitted to a decoder. Compared to including the conventional motion information difference value in the bitstream, the amount of bits can be saved by including the information on the compensation value of the motion information in the bitstream.

The TM (Template Matching) method is a method of compensating motion information by constructing a template using neighboring pixels of a current block and finding a matching area having the highest similarity with the template. Template matching (TM) is a method of performing motion prediction in a decoder without including motion information in a bitstream in order to reduce the size of an encoded bitstream. In this case, the decoder may roughly derive motion information for the current block using the already reconstructed neighboring blocks since there is no original image.

The DMVR (Decoder-side Motion Vector Refinement) method is a method of correcting motion information through correlation of previously reconstructed reference images to find more accurate motion information. This is a method of using, as a new bi-directional motion, a point where the reference blocks in a reference picture are best matched within a predetermined area. When such DMVR is performed, the encoder corrects motion information by performing DMVR in one block unit, then divides the block into sub-blocks and performs DMVR in each sub-block unit to correct motion information of the sub-block again. This can be referred to as MP-DMVR (Multi-pass DMVR).

The Local Illumination Compensation (LIC) method is a method of compensating for a luminance change between blocks. After deriving a linear model using neighboring pixels adjacent to the current block, the luminance information of the current block is compensated for through the linear model.

Since conventional video encoding methods perform motion compensation considering only vertical, horizontal, and horizontal movements, encoding efficiency is reduced when encoding videos including motions such as enlargement, reduction, rotation, etc. commonly encountered in real life. In order to express motions for such enlargement, reduction, and rotation, an Affine model-based motion prediction technique using a 4 (rotation) or 6 (magnification, reduction, rotation) parameter model may be applied.

BDOF (Bi-Directional Optical Flow) is used to correct a prediction block by estimating a change amount of a pixel based on an optical flow from a reference block of a block composed of bidirectional motion. The motion of the current block may be corrected using the motion information derived from the BDOF of the VVC.

Prediction refinement with optical flow (PROF) is a technique for improving the accuracy of affine motion prediction in sub-block units to be similar to that of pixel-unit motion prediction. Similar to BDOF, PROF is a technique for obtaining a final prediction signal by calculating correction values in units of pixels for pixel values affine motion compensated in units of sub-blocks based on optical-flow.

The CIIP (Combined Inter-/Intra-picture Prediction) method, when generating a prediction block for the current block, weights the prediction block generated by the intra-prediction method and the prediction block generated by the inter-prediction method to obtain the final prediction block. how to create

An intra block copy (IBC) method is a method in which a part most similar to a current block is found in an already reconstructed region within a current picture, and a corresponding reference block is used as a prediction block for the current block. In this case, information related to a block vector, which is a distance between the current block and the reference block, may be included in the bitstream. The decoder may calculate or set a block vector for the current block by parsing information related to the block vector included in the bitstream.

The BCW (Bi-prediction with CU-level Weights) method does not generate a prediction block as an average for two prediction blocks motion-compensated from different reference pictures, but adaptively applies weights on a block-by-block basis to compensate for motion. This is a method of performing a weighted average on two predicted blocks.

A multi-hypothesis prediction (MHP) method is a method of performing weight prediction through various prediction signals by transmitting additional motion information to unidirectional and bidirectional motion information during inter-screen prediction.

Cross-component linear model (CCLM) is a method of constructing a linear model using a high correlation between a luminance signal and a chrominance signal located at the same position as the luminance signal, and then predicting the chrominance signal through the linear model. After constructing a template using a restored block among neighboring blocks adjacent to the current block, parameters for the linear model are derived through the template. Next, the current luminance block selectively reconstructed according to the size of the chrominance block according to the image format is downsampled. Finally, the chrominance block of the current block is predicted using the downsampled luminance block and the corresponding linear model. At this time, a method of using two or more linear models is called a multi-model linear mode (MMLM).

In independent scalar quantization, a reconstructed coefficient t' _k for an input coefficient t _k depends only on a related quantization index q _k . That is, a quantization index of a certain reconstructed coefficient has a different value from quantization indices of other reconstructed coefficients. In this case, t' _k may be a value including a quantization error in t _k , and may be different or the same according to quantization parameters. Here, t' _k may be referred to as a reconstructed transform coefficient or an inverse quantized transform coefficient, and a quantization index may be referred to as a quantized transform coefficient.

In Uniform Reconstruction Quantizers (URQ), reconstructed coefficients have a characteristic of being equally spaced. In this case, the distance between two adjacent reconstruction values may be referred to as a quantization step size. Among the reconstructed values, 0 may be included, and the entire set of usable reconstructed values may be uniquely defined according to the size of the quantization step. The quantization step size may vary depending on the quantization parameter.

In the existing method, a set (set) of allowable reconstructed transform coefficients is reduced due to quantization, and the elements of this set may be finite. For this reason, there is a limit to minimizing the average error between the original image and the reconstructed image. Vector quantization can be used as a method for minimizing this average error.

A simple vector quantization method used in video encoding includes sign data hiding. This is a method in which the encoder does not encode the sign of one non-zero coefficient, and the decoder determines the sign of the corresponding coefficient according to whether the sum of the absolute values of all coefficients is an even number or an odd number. To this end, in the encoder, at least one coefficient may be increased or decreased by '1', which is selected so that at least one coefficient is optimal in terms of cost for rate-distortion, and the value is can be adjusted As an example, a coefficient having a value close to the boundary of the quantization interval may be selected.

Another vector quantization method includes trellis-coded quantization, and in video encoding, it is used as an optimal path search technique for obtaining an optimized quantization value in dependent quantization. On a block-by-block basis, quantization candidates for all coefficients in the block are placed in the Trellis graph, and the optimal Trellis path between the optimized quantization candidates is considered at the cost of rate-distortion. to explore Specifically, dependent quantization applied to video encoding may be designed such that a set of allowable reconstructed transform coefficients for a transform coefficient depends on a value of a transform coefficient that precedes the current transform coefficient in the reconstruction order. In this case, by selectively using a plurality of quantizers according to transform coefficients, an average error between an original image and a reconstructed image is minimized, thereby increasing coding efficiency.

Among the intra prediction encoding technologies, the MIP (Matrix Intra Prediction) method is a matrix-based intra prediction method. Unlike prediction methods that have directionality from pixels of neighboring blocks adjacent to the current block, the MIP (Matrix Intra Prediction) method is a matrix-based predefined matrix matrix This is a method of obtaining a prediction signal using the offset value and .

In order to derive the intra prediction mode of the current block, based on a template, which is an arbitrary region adjacent to the current block and reconstructed, the intra prediction mode for the template derived through the neighboring pixels of the template is reconstructed of the current block. can be used for First of all, a decoder may generate a prediction template for a template using neighboring pixels (references) adjacent to the template, and may use an intra prediction mode in which a prediction template most similar to an already reconstructed template is generated to reconstruct a current block. This method may be referred to as template intra mode derivation (TIMD).

In general, an encoder may determine a prediction mode for generating a prediction block and generate a bitstream including information about the determined prediction mode. The decoder may set the intra prediction mode by parsing the received bitstream. In this case, the amount of bits of information about the prediction mode may be about 10% of the size of the entire bitstream. In order to reduce the amount of bits of information about the prediction mode, the encoder may not include information about the intra prediction mode in the bitstream. Accordingly, the decoder may derive (determine) an intra prediction mode for reconstruction of the current block using characteristics of neighboring blocks, and may reconstruct the current block using the derived intra prediction mode. At this time, in order to derive the intra prediction mode, the decoder infers directional information by applying Sobel filters in horizontal and vertical directions to neighboring pixels (pixels) adjacent to the current block, and converts the directional information into the intra prediction mode. A mapping method can be used. A method in which a decoder derives an intra prediction mode using neighboring blocks may be described as decoder side intra mode derivation (DIMD).

Neighboring blocks may be spatially positioned blocks or temporally positioned blocks. Neighboring blocks that are spatially adjacent to the current block are Left (A1) blocks, Left Below (A0) blocks, Above (B1) blocks, Above Right (B0) blocks, or Above Left (Above Left) blocks. , B2) block. A neighboring block temporally adjacent to the current block may be a block including a top left pixel position of a bottom right (BR) block of the current block in a collocated picture. If a neighboring block temporally adjacent to the current block is coded in intra mode or if a neighboring block temporally adjacent to the current block exists in an unusable position, the horizontal and vertical directions of the current block in a collocated picture corresponding to the current picture A block including the center (Ctr) pixel location of may be used as a temporal neighboring block. Motion candidate information derived from a corresponding picture may be referred to as Temporal Motion Vector Predictor (TMVP). Only one TMVP can be derived from one block, and after one block is divided into several sub-blocks, each TMVP candidate can be derived for each sub-block. A TMVP derivation method in units of sub-blocks may be referred to as a sub-block temporal motion vector predictor (sbTMVP).

Whether or not the methods described in this specification will be applied depends on slice type information (eg, I slice, P slice, or B slice), whether it is a tile, whether it is a sub picture, the size of the current block, the depth of the coding unit, and the current block. It may be determined based on at least one of information about whether the luminance block is a chrominance block, whether it is a reference frame or a non-reference frame, a temporal layer according to a reference order and a layer, and the like. Information used to determine whether the methods described in this specification are to be applied may be information previously agreed between a decoder and an encoder. Also, these pieces of information may be determined according to profiles and levels. Such information may be expressed as a variable value, and information on the variable value may be included in the bitstream. That is, the decoder may determine whether the above methods are applied by parsing information on variable values included in the bitstream. For example, whether the above-described methods are to be applied may be determined based on a horizontal length or a vertical length of a coding unit. If the horizontal length or the vertical length is 32 or more (eg, 32, 64, 128, etc.), the above methods can be applied. In addition, the above-described methods may be applied when the horizontal or vertical length is less than 32 (eg, 2, 4, 8, or 16). In addition, when the horizontal length or the vertical length is 4 or 8, the above methods can be applied.

8(a) shows a process of outputting new motion information by revising motion information derived from neighboring blocks of a current block. Referring to FIG. 8(a), the decoder may obtain corrected motion information by correcting motion information of neighboring blocks of the current block using various motion compensation methods. Referring to FIG. 8(b), the decoder derives a motion candidate list from neighboring blocks of the current block, and then corrects one or more motion candidates in the derived motion candidate list using various motion compensation methods to obtain a corrected motion candidate list. can The motion candidate list may be constructed using motion information derived from neighboring blocks of the current block. The decoder may obtain a corrected motion candidate list including the corrected one or more motion candidates by performing a motion correction process on each or all of the one or more motion candidates in the motion candidate list. Referring to FIG. 8(c), the decoder may obtain corrected motion information by correcting initial motion information of the current block using various motion compensation methods.

9 is a diagram illustrating a method of correcting motion information for a current block by recursively performing a motion correction method according to an embodiment of the present invention.

The decoder may correct initial motion information for the current block by recursively performing the motion compensation method. The decoder may construct a motion candidate list for the current block using neighboring blocks of the current block and recursively perform one or more motion compensation methods to correct motion information. At this time, one or more motion compensation methods include Motion Vector Difference (MVD), Template Matching (TM), Bilateral Matching (BM), Merge mode with MVD (MMVD), Merge mode with MVD (MMVD)-based TM, Optical flow-based TM , Multi pass DMVR, etc. MVD may be a method in which an encoder generates a correction value for motion information by including it in a bitstream, and a decoder obtains a correction value for motion information through the bitstream to correct motion information (MV difference value in FIG. 9). correction). TM may be a method in which a decoder constructs a template based on neighboring pixels of a current block, finds a matching region having the highest similarity to the configured template, and corrects motion information. BM may be a method in which a decoder corrects motion information based on a similarity between a reference block in a picture included in the L0 picture list derived based on the motion information of the current block and a reference block in a picture included in the L1 picture list. The MMVD method is a method of correcting motion information using one of one or more motion difference value candidates. The encoder may generate a bitstream including information about an index representing one of one or more motion difference value candidates. The decoder may obtain a difference value candidate indicated by the index by parsing information about an index included in the bitstream, and correct motion information based on the obtained difference value candidate. In the MMVD-based TM method, an extended motion candidate list constructed using a motion candidate list and one or more motion difference value candidates is rearranged based on a TM cost value, and motion information of a current block is obtained using a motion candidate in the rearranged list. way to correct it. The encoder may generate a bitstream containing information about an index representing any one of the candidates in the rearranged list. The decoder may parse information about an index included in the bitstream and use a motion compensation candidate indicated by the corresponding index as a compensation value of motion information of the current block. The optical flow-based TM method may be a method in which a decoder configures a template of an area adjacent to a current block as an optical flow map, finds an area similar to the optical flow map of a reference picture, and corrects motion information.

One or more motion compensation methods may be applied in merge or AMVP mode. Referring to FIG. 9 , the decoder may derive (construct) a motion candidate list (eg, a merge candidate list) for a current block. Also, the decoder may correct each or all of the motion information (eg, merge candidates) in the motion candidate list using one or more motion compensation methods. The decoder may rearrange the motion candidate list based on a cost value of the corrected motion information. As described above, the decoder may perform the above-described correction method on each or all of the motion candidates in the rearranged motion candidate list and rearrange the motion candidate list. That is, the decoder may recursively perform correction of the above-described motion candidates and rearrangement of the motion candidate list. When this method is applied, accuracy of motion information for a motion candidate in the motion candidate list increases, and thus, a residual signal may be reduced, which may have an effect of reducing the amount of bits for the residual signal. The decoder may separately signal an index indicating one of the motion candidates in the rearranged motion candidate list and predict (reconstruct) a current block based on the motion candidate indicated by the index. Alternatively, the decoder may select a motion candidate having the lowest cost value and predict (reconstruct) the current block based on the motion candidate having the lowest cost value.

In order to increase the accuracy of the merge candidate, the encoder may generate a bitstream including information about an index representing one of correction values of motion information acquired using a Merge mode with MVD (MMVD) method, and the decoder A correction value of motion information may be obtained through an index obtained by parsing information on an index included in a bitstream, and a correction value of motion information may be used to predict (reconstruct) a current block. The MMVD method is a method of selecting one of a plurality of motion difference value candidates, and has an effect of saving a large amount of bits, although the accuracy is somewhat less than that of accurately transmitting the existing motion information difference value. In order to slightly increase the accuracy, the decoder uses TM, BM, MMVD (Merge mode with MVD), MMVD (Merge mode with MVD) to the first corrected motion information that is corrected based on the correction value of the motion information obtained using the MMVD method. Second correction motion information may be obtained by additionally applying at least one method of the based TM, the optical flow-based TM, and the multi-pass DMVR method. Alternatively, the encoder corrects motion information by applying at least one method of TM, BM, Merge mode with MVD (MMVD), Merge mode with MVD (MMVD)-based TM, Optical flow-based TM, and multi-pass DMVR method. Afterwards, a bitstream may be generated by additionally including information on a correction value for motion information obtained by applying at least one of the above methods.

For AMVP, a difference value of motion information may be included in a bitstream. The decoder may generate a prediction block for the current block using a difference value of motion information included in the bitstream. Since the difference value of the motion information is included in the bitstream, there is a problem in that the amount of bits is increased. To this end, the above-described method can also be applied to the AMVP candidate list. That is, each or all of the one or more candidates in the candidate list obtained using AMVP are TM, BM, MMVD (Merge mode with MVD), MMVD (Merge mode with MVD)-based TM, Optical flow-based TM, and multi pass DMVR. It can be corrected based on at least one method. Since the corrected motion information is used, there is an effect of reducing the amount of bits for difference values of motion information included in an actual bitstream.

The MVD method is performed first, and a correction method using at least one of TM, BM, MMVD (Merge mode with MVD), MMVD (Merge mode with MVD)-based TM, Optical flow-based TM, and multi pass DMVR Can be performed there is. Alternatively, the encoder may generate a bitstream including correction values for initial motion information using MVD. The decoder may perform the above-described motion compensation method based on a compensation value for initial motion information included in the bitstream.

Hereinafter, MVD (Motion Vector Difference), TM (Template Matching), BM (Bilateral Matching), MMVD (Merge mode with MVD), MMVD (Merge mode with MVD)-based TM, Optical flow-based TM, Multi pass DMVR, etc. Explain how to apply.

A motion compensation method to be used may be determined based on an encoding mode (prediction mode) of the current block. For example, when the current block is coded in the GPM mode, first, at least one of TM, BM, and optical flow-based TM methods may be performed on MV differential values corrected using MMVD. For example, when the current block is coded in AMVP mode, at least one of TM, BM, and optical flow-based TM methods may be performed on MV differential values corrected using MMVD of merge mode. In this case, MVD of AMVP mode may not be applied.

For example, when the motion information of neighboring blocks adjacent to the current block is the same or similar to each other, correction of the MV difference value is not performed, and at least one of TM, BM, and optical flow-based TM methods may be performed. This is because the motion of the current block is highly likely to be similar to that of neighboring blocks. Meanwhile, when the distribution of motion information of neighboring blocks adjacent to the current block is not similar to each other, at least one of TM, BM, and optical flow-based TM methods may be performed on the MV difference value corrected using MVD or MMVD. . This is because the motion of the current block may be different from that of neighboring blocks.

For example, the size of the current block, whether the current block is a luminance component block or a chrominance component block, quantization parameter information for the current block, motion resolution information of the current block, whether a differential signal exists in the current block, current block A motion correction method (e.g., Motion Vector Difference (MVD), Template Matching (TM), Bilateral Matching (BM), Optical flow-based TM, multi pass DMVR, etc.) can be selected. When the size of the current block is larger than a certain size or the motion resolution of the current block is 1/16 pixel unit, the TM method may not be selected. This is because the TM method has a high complexity. When the current block is a chrominance component block, the TM method is not performed, and motion information corrected through the TM method in the luminance component block of the current block may be used. For example, motion information corrected through the TM method in the luminance component block of the current block may be scaled and used in the chrominance block of the current block.

For example, a motion compensation method (e.g., MVD (Motion Vector Difference), TM (Template Matching), BM (Bilateral Matching), MMVD (Merge mode with MVD), MMVD (Merge mode with MVD) according to the characteristics of the current block based TM, Optical flow based TM, multi pass DMVR, etc.) can be selected. This is because there exists a trade-off between complexity and accuracy for each motion compensation method. For example, TM has high complexity and cannot be parallelized, but shows the highest performance. BM has lower performance than TM but can be parallelized. Optical flow has low complexity and can be parallelized, but has high performance. There is a downside to being low. In this case, the selected motion compensation method may be separately signaled. For example, a decoder may determine a motion compensation method based on a syntax element included in a bitstream. In this case, the syntax element may be signaled at the SPS level, PPS level, picture level, slice level, or CU (Coding Unit) level.

Referring to FIG. 10, the decoder can obtain initial motion information (initial MV, reference index) derived from neighboring blocks. The decoder may set a search region within the reference image based on the initial motion information. The decoder can select several candidate positions within the search area according to a predefined search pattern. The decoder may construct a template for the current block using neighboring blocks of the current block, and construct a reference image template having the same size as the template for the current block based on the candidate position. The decoder may obtain a cost value between the template for the current block and the reference picture template. The cost value may be obtained through Sum of Absolute Differences (SAD) or Mean-Removed SAD (MRSAD). The decoder may obtain cost values for all candidate positions within the search area, and may use information of a motion candidate of a position corresponding to the minimum cost value as final motion information (enhanced motion information in FIG. 10 ). In this specification, the meaning of obtaining a cost value may be the same as meaning that a decoder calculates a cost value.

Referring to FIG. 11, in order to find a reference block corresponding to the current block in a reference picture, the decoder may set a position shifted by initial motion information (initial MV) based on the upper left position of the current block as the position of the reference block. The decoder may set a search range of an arbitrary (m x n) size based on the upper left position of the reference block. In this case, m x n may be 16 x 16. For example, the search area may have a range of -8 to 8 in a horizontal direction and a range of -8 to 8 in a vertical direction based on the position of the initial motion information. Specifically, if the position indicated by the initial motion information is expressed in the form of horizontal and vertical coordinates, it may be (x, y). In this case, the coordinates in the horizontal direction of the search area may range from x−8 to x+8, and the coordinates in the vertical direction may range from y−8 to y+8. Thereafter, the decoder may construct a left template of the current block and an upper template of the current block using blocks adjacent to the current block. Also, the decoder configures the left template of the reference block and the upper template of the reference block based on the position of the reference block set in the reference picture. In this case, the left template of the current block and the left template of the reference block may have the same size, and the size of the upper template of the current block and the upper template of the reference block may be the same.

Referring to FIG. 12 , the position of a motion candidate searched for in the search area for the TM method may be set based on the position of initial motion information (center point in FIG. 12 ). A position where a motion candidate is searched may vary according to a search pattern. The search pattern may include a diamond pattern, a cross pattern, and the like. In FIG. 12 , ◇ may indicate a location where a motion candidate is searched according to a diamond pattern, and + may indicate a location where a motion candidate is searched according to a cross pattern. The interval of the pattern may be set to become wider or narrower as the distance from the initial motion information increases.

Specifically, FIG. 13 is a flowchart illustrating a search process for a location where a motion candidate for the TM method is searched for. Referring to FIG. 13 , the decoder may obtain (calculate) a pixel (pixel) based cost value for initial motion information. If the number of iterations of search performance is '0', the decoder may end without performing the TM method. Otherwise (ie, if the number of iterations of performing the search is not '0'), the decoder may perform the TM method. In this case, the number of repetitions may be an arbitrary integer value of 1 or more. Also, the number of repetitions, the initial search pattern, and the initial search interval may be set before performing the search process described with reference to FIG. 13 . For example, the number of repetitions may be set to '375', the initial search pattern to 'DIAMOND', and the initial search interval to '6'.

Next, the search pattern and search interval may be reset. The search pattern and search interval are at least one of the size of the current block, whether the current block is a luminance component block or a chrominance component block, the current motion resolution, the number of iterations, and the distribution of cost values for motion candidate positions calculated in the previous iteration step. can be determined based on Hereinafter, a method of setting a search pattern and a search interval will be described.

The search interval may be determined according to the motion information resolution of the current block. The motion information resolution may be units of 1 integer pixels, 4 integer pixels, 1/2 integer pixels, 1/4 integer pixels, and 1/16 integer pixels. When the motion information resolution is 1/4 integer pixels, the initial search interval may be set to 6, and in other cases, the initial search interval may be set to 4.

The search pattern may be determined as a diamond pattern or a cross pattern. The search interval may be adjusted by decreasing or increasing an arbitrary interval from the initial search interval. For example, the search pattern and search interval may vary depending on the iteration step. The repetition step may indicate how many times the resetting of the search pattern and search interval is repeated when the number of repetitions is not 0. That is, search patterns and search intervals may vary depending on the number of repetition steps. For example, in the first iteration step, the search pattern and search interval can be set to the diamond search pattern and the initial search interval. In the second step, the search pattern and search interval may be set to a search interval reduced by 1 from the cross pattern and the initial search interval. In steps after the second step, a search pattern and a search interval may be set to a cross pattern and a search interval reduced by 1 from the previous step.

The search pattern and search interval may be set according to the color component of the current block. The search pattern and search interval for the chrominance component may be set wider than the search pattern and search interval for the luminance component. This is because the spatial correlation between the luminance component (signal) and the chrominance component (signal) is high. Alternatively, in order to improve performance, the search pattern and search interval for the chrominance component may be set shorter than the search pattern and search interval for the luminance component.

A search pattern and a search interval may be set based on the size of the current block. When the size of the current block is larger than a predetermined value, a cross pattern is used as a search pattern and a search interval may be set wider than the initial search interval. For example, the search interval may be '7'. Alternatively, if the size of the current block is larger than a predetermined number to improve performance, a diamond pattern may be used as a search pattern and the search interval may be set shorter than the initial search interval. For example, the search interval may be '5'. The size of the current block may be 16x16 or 32x32, and the search pattern and search interval may be set based on the sum of the horizontal and vertical sizes of the current block.

Next, the decoder may set a correction value (offset) for the position of the motion candidate to be searched for using the search pattern and the search interval, and perform evaluation on the searched motion candidates. Evaluation in this specification may mean obtaining a cost value. A position where a motion candidate is searched may vary according to a search pattern. For example, if the search pattern is a cross pattern, the correction values are (0, 1), (1, 0), (0, -1), (-1, 0), and if the search pattern is a diamond pattern, Calibration values are (0, 2), (1, 1), (2, 0), (1, -1), (0, -2), (-1, -1), (-2, 0), is (-1, 1). In this case, the correction values (x, y) are (horizontal, vertical), where x may be a correction value for a horizontal direction and y may be a correction value for a vertical direction.

The method described with reference to FIG. 13 may be recursively performed for a predetermined number of repetitions. For example, if the number of iterations is 1, it may be performed more than once. If the decoder searches for motion candidates and evaluates all of the searched motion candidates, it may use motion information of a motion candidate corresponding to the smallest cost value as final motion information.

The initial motion candidate of FIG. 13 may be reset based on the smallest cost value in the previous iteration step. An initial motion candidate in a next step may be reset based on a motion candidate corresponding to the smallest cost value in the first step. For example, if the motion candidate corresponding to the smallest cost value is the upper-left motion candidate in the first step, the decoder may evaluate a motion candidate adjacent to the upper-left motion candidate in the next iteration step.

The search process described with reference to FIG. 13 may proceed differently depending on whether the current block is a coding unit (block) or a sub-block (sub-coding block).

When the current block is a coding block and the AMVP mode is applied to the current block, an L0 motion candidate list for L0 prediction of the current block and an L1 motion candidate list for L1 prediction of the current block may be derived. Corrected motion information may be derived by performing a search process on some or all motion candidates of the derived candidate list. Meanwhile, when the current block is a coding block and the merge mode is applied to the current block, one motion candidate list for L0 and L1 prediction of the current block may be derived. A search process may be performed for some or all motion candidates in one derived motion candidate list. L0 described herein may mean L0 prediction, and L1 may mean L1 prediction.

L0 uni-prediction, L1 uni-prediction, or bi-prediction may be applied to the current block. Whether L0 uni-prediction, L1 uni-prediction, or bi-prediction is applied to the current block may be indicated by reference direction indication information. Reference direction indication information may be reset based on a cost value. For example, bi-directional prediction is performed using the cost value of the prediction block generated using the initial motion information of L0, the cost value of the prediction block generated using the initial motion information of L1, and the initial motion information of L0 and L1. A cost value of a prediction block generated by weight averaging two prediction blocks, a cost value of a prediction block generated using corrected motion information of L0, a cost value of a prediction block generated using corrected motion information of L1, Bi-directional prediction is performed through the corrected motions of L0 and L1, and motion information and reference direction indication information corresponding to the smallest cost value among the cost values of the prediction block generated by averaging the weight of the two prediction blocks are reset to the current block. can

A current coding block can be divided into several sub-blocks. Initial motion information of each sub-block may be reset to corrected motion information according to a search process. A template may be different for each sub-block, and a pixel (pixel) of an adjacent sub-block may be used as a template. However, since the decoder can search for the next subblock only when the adjacent subblock is reconstructed, there is a problem in that the search process is not processed in parallel for each subblock. To this end, a search process may be performed only for subblocks located at the boundary of the current block. Alternatively, for subblocks located at the boundary of the current block, the decoder derives corrected motion information using the TM method, and for subblocks not located at the boundary of the current block, the decoder uses BM, optical flow-based TM, and multi-pass DMVR method. Corrected motion information may be derived using one or more of the above.

When the current block is processed as a coding block, the decoder may calculate a cost value for the entire current block using initial motion information for L0 and L1, and derive corrected motion information based on the calculated cost value. . In this case, the motion of a part of the lower right corner of a block processed as a coding block may be slightly different from the entire motion of the coding block. Depending on how the template is configured, the discovery process may vary and the corrected motion information may also vary. Therefore, even if the current block is processed as a coding block, corrected motion information can be derived in units of sub-blocks based on a cost value for a template configured based on sub-blocks.

Specifically, FIG. 14 is a flowchart illustrating a process of evaluating a search candidate using correction values for candidate positions selected according to the search pattern and search interval described with reference to FIG. 13 . The decoder may store initial motion information as final motion information. Also, the decoder may perform a process described later on correction values for all search candidates.

The decoder can select one of the correction values for the candidate position to be searched for. The correction value may be reset to suit the motion resolution to be used in the current block. The decoder may reconstruct motion information to be evaluated by adding the reset correction value to the initial motion information. A cost value may be obtained based on the reconstructed motion information. At this time, the cost value obtained based on the motion information is obtained by adding a difference between absolute values of components in the horizontal direction and absolute values of components in the vertical direction between the initial motion information and the reconstructed motion information, and then multiplying by an arbitrary weight value. can be calculated. Any weight value may be '4'. The decoder may calculate a pixel (pixel)-based cost value for the reconstructed motion information only when the cost value obtained based on the motion information is smaller than the cost value of the initial motion information obtained based on the pixel (pixel).

After evaluating correction values for all search candidates, the decoder may set motion information corresponding to the smallest cost value as final motion information.

Each of the cost values obtained based on the motion information may be obtained through a difference value between pieces of motion information obtained by adding a correction value to each of a plurality of motion candidates corresponding to a position selected according to the initial motion information. A cost value obtained based on motion information may vary according to the size of a correction value. That is, the smaller the correction value, the smaller the cost value may be. Since motion information corresponding to the smallest cost value is set as final motion information, evaluation may be performed only on motion candidates surrounding a position indicated by initial motion information having a small correction value. However, a motion candidate having a large correction value may be an optimal motion candidate. Accordingly, in order to select an optimal motion candidate by evaluating various motion candidates, a cost value may be obtained using a method described later. The cost value may be obtained using a difference between motion information values of neighboring blocks, a quantization parameter, a size of a current block, and the like.

A cost value may be obtained using a distribution of motion information of neighboring blocks. For evaluation of various motion candidates, a decoder may obtain a cost value using a difference value between corrected motion information and motion information of a neighboring block. For example, a cost value may be obtained according to a comparison result by comparing a difference value between the corrected motion information and motion information of a neighboring block with a predetermined value. Specifically, when the difference value between the corrected motion information and the motion information of the neighboring block is larger than (or smaller than or equal to) a predetermined value, the decoder may obtain a cost value. In this case, the neighboring block may be a neighboring block adjacent to the current block or a temporal neighboring block located at the same location as the current block in a collocated picture.

A cost value may be obtained based on the size of the current block. For example, a weight for obtaining the aforementioned cost value may be set according to the size of the current block. The weight may be set in inverse proportion to the size of the current block. That is, the larger the size of the current block, the lower the weight may be set. This is to evaluate a wider range of motion candidates in order to select an appropriate motion candidate. Meanwhile, the weight may be set in proportion to the size of the current block. That is, the larger the size of the current block, the higher the weight may be set. This is to reduce complexity. For example, the size of the current block may be 16x16 or 32x32, and may be set to the sum of the horizontal and vertical sizes of the current block. The weight can be an integer value such as 1, 2, 3, 4, 5, or 6. Also, since the cost value increases as the weight value increases, the decoder may not perform evaluation to obtain the cost value when the weight value is greater than or equal to a specific value.

In general, motion information of a current block has a high correlation with motion information of neighboring blocks. However, when the current block is located at the boundary of the object, the current block may be a background area. In this case, the movement of neighboring blocks may be different from the movement of the current block. Also, in the case of an image captured without changing the position (viewpoint) of the camera, since the background does not move, motion information about the background may be a zero vector (0, 0). The decoder may search for motion candidates for TM using motion information of neighboring blocks. In this case, when the current block corresponds to the background, searching for a zero vector may be more effective than searching for motion candidates using motion information of neighboring blocks. Therefore, when TM is performed, a zero vector may be included in a motion candidate to be searched for.

In the case of an image captured while the camera moves at a constant speed, motion information on the background part may have a constant value (global motion). In this case, a global motion candidate may be included in motion candidates searched for for TM.

A reference picture for the current picture may be configured differently according to an environment in which content is used. For example, in a real-time broadcasting environment, a reference picture such as low delay B having a low delay rate may be used. That is, only a decoded picture located in the past in the reproduction time order of pictures can be used as a reference picture. In addition, since random access is important in a streaming service such as Video on Demand (VoD), a reference picture configuration such as random access in which I and P pictures are coded in units of Group of Pictures (GOP) may be used. That is, decoded pictures located in the past and in the future in the reproduction time order of pictures can be used as reference pictures. Since a method of constructing a reference picture for the current picture is signaled through a slice header, the decoder can check pictures to be used as reference pictures when coding the current picture. In this case, reference pictures used in units of blocks may be different, and information on which reference picture is used may be signaled in units of blocks.

A reference picture is a picture including a block most similar to the current block, and a decoder can find a reference picture through template matching. Information related to the reference picture is not included in the bitstream and can be derived through template matching.

First, the decoder can select reference picture candidates to be evaluated and motion candidates to be evaluated. Next, the decoder can evaluate the motion candidates for each reference picture. In this case, each motion candidate may be scaled based on a distance between the reference picture and the current picture and a distance between the current picture and reference pictures of the motion candidates (ie, Picture Order Count (POC) difference). A decoder may perform evaluation on scaled motion candidates. If evaluation is completed for combinations of all reference picture candidates and all motion candidates, motion information of a motion candidate of a reference picture corresponding to the smallest cost value may be final motion information. For example, when the current picture is a B picture (when bidirectional prediction is used), the decoder obtains motion information of a motion candidate of a reference picture corresponding to the smallest cost value and a reference picture corresponding to the second smallest cost value. Bi-directional motion information may be configured using motion information of a motion candidate of . Alternatively, final motion information may be obtained using a motion candidate of each reference picture having the smallest cost value among reference pictures in each reference picture list. The number of reference pictures in the reference picture list may be limited in terms of complexity. In this case, the limited number of reference pictures may be set through separate signaling.

In another method, if evaluation is completed for combinations of all reference picture candidates and all motion candidates, the encoder constructs a list of combination information, rearranges it in ascending order based on the cost value, and then determines the optimal combination information. It is possible to generate a bitstream including index information for The decoder may determine a reference picture or motion candidate for the current block based on index information included in the bitstream.

16(a) shows a reference picture configuration in a random access environment. Referring to FIG. 16(a), a reference block of a first reference picture (reference index 0) among L0 reference pictures may have the smallest cost value, and a second reference picture (reference index 1) among L1 reference pictures may have the smallest cost value. A reference block may have the smallest cost value. The decoder uses the motion information of the reference block of the first reference picture (reference index 0) among the L0 reference pictures and the motion information of the reference block of the second reference picture (reference index 1) of the L1 reference pictures to determine the current block. Motion information for can be obtained. In this case, reference picture index information for L0 and L1 may be derived through template matching without being signaled through a bitstream.

16(b) shows a reference picture configuration in a low-delay environment. Referring to FIG. 16(b), the L0 reference picture and the L1 reference picture may be configured identically. A reference block of a second reference picture (reference index 1) among L0 reference pictures may have the smallest cost value, and a reference block of a third reference picture (reference index 2) excluding the L0 reference block among L1 reference pictures can have this smallest cost value. The decoder uses the motion information for the reference block of the second reference picture (reference index 1) among the L0 reference pictures and the motion information for the reference block of the third reference picture (reference index 2) among the L1 reference pictures to determine the current block. Motion information for can be obtained. In this case, reference picture index information for L0 and L1 may be derived through template matching without being signaled through a bitstream.

The number of reference pictures in the reference picture list to which the above method is applied can be limited. Complexity can be reduced by applying only within a limited number of reference pictures. The limited number may be set to a predetermined value or signaled at the SPS level, PPS level, picture level, slice level, or CU (Coding Unit) level to be included in the bitstream, and the decoder obtains the limited number from the bitstream and adapts Ideally, the number of reference pictures in the reference picture list can be changed.

Referring to FIG. 17 , first, a motion candidate may be derived from neighboring blocks of a current block.

A first initial motion candidate may be derived through an upper block of the current block. A position for searching for a motion candidate within a reference picture may be selected based on the first initial motion candidate. A location where a motion candidate is searched may be a location indicated by ◇ (diamond pattern) or + (cross pattern). The decoder may perform evaluation in the order of a motion candidate at a position farthest from an initial motion candidate and a motion candidate at a position closest to the initial motion candidate. In addition, the position of the motion candidate corrected in the previous iteration step may be set as an initial motion candidate position in the next iteration step. Motion information of a motion candidate corresponding to the smallest cost value among motion candidates may become a first corrected motion candidate 1710 .

Next, a second initial motion candidate may be derived through the left block of the current block. The decoder may derive the second corrected motion candidate 1720 corresponding to the smallest cost value using template matching.

A corrected motion candidate corresponding to a smaller cost value between the cost value of the first corrected motion candidate 1710 and the cost value of the second corrected motion candidate 1720 becomes the final motion candidate for the current block.

A more accurate final motion candidate can be obtained as the range in which motion candidates are searched is wide and the number of locations to be searched is large. Even in the TM described in this specification, different results may be derived depending on the search range and search position. Referring to FIG. 18, when the distributions of initial motion information (1801, 1802, 1803) derived from neighboring blocks of the current block are far apart from each other, since searched positions do not overlap each other, a wide range of motion candidates can be evaluated. and the most optimal motion information can be derived.

The decoder may further evaluate motion candidates of the new position. Hereinafter, a new position of a motion candidate to be additionally evaluated will be described.

A motion candidate at a center position derived using motion information derived from neighboring blocks of the current block may be additionally evaluated. That is, the decoder may additionally evaluate a motion candidate corresponding to a central position of two or more motion candidates.

The decoder may additionally evaluate motion information derived from neighboring blocks of the current block and motion candidates at different positions. The decoder may additionally evaluate a motion candidate indicating an empty space in the global search range after setting a global search range of a larger area including all of the search ranges for each of the initial motion candidates. First, the decoder may set regions including all reference blocks indicated by motion candidates derived from neighboring blocks of the current block as the global search range. Alternatively, the decoder may set a pre-promised range as the global search range based on the position of the upper left pixel of the current block and the size of the current block. The global search range is divided into four equal parts, and when the number of motion candidates in each quadrant is smaller than a predetermined number, the center position of the quadrant corresponding to the number of motion candidates smaller than the predetermined number is used as a new motion candidate position. After being set, the decoder can perform motion candidate evaluation of the new motion candidate position.

If the current block is coded in Merge mode, the decoder can perform additional evaluation on all motion candidates that can be expressed by the MMVD method. MMVD is a distance table for predetermined motion differential values and a code for a vertical or horizontal direction. This is a method of defining a table and signaling index information on the distance and index information on the direction of difference values. All motion candidates that can be derived through MMVD can be used for TM. In this case, a motion candidate corresponding to the smallest cost value among motion candidates may be used for prediction of the current block without separate signaling. Alternatively, the encoder constructs a motion candidate list using all motion candidates that can be induced through MMVD, rearranges the motion candidates in the motion candidate list in ascending order based on cost values, and assigns an index to an optimal motion candidate. It is possible to obtain a bitstream including information about. The decoder can predict (reconstruct) a current block through a motion candidate indicated by the index by parsing information about the index from the bitstream.

When the pieces of

initial motion information

1901 , 1902 , and 1903 are close together, a part of the search position set by the pieces of initial motion information may overlap. However, since the location where the motion candidates are searched is more subdivided and the evaluation of the motion candidates can be performed, there is an effect that precise motion information can be obtained. This can be applied in an environment (process) where more precise motion information is required.

The distribution of initial motion information is determined according to motion information of neighboring blocks of the current block. Accordingly, the decoder can adjust the search range by correcting the position of the initial motion information when the initial motion information is adjacent to each other or too far apart. At this time, the distribution of motion information may be determined based on a specific value. For example, when the distance between pieces of motion information is larger than a specific value, the pieces of motion information may be determined to be separated, and when the distance between pieces of motion information is smaller than a specific value, the pieces of motion information may be determined to be attached to each other. In this case, the specific value is a value that varies according to motion resolution and may be an integer or a decimal value.

FIG. 20 shows an example of changing the initial motion information so that it becomes closer when the initial motion information is far apart, and FIG. 21 shows an example of changing the initial motion information so that when the initial motion information is close to each other, they are separated. 20 and 21, dotted arrows indicate initial motion candidates (information) (Initial MV Candidate 1, Initial MV Candidate 2), and solid arrows indicate changed motion candidates (information) (Modified MV Candidate 1, Modified MV Candidate 2). indicates A dotted circle represents a search area according to an initial motion candidate, and a solid line circle represents a search area according to changed motion information. By adjusting the horizontal or vertical value of the initial motion information, the motion information may be changed to become closer or apart.

The decoder may search for motion candidates at positions separated by a predetermined interval based on initial motion information. At this time, any predetermined interval may be set in various ways. A motion candidate at a position changed by a predetermined interval in the horizontal direction may be searched for, a motion candidate at a position changed by a predetermined interval in the vertical direction may be searched, and a motion candidate at a position changed by a predetermined interval in the horizontal and vertical directions may be searched for. A motion candidate may be searched for. For example, FIG. 22(a) shows a basic position where a motion candidate derived through initial motion information is searched for. 22(b) shows positions of motion candidates to be searched that are changed by a predetermined interval in the vertical direction. Referring to FIG. 22(b), the position where the motion candidate is searched may be reduced by 1/2 in the vertical direction. 22(c) shows positions of motion candidates to be searched that have been changed by a predetermined interval in the horizontal direction. Referring to FIG. 22(c), the position where the motion candidate is searched may be reduced by 1/2 in the horizontal direction.

23 is a diagram illustrating a method of changing a search position where a motion candidate is searched according to an embodiment of the present invention.

Referring to FIG. 23 , a location where a motion candidate is searched for may be adaptively set using motion information (Initial MV Candidate 1, Initial MV Candidate 2) derived from neighboring blocks of the current block. A search interval and a search position for motion candidate search may be changed to positions of points where derived motion information is connected to each other in a straight line. The decoder may change a position where a motion candidate is searched for by reducing the first search range for the first initial motion information in at least one of a vertical direction and a horizontal direction. Further, the decoder may rotate the second search range for the second motion information so that a motion candidate search position within the second search range is aligned with a changed motion candidate search position for the first initial motion information. Then, the decoder may evaluate motion candidates of the changed motion candidate search positions. Similarly, the decoder may change the position where the motion candidate is searched for by reducing the second search range for the second initial motion information in at least one of the vertical direction and the horizontal direction. Further, the decoder may rotate the first search range for the first motion information so that a motion candidate search position within the first search range is aligned with a changed motion candidate search position for the second initial motion information. Then, the decoder may evaluate motion candidates of the changed motion candidate search positions. Even if a position where a motion candidate is searched for is within the search range, if it is located outside the picture, the motion candidate located outside the picture is not evaluated. When the search range is changed (decreased or increased), a location where a motion candidate is searched for may not match the motion resolution. In this case, a location where a motion candidate is searched may be adjusted to match a nearby motion resolution.

Since the TM uses a cost value obtained using neighboring pixels (pixels) adjacent to the current block, neighboring blocks of the current block must be reconstructed. Therefore, since the current block and neighboring blocks are dependent on each other, TMs cannot be processed in parallel, which is disadvantageous to processing speed. To solve this problem, it is necessary to remove the dependency between the current block and neighboring blocks. Specifically, in order to remove the dependence, the decoder may obtain a cost value using motion information by inducing motion information of neighboring blocks in units of pixels instead of obtaining a cost value using neighboring pixels (pixels) of the current block. . In this case, motion information of neighboring blocks can be reconstructed in parallel as the decoder parses the bitstream, so there is an effect that inter-blocks can be reconstructed in parallel. That is, the decoder can perform block reconstruction using TM in parallel.

Referring to FIG. 24 , a search area and location for searching motion candidates may be set using initial motion information (Initial MV Candidate) derived from neighboring blocks of the current block. In order to obtain a cost value of a motion vector, an optical flow MV map in units of pixels (pixels) may be generated using motion information around a reference block indicated by initial motion information. An optical flow MV template in units of pixels (pixels) may also be generated through neighboring motion information for neighboring templates adjacent to the current block. The decoder may obtain an optical flow MV reference template most similar to the current block MV template among locations for motion candidate search and use it as corrected motion information. A cost value between MV templates can be obtained in various ways. For example, a cost value between MV templates may be calculated through an absolute sum of MV differences in a horizontal or vertical direction. In order to reduce complexity, a 2x2 or 4x4 MV template may be derived instead of a pixel unit.

Hereinafter, a method of correcting motion information using DMVR will be described.

DMVR is a method of acquiring corrected motion information for a current block using a Bilateral Matching (BM) method. The BM (Bilateral Matching) method finds the most similar parts in the search area of the L0 reference block and the search area of the L1 reference block in blocks having bidirectional motion, corrects the initial motion information, and corrects the corrected motion. This is a method of using information for prediction of the current block. The size of the search area can be set to an arbitrary (m x n) size based on a specific point of the reference block. For example, the specific point may be the top-left position of the reference block or the center position of the reference block, and may have an arbitrary size of 16 x 16. The most similar part may be a point corresponding to the smallest cost value by calculating a cost value in units of pixels between blocks. The cost value may be calculated using Sum of Absolute Differences (SAD) or Mean-Removed SAD (MRSAD). Depending on the search area, the cost value may vary, and the corrected motion information may also vary. The decoder can divide the current block into a plurality of sub-blocks and correct motion information using DMVR for each sub-block. This is because motion information for small-sized blocks is more accurate than for large-sized blocks. In this case, DMVR may not be performed on a block having a large size, and DMVR may be performed only on a divided small block (eg, a sub-block). Referring to FIG. 26, one block may be divided into four sub-blocks. The decoder can obtain corrected motion information using DMVR for each divided sub-block. Alternatively, the decoder may use a multi-DMVR method in which more accurate motion information is derived through DMVR in a small block divided by using corrected motion information found through DMVR in a large block.

The multi-DMVR method will be described below.

After the parsing step of the current coding block, before starting decoding of the current coding block, the decoder determines whether to perform multi-DMVR correction by determining conditions described below in the step of deriving motion information for the current coding block. can Conditions for performing multiple DMVR are i) when motion in merge mode is bi-directional, ii) when distances between reference pictures and the current picture are the same, iii) when there is a motion differential value in merge mode, iv) weight between reference blocks This may be the case of a block to which prediction is not applied. The decoder may perform multi-DMVR when any one of conditions i) to iv) is satisfied.

If the motion difference between the first merge motion candidate in the merge candidate list of the current coding block and the other merge candidates is smaller than a certain threshold, multi-DMVR may not be performed. In addition, when there are a plurality of merge motion candidates different from the first merge motion candidate in the merge candidate list, if there is even one case where the motion difference is smaller than a certain threshold value, multiple DMVR may not be performed. At this time, among the merge candidates of the merge candidate list, only a merge candidate indicated by an index lower than an index indicating a merge motion candidate of the current coding block may compare motion differences. This is to reduce complexity. That is, if motion differences between each of the merge motion candidates in the merge candidate list and the merge candidate of the current coding block are calculated, and all of the calculated motion differences are greater than a predetermined threshold value, the decoder may perform multi-DMVR. Comparing the similarity between the merge motion candidate of the current coding block and the merge motion candidates in the merge candidate list is to determine whether neighboring blocks have various motions. In general, if the motions of neighboring blocks are similar, the motion of the current block is highly likely to be similar to that of neighboring blocks. In this case, the neighboring blocks and the current block may be inferred as blocks within the object having the same motion. Conversely, if the motions of neighboring blocks are not similar and exhibit different motion characteristics, the neighboring blocks including the current coding block may not be the same object, and thus a more accurate motion search process may be required.

27(a) shows a general process of performing multiple DMVRs, and FIG. 27(b) shows a general process of multiple DMVRs in more detail.

Referring to FIG. 27(a), multiple DMVRs may obtain one or more corrected motion information by performing DMVR in units of coding units (blocks) based on initial motion information (S2710). If TM is applied to the current coding block, the decoder may perform TM using one or more corrected motion information obtained in step S2710 (S2720). If the corrected motion information of the current coding block determined by performing TM is changed from bidirectional to unidirectional, the DMVR process cannot be performed, so steps S2730 and S2740 after S2720 are not performed, and the motion of the current block is unidirectional is finally determined by As a result of performing the TM, if the corrected motion information of the current coding block is bi-directional, the decoder may perform DMVR in sub-block units to obtain corrected motion information in sub-block units for each sub-block (S2730). Further, the decoder may re-calibrate the corrected motion information in units of sub-blocks obtained in step S2740 based on the BDOF, and finally acquire motion information corrected based on the BDOF.

Meanwhile, if the smallest cost value among the cost values of the one or more pieces of corrected motion information acquired in step S2710 is greater than an arbitrary value, step S2730 may be performed. On the other hand, if the smallest cost value is equal to or smaller than an arbitrary value, the decoder stores one or more corrected motion information obtained in step S2710 in units of sub-blocks, resets the motion information of the current coding block to initial motion information, Steps to S2740 may be performed. At this time, one or more pieces of corrected motion information stored in units of sub-blocks may be used for motion correction in step S2740. Finally, by comparing the cost value of the coding block using the initial motion information and the cost value of the coding block using the motion corrected in each sub-block obtained by performing step S2740, the block unit having a small cost value and the motion information thereof are obtained. Finally, it can be used for the current coding block. An arbitrary value may be the number of pixels (pixels) in the current coding block or the size of the current coding block (either horizontal or vertical, or the sum of the horizontal and vertical dimensions).

Referring to FIG. 27(b), step S2710 of FIG. 27(a) is a step of performing DMVR in coding unit (block) units using integer-unit search (S2701) and coding using half-pixel unit search It can be subdivided into a step (S2702) of performing unit (block) unit DMVR. In this case, the motion resolution may be set in integer units in step S2701 and in half-pixel units in step S2702. In step S2701, the decoder may obtain corrected motion information using the initial motion information and an integer-unit correction value. In step S2702, the decoder may obtain corrected motion information using the initial motion information and the half-pixel unit correction value. The decoder may use the corrected motion information acquired in step S2701 as reference motion information for step S2702. That is, the decoder may generate a new motion candidate based on the corrected motion information obtained in step S2701 and evaluate the new motion candidate. In this case, if the cost value of the new motion candidate is greater than the cost value of the corrected motion information obtained in step S2701, the corrected motion information obtained in step S2701 may be an output of the coding unit (block) unit DMVR. Conversely, if the cost value of the new motion candidate is smaller than the cost value of the corrected motion information obtained in step S2701, the corrected motion information obtained in step S2702 can be an output of the coding unit (block) unit DMVR. That is, the corrected motion information corresponding to the smaller cost value between the cost value obtained in step S2701 and the cost value obtained in step S2702 can be used for the TM (S2703 and S2720). At this time, when cost values for all new motion candidates are calculated, complexity increases. To this end, the decoder may calculate a motion information (MV)-based cost value, and calculate a pixel (pixel)-based cost value for a new motion candidate only when the cost value obtained based on the motion information is smaller than an arbitrary value. . A cost value obtained based on motion information may be obtained by multiplying a weighted sum of absolute values of differences between horizontal and vertical component values between a new motion candidate and initial motion information. Also, the cost value obtained based on the motion information may be obtained using a difference between the new motion candidate and the corrected motion information obtained in step S2701.

A cost value may be obtained using a distribution of motion information of neighboring blocks. To evaluate various motion candidates, a decoder may obtain a cost value using a difference value between corrected motion information and motion information of a neighboring block. For example, a cost value may be obtained according to a comparison result by comparing a difference value between the corrected motion information and motion information of a neighboring block with a predetermined value. Specifically, when the difference value between the corrected motion information and the motion information of the neighboring block is larger than (or smaller than or equal to) a predetermined value, the decoder may obtain a cost value. In this case, the neighboring block may be a neighboring block adjacent to the current block or a temporal neighboring block located at the same location as the current block in a collocated picture.

A cost value may be obtained based on the size of the current block. For example, a weight for obtaining the aforementioned cost value may be set according to the size of the current block. The weight may be set in inverse proportion to the size of the current block. That is, the larger the size of the current block, the lower the weight may be set. This is to evaluate a wider range of motion candidates in order to select an appropriate motion candidate. Meanwhile, the weight may be set in proportion to the size of the current block. That is, the larger the size of the current block, the higher the weight may be set. This is to reduce complexity. For example, the size of the current block may be 16x16 or 32x32, and may be set to the sum of the horizontal and vertical sizes of the current block. The weight can be an integer value such as 1, 2, 3, 4, 5, 6, etc. Also, since the cost value increases as the weight value increases, the decoder may not perform evaluation to obtain the cost value when the weight value is greater than or equal to a specific value.

Hereinafter, step S2730 of FIG. 27(a) will be described in detail.

After the decoder divides the current coding block into several sub-blocks in step S2730, steps S2704 and S2705 of FIG. 27(b) may be performed for each sub-block. As for the width of the subblock, a value smaller than the width of the coding block and the preset maximum width may become the width of the subblock. The height of the sub-block may be compared with the height of the coding block and the preset maximum height, and the minimum value may be the height of the sub-block. In this case, the sub-block size may be set to a maximum of 16x16.

The decoder may set the corrected motion information obtained in step S2730 as initial motion information for steps S2704 and S2705. The decoder may perform global search in integer units using the initial motion information, and obtain optimal motion information for the current sub-block and a cost value for the optimal motion information (S2704). In order to obtain optimal motion information, a cost value for each piece of motion information for a sub-block may be obtained. If the cost value of an arbitrary sub-block is smaller than the arbitrary value, motion information on the current sub-block acquired through global search in integer units is stored and subsequent processes may not be performed. If the cost value of an arbitrary sub-block is equal to or greater than the arbitrary value, the cost value of the arbitrary sub-block may be set to another predetermined value. Any value can be the number of pixels of the current sub-block or the size of the current sub-block. For example, the size of a sub block may be 16 x 16. Also, another predetermined value may be a maximum value in a range of numbers that can be expressed by the encoder and the decoder. After step S2704, the decoder may perform 3x3 Square search in half-pixel (1/2) units. Motion information acquired through step S2704 may be used as reference motion information in step S2705. That is, a new motion candidate is obtained based on the information acquired through step S2704, and the decoder can evaluate the new motion candidate (S2705). The decoder may evaluate a new motion candidate and store final motion information for the current sub-block. Steps S2704 and S2705 may be repeatedly performed for all subblocks. The DMVR process in units of sub-blocks has an advantage in that all sub-blocks can perform DMVR in parallel because there is no dependency between sub-blocks.

Referring to FIG. 28 , the decoder may obtain initial motion information and set motion candidates to be searched for based on the obtained initial motion information. Further, the decoder may obtain corrected motion information based on a cost value obtained by evaluating the set motion candidates. Further, the decoder may obtain final corrected motion information using model-based fractional MVD optimization according to the motion information resolution of the current block. The corrected motion information obtained in steps S2701, S2702, and S2705 of FIG. 27(b) may be obtained through the method described with reference to FIGS. 28 and 29.

Hereinafter, with reference to FIG. 29, a method of obtaining the final corrected motion information described with reference to FIG. 28 will be described in more detail.

The decoder may acquire (calculate) a cost value for the motion information corrected in the previous step (S2901). In this case, if a cost value for the previously acquired corrected motion information exists, step S2901 may be omitted. The cost value obtained in step S2901 may be obtained in different ways according to motion resolution. For example, if the motion resolution is an integer unit, it may be obtained as a pixel-based cost value for the corrected motion information and used as a reference cost value. If the motion resolution is a half-pixel (1/2) unit, the decoder obtains a cost value based on the obtained motion information using the initial motion information and the corrected motion information and obtains a pixel-based cost value for the corrected motion information. . Next, if the motion information-based cost value is greater than the pixel-based cost value, the corrected motion information is stored and the next process is omitted without being performed. If the cost value based on motion information is equal to or smaller than the cost value calculated based on pixels (pixels), a cost value obtained by adding a cost value based on motion information to a cost value based on pixels (pixels) is used as the reference cost value. The decoder may set correction values for motion candidates to be searched for (S2902). Specifically, motion compensation candidate values may be derived according to a predetermined 3x3 square search pattern. In this case, the motion compensation candidate value may be set based on the motion resolution. Depending on the motion resolution, the motion compensation candidate value may be scaled to expand its range. For example, the horizontal and vertical directions of a given 3x3 Square search pattern are Mv(-1, 1), Mv(0, 1), Mv(1, 1), Mv(1, 0), Mv(1, -1) ), Mv(0, -1), Mv(-1, -1), or Mv(-1, 0). When the motion resolution is a half-pixel (1/2) unit, only odd-numbered evaluations may be performed and even-numbered evaluations may not be performed in the 3x3 Square search pattern in order to reduce calculation complexity. The decoder may set a new motion candidate using the corrected motion candidate and the motion compensation candidate value. The decoder may reconstruct motion information of a reference block of a picture in the L0 picture list and motion information of a reference block of a picture in the L1 picture list (S2904). In order to evaluate the L0 and L1 motions having linear characteristics, the decoder may generate a new motion candidate by adding the motion compensation candidate values to the corrected L0 motion candidate and differentiating the motion compensation candidate values from the corrected L1 motion candidate. there is. In addition, in order to evaluate L0 and L1 motions having nonlinear characteristics, the decoder may generate a new motion candidate by adding a motion compensation candidate value to the corrected L0 motion candidate without modifying the corrected L0 motion candidate. Alternatively, the decoder may generate a new motion candidate by adding motion compensation candidate values to the corrected L0 motion candidate without modifying the corrected L1 motion candidate. The decoder may calculate a cost value based on motion information for a new motion candidate (S2905). The decoder may compare a cost value based on motion information of a new motion candidate with a reference cost value (S2906). When the cost value based on the motion information is greater than the reference cost value, the decoder may store a value corresponding to twice the cost value based on the motion information in the memory. And, the decoder may repeatedly perform again from step S2902 without performing the process after step S2906. If the cost value based on the motion information is smaller than the reference cost value, the decoder may obtain a pixel-based cost value for a new motion candidate (S2907). In addition, the reference cost value may be updated to a value obtained by adding a cost value based on motion information and a pixel-based cost value for a new motion candidate. If the motion resolution is a half-pixel (1/2) unit, the decoder may repeatedly perform the process from step S2901 to evaluate another new motion candidate. That is, when the motion resolution is a half-pixel (1/2) unit, the decoder may store only the evaluation result for the new motion candidate and may not use it as an optimal motion candidate. The stored evaluation result (cost value) of the new motion candidate can be used in a model-based fractional MVD optimization step that can be performed later. If the motion resolution is an integer unit, if the value obtained by summing the motion information-based cost value for the new motion candidate and the pixel-based cost value is smaller than the pixel-based cost value for the initially corrected motion information , the decoder can store the new motion candidate as optimal motion information. If the motion resolution is an integer unit, the decoder may store the pixel-based cost value for the new motion candidate obtained in step S2907 as a cost value for optimal motion information. Thereafter, the decoder may check whether there is a motion candidate that has not yet been searched for (S2908). If there is a motion candidate that has not yet been searched for, the decoder may repeatedly perform step S2901 in order to evaluate a new motion candidate. A motion candidate to be determined may be stored as a final corrected motion (S2909). When the motion resolution is in the half-pixel (1/2) unit, the decoder performs a model-based fractional MVD optimization process using the cost values of the motion candidates stored in the previous step to recalculate the optimal half-pixel unit corrected motion information. It can (S2910). Step S2910 may be similar to a process of inducing motion in units of fractional pixels using a cost value based on pixels (pixels) in units of integers in the DMVR of Versatile Video Coding (VVC). The decoder may obtain optimal motion information using cost values of motion information adjacent to the position of the optimal motion information (left, right, up, down).

The search method described with reference to FIG. 29 may be repeatedly performed a predetermined number of times, and when it is repeatedly performed, the decoder may search only the surrounding positions of the optimal motion information acquired in the previous search.

Referring to FIG. 30 , the global search in integer units may be a search performed by dividing a search area into several areas centered on a central position and setting different weights for each area. The middle position of the search area may be the position of the upper left pixel (pixel) of the reference block indicated by the initial motion information. A weight for each zone may be set differently according to the motion characteristics of the neighboring blocks of the current block and the current block. For example, when similarity between motions of the current block and neighboring blocks of the current block is high, weights may be set to increase in order from a center area to an edge area. Conversely, when similarity between motions of the current block and neighboring blocks of the current block is low, weights may be set to decrease in order from the center area to the edge area. If the pixel-based cost value for the corrected motion information obtained in step S2710 of FIG. 27 (a) is smaller than an arbitrary value, integer-unit global search may not be performed. In addition, if the pixel-based cost value for an arbitrary motion compensation value within the search area is smaller than an arbitrary value, the arbitrary motion compensation value becomes the final corrected motion information, and the global search in integer units may be terminated. there is. An arbitrary value can be the number of pixels (pixels) of the current block or the size of the current block. For example, the size of the current block may be 16x16.

Referring to FIG. 31, the decoder may set motion candidates to be searched for based on input initial motion information. The decoder can then perform evaluation on the motion candidates to obtain the final corrected motion information. As shown in FIG. 30 , global search in integer units may be performed for all locations in each area in the order from the middle area to the edge area.

Hereinafter, global search in integer units will be described in more detail with reference to FIG. 32 . Referring to FIG. 32, the decoder may reset the search position for each area within the search area according to the size of the block (S3201). At this time, if the size of the block is smaller than a certain value, the decoder evaluates only the search region reduced by a certain value based on the center position of FIG. 30 in order to reduce computational complexity. The arbitrary value may be half the size of the current block or a positive integer, for example '8'. After calculating a pixel-based cost value using the initial motion information, it is set as the minimum cost value. The decoder may set a motion compensation value for a search position of a motion candidate and reset the motion compensation value according to a motion resolution to be used in the current block (S3202 and S3203). The decoder may set a new motion candidate using the motion compensation value reset in step S3203. The decoder may reconstruct motion information of a reference block of a picture in the L0 picture list and motion information of a reference block of a picture in the L1 picture list (S3204). In order to evaluate the L0 and L1 motions having linear characteristics, the decoder may generate a new motion candidate by adding the motion compensation candidate values to the corrected L0 motion candidate and differentiating the motion compensation candidate values from the corrected L1 motion candidate. there is. In addition, in order to evaluate L0 and L1 motions having nonlinear characteristics, the decoder may generate a new motion candidate by adding a motion correction candidate value to the corrected L1 motion candidate without modifying the corrected L0 motion candidate. In addition, the decoder may generate a new motion candidate by adding motion compensation candidate values to the corrected L0 motion candidate without modifying the corrected L1 motion candidate. The decoder may obtain a pixel-based cost value for a new motion candidate (S3205). In order to differentiate the cost value for each region, the decoder may reacquire the cost value using a predefined weight value for each region (S3206). The weight value is a value obtained through an experiment and may be a positive integer value. For example, the weight values are '1', '2', ... , may be '63'. The decoder may update the minimum cost value to the reacquired cost value if the reacquired cost value is smaller than the minimum cost value. The decoder may evaluate whether evaluation of all motion candidates has been completed (S3207). If there is a motion candidate whose evaluation has not yet been completed, it can be performed again from step S3202 using the minimum cost value. When evaluation of all motion candidates is completed, the decoder may set information of a motion candidate corresponding to the smallest cost value among cost values of each of the motion candidates as final corrected motion information. An early termination method may be applied to global search in integer units to reduce complexity. If the cost value reacquired in step S3206 is smaller than the minimum cost value and the difference value obtained by subtracting the minimum cost value from the reacquired cost value is smaller than an arbitrary value, step S3208 is performed without checking step S3207, and A global search may be terminated. An arbitrary value may be the size of the current block or the number of pixels, for example, 16x16.

The BDOF-based motion information correction in FIGS. 33 and 34 specifically represents the BDOF-based motion information correction in steps S2740 and S2706 of FIG. 27 .

Referring to FIG. 33 , after dividing a current block into sub-blocks, the decoder may acquire final corrected motion information by calculating a correction value of BDOF-based motion information based on input initial motion information. BDOF may be used to correct a prediction block by estimating a change amount of a pixel from a reference block of a block composed of bidirectional motion. The motion of the current block may be corrected using motion information derived from BDOF. If the current block is encoded in at least one mode among affine, LIC, OBMC, sub-block MC, CIIP, SMVD, BCW with different weight, and MMVD, BDOF-based motion correction may not be performed. Meanwhile, BDOF-based motion correction may be performed when any one of the following conditions is satisfied. The conditions for performing BDOF-based motion correction are that the current block is i) when the motion of the merge mode is bidirectional, ii) when the distance between the reference pictures and the current picture is the same, iii) when weight prediction between reference blocks is not applied. In the case of a block, iv) it may be a case where the size of the current block is greater than or equal to a predetermined size. Any given size may be the width or length of a block. For example, the block may have a horizontal size of '8' and a vertical size of '8'. In addition, BDOF-based motion compensation may be performed in units of sub-blocks, and the size of a sub-block may be up to 16x16.

Hereinafter, a process of performing BDOF-based motion correction for each sub-block will be described. Referring to FIG. 34 , motion information may be the same between adjacent subblocks, and BDOF may be performed only once by merging subblocks having the same motion information (S3401 and S3402). Depending on the size difference between the width and height of the current block, it can be determined whether sub-blocks are merged horizontally or vertically. For example, if the height is greater than the width of the current block, sub-blocks may be merged in a vertical direction, and if the width is greater than the height of the current block, sub-blocks may be merged in a horizontal direction. The decoder may obtain corrected motion information or a prediction block by repeatedly performing steps S3403 to S3414 for each sub-block or merged sub-blocks.

The decoder may obtain a BDOF parameter using the L0 reference block and the L1 reference block (S3403). The BDOF parameter may include a variation amount between pixel values. The decoder may divide the current block into 8x8 sub-blocks (S3404), and then repeatedly perform steps S3405 to S3412 for each sub-block.

The decoder may obtain a pixel-based cost value between L0 and L1 reference blocks (S3405). The decoder may compare the cost value with a preset limit value (S3406). If the cost value as a result of comparison is smaller than the preset threshold value, the decoder may generate a prediction block for the subblock using L0 and L1 reference blocks by substituting (0,0) for the motion compensation value without performing motion compensation. Yes (S3407, S3409). Conversely, if the cost value as a result of the comparison is equal to or greater than the preset limit value, the decoder can obtain a BDOF-based motion compensation value (S3408). The decoder may check whether the motion compensation value is (0,0) (S3410). If the motion compensation value is (0,0), the decoder can generate a prediction block based on BDOF, and if the motion compensation value is not (0,0), it can store the compensation value obtained in step S3408 (S3411, S3412). The corrected motion information stored in step S3412 may be the final corrected motion information of FIG. 33 .

The decoder can check whether steps S3403 to S3412 have been performed for every 8x8 subblock (S3413). If there is an 8x8 subblock in which steps S3405 to S3412 have not been performed, the decoder may perform steps S3405 to S3412 repeatedly. If steps S3405 to S3412 have been performed on all 8x8 subblocks, the decoder can check whether steps S3403 to S3412 have been performed on all 16x16 subblocks (S3414). If there is a 16x16 subblock in which steps S3403 to S3412 have not been performed, the decoder may repeatedly perform steps S3403 to S3412. If steps S3403 to S3412 are performed on all 16x16 sub-blocks, BDOF-based motion correction can be terminated. In this case, the 8x8 sub-block may be a sub-block constituting the 16x16 sub-block.

Referring to FIG. 35, the decoder may generate a prediction block for the current block using final corrected motion information obtained by performing the processes described in FIGS. 33 and 34. The decoder may divide the current block into sub-blocks and generate a prediction block for the current block based on BDOF using motion information finally corrected for each sub-block.

Hereinafter, a specific process of generating a prediction block based on BDOF will be described.

Referring to FIG. 36, the decoder may divide the current block into 8x8 sub-blocks (S3601). The decoder may obtain final corrected motion information by summing the corrected motion information obtained by performing steps S2730, S2704, or S2705 of FIG. 27 and the motion correction value obtained by performing step S2740 or S2706 (S3602). At this time, the decoder may check whether the motion compensation value obtained in step S2740 or S2706 of FIG. 27 is (0,0) (S3603). As a result of checking, if the motion compensation value is (0,0), the decoder can use the prediction block obtained in steps S3409 and S3411 of FIG. 34 as a prediction block for the luminance component (S3604). Also, if the motion compensation value is (0,0) as a result of the check, the decoder may generate a prediction block for the color difference component (S3606). Meanwhile, if the motion compensation value is not (0,0) as a result of the check, the decoder may generate prediction blocks for the luminance component and the chrominance component based on the BDOF (S3605). The decoder may check whether steps S3602 to S3606 have been performed for all 8x8 subblocks, and if there are subblocks that have not been performed, steps S3602 to S3606 may be repeatedly performed (S3607).

Hereinafter, various methods of performing the above-described multi-DMVR will be described with reference to FIG. 27 .

Whether each step of the DMVR described with reference to FIG. 27 is performed depends on the size of the current block, the color component of the current block (whether the current block is a luminance component block or a chrominance component block), quantization parameter information for the current block, and motion of the current block. It may be determined based on at least one of resolution information and information related to the presence or absence of residual signals of neighboring blocks of the current block. For example, when the current block is a chrominance component block or the residual signals of neighboring blocks of the current block are within a certain value, some of the steps of FIG. 27 or steps after a specific step may be omitted. This is to reduce the computational complexity.

Whether to perform each step of the DMVR described with reference to FIG. 27 may be determined according to the size of the current block. Since the number of divided sub-blocks increases as the block size increases, if the size of the current block is equal to or larger than an arbitrary value, some or all of the DMVR steps may not be performed. In this case, the arbitrary value may be the sum of the horizontal and vertical sizes of the current block, and may be specifically 128.

Meanwhile, if the size of the current block (eg, coding block) is larger than an arbitrary value, each step of the DMVR described with reference to FIG. 27 may be performed by setting the size of the sub-block to be proportionally divided. An arbitrary value at this time may be 16. If the current block is larger than a certain value, the sub-block may be divided into the expanded size by expanding the width and height of the sub-block by n (eg, a natural number) times the preset size. Alternatively, the maximum number of subblocks applicable to the current block may be set, and the width and height of the subblock may be determined based on the maximum number.

Depending on the allowable number of parallel processes of the video signal processing apparatus, whether to perform each step of the DMVR described with reference to FIG. 27 may be determined. This is because the number of resources (eg, processes and threads) that can be processed in parallel is limited for each video signal processing device. Therefore, under limited processable resources, whether DMVR is performed in stages may vary, and DMVR may be selectively applied to each block. For example, when the number of processable resources is equal to or smaller than an arbitrary value, part of each step of the DMVR described with reference to FIG. 27 may be omitted or steps after a specific step may be omitted. Also, DMVR may be applied only to a sub-block at a specific location among sub-blocks. A specific position may be an odd or even block among divided subblocks. Alternatively, as shown in FIG. 37, the sub-blocks to which DMVR is applied may be sub-blocks located at the upper left, upper right, lower left, and lower right. Motion information on a subblock to which DMVR is not applied may be obtained using corrected motion information of neighboring blocks of a block to which DMVR is applied.

The size of divided sub-blocks may be determined according to the number of resources that can be processed in parallel. As the number of divided sub-blocks increases, the processing speed of the video signal processing apparatus may decrease. The number of resources that can be processed in parallel and the size of a subblock may be inversely proportional. For example, as the number of resources that can be processed in parallel increases, the size of subblocks to be divided may decrease. That is, as the number of resources that can be processed in parallel increases, the number of divided subblocks may increase. Conversely, as the number of resources that can be processed in parallel decreases, the size of the divided sub-blocks may increase. In addition, whether the number of resources that can be processed in parallel is greater or less than an arbitrary value may be determined, and the size of a subblock may be determined according to the number of resources determined according to an arbitrary value. In addition, the size of a sub-block includes not only the number of resources that can be processed in parallel, but also quantization parameter information of the current block, size of the current block or color component (luminance or chrominance component) of the current block, motion resolution information of the current block, and It may be determined based on at least one of information related to the presence or absence of a residual signal of a neighboring block. In this case, the size of the subblock and the number of divided subblocks may be separately signaled. For example, a decoder may parse syntax elements included in a bitstream to determine the size of subblocks and the number of divided subblocks. In this case, the syntax element may be signaled at the SPS level, PPS level, picture level, slice level, or CU (Coding Unit) level.

In the multi-DMVR process, the motion compensation process may be performed only on the luminance component of the current block and may not be performed on the chrominance component. In this case, the corrected motion information for the luminance component of the current block acquired through multiple DMVRs may be used for the chrominance component of the current block.

The DMVR method is a method for finding more accurate motion information, and a decoder can generate one prediction block using the finally corrected motion information. Since DMVR is a method of correcting motion information through correlation of already reconstructed reference images, accuracy may be lower than motion information found in an actual original image. In order to compensate for such low accuracy, referring to FIG. 38, the decoder can generate several prediction blocks by selectively utilizing motion information for each step obtained in the process of performing DMVR. The decoder may finally generate a prediction block for the current block through a weight average between prediction blocks. The controller can determine whether to use the prediction blocks generated in each step, and can set the weight of each prediction block in the weight averaging step. Whether or not to use the prediction blocks for each step may be determined based on a pixel (pixel)-based cost value already obtained when generating the prediction block. Also, the number of prediction blocks used may be determined according to a result of comparing a cost value with an arbitrary value. For example, if the cost value is less than a certain value, X number of preset prediction blocks may be used, and if the cost value is greater than a certain value, Y number of prediction blocks may be used. In this case, X and Y may be the same or different integer values. The controller can also determine whether to perform each stage of DMVR. Whether DMVR is performed in each step is related to the quantization parameter of the current block, the size of the current block, the color component of the current block (whether a luminance component or a chrominance component), the motion resolution information of the current block, and the presence or absence of residual signals of blocks adjacent to the current block. It may be determined based on at least one of the pieces of information.

39 is a flowchart illustrating a method of predicting a current block according to an embodiment of the present invention.

39 illustrates a method of predicting a current block based on the cost value described with reference to FIGS. 1 to 38 .

Referring to FIG. 39 , the decoder may obtain a first motion information list including one or more pieces of first motion information related to a current block (S3910). The decoder may obtain one or more pieces of first corrected motion information by correcting the one or more pieces of first motion information (S3920). The decoder may obtain one or more first cost values for each of the one or more pieces of first corrected motion information (S3930). The decoder may obtain a second motion information list by rearranging the one or more pieces of first corrected motion information based on the first cost values (S3940). The decoder may obtain one or more pieces of second motion information based on the first motion information included in the second motion information list (S3950). The decoder may acquire one or more pieces of second corrected motion information by correcting the one or more pieces of second motion information (S3960). The decoder may obtain one or more second cost values for each of the second corrected motion information (S3970). The decoder may predict the current block based on the second motion information determined based on the second cost values (S3980).

In this case, the one or more first cost values may be values related to a similarity between the current block and a reference block corresponding to each of the one or more pieces of first corrected motion information. The one or more second cost values may be values related to similarity between the current block and a reference block corresponding to each of the one or more pieces of second corrected motion information.

The first corrected motion information pieces of the second motion information list may be arranged in an ascending order of cost values respectively corresponding to the one or more pieces of first corrected motion information.

The first motion information may be corrected motion information corresponding to the smallest of the first cost values, and the second motion information may be corrected motion information corresponding to the smallest of the second cost values. there is.

The one or more pieces of first motion information may be included in different pictures, and the one or more pieces of second motion information may be included in different pictures.

The one or more pieces of first corrected motion information and the one or more pieces of second corrected motion information are Motion Vector Difference (MVD), Template Matching (TM), Bilateral Matching (BM), Merge mode with MVD (MMVD), MMVD It may be corrected based on at least one of a (Merge mode with MVD)-based TM, an optical flow-based TM, and multi-pass decoder-side motion vector refinement (DMVR).

When the encoding mode of the current block is a merge mode, the one or more pieces of first motion information and the one or more pieces of second motion information may be motion information derived by merge mode with MVD (MMVD).

Blocks corresponding to each of the one or more pieces of first motion information are blocks located in a first search area, blocks corresponding to each of the one or more pieces of second motion information are blocks located in a second search area, and The first search area and the second search area may be different from each other.

The first cost values may be sequentially calculated for each of the one or more pieces of first corrected motion information, and the second cost values may be sequentially calculated for each of one or more pieces of second corrected motion information. In this case, when a cost value smaller than the preset first value is calculated while the first cost values are sequentially calculated, cost values for the remaining corrected motion information may not be calculated. If a cost value smaller than the preset second value is calculated while the second cost values are sequentially calculated, cost values for the remaining corrected motion information may not be calculated.

The methods described above in this specification may be performed through a processor of a decoder or encoder. Also, the encoder may generate a bitstream to be decoded by the above methods. In addition, the bitstream generated by the encoder may be stored in a computer-readable non-transitory storage medium (recording medium).

Although this specification is mainly described from the viewpoint of a decoder, the same operation may be performed in an encoder. The term parsing in this specification has been described focusing on the process of obtaining information from a bitstream, but from the encoder side, it can be interpreted as constructing corresponding information in a bitstream. Therefore, the term parsing is not limited to a decoder operation, but can also be interpreted as an act of constructing a bitstream in an encoder. In addition, such a bitstream may be configured by being stored in a computer readable recording medium.

The above-described embodiments of the present invention may be implemented through various means. For example, embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

In the case of hardware implementation, the method according to the embodiments of the present invention includes one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices) , Field Programmable Gate Arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.

In the case of implementation by firmware or software, the method according to the embodiments of the present invention may be implemented in the form of a module, procedure, or function that performs the functions or operations described above. The software code can be stored in memory and run by a processor. The memory may be located inside or outside the processor, and may exchange data with the processor by various means known in the art.

Some embodiments may be implemented in the form of a recording medium including instructions executable by a computer, such as program modules executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures or other data in a modulated data signal, such as program modules, or other transport mechanism, and includes any information delivery media.

The above description of the present invention is for illustrative purposes, and those skilled in the art can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, the embodiments described above are illustrative in all respects and should be construed as being limited. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts should be construed as being included in the scope of the present invention. do.

Claims

In the video signal decoding device,

contains a processor;

the processor,

Obtaining a first motion information list including one or more pieces of first motion information related to a current block;

Obtaining one or more pieces of first corrected motion information by correcting the one or more pieces of first motion information;

Obtaining one or more first cost values for each of the one or more pieces of first corrected motion information;

obtaining a second motion information list by rearranging the one or more pieces of first corrected motion information based on the first cost values;

Obtaining one or more pieces of second motion information based on first motion information included in the second motion information list;

Obtaining one or more pieces of second corrected motion information by correcting the one or more pieces of second motion information;

Obtaining one or more second cost values for each of the second corrected motion information;

and predicting the current block based on second motion information determined based on the second cost values.
According to claim 1,

The one or more first cost values are values related to similarity between the current block and a reference block corresponding to each of the one or more pieces of first corrected motion information,

The one or more second cost values are values related to a similarity between the current block and a reference block corresponding to each of the one or more pieces of second corrected motion information.
According to claim 1,

The second motion information list is arranged in ascending order of cost values respectively corresponding to the one or more pieces of first corrected motion information.
According to claim 1,

The first motion information is the video signal decoding device, characterized in that the corrected motion information corresponding to the smallest cost value among the first cost values.
According to claim 1,

The second motion information is the video signal decoding device, characterized in that the corrected motion information corresponding to the smallest cost value among the second cost values.
According to claim 1,

The one or more pieces of first motion information are included in different pictures,

The video signal decoding device, characterized in that the one or more pieces of second motion information are included in different pictures.
According to claim 1,

The one or more first corrected motion information and the one or more second corrected motion information,

MVD (Motion Vector Difference), TM (Template Matching), BM (Bilateral Matching), MMVD (Merge mode with MVD), MMVD (Merge mode with MVD) based TM, optical flow based TM, multi A video signal decoding device characterized in that it is corrected based on at least one or more of the pass DMVR (Multi pass Decoder-side Motion Vector Refinement).
According to claim 1,

When the coding mode of the current block is a merge mode,

The one or more pieces of first motion information and the one or more pieces of second motion information are motion information derived by merge mode with MVD (MMVD).
According to claim 1,

Blocks corresponding to each of the one or more pieces of first motion information are blocks located in a first search area,

Blocks corresponding to each of the one or more pieces of second motion information are blocks located in a second search area,

The video signal decoding device, characterized in that the first search area and the second search area are different from each other.
According to claim 1,

The first cost values are sequentially calculated for each of the one or more pieces of first corrected motion information,

The second cost values are sequentially calculated for each of the one or more pieces of second corrected motion information.
According to claim 10,

If a cost value smaller than the preset first value is calculated while the first cost values are sequentially calculated, cost values for the remaining corrected motion information are not calculated,

When the second cost values are sequentially calculated and a cost value smaller than the preset second value is calculated, cost values for the remaining corrected motion information are not calculated.
In the video signal encoding device,

contains a processor;

the processor,

Obtaining a bitstream decoded by a decoding method;

The decoding method,

obtaining a first motion information list including one or more pieces of first motion information related to a current block;

obtaining one or more pieces of first corrected motion information by correcting the one or more pieces of first motion information;

obtaining one or more first cost values for each of the one or more pieces of first corrected motion information;

acquiring a second motion information list by rearranging the one or more pieces of first corrected motion information based on the first cost values;

obtaining one or more pieces of second motion information based on first motion information included in the second motion information list;

correcting the one or more pieces of second motion information to obtain one or more pieces of second corrected motion information;

obtaining one or more second cost values for each of the second corrected motion information pieces; and

and predicting the current block based on second motion information determined based on the second cost values.
According to claim 12,

The second motion information list is arranged in ascending order of cost values respectively corresponding to the one or more pieces of first corrected motion information.
According to claim 12,

The first motion information is a video signal encoding device, characterized in that the corrected motion information corresponding to the smallest cost value among the first cost values.
According to claim 12,

The second motion information is the video signal encoding device, characterized in that the corrected motion information corresponding to the smallest cost value among the second cost values.
According to claim 12,

The one or more pieces of first motion information are included in different pictures,

The video signal encoding device, characterized in that the one or more pieces of second motion information are included in different pictures.
According to claim 12,

When the coding mode of the current block is a merge mode,

The one or more pieces of first motion information and the one or more pieces of second motion information are motion information derived by merge mode with MVD (MMVD).
According to claim 12,

The first cost values are sequentially calculated for each of the one or more pieces of first corrected motion information,

The second cost values are sequentially calculated for each of the one or more pieces of second corrected motion information.
According to claim 12,

If a cost value smaller than the preset first value is calculated while the first cost values are sequentially calculated, cost values for the remaining corrected motion information are not calculated,

If a cost value smaller than the preset second value is calculated while the second cost values are sequentially calculated, cost values for the remaining corrected motion information are not calculated.
A computer-readable non-transitory storage medium storing a bitstream, wherein the bitstream is decoded by a decoding method,

The decoding method,

obtaining a first motion information list including one or more pieces of first motion information related to a current block;

obtaining one or more pieces of first corrected motion information by correcting the one or more pieces of first motion information;

obtaining one or more first cost values for each of the one or more pieces of first corrected motion information;

acquiring a second motion information list by rearranging the one or more pieces of first corrected motion information based on the first cost values;

obtaining one or more pieces of second motion information based on first motion information included in the second motion information list;

obtaining one or more pieces of second corrected motion information by correcting the one or more pieces of second motion information;

obtaining one or more second cost values for each of the second corrected motion information pieces; and

and predicting the current block based on second motion information determined based on the second cost values.