RU2803564C2 - Method and device for mutual prediction based on details of motion vectors on the side of the decoder (dmvr) and bidirectional optical flow (bdof) - Google Patents

Abstract

FIELD: image encoding.

SUBSTANCE: invention is related to image encoding, and more particularly to a method and equipment for performing mutual prediction based on decoder-side motion vector refinement (DMVR) and bi-directional optical flow (BDOF). An image decoding method is provided in which DMVR is applied to extract detailed L0 and L1 motion vectors based on the L0 and L1 motion vectors of the current block; prediction samples of the current block are derived based on L0 prediction samples derived based on the detailed L0 motion vector and L1 prediction samples derived based on the detailed L1 motion vector; and reconstruction samples of the current block are generated based on the prediction samples, wherein the L0 prediction samples are derived based on the L0 reference frame and the detailed L0 motion vector, the L1 prediction samples are derived based on the L1 reference frame and the detailed L1 motion vector, and in the step for extracting predictive samples, whether or not to apply the BDOF extraction process is determined based on whether or not the first POC difference between the current frame and the L0 reference frame and the second POC difference between the current frame and the L1 reference frame are identical, wherein the prediction samples for the current block are derived by applying BDOF to the current block based on BDOF flag information indicating whether or not to apply BDOF to the current block.

EFFECT: increasing the efficiency of image encoding/decoding.

3 cl, 12 dwg, 30 tbl

Description

State of the art

Field of technology to which the invention relates

[1] This document relates to image coding technology, and more specifically to a method and equipment for performing inter-prediction based on decoder-side motion vector granularity (DMVR) and bidirectional optical flow (BDOF).

Description of the Prior Art

[2] Demands for high-quality images and high-definition video, such as ultra-high-definition (UHD) images and 4K or 8K or higher video, have recently been growing in various technology fields. As image data and video data have high resolution and high quality, the amount of information or the number of bits that are relatively transmitted increases compared to existing image data and video data. Accordingly, if image data is transmitted using a medium such as an existing wired or wireless broadband line, or image data and video data are stored using an existing storage medium, transmission costs and storage costs increase.

[3] Additionally, interests and demands for immersive media such as virtual reality (VR), artificial reality (AR) or hologram content have been growing recently. Broadcasting of images and videos having image characteristics different from those of real images, such as game images, is increasing.

[4] Accordingly, there is a need for high-performance image and video compression technology to efficiently compress and transmit or store and reproduce high-quality image and high-resolution video information having such various characteristics.

The essence of the invention

[5] This document provides a method and equipment for improving the efficiency of image encoding.

[6] This document also provides a method and equipment for efficient cross-forecasting.

[7] This document also provides a method and hardware for performing inter-prediction based on motion vector detail (DMVR) at the decoder side.

[8] This document also provides a method and hardware for performing inter-prediction based on bidirectional optical flow (BDOF).

[9] This document also provides a method and equipment for improving prediction performance by providing a condition for determining whether or not to apply DMVR to improve image coding efficiency and/or a condition for determining whether or not to apply BDOF.

[10] According to one exemplary embodiment of this document, a method for decoding images performed by decoding equipment is provided. The method includes: extracting a detailed L0 motion vector and a detailed L1 motion vector by applying decoder-side motion vector granularity (DMVR) based on the L0 motion vector and L1 motion vector of the current block; extracting predictive samples for the current block based on the predictive L0 samples extracted based on the detailed L0 motion vector and the predictive L1 samples extracted based on the detailed L1 motion vector; and generating reconstructed samples for the current block based on the predicted samples, wherein the predicted L0 samples are extracted based on the L0 reference frame and the detailed L0 motion vector, and the predicted L1 samples are extracted based on the L1 reference frame and the detailed L1 motion vector. wherein the extraction of the prediction samples comprises determining whether or not to apply a bidirectional optical flow (BDOF) extraction process based on whether or not the first POC difference between the current frame and the reference L0 frame and the second POC difference between the current frame and reference L1 frame.

[11] According to another exemplary embodiment of this document, an image encoding method carried out by encoding equipment is provided. The method includes: extracting a detailed L0 motion vector and a detailed L1 motion vector by applying decoder-side motion vector granularity (DMVR) based on the L0 motion vector and L1 motion vector of the current block; extracting predictive samples for the current block based on the predictive L0 samples extracted based on the detailed L0 motion vector and the predictive L1 samples extracted based on the detailed L1 motion vector; and extracting residual samples based on predictive samples; and encoding image information including information regarding residual samples; wherein the L0 prediction samples are extracted based on the L0 reference frame and the detailed L0 motion vector, and the L1 prediction samples are extracted based on the L1 reference frame and the detailed L1 motion vector, wherein extracting the prediction samples comprises determining whether or not do not apply a bidirectional optical flow (BDOF) extraction process based on whether the first POC difference between the current frame and the L0 reference frame and the second POC difference between the current frame and the L1 reference frame are identical.

[12] According to this document, the overall efficiency of image/video compression can be increased.

[13] According to this document, computational complexity can be reduced and overall coding efficiency can be improved through efficient inter-prediction.

[14] According to this document, various application conditions are proposed when applying DMVR and/or BDOF to detail motion information during the motion compensation process to improve efficiency in terms of complexity and performance.

Brief description of drawings

[15] FIG. 1 schematically illustrates an example of a video/image coding system that is applicable to embodiments of this document.

[16] FIG. 2 is a schematic explanation of a video/image encoding equipment circuit configuration that is applicable to embodiments of this document.

[17] FIG. 3 is a schematic explanation of a video/image decoding equipment circuit configuration that is applicable to embodiments of this document.

[18] FIG. 4 is a diagram for explaining an embodiment of a process for performing decoder-side motion vector refinement (DMVR) in true bi-prediction.

[19] FIG. 5 is a diagram for explaining an embodiment of a process for performing decoder-side motion vector refinement (DMVR) using sum of absolute differences (SAD).

[20] FIG. 6 illustrates an example showing a method for performing a decoding process by checking application conditions of DMVR and BDOF.

[21] FIG. 7 and 8 illustrate another example showing a method for performing a decoding process by checking application conditions of DMVR and BDOF.

[22] FIG. 9 is a diagram illustrated to explain the concept of BDOF.

[23] FIG. 10 is a flowchart schematically illustrating an encoding method that can be performed by encoding equipment according to an embodiment of this document.

[24] FIG. 11 is a flowchart schematically illustrating a decoding method that can be performed by decoding equipment according to an embodiment of this document.

[25] FIG. 12 illustrates an example of a content streaming system to which the embodiments disclosed herein are applicable.

Detailed Description of Embodiments

[26] This document may be modified in various ways and may have various embodiments, and in the following, specific embodiments are illustrated in the drawings and described in detail. However, it is not the intention to limit this document to specific embodiments. Terms broadly used in this detailed description are used to describe a particular embodiment and are not used to limit the technical substance of this document. Expressions in the singular include expressions in the plural unless otherwise obvious from the context. It should be understood that a term such as "include" or "have" in this detailed description indicates the existence of a feature, number, step, operation, element, part, or combination thereof set forth in the specification and does not exclude the existence or possibility of adding one or more other characteristics, numbers, steps, operations, elements, parts, or combinations thereof.

[27] Meanwhile, the elements described in the drawings herein are independently illustrated for convenience of description in connection with various characteristic functions. This does not mean that each of the elements is implemented as separate hardware or as separate software. For example, at least two elements may be combined to form one element, or one element may be divided into multiple elements. An embodiment in which elements are combined and/or separated is also included within the scope of this document if it does not depart from the spirit of this document.

[28] Hereinafter, preferred embodiments of this document are described in more detail with reference to the accompanying drawings. Hereinafter in this document, in the drawings, an identical reference number is used in the identical element, and redundant description of the identical element may be omitted.

[29] FIG. 1 schematically illustrates an example of a video/image coding system to which embodiments of this document may be applied.

[30] Referring to FIG. 1, a video/image encoding system may include a first device (source device) and a second device (receiver device). A source device may deliver encoded information or video/image data in file or streaming form to a receiving device via a digital storage medium or a network.

[31] The source device may include a video source, encoding equipment, and a transmitter. The receiving device may include a receiver, decoding equipment, and a rendering module. The encoding equipment may be referred to as "video/image encoding equipment" and the decoding equipment may be referred to as "video/image decoding equipment". The transmitter may be included in the encoding equipment. The receiver may be included in the decoding equipment. The rendering module may include a display, and the display may be configured as a separate device or an external component.

[32] A video source may obtain video/image through a video/image capture, synthesis, or shaping process. The video source may include a video/image capture device and/or a video/image generation device. The video/image capturing device may include, for example, one or more cameras, video/image archives including previously captured video/images, and the like. The video/image generating apparatus may include, for example, computers, tablet computers and smartphones and may (electronically) generate video/images. For example, a virtual video/image may be generated via a computer or the like. In this case, the video/image capturing process can be replaced by the associated data generation process.

[33] Encoding equipment can encode the input video/image. Encoding hardware may perform a sequence of procedures such as prediction, transform, and quantization to achieve compression and encoding efficiency. Encoded data (encoded video/image information) may be output in the form of a bitstream.

[34] A transmitter may transmit information or encoded image/image data output in the form of a bitstream to a receiver of a receiving device via a digital storage medium or network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, and the like. The transmitter may include an element for generating a multimedia file through a predefined file format and may include an element for transmission through a broadcast/communication network. The receiver may receive/extract the bit stream and transmit the received bit stream to decoding equipment.

[35] The decoding equipment can decode a video/image by performing a sequence of procedures such as dequantization, deconversion and prediction corresponding to the operation of the encoding equipment.

[36] The rendering module may prepare a decoded video/image by rendering. The video/image prepared by rendering can be displayed through the display.

[37] This document relates to video/image coding. For example, the methods/embodiments disclosed in this document may be applied to the method disclosed in the Versatile Video Coding (VVC) standard, the EVC (Fundamental Video Coding) standard, the AOMedia Video 1 (AV1) standard, the second generation audio/video coding standard (AVS2) or next generation video/image coding standard (eg H.267 or H.268, etc.).

[38] This document presents various video/image encoding embodiments, and the embodiments may be performed in combination with each other unless otherwise noted.

[39] In this document, video may refer to a sequence of images over time. A frame generally means a unit representing one image in a particular time zone, and a slice/tile is a unit that constitutes part of a frame when encoded. A slice/tile may include one or more coding tree units (CTUs). One frame can consist of one or more slices/tiles. One frame can consist of one or more groups of tiles. One tile group may include one or more tiles. A brick can represent a rectangular area of CTU rows in a tile in a frame. A tile can be segmented into multiple bricks, each of which consists of one or more CTU rows in the tile. A tile that is not segmented into multiple bricks may also be called a "brick". A brick scan is a specific sequential ordering of CTUs segmenting a frame, where CTUs are ordered sequentially in a raster CTU scan within a brick, bricks within a tile are ordered sequentially in a raster scan of tile bricks, and tiles within a frame are ordered sequentially in a raster scan of frame tiles. A tile is a rectangular CTU area in a particular column of tiles and a particular row of tiles in a frame. A column of tiles is a rectangular area of CTUs having a height equal to the height of the frame and a width specified by syntax elements in the frame parameter set. A row of tiles is a rectangular area of CTUs having a height specified by syntax elements in the frame parameter set and a width equal to the width of the frame. A tile scan is a specific sequential ordering of the CTUs segmenting a frame, in which CTUs are ordered sequentially in a raster scan of the CTUs in a tile, whereas tiles in a frame are ordered sequentially in a raster scan of the frame's tiles. A slice includes an integer number of frame bricks that can be contained exclusively in a single NAL unit. A slice can consist of either a specific number of full tiles, or only a rigid sequence of full bricks of a single tile. Tile groups and slices can be used interchangeably in this document. For example, in this document, a group of tiles/header of a group of tiles may be referred to as a "slice/slice header".

[40] Pixel or pel can mean the smallest unit that makes up one frame (or image). Additionally, "sample" can be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel/pixel value of a luma component or only a pixel/pixel value of a chroma component.

[41] The unit may represent a basic image processing unit. The unit may include at least one of a specific frame area and information associated with the area. One unit may include one block of luma signals and two blocks of chrominance signals (eg, Cb, Cr). Unit may be used interchangeably with terms such as block or zone in some cases. In general, an MxN block may include samples (or arrays of samples) or a set (or array) of transform coefficients of M columns and N rows.

[42] In this document, the sign "/" and "," shall be interpreted to indicate "and/or". For example, the expression "A/B" could mean "A and/or B". Additionally, "A, B" may mean "A and/or B". Additionally, "A/B/C" may mean "at least one of A, B and/or C". In addition, "A, B, C" may mean "at least one of A, B and/or C".

[43] Additionally, as used herein, the term “or” should be interpreted to indicate “and/or.” For example, the expression "A or B" may contain 1) only A, 2) only B, and/or 3) both A and B. In other words, the term "or" in this document should be interpreted to indicate "in addition or alternatively" .

[44] FIG. 2 is a circuit diagram illustrating the configuration of video/image encoding equipment to which the embodiment(s) of the present document may be applied. Hereinafter herein, video encoding equipment may include image encoding equipment.

[45] Referring to FIG. 2, the encoding equipment 200 includes an image segmentation module 210, a prediction module 220, a residual processor 230 and an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The prediction module 220 may include an inter-prediction module 221 and an internal prediction module 222. forecasting. Residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. Residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a “recovery module” or a “recovery block generator.” The image segmentation module 210, prediction module 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 may be configured by at least one hardware component (eg, a chipset or encoder processor) according to an embodiment. In addition, storage device 270 may include a decoded frame buffer (DPB) or may be configured with a digital storage medium. The hardware component may further include a storage device 270 as an internal/external component.

[46] The image segmentation module 210 may segment an input image (either a frame or a movie frame) input to the encoding equipment 200 into one more processing units. For example, a processing unit may be called a "coding unit (CU)". In this case, the coding unit may be recursively segmented according to a quadtree, binary tree, and ternary tree (QTBTTT) structure from a coding tree unit (CTU) or largest coding unit (LCU). For example, one coding unit may be segmented into multiple coding units of greater depth based on a quadtree structure, a binary tree structure, and/or a ternary structure. In this case, for example, a quadtree structure may be applied first, and a binary tree structure and a ternary structure may be subsequently applied. Alternatively, a binary tree structure may be used first. The encoding procedure according to this document may be performed based on a final encoding unit that is no longer segmented. In this case, the largest coding unit can be used as the final coding unit based on the coding efficiency according to the characteristics of the images, or if necessary, the coding unit can be recursively segmented into coding units of greater depth, and the coding unit having the optimal size can be used as the final coding units. Here, the encoding procedure may include a prediction, transformation and reconstruction procedure, which is described below. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit may be split or segmented from the above final encoding unit. The prediction unit may be a sample prediction unit, and the transformation unit may be a unit for extracting a transformation coefficient and/or a unit for extracting a residual signal from the transformation coefficient.

[47] Unit may be used interchangeably with terms such as block or zone in some cases. In general, an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample may generally represent a pixel or a pixel value, may represent only a pixel/pixel value of a luma component, or may represent only a pixel/pixel value of a chrominance signal component. Sampling can be used as a term corresponding to one frame (or image) for a pixel or pel.

[48] In the encoding equipment 200, the prediction signal (predicted block, prediction sample array) output from the inter prediction unit 221 or intra prediction unit 222 is subtracted from the input image signal (original block, original sample array) to generate a residual signal ( residual block, residual sample array), and the generated residual signal is transmitted to converter 232. In this case, as shown, a module for subtracting the prediction signal (predicted block, prediction sample array) from the input image signal (source block, source sample array) into encoder 200 may be referred to as "subtractor 231". The prediction module may perform prediction on a block to be processed (hereinafter referred to as a "current block") and generate a prediction block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied based on the current block or CU. As described below in the description of each prediction mode, the prediction module may generate various prediction-related information, such as prediction mode information, and transmit the generated information to the entropy encoder 240. Information regarding the prediction may be encoded in the entropy encoder 240 and output in the form of a stream bits

[49] The intra prediction module 222 may predict the current block by reference to the samples in the current frame. Samples for reference may be located in the surroundings of the current block or may be spaced according to the prediction mode. In intraprediction, prediction modes may include a plurality of non-directional modes and a plurality of directed modes. The omnidirectional mode may include, for example, DC mode and planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the granularity of the prediction direction. However, this is merely an example; more or less directional prediction modes may be used depending on the setting. The intra prediction module 222 may determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[50] The inter-prediction unit 221 can extract a predicted block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. Here, in order to reduce the amount of motion information transmitted in the inter-prediction mode, motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0-prediction, L1-prediction, bi-prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block present in the current frame and a temporal neighbor block present in the reference frame. The reference frame including the reference block and the reference frame including the temporary adjacent block may be identical or different. A temporary neighbor block may be called a “co-located reference block”, a “col-located CU (colCU)”, etc., and a reference frame including a temporary neighboring block may be called a “col-located frame (colPic)”. For example, the inter-prediction module 221 may configure a list of motion information candidates based on neighboring blocks and generate information indicating which candidate is used to retrieve the motion vector and/or reference frame index of the current block. Mutual prediction can be performed based on different prediction modes. For example, in the case of the skip mode and the merge mode, the inter-prediction unit 221 may use the motion information of the adjacent block as the motion information of the current block. In the skip mode, unlike the combine mode, the residual signal may not be transmitted. In the case of the motion vector prediction (MVP) mode, the motion vector of an adjacent block may be used as a motion vector predictor, and the motion vector of the current block may be indicated by signaling a motion vector difference.

[51] The prediction module 220 may generate a prediction signal based on various prediction methods described below. For example, the prediction module may not only apply intra-prediction or inter-prediction to predict one block, but also simultaneously apply both intra-prediction and inter-prediction. This may be referred to as "combined inter-intraprediction (CIIP)". In addition, the prediction module can be based on intra-block copy (IBC) prediction mode or palette mode for block prediction. The IBC prediction mode or palette mode can be used for encoding image/video content of a game and the like, for example, screen content encoding (SCC). IBC essentially performs prediction in the current frame, but can be performed similarly to inter-prediction in which the reference block is retrieved in the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described in this document. Palette mode can be considered an example of intra-coding or intra-prediction. When palette mode is applied, a sampled value in a frame may be signaled based on information regarding the palette table and palette index.

[52] The prediction signal generated by the prediction module (including inter prediction module 221 and/or intra prediction module 222) can be used to generate a reconstructed signal or generate a residual signal. Converter 232 may generate conversion coefficients by applying conversion technology to the residual signal. For example, the transform technology may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally nonlinear transform (CNT). ). Here, GBT means a graph-derived transformation when the relationship information between pixels is represented by the graph. CNT means a transform generated based on a prediction signal generated using all previously reconstructed pixels. In addition, the conversion process can be applied to square pixel blocks that are identical in size, or can be applied to blocks that are variable in size rather than square.

[53] The quantizer 233 may quantize the transform coefficients and transmit them to the entropy encoder 240, and the entropy encoder 240 may encode the quantized signal (information regarding the quantized transform coefficients) and output a bit stream. Information regarding the quantized transform coefficients may be referred to as "residual information". Quantizer 233 may re-arrange the block quantized transform coefficients into one-dimensional vector form based on the scanning order of the coefficients, and generate information regarding the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form. Information regarding conversion coefficients can be generated. The entropy encoder 240 may implement various encoding techniques, such as, for example, exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like. Entropy encoder 240 may encode information needed for video/image reconstruction other than quantized transform coefficients (eg, syntax element values, etc.) together or separately. Encoded information (eg, encoded video/image information) may be transmitted or stored in NAL (Network Abstraction Layer) units in the form of a bitstream. The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. In this document, information and/or syntax elements transmitted/signaled from encoding equipment to decoding equipment may be included in video/frame information. The video/image information may be encoded through the encoding procedure described above and included in the bit stream. The bit stream may be transmitted over a network or may be stored on a digital storage medium. The network may include a broadcast network and/or a communications network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, and the like. A transmitter (not shown) transmitting the signal output from the entropy encoder 240 and/or a storage module (not shown) storing the signal may be included as an internal/external element of the encoding equipment 200, and alternatively, the transmitter may be included in the entropy encoder 240 .

[54] The quantized transform coefficients output from quantizer 233 can be used to generate a predictive signal. For example, the residual signal (residual block or residual samples) may be recovered by applying dequantization and inverse transform to the quantized transform coefficients through dequantizer 234 and inverse transformer 235. Adder 250 adds the reconstructed residual signal with the prediction signal output from interprediction module 221 or module 222 internal prediction to generate a reconstructed signal (reconstructed frame, reconstructed block, array of reconstructed samples). If there is no remainder for a block to be processed, for example in a case in which the skip mode is applied, the predicted block can be used as a reconstructed block. The adder 250 may be referred to as a "recovery module" or a "recovered block generator." The generated reconstructed signal can be used to internally predict the next block to be processed in the current frame, and can be used to inter-predict the next frame through filtering, as described below.

[55] Meanwhile, luma conversion with chrominance scaling (LMCS) can be applied during encoding and/or reconstruction of frames.

[56] Filter 260 may enhance the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, filter 260 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and storing the modified reconstructed frame in storage device 270, namely, a DPB of storage device 270. Various filtering techniques may include, for example, deblocking filtering. , sampled adaptive bias, adaptive contour filter, bilateral filter, etc. The filter 260 may generate various filtering-related information and transmit the generated information to the entropy encoder 240, as described below in the description of each filtering method. Information associated with filtering may be encoded by entropy encoder 240 and output in the form of a bitstream.

[57] The modified reconstructed frame transferred to the storage device 270 can be used as a reference frame in the inter-prediction unit 221. When inter prediction is applied by the encoding equipment, prediction mismatch between the encoding equipment 200 and the decoding equipment can be eliminated, and encoding efficiency can be improved.

[58] The DPB of the storage device 270 may store the modified reconstructed frame for use as a reference frame in the inter-prediction module 221. Memory 270 may store block motion information from which motion information in the current frame is retrieved (or encoded) and/or motion information of blocks in a frame that have already been reconstructed. The stored motion information may be transmitted to the inter-prediction unit 221 and used as spatial neighbor block motion information or temporal neighbor block motion information. Memory 270 may store reconstructed samples of reconstructed blocks in the current frame and may transmit reconstructed samples to intra prediction module 222.

[59] FIG. 3 is a circuit diagram illustrating the configuration of video/image decoding equipment that is applicable to embodiments of this document.

[60] Referring to FIG. 3, the decoding equipment 300 may include an entropy decoder 310, a residual processor 320, a prediction module 330, an adder 340, a filter 350, a memory 360. The prediction module 330 may include an inter-prediction module 331 and an intra-prediction module 332. Residual processor 320 may include a dequantizer 321 and an inverse converter 321. Entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 may be configured by a hardware component (eg, a chipset or decoder processor) according to an embodiment. In addition, storage device 360 may include a decoded frame buffer (DPB) or may be configured with a digital storage medium. The hardware component may further include storage device 360 as an internal/external component.

[61] When a bitstream including video/image information is input, the decoding equipment 300 can reconstruct an image corresponding to the process in which the video/image information is processed in the encoding equipment of FIG. 2. For example, decoding equipment 300 may extract units/blocks based on block segmentation-related information obtained from the bitstream. The decoding equipment 300 may perform decoding using a processing unit used in the encoding equipment. That is, the decoding processing unit, for example, may be an encoding unit, and the encoding unit may be segmented according to a quadtree structure, a binary tree structure, and/or a ternary tree structure from an encoding tree unit or a largest encoding unit. One or more transformation units may be derived from a coding unit. The reconstructed image signal decoded and output by the decoding equipment 300 can be reproduced by the playback equipment.

[62] The decoding equipment 300 may receive a signal output from the encoding equipment of FIG. 2 in the form of a bit stream, and the received signal may be decoded through entropy decoder 310. For example, entropy decoder 310 may parse the bit stream to extract information (eg, video/image information) needed for image reconstruction (or frame reconstruction). The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The decoding equipment may further decode the frame based on information regarding the parameter set and/or general constraint information. The signaling/receiving information and/or syntax elements described later in this document may be decoded, may be decoded by a decoding procedure, and obtained from the bitstream. For example, entropy decoder 310 decodes information in a bitstream based on an encoding method, such as exponential Golomb, CAVLC, or CABAC encoding, and the output syntax elements required for image reconstruction and the quantized values of the transform coefficients for the remainder. More specifically, the entropy CABAC decoding method may receive a resolution element corresponding to each syntax element in the bitstream, determine a context model using information of a target decoding syntax element, decoding information of a target decoding block, or information of a symbol/resolution element decoded in a previous stage, and perform arithmetic decoding for the permission element by predicting the probability of occurrence of the permission element according to the determined context model and generating a symbol corresponding to the meaning of each syntactic element. In this case, the entropy CABAC decoding method can update the context model by using the decoded symbol/grant element information for the context model of the next symbol/grant element after determining the context model. The prediction-related information from the information decoded by the entropy decoder 310 may be provided to the prediction module (inter-prediction module 332 and intra-prediction module 331), and the residual value for which entropy decoding is performed in the entropy decoder 310, i.e. the quantized transform coefficients and associated parameter information may be input to residual processor 320. Residual processor 320 may extract a residual signal (residual block, residual samples, array of residual samples). In addition, information regarding filtering from information decoded by the entropy decoder 310 may be provided to the filter 350. Meanwhile, a receiver (not shown) for receiving a signal output from the encoding equipment may be further configured as an internal/external element of the equipment 300 decoding, or the receiver may be a component of the entropy decoder 310. Meanwhile, the decoding equipment according to this document may be called "video/image/frame decoding equipment", and the decoding equipment may be classified into information decoder (video/image/frame information decoder) and selective decoder (video/image/frame sampling decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverter 322, an adder 340, a filter 350, a memory 360, an inter-prediction module 332, and an intra-prediction module 331 .

[63] The dequantizer 321 may dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 321 may re-arrange the quantized transform coefficients into a two-dimensional block form. In this case, re-arrangement may be performed based on the order of coefficient scanning performed in the encoding equipment. The dequantizer 321 may perform dequantization on the quantized transform coefficients by using a quantization parameter (eg, quantization step size information) and obtain the transform coefficients.

[64] The inverse converter 322 inversely converts the transform coefficients to obtain a residual signal (residual block, residual sample array).

[65] The prediction module may perform prediction for the current block and may generate a predicted block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied to the current block based on the prediction information output from the entropy decoder 310, and may determine a particular intra/inter prediction mode.

[66] The prediction module 320 may generate a prediction signal based on various prediction methods described below. For example, the prediction module may not only apply intra-prediction or inter-prediction to predict one block, but also simultaneously apply intra-prediction and inter-prediction. This may be referred to as "combined inter-intraprediction (CIIP)". In addition, the prediction module can be based on intra-block copy (IBC) prediction mode or palette mode for block prediction. The IBC prediction mode or palette mode can be used for encoding image/video content of a game and the like, for example, screen content encoding (SCC). IBC essentially performs prediction in the current frame, but can be performed similarly to inter-prediction in which the reference block is retrieved in the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described in this document. Palette mode can be considered an example of intra-coding or intra-prediction. When palette mode is applied, a sampled value in a frame may be signaled based on information regarding the palette table and palette index.

[67] The intra prediction module 331 may predict the current block by reference to the samples in the current frame. Samples for reference may be located in the surroundings of the current block or may be spaced according to the prediction mode. In intraprediction, prediction modes may include a plurality of non-directional modes and a plurality of directed modes. The intra prediction unit 331 may determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[68] The inter-prediction unit 332 may extract a predicted block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0-prediction, L1-prediction, bi-prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block present in the current frame and a temporal neighbor block present in the reference frame. For example, the inter-prediction module 332 may configure a motion information candidate list based on neighboring blocks and extract the current block's motion vector and/or reference frame index based on the received candidate selection information. The inter-prediction may be performed based on various prediction modes, and the information regarding the prediction may include information indicating the inter-prediction mode for the current block.

[69] The adder 340 may generate a reconstructed signal (recovered frame, reconstructed block, reconstructed sample array) by adding the resulting residual signal with a prediction signal (predicted block, predicted sample array) output from a prediction module (including inter-prediction module 332 and /or internal prediction module 331). If there is no remainder for a block to be processed, for example when a skip mode is applied, the predicted block can be used as the reconstructed block.

[70] The adder 340 may be referred to as a “recovery module” or a “recovered block generator.” The generated reconstructed signal may be used to internally predict the next block to be processed in the current frame, may be output by filtering as described below, or may be used to inter-predict the next frame.

[71] Meanwhile, luma conversion with chrominance scaling (LMCS) can be applied in the frame decoding process.

[72] Filter 350 may enhance the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, filter 350 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and store the modified reconstructed frame in storage device 360, namely, a DPB of storage device 360. Various filtering techniques may include, for example, deblocking filtering. , sampled adaptive bias, adaptive contour filter, bilateral filter, etc.

[73] The (modified) reconstructed frame stored in the DPB of the storage device 360 can be used as a reference frame in the inter-prediction unit 332. Memory 360 may store block motion information from which motion information in the current frame is retrieved (or decoded) and/or motion information of blocks in a frame that have already been reconstructed. The stored motion information may be transmitted to the inter-prediction unit 260 so that it is used as spatial neighbor block motion information or temporal neighbor block motion information. Storage device 360 may store reconstructed samples of reconstructed blocks in the current frame and transmit the reconstructed samples to intra prediction module 331.

[74] In this document, the embodiments described in the filter 260, the inter-prediction module 221 and the intra-prediction module 222 of the encoding equipment 200 may be identical or appropriately applied so that they correspond to the filter 350, the inter-prediction module 332 and the module 331 internal prediction of the decoding equipment 300. The same may also apply to module 332 and internal prediction module 331.

[75] As described above, when performing video encoding, prediction is performed in order to improve compression efficiency. A predicted block including predicted samples for the current block, i.e. target encoding block, can be generated through prediction. In this case, the predicted block includes predicted samples into a spatial domain (or pixel domain). The predicted block is identically retrieved in the encoding equipment and decoding equipment. The encoding equipment can improve the efficiency of image encoding by signaling, to the decoding equipment, information regarding the remainder (residual information) between the original block, rather than the original sample value of the original block itself, and the predicted block. The decoding equipment can extract a residual block including residual samples based on the residual information, can generate a reconstructed block including the reconstructed samples by adding the residual block with a predicted block, and can generate a reconstructed frame including the reconstructed blocks.

[76] Residual information can be generated through a transformation and quantization procedure. For example, the encoding equipment may extract a residual block between an original block and a predicted block, may extract transform coefficients by performing a transform procedure on residual samples (an array of residual samples) included in the residual block, may extract quantized transform coefficients by performing a quantization procedure on the transform coefficients, and may signal associated residual information to decoding equipment (via a bit stream). In this case, the residual information may include information such as value information, location information, a transform circuit, a transform kernel, and a quantization parameter for the quantized transform coefficients. The decoding equipment may perform a dequantization/deconversion procedure based on the residual information and may extract the residual samples (or residual block). The decoding equipment can generate a reconstructed image based on the predicted block and the residual block. In addition, the encoding equipment may extract a residual block by dequantizing/de-converting the quantized transform coefficients to reference the inter-prediction of a subsequent frame, and may generate a reconstructed frame.

[77] Meanwhile, as described above, intra prediction or inter prediction may be applied when performing prediction for the current block. Later in this document, the case of applying mutual prediction to the current block is described. Meanwhile, as described above, intra prediction or inter prediction may be applied when performing prediction for the current block. Later in this document, the case of applying mutual prediction to the current block is described.

[78] A prediction module (more specifically, an inter-prediction module) of the encoding/decoding equipment can extract prediction samples by performing inter-prediction in units of blocks. The inter-prediction may represent a prediction extracted in a manner dependent on data elements (eg, sample values or motion information) of frame(s) other than the current frame. When inter-prediction is applied to the current block, a predicted block (prediction sample array) for the current block can be retrieved based on a reference block (reference sample array) indicated by a motion vector in a reference frame in a reference frame indicated by a reference frame index. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information of the current block may be predicted in units of blocks, sub-blocks or samples based on the correlation of the motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction type information (L0 prediction, L1 prediction, bi-prediction, etc.). When inter-prediction is applied, the neighbor block may include a spatial neighbor block that is present in the current frame and a temporal neighbor block that is present in the reference frame. The reference frame including the reference block and the reference frame including the temporary adjacent block may be identical to each other or different from each other. A temporary neighbor block may be referred to by a name such as a "co-located reference block", a "co-located CU (colCU)", etc., and a reference frame including a temporary neighboring block may be called a "co-located frame ( colPic)". For example, a list of motion information candidates may be configured based on neighboring blocks relative to the current block, and flag or index information indicating which candidate is selected (used) may be signaled so as to extract a motion vector and/or index of the current block's reference frame. Mutual prediction may be performed based on different prediction modes, and, for example, in the case of a skip mode and a merge mode, the motion information of the current block may be identical to the motion information of the selected neighboring block. In the case of the skip mode, the residual signal may not be transmitted, unlike the combine mode. In the case of a motion vector prediction (MVP) mode, the motion vector of a selected neighboring block can be used as a motion vector predictor, and the motion vector difference can be signaled. In this case, the motion vector of the current block can be extracted by using the sum of the motion vector predictor and the motion vector difference.

[79] The motion information may further include L0 motion information and/or L1 motion information according to a type of inter-prediction (L0 prediction, L1 prediction, bi-prediction, etc.). The motion vector for the L0 direction may be referred to as the "L0 motion vector" or "MVL0", and the motion vector for the L1 direction may be referred to as the "L1 motion vector" or "MVL1". Prediction based on L0 motion vector may be referred to as “L0 prediction,” prediction based on L1 motion vector may be referred to as “L1 prediction,” and prediction based on both L0 motion vector and L1 motion vector may be referred to as “biprediction.” ". Here, the L0 motion vector may indicate a motion vector associated with the reference frame list L0, and the L1 motion vector may indicate a motion vector associated with the reference frame list L1. The reference frame list L0 may include frames before the current frame in output order, and the reference frame list L1 may include frames subsequent to the current frame in output order as reference frames. Preceding frames may be referred to as a "forward (reference) frame" and subsequent frames may be referred to as a "reverse (reference) frame." The reference frame list L0 may further include frames subsequent to the current frame in the order of output as reference frames. In this case, previous frames may first be indexed in the reference frame list L0, and subsequent frames may then be indexed. The reference frame list L1 may further include frames up to the current frame in the order of output as reference frames. In this case, subsequent frames may first be indexed in the reference frame list L1, and previous frames may then be indexed. Here, the order of output may correspond to the order of numbers in the sequence of frames (POC).

[80] Additionally, various inter-prediction modes can be used when applying inter-prediction to the current block. For example, various modes may be used, including a merge mode, a skip mode, a motion vector prediction (MVP) mode, an affine mode, a statistical motion vector prediction (HMVP) mode, and the like. Decoder Motion Vector Resolution (DMVR), Adaptive Motion Vector Resolution (AMVR), Bidirectional Optical Flow (BDOF), etc. can additionally be used as an auxiliary mode. The affine mode may be called "affine motion prediction mode". The MVP mode may be referred to as "Advanced Motion Vector Prediction (AMVP) mode". Herein, some modes and/or motion information options extracted by some modes may be included in one of the motion information-related options of other modes.

[81] Prediction mode information indicating the inter-prediction mode of the current block may be signaled from the encoding equipment to the decoding equipment. In this case, prediction mode information may be included in the bitstream and received by the decoding equipment. The prediction mode information may include index information indicating one of several possible mode options. Alternatively, the inter-prediction mode may be indicated via hierarchical signaling of flag information. In this case, the prediction mode information may include one or more flags. For example, whether or not to apply the skip mode may be indicated by signaling a skip flag, whether or not to apply the combining mode may be indicated by signaling a combining flag when the skip mode is not applied, and indicating whether that the MVP mode is applied, or a flag for additional discrimination may optionally be signaled when the combining mode is not applied. The affine mode may be signaled as an independent mode or signaled as a dependent mode in a pooling mode or MVP mode. For example, an affine mode may include an affine union mode and an affine MVP mode.

[82] Additionally, the motion information of the current block can be used when applying inter-prediction to the current block. The encoder can extract optimal motion information for the current block through a motion estimation procedure. For example, the encoding apparatus may search a similar reference block having high correlation in fractional pixel units in a predetermined search range in a reference frame by using the source block in the source frame for the current block, and extract motion information through the searched reference block. The block similarity can be extracted based on the difference of the phase-based sample values. For example, block similarity may be calculated based on the sum of absolute differences (SAD) between the current block (or the current block's reference) and the reference block (or the reference block's reference). In this case, motion information can be extracted based on the reference block having the smallest SAD in the search area. The extracted motion information may be signaled to a decoding device according to various methods based on an inter-prediction mode.

[83] A predicted block for a current block may be extracted based on motion information extracted according to the prediction mode. The predicted block may include the predicted samples (an array of predicted samples) of the current block. When the motion vector (MV) of the current block indicates a fractional unit of samples, an interpolation procedure can be performed, and the prediction samples of the current block can be extracted based on reference samples with a fractional unit of samples in the reference frame through the interpolation procedure. When affine cross-prediction is applied to the current block, prediction samples can be generated based on MVs in sample/sub-block units. When bi-prediction is applied, prediction samples extracted through the weighted sum or weighted average of prediction samples retrieved based on L0 prediction (i.e. prediction using a reference frame in a list of L0 reference frames and MVL0), and prediction samples (according to phase), extracted based on L1 prediction (ie, prediction using a reference frame in the L1 reference frame list and MVL1) can be used as prediction samples of the current block. When bi-prediction is applied, if the reference frame used for L0 prediction and the reference frame used for L1 prediction are located in different temporal directions based on the current frame (i.e., if the prediction corresponds to bi-prediction and bidirectional prediction), it may be called "true biforecasting".

[84] Reconstructed samples and reconstructed frames can be generated based on the extracted predictive samples, and a procedure such as in-loop filtering, etc., can then be performed as described above.

[85] Meanwhile, since the motion of the current block is predicted based on the motion vector of the adjacent block without motion vector difference (MVD) in the skip mode and/or merge mode, the skip mode and/or merge mode shows a limitation in motion prediction. To improve the limitation of the skip mode and/or the combining mode, the motion vector can be refined by applying a decoder-side motion vector detail (DMVR) mode, a bidirectional optical flow (BDOF) extraction mode, etc. DMVR and BDOF modes can be used when true biprediction is applied to the current block.

[86] FIG. 4 is a diagram for describing an embodiment of a process for performing decoder-side motion vector refinement (DMVR) in true pairwise prediction.

[87] DMVR is a method in which a decoder performs motion prediction by refining motion information of an adjacent block. When DMVR is applied, the decoder can extract detailed motion information through cost comparison based on a reference generated by using motion information of an adjacent block in a merge/skip mode. In this case, the motion prediction accuracy can be increased, and the compression performance can be improved without additional overhead information.

[88] This document mainly describes decoding equipment for convenience of description, but DMVR according to an embodiment of this document can be performed in an identical manner even in encoding equipment.

[89] Referring to FIG. 4, the decoding equipment can extract prediction blocks (i.e., reference blocks) identified by initial motion vectors for list 0 and list 1 direction (or motion information) (for example, MV0 and MV1), and generate a template (or bilateral template) through a weighted sum (e.g., averaging) of the extracted predictive samples (step 1). Here, the initial motion vectors (MV0 and MV1) may represent motion vectors extracted by using motion information of a neighboring block in a merge/skip mode.

[90] In addition, the decoding equipment may extract motion vectors (eg, MV0' and MV1') to minimize the difference between the template and the sample area of the reference frame through a template matching operation (step 2). Here, the sample zone may indicate a neighboring zone of the initial prediction block in the reference frame, and the sample zone may be called a “neighbor zone”, “reference zone”, “search zone”, “search range”, “search space”, etc. The pattern matching operation may include the operation of calculating a cost measurement value between the template and the sample area of the reference frame. For example, sum of absolute differences (SAD) can be used to measure costs. As one example, normalized SAD can be used as a cost function. In this case, the matching cost can be specified as SAD (T-mean(T), 2*P[x]-2*mean(P[x])). Here, T represents the template, and P[x] represents the block in the search area. In addition, the motion vector for calculating the minimum reference cost for each of the two reference frames can be considered as an updated motion vector (replacing the initial motion vector). As illustrated in FIG. 8, the decoding equipment can generate the final result of the bilateral prediction (ie, the final block of the bilateral prediction) by using the updated motion vectors MV0' and MV1'. As an embodiment, multi-iteration to extract the updated (or new) motion vector can be used to obtain the final bilateral prediction result.

[91] In an embodiment, the decoding equipment may invoke the DMVR process to improve the accuracy of the initial motion-compensated prediction (ie, motion-compensated prediction via a conventional combine/skip mode). For example, the decoding equipment may perform a DMVR process where the prediction mode of the current block is a merge mode or a skip mode, and bilateral biprediction, in which bilateral reference frames are arranged in opposite directions based on the current frame in display order, is applied to the current block.

[92] FIG. 5 is a diagram for explaining an embodiment of a process for performing decoder-side motion vector refinement (DMVR) by using sum of absolute differences (SAD).

[93] As described above, decoding equipment can measure mapping costs through the use of SAD in executing DMVR. As an embodiment, FIG. 5, a method for refining a motion vector by calculating the mean sum of absolute differences (MRSAD) between prediction samples in two reference frames without generating a template is further described. In other words, the method of FIG. 5 shows an embodiment of bilateral matching using MRSAD.

[94] Referring to FIG. 5, the decoding equipment may extract an adjacent pixel relative to a pixel (sample) indicated by motion vector MV0 for list direction 1 (L0) in the reference L0 frame, and extract an adjacent pixel relative to a pixel (sample) indicated by motion vector MV1 for list direction 1 (L1) in the L1 reference frame. In addition, the decoding equipment may measure the matching cost by calculating the MRSAD between the L0 prediction block (i.e., the L0 reference block) identified by a motion vector indicating an adjacent pixel extracted in the L0 reference frame and the L1 prediction block (ie, an L1 reference block) identified by a motion vector indicating an adjacent pixel extracted in the L1 reference frame. In this case, the decoding equipment may select a search point (ie, a search area having a minimum SAD between an L0 prediction block and an L1 prediction block) having a minimum cost as a pair of detailed motion vectors. In other words, a pair of detailed motion vectors may include a detailed L0 motion vector indicating a pixel position (L0 prediction block) having a minimum cost in the L0 reference frame, and a detailed L1 motion vector indicating a pixel position (L1 block prediction) that has minimal costs in the reference L1 frame.

[95] As an embodiment, when calculating the matching cost, after the reference frame search area is set, one-way prediction can be performed by using a regular 8-tap DCTIF interpolation filter. Additionally, as one example, 16-bit precision may be used to calculate the MRSAD, and trimming and/or rounding operations may not be applied before calculating the MRSAD given the internal buffer.

[96] When true biprediction is applied to the current block, as described above, BDOF can be used to refine the biprediction signal. When bi-prediction is applied to the current block, bidirectional optical flow (BDOF) can be used to calculate improved motion information and generate prediction samples based on the calculated motion information. For example, BDOF can be applied at the 4x4 subblock level. In other words, BDOF can be performed in units of 4x4 subblocks in the current block. Alternatively, BDOF may be applied to only the luminance signal component. Alternatively, BDOF may be applied to only the chrominance signal component and applied to the luma signal component and the chrominance signal component.

[97] BDOF mode is based on the concept of optical flow, provided that the motion of the object is smooth, as indicated by the name "BDOF mode". The motion granularity (vx, vy) can be calculated by minimizing the difference between the predicted L0 and L1 samples for each of the 4x4 sub-blocks. In addition, motion detail can be used to adjust sample biprediction values in 4x4 sub-blocks.

[98] Meanwhile, it can be seen that DMVR and BDOF as technologies that perform prediction by refining motion information (in this case, true bi-prediction represents the case of performing motion prediction/compensation in a reference frame of another direction based on the frame of the current block) in time of application of true biprediction is a granularity technology that has a similar concept in that it assumes that the movement of an object in a frame is performed at a predetermined speed and in a predetermined direction. However, when true bi-forecasting is performed, since the condition for applying DMVR and the condition for applying BDOF are different from each other, the process of checking the conditions multiple times for each technology must be performed. Therefore, this document proposes a method that can improve efficiency in terms of decoder complexity and performance by improving the condition checking process when determining the prediction mode to apply to the current block.

[99] Table 1 below shows the condition for applying DMVR during traditional true biforecasting. When all of the following conditions are met, DMVR can be applied.

[100] Tab. 1

- sps_dmvr_enabled_flag is set to 1: Signaling to SPS
- merge_flag[xCb][yCb] is equal to 1: Applies in case of merge/skip
- mmvd_flag[xCb][yCb] is equal to 0: Applies in case of non-MMVD
- predFlagL0[0][0]=1 and predFlagL0[1][1]=1: Bidirectional prediction
- DiffPicOrderCnt(currPic, RefPicList0[refIdxL0])-DiffPicOrderCnt(currPic, RefPicList1[refIdxL1]) is equal to 0: TrueBi and the distances from the current frame and the bidirectional reference frame are equal to each other (hereinafter in this document, "distance to reference frame" can represent the above meaning).
- CbHeight is greater than or equal to 8: In case the length (or size) of the block is greater than a threshold value (for example, 8) (here, the threshold value can be roughly illustrated in various ways),
- CbHeight*CbWidth is greater than or equal to 64: In case the block size (or absolute value) is greater than a threshold value (for example, 64) (here, the threshold value can be roughly illustrated in various ways)

[101] Referring to Table 1, 1) whether or not to apply DMVR can be determined based on flag information (eg, sps_dmvr_enabled_flag) signaled in Sequence Parameter Set (SPS) syntax. Here, the flag information (eg, sps_dmvr_enabled_flag) may represent whether or not the DMVR is enabled based on the true bi-prediction. For example, when sps_dmvr_enabled_flag is equal to 1 (ie, when the true biprediction-based DMVR is enabled), it may be determined that a condition for whether or not the DMVR is enabled is satisfied.

[102] 2) Whether or not to apply DMVR may be determined based on flag information (eg, merge_flag) representing whether or not mutual prediction is performed by using the merge mode/skip mode. For example, when merge_flag is 1 (ie, when mutual prediction is performed by using the merge mode/skip mode), it may be determined that a condition for whether or not to apply the merge mode/skip mode is satisfied.

[103] 3) Whether or not to apply DMVR can be determined based on flag information (eg, mmvd_flag) representing whether or not inter-prediction is performed by using a motion vector difference (MMVD) combining mode. For example, when mmvd_flag is 0 (ie, when the MMVD mode is not used), it may be determined that the condition for whether or not to use the MMVD mode is satisfied.

[104] 4) Whether or not DMVR should be used can be determined based on whether bilateral prediction (biprediction) is used or not. Here, bilateral prediction may represent mutual prediction performed based on reference frames that exist in different directions based on the current frame. For example, when predFlagL0[0][0]=1 and predFlagL0[1][1]=1, it can be determined that bilateral prediction is applied, and it can be determined that a condition for whether or not to perform bilateral prediction is satisfied .

[105] 5) Whether or not to apply DMVR can be determined based on whether true bi-prediction is performed or not, and the distance between the current frame and bilateral reference frames are identical to each other. In other words, it can be determined whether or not the distance between the current frame and the L0 reference frame (i.e., the reference frame in the L0 reference frame list) and the distance between the current frame and the L1 reference frame (i.e., the reference frame in list L1 of reference frames) identical to each other. For example, when DiffPicOrderCnt(currPic, RefPicList0[refIdxL0])-DiffPicOrderCnt(currPic, RefPicList1[refIdxL1])=0, it is determined that true bi-prediction is performed and the distances between the current frame and bilateral reference frames are identical to each other to determine that that the condition for whether or not the distances of the bilateral reference frames are identical to each other is satisfied.

[106] 6) Whether or not to apply DMVR can be determined based on whether the height of the current block is greater than a threshold value. For example, when the height of the current block is equal to or greater than 8, it may be determined that the size (height) condition of the current block is satisfied.

[107] 7) Whether or not to apply DMVR may be determined based on whether or not the current block size is larger than a threshold value. For example, when the current block size, height*width, is equal to or greater than 64, it may be determined that the current block size (height*width) condition is satisfied.

[108] The decoding equipment can determine whether or not to apply the DMVR according to whether conditions 1) to 7) of Table 1 above are satisfied or not. In other words, when conditions 1) to 7) of Table 1 above are satisfied, the decoding equipment can perform true bi-prediction by applying DMVR, and when even any of the conditions in Table 1 above are not satisfied, the decoding equipment does not apply DMVR.

[109] Table 2 below shows the condition for applying BDOF during traditional true biforecasting. When all of the following conditions are met, BDOF can be applied.

[110] Tab. 2

- sps_bdof_enabled_flag is equal to 1: Signaling via SPS
- predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx] are equal to 1: Bidirectional prediction
- DiffPicOrderCnt(currPic, refPicList0[refIdxL0])*DiffPicOrderCnt(currPic, refPicList1[refIdxL1]) less than 0: TrueBi
- MotionModelIdc[xCb][yCb] is equal to 0: In case of non-affine mode
- merge_subblock_flag[xCb][yCb] is equal to 0: subblock andand w>=8 andand h>=8
- GbiIdx[xCb][yCb] is equal to 0: In case the GBi index is the default value
- cIdx is 0: Applies only to the luminance signal

[111] Referring to Table 2 above, 1) whether or not BDOF should be applied can be determined based on flag information (eg, sps_bdof_enabled_flag) signaled in Sequence Parameter Set (SPS) syntax. Here, the flag information (eg, sps_dmvr_enabled_flag) may represent whether the BDOF is enabled or not based on the true biprediction. For example, when sps_bdof_enabled_flag is equal to 1 (ie, when the true biprediction-based BDOF is enabled), it may be determined that the condition for whether or not the BDOF is enabled is satisfied.

[112] 2) Whether or not BDOF should be used can be determined based on whether bilateral prediction is used or not. Here, bilateral prediction may represent mutual prediction performed based on reference frames that exist in different directions based on the current frame. For example, when predFlagL0] and predFlagL1 are 1, it can be determined that bilateral prediction is applied, and it can be determined that a condition for whether or not to perform bilateral prediction is satisfied.

[113] 3) Whether or not BDOF should be applied can be determined based on whether true biprediction is performed or not. In other words, it can be determined whether or not the L0 reference frame (i.e., the reference frame in the reference frame list L0) and the L1 reference frame (i.e., the reference frame in the reference frame list L1) are located in temporarily different directions on based on the current frame. For example, when DiffPicOrderCnt(currPic, refPicList0[refIdxL0])*DiffPicOrderCnt(currPic, refPicList1[refIdxL1]) is less than 0, it is determined that the bilateral reference frames are located in different directions based on the current frame to determine that the condition for , whether true vi-prediction holds or fails.

[114] 4) Whether or not BDOF should be used can be determined based on whether the affine mode is used or not. Here, whether or not the affine mode is used can be determined by retrieving the MotionModelIdc. For example, when the retrieved MotionModelIdc is 0, it may be determined that the affine mode is not used, in which case, it may be determined that the condition for whether or not to use the affine mode is satisfied.

[115] 5) Whether or not BDOF should be applied may be determined based on flag information (eg, merge_subblock_flag) representing whether or not inter-prediction is performed in subblock units. For example, when merge_subblock_flag is 0 (ie, when the merge mode is not applied in the subblock units), it may be determined that the condition for whether or not to apply the merge mode in the subblock units is satisfied.

[116] 6) Whether or not BDOF should be applied can be determined based on whether GBi exists or not. Here, whether or not a GBi exists can be determined based on the GBi index information (eg, GbiIdx). For example, when GbiIdx is 0 (ie, when GbiIdx is the default value), it may be determined that the condition for whether or not GBi exists is satisfied.

[117] 7) Whether or not BDOF should be applied may be determined based on whether or not the current block is a luma signal block including a luma signal component. For example, when an index (eg, cIdx) indicating whether or not the current block is a luma block (i.e., when the current block is a luma block), it may be determined that a condition for whether the current block is or not the brightness signal block is satisfied.

[118] The decoding equipment can determine whether or not to apply the DMVR according to whether conditions 1) to 7) of Table 2 above are satisfied or not. In other words, when conditions 1) to 7) of Table 2 above are satisfied, the decoding equipment can perform true bi-prediction by applying BDOF and when even any of the conditions in Table 2 above are not satisfied, the decoding equipment does not apply BDOF.

[119] As described above, some conditions for use of DMVR and BDOF are identical, and some are similar or different. In the traditional scheme, since condition checking is performed for each technology, even when the condition is identical, the complexity to perform bi-prediction increases. Therefore, the present disclosure provides an effective condition for applying DMVR and BDOF during bi-forecasting.

[120] When the combine/skip mode is compared with the AMVP mode, the combine/skip mode has relatively lower accuracy than the AMVP mode, and as a result, the motion information granularity by using the DMVR method is efficient in terms of performance. However, BDOF mode can be applied even in AMVP mode in addition to the merge/skip mode as opposed to DMVR, and as such, the complexity to perform BDOF relative to performance may increase when BDOF is applied in AMVP mode. Accordingly, the embodiment proposes a method for applying even BDOF in a merge/skip mode similarly to DMVR.

[121] In this case, as an embodiment proposed in the present disclosure, the BDOF application condition may include the conditions presented in Table 3 below.

[122] Tab. 3

- sps_bdof_enabled_flag is equal to 1.
- merge_flag[xCb][yCb] is equal to 1.
- predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx] are equal to 1.
- DiffPicOrderCnt(currPic, refPicList0[refIdxL0])*DiffPicOrderCnt(currPic, refPicList1[refIdxL1]) is less than 0.
- MotionModelIdc[xCb][yCb] is equal to 0.
- merge_subblock_flag[xCb][yCb] is equal to 0.
- GbiIdx[xCb][yCb] is equal to 0.
- cIdx is 0.

[123] Referring to the above Table 3, whether or not to apply BDOF can be determined based on flag information (eg, merge_flag) representing whether or not mutual prediction is performed by using the merge mode/skip mode. For example, when merge_flag is 1 (ie, when mutual prediction is performed by using the merge mode/skip mode), it may be determined that a condition for whether or not to apply the merge mode/skip mode is satisfied. Accordingly, BDOF can also be used in a merge/skip mode, similar to DMVR.

[124] In other words, in the embodiment, together with the condition applied in the case of the combine/skip mode, whether or not BDOF should be applied can be determined based on the condition applied when the BDOF is activated, the condition applied in the case of bilateral prediction, a condition applied when true biprediction is performed, a condition applied when affine prediction is used, a condition applied when subblock-based aggregation mode is not applied, a condition applied when the GBi index is the default, and a condition applied when the current block is a luma block.

[125] Accordingly, the decoding equipment can determine whether all the conditions listed in Table 3 above are satisfied or not, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying BDOF. When even any of the conditions listed in Table 3 above are not satisfied, the decoding equipment may not apply BDOF. The encoding equipment can also apply the conditions of the above Table 3, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[126] The method for performing true bi-prediction based on the conditions listed in Table 3 above can be independently applied to DMVR and BDOF, or can be applied under identical conditions for DMVR and BDOF.

[127] Meanwhile, when the conditions listed in the embodiment are identical to the conditions described in the above Tables 1 and 2, the detailed operation or meaning is similarly applied, and as a result, the detailed description for each condition is omitted. Additionally, duplicate content is omitted even in the embodiments that are described below.

[128] The encoding/decoding equipment may be configured with different hardware, and the preference for complexity versus performance may differ. Therefore, the embodiment provides a method that can refine motion information by applying DMVR even in AMVP mode in addition to the combine/skip mode.

[129] In this case, as an embodiment proposed in the present disclosure, the application condition of the DMVR may include the conditions presented in Table 4 below.

[130] Tab. 4

- sps_dmvr_enabled_flag is set to 1.
- merge_flag[xCb][yCb] is equal to 1. (Removing conditions limited by merge/skip mode)),
- mmvd_flag[xCb][yCb] is equal to 0.
- predFlagL0[0][0]=1 and predFlagL0[1][1]=1.
- DiffPicOrderCnt(currPic, RefPicList0[refIdxL0])-DiffPicOrderCnt(currPic, RefPicList1[refIdxL1]) is 0.
- CbHeight is greater than or equal to 8.
- CbHeight*CbWidth is greater than or equal to 64.

[131] Referring to the above Table 4, the process of determining whether or not to apply the DMVR may be omitted based on flag information (eg, merge_flag) representing whether or not mutual prediction is performed by using the merge mode/skip mode. In this regard, by omitting the condition of whether or not to apply the combining mode/skip mode, DMVR can be applied even in the AMVP mode in addition to the combining mode/skip mode.

[132] According to the above Table 4, whether or not to apply the DMVR can be determined based on the condition applied when the DMVR is activated, the condition applied when the MMVD mode is not used, the condition applied in the case of bilateral prediction, the condition applied in the case of true biprediction, in which the distances between the current frame and bilateral reference frames are identical to each other, a condition applied when the height of the current block is equal to or greater than 8, and a condition applied when the size (height*width) of the current block is equal to or greater 64.

[133] In other words, the decoding equipment can determine whether all the conditions listed in Table 4 above are satisfied or not, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying DMVR. When even any of the conditions listed in Table 4 above are not satisfied, the decoding equipment may not apply the DMVR. The encoding equipment can also apply the conditions of the above Table 4, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[134] The method for performing true bi-prediction based on the conditions listed in Table 4 above can be independently applied to DMVR and BDOF, or can be applied under identical conditions for DMVR and BDOF.

[135] Meanwhile, as an embodiment of the present disclosure, both DMVR and BDOF can be applied to the normal combining mode. In other words, DMVR and BDOF can be used when the Advanced Temporal Motion Vector Prediction (ATMVP) mode is not used, the affine mode is not used, and CPR is not used. In this case, the condition for applying the DMVR may include the conditions presented in Table 5 below.

[136] Tab. 5

- sps_dmvr_enabled_flag is set to 1.
- merge_flag[xCb][yCb] is equal to 1.
- mmvd_flag[xCb][yCb] is equal to 0.
- predFlagL0[0][0]=1 and predFlagL0[1][1]=1.
- DiffPicOrderCnt(currPic, RefPicList0[refIdxL0])-DiffPicOrderCnt(currPic, RefPicList1[refIdxL1]) is 0.
- CbHeight is greater than or equal to 8.
- CbHeight*CbWidth is greater than or equal to 64.
- MotionModelIdc[xCb][yCb] is equal to 0: ! Affine
- merge_subblock_flag[xCb][yCb] is equal to 0: subblock andand w>=8 andand h>=8

[137] Referring to the above Table 5, whether or not the condition applied when the affine mode is not used (for example, when MotionModelIdc is 0) and the condition applied when the subblock-based merge mode is not used (for example, when merge_subblock_flag is 0) is defined to apply DMVR only in normal aggregation mode.

[138] Additionally, in an embodiment, together with the condition for whether or not the affine mode is used, and the condition for whether or not the subblock-based aggregation mode is used, whether or not the DMVR should be applied can be determined based on the condition applied when DMVR is activated, the condition applied when the merge mode/skip mode is used, the condition applied when the MMVD mode is not used, the condition applied in the case of bilateral prediction, the condition applied in the case of true biprediction in which the distances between the current frame and bilateral reference frames are identical to each other, the condition applied when the height of the current block is equal to or greater than 8, and the condition applied when the size (height*width) of the current block is equal to or greater than 64.

[139] In other words, the decoding equipment can determine whether or not all the conditions listed in Table 5 above are satisfied, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying DMVR. When even any of the conditions listed in Table 5 above are not satisfied, the decoding equipment may not apply the DMVR. The encoding equipment can also apply the conditions of the above Table 5, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[140] The condition for whether or not to apply the subblock-based merge mode (eg, merge_subblock_flag) from the application conditions of Table 5 above includes a duplicate condition from among the traditional DMVR application conditions. Accordingly, as an embodiment of the present disclosure, a condition duplicated with a condition (eg, merge_subblock_flag) for whether or not to apply the subblock-based merging mode may be removed. In this case, the corresponding condition may be deleted as suggested in Table 6 below.

[141] Tab. 6

- sps_dmvr_enabled_flag is set to 1.
- merge_flag[xCb][yCb] is equal to 1.
- mmvd_flag[xCb][yCb] is equal to 0.
- predFlagL0[0][0]=1 and predFlagL0[1][1]=1.
- DiffPicOrderCnt(currPic, RefPicList0[refIdxL0])-DiffPicOrderCnt(currPic, RefPicList1[refIdxL1]) is 0.
- CbHeight is greater than or equal to 8.
- CbHeight*CbWidth is greater than or equal to 64.
- MotionModelIdc[xCb][yCb] is equal to 0: ! Affine
- merge_subblock_flag[xCb][yCb] is equal to 0: subblock andand w>=8 andand h>=8

[142] Referring to the above Table 6, the subblock-based aggregation mode can be applied when the current block size is equal to or greater than 8×8. Accordingly, since the condition (for example, merge_subblock_flag=0) for whether or not to apply the subblock-based merging mode includes a condition related to the size of the current block, conditions (for example, CbHeight and CbHeight*CbWidth) related to the size of the current block, may be excluded from the conditions of use of traditional DMVR. For example, the condition for whether the height of the current block 8 is equal to or greater than or not, and the condition for whether the height*width of the current block 64 is equal to or greater than or not may be omitted, and whether or not the DMVR should be applied may be determined by using the remaining conditions listed in Table 6 above.

[143] In other words, the decoding equipment can determine whether all the conditions listed in Table 6 above are satisfied or not, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying DMVR. When even any of the conditions listed in Table 6 above are not satisfied, the decoding equipment may not apply the DMVR. The encoding equipment can also apply the conditions of the above Table 6, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[144] The method for performing true bi-prediction based on the conditions listed in Table 5 or 6 above can be independently applied to DMVR and BDOF, or can be applied under identical conditions for DMVR and BDOF.

[145] As an embodiment of the present disclosure, in order to increase motion vector accuracy at low complexity, when the block size is small, granularity technologies such as DMVR and BDOF may not be applied. In the traditional scheme, granularity technology can be applied when the current block is a block that is equal to or higher than 8x8, and in the case of DMVR granularity technologies, when the size of the current block is large, granularity is applied by dividing the current block into 16x16 units. and as a result, DMVR may not apply to a block that is smaller than 16x16. In this case, the condition for applying the DMVR may include the conditions presented in Table 7 below.

[146] Tab. 7

- sps_dmvr_enabled_flag is set to 1.
- merge_flag[xCb][yCb] is equal to 1.
- mmvd_flag[xCb][yCb] is equal to 0.
- predFlagL0[0][0]=1 and predFlagL0[1][1]=1.
- DiffPicOrderCnt(currPic, RefPicList0[refIdxL0])-DiffPicOrderCnt(currPic, RefPicList1[refIdxL1]) is 0.
- IntraPredMode is greater than or equal to 16
- CbWidth is greater than or equal to 16.

[147] Referring to Table 7 above, DMVR may not apply to a block that is smaller than 16×16 by changing conditions (eg, CbHeight and CbWidth) associated with the size of the current block. For example, a condition applied when the height of the current block (for example, CbHeight) is equal to or greater than 16, and a condition applied when the width of the current block (for example, CbWidth) is equal to or greater than 16 can be used. When application conditions related to the size of the current block , are satisfied (i.e., when the current block size is equal to or greater than 16x16, DMVR can be applied, and when the application conditions related to the current block size are not satisfied (i.e., when the current block size is less than 16x16), DMVR may not apply.

[148] Additionally, in an embodiment, together with conditions (e.g., CbHeight and CbWidth) associated with the size of the current block, whether or not to apply DMVR can be determined based on the remaining application conditions listed in Table 7 above.

[149] In other words, the decoding equipment can determine whether all the conditions listed in Table 7 above are satisfied or not, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying DMVR. When even any of the conditions listed in Table 7 above are not satisfied, the decoding equipment may not apply the DMVR. The encoding equipment can also apply the conditions of the above Table 7, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[150] As an embodiment of the present disclosure, when the current block is a block that is smaller than 16x16, BDOF may not be applied in addition to DMVR. In this case, the condition for applying the BDOF may include the conditions presented in Table 8 below.

[151] Tab. 8

- sps_bdof_enabled_flag is equal to 1.
- predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx] are equal to 1.
- DiffPicOrderCnt(currPic, refPicList0[refIdxL0])*DiffPicOrderCnt(currPic, refPicList1[refIdxL1]) is less than 0.
- MotionModelIdc[xCb][yCb] is equal to 0.
- merge_subblock_flag[xCb][yCb] is equal to 0.
- IntraPredMode is greater than or equal to 16
- CbWidth greater than or equal to 16
- GbiIdx[xCb][yCb] is equal to 0.
- cIdx is 0.

[152] Referring to Table 8 above, BDOF may not apply to a block that is smaller than 16×16 by changing conditions (eg, CbHeight and CbWidth) associated with the size of the current block. For example, a condition applied when the height of the current block (for example, CbHeight) is equal to or greater than 16, and a condition applied when the width of the current block (for example, CbWidth) is equal to or greater than 16 can be used. When application conditions related to the size of the current block , are satisfied (i.e. when the current block size is equal to or greater than 16x16), BDOF can be applied, and when the application conditions related to the current block size are not satisfied (i.e. when the current block size is less than 16x16) , BDOF may not apply.

[153] Additionally, in an embodiment, together with conditions (e.g., CbHeight and CbWidth) associated with the size of the current block, whether or not BDOF should be applied can be determined based on the remaining application conditions listed in Table 8 above.

[154] In other words, the decoding equipment can determine whether or not all the conditions listed in Table 8 above are satisfied, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying BDOF. When even any of the conditions listed in Table 8 above are not satisfied, the decoding equipment may not apply BDOF. The encoding equipment can also apply the conditions of the above Table 8, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[155] The method for performing true bi-prediction based on the conditions listed in Table 7 or 8 above can be independently applied to DMVR and BDOF, or can be applied under identical conditions for DMVR and BDOF.

[156] As described above, DMVR is applied when the distances between the current frame and the bilateral reference frames are identical to each other, while BDOF is continuously applied in the case of true biprediction, even if the distances between the current frame and the bilateral reference frames are different from each other . Accordingly, the present disclosure proposes a method that can integrally apply a condition related to the distance between bilateral reference frames in DMVR and BDOF to improve coding efficiency.

[157] As an embodiment proposed in the present disclosure, the BDOF application condition may include the conditions presented in Table 9 below.

[158] Tab. 9

- sps_bdof_enabled_flag is equal to 1.
- predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx] are equal to 1.
- DiffPicOrderCnt(currPic, RefPicList0[refIdxL0])-DiffPicOrderCnt(currPic, RefPicList1[refIdxL1]) is equal to 0: TrueBi and distances to bidirectional reference frame are equal to each other
- MotionModelIdc[xCb][yCb] is equal to 0.
- merge_subblock_flag[xCb][yCb] is equal to 0.
- GbiIdx[xCb][yCb] is equal to 0.
- cIdx is 0.

[159] Referring to the above Table 9, the condition associated with the distance to the reference frame from the BDOF application conditions (eg, DiffPicOrderCnt) is changed to similarly apply the corresponding condition to DMVR and BDOF. For example, whether or not DiffPicOrderCnt(currPic, RefPicList0[refIdxL0])-DiffPicOrderCnt(currPic, RefPicList1[refIdxL1]) is equal to 0 is defined to determine whether or not the distance between the current frame and the reference L0 frame (i.e., the reference frame in the reference frame list L0) and the distance between the current frame and the L1 reference frame (ie, the reference frame in the reference frame list L1) are identical to each other. In other words, BDOF can only be applied when the distances between the current frame and the bilateral reference frames are identical to each other. In this regard, as a condition under which true bi-prediction is performed and the distances between bilateral reference frames are identical to each other, it is added that the range of BDOF application is limited, thereby reducing decoding complexity.

[160] Additionally, in an embodiment, together with conditions (eg, DiffPicOrderCnt) associated with the distance to the reference frame, whether or not BDOF should be applied can be determined based on the remaining application conditions listed in Table 9 above.

[161] In other words, the decoding equipment can determine whether all the conditions listed in Table 9 above are satisfied or not, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying BDOF. When even any of the conditions listed in Table 9 above are not satisfied, the decoding equipment may not apply BDOF. The encoding equipment can also apply the conditions of the above Table 9, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[162] As an embodiment proposed in the present disclosure, the DMVR application condition may include the conditions presented in Table 10 below.

[163] Tab. 10

- sps_dmvr_enabled_flag is set to 1.
- merge_flag[xCb][yCb] is equal to 1.
- mmvd_flag[xCb][yCb] is equal to 0.
- predFlagL0[0][0]=1 and predFlagL0[1][1]=1.
- DiffPicOrderCnt(currPic, RefPicList0[refIdxL0])-DiffPicOrderCnt(currPic, RefPicList1[refIdxL1]) is equal to 0: TrueBi and distances to bidirectional reference frame are equal to each other
- DiffPicOrderCnt(currPic, refPicList0[refIdxL0])*DiffPicOrderCnt(currPic, refPicList1[refIdxL1]) less than 0: TrueBi
- CbHeight is greater than or equal to 8
- CbHeight*CbWidth is greater than or equal to 64

[164] Referring to the above Table 10, the condition associated with the distance to the reference frame from the DMVR application conditions (eg, DiffPicOrderCnt) is changed to similarly apply the corresponding condition to the DMVR and BDOF. For example, whether or not DiffPicOrderCnt(currPic, refPicList0[refIdxL0])*DiffPicOrderCnt(currPic, refPicList1[refIdxL1])0 is determined to determine whether or not true biprediction is performed, in which bilateral reference frames are located in different directions based on the current frame . In other words, DMVR can be continuously applied in the case of true biprediction, even if the distances between the current frame and the bilateral reference frames are not identical to each other. In this regard, when a condition for whether true bi-prediction is satisfied or not is applied, the motion vector extracted taking into account the decoding complexity can be used without scaling even when the distances between bilateral reference frames are different.

[165] Additionally, in an embodiment, together with a condition (eg, DiffPicOrderCnt) associated with the distance to the reference frame, whether or not to apply DMVR can be determined based on the remaining application conditions listed in Table 10 above.

[166] In other words, the decoding equipment can determine whether all the conditions listed in Table 10 above are satisfied or not, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying DMVR. When even any of the conditions listed in Table 10 above are not satisfied, the decoding equipment may not apply the DMVR. The encoding equipment can also apply the conditions of the above Table 10, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[167] The method for performing true bi-prediction based on the conditions listed in Table 9 or 10 above can be independently applied to DMVR and BDOF, or can be applied under identical conditions for DMVR and BDOF.

[168] Meanwhile, in each reference block, a case may arise in which motion compensation is performed by a weighted sum by changing the light. In this case, since the phenomenon can be determined by GBi or local illuminance compensation (LIC), the application conditions of DMVR and BDOF can be determined taking into account the GBi or LIC condition.

[169] As an embodiment of the present disclosure, a method is provided for determining whether or not to apply DMVR taking into account the GBi and LIC conditions. In this case, the condition for applying the DMVR may include the conditions presented in Table 11 below.

[170] Tab. eleven

- sps_dmvr_enabled_flag is set to 1.
- merge_flag[xCb][yCb] is equal to 1.
- mmvd_flag[xCb][yCb] is equal to 0.
- predFlagL0[0][0]=1 and predFlagL0[1][1]=1.
- DiffPicOrderCnt(currPic, RefPicList0[refIdxL0])-DiffPicOrderCnt(currPic, RefPicList1[refIdxL1]) is 0.
- CbHeight is greater than or equal to 8.
- CbHeight*CbWidth is greater than or equal to 64.
- GbiIdx[xCb][yCb] is equal to 0.
- LICFlag is 0.

[171] Referring to Table 11, it is possible to determine whether or not to apply DMVR by adding a GBi condition (eg, GbiIdx) and an LIC condition (eg, LICFlag). Here, GBi may represent a generalized biprediction allowing different weights to be applied to the L0 prediction and L1 prediction, and, for example, represent a generalized biprediction using GbiIdx. GbiIdx may exist in the case of biforecasting and represent the biforecasting weight index. For example, GbiIdx may be retrieved from an adjacent block in aggregation mode or signaled from encoding equipment to decoding equipment via a GbiIdx syntax element (eg, gbi_idx) in MVP mode. As an example, GbiIdx may indicate a weight w applied to the L1 prediction, in which case, a weight (1-w) may be applied to the L0 prediction. As another example, GbiIdx may indicate a weight w applied to the L0 prediction, in which case, a weight (1-w) may be applied to the L1 prediction.

[172] According to Table 11, if GbiIdx is 0 (ie, if GbiIdx is the default value), it may be determined to satisfy the GBi condition, and if LICFlag is 0 (ie, if LIC exists), it may be defined as satisfying the LIC condition.

[173] Additionally, in an embodiment, together with the GBi condition (eg, GbiIdx) and the LIC condition (eg, LICFlag), whether or not to apply the DMVR can be determined based on the remaining application conditions listed in Table 11 above .

[174] In other words, the decoding equipment can determine whether or not all the conditions listed in Table 11 above are satisfied, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying DMVR. When even any of the conditions listed in Table 11 above are not satisfied, the decoding equipment may not apply the DMVR. The encoding equipment can also apply the conditions of the above Table 11, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[175] As an embodiment of the present disclosure, a method is provided for determining whether or not BDOF should be applied taking into account the GBi and LIC conditions. In this case, the condition for applying the BDOF may include the conditions presented in Table 12 below.

[176] Tab. 12

- sps_bdof_enabled_flag is equal to 1.
- predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx] are equal to 1.
- DiffPicOrderCnt(currPic, refPicList0[refIdxL0])*DiffPicOrderCnt(currPic, refPicList1[refIdxL1]) is less than 0.
- MotionModelIdc[xCb][yCb] is equal to 0.
- merge_subblock_flag[xCb][yCb] is equal to 0.
- GbiIdx[xCb][yCb] is equal to 0.
- LICFlag is 0.
- cIdx is 0.

[177] According to the above Table 12, whether or not BDOF should be applied can be determined by adding an LIC condition (eg, LICFlag) together with a traditional GBi condition (eg, GbiIdx). For example, when GbiIdx is 0 (i.e., when GbiIdx is the default value), it is determined that the condition for whether or not a GBi exists is satisfied, and when LICFlag is 0 (i.e., when LIC exists), may determined that the condition for whether or not an LIC exists is satisfied.

[178] Accordingly, in an embodiment, together with the GBi condition (eg, GbiIdx) and the LIC condition (eg, LICFlag), whether or not BDOF should be applied can be determined based on the remaining application conditions listed in Table 12 above .

[179] In other words, the decoding equipment can determine whether or not all the conditions listed in Table 12 above are satisfied, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying BDOF. When even any of the conditions listed in Table 12 above are not satisfied, the decoding equipment may not apply BDOF. The encoding equipment can also apply the conditions of the above Table 12, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[180] The method for performing true bi-prediction based on the conditions listed in Table 11 or 12 above can be independently applied to DMVR and BDOF, or can be applied under identical conditions for DMVR and BDOF.

[181] Meanwhile, since DMVR and BDOF extract motion information through a granularity process in decoding equipment, the problem of decoding complexity arises. Accordingly, the present disclosure proposes a method that can reduce decoding complexity by allowing a determination of whether or not to apply DMVR and BDOF through the use of a pooling index. In this case, since both DMVR and BDOF perform motion vector detail within a limited range, the benefit of detail may be reduced when the motion vector is imprecise. Accordingly, the present disclosure proposes a method that can limitedly apply DMVR and BDOF only when the value indicating the join index is small in view of the granularity efficiency.

[182] Here, the combining index may be a syntax element signaled from the encoding equipment to the decoding equipment. For example, the encoding/decoding equipment may configure a list of possible combining options based on neighboring blocks relative to the current block when the combining mode/skip mode is applied to the current block. In this case, the encoding equipment may select the optimal combining candidate from the combining candidate candidates included in the combining candidate list based on the rate-distortion (RD) distortion cost function, and signal combining index information indicating the selected one. a possible option for combining into decoding equipment. The decoding equipment may select a combine candidate to apply to the current block based on the list of combine candidates and the combine index information.

[183] As an embodiment of the present disclosure, a method for determining whether or not to apply DMVR by using a union index may include the conditions presented in Table 13 below.

[184] Tab. 13

- sps_dmvr_enabled_flag is set to 1.
- merge_flag[xCb][yCb] is equal to 1.
- mmvd_flag[xCb][yCb] is equal to 0.
- merge_idx[xCb][yCb]<2.
- predFlagL0[0][0]=1 and predFlagL0[1][1]=1.
- DiffPicOrderCnt(currPic, RefPicList0[refIdxL0])-DiffPicOrderCnt(currPic, RefPicList1[refIdxL1]) is 0.
- CbHeight is greater than or equal to 8.
- CbHeight*CbWidth is greater than or equal to 64.

[185] Referring to Table 13 above, whether or not DMVR should be applied can be determined by adding a condition on the merge index (eg, merge_idx). For example, when the merge index (eg, merge_idx) is less than 2, it may be determined that the condition on the merge index is satisfied. Here, the value (threshold value) of the combining index may be set to 2, but this is only one example, and the corresponding value may vary according to the encoding efficiency.

[186] Accordingly, in an embodiment, together with the condition on the merge index (for example, merge_idx), whether or not to apply the DMVR can be determined based on the remaining application conditions listed in Table 13 above.

[187] In other words, the decoding equipment can determine whether or not all the conditions listed in Table 13 above are satisfied, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying DMVR. When even any of the conditions listed in Table 13 above are not satisfied, the decoding equipment may not apply the DMVR. The encoding equipment can also apply the conditions of the above Table 13, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[188] As an embodiment of the present disclosure, a method for determining whether or not to apply BDOF by using a union index may include the conditions presented in Table 14 below.

[189] Tab. 14

- sps_bdof_enabled_flag is equal to 1.
- predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx] are equal to 1.
- DiffPicOrderCnt(currPic, refPicList0[refIdxL0])*DiffPicOrderCnt(currPic, refPicList1[refIdxL1]) is less than 0.
- MotionModelIdc[xCb][yCb] is equal to 0.
- merge_subblock_flag[xCb][yCb] is equal to 0.
- GbiIdx[xCb][yCb] is equal to 0.
-! (merge_flag[xCb][yCb] is equal to 1 andand merge_idx[xCb][yCb]>=2)
- cIdx is 0.

[190] Referring to the above Table 14, whether or not BDOF should be applied can be determined by adding a condition on whether the merge mode/skip mode is satisfied or not (eg, merge_flag), and a condition on the merge index (eg, merge_idx). For example, when merge_flag is not equal to 1 (i.e., when mutual prediction is performed by using merge mode/skip mode), and merge_idex is not equal to or greater than 2, only if the merge index value is small, it can be determined that the condition for limited application BDOF is satisfied. In other words, when merge_flag is 1 (that is, when mutual prediction is performed by using merge mode/skip mode), and merge_idex is less than 2, only if the value of the merge index is small, it can be determined that the condition on the merge index is satisfied, and BDOF may apply. Here, the value (threshold value) of the combining index may be set to 2, but this is only one example, and the corresponding value may vary according to the encoding efficiency.

[191] In other words, in an embodiment, together with a condition on whether or not the merge mode/skip mode is satisfied (eg, merge_flag), and a condition on the merge index (eg, merge_idx), whether or not BDOF should be applied may determined based on the remaining application conditions listed in Table 14 above.

[192] In other words, the decoding equipment can determine whether or not all the conditions listed in Table 14 above are satisfied, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying BDOF. When even any of the conditions listed in Table 14 above are not satisfied, the decoding equipment may not apply BDOF. The encoding equipment can also apply the conditions of the above Table 14, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[193] The method for performing true bi-prediction based on the conditions listed in Table 13 or 14 above can be independently applied to DMVR and BDOF, or can be applied under identical conditions for DMVR and BDOF.

[194] Meanwhile, in the merge/skip mode, motion information can be detailed through MMVD, and in this case, since the decoding complexity increases, DMVR is not performed when MMVD is applied. However, when DMVR is applied without considering the MMVD condition, DMVR can be applied without the MMVD condition considering the performance improvement. In this case, according to an embodiment of the present disclosure, the DMVR application condition may include the conditions presented in Table 15 below.

[195] Tab. 15

- sps_dmvr_enabled_flag is set to 1.
- merge_flag[xCb][yCb] is equal to 1.
- mmvd_flag[xCb][yCb] is equal to 0.
- predFlagL0[0][0]=1 and predFlagL0[1][1]=1.
- DiffPicOrderCnt(currPic, RefPicList0[refIdxL0])-DiffPicOrderCnt(currPic, RefPicList1[refIdxL1]) is 0.
- CbHeight is greater than or equal to 8.
- CbHeight*CbWidth is greater than or equal to 64.

[196] Referring to the above Table 15, the condition (eg, mmvd_flag) for whether or not to apply the MMVD mode from the application conditions of the traditional DMVR may be excluded. In other words, the process of determining whether or not mmvd_flag is 0 (i.e., not used or MMVD mode is used) may be omitted, and whether or not DMVR should be applied may be determined based on the conditions listed in Table 15 above.

[197] In other words, the decoding equipment can determine whether or not all the conditions listed in Table 15 above are satisfied (except for the condition (eg, mmvd_flag) for whether or not to apply the MMVD mode), and when all conditions are satisfied ,The decoding equipment can perform true bi-prediction by ,applying DMVR. When even any of the conditions listed in Table 15 above are not satisfied, the decoding equipment may not apply the DMVR. The encoding equipment can also apply the conditions of the above Table 15, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[198] Additionally, unlike the embodiment of Table 15 described above, when motion information is detailed through MMVD in the merge/skip mode, whether or not to perform BDOF can be determined taking into account the decoding complexity. Accordingly, the present disclosure provides a method for determining whether or not BDOF should be applied, taking into account the MMVD condition. In this case, according to an embodiment of the present disclosure, the BDOF application condition may include the conditions presented in Table 16 below.

[199] Tab. 16

- sps_bdof_enabled_flag is equal to 1.
- predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx] are equal to 1.
- DiffPicOrderCnt(currPic, refPicList0[refIdxL0])*DiffPicOrderCnt(currPic, refPicList1[refIdxL1]) is less than 0.
- MotionModelIdc[xCb][yCb] is equal to 0.
- merge_subblock_flag[xCb][yCb] is equal to 0.
- GbiIdx[xCb][yCb] is equal to 0.
- mmvd_flag[xCb][yCb] is equal to 0
- cIdx is 0.

[200] Referring to Table 16 above, BDOF may not be applied when motion information is detailed through MMVD by adding a condition (eg, mmvd_flag) for whether or not the MMVD mode should be applied. For example, when mmvd_flag is 0 (ie, when the MMVD mode is not used), it may be determined that the condition for whether or not to use the MMVD mode is satisfied.

[201] Accordingly, in an embodiment, along with a condition (eg, mmvd_flag) for whether or not to apply the MMVD mode, whether or not to apply BDOF can be determined based on the remaining application conditions listed in Table 16 above.

[202] In other words, the decoding equipment can determine whether or not all the conditions listed in Table 16 above are satisfied, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying BDOF. When even any of the conditions listed in Table 16 above are not satisfied, the decoding equipment may not apply BDOF. The encoding equipment can also apply the conditions of the above Table 16, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[203] The method for performing true bi-prediction based on the conditions listed in Table 15 or 16 above can be independently applied to DMVR and BDOF, or can be applied under identical conditions for DMVR and BDOF.

[204] Meanwhile, in the case of AMVP, adaptive motion vector resolution (AMVR) technology can be applied. In this case, when the resolution of the motion vector is high, i.e. When integer-point rounding or 4 integer-point rounding is provided, it may not be appropriate to apply the technology in the case of BDOF performing drilling in a limited area. Accordingly, the present disclosure provides a method that can determine the execution of a BDOF according to an AMVR condition. In this case, according to an embodiment of the present disclosure, the BDOF application condition may include the conditions presented in Table 17 below.

[205] Tab. 17

- sps_bdof_enabled_flag is equal to 1.
- predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx] are equal to 1.
- DiffPicOrderCnt(currPic, refPicList0[refIdxL0])*DiffPicOrderCnt(currPic, refPicList1[refIdxL1]) is less than 0.
- MotionModelIdc[xCb][yCb] is equal to 0.
- merge_subblock_flag[xCb][yCb] is equal to 0.
- amvr_flag!=0
- GbiIdx[xCb][yCb] is equal to 0.
- cIdx is 0.

[206] Referring to Table 17 above, whether or not BDOF should be applied can be determined by adding an AMVR condition (eg, amvr_flag). Here, amvr_flag may be information representing a motion vector difference (MVD) solution. For example, in case amvr_flag is equal to 0, the case may indicate that the MVD resolution is extracted in 1/4 sampling units (a quarter of the luminance signal sample), and in the case that amvr_flag is not equal to 0, the case may indicate that the MVD resolution is extracted in units of an integer luminance sample or four luminance samples. Alternatively, the opposite case to this may be defined. According to an embodiment, as presented in the above Table 20, in the case where amvr_flag is not 0, a condition that BDOF is applied may be set. In other words, if amvr_flag is 0, BDOF may be limited such that it does not apply.

[207] Accordingly, in an embodiment, together with the AMVR condition (e.g., amvr_flag), whether or not BDOF should be applied can be determined based on the remaining application conditions listed in Table 17 above.

[208] In other words, the decoding equipment can determine whether or not all the conditions listed in Table 17 above are satisfied, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying BDOF. When even any of the conditions listed in Table 17 above are not satisfied, the decoding equipment may not apply BDOF. The encoding equipment can also apply the conditions of the above Table 17, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[209] Meanwhile, DMVR and BDOF may be signaled in Sequence Parameter Set (SPS) syntax. Table 18 below shows one example of a syntax element indicating whether or not the DMVR signaled through the SPS syntax is activated and whether the BDOF is activated or not.

[210] Tab. 18

seq_parameter_set_rbsp(){ Descriptor ...... ...... sps_dmvr_enabled_flag u(1) sps_bdof_enabled_flag u(1) ...... ...... }

[211] Referring to the above Table 18, the sps_dmvr_enabled_flag may be signaled in the SPS syntax, and whether or not the DMVR is enabled based on the true bi-prediction may be represented based on the syntax element. For example, in the case where sps_dmvr_enabled_flag is equal to 1, the case may indicate that the true bi-prediction-based DMVR is activated, and in the case that sps_dmvr_enabled_flag is equal to 0, the case may indicate that the true bi-prediction-based DMVR is not activated.

[212] Additionally, sps_bdof_enabled_flag may be signaled in SPS syntax, and whether or not BDOF is enabled based on true biprediction may be represented based on the syntax element. For example, in the case where sps_bdof_enabled_flag is equal to 1, the case may indicate that the true biprediction BDOF is activated, and in the case that sps_bdof_enabled_flag is equal to 0, the case may indicate that the true biprediction BDOF is not activated.

[213] As shown in Table 18, DMVR and BDOF application conditions can be checked by using a syntax element (eg, sps_dmvr_enabled_flag) representing whether the DMVR is enabled or not, and a syntax element (eg, sps_bdof_enabled_flag) representing whether BDOF is enabled or not. .

[214] FIG. 6 illustrates one example illustrating a method for performing a decoding process by checking application conditions of DMVR and BDOF.

[215] As shown in Table 18, the method of FIG. 6 may be used by using a syntax element (eg, sps_dmvr_enabled_flag) representing whether the DMVR is enabled or not, and a syntax element (eg, sps_bdof_enabled_flag) representing whether the BDOF is enabled or not.

[216] Referring to FIG. 6, the decoding equipment can extract motion information (eg, motion vector, reference frame index, etc.) for the current block (S600).

[217] The decoding equipment may check the application of DMVR (S610). In this case, the DMVR application condition may be checked based on a syntax element (eg, sps_dmvr_enabled_flag) representing whether the DMVR is enabled or not. For example, when DMVR is enabled (eg, when sps_dmvr_enabled_flag is 1), the DMVR application condition may be checked.

[218] The decoding equipment can determine whether or not to apply the DMVR process according to whether the DMVR application condition is satisfied or not (S620).

[219] When all DMVR application conditions are satisfied, the decoding equipment can extract detailed motion information by applying the DMVR process (S630). When at least one of the DMVR application conditions is not satisfied, the decoding equipment may not apply the DMVR process.

[220] The decoding equipment can extract predictive samples of the current block based on the extracted motion information (not detailed) when the detailed motion information or DMVR extracted when applying the DMVR is not applied (S640).

[221] In addition, the decoding equipment may check the BDOF application condition (S650). In this case, the BDOF DMVR application condition may be checked based on a syntax element (eg, sps_bdof_enabled_flag) representing whether the BDOF is enabled or not. For example, when the DMVR is enabled (for example, when sps_bdof_enabled_flag is 1), the DMVR application condition may be checked.

[222] When all BDOF application conditions are satisfied, the decoding equipment can refine the prediction samples by applying the BDOF process (S670). When at least one of the BDOF application conditions is not satisfied, the decoding equipment may not apply the BDOF process.

[223] The decoding equipment may extract residual samples for the current block (S680) and extract reconstructed samples based on detailed prediction samples extracted when using residual samples and BDOF or predictive samples (non-detailed) extracted when not using BDOF (S690).

[224] The present disclosure proposes various embodiments that can improve coding efficiency and reduce complexity by harmonizing the mutual application conditions of DMVR and BDOF during application of DMVR and BDOF, as described above. When checking the application conditions of DMVR and BDOF according to embodiments of the present disclosure and applying the application conditions to the decoding process, the corresponding conditions can be separately checked and applied, but the application conditions can be checked immediately to improve encoding efficiency. In other words, the present disclosure provides a method that can integrate and verify the application conditions of DMVR and BDOF at once.

[225] As an embodiment of the present disclosure, information (for example) indicating whether or not to apply granularity in the decoding equipment in sequence parameter set (SPS) syntax is signaled to perform a DMVR/BDOF application condition checking process. . Next, Table 19 shows one example of a syntax element (eg, sps_refinement_enabled_flag) representing whether or not refinement should be applied to the decoding equipment signaled through the SPS syntax.

[226] Tab. 19

seq_parameter_set_rbsp(){ Descriptor ...... ...... sps_refinement_enabled_flag u(1) if(sps_refinement_enabled_flag){ sps_dmvr_enabled_flag u(1) sps_bdof_enabled_flag u(1) } ...... ...... }

[227] Referring to Table 19 above, sps_refinement_enabled_flag may be signaled in SPS syntax and may represent whether or not the granularity is usable in decoding equipment based on the syntax element. For example, when sps_refinement_enabled_flag exists (ie, when sps_refinement_enabled_flag is true), it can be determined that the granularity is usable in the decoding hardware. In this case, the decoding equipment receives an sps_dmvr_enabled_flag syntax element representing whether the DMVR is enabled or not, and an sps_bdof_enabled_flag syntax element representing whether the BDOF is enabled or not, so as to determine application conditions of the DMVR and the BDOF.

[228] FIG. 7 and 8 illustrate another example illustrating a method for performing a decoding process by checking application conditions of DMVR and BDOF.

[229] The method of FIG. 7 and the method according to FIG. 8 may be applied when a syntax element (eg, sps_refinement_enabled_flag) representing whether or not the granularity is usable in decoding equipment is used, as shown in Table 19 above. Additionally, the description of the duplicated contents of FIG. 7 and 8 from fig. 6 described above is omitted.

[230] Referring to FIG. 7, it can be seen that the application conditions are checked directly in the previous step without checking the corresponding application conditions of DMVR and BDOF by comparison with the above process in FIG. 6. As an embodiment, in step S710 of FIG. 7, the decoding equipment can check the granularity application condition. In this case, the granularity application condition may be checked based on the sps_refinement_enabled_flag shown in the above Table 19. For example, the decoding equipment may obtain an sps_dmvr_enabled_flag syntax element representing whether the DMVR is enabled or not, and an sps_bdof_enabled_flag syntax element representing whether the BDOF is enabled or not then when sps_refinement_enabled_flag is 1, and check the DMVR and BDOF application conditions based on the received syntax elements.

[231] Additionally, referring to FIG. 8, the application conditions are checked directly in the previous step without checking the corresponding application conditions of DMVR and BDOF by comparison with the above process in FIG. 6, and a simple verification process (BDOF application condition) can be performed with respect to another condition. As an embodiment, in step S810 of FIG. 8, the decoding equipment can check the granularity application condition. Thereafter, in step S850 of FIG. 8, the decoding equipment may further simply check the application condition of a BDOF having a condition different from the granularity application condition.

[232] Meanwhile, when the height or size (height*width) of the current block is smaller than a specific length or specific size, the multiplication/addition calculation ratio for motion compensation is increased. Accordingly, in an embodiment of the present disclosure, the application of BDOF to a block having a small height or small size may be limited in quality under the condition of applying the DMVR to reduce the worst case. In this case, the condition for applying the BDOF may include the conditions presented in Table 20 below.

[233] Tab. 20

- sps_bdof_enabled_flag is equal to 1.
- predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx] are equal to 1.
- DiffPicOrderCnt(currPic, refPicList0[refIdxL0])*DiffPicOrderCnt(currPic, refPicList1[refIdxL1]) is less than 0.
- MotionModelIdc[xCb][yCb] is equal to 0.
- merge_subblock_flag[xCb][yCb] is equal to 0.
- GbiIdx[xCb][yCb] is equal to 0.
- CbHeight is greater than or equal to 8: In case the length (or size) of the block is greater than the threshold value (here, the threshold value can be roughly illustrated in various ways),
- CbHeight*CbWidth is greater than or equal to 64: In case the block size (or absolute value) is greater than the threshold value (here, the threshold value can be roughly illustrated in various ways),
- cIdx is 0.

[234] Referring to Table 20 above, BDOF may not apply to a block that is smaller than a specific size by adding conditions (eg, CbHeight and CbWidth) related to the size of the current block. For example, a condition applied when the height of the current block (for example, CbHeight) is equal to or greater than 8, and a condition applied when the size of the current block (for example, CbHeight*CbWidth) is equal to or greater than 64 can be used. When application conditions related to size of the current block are satisfied (i.e., when the height of the current block is equal to or greater than 8, and the height*width of the current block is equal to or greater than 64), BDOF may be applied, and when the application conditions associated with the size of the current block are not satisfied, BDOF may do not apply.

[235] Additionally, in an embodiment, along with conditions (e.g., CbHeight and CbHeight*CbWidth) associated with the size of the current block, whether or not BDOF should be applied can be determined based on the remaining application conditions listed in Table 20 above.

[236] In other words, the decoding equipment can determine whether or not all the conditions listed in Table 20 above are satisfied, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying BDOF. When even any of the conditions listed in Table 20 above are not satisfied, the decoding equipment may not apply BDOF. The encoding equipment can also apply the conditions of the above Table 20, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[237] Meanwhile, when the MMVD is applied to the current block, motion information can be extracted based on information (eg, MMVD index) regarding the MMVD. Here, information regarding the MMVD may include a base MV index, a distance index, a direction index, and the like. In particular, a distance index (more specifically, mmvd_distance_index[xCb][yCb]) can be used to represent the distance from the base MV, and, for example, distance indices 0-7 can be represented as {1/4, 1/2, 1, 2, 4, 8, 16, 32}, respectively. When determining the granularity of motion information in DMVR and BDOF, whether or not the granularity is performed taking into account the adjacent pixel (adjacent sample), and in this case, when the distance between the adjacent pixel and the base MV is large, the distance index value is also increased. What is difficult in such a case is that considering the adjacent pixel helps in improving the performance of DMVR and BDOF. Accordingly, the present disclosure provides a method that can determine whether or not DMVR and BDOF should be applied according to a distance index (more specifically, the mmvd_distance_index[xCb][yCb] value).

[238] As an embodiment of the present disclosure, whether or not to apply the DMVR may be determined taking into account the distance index, in which case, the condition for applying the DMVR may include the conditions presented in Table 21 below.

[239] Tab. 21

- sps_dmvr_enabled_flag is set to 1.
- merge_flag[xCb][yCb] is equal to 1.
- mmvd_flag[xCb][yCb] is equal to 0.
- mmvd_flag[xCb][yCb] is equal to 1 and mmvd_distance_index[xCb][yCb])>4.
- predFlagL0[0][0]=1 and predFlagL0[1][1]=1.
- DiffPicOrderCnt(currPic, RefPicList0[refIdxL0])-DiffPicOrderCnt(currPic, RefPicList1[refIdxL1]) is 0.
- CbHeight is greater than or equal to 8.
- CbHeight*CbWidth is greater than or equal to 64.

[240] Referring to the above Table 21, the condition (eg, mmvd_flag) associated with MMVD from the DMVR application conditions is changed to restrict the application of DMVR in MMVD mode. For example, when mmvd_flag is 1 and mmvd_distance_index is greater than 4, it may be determined that the distance index condition MMVD is satisfied. Accordingly, when the MMVD mode is applied, whether or not DMVR should be applied can be determined according to the distance index (more specifically, the mmvd_distance_index[xCb][yCb] value).

[241] Here, the value (threshold value) mmvd_distance_index may be set to 4, but this is only one example, and the corresponding value may be changed to different values according to performance and encoding efficiency.

[242] Accordingly, in an embodiment, together with a condition (eg, mmvd_flag) for whether or not MMVD should be applied, and a distance index condition (eg, mmvd_distance_index) mMVD, whether or not DMVR should be applied can be determined based on the remaining conditions of use listed in Table 21 above.

[243] In other words, the decoding equipment can determine whether or not all the conditions listed in Table 21 above are satisfied, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying DMVR. When even any of the conditions listed in Table 21 above are not satisfied, the decoding equipment may not apply the DMVR. The encoding equipment can also apply the conditions of the above Table 21, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[244] Additionally, as an embodiment of the present disclosure, whether or not BDOF should be applied may be determined taking into account the distance index, in which case, the condition for applying BDOF may include the conditions presented in Table 22 below.

[245] Tab. 22

- sps_bdof_enabled_flag is equal to 1.
- predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx] are equal to 1.
- DiffPicOrderCnt(currPic, refPicList0[refIdxL0])*DiffPicOrderCnt(currPic, refPicList1[refIdxL1]) is less than 0.
- MotionModelIdc[xCb][yCb] is equal to 0.
- merge_subblock_flag[xCb][yCb] is equal to 0.
-! merge_flag || (merge_flag andand mmvd_flag[xCb][yCb] is 1 and mmvd_distance_index[xCb][yCb])>4)
- GbiIdx[xCb][yCb] is equal to 0.
- cIdx is 0.

[246] Referring to Table 22 above, BDOF can be applied to a limited extent in MMVD mode by adding a condition for whether merge mode/skip mode is satisfied or not (eg, merge_flag), and conditions (eg, mmvd_flag and mmvd_distance_index) associated with MMVD . For example, when merge_flag does not exist (ie, when merge_flag is not equal to 1), or when merge_flag is equal to 1, mmvd_flag is equal to 1, and mmvd_distance_index is greater than 4, it can be determined that the condition for limited application of BDOF in MMVD mode is satisfied. Accordingly, when the MMVD mode is applied, whether or not BDOF should be applied can be determined according to the distance index (eg, the mmvd_distance_index[xCb][yCb] value).

[247] Here, the value (threshold value) of mmvd_distance_index may be set to 4, but this is only one example, and the corresponding value may be changed to different values according to performance and encoding efficiency.

[248] Accordingly, in an embodiment, together with a condition for whether or not the merge mode/skip mode is satisfied (eg, merge_flag), and a condition associated with the MMVD (eg, mmvd_flag, mmvd_distance_index), whether or not BDOF should be applied , may be determined based on the remaining application conditions listed in Table 22 above.

[249] In other words, the decoding equipment can determine whether or not all the conditions listed in Table 22 above are satisfied, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying BDOF. When even any of the conditions listed in Table 22 above are not satisfied, the decoding equipment may not apply BDOF. The encoding equipment can also apply the conditions of the above Table 22, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[250] The method for performing true bi-prediction based on the conditions listed in Table 21 or 22 above can be independently applied to DMVR and BDOF, or can be applied under identical conditions for DMVR and BDOF.

[251] Meanwhile, a combined intra-inter prediction (CIIP) mode for simultaneously performing intra prediction and inter prediction can be applied to the current block as described above. In this case, a prediction block (mutual block) in which mutual prediction is performed is combined with an intra-prediction method to ultimately generate predictive sample values, thereby improving the prediction accuracy. However, since DMVR and BDOF are mutual block granularity technologies, the use of CIIP mode may not be necessary in terms of performance versus complexity. Accordingly, the present disclosure provides a method that can determine whether or not DMVR and BDOF should be applied, taking into account the CIIP.

[252] As an embodiment of the present disclosure, whether or not to apply the DMVR may be determined taking into account the CIIP, in which case, the condition for applying the DMVR may include the conditions presented in Table 23 below.

[253] Tab. 23

- sps_dmvr_enabled_flag is set to 1.
- merge_flag[xCb][yCb] is equal to 1.
- mmvd_flag[xCb][yCb] is equal to 0.
- predFlagL0[0][0]=1 and predFlagL0[1][1]=1.
- DiffPicOrderCnt(currPic, RefPicList0[refIdxL0])-DiffPicOrderCnt(currPic, RefPicList1[refIdxL1]) is 0.
- CbHeight is greater than or equal to 8.
- CbHeight*CbWidth is greater than or equal to 64.
- ciip_flag is 0: So when CIIP is not applied, DMVR can be applied.

[254] Referring to the above Table 23, DMVR may be limited in application according to whether CIIP should be applied or not by adding a condition on whether CIIP mode should be applied or not (eg, ciip_flag). For example, when ciip_flag is 0 (ie, when CIIP mode is not applied), DMVR may be applied by determining that a condition for whether or not to apply CIIP mode is satisfied.

[255] Accordingly, in an embodiment, along with a condition (for example, ciip_flag) for whether or not to apply the CIIP mode, whether or not to apply DMVR can be determined based on the remaining application conditions listed in Table 23 above.

[256] In other words, the decoding equipment can determine whether all the conditions listed in Table 23 above are satisfied or not, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying DMVR. When even any of the conditions listed in Table 23 above are not satisfied, the decoding equipment may not apply the DMVR. The encoding equipment can also apply the conditions of the above Table 23, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[257] Additionally, as an embodiment of the present disclosure, whether or not BDOF should be applied may be determined taking into account the CIIP, in which case, the condition for applying BDOF may include the conditions presented in Table 24 below.

[258] Tab. 24

- sps_bdof_enabled_flag is equal to 1.
- predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx] are equal to 1.
- DiffPicOrderCnt(currPic, refPicList0[refIdxL0])*DiffPicOrderCnt(currPic, refPicList1[refIdxL1]) is less than 0.
- MotionModelIdc[xCb][yCb] is equal to 0.
- merge_subblock_flag[xCb][yCb] is equal to 0.
- GbiIdx[xCb][yCb] is equal to 0.
- ciip_flag is 0: So when CIIP is not applied, BDOF can be applied.
- cIdx is 0.

[259] Referring to Table 24 above, BDOF may be limited in application according to whether CIIP should be applied or not by adding a condition on whether CIIP mode should be applied or not (eg, ciip_flag). For example, when ciip_flag is 0 (ie, when CIIP mode is not applied), BDOF may be applied by determining that a condition for whether or not to apply CIIP mode is satisfied.

[260] Accordingly, in an embodiment, along with a condition (for example, ciip_flag) for whether or not to apply the CIIP mode, whether or not to apply BDOF can be determined based on the remaining application conditions listed in Table 24 above.

[261] In other words, the decoding equipment can determine whether or not all the conditions listed in Table 24 above are satisfied, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying BDOF. When even any of the conditions listed in Table 24 above are not satisfied, the decoding equipment may not apply BDOF. The encoding equipment may also apply the conditions of the above Table 24, and the encoding equipment may perform true bi-prediction through a corresponding method in the decoding equipment.

[262] The method for performing true bi-prediction based on the conditions listed in Table 23 or 24 above can be independently applied to DMVR and BDOF, or can be applied under identical conditions for DMVR and BDOF.

[263] The methods listed in Tables 3 - 24 according to the embodiment of the present disclosure described above may be used in combination. In other words, whether DMVR and BDOF should be applied or not can be determined by checking the granularity application condition, and the conditions shown in Table 25 below may be applied.

[264] Tab. 25

DMVR - sps_dmvr_enabled_flag is set to 1.
- merge_flag[xCb][yCb] is equal to 1.
- mmvd_flag[xCb][yCb] is equal to 0.
- predFlagL0[0][0]=1 and predFlagL0[1][1]=1.
- DiffPicOrderCnt(currPic, RefPicList0[refIdxL0])-DiffPicOrderCnt(currPic, RefPicList1[refIdxL1]) is 0.
- CbHeight is greater than or equal to 8.
- CbHeight*CbWidth is greater than or equal to 64. BDOF - sps_bdof_enabled_flag is equal to 1.
- predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx] are equal to 1.
- DiffPicOrderCnt(currPic, refPicList0[refIdxL0])*DiffPicOrderCnt(currPic, refPicList1[refIdxL1]) is less than 0.
- MotionModelIdc[xCb][yCb] is equal to 0.
- merge_subblock_flag[xCb][yCb] is equal to 0.
- GbiIdx[xCb][yCb] is equal to 0.
- cIdx is 0. DMVR+BDOF - predFlagL0[0][0]=1 and predFlagL0[1][1]=1.
- DiffPicOrderCnt(currPic, RefPicList0[refIdxL0])-DiffPicOrderCnt(currPic, RefPicList1[refIdxL1]) is 0.
- mmvd_flag[xCb][yCb] is equal to 0.
- MotionModelIdc[xCb][yCb] is equal to 0.
- merge_subblock_flag[xCb][yCb] is equal to 0.
- GbiIdx[xCb][yCb] is equal to 0.

[265] Referring to the above Table 25, a condition may be used in bilateral prediction when DMVR or BDOF is applied, a condition applied in the case of true bi-prediction in which the distances between the current frame and bilateral reference frames are identical to each other, a condition applied when MMVD mode is not used, a condition applied when affine prediction is not satisfied, a condition applied when subblock-based aggregation mode is applied, and a condition applied when the GBi index is the default value. In other words, whether or not to apply DMVR or BDOF can be determined according to whether the conditions are satisfied or not.

[266] Additionally, a condition for determining whether or not the combining mode for DMVR is satisfied, or for determining whether or not the current block is a luminance block for BDOF may be added.

[267] The application conditions listed in Table 25 above are examples, and it is obvious that the various conditions listed in the above embodiments (in the embodiments in Tables 3-24 above) can be used in combination.

[268] Meanwhile, in DMVR, the SAD function is adapted as a cost function instead of the mean removed SAD (MRSAD) function taking into account the decoding complexity. However, when the GBi index is not the default (eg, when GbiIdx is not 0), the two reference blocks may have different weights, and as a result, DMVR using SAD may not be preferred. Accordingly, the DMVR condition can be fixed taking into account the GBi index. According to the experimental result, a 0.00% change in RD rate is shown with 100% encoding and decoding execution time by comparing with the VVC Test Model (VTM) 4.0.

[269] In the case of the Versatile Video Coding (VVC) standard, the DMVR process may be executed when all the conditions listed in Table 26 below are satisfied.

[270] Tab. 26

- sps_dmvr_enabled_flag is equal to 1
- merge_flag[xCb][yCb] is equal to 1
- both predFlagL0[0][0] and predFlagL1[0][0] are equal to 1
- mmvd_flag[xCb][yCb] is equal to 0
DiffPicOrderCnt(currPic, RefPicList[0][refIdxL0]) is equal to DiffPicOrderCnt(RefPicList[1][refIdxL1], currPic)
- cbHeight is greater than or equal to 8
- cbHeight*cbWidth is greater than or equal to 64

[271] In the current DMVR, a block that does not match can be searched by comparison with the SAD of reference blocks that must be weighted and averaged further. In the present disclosure, since the two reference blocks may have different weights, the DMVR condition may be determined taking into account such a case. According to an embodiment of the present disclosure, DMVR may not be performed for a block in which the GBi index is not the default value. In this case, the application condition of DMVR can be shown in Table 27 below.

[272] Tab. 27

- sps_dmvr_enabled_flag is equal to 1
- merge_flag[xCb][yCb] is equal to 1
- both predFlagL0[0][0] and predFlagL1[0][0] are equal to 1
- mmvd_flag[xCb][yCb] is equal to 0
- GbiIdx[xCb][yCb] is equal to 0.
- DiffPicOrderCnt(currPic, RefPicList[0][refIdxL0]) is equal to DiffPicOrderCnt(RefPicList[1][refIdxL1], currPic)
- cbHeight is greater than or equal to 8
- cbHeight*cbWidth is greater than or equal to 64

[273] As shown in the above Table 27, when the GBi index value (eg, GbiIdx) is 0, a DMVR execution condition may be added. In other words, when the value of the GBi index (eg, GbiIdx) is not equal to 0, different weights are applied to the two reference blocks (i.e., the reference block referenced for L0 prediction and the reference block referenced for L1 prediction), and as a result, in this case, DMVR may be limited such that it does not hold.

[274] Meanwhile, BDOF is executed when the current block (ie, the current coding unit; the current CU) satisfies the true biprediction condition. When the optical flow equation is considered to be designed to predict the motion of an object that is moving at a predetermined speed (i.e., momentum), the current true biprediction condition does not represent the optimal condition for applying BDOF. Accordingly, the BDOF condition may be fixed taking into account the distance of the reference frame. According to the experimental result, a 0.01% change in RD rate is shown with 100% encoding and decoding execution time by comparing with the VVC Test Model (VTM) 4.0.

[275] FIG. 9 is a diagram illustrated to describe the principle of BDOF.

[276] As described above, BDOF is designed to improve motion compensation performance by using the concept of optical flow. According to the BDOF, as illustrated in FIG. 9, it can be assumed that the object moves at a predetermined speed (constant motion), and while the object moves, the brightness of each pixel does not change. In case of assumption, the optical flow equation can be expressed as Equation 1 below.

[277] equation 1

δ(i, j)=A-B

=I(x+δx, y+δy, t+δt)- I(x-δx, y-δy, t-δt)

[278] As described above, when the current CU satisfies the true biprediction condition, BDOF is executed. However, the true biprediction condition does not mean the case in which the object is moving at a predetermined speed. Accordingly, the present disclosure provides a method that can apply BDOF when an object has a predetermined motion, and can improve motion compensation performance.

[279] According to an embodiment of the present disclosure, in the above equation 1, as δt, BDOF can be applied when the distance from the L0 reference frame (link 0 of FIG. 9) and the distance from the L1 reference frame (link 1 of FIG. 9) based on the current frame are identical to each other. In this case, the application condition of BDOF may change as shown in Table 28 below.

[280] Tab. 28

The currPic variable indicates the current frame, and the bdofFlag variable is retrieved as follows:
- If all of the following conditions are true, bdofFlag is set to true.
--- sps_bdof_enabled_flag is equal to 1.
--- predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx] are equal to 1.
--- DiffPicOrderCnt(currPic, RefPicList[0][refIdxL0])*DffPicOrderCnt(currPic, RefPicList[1][refIdxL1]) is less than 0.
--- DiffPicOrderCnt(currPic, RefPicList[0][refIdxL0]) is equal to DiffPicOrderCnt(RefPicList[1][refIdxL1], currPic)
--- MotionModelIdc[xCb][yCb] is equal to 0.
--- merge_subblock_flag[xCb][yCb] is equal to 0.
--- GbiIdx[xCb][yCb] is equal to 0.
--- cIdx is 0.
- Otherwise, bdofFlag is set to false.

[281] Referring to Table 28 above, the reference frame distance condition from the BDOF application conditions (eg, DiffPicOrderCnt) is modified to apply BDOF only when the object has a predetermined moving speed. For example, whether or not DiffPicOrderCnt(currPic, RefPicList[0][refIdxL0]) and DiffPicOrderCnt(RefPicList[1][refIdxL1], currPic) are identical to each other is determined to determine whether or not the distance between the current frame and the L0 reference frame (ie, the reference frame in the L0 reference frame list) and the distance between the current frame and the L1 reference frame (ie, the reference frame in the L1 reference frame list) are identical to each other. In other words, BDOF can only be applied when the distance from the L0 reference frame and the distance from the L1 reference frame based on the current frame are identical to each other. In this regard, a condition in which the distances between bilateral reference frames are identical based on the current frame is used to determine whether true bi-prediction is performed or not, and an object that moves at a predetermined speed is included. BDOF is applied to the block satisfying the condition to obtain a more refined motion information detail result.

[282] Additionally, in an embodiment, together with conditions (e.g., DiffPicOrderCnt) associated with the distance to the reference frame, whether or not BDOF should be applied can be determined based on the remaining application conditions listed in Table 28 above.

[283] In other words, the decoding equipment can determine whether or not all the conditions listed in Table 28 above are satisfied, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying BDOF. When even any of the conditions listed in Table 28 above are not satisfied, the decoding equipment may not apply BDOF. The encoding equipment may also apply the conditions of the above Table 28, and the encoding equipment may perform true bi-prediction through a corresponding method in the decoding equipment.

[284] Meanwhile, the present disclosure provides a method for determining whether or not BDOF should be applied according to the block size. Table 29 below shows a case where a block size limit according to an embodiment of the present disclosure is included as an application condition.

[285] Tab. 29

The currPic variable indicates the current frame, and the bdofFlag variable is retrieved as follows:
- If all of the following conditions are true, bdofFlag is set to true.
--- sps_bdof_enabled_flag is equal to 1.
--- predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx] are equal to 1.
--- DiffPicOrderCnt(currPic, RefPicList[0][refIdxL0])*DffPicOrderCnt(currPic, RefPicList[1][refIdxL1]) is less than 0.
--- MotionModelIdc[xCb][yCb] is equal to 0.
--- merge_subblock_flag[xCb][yCb] is equal to 0.
--- GbiIdx[xCb][yCb] is equal to 0.
--- cIdx is 0.
--- cbHeight is greater than or equal to 8
--- cbHeight*cbWidth is greater than or equal to 64
- Otherwise, bdofFlag is set to false.

[286] Referring to the above table 29, when applying BDOF, when the height of the current block (for example, CbHeight) is equal to or greater than 8, the case in which the size (for example, CbHeight*CbWidth) of the current block is equal to or greater than 64 may be added as conditions.

[287] Accordingly, in the embodiment, together with the condition for whether the height of the current block 8 is equal to or greater than or not, and the condition for whether the height*width of the current block 64 is equal to or greater than or not, whether or not BDOF should be applied, may be determined by using the remaining conditions listed in Table 29 above.

[288] In other words, the decoding equipment can determine whether or not all the conditions listed in Table 29 above are satisfied, and when all the conditions are satisfied, the decoding equipment can perform true bi-prediction by applying BDOF. When even any of the conditions listed in Table 29 above are not satisfied, the decoding equipment may not apply BDOF. The encoding equipment can also apply the conditions of the above Table 29, and the encoding equipment can perform true bi-prediction through a corresponding method in the decoding equipment.

[289] The DMVR process described in the present disclosure may be implemented according to the specification shown in Table 30 below. Table 30 below shows one example of a SAD-based motion vector refinement process as an embodiment of the present disclosure.

[290] Tab. thirty

8.5.3.1. General information
The inputs to this process are as follows:
- luminance signal location (xSb, ySb) indicating the upper left sample of the current encoding subblock relative to the upper left luminance signal sample of the current frame,
- sbWidth variable indicating the width of the current encoding subblock in luminance signal samples,
- variable sbHeight indicating the height of the current encoding subblock in luminance signal samples,
- vectors mvL0 and mvL1 of the brightness signal motion with fractional sampling accuracy of 1/16,
- selected arrays refPicL0L and refPicL1L of samples of reference frames of brightness signals. The findings from this process are:
- delta vectors dMvL0 and dMvL1 of the brightness signal movement. The variable subPelFlag is set to 0. and the variables srRange, offsetH0, offsetH1, offsetV0 and offsetV1 are set to 2.
Both components of the delta vectors dMvL0 and dMvL1 of the motion of the luminance signal are set equal to zero and are modified as follows:
For each X equal to 0 or 1, a predSamplesLXL array of prediction luminance signal sample values at (sbWidth+2*srRange)x(sbHeight+2*srRange) is retrieved by activating the fractional example bilinear interpolation process specified in 8.5.3.2.1, with location (xSb, ySb) of the luminance signal, prediction block width equal to (sbWidth+2*srRange), prediction block height equal to (sbHeight+2*srRange), refPicLXL array of reference frame samples, motion vector mvLX and detail search range srRange in quality of inputs.
The list sadList[i] with i=0...8 is retrieved by activating the sum of absolute differences process specified in 8.5.3.3 with sbWidth, sbHeight, offsetH0, offsetH1, offsetV0, offsetV1, predSamplesL0L and predSamplesL1L as inputs.
When sadList[4] is greater than or equal to 4*(sbHeight>>1)*sbWidth, the following applies:
The variable bestIdx is retrieved by activating the array entry selection process specified in section 8.5.3.4 with the list sadList[i] with i=0...8 as input.
If bestIdx is 4, subPelFlag is set to 1.
Otherwise, the following applies: dX=bestIdx% 3-1 (8,436)
- dY=bestIdx/3-l (8 437)
- dMvL0[0]+=16*dX (8 438)
- dMvL0[1]+=16*dY (8 439)
- offsetH0+=dX (8 440)
- offsetV0+=dY (8 441)
- offsetH1 -=dX (8 442)
- offsetV1 -=dY (8 443)
The list sadList[i] with i=0...8 is modified by activating the sum of absolute differences process specified in 8.5.3.3 with sbWidth, sbHeight, offsetH0, offsetH1, offsetV0, offsetV1, predSamplesL0L and predSamplesL1L as inputs.
[291] The variable bestIdx is modified by activating the array entry selection process specified in section 8.5.3.4 with the list sadList[i] with i=0... 8 as input.
If bestIdx is 4, subPelFlag is set to 1
Otherwise (bestIdx is not equal to 4), the following applies:
- dMvL0[0]+=16*(bestIdx% 3-1) (8 444)
- dMvL0[1]+=16*(bestIdx/3-1) (8 445)
When subPelFlag is 1, the parametric motion vector refinement process specified in section 8.5.3.5 is activated, with sadList[i] with i=0...8 and delta motion vector dMvL0 as inputs and modified dMvL0 as output.
The motion delta vector dMvL1 is extracted as follows:
- dMvL1[0]=-dMvL0[0] (8 446)
- dMvL1[1]=-dMvL0[1] (8 447)

[292] FIG. 10 is a flowchart schematically illustrating an encoding method that can be performed by encoding equipment according to an embodiment of this disclosure.

[293] The method disclosed in FIG. 10 may be performed by the encoding equipment 200 disclosed in FIG. 2. In particular, steps S1000 to S1010 in FIG. 10 may be performed by the prediction module 220 and the inter-prediction module 221 disclosed in FIG. 2, and step S1020 of FIG. 10 may be executed by the residual processor 230 disclosed in FIG. 2, and step S1030 of FIG. 10 may be performed by the entropy encoder 240 disclosed in FIG. 2. Additionally, the method disclosed in FIG. 10 may include the above-described embodiments in this disclosure. Accordingly, a detailed explanation of the contents duplicated with respect to the above-described embodiments in FIGS. 10, omitted or simplified.

[294] Referring to FIG. 10, the encoding equipment can extract the L0 detailed motion vector and the L1 detailed motion vector by applying DMVR based on the L0 motion vector and L1 motion vector of the current block (S1000).

[295] As an embodiment, when inter-prediction is performed on the current block, the encoding equipment may extract motion information (motion vector, reference frame index, etc.) of the current block. For example, the encoding equipment can search for blocks similar to the current block in a predetermined area (search area) of reference frames through motion estimation, and can extract a reference block having a difference with respect to the current block that is minimal or equal to or less than a predetermined level. Based on this, the encoding equipment can extract a reference frame index indicating the reference frame in which the reference block is located, and can extract a motion vector based on the difference in location between the reference block and the current block.

[296] Additionally, the encoding equipment may determine the inter-prediction mode applied to the current block from various prediction modes. The encoding equipment may compare the RD costs for different prediction modes with each other and may determine the optimal prediction mode for the current block.

[297] For example, the encoding equipment may determine whether or not to apply the combining mode as the optimal prediction mode for the current block. In the case of applying the combining mode to the current block, the encoding equipment can configure a list of possible combining options based on neighboring blocks relative to the current block and can generate combining index information. In particular, the encoding equipment may extract a reference block having a difference with respect to the current block that is minimal or equal to or less than a predetermined level from the reference blocks (i.e., neighboring blocks) indicated by combination candidates included in the candidate list associations. In this case, the combining candidate associated with the extracted reference block can be selected, and combining index information indicating the selected combining candidate can be generated and signaled to the decoding equipment. The motion information of the current block may be extracted using the motion information of the selected combining candidate.

[298] Here, the motion information may include information such as a motion vector, a reference frame index, and the like. and may include L0 motion information and/or L1 motion information depending on the type of inter-prediction (L0 prediction, L1 prediction, bi-prediction, etc.). When bi-prediction is applied to the current block, the motion information may include a motion vector (L0 motion vector) in the L0 direction and a motion vector (L1 motion vector) in the L1 direction. Additionally, the motion information may include an L0 reference frame index and an L0 reference frame indicated by an L0 reference frame index in the L0 reference frame list, and an L1 reference frame index and an L1 reference frame indicated by an L1 reference frame index. -frame in the list of reference L1 frames.

[299] Thus, when the combining mode is applied, the encoding equipment can extract the L0 motion vector and L1 motion vector of the neighboring block indicated by the combining index information from neighboring blocks relative to the current block, and can use them as L0 and L1 -movement vectors of the current block.

[300] The encoding equipment can extract detailed L0 and L1 motion vectors by applying DMVR to the L0 and L1 motion vectors of the current block extracted as described above, taking into account encoding efficiency, complexity, prediction performance, and the like. In this case, the encoding equipment can determine whether or not to apply DMVR to the current block using various application conditions described in Tables 1-30 above.

[301] As an embodiment, the encoding equipment may extract DMVR flag information associated with indicating whether the DMVR is applied or not, and may determine whether or not the DMVR should be applied based on the DMVR flag information. For example, when the combining mode is applied to the current block, the encoding equipment may set a value (eg, true or 1) of the DMVR flag information to indicate that the DMVR is applied to the current block. Further, according to an embodiment, the encoding equipment can extract DMVR flag information by determining whether conditions such as the case in which DMVR-based mutual bi-prediction is activated, the case in which bi-prediction performed based on the L0 reference frame, and reference L1 frame is applied to the current block, the case in which the motion vector difference (MMVD) combining mode is not applied to the current block, the case in which the prediction mode in which inter prediction and intra prediction are combined is not applied to the current block , the case in which the first POC difference between the current frame and the L0 reference frame and the second POC difference between the current frame and the L1 reference frame are identical, the case in which the biprediction weight index information value of the current block is 0, the case in in which the height of the current block is equal to or greater than 8, the case in which the width of the current block is equal to or greater than 8, the case in which the height x width of the current block is greater than 8x8. For example, if all of the above conditions are satisfied, the value of the DMVR flag information may be set to true or 1, and if at least one of the above conditions is not satisfied, the value of the DMVR flag information may be set to false or 0.

[302] Here, when retrieving DMVR flag information, the above-mentioned DMVR application conditions are merely exemplary, and various combinations of the conditions of Tables 1 to 30 may be used.

[303] When it is determined that DMVR is applied based on the DMVR flag information (eg, when the value of the DMVR flag information is retrieved to true or 1), the encoding equipment may apply DMVR to the L0 and L1 motion vectors of the current block .

[304] In an embodiment, the encoding equipment may extract minimum sum of absolute differences (SAD) based on reference samples (i.e., L0 prediction samples) in the L0 reference frame extracted based on the L0 motion vector, and reference samples ( i.e., predictive L1 samples) in a reference L1 frame extracted based on the L1 motion vector. Also, the encoding equipment can extract the detailed L0 motion vector and the detailed L1 motion vector based on the sample position corresponding to the minimum SAD. Thus, the detailed L0 motion vector may be a motion vector associated with indicating a sample location corresponding to a minimum SAD in the L0 reference frame, and the detailed L1 motion vector may be a motion vector associated with indicating a sample location corresponding to the minimum SAD in the reference L1 frame. Since the process of extracting detailed motion vector by applying DMVR is described in detail with reference to FIG. 4 and 5, its explanation is omitted below.

[305] The encoding equipment may extract prediction samples for the current block based on L0 prediction samples extracted based on the detailed L0 motion vector and L1 prediction samples extracted based on the detailed L1 motion vector (S1010).

[306] Here, L0 prediction samples are extracted based on reference samples indicated by the detailed L0 motion vector in the L0 reference frame, and L1 prediction samples are extracted based on reference samples indicated by the detailed L1 motion vector in the L1 reference frame. frame.

[307] When retrieving prediction samples, the encoding equipment may apply a BDOF process to the current block taking into account encoding efficiency, complexity, prediction performance, and the like.

[308] For example, the encoding equipment may determine whether or not to apply the BDOF process based on whether the first POC difference between the current frame and the L0 reference frame and the second POC difference between the current frame and the L1 reference are identical. -frame. Here, the number in the sequence of frames (POC) may represent the order output from the DPB. The first POC may represent the distance between the current frame and the L0 reference frame and may be calculated using, for example, the function DiffPicOrderCnt(currPic, RefPicList0[refIdxL0]). The second POC may represent the distance between the current frame and the L1 reference frame and may be calculated using, for example, the function DiffPicOrderCnt(currPic, RefPicList1[refIdxL1]).

[309] In this case, the encoding equipment can determine whether or not to apply BDOF to the current block by using various application conditions described in Tables 1-30 above.

[310] In an embodiment, the encoding equipment can extract BDOF flag information associated with indicating whether BDOF is applied or not, and can determine whether BDOF should be applied or not based on the BDOF flag information. For example, when the first POC difference and the second POC difference are identical, the encoding equipment may set a value (eg, true or 1) of the BDOF flag information to indicate that BDOF is applied to the current block. Further, according to the embodiment, the encoding equipment can extract BDOF flag information by determining whether conditions such as the case in which BDOF-based inter-prediction is activated, the case in which bi-prediction performed based on the L0 reference frame, and reference L1 frame is applied to the current block, a case in which the affine mode is not applied to the current block, a case in which the sub-block-based aggregation mode is not applied to the current block, a case in which the prediction mode is in which mutual prediction and internal prediction, does not apply to the current block, the case in which the biprediction weight index information value of the current block is 0, the case in which the height of the current block is equal to or greater than 8, the case in which the width of the current block is equal to or greater than 8, the case in where the height x width of the current block is greater than 8x8. For example, if all of the above conditions are satisfied, the value of the BDOF flag information may be set to true or 1, and if at least one of the above conditions is not satisfied, the value of the BDOF flag information may be set to false or 0.

[311] Here, when retrieving BDOF flag information, the above-mentioned BDOF application conditions are merely exemplary, and various combinations of the conditions of Tables 1 to 30 may be used.

[312] When a BDOF process is applied to the current block based on the value of the BDOF flag information (eg, when the value of the BDOF flag information is retrieved as true or 1), the encoding equipment may calculate a gradient for the predicted L0 samples retrieved based on detailed L0 motion vector, and a gradient for predictive L1 samples extracted based on the detailed L1 motion vector. Also, the encoding equipment can finally extract (fine-grained) prediction samples based on the gradient for L0 prediction samples and the gradient for L1 prediction samples.

[313] The encoding equipment can extract residual samples based on the predictive samples (S1020) and can encode image information including information regarding the residual samples (S1030).

[314] Thus, the encoding equipment can extract residual samples based on the original samples of the current block and the predicted samples of the current block. Also, encoding equipment can generate information regarding residual samples. Here, information regarding the residual samples may include value information, location information, transform technology, transform kernel, and quantization parameter information for quantized transform coefficients extracted by performing transform and quantization of the residual samples.

[315] The encoding equipment may encode information regarding the residual samples to output the encoded information in the form of a bitstream, and may transmit the encoded information to the decoding equipment via a network or storage medium.

[316] Additionally, the encoding equipment may encode the information extracted in steps S1000 to S1030 described above, and output the encoded information as a bit stream. For example, association flag information, association index information, L0 reference frame index, L1 reference frame index, etc. may be included in image information to be encoded, and the encoded image information may be signaled to decoding equipment.

[317] FIG. 11 is a flowchart schematically illustrating a decoding method that can be performed by decoding equipment according to an embodiment of this document.

[318] The method disclosed in FIG. 11 may be performed by the decoding equipment 300 disclosed in FIG. 3. In particular, steps S1100-S1110 of FIG. 11 may be performed by the prediction module 330 and the inter-prediction module 332 disclosed in FIG. 3, and step S1120 of FIG. 11 may be performed by the adder 340 disclosed in FIG. 3. Additionally, the method disclosed in FIG. 11 may include the above-described embodiments herein. Accordingly, a detailed explanation of the contents duplicated with respect to the above-described embodiments in FIGS. 11, omitted or simplified.

[319] Referring to FIG. 11, the decoding equipment can extract the detailed L0 motion vector and the detailed L1 motion vector by applying DMVR based on the L0 motion vector and L1 motion vector of the current block (S1100).

[320] In an embodiment, the decoding equipment may determine a prediction mode for the current block based on prediction information signaled from the encoding equipment. Also, the decoding equipment can extract motion information (motion vector, reference frame index, etc.) of the current block based on the prediction mode. Here, the prediction mode may include a skip mode, an aggregation mode and an (A)MVP mode and the like.

[321] For example, when a combining mode is applied to a current block, the decoding equipment may construct a list of combining candidates based on neighboring blocks relative to the current block and select one combining candidate from among the combining candidates included in the list of combining candidates. In this case, one join candidate may be selected from the list of join candidates based on the join index information described above. The decoding equipment can extract the motion information of the current block by using the motion information of the selected combining candidate. That is, motion information of a merge candidate selected by the merge index from the merge candidates included in the list of merge candidates can be used as motion information of the current block.

[322] Here, the motion information may include information such as a motion vector, a reference frame index, and the like. and may include L0 motion information and/or L1 motion information depending on the type of inter-prediction (L0 prediction, L1 prediction, bi-prediction, etc.). When bi-prediction is applied to the current block, the motion information may include a motion vector (L0 motion vector) in the L0 direction and a motion vector (L1 motion vector) in the L1 direction. Additionally, the motion information may include an L0 reference frame index and an L0 reference frame indicated by an L0 reference frame index in the L0 reference frame list, and an L1 reference frame index and an L1 reference frame indicated by an L1 reference frame index. -frame in the list of reference L1 frames.

[323] Thus, when the combining mode is applied, the decoding equipment can extract the L0 motion vector and L1 motion vector of the neighboring block indicated by the combining index information from neighboring blocks relative to the current block, and can use them as L0 and L1 -movement vectors of the current block.

[324] Decoding equipment can extract detailed L0 and L1 motion vectors by applying DMVR to the L0 and L1 motion vectors of the current block extracted as described above. In this case, the decoding equipment can determine whether or not to apply the DMVR to the current block using various application conditions described in Tables 1-30 above.

[325] In an embodiment, the decoding equipment may extract DMVR flag information associated with indicating whether DMVR is applied or not, and determine whether DMVR should be applied or not based on the DMVR flag information. For example, when the combining mode is applied to the current block, the decoding equipment may set a value (eg, true or 1) of the DMVR flag information to indicate that the DMVR is applied to the current block. Further, according to the embodiment, the decoding equipment can extract the DMVR flag information by determining whether conditions such as the case in which DMVR-based mutual bi-prediction is activated, the case in which bi-prediction performed based on the L0 reference frame, and reference L1 frame is applied to the current block, the case in which the motion vector difference (MMVD) combining mode is not applied to the current block, the case in which the prediction mode in which inter prediction and intra prediction are combined is not applied to the current block , the case in which the first POC difference between the current frame and the L0 reference frame and the second POC difference between the current frame and the L1 reference frame are identical, the case in which the biprediction weight index information value of the current block is 0, the case in in which the height of the current block is equal to or greater than 8, the case in which the width of the current block is equal to or greater than 8, the case in which the height x width of the current block is greater than 8x8. For example, if all of the above conditions are satisfied, the value of the DMVR flag information may be set to true or 1, and if at least one of the above conditions is not satisfied, the value of the DMVR flag information may be set to false or 0.

[326] Here, when retrieving DMVR flag information, the above-mentioned DMVR application conditions are merely exemplary, and various combinations of the conditions of Tables 1 to 30 may be used.

[327] When it is determined that DMVR is applied based on the DMVR flag information (eg, when the value of the DMVR flag information is retrieved as true or 1), the decoding equipment may apply the DMVR to the L0 and L1 motion vectors of the current block .

[328] In an embodiment, the decoding equipment may extract the minimum sum of absolute differences (SAD) based on the reference samples (i.e., L0 prediction samples) in the L0 reference frame extracted based on the L0 motion vector, and the reference samples ( i.e., predictive L1 samples) in a reference L1 frame extracted based on the L1 motion vector. Also, the decoding equipment can extract the detailed L0 motion vector and the detailed L1 motion vector based on the sample position corresponding to the minimum SAD. Thus, the detailed L0 motion vector may be a motion vector associated with indicating a sample location corresponding to a minimum SAD in the L0 reference frame, and the detailed L1 motion vector may be a motion vector associated with indicating a sample location corresponding to the minimum SAD in the reference L1 frame. Since the process of extracting detailed motion vector by applying DMVR is described in detail with reference to FIG. 4 and 5, its explanation is omitted below.

[329] The decoding equipment may extract prediction samples for the current block based on L0 prediction samples extracted based on the detailed L0 motion vector and L1 prediction samples extracted based on the detailed L1 motion vector (S1110).

[330] Here, L0 prediction samples are extracted based on reference samples indicated by the detailed L0 motion vector in the L0 reference frame, and L1 prediction samples are extracted based on reference samples indicated by the detailed L1 motion vector in the L1 reference frame. frame.

[331] When retrieving prediction samples, the decoding hardware may apply a BDOF process to the current block to refine the prediction samples.

[332] For example, the decoding equipment may determine whether or not to apply the BDOF process based on whether the first POC difference between the current frame and the L0 reference frame and the second POC difference between the current frame and the L1 reference are identical. -frame. Here, the number in the sequence of frames (POC) may represent the order output from the DPB. The first POC may represent the distance between the current frame and the L0 reference frame and may be calculated using, for example, the function DiffPicOrderCnt(currPic, RefPicList0[refIdxL0]). The second POC may represent the distance between the current frame and the L1 reference frame and may be calculated using, for example, the function DiffPicOrderCnt(currPic, RefPicList1[refIdxL1]).

[333] In this case, the decoding equipment can determine whether or not to apply BDOF to the current block by using various application conditions described in Tables 1-30 above.

[334] In an embodiment, the decoding equipment can extract BDOF flag information associated with indicating whether BDOF is applied or not, and can determine whether BDOF should be applied or not based on the BDOF flag information. For example, when the first POC difference and the second POC difference are identical, the decoding equipment may set a value (eg, true or 1) of the BDOF flag information to indicate that BDOF is applied to the current block. Further, according to the embodiment, the decoding equipment can extract BDOF flag information by determining whether conditions such as the case in which BDOF-based inter-prediction is activated, the case in which bi-prediction performed based on the L0 reference frame, and reference L1 frame is applied to the current block, a case in which the affine mode is not applied to the current block, a case in which the sub-block-based aggregation mode is not applied to the current block, a case in which the prediction mode is in which mutual prediction and internal prediction, does not apply to the current block, the case in which the biprediction weight index information value of the current block is 0, the case in which the height of the current block is equal to or greater than 8, the case in which the width of the current block is equal to or greater than 8, the case in where the height x width of the current block is greater than 8x8. For example, if all of the above conditions are satisfied, the value of the BDOF flag information may be set to true or 1, and if at least one of the above conditions is not satisfied, the value of the BDOF flag information may be set to false or 0.

[335] Here, when retrieving BDOF flag information, the above-mentioned BDOF application conditions are merely exemplary, and various combinations of the conditions of Tables 1 to 30 may be used.

[336] When a BDOF process is applied to the current block based on the value of the BDOF flag information (eg, when the value of the BDOF flag information is extracted to true or 1), the decoding equipment may calculate a gradient for the predicted L0 samples extracted based on detailed L0 motion vector, and a gradient for predictive L1 samples extracted based on the detailed L1 motion vector. Also, the decoding equipment may finally extract (fine-grained) prediction samples based on the gradient for the L0 prediction samples and the gradient for the L1 prediction samples.

[337] The decoding equipment may generate reconstructed samples for the current block based on the predicted samples (S1120).

[338] In an embodiment, the decoding equipment may directly use the predicted samples as reconstructed samples according to a prediction mode, or may generate reconstructed samples by adding residual samples to the predicted samples.

[339] If a residual sample for the current block exists, the decoding equipment may receive information regarding the residual for the current block. The residual information may include conversion factors regarding the residual samples. The decoding equipment may extract residual samples (or an array of residual samples) for the current block based on the residual information. The decoding equipment may generate reconstructed samples based on the prediction samples and residual samples, and may extract a reconstructed block or a reconstructed frame based on the reconstructed samples.

[340] In the above embodiments, although the methods are described based on process flow diagrams in the form of a sequence of steps or units, embodiments of this document are not limited to the order of the steps, and some steps may be performed in an order different from the order of other steps, or can be performed simultaneously with other steps. In addition, those skilled in the art will understand that the steps shown in the flowchart are not the only possible ones, and the steps may include additional steps, or that one or more steps in the flowchart can be deleted without affecting the scope of rights of this document.

[341] The above method according to this document may be implemented in software form, and the encoding equipment and/or decoding equipment according to this document may be included in equipment for performing image processing, such as a television, computer, smartphone, set-top box or display device.

[342] In this document, when embodiments are implemented in software form, the above method is implemented as a module (process, function, etc.) for performing the above function. The module may be stored in a storage device and executed by a processor. The storage device may be located inside or outside the processor and coupled to the processor through various known means. The processor may include application specific integrated circuits (ASICs), other chipsets, logic circuits, and/or data processing devices. The storage device may include read-only memory (ROM), random access memory (RAM), flash memory, memory cards, storage media, and/or other storage devices. Thus, the embodiments described herein may be implemented and executed on/on a processor, microprocessor, controller, or chip. For example, the functional modules illustrated in the drawings may be implemented and executed on a computer, processor, microprocessor, controller, or chip. In this case, information regarding the instructions or algorithm to be implemented may be stored in a digital storage medium.

[343] In addition, the decoding equipment and encoding equipment to which this document applies may be included in a multimedia broadcasting device, a mobile communication terminal, a home theater system video device, a digital cinema system video device, a monitoring camera, an interactive video device, a real-time communications such as video communications, mobile streaming device, storage medium, video recording camera, video-on-demand (VoD) service delivery device, over-the-top (OTT) video device, over-the-top streaming service device Internet, three-dimensional video device, virtual reality (VR) style device, augmented reality (AR) style device, video telephony system device, terminal in a vehicle (for example, a terminal in a vehicle (including an autonomous vehicle), terminal in aircraft and marine terminal) and medical video device and can be used to process a video or data signal. For example, an over-the-top (OTT) video device may include a game console, Blu-ray player, Internet-enabled television, home theater system, smartphone, tablet PC, and digital video recorder (DVR).

[344] Moreover, the processing method to which this document applies may be created in the form of a program executed by a computer and may be stored on a computer-readable recording medium. Multimedia data having a data structure according to this document can also be stored on a computer-readable recording medium. A computer-readable recording medium includes all types of storage devices on which computer-readable data is stored. The computer-readable medium may include, for example, a Blu-ray disc (BD), a universal serial bus (USB), a ROM, a PROM, an EEPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical storage device. In addition, the computer-readable recording medium includes a medium implemented in the form of carriers (eg, when transmitted over the Internet). In addition, the bit stream generated using the encoding method may be stored in a computer-readable recording medium or may be transmitted over wired and wireless communication networks.

[345] Additionally, an embodiment of this document may be implemented as a computer program product using program code. The program code may be executed by a computer according to an embodiment of this document. The program code may be stored on a computer readable medium.

[346] FIG. 12 illustrates an example of a content streaming system to which the embodiments disclosed herein may be applied.

[347] Referring to FIG. 12, the content streaming system to which the embodiment(s) of this document applies may mainly include an encoding server, a streaming server, a web server, a media storage, a user device, and a media input device.

[348] The encoding server compresses content input from media input devices such as a smartphone, camera, video recorder, etc. into digital data to generate a bitstream and transmit the bitstream to the streaming server. As another example, when media input devices such as smartphones, cameras, video recorders, etc. directly generate a bit stream, the encoding server may be omitted.

[349] The bitstream may be generated by an encoding method or a bitstream generation method to which the embodiment(s) of this document applies, and the streaming server may temporarily store the bitstream while transmitting or receiving the bitstream.

[350] The streaming server transmits multimedia data to the user device based on the user's request through the web server, and the web server serves as a medium for informing the user regarding the service. When a user requests a required service from a web server, the web server delivers it to the streaming server and the streaming server transmits the media data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server serves to manage the command/response between devices in the content streaming system.

[351] The streaming server may receive content from a media storage and/or encoding server. For example, when content is received from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may store the bit stream for a predetermined time.

[352] Examples of a user device may include a mobile phone, a smartphone, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable media player (PMP), a navigation device, a tablet PC, tablet PCs, ultrabooks, wearable devices (such as smart watches, smart glasses, head-mounted displays), digital TVs, desktop computers, digital signage system, etc.

[353] Each server in a content streaming system may operate as a distributed server, in which case data received from each server may be distributed.

Claims (27)

1. A method for decoding images carried out by means of decoding equipment, wherein the method for decoding images comprises the steps of: extracting the L0 motion vector and L1 motion vector of the current block; extracting the detailed L0 motion vector and the detailed L1 motion vector by applying decoder-side motion vector detail (DMVR) based on the L0 motion vector and L1 motion vector of the current block; extracting prediction samples for the current block based on L0 prediction samples extracted based on the detailed L0 motion vector and L1 prediction samples extracted based on the detailed L1 motion vector; And generate reconstructed samples for the current block based on predicted samples, wherein the L0 prediction samples are extracted based on the L0 reference frame and the detailed L0 motion vector, and the L1 prediction samples are extracted based on the L1 reference frame and the detailed L1 motion vector, wherein predictive samples for the current block are retrieved by applying bidirectional optical flow (BDOF) to the current block based on BDOF flag information indicating whether BDOF should be applied to the current block, wherein the BDOF flag information is extracted based on whether or not the first POC difference between the current frame and the L0 reference frame and the second POC difference between the current frame and the L1 reference frame are identical, and whether or not the biprediction weight index information value of the current block is 0, and wherein, based on the case in which the first POC difference and the second POC difference are identical and the value of the bi-prediction weight index information of the current block is 0, the value of the BDOF flag information is set to a value associated with applying BDOF to the current block. 2. An image encoding method carried out by means of encoding equipment, wherein the image encoding method comprises the steps of: extracting the L0 motion vector and L1 motion vector of the current block; extracting the detailed L0 motion vector and the detailed L1 motion vector by applying decoder-side motion vector detail (DMVR) based on the L0 motion vector and L1 motion vector of the current block; extracting prediction samples for the current block based on L0 prediction samples extracted based on the detailed L0 motion vector and L1 prediction samples extracted based on the detailed L1 motion vector; extracting residual samples based on the predictive samples; And encode image information including information regarding residual samples, wherein the L0 prediction samples are extracted based on the L0 reference frame and the detailed L0 motion vector, and the L1 prediction samples are extracted based on the L1 reference frame and the detailed L1 motion vector, wherein predictive samples for the current block are retrieved by applying bidirectional optical flow (BDOF) to the current block based on BDOF flag information indicating whether BDOF should be applied to the current block, wherein the BDOF flag information is extracted based on whether or not the first POC difference between the current frame and the L0 reference frame and the second POC difference between the current frame and the L1 reference frame are identical, and whether or not the biprediction weight index information value of the current block is 0, and wherein, based on the case where the first POC difference and the second POC difference are identical and the value of the bi-prediction weight index information of the current block is 0, the value of the BDOF flag information is set to a value associated with applying BDOF to the current block. 3. A method for transmitting data to obtain image information, the method comprising the steps of: obtain a bitstream for an image, wherein the bitstream is generated based on the steps of: extracting the L0 motion vector and L1 motion vector of the current block, extracting the detailed L0 motion vector and the detailed L1 motion vector by applying decoder-side motion vector detail (DMVR) based on the L0 motion vector and L1 motion vector of the current block, extracting predictive samples for the current block based on predictive L0 samples extracted based on the detailed L0 motion vector and predictive L1 samples extracted based on the detailed L1 motion vector, extracting residual samples based on the predictive samples, encoding image information including includes information regarding residual samples; transmit data containing a stream of bits, wherein the L0 prediction samples are extracted based on the L0 reference frame and the detailed L0 motion vector, and the L1 prediction samples are extracted based on the L1 reference frame and the detailed L1 motion vector, wherein predictive samples for the current block are retrieved by applying bidirectional optical flow (BDOF) to the current block based on BDOF flag information indicating whether BDOF should be applied to the current block, wherein the BDOF flag information is extracted based on whether or not the first POC difference between the current frame and the L0 reference frame and the second POC difference between the current frame and the L1 reference frame are identical, and whether or not the biprediction weight index information value of the current block is 0, and wherein, based on the case where the first POC difference and the second POC difference are identical and the value of the bi-prediction weight index information of the current block is 0, the value of the BDOF flag information is set to a value associated with applying BDOF to the current block.

Images