RU2820148C2 - Image encoding and decoding equipment and image data transmission equipment - Google Patents

Abstract

FIELD: image and video encoding technologies.

SUBSTANCE: result is achieved by the fact that forming a list of possible options for combining current block based on information of mutual prediction mode, extracting movement information of the current unit based on a candidate selected from a list of possible combinations, predictive samples of the current block are formed based on the motion information and a reconstructed frame is formed based on the predicted samples. Image information includes first activation flag for combined interframe and intraframe prediction (CIIP) mode and second activation flag for prediction mode with segmentation, wherein based on the first activation flag value equal to 0, the second activation flag value equal to 0, the subunit combination flag value equal to 0, and the MMVD-combination flag value equal to 0, regular merging mode is applied and merging index information for the regular merging mode is included in the syntax of merging data.

EFFECT: high efficiency of encoding images.

11 cl, 20 dwg, 10 tbl

Description

State of the art

Field of technology to which the invention relates

[1] The present disclosure relates to a method for decoding images to perform inter prediction when a prediction mode is ultimately not selected for the current block, and to equipment for doing so.

Description of the Prior Art

[2] Recently, the demand for high-quality high-definition images/videos, for example, 4K, 8K or even higher ultra-high definition (UHD) images/videos, has been growing in various fields of technology. As the resolution or quality of images/videos becomes higher, a relatively larger amount of information or number of bits is transmitted than for traditional image/video data. Therefore, if image/video data is transmitted over a medium such as an existing wired/wireless broadband line or stored on a legacy storage medium, transmission and storage costs increase significantly.

[3] In addition, there is growing interest and demand for virtual reality (VR) and artificial reality (AR) content and immersive multimedia, such as holograms; and there is also an increase in broadcasting of images/videos exhibiting characteristics of the images/videos that differ from those of the actual image/video, for example, game images/videos.

[4] Therefore, a highly efficient image/video compression technology is required in order to efficiently compress and transmit, store or play back high-quality, high-resolution images/videos exhibiting various characteristics as described above.

The essence of the invention

[5] The present disclosure provides a method and equipment for improving the efficiency of image encoding.

[6] The present disclosure also provides a method and apparatus for extracting a predictive sample based on the default pooling mode.

[7] The present disclosure also provides a method and equipment for extracting a predictive sample by applying a regular merging mode as the default merging mode.

[8] In an aspect, a method for decoding images carried out by decoding equipment is provided. The method includes: receiving image information including inter-prediction mode information through a bit stream; generating a list of possible options for combining the current block based on information from the mutual prediction mode; retrieving motion information of the current block based on a candidate selected from a list of candidate combinations; and generating predictive samples of the current block based on the motion information, wherein a regular combining mode is applied to the current block based on the fact that a combining mode is available for the current block based on a common combining flag, and a motion vector difference (MMVD) combining mode, a sub-block mode combining, combined inter-combining and intra-prediction (CIIP) mode and segmentation mode, in which prediction is performed by dividing the current block into two segments, are not available and a list of possible combining options is generated by applying a regular combining mode to the current block.

[9] In an aspect, an image encoding method carried out by encoding equipment is provided. The method includes: determining the inter-prediction mode of the current block and generating inter-prediction mode information indicating the inter-prediction mode; generating a list of possible options for combining the current block based on the mutual prediction mode; generating selection information indicating one of the possible options included in the list of possible combination options; and encoding image information including inter-prediction mode information and selection information, wherein a regular combining mode is applied to the current block based on the combining mode being available for the current block based on a common combining flag, and a combining mode with a motion vector difference ( MMVD), sub-block combining mode, combined inter-combining and intra-prediction (CIIP) mode, and segmentation mode, in which prediction is performed by dividing the current block into two segments, are not available and a list of possible combining options is generated by applying the regular combining mode to the current block .

[10] In another aspect, there is provided a computer-readable storage medium storing encoded information instructing image decoding equipment to perform an image decoding method, wherein the image decoding method includes: obtaining image information including inter-prediction mode information and residual information through a stream of bits; generating a list of possible options for combining the current block based on information from the mutual prediction mode; retrieving motion information of the current block based on a candidate selected from a list of candidate combinations; and generating predictive samples of the current block based on the motion information, wherein a regular combining mode is applied to the current block based on the fact that a combining mode is available for the current block based on a common combining flag, and a motion vector difference (MMVD) combining mode, a sub-block mode combining, combined inter-combining and intra-prediction (CIIP) mode and segmentation mode, in which prediction is performed by dividing the current block into two segments, are not available and a list of possible combining options is generated by applying a regular combining mode to the current block.

Advantages of the invention

[11] According to the present disclosure, the overall efficiency of image/video compression can be improved.

[12] According to the present disclosure, inter-prediction can be efficiently performed by applying a default combining mode when the combining mode is not ultimately selected.

[13] According to the present disclosure, when the combining mode is not ultimately selected, the regular combining mode is applied, and motion information is extracted based on the candidate indicated by the combining index information, thereby effectively performing mutual prediction.

Brief description of drawings

[14] FIG. 1 schematically shows an example of a video/image coding system to which embodiments of the present disclosure apply.

[15] FIG. 2 is a diagram fundamentally illustrating the configuration of video/image encoding equipment to which embodiments of the present document can be applied.

[16] FIG. 3 is a diagram schematically illustrating the configuration of video/image decoding equipment to which embodiments of the present document can be applied.

[17] FIG. 4 illustrates the inter-prediction procedure.

[18] FIG. 5 is a view illustrating the merging mode of mutual prediction.

[19] FIG. 6 is a view illustrating a mode of a motion vector difference (MMVD) combining mode in mutual prediction.

[20] FIG. 7 roughly shows the motion expressed through the affine motion model.

[21] FIG. 8A and 8B approximately show CPMV for affine motion prediction.

[22] FIG. 9 roughly shows the case in which the affine MVF is defined in sub-block units.

[23] FIG. 10 is a view illustrating an affine combining mode or a subblock-based combining mode in inter-prediction.

[24] FIG. 11 is a view illustrating candidate positions in the affine combining mode or the subblock-based combining mode.

[25] FIG. 12 is a view illustrating SbTMVP in inter-prediction.

[26] FIG. 13 is a view illustrating a combined inter-frame combining and intra-frame prediction (CIIP) mode in inter-prediction.

[27] FIG. 14 is a view illustrating a segmentation mode in inter-prediction.

[28] FIG. 15 and 16 schematically show an example of a method for encoding video/images and related components according to embodiment(s) of the present disclosure.

[29] FIG. 17 and 18 schematically show an example of a method for decoding video/images and related components according to embodiment(s) of the present disclosure.

[30] FIG. 19 illustrates an example of a content streaming system to which the embodiments disclosed in the present disclosure may be applied.

Detailed Description of Embodiments

[31] The present disclosure may be modified in various ways and may have several exemplary embodiments. Accordingly, specific exemplary embodiments of the present disclosure are illustrated in the accompanying drawings and described in detail below. However, it is not intended to limit the present disclosure to specific embodiments. The terms used in the present description of the invention are used only to describe specific exemplary embodiments and not to limit the present disclosure. Singular forms are intended to include plural forms unless the context clearly indicates otherwise. It should be understood that the terms “include,” “have,” and the like, as used herein, indicate the presence of features, numbers, steps, operations, components, parts, or combinations thereof set forth in this specification, but do not exclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations of the foregoing.

[32] Meanwhile, each configuration in the drawings described herein is shown independently for convenience of description regarding various characteristic functions and does not mean that each configuration is implemented as separate hardware or separate software. For example, two or more components for each component may be combined to form one component, or one component may be divided into multiple components. Embodiments in which each component is integrated and/or separated are also included within the scope of this disclosure.

[33] In the present disclosure, “A or B” may mean “A only,” “B only,” or “both A and B.” In other words, "A or B" in the present disclosure may be interpreted as "A and/or B". For example, in the present disclosure, "A, B or C" means "A only", "B only", "C only" or "any and any combination of A, B and C".

[34] The slash (/) or comma (comma) used in this disclosure may indicate “and/or.” For example, "A/B" can mean "A and/or B". Accordingly, "A/B" can mean "only A", "only B" or "both A and B". For example, "A, B, C" can mean "A, B or C".

[35] In the present specification, “at least one of A and B” may mean “A only,” “B only,” or “both A and B.” Moreover, in the present specification, the expression “at least one of A or B” or “at least one of A and/or B” may be interpreted identically to “at least one of A and B”.

[36] Additionally, as used herein, “at least one of A, B, and C” may mean “A only,” “B only,” “C only,” or “any combination of A, B, and C.” In addition, "at least one of A, B or C" or "at least one of A, B and/or C" may mean "at least one of A, B and C".

[37] Additionally, parentheses used in the present description of the invention may mean "for example." In particular, in the case where "forecasting (intra) forecasting" is expressed, it may be indicated that "intra forecasting" is offered as an example of "forecasting". In other words, the term "prediction" in the present specification is not limited to "internal prediction", and it may be stated that "internal prediction" is offered as an example of "prediction". Additionally, even if "forecasting (ie, internal forecasting)" is expressed, it may be indicated that "internal forecasting" is offered as an example of "forecasting".

[38] In the present specification, technical features separately explained in one drawing may be implemented separately or may be implemented simultaneously.

[39] Exemplary embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. In addition, like reference numerals are used to identify like elements throughout the drawings, and identical descriptions of like elements may be omitted.

[40] FIG. 1 illustrates an example of a video/image coding system to which embodiments of the present disclosure may be applied.

[41] Referring to FIG. 1, the video/image encoding system may include a first device (source device) and a second device (receiver device). A source device may transmit encoded information or video/image data to a receiving device via a digital storage medium or network in the form of a file or stream.

[42] The source device may include a video source, encoding equipment, and a transmitter. The receiving device may include a receiver, decoding equipment, and a rendering module. The encoding equipment may be referred to as "video/image encoding equipment" and the decoding equipment may be referred to as "video/image decoding equipment". The transmitter may be included in the encoding equipment. The receiver may be included in the decoding equipment. The rendering module may include a display, and the display may be configured as a separate device or an external component.

[43] A video source may obtain video/image through a video/image capture, synthesis, or shaping process. The video source may include a video/image capture device and/or a video/image generation device. The video/image capturing device may include, for example, one or more cameras, video/image archives including previously captured video/images, and the like. The video/image generating apparatus may include, for example, computers, tablet computers and smartphones and may (electronically) generate video/images. For example, a virtual video/image may be generated via a computer or the like. In this case, the video/image capturing process can be replaced by the associated data generation process.

[44] Encoding equipment can encode the input video/image. Encoding hardware may perform a sequence of procedures such as prediction, transform, and quantization to achieve compression and encoding efficiency. Encoded data (encoded video/image information) may be output in the form of a bitstream.

[45] A transmitter may transmit information or encoded image/image data output in the form of a bitstream to a receiver of a receiving device via a digital storage medium or network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, and the like. The transmitter may include an element for generating a multimedia file through a predefined file format and may include an element for transmission through a broadcast/communication network. The receiver may receive/extract the bit stream and transmit the received bit stream to decoding equipment.

[46] The decoding equipment can decode the video/image by performing a sequence of procedures such as dequantization, deconversion and prediction corresponding to the operation of the encoding equipment.

[47] The rendering module may prepare a decoded video/image by rendering. The video/image prepared by rendering can be displayed through the display.

[48] The present disclosure relates to video/image coding. For example, the method/embodiment disclosed in this document may be applied to the method disclosed in the Universal Video Coding (WC) standard, the Fundamental Video Coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the second generation audio/video coding standard (AVS2) or next generation video/image coding standard (for example, H.267, H.268, etc.).

[49] This document proposes various embodiments of video/image encoding, and the above embodiments can also be performed in combination with each other unless otherwise noted.

[50] In this document, video may refer to a sequence of images over time. A frame generally means a unit representing one image in a particular time frame, and a slice/tile means a unit constituting a part of a frame from an encoding point of view. A slice/tile may include one or more coding tree units (CTUs). One frame can consist of one or more slices/tiles.

[51] A tile is a rectangular CTU area in a particular column of tiles and a particular row of tiles in a frame. A column of tiles is a rectangular area of CTUs having a height equal to the height of the frame and a width specified by syntax elements in the frame parameter set. A row of tiles is a rectangular area of CTUs having a height specified by syntax elements in the frame parameter set and a width equal to the width of the frame. A tile scan is a specific sequential ordering of the CTUs segmenting a frame, in which CTUs are ordered sequentially in a raster scan of the CTUs in a tile, whereas tiles in a frame are ordered sequentially in a raster scan of the frame's tiles. A slice can contain the number of complete tiles or the number of consecutive CTU lines in one tile of a frame that can be contained in one NAL unit. In this document, tile group and slice can be used interchangeably. For example, in this document, a group of tiles/header of a group of tiles may be referred to as a "slice/slice header".

[52] Meanwhile, one frame may be divided into two or more subframes. A subframe may be a rectangular area of one or more slices in a frame.

[53] Pixel or pel can mean the smallest unit that makes up one frame (or image). Additionally, "sample" can be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel/pixel value of a luma component or only a pixel/pixel value of a chroma component.

[54] The unit may represent a basic image processing unit. The unit may include at least one of a specific frame area and information associated with the area. One unit may include one block of luma signals and two blocks of chrominance signals (eg, Cb, Cr). Unit may be used interchangeably with terms such as block or zone in some cases. In general, an M×N block may include samples (or arrays of samples) or a set (or array) of transform coefficients of M columns and N rows. Alternatively, a sample may refer to a pixel value in the spatial domain, and when such a pixel value is converted to a frequency domain, it may refer to a frequency domain conversion factor.

[55] FIG. 2 is a diagram fundamentally illustrating the configuration of video/image encoding equipment to which the disclosure of this document may be applied. Hereinafter in this document, what is referred to as “video encoding equipment” may include image encoding equipment.

[56] Referring to FIG. 2, encoding equipment 200 may include and be configured with an image segmentation module 210, a prediction module 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The prediction module 220 may include a mutual module 221. forecasting and internal forecasting module 222. Residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. Residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a “recovery module” or a “recovery block generator.” The image segmentation module 210, prediction module 220, residual processor 230, entropy encoder 240, adder 250, and filter 260, which are described above, may be configured by one or more hardware components (eg, encoder chipsets or processors) according to an embodiment. In addition, storage device 270 may include a decoded frame buffer (DPB) and may also be configured by a digital storage medium. The hardware component may further include a storage device 270 as an internal/external component.

[57] The image segmentation module 210 may split an input image (or frame) input to the encoding equipment 200 into one or more processing units. As an example, the processing unit may be called a "coding unit (CU)". In this case, the coding unit may be recursively partitioned according to a quadtree, binary tree, and ternary tree (QTBTTT) structure from a coding tree unit (CTU) or largest coding unit (LCU). For example, one encoding unit may be decomposed into multiple encoding units of greater depth based on a quadtree structure, a binary tree structure, and/or a ternary tree structure. In this case, for example, a quadtree structure is first applied, and a binary tree structure and/or a ternary tree structure may subsequently be applied. Alternatively, a binary tree structure can also be used first. The encoding procedure of the present disclosure may be performed based on a final encoding unit that is no longer split. In this case, based on the coding efficiency according to the characteristics of images and the like, the maximum coding unit can be directly used as the final coding unit, or if necessary, the coding unit can be recursively divided into coding units of greater depth, so that a coding unit having optimal size, can be used as the final coding unit. Here, the encoding procedure may include a procedure such as prediction, transformation and reconstruction, which is described below. As another example, the processing unit may further include a predictor (PU) or transformation unit (TU). In this case, each of the predictor and transformation unit may be split or segmented from the above final encoding unit. The predictor may be a sample prediction unit, and the transformation unit may be a unit for deriving a transformation coefficient and/or a unit for deriving a residual signal from the transformation coefficient.

[58] Unit may be used interchangeably with a term such as block or zone in some cases. Typically, an M×N block may represent samples consisting of M columns and N rows or a group of transform coefficients. A sample may generally represent a pixel or a pixel value, and may also represent only a pixel/pixel value of a luma component, and may also represent only a pixel/pixel value of a chrominance signal component. Sample can be used as a term corresponding to a pixel or pel configuring one frame (or image).

[59] The encoding equipment 200 may subtract a prediction signal (predicted block, prediction sample array) output from the inter prediction module 221 or intra prediction module 222 from the input image signal (source block, source sample array) to generate a residual signal (residual block, residual sample array), and the generated residual signal is transmitted to converter 232. In this case, as illustrated, a module for subtracting a prediction signal (prediction block, prediction sample array) from an input image signal (source block, source sample array) in the encoder 200 may be referred to as "subtractor 231". The prediction module may perform prediction for a target processing block (hereinafter referred to as a "current block"), and generate a prediction block including prediction samples for the current block. The prediction module may determine whether intra-prediction or inter-prediction is applied in units of the current block or CU. The prediction module may generate various information regarding prediction, such as prediction mode information, and transmit the generated information to the entropy encoder 240, as described below in the description of each prediction mode. The prediction information may be encoded by entropy encoder 240 and output in the form of a bit stream.

[60] Intra prediction module 222 may predict the current block with reference to samples within the current frame. The referenced samples may be located adjacent to the current block or may also be located at a large distance from the current block according to the prediction mode. Prediction modes in intra prediction may include a plurality of non-directional modes and a plurality of directed modes. The omnidirectional mode may include, for example, DC mode or planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the precise degree of the prediction direction. However, this is exemplary, and directional prediction modes that are greater or less than the above number can be used as configured. The intra prediction module 222 can also determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[61] The inter-prediction unit 221 may logically output a predicted block relative to a current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. At this time, in order to reduce the amount of motion information transmitted in the inter-prediction mode, motion information may be predicted in units of blocks, sub-blocks or samples based on the correlation of motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0-prediction, L1-prediction, bi-prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block existing in the current frame and a temporal neighbor block existing in the reference frame. The reference frame including the reference block and the reference frame including the temporary adjacent block may be identical to each other or different from each other. A temporary neighbor block may be referred to by a name such as a "co-located reference block", a "co-located CU (colCU)" and the like, and a reference frame including a temporary neighboring block may also be called a "co-located frame ( colPic)". For example, the inter-prediction module 221 may configure a list of motion information candidates based on neighboring blocks and generate information indicating which candidate is used to retrieve the motion vector and/or reference frame index of the current block. Inter prediction can be performed based on various prediction modes, and, for example, in the case of the skip mode and the merge mode, the inter prediction unit 221 can use the motion information of a neighboring block as the motion information of the current block. In the case of the skip mode, the residual signal may not be transmitted, unlike the combine mode. A motion vector prediction (MVP) mode may indicate the motion vector of a current block by using the motion vector of an adjacent block as a motion vector predictor and transmitting motion vector difference signals.

[62] The prediction module 220 may generate a prediction signal based on various prediction methods, which are described below. For example, the prediction module may apply intra-prediction or inter-prediction to predict one block, and may simultaneously apply intra-prediction and inter-prediction. This may be referred to as "combined inter-intraprediction (CIIP)". In addition, the prediction module may be based on an intra-block copy (IBC) prediction mode or based on a palette mode for block prediction. The IBC prediction mode or palette mode can be used for encoding image/video content of a game and the like, for example, screen content encoding (SCC). IBC essentially performs prediction in the current frame, but can be performed similarly to inter-prediction in that the reference block is retrieved in the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described in this document. Palette mode can be considered an example of intra-coding or intra-prediction. When palette mode is applied, a sampled value in a frame may be signaled based on information regarding the palette table and palette index.

[63] The prediction signal generated by the prediction module (including inter prediction module 221 and/or intra prediction module 222) may be used to generate a reconstructed signal, or may be used to generate a residual signal. Converter 232 may generate conversion coefficients by applying conversion technology to the residual signal. For example, the transform technology may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally nonlinear transform (CNT). Here, GBT means the transformation obtained from a graph when expressing the relationship information between pixels in the graph. CNT means the transformation obtained from the prediction signal generated using all the previously reconstructed pixels. In addition, the conversion process may be applied to a block of pixels that is identical in size to a square, or may be applied to a variable size block that is not a square.

[64] The quantizer 233 quantizes the transform coefficients and passes them to the entropy encoder 240, and the entropy encoder 240 encodes the quantized signal (information regarding the quantized transform coefficients) and outputs the encoded signal as a bitstream. Information regarding the quantized transform coefficients may be referred to as "residual information". Quantizer 233 may re-arrange the quantized transform coefficients in block form into one-dimensional vector form based on the scan order of the coefficients, and may generate information regarding the transform coefficients based on the quantized transform coefficients in one-dimensional vector form. Entropy encoder 240 can perform various encoding techniques, such as, for example, exponential Golomb coding, context adaptive variable length coding (CAVLC), and context adaptive binary arithmetic coding (CABAC). Entropy encoder 240 may encode information needed to reconstruct video/images (eg, values of syntax elements, etc.) other than the quantized transform coefficients, together or separately. Encoded information (eg, encoded video/image information) may be transmitted or stored in Network Abstraction Layer (NAL) units in the form of a bitstream. The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. In this document, information and/or syntax elements transmitted/signaled from encoding equipment to decoding equipment may be included in video/image information. The video/image information may be encoded through the encoding procedure described above and included in the bit stream. The bit stream may be transmitted over a network or may be stored on a digital storage medium. Here, the network may include a broadcast network and/or a communications network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD and SSD. A transmitting module (not shown) and/or a storage module (not shown) for transmitting or storing the signal output from the entropy encoder 240 may be configured as internal/external elements of the encoding equipment 200, or the transmitting module may be included in the entropy encoder 240.

[65] The quantized transform coefficients output from quantizer 233 can be used to generate a predictive signal. For example, the residual signal (residual block or residual samples) can be recovered by applying dequantization and inverse transform to the quantized transform coefficients through dequantizer 234 and inverse transform module 235. The adder 250 may add the reconstructed residual signal with the prediction signal output from the inter-prediction module 221 or the intra-prediction module 222 to generate a reconstructed signal (reconstructed frame, reconstructed block, reconstructed sample array). When there is no remainder for a target processing block, for example when a skip mode is applied, the predicted block can be used as a reconstructed block. The adder 250 may be referred to as a "recovery module" or a "recovery block generator." The generated reconstructed signal may be used to internally predict the next target block for processing in the current frame, or may be used to inter-predict the next frame after filtering, as described below.

[66] Meanwhile, luma conversion with chrominance scaling (LMCS) can be applied during the encoding and/or frame restoration process.

[67] Filter 260 may enhance the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, filter 260 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and store the modified reconstructed frame in storage device 270, namely, a DPB of storage device 270. Various filtering techniques may include, for example, deblocking filtering. , sampled adaptive bias, adaptive contour filter, bilateral filter, etc. The filter 260 may generate various types of filtering-related information and transmit the generated information to the entropy encoder 240, as described below in the description of each filtering method. Information associated with filtering may be encoded by entropy encoder 240 and output in the form of a bitstream.

[68] The modified reconstructed frame transferred to the storage device 270 can be used as a reference frame in the inter-prediction unit 221. When inter prediction is applied by the encoding equipment, prediction mismatch between the encoding equipment 200 and the decoding equipment can be eliminated, and encoding efficiency can be improved.

[69] The DPB of the storage device 270 may store the modified reconstructed frame for use as a reference frame in the inter-prediction module 221. Memory 270 may store block motion information from which motion information in the current frame is retrieved (or encoded) and/or motion information of already reconstructed blocks in the frame. The stored motion information may be transmitted to the inter-prediction unit 221 to be used as spatial neighbor block motion information or temporal neighbor block motion information. Memory 270 may store reconstructed samples of reconstructed blocks in the current frame and may transmit reconstructed samples to intra prediction module 222.

[70] Meanwhile, in this document, at least one of quantization/dequantization and/or transformation/inverse transformation may be omitted. When quantization/dequantization is omitted, the quantized conversion factor may be referred to as a "transformation factor". When the transform/inverse transform is omitted, the transform coefficients may be called "coefficient" or "residual coefficient" or may still be called "transform coefficient" for consistency of expression.

[71] Additionally, in this document, the quantized transform coefficient and the transform coefficient may be referred to as the “transform coefficient” and the “scaled transform coefficient,” respectively. In this case, the residual information may include information regarding the transform coefficient(s), and the information regarding the transform coefficient(s) may be signaled through a residual encoding syntax. Transform coefficients can be extracted based on residual information (or information regarding the transform coefficient(s)), and scaled transform coefficients can be extracted via inverse transform (scaling) of the transform coefficients. Residual samples can be extracted based on the inverse transform (transform) of the scaled transform coefficients. This may also be applied/expressed elsewhere in this document.

[72] FIG. 3 is a diagram for schematically explaining the configuration of video/image decoding equipment to which the disclosure of this document may be applied.

[73] Referring to FIG. 3, decoding equipment 300 may include and be configured with an entropy decoder 310, a residual processor 320, a prediction module 330, an adder 340, a filter 350, and a memory 360. The prediction module 330 may include an intra prediction module 331 and a mutual prediction module 332. forecasting. Residual processor 320 may include a dequantizer 321 and an inverse converter 322. Entropy decoder 310, residual processor 320, prediction module 330, adder 340, and filter 350, which are described above, may be configured by one or more hardware components (e.g., chipsets or decoder processors) according to an embodiment. Additionally, storage device 360 may include a decoded frame buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include storage device 360 as an internal/external component.

[74] When a bit stream including video/image information is input, the decoding equipment 300 can reconstruct the image in response to the process in which the video/image information is processed in the encoding equipment illustrated in FIG. 2. For example, decoding equipment 300 may extract units/blocks based on blocking-related information obtained from the bit stream. The decoding equipment 300 may perform decoding using a processing unit applied to the encoding equipment. Therefore, a processing unit for decoding, for example, may be an encoding unit, and the encoding unit may be partitioned according to a quadtree structure, a binary tree structure, and/or a ternary tree structure from an encoding tree unit or a maximum encoding unit. One or more transformation units may be derived from a coding unit. In addition, the reconstructed image signal decoded and output by the decoding equipment 300 can be reproduced by the playback equipment.

[75] The decoding equipment 300 may receive a signal output from the encoding equipment of FIG. 2 in the form of a bit stream, and the received signal may be decoded through entropy decoder 310. For example, entropy decoder 310 may parse the bit stream to extract information (eg, video/image information) needed for image reconstruction (or frame reconstruction). The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The decoding equipment may further decode the frame based on information regarding the parameter set and/or general constraint information. The signaling/receiving information and/or syntax elements described later in this document may be decoded, may be decoded by a decoding procedure, and obtained from the bit stream. For example, entropy decoder 310 decodes information in a bit stream based on an encoding method, such as exponential Golomb coding, context adaptive variable length coding (CAVLC), or context adaptive arithmetic coding (CABAC), and the output syntax elements required for image reconstruction and quantized values of the transformation coefficients for the remainder. More specifically, the entropy CABAC decoding method may receive a resolution element corresponding to each syntax element in the bit stream, determine a context model by using information of a target decoding syntax element, decoding information of a target decoding block, or information of a symbol/resolution element decoded in a previous stage, and perform arithmetic decoding for the permission element by predicting the probability of occurrence of the permission element according to the determined context model and generating a symbol corresponding to the meaning of each syntactic element. In this case, the entropy CABAC decoding method can update the context model by using the decoded symbol/grant element information for the context model of the next symbol/grant element after determining the context model. The prediction-related information from the information decoded by the entropy decoder 310 may be provided to the prediction module (inter-prediction module 332 and intra-prediction module 331), and the residual values for which entropy decoding is performed in the entropy decoder 310, i.e. the quantized transform coefficients and associated parameter information may be input to residual processor 320.

[76] Dequantizer 321 may dequantize the quantized transform coefficients to output transform coefficients. Dequantizer 321 may re-arrange the quantized transform coefficients in two-dimensional block form. In this case, re-arrangement can be performed based on the scanning order of the coefficients performed by the encoding equipment. The dequantizer 321 may perform dequantization on the quantized transform coefficients using a quantization parameter (eg, quantization step size information) and obtain the transform coefficients.

[77] An inverse converter 322 inversely converts the transform coefficients to obtain a residual signal (residual block, residual sample array).

[78] Prediction module 330 may perform prediction on the current block and generate a predicted block including prediction samples of the current block. The prediction module may determine whether intra prediction is applied or inter prediction is applied to the current block based on the prediction information output from the entropy decoder 310, and determine a particular intra/inter prediction mode.

[79] The prediction module 330 may generate a prediction signal based on various prediction methods, which are described below. For example, the prediction module may apply intra-prediction or inter-prediction to predict one block, and may simultaneously apply intra-prediction and inter-prediction. This may be referred to as "combined inter-intraprediction (CIIP)". In addition, the prediction module may be based on an intra-block copy (IBC) prediction mode or based on a palette mode for block prediction. The IBC prediction mode or palette mode can be used for image/video encoding for content such as games and the like, such as Screen Content Coding (SCC). IBC may essentially perform prediction in the current frame, but may perform similar to inter-prediction in that the reference block is retrieved in the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described in this document. Palette mode can be considered an example of intra-coding or intra-prediction. When palette mode is applied, information regarding the palette table and palette index may be included in video/image information and signaled.

[80] The intra prediction module 3321 may predict the current block by reference to the samples in the current frame. Samples for reference may be located in the surroundings of the current block or may be located spaced apart from the current block according to a prediction mode. In intraprediction, prediction modes may include a plurality of non-directional modes and a plurality of directed modes. The intra prediction module 331 may determine the prediction mode to be applied to the current block by using the prediction mode applied to the adjacent block.

[81] The inter-prediction unit 332 may extract a predicted block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include information regarding the direction of the inter-prediction (L0-prediction, L1-prediction, bi-prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block existing in the current frame and a temporal neighbor block existing in the reference frame. For example, the inter-prediction module 332 may construct a list of candidate motion information based on neighboring blocks and extract the current block's motion vector and/or reference frame index based on the received candidate selection information. The inter-prediction may be performed based on various prediction modes, and the information regarding the prediction may include information indicating the inter-prediction mode for the current block.

[82] The adder 340 may generate a reconstructed signal (a reconstructed frame, a reconstructed block, or an array of reconstructed samples) by adding the resulting residual signal with a prediction signal (a predicted block or an array of predicted samples) output from a prediction module (including an inter-prediction module 332 and /or internal prediction module 331). If there is no remainder for the target block to process, for example in the case where the skip mode is applied, the predicted block can be used as the reconstructed block.

[83] The adder 340 may be referred to as a “recovery module” or a “recovered block generator.” The generated reconstructed signal may be used to internally predict the next block to be processed in the current frame, and, as described below, may also be output by filtering, or may also be used to inter-predict the next frame.

[84] Meanwhile, luma conversion with chrominance scaling (LMCS) can also be applied in the frame decoding process.

[85] Filter 350 may enhance the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, filter 350 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and store the modified reconstructed frame in storage device 360, namely, a DPB of storage device 360. Various filtering techniques may include, for example, deblocking filtering. , sampled adaptive bias, adaptive contour filter, bilateral filter, etc.

[86] The (modified) reconstructed frame stored in the DPB of the storage device 360 can be used as a reference frame in the inter-prediction unit 332. Storage device 360 may store block motion information from which motion information in the current frame is retrieved (or decoded) and/or motion information of already reconstructed blocks in the frame. The stored motion information may be transmitted to the inter-prediction unit 332 so that it is used as spatial neighbor block motion information or temporal neighbor block motion information. Storage device 360 may store reconstructed samples of reconstructed blocks in the current frame and transmit the reconstructed samples to intra prediction module 331.

[87] In this disclosure, the embodiments described in the filter 260, the inter-prediction module 221, and the intra-prediction module 222 of the encoding equipment 200 may be applied the same or correspond to the filter 350, the inter-prediction module 332, and the intra-prediction module 331.

[88] Meanwhile, as described above, when performing video encoding, prediction is performed in order to improve compression efficiency. Due to this, a predicted block including predictive samples for the current block can be generated as the block to be encoded (ie, the encoding target block). Here, the predicted block includes predictive samples in a spatial domain (or pixel domain). The predicted block is extracted identically in the encoding equipment and the decoding equipment, and the encoding equipment can signal information (residual information) regarding the remainder between the original block and the predicted block, rather than the original sample value of the original block, to the decoding equipment, thereby improving efficiency image coding. The decoding equipment may extract a residual block including residual samples based on the residual information, add the residual block and the predicted block to generate reconstructed blocks including the reconstructed samples, and generate a reconstructed frame including the reconstructed blocks.

[89] Residual information can be generated through a transformation and quantization procedure. For example, the encoding equipment may extract a residual block between an original block and a predicted block, perform a transform procedure on the residual samples (an array of residual samples) included in the residual block to extract transform coefficients, perform a quantization procedure on the transform coefficients to extract quantized transform coefficients, and residual information associated with the signals to the decoding equipment (via the bit stream). Here, the residual information may include quantized transform coefficient value information, location information, transform technology, transform kernel, quantization parameter, and the like. The decoding equipment may perform a dequantization/deconversion procedure based on the residual information and extract the residual samples (or residual blocks). The decoding equipment can generate a reconstructed frame based on the predicted block and the residual block. In addition, for reference to inter-picture prediction later, the encoding equipment may also dequantize/de-convert the quantized transform coefficients to extract a residual block and generate a reconstructed frame based on it.

[90] Meanwhile, various inter-prediction modes can be used to predict the current block in a frame. Various inter-prediction modes are described below with reference to FIG. 5.

[91] Meanwhile, information indicating whether list 0 (L0) prediction, list 1 (L1) prediction or bi-prediction described above is used in the current block (current coding unit) may be transmitted in overhead signals to the current block. This information, for example, may be called "motion prediction direction information", "inter prediction direction information" or "inter prediction indicator information" and may be configured/encoded/signaled in the form of an inter_pred_idc syntax element. Thus, the inter_pred_idc syntax element may indicate whether the above-described List 0 (L0) prediction, List 1 (L1) prediction, or bi-prediction is used for the current block (current encoding unit). In the present disclosure, for convenience of description, the inter-prediction type (L0-prediction, L1-prediction, or bi-prediction) indicated by the inter_pred_idc syntax element may be represented as a motion prediction direction. L0 prediction may be represented by pred_L0; L1 prediction may be represented by pred_L1; and biforecasting may be represented by pred_BI. For example, after prediction, the type may be specified according to the value of the inter_pred_idc syntax element.

[92] As described above, one frame may include one or more slices. A slice may have one of the slice types including an internal (I) slice, a predictive (P) slice, and a bipredictive (B) slice. The slice type may be specified based on the slice type information. For blocks in an I-slice, inter-prediction is not used for prediction, and only intra-prediction can be used. Of course, even in this case, the original sample value may be encoded and signaled without prediction. For blocks in a P-slice, intra-forecasting or inter-forecasting can be used, and when inter-forecasting is used, only uni-forecasting can be used. Meanwhile, intra prediction or inter prediction can be used for blocks in the B-slice, and when inter prediction is used, up to maximum bi prediction can be used.

[93] L0 and L1 may include reference frames encoded/decoded before the current frame. For example, L0 may include reference frames before and/or after the current frame in POC order, and L1 may include reference frames after and/or before the current frame in POC order. In this case, the index of a reference frame lower than the reference frames earlier than the current frame in the POC order may be allocated L0, and the index of the reference frame lower than the reference frames later than the current frame in the POC order may be allocated L1. In the case of a B-slice, bi-prediction can be applied, in which case, unidirectional bi-prediction can be used, or bi-directional bi-prediction can be used. Bidirectional biforecasting may be referred to as "true biforecasting."

[94] For example, information regarding the inter-prediction mode of the current block may be encoded and signaled at the CU (CU-syntax) layer and the like. or may be implicitly specified according to a condition. In this case, some modes may be explicitly signaled, and other modes may be implicitly retrieved.

[95] For example, CU syntax may carry information regarding (mutual) prediction mode, etc. The CU syntax can be as shown in Table 1 below.

[96]

[97]

[98]

[99]

[100]

[101]

[102]

[103]

[104]

[105]

[106] In Table 1, cu_skip_flag may indicate whether or not the skip mode is applied to the current block (CU).

[107] pred_mode_flag equal to 0 may indicate that the current coding unit is encoded in inter-prediction mode. Pred_mode_flag equal to 1 may indicate that the current coding unit is encoded in intra prediction mode.

[108] pred_mode_ibc_flag equal to 1 may indicate that the current coding unit is encoded in IBC prediction mode. Pred_mode_ibc_flag equal to 0 may indicate that the current encoding unit is not encoded in IBC prediction mode.

[109] pcm_flag[x0][y0] equal to 1 may indicate that a pcm_sample() syntax structure is present and a transform_tree() syntax structure is not present in a coding unit including a luma coding block at location (x0, y0). Pcm_flag[x0][y0] equal to 0 may indicate that the pcm_sample() syntax structure is not present. Thus, pcm_flag may represent whether or not a pulse code modulation (PCM) mode is applied to the current block. If PCM mode is applied to the current block, prediction, transformation, quantization, etc. are not applied, and the original sample values in the current block can be encoded and signaled.

[110] intra_mip_flag[x0][y0] equal to 1 may indicate that the intra prediction type for the luminance samples is matrix intra prediction (MIP). Intra_mip_flag[x0][y0] equal to 0 may indicate that the intra prediction type for the luminance samples is not matrix intra prediction. Thus, intra_mip_flag may represent whether or not a mip prediction mode is applied to the luminance sample of the current block.

[111] intra_chroma_pred_mode[x0][y0] may indicate the intra-chroma prediction mode for the chroma samples in the current block.

[112] general_merge_flag[x0][y0] may indicate whether or not inter-prediction parameters for the current coding unit are inferred from an adjacent inter-prediction segment. Thus, general_merge_flag may represent that general merging is available, and when the value of general_merge_flag is 1, regular merging mode, MMVD mode, and subblock merging mode (subblock-based merging mode) may be available. For example, when the value of general_merge_flag is 1, the merging data syntax can be parsed from the encoded video/image information (or bitstream) and the merging data syntax configured/encoded to include the information as shown in Table 2 below.

[113]

[114]

[115] In Table 2, regular_merge_flag[x0][y0] equal to 1 may indicate that the regular merge mode is used to generate the inter-prediction parameters of the current coding unit. Thus, regular_merge_flag can represent whether or not the merge mode (regular merge mode) is applied to the current block.

[116] mmvd_merge_flag[x0][y0] equal to 1 may indicate that the motion vector difference merging mode is used to generate the inter-prediction parameter of the current block. Thus, mmvd_merge_flag represents whether or not MMVD is applied to the current block.

[117] mmvd_cand_flag[x0][y0] may indicate whether the first (0) or second (1) candidate in the list of candidate combinations with the motion vector difference extracted from mmvd_distance_idx[x0][y0] and mm vd_di is used or not recti on_idx[x0][y0].

[118] mmvd_distance_idx[x0][y0] may indicate the index used to retrieve MmvdDistance[x0][y0].

[119] mmvd_direction_idx[x0][y0] may indicate the index used to retrieve MmvdSign[x0][y0].

[120] merge_subblock_flag[x0][y0] may indicate subblock-based inter-prediction parameters for the current block. Thus, merge_subblock_flag can represent whether or not the subblock-based merge mode (or affine merge mode) is applied to the current block.

[121] merge_subblock_idx[x0][y0] may indicate a merge candidate index of a list of subblock-based merge candidates.

[122] ciip_flag[x0][y0] may indicate whether or not combined inter-frame combining and intra-frame prediction (CIIP) is applied for the current coding unit.

[123] merge_triangle_idx0[x0][y0] may indicate that the index of the first possible merge option is a list of possible triangular-shaped motion compensation options.

[124] merge_triangle_idx1[x0][y0] may indicate that the index of the second possible merge option is a list of possible triangular-shaped motion compensation options.

[125] merge_idx[x0][y0] may indicate the index of a possible merge option of a list of possible merge options.

[126] Meanwhile, returning to CU syntax, mvp_10_flag[x0][y0] may indicate the motion vector predictor index of list 0. Thus, when MVP mode is applied, mvp_10_flag may represent the candidate selected for MVP retrieval of the current block from a list of 0 possible MVP options.

[127] ref_idx_11[x0][y0] has identical semantics to ref_idx_10 with 10, and list 0 can be replaced by 11 and list 1, respectively.

[128] inter_pred_idc[x0][y0] may indicate whether or not list0, list1, or biprediction is used for the current encoding unit.

[129] sym_mvd_flag[x0][y0] equal to 1 may indicate that the syntax elements ref_idx_10[x0][y0] and ref_idx_11[x0][y0] and the syntax structure mvd_coding(x0, y0, refList, cpIdx) for refList equal to 1 are not present. Thus, sym_mvd_flag represents whether or not symmetric MVD is used in MVD encoding.

[130] ref_idx_10[x0][y0] may indicate the list 0 reference frame index for the current block.

[131] ref_idx_11[x0][y0] has semantics identical to those of ref_idx_10, with 10, L0 and list 0 replaced by 11, L1 and list 1, respectively.

[132] inter_affine_flag[x0][y0] equal to 1 may indicate that affine model-based motion compensation is used to generate predictive samples of the current block when decoding the P- or B-slice.

[133] cu_affine_type_flag[x0][y0] equal to 1 may indicate that for the current coding unit, when decoding a P- or B-slice, motion compensation based on a 6-parameter affine model is used to generate predictive samples current coding unit. Cu_affine_type_flag[x0][y0] equal to 0 may indicate that motion compensation based on a 4-parameter affine model is used to generate predictive samples of the current block.

[134] amvr_flag[x0][y0] may indicate the motion vector difference resolution. The indices x0, y0 of the arrays indicate the location (x0, y0) of the upper left sample of the luminance signal of the encoding block in question relative to the upper left sample of the luminance signal of the movie frame. Amvr_flag[x0][y0] equal to 0 may indicate that the resolution of the motion vector difference is 1/4 of the luminance signal sample. Amvr_flag[x0][y0] equal to 1 may indicate that the motion vector difference resolution is further specified by amvr_precision_flag[x0][y0].

[135] amvr_precision_flag[x0][y0] equal to 0 may indicate that the motion vector difference resolution is one integer luminance sample if inter_affine_flag[x0][y0] is 0, and 1/16 of the luminance sample in otherwise. Amvr_precision_flag[x0][y0] equal to 1 may indicate that the motion vector difference resolution is four luminance samples if inter_affine_flag[x0][y0] is 0, and one integer luminance sample otherwise.

[136] bcw_idx[x0][y0] may indicate a biprediction weight index with CU weights.

[137] FIG. 4 roughly shows the inter-prediction procedure.

[138] Referring to FIG. 4, the inter-prediction procedure may include a step of determining the inter-prediction mode, a step of extracting motion information according to the determined prediction mode, and a step of performing prediction (forming a prediction sample) based on the extracted motion information. The inter-prediction procedure may be performed by the encoding equipment and decoding equipment as described above. In this document, the encoding equipment may include encoding equipment and/or decoding equipment.

[139] Referring to FIG. 4, the encoding equipment determines the inter-prediction mode for the current block (S400). Meanwhile, various inter-prediction modes can be used to predict the current block in a frame. Various inter-prediction modes can be used to predict the current block in a frame. For example, various modes such as fusion mode, skip mode, motion vector prediction (MVP) mode, affine mode, sub-block fusion mode, MVD fusion mode (MMVD), etc. Decoder Motion Vector Resolution (DMVR), Adaptive Motion Vector Resolution (AMVR), Bi-Prediction with CU Weights (BCW), Bidirectional Optical Flow (BDOF), etc. can be used in addition to this or instead as auxiliary modes. The affine mode may be called "affine motion prediction mode". The MVP mode may be referred to as "Advanced Motion Vector Prediction (AMVP) mode". In the present disclosure, certain modes and/or motion information options extracted by some modes may be included as one of the motion information-related options of other modes. For example, an HMVP candidate may be added as a merge/skip mode fusion candidate, or may be added as an MVP mode candidate. When an HMVP candidate is used as a combination mode or skip mode traffic information candidate, the HMVP candidate may be referred to as a “combined HMVP candidate.”

[140] Prediction mode information indicating the inter-prediction mode of the current block may be signaled from the encoding equipment to the decoding equipment. The prediction mode information may be included in the bitstream and received at the decoding equipment. The prediction mode information may include index information indicating one of several possible mode options. Additionally, the inter-prediction mode may be indicated through hierarchical transmission of flag information signaling. In this case, the prediction mode information may include one or more flags. For example, whether or not a skip mode is applied may be indicated by signaling a skip flag; may indicate whether or not the aggregation mode is applied by signaling an aggregation flag for not applying the skip mode; and may indicate that the MVP mode is applied, or a flag for the additional segment may optionally be signaled when the aggregation mode is not applied. The affine mode may be signaled as an independent mode or may be signaled as a mode dependent on an aggregation mode or an MVP mode and the like. For example, an affine mode may include an affine union mode and an affine MVP mode.

[141] The encoding equipment extracts motion information for the current block (S410). Motion information can be extracted based on a mutual prediction mode.

[142] The encoding equipment may perform inter-prediction using motion information of the current block. The encoding equipment can extract optimal motion information for the current block through a motion estimation procedure. For example, the encoding equipment may search for a similar reference block with high correlation in a predetermined search range in a reference frame by using the source block in the source frame relative to the current block in fractional pixel units, and extract motion information through it. Block similarity can be extracted based on the difference between phase-based sample values. For example, block similarity may be calculated based on the SAD between the current block (or a reference block of the current block) and a reference block (or a reference block reference). In this case, motion information can be extracted based on the reference block having the smallest SAD in the search area. The extracted motion information may be signaled to decoding equipment according to various methods based on an inter-prediction mode.

[143] The encoding equipment performs inter-prediction based on motion information regarding the current block (S420). The encoding equipment may extract predictive sample(s) for the current block based on the motion information. The current block including the predicted samples may be referred to as a "predicted block".

[144] FIG. 5 is a diagram for explaining the merging mode of mutual prediction.

[145] When the combining mode is applied, the motion information of the current prediction block is not directly transmitted, but the motion information of the current prediction block is extracted using the motion information of the adjacent prediction block. Accordingly, motion information of the current prediction block may be indicated by transmitting flag information indicating that the combining mode is used and a combining index indicating which prediction block in the neighborhood is being used. The combining mode may be called "regular combining mode".

[146] To perform the combining mode, the encoding equipment must search for a candidate combining block used to extract motion information regarding the current prediction block. For example, up to five possible combination block options may be used, but the embodiment(s) of the present disclosure are not limited to this. In addition, a maximum number of possible merge block options may be carried in the slice header or tile group header, but the embodiment(s) of the present disclosure is not limited to this. After finding the combiner candidate candidates, the encoding equipment may generate a list of combiner candidate candidates and may select the combiner candidate candidate having the lowest cost from the combiner block candidate candidates as the final combiner block candidate.

[147] The present disclosure may provide various embodiments of possible combination blocks constituting a list of possible combinations.

[148] For example, the list of merge candidates may include five possible merge block candidates. For example, four spatial join candidates and one temporal join candidate may be used. As a specific example, in the case of a spatial combining option, the blocks illustrated in FIG. 5 can be used as a spatial merging option. Hereinafter in this document, a spatial merging opportunity or a spatial merging opportunity, which is described below, may be referred to as "SMVP", and a temporal merging opportunity or a temporal merging opportunity, which are described below, may be referred to as "TMVP".

[149] The list of possible combinations for the current block can be configured, for example, based on the following procedure.

[150] The encoding equipment (encoding equipment/decoding equipment) may insert spatial join candidates retrieved by searching for spatial neighbors of the current block into the list of join candidates. For example, spatial neighbor blocks may include a lower left corner neighbor block, a left neighbor block, a right upper corner neighbor block, a top neighbor block, and a left upper corner neighbor block relative to the current block. However, this is an example, and in addition to the above-mentioned spatial neighbor blocks, additional neighboring blocks such as a right neighbor block, a lower neighbor block, and a right lower neighbor block may further be used as spatial neighbor blocks. The encoding equipment can detect available blocks by performing a priority-based spatial neighbor search, and can extract motion information of the detected blocks as spatial combining candidates. For example, the encoding equipment and/or decoding equipment may search the five blocks illustrated in FIG. 5, in the order A1->B1->B0->A0->B2, and can configure a list of combination candidates by sequentially indexing the available candidates.

[151] The encoding equipment may search for a temporary neighbor block relative to the current block and insert the retrieved temporary join candidate into the list of join candidates. The temporary neighbor block may be located in the reference frame, i.e. in a frame different from the current frame in which the current block is located. The reference frame in which temporary adjacent blocks are positioned may be called a "co-located frame" or "co-located frame". You can search for temporary neighboring blocks in order from the lower right corner neighboring block and the lower right center block relative to the co-located block relative to the current block in the co-locating block. frame. Meanwhile, when motion data compression is applied, specific motion information can be stored as characteristic motion information regarding each predetermined storage unit in the co-location. frame. In this case, it is not necessary to store motion information regarding all blocks in a predetermined storage unit, and thereby the advantage of compressing motion data can be obtained. In this case, the predetermined storage unit may be predefined as, for example, 16×16 sample units or 8×8 sample units, or size information regarding the predetermined storage unit may be signaled from the encoding equipment to the decoding equipment. When motion data compression is applied, motion information relative to temporarily adjacent blocks may be replaced with characteristic motion information relative to a predetermined storage unit in which temporarily adjacent blocks are positioned. Thus, in this case, from an implementation point of view, instead of a predicted block positioned at the coordinates of temporarily adjacent blocks, a temporary merging candidate can be retrieved based on motion information relative to the prediction block covering an arithmetic left-shifted position after an arithmetic right-shifted position by a certain value by based on the coordinates (top left sample position) of the temporary adjacent block. For example, when the predetermined storage unit is 2n×2n sample units, if the coordinates of temporarily adjacent blocks are (xTnb, yTnb), the motion information relative to the prediction block positioned at the adjusted position ((xTnb>>n)<<n), ( yTnb>>n)<<n)), can be used for a temporary possible merging option. Specifically, when the predetermined storage unit is 16×16 sample units, if the coordinates of temporarily adjacent blocks are (xTnb, yTnb), the motion information relative to the prediction block positioned at the adjusted position ((xTnb>>4)<<4), (yTnb>>4)<<4)), can be used for a temporary possible merging option. Alternatively, when the predetermined storage unit is 8×8 sample units, if the coordinates of temporarily adjacent blocks are (xTnb, yTnb), motion information relative to the prediction block positioned at the adjusted position ((xTnb>>3)<<3), ( yTnb>>3)<<3)), can be used for a temporary possible merging option.

[152] The encoding equipment may check whether the number of current possible combinations is less than or equal to the number of maximum possible combinations. The maximum number of possible combinations may be preset or signaled from the encoding equipment to the decoding equipment. For example, encoding equipment may generate and encode information regarding the maximum number of possible combinations and transmit the information to a decoder in the form of a bitstream. When the maximum number of candidate joins is filled, the subsequent process of adding candidates may not continue.

[153] As a result of the check, when the number of current combining candidates is less than the maximum number of combining candidates, the encoding equipment may insert an additional combining candidate into the list of combining candidates. For example, the additional pool candidate may include at least one of history-based pool candidate(s), pairwise average pool candidate(s), ATMVP, combined bi-predictive pool candidate (when the current group's slice/tile group type is slices/tiles is type B) and/or a possible variant of combining with a zero vector.

[154] As a result of the check, when the number of current possible combinations is not less than the maximum number of possible combinations, the encoding equipment may complete constructing the list of possible combinations. In this case, the encoding equipment may select an optimal combining candidate from among the combining candidate candidates constituting a list of combining candidate candidates based on a rate-based distortion cost function (RD), and signaling selection information indicating the selected candidate merging option (eg, merging index) into the decoding hardware. The decoder may select the optimal combining candidate based on the list of combining candidate candidates and the selection information.

[155] As described above, the motion information of the selected combining candidate can be used as the motion information of the current block, and the prediction samples of the current block can be extracted based on the motion information of the current block. The encoding equipment may extract residual samples of the current block based on the predicted samples, and may signal residual information regarding the residual samples to the decoding equipment. As described above, the decoding equipment can generate reconstructed samples based on residual samples extracted based on residual information and prediction samples, and can generate a reconstructed frame based on them.

[156] When the skip mode is applied, motion information regarding the current block can be retrieved identically to when the combining mode is applied. However, when the skip mode is applied, the residual signal for the corresponding block is omitted, and therefore the predicted samples can be directly used as reconstructed samples. Skip mode can be used, for example, when the value of the cu_skip_flag syntax element is 1.

[157] FIG. 6 is a view illustrating a motion vector difference (MMVD) combining mode in mutual prediction.

[158] The MMVD mode is a method of applying motion vector difference (MVD) to a combining mode that directly uses motion information extracted to generate predictive samples of the current block.

[159] For example, an MMVD flag (eg, mmvd_flag) indicating whether or not MMVD should be used for the current block (ie, the current CU) may be signaled, and MMVD may be executed based on this MMVD. flag. When MMVD is applied to the current block (eg, when mmvd_flag is 1), additional information regarding the MMVD may be signaled.

[160] Here, the additional information regarding the MMVD may include a combination candidate flag (eg, mmvd_cand_flag) indicating whether the first candidate or the second candidate in the list of combination candidates is used or not, together with an MVD, a distance index to indicate the absolute magnitude of the movement (for example, mmvd_distance_idx) and a direction index (for example, mmvd_direction_idx) to indicate the direction of movement.

[161] In MMVD mode, two possibilities (i.e., the first candidate or the second candidate) located in the first and second entries of the possible options in the list of possible merge options may be used, and one of the two possible options (i.e. .e. first possible option or second possible option) can be used as the base MV. For example, a combination candidate flag (eg, mmvd_cand_flag) may be signaled to indicate either one of two candidates (ie, the first candidate or the second candidate) in the list of candidate combinations.

[162] In addition, the distance index (eg, mmvd_distance_idx) indicates motion size information and may indicate a predetermined offset from the starting point. Referring to FIG. 5, the displacement may be added to the horizontal component or the vertical component of the origin motion vector. The relationship between the distance index and the predetermined offset can be shown in Table 3 below.

[163]

[164] Referring to Table 3, the MVD distance (eg, MmvdDistance) can be determined according to the distance index value (eg, mmvd_distance_idx), and the MVD distance (eg, MmvdDistance) can be extracted using integer sampling precision or fractional sampling precision based on the value of slice_fpel_mmvd_enabled_flag. For example, slice_fpel_mmvd_enabled_flag equal to 1 may indicate that the MVD distance is extracted using integer sampling units in the current slice, and slice_fpel_mmvd_enabled_flag equal to 0 may indicate that the MVD distance is extracted using fractional sampling units in the current slice.

[165] In addition, the direction index (eg, mmvd_direction_idx) indicates the direction of the MVD relative to the starting point and may indicate four directions, as shown in Table 4 below. In this case, the MVD direction may be indicated by the MVD sign. The relationship between the direction index and the MVD code can be expressed as shown in Table 4 below.

[166]

[167] Referring to Table 4, the MVD sign (eg, MmvdSign) may be determined according to the direction index value (eg, mmvd_direction_idx), and the MVD sign (eg, MmvdSign) may be extracted for the L0 reference frame and the L1 reference frame.

[168] Based on the distance index (eg, mmvd_distance_idx) and direction index (eg, mmvd_direction_idx) described above, the MVD offset can be calculated as shown in Equation 1 below.

[169] equation 1

[170] Thus, in MMVD mode, a combine candidate indicated by a combine candidate flag (eg, mmvd_cand_flag) is selected from among the combine candidates of a list of combine candidates retrieved based on the adjacent block, and the selected combine candidate may be used as a base case (e.g. MVP). In addition, motion information (ie, motion vector) of the current block can be extracted by summing the extracted MVD using a distance index (eg, mmvd_distance_idx) and a direction index (eg, mmvd_direction_idx) based on the base candidate.

[171] Meanwhile, traditionally, only one motion vector can be used to express the motion of a coding block. Thus, the model of translational motion in space is used. However, although this method can express the optimal motion in units of blocks, it does not actually represent the optimal motion of each sample, and coding efficiency can be improved if the optimal motion vector can be determined in a unit of samples. For this purpose, an affine motion model can be used. An affine motion prediction method for encoding using an affine motion model can effectively express four motions as follows.

[172] FIG. 7 roughly shows the motion expressed through the affine motion model.

[173] Referring to FIG. 7, the motion that can be expressed through the affine motion model may include translational motion in space, scaling motion, rotational motion, and shear motion. Thus, not only the translational movement in space in which (part of) the image moves in a plane with the passage of time shown in Fig. 7, but also scaling motion in which (part of) the image is scaled as time passes, rotational motion in which (part of) the image rotates as time passes, and shear motion in which (part of) the image is transformed into a parallelogram as time passes , can be effectively expressed through affine motion prediction.

[174] The encoding equipment/decoding equipment can predict an image distortion shape based on motion vectors at control points (CPs) of the current block through affine motion prediction, and thereby the prediction accuracy can be improved so as to improve image compression performance. In addition, the motion vector for at least one control point of the current block can be extracted using the motion vector of an adjacent block relative to the current block, thereby reducing the data load of added side information and greatly improving the efficiency of inter-prediction.

[175] Of the motions that an affine motion model can express, an affine motion model that expresses three motions (movement in space, scaling, rotation) can be called a “similarity-based (simplified) affine motion model.” However, the affine motion model is not limited to the motion model described above.

[176] An affine motion prediction method may express a motion vector in each block sample unit using two, three, or four motion vectors.

[177] FIG. 8A and 8B approximately show CPMV for affine motion prediction.

[178] Traditionally, only one motion vector can be used to express the motion of a coding block. Thus, the model of translational motion in space is used. However, although this method can express the optimal motion in units of blocks, it does not actually represent the optimal motion of each sample, and coding efficiency can be improved if the optimal motion vector can be determined in a unit of samples. For this purpose, an affine motion model can be used. A method for affine motion prediction for encoding using an affine motion model may be as follows.

[179] An affine motion prediction method may express a motion vector in each block sample unit using two, three, or four motion vectors. For example, an affine motion model can represent four types of motion. An affine motion model that expresses three motions (spatial translation, scaling, and rotation) of the motions that an affine motion model can express can be called a “similarity-based (simplified) affine motion model.” However, the affine motion model is not limited to the motion model described above.

[180] Affine motion prediction can determine the motion vector of a sample position included in a block using two or more control point motion vectors (CPMVs). In this case, the set of motion vectors may be called an "affine motion vector field (MVF)".

[181] For example, FIG. 8A may show a case where two CPMVs are used, which may be called a "4-parameter affine model". In this case, the motion vector at the sample position (x, y) can be defined, for example, as an equation.

[182] equation 2

[183] For example, FIG. 8B may show a case where three CPMVs are used, which may be called a "6-parameter affine model". In this case, the motion vector at the sample position (x, y) can be determined, for example, by Equation 3.

[184] equation 3

[185] In Equations 2 and 3, {v _x , v _y } may represent the motion vector at position (x, y). In addition, {v _0x , v _0y } may indicate a control point (CP) CPMV at the upper left corner position of the encoding block, {v _1x , v _1y } may indicate the CPMV CP at the upper right corner position, {v _2x , v _2y } may indicate CPMV CP in the lower left corner position. In addition, W may indicate the width of the current block, and H may indicate the height of the current block.

[186] FIG. 9 roughly illustrates the case in which the affine MVF is defined in terms of subblocks.

[187] During the encoding/decoding process, the affine MVF may be defined in units of samples or predefined subblocks. For example, when the affine MVP is defined in units of samples, a motion vector can be obtained based on each sample value. Alternatively, for example, when an affine MVP is defined in terms of subblocks, the motion vector of the corresponding block can be obtained based on the subblock center sample value (the lower right corner of the center, i.e., the bottom right sample of the four center samples). Thus, in affine motion prediction, the motion vector of the current block can be extracted in units of samples or sub-blocks.

[188] In the case of FIG. 9, the affine MVF is defined in terms of 4x4 subblocks, but the size of the subblocks can be modified in various ways.

[189] Thus, when affine prediction is available, the three motion models applicable to the current block may include a translational motion model in space, a 4-parameter affine motion model, and a 6-parameter affine motion model. A translational motion model in space can represent a model using an existing motion vector in units of blocks, a 4-parameter affine motion model can represent a model using two CPMVs, and a 6-parameter affine motion model can represent a model using three CPMVs.

[190] Meanwhile, affine motion prediction may include an affine MVP (or affine mutual) mode or an affine pooling mode.

[191] FIG. 10 is a view illustrating an affine combining mode or a subblock-based combining mode in inter-prediction.

[192] For example, in the affine combining mode, the CPMV may be determined according to the affine motion model of a neighboring block encoded by affine motion prediction. For example, neighboring blocks encoded as affine motion prediction in the search order can be used for the affine fusion mode. Thus, when at least one of the neighboring blocks is encoded in affine motion prediction, the current block may be encoded in the affine pooling mode. Here, the exact merging mode may be called "AF_MERGE".

[193] When the affine merging mode is applied, the CPMVs of the current block can be extracted using the CPMVs of neighboring blocks. In this case, the CPMVs of the neighboring block can be used as the CPMV of the current block as is, and the CPMVs of the neighboring block can be modified based on the size of the neighboring block and the size of the current block and used as the CPMV of the current block.

[194] On the other hand, in the case of an affine combining mode in which the motion vector (MV) is extracted in units of sub-blocks, it may be referred to as a “sub-block-based combining mode”, which may be indicated based on a sub-block-based combining flag (or a syntax element merge_subblock_flag). Alternatively, when the value of the merge_subblock_flag syntax element is 1, it may indicate that the subblock-based merging mode is being applied. In this case, the list of affine join candidates that is described below may be referred to as a “subblock-based join candidate list.” In this case, the list of subblock-based aggregation candidates may further include a candidate retrieved by SbTMVP, which is described below. In this case, the candidate retrieved by SbTMVP can be used as candidate index 0 of the subblock-based pool candidate list. In other words, a candidate retrieved from the SbTMVP may be positioned before an inherited affine candidate or a constructed affine candidate, which is described below, in the subblock-based pool candidate list.

[195] When the affine join mode is applied, a list of affine join candidates can be constructed to retrieve the CPMV for the current block. For example, the list of affine join possibilities may include at least one of the following possibilities. 1) Inherited affine possible join option. 2) Constructed affine possible union option. 3) Possible version of the zero motion vector (or zero vector). Here, the inherited affine merging candidate is a candidate extracted based on the CPMV of a neighboring block, when the neighboring block is encoded in affine mode, the constructed affine merging candidate is a candidate extracted by constructing the CPMV based on the MVs of neighboring blocks of the corresponding CP in units each CPMV, and a null motion vector candidate may indicate a candidate consisting of CPMVs whose value is 0.

[196] The list of affine possible combinations can be constructed, for example, as follows.

[197] Up to two legacy affine candidates can be provided, and the legacy affine candidates can be extracted from the affine motion patterns of neighboring blocks. Neighbor blocks can contain one left neighbor block and a top neighbor block. Possible block options can be positioned as illustrated in FIG. 4. The scan order for the left predictor may be A ₁ -->A ₀ , and the scan order for the top predictor may be B ₁ -->B ₀ -->B ₂ . Only one inherited candidate from each of the left and top corners can be selected. The cutoff check may not be performed between two inherited candidates.

[198] When a neighboring affine block is identified, motion vectors based on the control points of the checked block can be used to retrieve a candidate CPMVP from the list for the affine union of the current block. Here, a neighboring affine block may indicate a block encoded in the affine prediction mode from neighboring blocks of the current block. For example, referring to FIG. 10, when the lower left adjacent block A is encoded in the affine prediction mode, motion vector v2, v3 and v4 in the upper left corner, upper right corner and lower left corner of the adjacent block A can be obtained. When a neighboring block A is encoded using a 4-parameter affine motion model, the two CPMVs of the current block can be calculated according to v2 and v4. When the neighboring block A can be calculated according to v2 and v3. When a neighboring block A is encoded using a 6-parameter affine motion model, the three CPMVs of the current block can be calculated according to the three CPMVs v2, v3 and v4 of the current block.

[199] FIG. 11 is a diagram illustrating candidate positions in an affine combining mode or a subblock-based combining mode.

[200] An affine candidate constructed in an affine fusion mode or a subblock-based fusion mode may mean a candidate constructed by combining translational motion information in the space around each control point. Control point motion information can be extracted from specified spatial and temporal perimeters. CPMVk (k=0, 1, 2, 3) can represent the kth control point.

[201] Referring to FIG. 11, blocks can be checked in the order B2->B3->A2 for CPMV0, and the motion vector of the first available block can be used. For CPMV1, blocks can be checked according to the order B1->B0, and for CPMV2, blocks can be checked according to the order A1->A0. TMVP (Temporal Motion Vector Predictor) can be used with CPMV3 if available.

[202] Once the motion vectors of the four control points are obtained, affine fusion candidates can be generated based on the obtained motion information. The control point based motion vector combination may be any of {CPMV0, CPMV1, CPMV2}, {CPMV0, CPMV1, CPMV3}, {CPMV0, CPMV2, CPMV3}, {CPMV1, CPMV2, CPMV3}, {CPMV0, CPMV1} and {CPMV0, CPMV2}.

[203] A combination of three CPMVs can constitute a 6-parameter affine fusion candidate, and a combination of two CPMVs can constitute a 4-parameter affine fusion candidate. To eliminate the motion scaling process, if the reference indices of the control points are different, associated combinations of motion vectors based on the control points may be discarded.

[204] FIG. 12 is a view illustrating SbTMVP in inter-prediction.

[205] Subblock-based temporal motion vector prediction (SbTMVP) may also be referred to as “advanced temporal motion vector prediction (ATMVP)”. SbTMVP can use the motion field in a co-located frame to improve motion vector prediction and fusion mode for the CU in the current frame. Here, the co-located frame may be referred to as a “co-located frame”.

[206] For example, SbTMVP can predict motion at the sub-block (or sub-CU) level. In addition, SbTMVP may apply motion shifting before sampling temporal motion information from the co-location. frame. Here, the motion offset may be obtained from the motion vector of one of the spatially adjacent blocks of the current block.

[207] SbTMVP can predict the motion vector of a sub-unit (or sub-CU) in the current block (or CU) according to two steps.

[208] In the first step, spatial neighboring blocks may be tested according to the order of A1, B1, B0 and A0 in FIG. 5. The first spatial neighboring block having a motion vector using co-location. frame as its reference frame can be identified, and a motion vector can be selected as the motion offset to be applied. When such motion is not identified from a spatial neighbor block, the motion offset may be set to (0, 0).

[209] In the second stage, the motion offset identified in the first stage can be applied to obtain sub-block level motion information (motion vector and reference indices) from the co-located ones. frame. For example, the motion shift can be added to the coordinates of the current block. For example, the motion offset may be set equal to the motion A1 in FIG. 8. In this case, for each sub-block, the movement information of the corresponding block is co-located. frame can be used to extract sub-block motion information. Temporal motion scaling can be used to align reference frames of temporal motion vectors with reference frames of the current block.

[210] A combined subblock-based aggregation list including both the SbTVMP candidate and the exact aggregation candidate may be used to carry affine aggregation mode signaling. Here, the affine merging mode may be called “sub-block-based merging mode.” The SbTVMP mode may or may not be available according to a flag included in the Sequence Parameter Set (SPS). If the SbTMVP mode is available, the SbTMVP predictor may be added as the first entry in the subblock-based join candidate list, and the affine join candidates may be listed next. The maximum allowed size of the list of affine possible join options can be 5.

[211] The size of the sub-CU (or sub-block) used in SbTMVP can be set fixed to 8x8, and similar to the affine merge mode, the SbTMVP mode can only be applied to blocks having a width and height of 8 or greater. The logic for encoding the additional width and height may be identical to the logic for encoding other possible join options. Thus, for each CU in a P- or B-slice, an RD test using an additional rate-dependent distortion (RD) cost function can be performed to determine whether or not the SbTMVP candidate should be used.

[212] FIG. 13 is a view illustrating a combined inter-frame combining and intra-frame prediction (CIIP) mode in inter-prediction.

[213] CIIP can be applied to the current CU. For example, in the case in which the CU is encoded in the concatenation mode, the CU includes at least 64 luminance samples (i.e., when the product of CU-width and CU-height is 64 or greater), and as CU-width, Since the CU height is less than 128 luminance samples, an additional flag (eg, ciip_flag) may then be signaled to indicate whether or not the CIIP mode is applied to the current CU.

[214] In CIIP prediction, the mutual prediction signal and the internal prediction signal may be combined. In the CIIP mode, the mutual prediction signal P inter can be extracted using the same inter-prediction process applied to the regular combining mode. The internal prediction signal P_intra may be extracted according to an internal prediction process having a planar mode.

[215] The internal prediction signal and the mutual prediction signal can be combined using a weighted average and can be expressed in Equation 4 below. The weight coefficient can be calculated according to the coding mode of the top and left adjacent blocks shown in FIG. 13.

[216] equation 4

[217] In Equation 4, when the top neighbor block is accessible and internally encoded, isIntraTop may be set to 1, otherwise isIntraTop may be set to 0. If the left adjacent block is accessible and internally encoded, isIntraLeft may be set to 1, otherwise isIntraLeft can be set to 0. When (isIntraLeft+isIntraLeft) is 2, wt can be set to 3, and when (islntraLeft+isIntraLeft) is 1, wt can be set to 2. Otherwise, wt can be set to 1.

[218] FIG. 14 is a view illustrating a segmentation mode in inter-prediction.

[219] Referring to FIG. 14, when the segmentation mode is applied, the CU can be equally divided into two triangular-shaped segments using diagonal partitioning or anti-diagonal partitioning in the opposite direction. However, this represents only an example of the segmentation mode, and the CU may be equally or unevenly divided into segments having different shapes

[220] For each CU segment, only unidirectional prediction may be allowed. Thus, each segment can have one motion vector and one reference index. The unidirectional prediction constraint should ensure that only two motion-compensated predictions are required per CU, similar to bi-prediction.

[221] When the segmentation mode is applied, a flag indicating the partitioning direction (diagonal direction or opposite diagonal direction) and two merge indices (for each segment) may additionally be signaled.

[222] After predicting each segment, sample values based on the boundary line in the diagonal or the opposite diagonal can be adjusted using blending processing with adaptive weights based on the adaptive weights.

[223] Meanwhile, when the combining mode or the skip mode is applied, motion information can be extracted based on the regular combining mode, MMVD (motion vector difference combining mode), sub-block combining mode, CIIP mode (combined inter-frame and intra-frame combining mode). prediction) or segmentation mode can be used to extract motion information to generate predictive samples, as described above. Each mode can be activated or deactivated via an on/off flag in the Sequence Parameter Set (SPS). If the on-off flag for a particular mode is deactivated in the SPS, the syntax clearly conveyed for the prediction mode in CU or PU units may not be signaled.

[224] Table 5 below relates to the process of extracting the merge mode or skip mode from the traditional merge_data syntax. In Table 5 below, CUMergeTriangleFlag[x0][y0] may correspond to the on-off flag for the segmentation mode described above in FIG. 14, and merge_triangle_split_dir[x0][y0] may indicate the splitting direction (diagonal direction or opposite diagonal direction) when the segmentation mode is applied. In addition, merge_triangle_idx0[x0][y0] and merge_triangle_idx1[x0][y0] can specify two merge indexes for each segment when segmentation mode is applied.

[225]

[226]

[227] Meanwhile, each prediction mode including regular combining mode, MMVD mode, sub-block combining mode, CIIP mode and segmentation mode can be activated or deactivated from the Sequence Parameter Set (SPS), as shown in Table 6 below In Table 6 below, sps_triangle_enabled_flag may correspond to a flag that enables or disables the segmentation mode described above in FIG. 14, from SPS.

[228]

[229]

[230]

[231]

[232]

[233]

[234] The merge_data syntax of table 5 may be parsed or retrieved according to the SPS flag of table 6 and the condition under which each prediction mode can be used. A summary of all cases according to the conditions under which the SPS flag and each prediction mode can be used is shown in Tables 7 and B. Table 7 shows the number of cases in which the current block is in merge mode, and Table 8 shows the number of cases in which, the current the block is in skip mode. In Tables 7 and 8 below, regular may correspond to the regular join mode, mmvd may correspond to triangular, or TRI may correspond to the segmentation mode described above with reference to FIG. 12.

[235]

[236]

[237] As one example of the cases mentioned in Tables 7 and B, a case in which the current block is 4×16 and a skip mode are described. When subblock merge mode, MMVD mode, CIIP mode and segmentation mode are activated in SPS, if regular_merge_flag[x0][y0], mmvd_flag[x0][y0] and merge_subblock_flag[x0][y0] in merge_data syntax are 0, information motion for the current block must be extracted in segmentation mode. However, even if the segmentation mode is activated from the on-off flag in SPS, it can be used as a prediction mode only when the conditions in Table 9 below are additionally satisfied. In Table 9 below, MergeTriangleFlag[x0][y0] can correspond to the flag enable/disable for segmentation mode, and sps_triangle_enabled_flag may correspond to a flag that enables or disables segmentation mode from SPS.

[238]

[239] Referring to the above Table 9, if the current slice is a P-slice, since prediction samples cannot be generated through the segmentation mode, the decoder can no longer decode the bitstream. In order to resolve the problem that occurs in the exceptional case in which decoding is not performed because the final prediction mode cannot be selected according to each SPS on/off flag and the combining data syntax, in the present disclosure, a default combining mode is proposed. The default aggregation mode may be preset in various ways or may be retrieved through additional transmission in the syntax overhead.

[240] In an embodiment, the regular merging mode may be applied to the current block based on a case in which the MMVD mode, sub-block merging mode, CIIP mode, and segmentation mode for performing prediction by dividing the current block into two segments are not available. Thus, when the merge mode cannot ultimately be selected for the current block, the regular merge mode can be used as the default merge mode.

[241] For example, if the value of the general merge flag indicating whether or not a merge mode is available for the current block is 1, but the merge mode cannot ultimately be selected for the current block, the regular merge mode may be used as the default merge mode .

[242] In this case, the motion information of the current block can be extracted based on the merging index information indicating one of the merging candidates included in the merging candidate list of the current block, and prediction samples can be generated based on the extracted motion information.

[243] Accordingly, the syntax of the union data may be as shown in Table 10 below.

[244]

[245]

[246] Referring to Table 10 and Table 6, based on the case in which the MMVD mode is not available, the sps_mmvd_enabled_flag for activating or deactivating the MMVD mode from the SPS may be 0, or the first flag (mmvd_merge_flag[x0][y0] ), indicating whether the MMVD mode is applied or not, can be equal to 0.

[247] In addition, based on the case in which the subblock merging mode is not available, the sps_affine_enabled_flag for activating or deactivating the subblock merging mode from the SPS may be 0, or a second flag (merge_subblock_flag[x0][y0]) indicating whether Whether or not subblock merging mode is applied can be equal to 0.

[248] In addition, based on the case in which the CIIP mode is not available, the sps_ciip_enabled_flag for enabling or disabling the CIIP mode from the SPS may be 0, or a third flag (ciip_flag[x0][y0]) indicating whether CIIP mode is applied or not, can be equal to 0.

[249] In addition, based on the case in which the segmentation mode is not available, the sps_triangle_enabled_flag for activating or deactivating the segmentation mode from SPS may be 0, or a fourth flag (MergeTriangleFlag[x0][y0]) indicating whether or not no segmentation mode, can be equal to 0.

[250] In addition, for example, based on the case in which the segmentation mode is deactivated based on the sps_triangle_enabled_flag, a fourth flag (MergeTriangleFlag[x0][y0]) indicating whether or not the segmentation mode is applied may be set to 0.

[251] In another embodiment, based on the fact that the regular combining mode, MMVD mode, sub-block combining mode, CIIP mode and segmentation mode for performing prediction by dividing the current block into two segments, the regular combining mode can be applied to the current block . Thus, when the merge mode cannot ultimately be selected for the current block, the regular merge mode can be used as the default merge mode.

[252] For example, in a case in which the value of the general merge flag indicating whether or not the merge mode is available for the current block is 1, but the merge mode cannot ultimately be selected for the current block, the regular merge mode may be used as default merge mode.

[253] For example, based on the case in which the MMVD mode is not available, the sps_mmvd_enabled_flag for activating or deactivating the MMVD mode from the SPS may be 0, or the first flag (mmvd_merge_flag[x0][y0]) indicating whether or not MMVD mode, can be equal to 0.

[254] In addition, based on the case in which the subblock merging mode is not available, the sps_affine_enabled_flag for activating or deactivating the subblock merging mode from the SPS may be 0, or a second flag (merge_subblock_flag[x0][y0]) indicating whether Whether or not subblock merging mode is applied can be equal to 0.

[255] In addition, based on the case in which the CIIP mode is not available, the sps_ciip_enabled_flag for enabling or disabling the CIIP mode from the SPS may be 0, or a third flag (ciip_flag[x0][y0]) indicating whether CIIP mode is applied or not, can be equal to 0.

[256] In addition, based on the case in which the segmentation mode is not available, the sps_triangle_enabled_flag for activating or deactivating the segmentation mode from SPS may be 0, or a fourth flag (MergeTriangleFlag[x0][y0]) indicating whether or not no segmentation mode, can be equal to 0.

[257] In addition, based on the case in which the regular merge mode is not available, the fifth flag (regular_merge_flag[x0][y0]) indicating whether or not the regular merge mode is applied may be set to 0. Thus, even when the value of the fifth flag is 0, the regular aggregation mode can be applied to the current block based on the case in which MMVD mode, sub-block aggregation mode, CIIP mode and segmentation mode are not available.

[258] In this case, motion information of the current block may be extracted based on the first candidate of the merge candidates included in the list of merge candidates of the current block, and predictive samples may be generated based on the extracted motion information.

[259] In another embodiment, the regular combining mode may be applied to the current block based on the fact that the regular combining mode, MMVD mode, sub-block combining mode, CIIP mode and segmentation mode for performing prediction by dividing the current block into two segments are not available. Thus, when a merge mode is ultimately selected for the current block, the regular merge mode can be applied as the default merge mode.

[260] For example, in a case in which the value of the general merge flag indicating whether or not the merge mode is available for the current block is 1, but the merge mode is ultimately not selected for the current block, the regular merge mode may be used as the mode joins by default.

[261] For example, based on the case in which the MMVD mode is not available, the sps_mmvd_enabled_flag for activating or deactivating the MMVD mode from the SPS may be 0, or the first flag (mmvd_merge_flag[x0][y0]) indicating whether or not MMVD mode, can be equal to 0.

[262] In addition, based on the case in which the subblock merging mode is not available, the sps_affine_enabled_flag for activating or deactivating the subblock merging mode from the SPS may be 0, or a second flag (merge_subblock_flag[x0][y0]) indicating whether Whether or not subblock merging mode is applied can be equal to 0.

[263] In addition, based on the case in which the CIIP mode is not available, the sps_ciip_enabled_flag for enabling or disabling the CIIP mode from the SPS may be 0, or a third flag (ciip_flag[x0][y0]) indicating whether CIIP mode is applied or not, can be equal to 0.

[264] In addition, based on the case in which the segmentation mode is not available, the sps_triangle_enabled_flag for activating or deactivating the segmentation mode from SPS may be 0, or a fourth flag (MergeTriangleFlag[x0][y0]) indicating whether or not no segmentation mode, can be equal to 0.

[265] In addition, based on the case in which the regular merge mode is not available, the fifth flag (regular_merge_flag[x0][y0]) indicating whether or not the regular merge mode is applied may be set to 0. Thus, even when the value of the fifth flag is 0, the regular aggregation mode can be applied to the current block based on the case in which MMVD mode, sub-block aggregation mode, CIIP mode and segmentation mode are not available.

[266] In this case, the motion vector (0, 0) can be extracted as the motion information of the current block, and prediction samples of the current block can be generated based on the motion information (0, 0). For a motion vector (0, 0), prediction may be performed with reference to the zero reference frame of the L0 reference list. However, when the zero reference frame (RefPicList[0][0]) of the L0 reference list does not exist, prediction can be performed by referring to the zero reference frame (RefPicList[1][0]) of the L1 reference list.

[267] FIG. 15 and 16 schematically show an example of a method for encoding video/images and related components according to embodiment(s) of this document.

[268] The method disclosed in FIG. 15 may be performed by the encoding equipment disclosed in FIG. 2 or fig. 16. In particular, for example, steps S1500-S1520 in FIG. 15 may be performed by the prediction module 220 of the encoding equipment 200 of FIG. 16, and S1530 of FIG. 15 may be performed by the entropy encoder 240 of the encoding equipment 200 of FIG. 16. In addition, although not shown in FIG. 15, prediction samples or prediction-related information may be retrieved by the prediction module 220 of the encoding equipment 200 in FIG. 15, and the residual information may be extracted from the original samples or prediction samples by the residual processor 230 of the encoding equipment 200, and a bit stream may be generated from the residual information or prediction-related information by the entropy encoder 240 of the encoding equipment 200. The method disclosed in FIG. 15 may include the embodiments described above in this document.

[269] Referring to FIG. 15, the encoding equipment can determine the inter-prediction mode of the current block and generate inter-prediction mode information indicating the inter-prediction mode (S1700). For example, the encoding equipment may determine at least one of a regular combining mode, a skip mode, a motion vector prediction (MVP) mode, a motion vector difference (MMVD) combining mode, a sub-block combining mode, a CIIP mode (combined inter-frame and intra-frame combining mode). prediction) and a segmentation mode that performs prediction by dividing the current block into two segments, as an inter-prediction mode to be applied to the current block, and generate inter-prediction mode information indicating the inter-prediction mode.

[270] The encoding equipment may generate a list of possible combinations of the current block based on the inter-prediction mode (S1510). For example, the encoding equipment may generate a list of possible combinations according to a particular inter-prediction mode.

[271] For example, possibilities may be inserted into the list of possible combinations until the number of possibilities in the list of possible combinations is the maximum number of possible possibilities. Here, a candidate may indicate a candidate or block candidate for extracting motion information (or motion vector) of the current block. For example, a candidate block may be retrieved by searching neighboring blocks of the current block. For example, a neighbor block may include a spatial neighbor block and/or a temporal neighbor block relative to the current block, and preferably a spatial neighbor search (spatial join) can be performed to retrieve a candidate, and thereafter a temporary neighbor search can be performed and retrieved it as a candidate (temporary join), and the extracted candidates can be inserted into the list of candidate joins. For example, when the number of candidates in the merge candidate list is less than the maximum number of candidates in the merge candidate list even after candidates are inserted, an additional candidate may be inserted. For example, the additional candidate may include at least one of history-based pool candidate(s), pairwise average pool candidate(s), ATMVP, and combined bipredictive pool candidate(s) (when the slice group/tile type of the current slice group /tile represents type B) and/or a possible combination with a zero vector.

[272] The encoding equipment may generate selection information indicating one of the candidate options included in the combination candidate list (S1520). As described above, the list of join opportunities may include at least a portion of a spatial join opportunity, a temporal join opportunity, a pairwise opportunity, or a zero vector opportunity, and one of these opportunities may be selected to inter-predict the current block.

[273] For example, the selection information may include index information indicating one candidate among the combining candidates included in the list of combining candidates. For example, the selection information may be called “merging index information.”

[274] For example, the encoding equipment may generate predictive samples of the current block based on the candidate indicated by the combining index information. Alternatively, for example, the encoding equipment may extract motion information based on the candidate indicated by the combining index information, and may generate predictive samples of the current block based on the motion information.

[275] At this time, based on the fact that the MMVD mode (motion vector difference combining mode), sub-block combining mode, CIIP mode (combined inter-frame combining and intra-frame prediction mode) and segmentation mode to perform prediction by dividing the current block segments are not available, regular merging mode can be applied to the current block. Here, a list of possible merge options can be generated by applying a regular merge mode to the current block.

[276] For example, the inter-prediction mode information may include merging index information indicating one of the merging candidates included in the merging candidate list of the current block, and movement information of the current block may be extracted based on the candidate indicated by the merging index information . In addition, predictive samples of the current block can be generated based on the extracted motion information.

[277] For example, the inter-prediction mode information may include a first flag indicating whether or not the MMVD mode is applied, a second flag indicating whether or not a subblock combining mode is applied, and a third flag indicating whether or not CIIP is applied. -mode.

[278] For example, based on the case in which the MMVD mode, sub-block aggregation mode, CIIP mode and segmentation mode are not available, the values of the first flag, second flag and third flag may be 0.

[279] In addition, for example, the inter-prediction mode information may include a common merge flag indicating whether or not a merge mode is available for the current block, and the value of the common merge flag may be 1.

[280] For example, a flag for activating or deactivating a segmentation mode may be included in a sequence parameter set (SPS) of image information, and based on the case in which the segmentation mode is deactivated, the value of a fourth flag indicating whether or not the segmentation mode is applied may be set equal to 0.

[281] Meanwhile, the inter-prediction mode information may further include a fifth flag indicating whether or not a regular combining mode is applied. Even when the value of the fifth flag is 0, the regular combining mode can be applied to the current block based on the case in which the MMVD mode, sub-block combining mode, CIIP mode and segmentation mode are not available.

[282] In this case, motion information of the current block may be extracted based on the first merge candidate among the merge candidates included in the list of merge candidates of the current block. In addition, predictive samples may be generated based on current block motion information extracted based on the first possible combination.

[283] Alternatively, in this case, motion information of the current block may be extracted based on the motion vector (0,0), and prediction samples may be generated based on the motion information of the current block extracted based on the motion vector (0,0).

[284] The encoding equipment may encode image information including inter-prediction mode information and selection information (S1530). For example, image information may be referred to as "video information". The image information may include various information according to the embodiment(s) of the present disclosure described above. For example, the image information may include at least a portion of the prediction-related information or related residual information. For example, the prediction-related information may include at least a portion of inter-prediction mode information, selection information, and inter-prediction type information. For example, encoding equipment may encode image information including all or part of the above information (or syntax elements) to generate bitstream information or encoded information. Alternatively, the encoding equipment may output the information in the form of a bitstream. Additionally, the bit stream or encoded information may be transmitted to the decoding equipment via a network or storage medium.

[285] Alternatively, although not shown in FIG. 15, for example, the encoding equipment may extract residual samples based on the prediction samples and the original samples. In this case, related residual information can be extracted based on the residual samples. Residual samples can be extracted based on associated residual information. Reconstructed samples can be formed based on residual samples and predictive samples. The reconstructed block and the reconstructed frame can be retrieved based on the reconstructed samples. Alternatively, for example, the encoding equipment may encode image information including residual information or prediction-related information.

[286] For example, encoding equipment may generate bitstream information or encoded information by encoding image information including all or part of the above information (or syntax elements). The encoded image/video information may be output in the form of a bit stream. Additionally, the bit stream or encoded information may be transmitted to the decoding equipment via a network or storage medium. Alternatively, the bitstream or encoded information may be stored in a computer-readable storage medium, and the bitstream or encoded information may be generated by the above-mentioned image encoding method.

[287] FIG. 17 and 18 schematically show an example of a method for decoding video/images and related components according to embodiment(s) of the present disclosure.

[288] The method disclosed in FIG. 17 may be performed by the decoding equipment disclosed in FIG. 3 or fig. 18. In particular, for example, step S1700 in FIG. 17 may be performed by the entropy decoder 310 of the decoding equipment 300 in FIG. 1, and steps S1710-S1730 of FIG. 17 may be performed by the prediction module 330 of the decoding equipment 300 in FIG. 18.

[289] In addition, although not shown in FIG. 17, prediction-related information or residual information may be extracted from the bit stream by the entropy decoder 310 of the decoding equipment 300 in FIG. 18. The method disclosed in FIG. 17 may include the embodiments described above in this document.

[290] Referring to FIG. 17, the decoding equipment may receive image information including inter prediction mode information via a bit stream (S1700). For example, image information may be referred to as "video information". The image information may include various information according to the above embodiment(s) of the present disclosure. For example, the image information may include at least a portion of the prediction-related information or related residual information.

[291] For example, the prediction-related information may include intra-prediction mode information and/or intra-prediction type information. For example, the inter-prediction mode information may include information indicating at least some of the different inter-prediction modes. For example, various modes can be used, such as regular combining mode, skip mode, MVP (Motion Vector Prediction) mode, MMVD mode (motion vector difference combining mode), sub-block combining mode, CIIP mode (combined inter-frame and intra-frame combining mode). prediction) and segmentation mode, which performs prediction by dividing the current block into two segments. For example, the inter-prediction type information may include an inter_pred_idc syntax element. Alternatively, the inter-prediction type information may include information indicating any of L0 prediction, L1 prediction, and bi-prediction.

[292] The decoding equipment may generate a list of possible combinations of the current block based on the inter-prediction mode information (S1710). For example, the decoding equipment may generate a list of possible combining options according to a certain inter-prediction mode from a regular combining mode, a skip mode, an MVP mode, an MMVD mode, a sub-block combining mode, a CIIP mode, and a segmentation mode that performs prediction by dividing the current block into two segment as the inter-prediction mode of the current block based on the inter-prediction mode information.

[293] For example, candidates may be inserted into the pool candidate list until the number of candidates in the pool candidate list is the maximum number of possibilities. Here, a candidate may indicate a candidate or block candidate for extracting motion information (or motion vector) of the current block. For example, a candidate block may be retrieved by searching neighboring blocks of the current block. For example, a neighbor block may include a spatial neighbor block and/or a temporal neighbor block relative to the current block, and preferably a spatial neighbor search (spatial join) can be performed to retrieve a candidate, and thereafter a temporary neighbor search can be performed and retrieved it as a candidate (temporary join), and the extracted candidates can be inserted into the list of candidate joins. For example, when the number of candidates in the merge candidate list is less than the maximum number of candidates in the merge candidate list even after candidates are inserted, an additional candidate may be inserted. For example, the additional candidate may include at least one of history-based pool candidate(s), pairwise average pool candidate(s), ATMVP, and combined bipredictive pool candidate(s) (when the slice group/tile type of the current slice group /tile represents type B) and/or a possible combination with a zero vector.

[294] The decoding equipment may extract motion information of the current block based on the candidate selected from the combination candidate list (S1720).

[295] As described above, the list of join opportunities may include at least a portion of a spatial join opportunity, a temporal join opportunity, a pairwise opportunity, or a zero vector opportunity, and one of these opportunities may be selected for inter-prediction current block.

[296] For example, the selection information may include index information indicating one candidate among the combining candidates included in the list of combining candidates. For example, the selection information may be called “merging index information.”

[297] The decoding equipment may generate predictive samples of the current block based on the motion information (S1730). For example, the decoding equipment may generate predictive samples of the current block based on the candidate indicated by the combining index information. Alternatively, for example, the decoding equipment may extract motion information based on the candidate indicated by the combining index information, and may generate predictive samples of the current block based on the motion information.

[298] Meanwhile, according to an embodiment, based on the fact that the motion vector difference (MMVD) combining mode, the sub-block combining mode, the combined inter-frame combining and intra-prediction (CIIP) mode, and the segmentation mode for performing prediction by dividing the current block into two segments are not available, regular merging mode can be applied to the current block. Here, a list of possible merge options can be generated by applying a regular merge mode to the current block.

[299] For example, the inter-prediction mode information may include merging index information indicating one of the merging candidates included in the merging candidate list of the current block, and movement information of the current block may be extracted based on the candidate indicated by the merging index information . In addition, predictive samples of the current block can be generated based on the extracted motion information.

[300] For example, the inter-prediction mode information may include a first flag indicating whether or not the MMVD mode is applied, a second flag indicating whether or not a sub-block combining mode is applied, and a third flag indicating whether or not CIIP is applied. -mode.

[301] For example, based on the case in which the MMVD mode, sub-block aggregation mode, CIIP mode and segmentation mode are not available, the values of the first flag, second flag and third flag may be 0.

[302] In addition, for example, the inter-prediction mode information may include a common merge flag indicating whether or not a merge mode is available for the current block, and the value of the common merge flag may be 1.

[303] For example, a flag for activating or deactivating a segmentation mode may be included in a sequence parameter set (SPS) of image information, and based on the case in which the segmentation mode is deactivated, the value of a fourth flag indicating whether or not the segmentation mode is applied can be set equal to 0.

[304] Meanwhile, the inter-prediction mode information may further include a fifth flag indicating whether or not the regular combining mode is applied. Even when the value of the fifth flag is 0, the regular combining mode can be applied to the current block based on the case in which the MMVD mode, sub-block combining mode, CIIP mode and segmentation mode are not available.

[305] In this case, motion information of the current block may be extracted based on the first merge candidate among the merge candidates included in the list of merge candidates of the current block. In addition, predictive samples may be generated based on current block motion information extracted based on the first possible combination.

[306] Alternatively, in this case, motion information of the current block may be extracted based on the motion vector (0,0), and prediction samples may be generated based on the motion information of the current block extracted based on the motion vector (0,0).

[307] Although not shown in FIG. 17, for example, the decoding equipment may extract residual samples based on associated residual information included in the image information.

[308] For example, decoding equipment may obtain image information including all or parts of the above-described pieces of information (or syntax elements) by decoding a bit stream or encoded information. Additionally, the bitstream or encoded information may be stored on a computer-readable storage medium and may cause the above-described decoding method to be performed.

[309] Although the methods are described on the basis of a flowchart in which steps or blocks are listed in sequence in the above-described embodiments, the steps herein are not limited to a particular order, and a particular step may be performed in another step or in a different order or simultaneously regarding what is described above. Additionally, those skilled in the art will appreciate that steps in the flowcharts are not the only possible steps, and another step may be included or one or more steps in the flowchart may be deleted without affecting the scope of the present disclosure.

[310] The above method according to the present disclosure may be implemented in the form of software, and the encoding equipment and/or decoding equipment according to the present disclosure may be included in an apparatus for performing image processing, for example, a television, a computer, a smartphone, a set-top box, a display device, and etc.

[311] When embodiments of the present disclosure are implemented by software, the above method may be implemented by a module (process or function) that performs the above function. The module may be stored in a storage device and executed by a processor. The storage device may be mounted inside or outside the processor and may be coupled to the processor through various known means. The processor may include an application specific integrated circuit (ASIC), other chipsets, logic circuitry, and/or data processing apparatus. The storage device may include read-only memory (ROM), random access memory (RAM), flash memory, memory card, storage medium, and/or other data storage device. In other words, embodiments of the present disclosure may be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional modules illustrated in the respective drawings may be implemented and executed on a computer, processor, microprocessor, controller, or chip. In this case, implementation information (eg, instruction information) or algorithms may be stored on a digital storage medium.

[312] In addition, the decoding equipment and encoding equipment to which the embodiment(s) of this document applies may be included in a broadcast multimedia transceiver device, a mobile communication terminal, a home theater system video device, a digital cinema system video device, a surveillance camera, video chat device and real-time communication device, e.g., video communication device, mobile streaming device, storage medium, video recording camera, video-on-demand (VoD) service provider, video-over-the-top (OTT) device, service provider Internet streaming device, 3D video device, virtual reality (VR) style device, augmented reality (AR) style device, telephone video device, in-vehicle terminal (e.g., in-vehicle terminal (including autonomous vehicle), aircraft terminal or ship terminal) and medical video device; and can be used to process an image signal or data. For example, an OTT video device may include a game console, Blu-ray player, Internet-enabled television, home theater system, smartphone, tablet PC, and digital video recorder (DVR).

[313] In addition, the processing method to which the embodiment(s) of the present document is applied may be formed in the form of a program executed by a computer and may be stored on a computer-readable recording medium. Multimedia data having a data structure according to the embodiment(s) of the present document may also be stored on a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices on which computer-readable data is stored. The computer-readable recording medium may include, for example, a Blu-ray disc (BD), a universal serial bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical device. data storage. A computer-readable recording medium also includes media implemented in the form of a carrier wave (eg, Internet transmissions). In addition, the bitstream generated by the encoding method may be stored in a computer-readable recording medium or transmitted through a wired or wireless communication network.

[314] In addition, the embodiment(s) of the present document may be implemented as a computer program product based on program code, and the program code may be executed on a computer according to the embodiment(s) of the present document. The program code may be stored on a computer-readable medium.

[315] FIG. 19 represents an example of a content streaming system to which an embodiment of the present document may be applied.

[316] Referring to FIG. 19, a content streaming system to which embodiments of this document apply may generally include an encoding server, a streaming server, a web server, a media storage device, a user device, and a media input device.

[317] The encoding server functions to digitally compress content input from media input devices such as a smartphone, camera, video recorder, etc. to form a bitstream and transmit it to the streaming server. As another example, in case a media input device such as a smartphone, camera, video recorder, etc. directly generates a bitstream, the encoding server may be omitted.

[318] The bitstream may be generated by an encoding method or a bitstream generation method to which embodiments of this document apply. In addition, the streaming server may temporarily store the bit stream while transmitting or receiving the bit stream.

[319] The streaming server transmits multimedia data to the subscriber device based on the user's request through the web server, which functions as a tool that informs the user as to what service is provided. When a user requests a service that the user wants, the web server passes the request to the streaming server and the streaming server streams the media data to the user. In this regard, the content streaming system may include a separate control server, in which case, the control server functions to manage commands/responses between associated equipment in the content streaming system.

[320] The streaming server may receive content from a media storage and/or encoding server. For example, if the content is received from an encoding server, the content may be received in real time. In this case, the streaming server can store the bit stream for a predetermined period of time to smoothly provide the streaming service.

[321] For example, a subscriber device may include a mobile phone, a smartphone, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable media player (PMP), a navigation device, a tablet PC, a tablet PC, an ultrabook, a wearable device (for example, a watch terminal (smart watch), a glasses terminal (smart glasses), a head-mounted display (HMD)), a digital TV, a desktop computer, a digital signage system, etc.

[322] Each of the servers in the content streaming system may operate as a distributed server, in which case, data received through each server may be processed in a distributed manner.

[323] The claims herein may be combined in various ways. For example, the technical features in the claims for the method of the present description can be combined with the ability to be implemented or performed in equipment, and the technical features in the claims for equipment can be combined with the ability to be implemented or performed in the method. Additionally, the technical features in the method claim(s) and equipment claim(s) may be combined with the ability to be implemented or performed in the equipment. Additionally, the technical features in the method claim(s) and equipment claim(s) may be combined with the ability to be implemented or performed in the method.

Claims (45)

1. Decoding equipment for decoding images, wherein the decoding equipment comprises: Memory device; And at least one processor coupled to the storage device, wherein the at least one processor is configured to: receive image information including inter-prediction mode information via the bit stream; generate a list of possible options for combining the current block based on information from the mutual prediction mode; extract movement information of the current block based on a candidate selected from a list of candidate combinations; generate predictive samples of the current block based on motion information, and generate a reconstructed frame based on predictive samples; wherein the image information includes a first activation flag for a combined inter-frame and intra-prediction (CIIP) mode and a second activation flag for a segmentation prediction mode, wherein for a segmentation prediction mode, prediction is performed by block segmentation, wherein the segmentation includes itself triangular segmentation, wherein the combining data syntax is associated with the inter-prediction mode information, wherein the combining data syntax included in the image information includes at least one of a regular combining flag associated with whether or not a regular combining mode is applied to the current block, a flag an MMVD merging flag associated with whether or not the motion vector difference (MMVD) merging mode is applied to the current block, and a sub-block merging flag associated with whether or not the sub-block merging mode is applied to the current block, wherein the parsing order of the regular join flag precedes the parsing order of the MMVD join flag, and wherein, based on the value of the first activation flag being 0, the value of the second activation flag being 0, the value of the sub-block combining flag being 0, and the value of the MMVD combining flag being 0, the regular combining mode is applied, and the combining index information for the regular mode join is included in the join data syntax. 2. The decoding equipment according to claim 1, wherein the inter-prediction mode information includes a common combining flag, and wherein, the syntax of the concatenation data for the current block is included in the image information based on the value of the common concatenation flag being 1. 3. The decoding equipment of claim 1, wherein the first enable flag for the CIIP mode and the second enable flag for the segmentation prediction mode are included in the sequence parameter set (SPS) of the image information. 4. The decoding equipment according to claim 1, wherein, for the case in which the value of the first activation flag is 0, the value of the second activation flag is 0, the value of the sub-block combining flag is 0, and the value of the MMVD combining flag is 0, the regular join flag with a value of 1 is not explicitly signaled, the regular join mode is applied to the current block, and the join index information is included in the join data syntax. 5. The decoding equipment of claim 4, wherein the combining index information is further included in the combining data syntax based on a maximum number of possible combining options for the regular combining mode greater than 1. 6. Encoding equipment for encoding images, wherein the encoding equipment comprises: Memory device; And at least one processor coupled to the storage device, wherein the at least one processor is configured to: determine the inter-prediction mode of the current block and generate inter-prediction mode information indicating the inter-prediction mode; generate a list of possible options for combining the current block based on the mutual prediction mode; generate selection information indicating one of the possible options included in the list of possible combination options; And encode image information including inter-prediction mode information and selection information, wherein the image information includes a first activation flag for a combined inter-frame and intra-prediction (CIIP) mode and a second activation flag for a segmentation prediction mode, wherein for a segmentation prediction mode, prediction is performed by block segmentation, wherein the segmentation includes itself triangular segmentation, wherein the combining data syntax is associated with the inter-prediction mode information, wherein the combining data syntax included in the image information includes at least one of a regular combining flag associated with whether or not a regular combining mode is applied to the current block, a flag an MMVD merging flag associated with whether or not the motion vector difference (MMVD) merging mode is applied to the current block, and a sub-block merging flag associated with whether or not the sub-block merging mode is applied to the current block, wherein the parsing order of the regular join flag precedes the parsing order of the MMVD join flag, and wherein, based on the value of the first activation flag being 0, the value of the second activation flag being 0, the value of the sub-block combining flag being 0, and the value of the MMVD combining flag being 0, the regular combining mode is applied, and the combining index information for the regular mode join is included in the join data syntax. 7. The encoding equipment of claim 6, wherein the inter-prediction mode information includes a common combining flag, and wherein, the syntax of the concatenation data for the current block is included in the image information based on the value of the common concatenation flag being 1. 8. The encoding equipment of claim 6, wherein the first enable flag for the CIIP mode and the second enable flag for the segmentation prediction mode are included in the sequence parameter set (SPS) of the image information. 9. The encoding equipment according to claim 6, wherein for the case in which the value of the first activation flag is 0, the value of the second activation flag is 0, the value of the sub-block combining flag is 0, and the value of the MMVD combining flag is 0, the regular join flag with a value of 1 is not explicitly signaled, the regular join mode is applied to the current block, and the join index information is included in the join data syntax. 10. The encoding equipment of claim 9, wherein the combining index information is further included in the combining data syntax based on a maximum number of possible combining options for the regular combining mode greater than 1. 11. Equipment for transmitting image data, the equipment comprising: at least one processor configured to: determine the inter-prediction mode of the current block and generate inter-prediction mode information indicating the inter-prediction mode; generate a list of possible options for combining the current block based on the mutual prediction mode; generate selection information indicating one of the possible options included in the list of possible combination options; And encode the image information to generate a bitstream, wherein the image information includes inter-prediction mode information and selection information, wherein the image information includes a first activation flag for a combined inter-frame and intra-prediction (CIIP) mode and a second activation flag for a segmentation prediction mode, wherein for a segmentation prediction mode, prediction is performed by block segmentation, wherein the segmentation includes itself triangular segmentation, wherein the combining data syntax is associated with the inter-prediction mode information, wherein the combining data syntax included in the image information includes at least one of a regular combining flag associated with whether or not a regular combining mode is applied to the current block, a flag an MMVD merging flag associated with whether or not the motion vector difference (MMVD) merging mode is applied to the current block, and a sub-block merging flag associated with whether or not the sub-block merging mode is applied to the current block, wherein the parsing order of the regular join flag precedes the parsing order of the MMVD join flag, and wherein, based on the value of the first activation flag being 0, the value of the second activation flag being 0, the value of the sub-block combining flag being 0, and the value of the MMVD combining flag being 0, the regular combining mode is applied, and the combining index information for the regular mode join is included in the join data syntax.