RU2816199C1 - Video or image encoding based on intra-block encoding - Google Patents

Abstract

FIELD: video encoding.

SUBSTANCE: prediction mode for the current block in the current frame is extracted as a prediction mode based on intra-block coding (IBC). Flag information for the IBC prediction mode is generated. A list of candidates for the IBC prediction mode is retrieved, which includes candidate block vectors for the reference block in the current frame. Index information for the block vector of the current block is generated based on the list of possible options for the IBC prediction mode. Residual information is generated for the residual samples of the current block. Image information including flag information, index information and residual information is encoded. The image information includes information regarding the maximum number of possible block vectors included in the candidate list for the IBC prediction mode and information regarding the maximum number of possible motion vector candidates included in the candidate list for the inter-prediction mode, different from said information regarding the maximum number of possible block vector options.

EFFECT: increase of efficiency of video encoding.

4 cl, 13 dwg, 12 tbl

Description

State of the art

Field of technology to which the invention relates

[1] This document relates to video or image coding based on Intra Block Coding (IBC).

Description of the prior art

[2] Recently, the demand for high-quality, high-definition image/video, such as ultra-high-definition (UHD) image/video in 4K or 8K or higher, has been growing in various technology fields. Since the image/video data is of high resolution and high quality, the amount of information or bits that must be transmitted increases relative to the existing image/video data, and therefore, transmitting the image data using a medium such as an existing wired/wireless broadband line or existing storage media, or storing image/video data using an existing storage media increases transmission costs and storage costs.

[3] In addition, interest and demand for immersive multimedia such as virtual reality (VR) and artificial reality (AR) content or holograms has recently been increasing, and broadcasting for image/video having characteristics different from reality images , such as game images, increases.

[4] Accordingly, a high-performance image/video compression technology is required to efficiently compress, transmit, store and reproduce high-quality, high-resolution image/video information having various characteristics as described above.

[5] In addition, there is discussion about intra-block coding (IBC) technology to improve compression efficiency and improve encoding efficiency for screen content. To effectively utilize these technologies, there is a need for a method for efficiently signaling associated information.

The essence of the invention

[6] According to an embodiment of the present document, a method and equipment for improving the efficiency of image encoding are provided.

[7] According to an embodiment of the present document, a method and equipment for effectively applying filtration are provided.

[8] According to an embodiment of the present document, a method and equipment for efficient IBC application are provided.

[9] According to an embodiment of the present document, a method and equipment are provided for signaling information regarding the maximum number of candidate block vectors included in the candidate list for the IBC.

[10] According to an embodiment of the present document, a method and equipment are provided for separately transmitting, in signaling signals, information regarding the maximum number of possible block vector candidates included in the combination candidate list for an IBC merge, and information regarding the maximum number of possible motion vector candidates included to the list of possible join options for the cross-prediction join mode.

[11] According to an embodiment of the present document, a video/image decoding method performed by decoding equipment is provided.

[12] According to an embodiment of the present document, decoding equipment for performing video/image decoding is provided.

[13] According to an embodiment of the present document, a video/image encoding method performed by encoding equipment is provided.

[14] According to an embodiment of the present document, encoding equipment for performing video/image encoding is provided.

[15] According to one embodiment of the present document, a computer-readable digital storage medium is provided on which encoded video/image information generated according to a video/image encoding method disclosed in at least one embodiment of the present document is stored.

[16] According to an embodiment of the present document, a computer-readable digital storage medium is provided on which encoded information or encoded video/image information instructing the implementation of a video/image decoding method disclosed in at least one embodiment of this document is stored, through decoding equipment.

Advantages of the invention

[17] According to an embodiment of the present document, overall image/video compression efficiency can be improved.

[18] According to an embodiment of the present document, subjective/objective visual quality can be improved through efficient filtering.

[19] According to an embodiment of the present document, a list of possible combinations for the IBC can be constructed.

[20] According to an embodiment of the present document, information regarding the maximum number of candidates included in the list of combining candidates for the IBC may be signaled.

[21] According to an embodiment of the present document, the degree of freedom for the codec can be increased, and the coding efficiency can be improved.

Brief description of drawings

[22] FIG. 1 illustrates an example of a video/image coding system to which embodiments of this document may be applied.

[23] FIG. 2 is a diagram schematically illustrating the configuration of video/image encoding equipment to which embodiments of the present document can be applied.

[24] FIG. 3 is a diagram schematically illustrating the configuration of video/image decoding equipment to which embodiments of the present document can be applied.

[25] FIG. 4 is a drawing for explaining a method for extracting motion information through the inter-prediction mode.

[26] FIG. 5 and fig. 6 illustrates an image encoding method based on an IBC prediction mode and an encoding equipment prediction unit implementing the image encoding method.

[27] FIG. 7 and fig. 8 illustrates an image decoding method based on the IBC prediction mode and a decoding equipment prediction unit implementing the image decoding method.

[28] FIG. 9 and fig. 10 schematically shows an example of a method for encoding videos/images and related components according to embodiment(s) of the present document.

[29] FIG. 11 and fig. 12 schematically shows an example of an image/video decoding method and related components according to an embodiment of the present document.

[30] FIG. 13 shows an example of a content streaming system to which the embodiments disclosed herein may be applied.

Detailed Description of Embodiments

[31] The present document may be modified in various forms, and specific embodiments thereof are described and shown in the drawings. However, the embodiments are not intended to limit the present document. The terms used in the following description are used to simply describe specific embodiments, but are not intended to limit this document. The singular expression includes the plural expression as long as they are clearly interpreted differently. Terms such as “include” and “have” are intended to indicate that there are features, numbers, steps, operations, elements, components, or combinations thereof used in the following description, and as such are to be understood that the possibility of the presence or addition of one or more other features, numbers, steps, operations, elements, components or combinations of the above is not excluded.

[32] Meanwhile, each configuration in the drawings described herein is shown independently for convenience of description regarding various characteristic functions and does not mean that each configuration is implemented as separate hardware or separate software. For example, two or more components for each component may be combined to form one component, or one component may be divided into multiple components. Embodiments in which each component is integrated and/or separated are also included within the scope of the disclosure herein.

[33] Hereinafter, examples of the present embodiment will be described in detail with reference to the accompanying drawings. In addition, like reference numerals are used to identify like elements throughout the drawings, and identical descriptions of like elements are omitted.

[34] FIG. 1 illustrates an example of a video/image coding system to which embodiments of this document may be applied.

[35] Referring to FIG. 1, a video/image encoding system may include a first device (source device) and a second device (receiver device). A source device may transmit encoded information or video/image data to a receiving device via a digital storage medium or network in the form of a file or stream.

[36] The source device may include a video source, encoding equipment, and a transmitter. The receiving device may include a receiver, decoding equipment, and a rendering module. The encoding equipment may be referred to as "video/image encoding equipment" and the decoding equipment may be referred to as "video/image decoding equipment". The transmitter may be included in the encoding equipment. The receiver may be included in the decoding equipment. The rendering module may include a display, and the display may be configured as a separate device or an external component.

[37] A video source may obtain video/image through a video/image capture, synthesis, or shaping process. The video source may include a video/image capture device and/or a video/image generation device. The video/image capturing device may include, for example, one or more cameras, video/image archives including previously captured video/images, and the like. The video/image generating apparatus may include, for example, computers, tablet computers and smartphones and may (electronically) generate video/images. For example, a virtual video/image may be generated via a computer or the like. In this case, the video/image capturing process can be replaced by the associated data generation process.

[38] Encoding equipment can encode the input video/image. Encoding hardware may perform a sequence of procedures, such as prediction, transform, and quantization, for compression and encoding efficiency. Encoded data (encoded video/image information) may be output in the form of a bitstream.

[39] A transmitter may transmit information or encoded image/image data output in the form of a bitstream to a receiver of a receiving device via a digital storage medium or network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, and the like. The transmitter may include an element for generating a multimedia file through a predefined file format and may include an element for transmission through a broadcast/communication network. The receiver may receive/extract the bit stream and transmit the received bit stream to decoding equipment.

[40] The decoding equipment can decode a video/image by performing a sequence of procedures such as dequantization, deconversion and prediction corresponding to the operation of the encoding equipment.

[41] The rendering module may prepare a decoded video/image by rendering. The video/image prepared by rendering can be displayed through the display.

[42] This document relates to video/image coding. For example, the method/embodiment disclosed in this document may be applied to the method disclosed in the Versatile Video Coding (VVC) standard, the Fundamental Video Coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the second generation audio/video coding standard (AVS2) or next generation video/image coding standard (for example, H.267, H.268, etc.).

[43] This document proposes various embodiments of video/image encoding, and the above embodiments can also be performed in combination with each other unless otherwise specified.

[44] As used herein, video may refer to a sequence of images over time. A frame generally means a unit representing one image in a particular time frame, and a slice/tile means a unit that constitutes a part of a frame from an encoding point of view. A slice/tile may include one or more coding tree units (CTUs). One frame can consist of one or more slices/tiles. One frame can consist of one or more groups of tiles. One tile group may include one or more tiles. A brick can represent a rectangular area of CTU rows in a tile in a frame. A tile can be segmented into multiple bricks, each of which can be constructed with one or more CTU rows in the tile. A tile that is not segmented into multiple bricks may also be called a "brick". A brick scan may represent a particular sequential ordering of CTUs segmenting a frame, where CTUs may be ordered sequentially by a raster scan of CTUs in a brick, and bricks in a tile may be ordered sequentially by a raster scan of tile bricks, and tiles in a frame may be ordered sequentially by a raster scan of the frame's tiles. A tile is a rectangular CTU area in a particular column of tiles and a particular row of tiles in a frame. A column of tiles is a rectangular area of CTUs having a height equal to the height of the frame and a width specified by syntax elements in the frame parameter set. A row of tiles is a rectangular area of CTUs having a height specified by syntax elements in the frame parameter set and a width equal to the width of the frame. A tile scan is a specific sequential ordering of the CTUs segmenting a frame, in which CTUs are ordered sequentially in a raster scan of the CTUs in a tile, whereas tiles in a frame are ordered sequentially in a raster scan of the frame's tiles. A slice includes an integer number of frame bricks that can be contained exclusively in a single NAL unit. A slice can consist of either a specific number of full tiles, or only a rigid sequence of full bricks of a single tile. Herein, a tile group and a slice may be used instead of each other. For example, herein, a tile group/tile group header may be referred to as a "slice/slice header".

[45] Meanwhile, one frame may be divided into two or more subframes. A subframe may be a rectangular area of one or more slices in a frame.

[46] Pixel or pel can mean the smallest unit that makes up one frame (or image). Additionally, "sample" can be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel/pixel value of a luma component or only a pixel/pixel value of a chrominance signal component.

[47] The unit may represent a basic image processing unit. The unit may include at least one of a specific frame area and information associated with the area. One unit may include one block of luma signals and two blocks of chrominance signals (eg, Cb, Cr). Unit may be used interchangeably with terms such as block or zone in some cases. In general, an MxN block may include samples (or arrays of samples) or a set (or array) of transform coefficients of M columns and N rows. Alternatively, a sample may refer to a pixel value in the spatial domain, and when such a pixel value is converted to a frequency domain, it may refer to a frequency domain conversion factor.

[48] As used herein, “A or B” may mean “A only,” “B only,” or “both A and B.” In other words, "A or B" herein may be interpreted as "A and/or B". For example, as used herein, "A, B or C (A, B or C)" means "A only", "B only", "C only" or "any combination of A, B and C".

[49] A slash (/) or comma (comma) used herein may indicate “and/or.” For example, "A/B" could mean "A and/or B". Accordingly, "A/B" can mean "A only", "B only" or "both A and B". For example, "A, B, C" could mean "A, B or C".

[50] As used herein, “at least one of A and B” may mean “A only,” “B only,” or “both A and B.” Moreover, as used herein, the expression “at least one of A or B” or “at least one of A and/or B” may be interpreted identically to “at least one of A and B”.

[51] Additionally, as used herein, “at least one of A, B, and C” means “A only,” “B only,” “C only,” or “any combination of A, B, and C.” Additionally, “at least one of A, B, or C” or “at least one of A, B, and/or C” may mean “at least one of A, B, and C.”

[52] Additionally, parentheses used herein may indicate "for example." In particular, when "forecasting (internal forecasting)" is specified, "internal forecasting" may be offered as an example of "forecasting". In other words, “forecasting” herein is not limited to “internal forecasting,” and “internal forecasting” may be offered as an example of “forecasting.” Moreover, even when “forecasting (ie, internal forecasting)” is specified, “internal forecasting” may be offered as an example of “forecasting.”

[53] Technical features that are separately described in a single drawing herein may be implemented separately or simultaneously.

[54] FIG. 2 is a diagram schematically illustrating the configuration of video/image encoding equipment to which embodiments of the present document can be applied. Hereinafter in this document, what is referred to as “video encoding equipment” may include image encoding equipment.

[55] Referring to FIG. 2, the encoding equipment 200 includes an image segmentation module 210, a prediction module 220, a residual processor 230 and an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The prediction module 220 may include an inter-prediction module 221 and an internal prediction module 222. forecasting. Residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. Residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a “recovery module” or a “recovery block generator.” The image segmentation module 210, prediction module 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 may be configured by at least one hardware component (eg, a chipset or encoder processor) according to an embodiment. In addition, storage device 270 may include a decoded frame buffer (DPB) or may be configured with a digital storage medium. The hardware component may further include a storage device 270 as an internal/external component.

[56] The image segmentation module 210 may segment an input image (or frame or frame) input to the encoding equipment 200 into one more processing units. For example, a processing unit may be called a "coding unit (CU)". In this case, the coding unit may be recursively segmented according to a quadtree, binary tree, and ternary tree (QTBTTT) structure from a coding tree unit (CTU) or largest coding unit (LCU). For example, one coding unit may be segmented into multiple coding units of greater depth based on a quadtree structure, a binary tree structure, and/or a ternary structure. In this case, for example, a quadtree structure may be applied first, and a binary tree structure and a ternary structure may be subsequently applied. Alternatively, a binary tree structure may be used first. The encoding procedure of the present disclosure may be performed based on a final encoding unit that is no longer segmented. In this case, the largest coding unit can be used as the final coding unit based on the coding efficiency according to the characteristics of the images, or if necessary, the coding unit can be recursively segmented into coding units of greater depth, and the coding unit having the optimal size can be used as the final coding units. Here, the encoding procedure may include a prediction, transformation and reconstruction procedure, which is described below. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit may be split or segmented from the above final encoding unit. The prediction unit may be a sample prediction unit, and the transformation unit may be a unit for extracting a transformation coefficient and/or a unit for extracting a residual signal from the transformation coefficient.

[57] Unit may be used interchangeably with terms such as block or zone in some cases. In general, an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample may generally represent a pixel or a pixel value, may represent only a pixel/pixel value of a luma component, or may represent only a pixel/pixel value of a chrominance signal component. Sampling can be used as a term corresponding to one frame (or image) for a pixel or pel.

[58] In the encoding equipment 200, the prediction signal (predicted block, prediction sample array) output from the inter prediction unit 221 or intra prediction unit 222 is subtracted from the input image signal (original block, original sample array) to generate a residual signal ( residual block, residual sample array), and the generated residual signal is transmitted to converter 232. In this case, as shown, a module for subtracting the prediction signal (predicted block, prediction sample array) from the input image signal (source block, source sample array) into encoder 200 may be referred to as "subtractor 231". The prediction module may perform prediction on a block to be processed (hereinafter referred to as a "current block") and generate a prediction block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied based on the current block or CU. As described below in the description of each prediction mode, the prediction module may generate various prediction-related information, such as prediction mode information, and transmit the generated information to the entropy encoder 240. Information regarding the prediction may be encoded in the entropy encoder 240 and output in the form of a stream bits

[59] The intra prediction module 222 may predict the current block by reference to the samples in the current frame. The reference samples may be located in the surroundings of the current block or may be located independently according to the prediction mode. In intraprediction, prediction modes may include a plurality of non-directional modes and a plurality of directed modes. The omnidirectional mode may include, for example, DC mode and planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the granularity of the prediction direction. However, this is merely an example; more or less directional prediction modes may be used depending on the setting. The intra prediction module 222 may determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[60] The inter-prediction unit 221 can extract a predicted block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. Here, in order to reduce the amount of motion information transmitted in the inter-prediction mode, motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0-prediction, L1-prediction, bi-prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block present in the current frame and a temporal neighbor block present in the reference frame. The reference frame including the reference block and the reference frame including the temporary adjacent block may be identical or different. A temporary neighbor block may be called a "co-located reference block", a "co-located CU (colCU)", etc., and a reference frame including a temporary neighboring block may be called a "col-located frame (colPic)". For example, the inter-prediction module 221 may configure a list of motion information candidates based on neighboring blocks and generate information indicating which candidate is used to retrieve the motion vector and/or reference frame index of the current block. Mutual prediction can be performed based on different prediction modes. For example, in the case of the skip mode and the merge mode, the inter-prediction unit 221 may use the motion information of the adjacent block as the motion information of the current block. In skip mode, unlike combine mode, the residual signal may not be transmitted. In the case of a motion vector prediction (MVP) mode, the motion vector of an adjacent block may be used as a motion vector predictor, and the motion vector of the current block may be indicated by signaling a motion vector difference.

[61] The prediction module 220 may generate a prediction signal based on various prediction methods described below. For example, the prediction module may not only apply intra-prediction or inter-prediction to predict one block, but also simultaneously apply both intra-prediction and inter-prediction. This may be referred to as "combined inter-intraprediction (CIIP)". In addition, the prediction module can be based on intra-block copy (IBC) prediction mode or palette mode for block prediction. The IBC prediction mode or palette mode can be used for encoding image/video content of a game and the like, for example, screen content encoding (SCC). IBC essentially performs prediction in the current frame, but can be performed similarly to inter-prediction in which the reference block is retrieved in the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described in the present disclosure. Palette mode can be considered an example of intra-coding or intra-prediction. When palette mode is applied, a sampled value in a frame may be signaled based on information regarding the palette table and palette index.

[62] The prediction signal generated by the prediction module (including inter prediction module 221 and/or intra prediction module 222) can be used to generate a reconstructed signal or generate a residual signal. Converter 232 may generate conversion coefficients by applying conversion technology to the residual signal. For example, the transform technology may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a graph-based transform (GBT), or a conditionally nonlinear transform (CNT). Here, GBT means a graph-derived transformation when the relationship information between pixels is represented by the graph. CNT means a transform generated based on a prediction signal generated using all previously reconstructed pixels. In addition, the conversion process can be applied to square pixel blocks that are identical in size, or can be applied to blocks that are variable in size rather than square.

[63] Quantizer 233 may quantize the transform coefficients and transmit them to entropy encoder 240, and entropy encoder 240 may encode the quantized signal (information regarding the quantized transform coefficients) and output a bit stream. Information regarding the quantized transform coefficients may be referred to as "residual information". Quantizer 233 may re-arrange the block quantized transform coefficients into one-dimensional vector form based on the scanning order of the coefficients, and generate information regarding the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form. Information regarding conversion coefficients can be generated. The entropy encoder 240 may implement various encoding techniques, such as, for example, exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like. Entropy encoder 240 may encode information needed to reconstruct video/images other than quantized transform coefficients (eg, syntax element values, etc.), together or separately. Encoded information (eg, encoded video/image information) may be transmitted or stored in NAL (Network Abstraction Layer) units in the form of a bitstream. The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. In the present disclosure, information and/or syntax elements transmitted/signaled from encoding equipment to decoding equipment may be included in video/frame information. The video/image information may be encoded through the encoding procedure described above and included in the bitstream. The bit stream may be transmitted over a network or may be stored on a digital storage medium. The network may include a broadcast network and/or a communications network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, and the like. A transmitter (not shown) transmitting the signal output from the entropy encoder 240 and/or a storage module (not shown) storing the signal may be included as an internal/external element of the encoding equipment 200, and alternatively, the transmitter may be included in the entropy encoder 240 .

[64] The quantized transform coefficients output from quantizer 233 can be used to generate a predictive signal. For example, the residual signal (residual block or residual samples) may be recovered by applying dequantization and inverse transform to the quantized transform coefficients through dequantizer 234 and inverse transformer 235. Adder 250 adds the reconstructed residual signal with the prediction signal output from interprediction module 221 or module 222 internal prediction to generate a reconstructed signal (reconstructed frame, reconstructed block, array of reconstructed samples). If there is no remainder for a block to be processed, for example in a case in which the skip mode is applied, the predicted block can be used as a reconstructed block. The adder 250 may be referred to as a "recovery module" or a "recovered block generator." The generated reconstructed signal can be used to internally predict the next block to be processed in the current frame, and can be used to inter-predict the next frame through filtering, as described below.

[65] Meanwhile, luma conversion with chrominance scaling (LMCS) can be applied during encoding and/or reconstruction of frames.

[66] Filter 260 may enhance the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, filter 260 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and storing the modified reconstructed frame in storage device 270, namely, a DPB of storage device 270. Various filtering techniques may include, for example, deblocking filtering. , sampled adaptive bias, adaptive contour filter, bilateral filter, etc. The filter 260 may generate various filtering-related information and transmit the generated information to the entropy encoder 240, as described below in the description of each filtering method. Information associated with filtering may be encoded by entropy encoder 240 and output in the form of a bit stream.

[67] The modified reconstructed frame transferred to the storage device 270 can be used as a reference frame in the inter-prediction unit 221. When inter prediction is applied by the encoding equipment, prediction mismatch between the encoding equipment 200 and the decoding equipment 300 can be eliminated, and the encoding efficiency can be improved.

[68] The DPB of the storage device 270 may store the modified reconstructed frame for use as a reference frame in the inter-prediction module 221. Memory 270 may store block motion information from which motion information in the current frame is retrieved (or encoded) and/or motion information of blocks in a frame that have already been reconstructed. The stored motion information may be transmitted to the inter-prediction unit 221 and used as spatial neighbor block motion information or temporal neighbor block motion information. Memory 270 may store reconstructed samples of reconstructed blocks in the current frame and may transmit reconstructed samples to intra prediction module 222.

[69] FIG. 3 is a circuit diagram illustrating a configuration of video/image decoding equipment to which embodiment(s) of the present disclosure may be applied.

[70] Referring to FIG. 3, the decoding equipment 300 may include an entropy decoder 310, a residual processor 320, a prediction module 330, an adder 340, a filter 350, a memory 360. The prediction module 330 may include an inter-prediction module 331 and an intra-prediction module 332. Residual processor 320 may include a dequantizer 321 and an inverse converter 321. Entropy decoder 310, residual processor 320, prediction module 330, adder 340, and filter 350 may be configured by a hardware component (eg, a chipset or decoder processor) according to an embodiment. In addition, storage device 360 may include a decoded frame buffer (DPB) or may be configured with a digital storage medium. The hardware component may further include storage device 360 as an internal/external component.

[71] When a bitstream including video/image information is input, the decoding equipment 300 can reconstruct an image corresponding to the process in which the video/image information is processed in the encoding equipment of FIG. 2. For example, decoding equipment 300 may extract units/blocks based on block segmentation-related information obtained from the bitstream. The decoding equipment 300 may perform decoding using a processing unit used in the encoding equipment. Thus, the decoding processing unit, for example, may be a coding unit, and the coding unit may be segmented according to a quadtree structure, a binary tree structure, and/or a ternary tree structure from a coding tree unit or a largest coding unit. One or more transformation units may be derived from a coding unit. The reconstructed image signal decoded and output by the decoding equipment 300 can be reproduced by the playback equipment.

[72] The decoding equipment 300 may receive a signal output from the encoding equipment of FIG. 2 in the form of a bit stream, and the received signal may be decoded through entropy decoder 310. For example, entropy decoder 310 may parse the bit stream to extract information (eg, video/image information) needed for image reconstruction (or frame reconstruction). The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The decoding equipment may further decode the frame based on information regarding the parameter set and/or general constraint information. The signaling/receiving information and/or syntax elements described later in the present disclosure may be decoded, may be decoded by a decoding procedure, and obtained from the bit stream. For example, entropy decoder 310 decodes information in a bitstream based on an encoding method, such as exponential Golomb, CAVLC, or CABAC encoding, and the output syntax elements required for image reconstruction and the quantized values of the transform coefficients for the remainder. More specifically, the entropy CABAC decoding method may receive a resolution element corresponding to each syntax element in the bitstream, determine a context model using information of a target decoding syntax element, decoding information of a target decoding block, or information of a symbol/resolution element decoded in a previous stage, and perform arithmetic decoding for the permission element by predicting the probability of occurrence of the permission element according to the determined context model and generating a symbol corresponding to the meaning of each syntactic element. In this case, the entropy CABAC decoding method can update the context model by using the decoded symbol/grant element information for the context model of the next symbol/grant element after determining the context model. The prediction-related information from the information decoded by the entropy decoder 310 may be provided to the prediction module (inter-prediction module 332 and intra-prediction module 331), and the residual value for which entropy decoding is performed in the entropy decoder 310, i.e. the quantized transform coefficients and associated parameter information may be input to residual processor 320. Residual processor 320 may extract a residual signal (residual block, residual samples, array of residual samples). In addition, information regarding filtering from information decoded by the entropy decoder 310 may be provided to the filter 350. Meanwhile, a receiver (not shown) for receiving a signal output from the encoding equipment may be further configured as an internal/external element of the equipment 300 decoding, or the receiver may be a component of the entropy decoder 310. Meanwhile, the decoding equipment according to this document may be called "video/image/frame decoding equipment", and the decoding equipment may be classified into information decoder (video/image/frame information decoder) and selective decoder (video/image/frame sampling decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverter 322, an adder 340, a filter 350, a memory 360, an inter-prediction module 332, and an intra-prediction module 331. .

[73] The dequantizer 321 may dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 321 may re-arrange the quantized transform coefficients into a two-dimensional block form. In this case, re-arrangement may be performed based on the order of coefficient scanning performed in the encoding equipment. The dequantizer 321 may perform dequantization on the quantized transform coefficients by using a quantization parameter (eg, quantization step size information) and obtain the transform coefficients.

[74] The inverse converter 322 inversely converts the transform coefficients to obtain a residual signal (residual block, residual sample array).

[75] Prediction module 230 may perform prediction for the current block and may generate a predicted block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied to the current block based on the prediction information output from the entropy decoder 310, and may determine a particular intra/inter prediction mode.

[76] Prediction module 320 may generate a prediction signal based on various prediction methods described below. For example, the prediction module may not only apply intra-prediction or inter-prediction to predict one block, but also simultaneously apply intra-prediction and inter-prediction. This may be referred to as "combined inter-intraprediction (CIIP)". In addition, the prediction module can be based on intra-block copy (IBC) prediction mode or palette mode for block prediction. The IBC prediction mode or palette mode can be used for encoding image/video content of a game and the like, for example, screen content encoding (SCC). IBC essentially performs prediction in the current frame, but can be performed similarly to inter-prediction in which the reference block is retrieved in the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described in the present disclosure. Palette mode can be considered an example of intra-coding or intra-prediction. When palette mode is applied, a sampled value in a frame may be signaled based on information regarding the palette table and palette index.

[77] The intra prediction module 331 may predict the current block by reference to the samples in the current frame. Samples for reference may be located in the surroundings of the current block or may be located independently according to the prediction mode. In intraprediction, prediction modes may include a plurality of non-directional modes and a plurality of directed modes. The intra prediction unit 331 may determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[78] The inter-prediction unit 332 may extract a predicted block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0-prediction, L1-prediction, bi-prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block present in the current frame and a temporal neighbor block present in the reference frame. For example, the inter-prediction module 332 may configure a motion information candidate list based on neighboring blocks and extract the current block's motion vector and/or reference frame index based on the received candidate selection information. The inter-prediction may be performed based on various prediction modes, and the information regarding the prediction may include information indicating the inter-prediction mode for the current block.

[79] The adder 340 may generate a reconstructed signal (recovered frame, reconstructed block, reconstructed sample array) by adding the resulting residual signal with a prediction signal (predicted block, predicted sample array) output from a prediction module (including inter-prediction module 332 and /or internal prediction module 331). If there is no remainder for a block to be processed, for example when a skip mode is applied, the predicted block can be used as the reconstructed block.

[80] The adder 340 may be referred to as a “recovery module” or a “recovered block generator.” The generated reconstructed signal may be used to internally predict the next block to be processed in the current frame, may be output by filtering as described below, or may be used to inter-predict the next frame.

[81] Filter 350 can enhance the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, filter 350 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and store the modified reconstructed frame in storage device 360, namely, a DPB of storage device 360. Various filtering techniques may include, for example, deblocking filtering. , sampled adaptive bias, adaptive contour filter, bilateral filter, etc.

[82] The (modified) reconstructed frame stored in the DPB of the storage device 360 can be used as a reference frame in the inter-prediction unit 332. Storage device 360 may store block motion information from which motion information in the current frame is retrieved (or decoded) and/or motion information of blocks in a frame that have already been reconstructed. The stored motion information may be transmitted to the inter-prediction unit 260 so that it is used as spatial neighbor block motion information or temporal neighbor block motion information. Storage device 360 may store reconstructed samples of reconstructed blocks in the current frame and transmit the reconstructed samples to intra prediction module 331.

[83] Herein, the embodiments described in the filter 260, the inter-prediction module 221, and the intra-prediction module 222 of the encoding equipment 200 may be identical or appropriately applied so that they correspond to the filter 350, the inter-prediction module 332, and the module 331 internal prediction of the decoding equipment 300. The same may also apply to module 332 and internal prediction module 331.

[84] As described above, when encoding video, prediction is performed in order to improve compression efficiency. Through this, a predicted block can be generated including predictive samples for the current block, which is the block to be encoded. Here, the predicted block includes predictive samples in a spatial domain (or pixel domain). The predicted block is retrieved equally from the encoding apparatus and the decoding apparatus, and the encoding apparatus decodes information (residual information) regarding the remainder between the original block and the predicted block, rather than the original sample value of the original block itself. By transmitting overhead signals to the device, the efficiency of image encoding can be increased. The decoding equipment may extract a residual block including residual samples based on the residual information, and generate a reconstructed block including the reconstructed samples by adding the residual block with the predicted block, and generate a reconstructed frame including the reconstructed blocks.

[85] Residual information can be generated through transformation and quantization processes. For example, the encoding equipment may extract a residual block between an original block and a predicted block, and perform a transform process on the residual samples (an array of residual samples) included in the residual block to extract transform coefficients, and then, by performing a quantization process on the transform coefficients, extract quantized transform coefficients to signal the residual associated information to the decoding equipment (via the bitstream). Here, the residual information may include location information, transformation technology, transformation kernel and quantization parameter, quantized transformation coefficient value information, etc. The decoding equipment may perform a dequantization/deconversion process based on the residual information and extract the residual samples (or residual blocks). The decoding equipment can generate a reconstructed image based on the predicted block and the residual block. The encoding equipment may also dequantize/de-quantize the quantized transform coefficients for the inter-prediction reference of a later frame to extract the residual block and generate a reconstructed frame based on it. Herein, at least one of quantize/dequantize and/or transform/de-transform may be omitted. When quantization/dequantization is omitted, the quantized conversion factor may be referred to as a "transformation factor". When the transform/inverse transform is omitted, the transform coefficients may be called "coefficients" or "residual coefficients" or may still be called "transform coefficients" for consistency of expression. Herein, the quantized transform coefficient and the transform coefficient may be referred to as the “transform coefficient” and the “scaled transform coefficient,” respectively. In this case, the residual information may include information regarding the transform coefficient(s), and the information regarding the transform coefficient(s) may be signaled through a residual encoding syntax. Transform coefficients can be extracted based on residual information (or information regarding the transform coefficient(s)), and scaled transform coefficients can be extracted via inverse transform (scaling) of the transform coefficients. Residual samples can be extracted based on the inverse transform (transform) of the scaled transform coefficients. This may also be applied/expressed elsewhere in this document.

[86] Intra prediction may refer to prediction that generates prediction samples for the current block based on reference samples in the frame to which the current block belongs (hereinafter referred to as the “current frame”). When intra prediction is applied to the current block, neighboring reference samples that should be used for intra prediction of the current block can be retrieved. The current block's adjacent reference samples may include samples adjacent to the left border of the current block having size nWxnH and a total of 2xnH samples bordering the bottom left, samples adjacent to the top border of the current block, and a total of 2xnW samples , bordering the top right part, and one selection bordering the top left part relative to the current block. Alternatively, the neighboring reference samples of the current block may include a plurality of top neighboring samples and a plurality of left neighboring samples. In addition, the adjacent reference samples of the current block may include a total of nH samples adjacent to the right border relative to the current block having size nWxnH, a total of nW samples adjacent to the lower border relative to the current block, and one sample adjacent to the lower right relative to current block.

[87] However, some adjacent reference samples of the current block may not yet be decoded or available. In this case, the decoder can configure adjacent reference samples to be used for prediction by substituting available samples for samples that are not available. Alternatively, neighboring reference samples to be used for prediction can be configured through interpolation of the available samples.

[88] When neighboring reference samples are retrieved, (i) the prediction sample may be retrieved based on the average or interpolation of the current block's neighboring reference samples, and (ii) the prediction sample may be retrieved based on the reference sample present in a particular (prediction) direction for the prediction sample. from peripheral reference samples of the current block. Case (i) may be referred to as "non-directional mode" or "non-angular mode", and case (ii) may be referred to as "directional mode" or "angular mode".

[89] In addition, the prediction sample may also be generated through interpolation between the second adjacent sample and the first adjacent sample located in the opposite direction to the prediction direction of the intra prediction mode of the current block based on the prediction sample of the current block from the adjacent reference samples. The above case may be called "linear interpolation intraprediction (LIP)". In addition, chroma prediction samples can be generated from the luminance samples using a linear model. This case may be called "LM mode".

[90] In addition, the temporary prediction sample of the current block may be extracted based on the filtered neighboring reference samples and at least one reference sample extracted according to the intra prediction mode from the existing neighboring reference samples, i.e. the unfiltered neighboring reference samples and the temporary prediction sample can be summed with weighting to extract the current block's prediction sample. The above case may be called "position-dependent intraprediction (PDPC)".

[91] In addition, a reference sample line having the highest prediction accuracy among adjacent multiple reference sample lines of the current block may be selected to extract a prediction sample by using a reference sample located in the prediction direction on the corresponding line, and then a reference sample line used herein, may be indicated (signalled) to the decoding equipment, thereby performing internal predictive coding. The above-described case may be called "multiple reference line (MRL) intra-prediction" or "MRL-based intra-prediction".

[92] In addition, intra prediction can be performed based on the same intra prediction mode by dividing the current block into vertical or horizontal sub-segments, and adjacent reference samples can be retrieved and used in a sub-segment unit. Thus, in this case, the intra-prediction mode for the current block is equally applied to the sub-segments, and the intra-prediction performance can be improved in some cases by retrieving and using adjacent reference samples in a unit of sub-segments. This forecasting method may be referred to as "internal subsegment-based (ISP) or ISP-based forecasting."

[93] The above-described intra prediction methods may be called "intra prediction type" separately from the intra prediction mode. The intra-prediction type may be referred to by various terms such as "intra-prediction technology" or "additional intra-prediction mode". For example, the intra prediction type (or additional intra prediction mode) may include at least one of the above-described LIP, PDPC, MRL, and ISP. The general intra-prediction method, excluding a specific intra-prediction type such as LIP, PDPC, MRL or ISP, may be referred to as a “normal intra-prediction type”. The normal intra-prediction type can generally be applied when a specific intra-prediction type is not applied, and prediction can be performed based on the intra-prediction mode described above. Meanwhile, post-filtering can be performed on the predicted sample, retrieved as needed.

[94] In particular, the intra prediction procedure may include a step of determining an intra prediction mode/type, a step of retrieving neighboring reference samples, and a step of retrieving prediction samples based on the intra prediction mode/type. In addition, a post-filtering step can be performed on the predicted sample, extracted as needed.

[95] When intra prediction is applied, the intra prediction mode applied to the current block can be determined using the intra prediction mode of the adjacent block. For example, the decoding equipment may select one of the possible most probable mode (MPM) options of an MPM list retrieved based on the intra prediction mode of an adjacent block (e.g., left and/or upper adjacent blocks) relative to the current block based on the received MPM index and select one of the other remaining input prediction modes not included in the MPM options (and the planar mode), based on information from the remaining intra prediction mode. The MPM list may be configured to include or not include planar mode as an option. For example, if the MPM list includes planar mode as a possible option, the MPM list may have six possible options. If the MPM list does not include planar mode as an option, the MPM list may have three options. When the MPM list does not include planar mode as an option, a non-planar flag (eg, intra_luma_not_planar_flag) indicating whether the current block's intraprediction mode is not planar mode may be signaled. For example, the MPM flag may be signaled first, and the MPM index and the non-planar flag may be signaled when the value of the MPM flag is 1. In addition, the MPM index may be signaled when the value of the non-planar flag is 1. Here, the MPM list is configured to not include the planar mode, since the option is not to signal the non-planar flag first to first check whether it is a planar mode or not, since the planar mode is always considered as MPM.

[96] For example, whether the intra prediction mode applied to the current block is in the MPM candidates (and planar mode) or in the remaining mode can be indicated based on the MPM flag (eg, intra_luma_mpm_flag). An MPM flag value of 1 may indicate that the intra-prediction mode for the current block is in the possible MPM options (and in planar mode), and a MPM flag value of 0 may indicate that the intra-prediction mode for the current block is not in the possible MPM options. MPM variants (and planar mode). A non-planar flag value of 0 (eg, intra_luma_not_planar_flag) may indicate that the intra prediction mode for the current block is a planar mode, and a non-planar flag value of 1 may indicate that the intra prediction mode for the current block is not a planar mode. The MPM index may be signaled in the form of an mpm_idx or intra_luma_mpm_idx syntax element, and the remaining intra prediction mode information may be signaled in the form of a rem_intra_luma_pred_mode or intra_luma_mpm_remainder syntax element. For example, the remaining intra prediction mode information may index the remaining inter prediction modes not included in the MPM candidates (and the planar mode) from all intra prediction modes in order of prediction mode number to indicate one of them. The intra prediction mode may be an intra prediction mode for the luminance signal component (sample). Hereinafter herein, the intra prediction mode information may include at least one of an MPM flag (eg, intra_luma_mpm_flag), a non-planar flag (eg, Intra_luma_not_planar_flag), an MPM index (eg, mpm_idx or intra_luma_mpm_idx) and information remaining internal prediction mode (rem_intra_luma_pred_mode or intra_luma_mpm_remainder). As used herein, an MPM list may be referred to by various terms such as "MPM candidate list" and "candModeList". When a MIP is applied to the current block, a separate MPM flag (eg, intra_mip_mpm_flag), MPM index (eg, intra_mip_mpm_idx), and intra prediction remaining mode information (eg, intra_mip_mpm_remainder) for the MIP may be signaled, and the non-planar flag is not sent to service signals.

[97] In other words, in general, when block partitioning is performed on an image, the current block and the adjacent block to be encoded have similar image characteristics. Therefore, the current block and the neighboring block have a high probability of having an identical or similar intra-prediction mode. Thus, the encoder can use the intra-prediction mode of an adjacent block to encode the intra-prediction mode of the current block.

[98] For example, the encoder/decoder may configure a list of most probable modes (MPM) for the current block. An MPM list may also be referred to as an "MPM candidate list". As used herein, MPM may refer to a mode used to improve coding efficiency by taking into account the similarity between a current block and a neighboring block when encoding an intra prediction mode. As described above, the MPM list may be configured to include a planar mode or may be configured to exclude a planar mode. For example, when the MPM list includes planar mode, the number of possible options in the MPM list may be 6. Also, if the MPM list does not include planar mode, the number of possible options in the MPM list can be 5.

[99] The encoder/decoder can configure an MPM list that includes 5 or 6 MPM.

[100] To configure the MPM list, three types of modes can be considered: default internal modes, adjacent internal modes, and extracted internal modes.

[101] For adjacent internal modes, two adjacent blocks may be considered, i.e. left adjacent block and top adjacent block.

[102] As described above, if the MPM list is configured not to include a planar mode, the planar mode is excluded from the list, and the number of possible MPM list options can be set to 5.

[103] In addition, the non-directional mode (or non-angular mode) of the intra prediction modes may include a DC mode based on the average of neighboring reference samples of the current block or a planar mode based on interpolation.

[104] When inter prediction is applied, the encoding equipment/decoding equipment prediction module can extract the prediction sample by performing inter prediction in units of blocks. The inter-prediction may be a prediction derived in a manner that depends on data elements (eg, sample values or motion information) of frame(s) other than the current frame. When inter-prediction is applied to the current block, a predicted block (prediction sample array) for the current block can be retrieved based on a reference block (reference sample array) indicated by a motion vector on a reference frame indicated by a reference frame index. Here, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information of the current block may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction type information (L0 prediction, L1 prediction, bi-prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block present in the current frame and a temporal neighbor block present in the reference frame. The reference frame including the reference block and the reference frame including the temporary adjacent block may be identical or different. A temporary neighbor block may be called a "co-located reference block", a "co-located CU (colCU)", etc., and a reference frame including a temporary neighboring block may be called a "col-located frame (colPic)". For example, a list of motion information candidates may be configured based on neighboring blocks of the current block, and flag or index information indicating which candidate is selected (used) may be signaled to retrieve a motion vector and/or reference frame index current block. Mutual prediction can be performed based on different prediction modes. For example, in the case of the skip mode and the merge mode, the motion information of the current block may be identical to the motion information of the adjacent block. In skip mode, unlike combine mode, the residual signal may not be transmitted. In the case of a motion vector prediction (MVP) mode, the motion vector of a selected neighboring block can be used as a motion vector predictor, and the motion vector of the current block can be signaled. In this case, the motion vector of the current block can be extracted using the sum of the motion vector predictor and the motion vector difference.

[105] The motion information may include L0 motion information and/or L1 motion information according to a type of inter-prediction (L0 prediction, L1 prediction, bi-prediction, etc.). The motion vector in the L0 direction may be referred to as the "L0 motion vector" or "MVL0", and the motion vector in the L1 direction may be referred to as the "L1 motion vector" or "MVL1". Prediction based on L0 motion vector may be referred to as “L0 prediction,” prediction based on L1 motion vector may be referred to as “L1 prediction,” and prediction based on both L0 motion vector and L1 motion vector may be referred to as “biprediction.” ". Here, the L0 motion vector may indicate a motion vector associated with a reference frame list (L0) L0, and the L1 motion vector may indicate a motion vector associated with a reference frame list (L1) L1. The reference frame list L0 may include frames that are earlier in output order than the current frame as reference frames, and the reference frame list L1 may include frames that are later in output order than the current frame. Previous frames may be referred to as "forward (reference) frames" and subsequent frames may be referred to as "reverse (reference) frames". The reference frame list L0 may further include frames that are later in the output order than the current frame as reference frames. In this case, previous frames may be indexed first in the reference frame list L0, and subsequent frames may be indexed later. The reference frame list L1 may further include previous frames in output order relative to the current frame as reference frames. In this case, subsequent frames may be indexed first in reference frame list 1, and previous frames may be indexed later. The order of output may correspond to the order of numbers in the sequence of frames (POC).

[106] FIG. 4 is a drawing for explaining a method for extracting motion information through the inter-prediction mode. Here, from the mutual prediction modes, the fusion mode, the motion vector prediction (MVP) mode (or advanced motion vector prediction (AMVP)), the pairwise average pooling mode, and the history-based MVP (HMVP) mode are described below.

[107] When the combining mode is applied, the motion information of the current prediction block is not directly transmitted, and the motion information of the current prediction block is extracted using the motion information of the adjacent prediction block. Therefore, motion information of the current prediction block may be indicated by transmitting flag information indicating that the combining mode is used and a combining index indicating which neighboring prediction blocks are being used.

[108] The encoder must search for a candidate merging block used to extract motion information of the current prediction block to perform the merging mode. For example, up to five possible merge block options may be used, but this document is not limited to this. A maximum number of possible merge block options may be carried in a slice header, a tile group header, or a parameter set (ie, a sequence parameter set (SPS)), and this document is not limited to this. After finding the combiner candidate candidates, the encoder may generate a list of combiner candidate candidates and select the combiner candidate candidate that has the least cost among them as the final combiner candidate candidate.

[109] This document provides various embodiments of an exemplary merge block configuring a list of possible merge options.

[110] The merge candidate list may use, for example, five possible merge block candidates. For example, four spatial join candidates and one temporal join candidate may be used. As a specific example, in the case of a spatial combining option, the blocks shown in FIG. 4 can be used as a spatial merging option.

[111] The list of possible combinations for the current block can be configured, for example, based on the following procedure.

[112] The encoding equipment (encoder/decoder) inserts the spatial combining candidates extracted by searching the spatial neighbors of the current block into the combining candidate list (S310). For example, spatial neighbors may include a lower left corner neighbor, a left neighbor, a right upper corner neighbor, a top neighbor, and a left upper corner neighbor relative to the current block. However, this is merely an example, and in addition to the above-described spatial neighbor blocks, additional neighboring blocks such as a right neighbor block, a lower neighbor block, and a right lower neighbor block may be used as spatial neighbor blocks. The encoding equipment may perform a priority-based spatial neighbor search to detect available blocks, and extract motion information of the detected blocks as spatial combining candidates. For example, the encoder and decoder may search the five blocks shown in FIG. 1.3.2-1 in the order A1, B1, B0, A0, B2, and sequentially index the available candidates to configure the list of possible join options.

[113] The encoding equipment inserts a temporary combining candidate retrieved by performing a search in a temporary adjacent block of the current block into the combining candidate list (S320). The temporary neighbor block may be located in the reference frame, i.e. in a frame different from the current frame in which the current block is located. The reference frame in which the temporary adjacent block is located may be referred to as a "co-located frame" or "co-located frame". You can search for a temporary neighbor block in order from the bottom right corner neighbor block and the bottom right center block relative to the co-located block for the current block in the co-location. frame. Meanwhile, when motion data compression is applied, specific motion information can be stored as characteristic motion information for each predetermined storage unit in the co-location. frame. In this case, it is not necessary to store motion information for all blocks in a predetermined storage unit, thereby obtaining the advantage of compressing motion data. In this case, the predetermined storage unit may be predetermined, for example, in 16×16 sampling units, 8×8 sampling units, etc., or size information regarding the predetermined storage unit may be signaled from the encoder to the decoder. When motion data compression is applied, motion information of a temporary neighbor block may be replaced with characteristic motion information of a predetermined storage unit in which the temporary neighbor block is located. Thus, in this case, from an implementation point of view, the predetermined value is arithmetically shifted to the right based on the coordinates (upper left sample position) of the temporary adjacent block, and thereafter, the temporary merging candidate can be retrieved based on the motion information of the prediction block covering the arithmetically position shifted to the left. For example, in the case where a sampling unit having a predetermined storage unit is 2nx2n, if the coordinates of the temporary neighboring block are (xTnb, yTnb), the motion information of the prediction block located at the modified position ((xTnb>>n)<<n ), (yTnb>>n)<<n)). Particularly, for example, in the case where the predetermined storage unit is a 16×16 sampling unit, if the coordinates of the temporary neighboring block are (xTnb, yTnb), the motion information of the prediction block located at the modified position ((xTnb>>4)<< 4), (yTnb>>4)<<4)), can be used for a temporary possible merging option. Alternatively, for example, in the case where the predetermined storage unit is an 8×8 sample unit, if the coordinates of the temporary neighboring block are (xTnb, yTnb), the motion information of the prediction block located at the modified position ((xTnb>>3)<<3 ), (yTnb>>3)<<3)), can be used for a temporary possible merging option.

[114] The encoding equipment can determine whether or not the number of current combination possibilities is less than the maximum number of combination possibilities (the maximum number of possibilities included in the combination candidate list) (S330). The maximum number of possible joins may be predefined or signaled from the encoder to the decoder (ie, through the tile group header or SPS). For example, the encoder may generate information regarding the maximum number of possible combinations (the maximum number of possible combinations included in the list of possible combinations), encode the information, and transmit the encoded information to the decoder in the form of a bitstream. If the maximum number of candidate joins is filled, the subsequent process of adding a candidate may not occur.

[115] As a result of the check, if the number of current combining candidates is less than the maximum number of combining candidates, the encoding equipment inserts an additional combining candidate into the combining candidate list (S340). Additional merging candidate may include, for example, ATMVP and a combined bipredictive merging candidate (when the slice/tile type of the current slice/tile group is B) and/or a null vector merging candidate.

[116] As a result of the check, if the number of current possible combinations is not less than the number of maximum possible combinations, the encoding equipment may complete constructing the list of possible combinations. In this case, the encoder may select the optimal combining candidate from the combining candidate options configuring the list of combining candidate candidates based on a rate-dependent distortion cost function (RD) and signal selection information (eg, combining index) indicating the selected candidate combination option for the decoder. The decoder may select the optimal combining candidate based on the list of combining candidate candidates and the selection information.

[117] As described above, the motion information of the selected combining candidate can be used as the motion information of the current block, and the prediction samples of the current block can be extracted based on the motion information of the current block. The encoder may extract residual samples of the current block based on the predicted samples and may signal residual information regarding the residual samples to the decoder. The decoder may generate reconstructed samples based on the residual samples and predicted samples extracted based on the residual information, and generate a reconstructed frame based on it, as described above.

[118] When the skip mode is applied, motion information of the current block can be extracted in the same manner as motion information for the case in which the combining mode is applied. However, when the skip mode is applied, the residual signal for the corresponding block is omitted, and therefore the predicted samples can be used as reconstructed samples.

[119] When the motion vector prediction (MVP) mode is applied, a list of motion vector predictor (MVP) candidate candidates may be generated using the motion vector of the reconstructed spatial neighbor block (which may be the neighbor block of FIG. 4) and/or the motion vector of the temporal neighbor block. adjacent block (or Col block). Thus, a motion vector corresponding to a reconstructed spatial neighbor block and/or a motion vector corresponding to a temporal neighbor block may be used as a candidate motion vector predictor. When bi-prediction is applied, a list of candidate MVPs for extracting L0 motion information and a list of candidate MVPs for extracting L1 motion information can be generated and used separately. The above-described prediction information (or information regarding prediction) may include selection information (eg, an MVP flag or an MVP index) indicating an optimal motion vector predictor candidate selected from motion vector predictor candidates included in the list. In this case, the prediction module may select a motion vector predictor of the current block from motion vector predictor candidates included in the motion vector candidate list using the selection information. The encoding hardware prediction module may obtain a motion vector difference (MVD) between the motion vector of the current block and the motion vector predictor, encode it, and output it in the form of a bitstream. Thus, the MVD can be obtained as a value obtained by subtracting the motion vector predictor from the motion vector of the current block. In this case, the prediction unit of the decoding equipment can obtain the motion vector difference included in the prediction information and extract the motion vector of the current block by summing the motion vector difference and the motion vector predictor. The decoding equipment prediction module may obtain or extract a reference frame index indicating a reference frame from the prediction information.

[120] Pool candidates for history-based MVP (HMVP) can be added to the pool list after the spatial MVP and TMVP. In this method, the motion information of a previously encoded block is stored in a table and used as an MVP for the current CU. A table with several possible HMVP variants is maintained during the encoding/decoding process. The table is reset (emptied) when a new CTU row is encountered. Each time there is a non-subblock mutually encoded CU, the associated motion information is added to the last table entry as a new HMVP candidate.

[121] The size S of the HMVP table is set to 6, indicating that up to 6 possible history-based MVPs (HMVPs) can be added to the table. When a new motion candidate is inserted into the table, a constrained first-in, first-out (FIFO) rule is used, with redundancy checking first applied to detect whether or not an identical HMVP exists in the table. If found, an identical HMVP is removed from the table and all possible HMVP variants after it are moved forward.

[122] HMVP candidates can be used in the process of constructing lists of candidate combinations. The last few possible HMVP options in the table are checked in order and inserted into the list of possible options after the possible TMVP option. Redundancy control is applied to HMVP candidates relative to the spatial or temporal aggregation candidate.

[123] To reduce the number of redundancy checks, the following simplifications are introduced:

[124] 1) The number of possible HMPV variants used to generate lists for merging is given as (N<=4)? M: (8-N), where N indicates the number of existing candidates in the list to be merged, and M indicates the number of available HMVP candidates in the table.

[125] 2) Once the total number of available join options reaches the maximum allowed join options minus 1, the process of constructing join candidate lists from the HMVP is completed.

[126] Pairwise candidate averages are generated by averaging predefined candidate pairs in an existing pool candidate list, and the predefined candidate pairs are specified as {(0, 1), (0, 2), (1, 2), (0, 3), (1, 3), (2, 3)}, where the numbers indicate the indices of combining into a list of possible combining options. Average motion vectors are calculated separately for each reference list. If both motion vectors are available in the same list, the two motion vectors are averaged even when they point to different reference frames; if only one motion vector is available, it should be used directly; if the motion vector is not available, this list should be kept invalid.

[127] When the list to join is not complete after pairwise averages of possible joins are added, zero MVPs are inserted into the list until the maximum number of possible joins is encountered.

[128] Later in this document, a detailed description is given of the IBC that can be performed by the hardware prediction module of FIG. 2 or fig. 3. IBC can be used to encode game image/video content, etc., such as screen content encoding (SCC). IBC essentially performs prediction in the current frame, but can be performed similarly to inter-prediction in which the reference block is retrieved in the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described herein. For example, the IBC may use at least one of the above methods to extract motion information (motion vector). The IBC may refer to the current frame and may therefore be called a "Current Frame Reference (CPR)". When the above-described prediction methods are applied in IBC, the motion vector (motion information) may generally be called (replaced or mixed) as a “block vector (block information).”

[129] For IBC, the encoding hardware may perform block matching (BM) to extract the optimal block vector (or motion vector) for the current block (eg, CU). The extracted block vector (or motion vector) may be signaled to the decoding equipment via a bit stream using a method similar to the signaling of inter-prediction block information (motion vector) described above. The decoding equipment may extract a reference block for the current block in the current frame via a signaled block vector (motion vector), thereby triggering a prediction signal (predicted block or predicted samples) for the current block. Here, a block vector (or motion vector) may represent an offset from the current block to a reference block located in an area already reconstructed in the current frame. Thus, the block vector (or motion vector) may be called a "displacement vector". Later in this document, in IBC, the motion vector may correspond to a block vector or a displacement vector. The motion vector of the current block may include a motion vector for the luma signal component (luminance motion vector) or a motion vector for the chrominance signal component (chrominance motion vector). For example, the luma motion vector for an IBC-coded CU may be an integer unit of samples (ie, integer precision). Chroma motion vectors can also be clipped in integer sample units. As mentioned above, the IBC may use at least one of the inter-prediction technologies, for example, 1-pel and 4-pel motion vector accuracies can be switched when an IBC, eg AMVR, is applied.

[130] To reduce memory consumption and decoder complexity, only the reconstructed portion of the predefined area including the current CTU can be used. This limitation ensures that IBC mode can be implemented using local on-chip storage for hardware implementations.

[131] On the encoder side, hash-based motion estimation is performed for IBC. The encoder performs RD testing for blocks with a width or height of no more than 16 luminance samples. For non-joining mode, block vector search is performed using hash-based search first. If the hash search does not return a valid candidate, then a local search must be performed based on block-by-block matching.

[132] In a hash-based search, the hash key mapping (32-bit CRC) between the current block and the reference block is extended to all allowed block sizes. The calculation of hash keys for each position in the current frame is based on 4x4 subblocks. For the current larger block, a hash key is determined to match the hash key of the reference block when all hash keys of all 4x4 subblocks match the hash keys at the corresponding reference locations. If the hash keys of multiple reference blocks are found to match the hash key of the current block, the costs in the form of block vectors of each matching link are calculated, and the block vector with the minimum cost is selected.

[133] In block-based search, the search range is set to N samples left and up relative to the current block in the current CTU. At the beginning of the CTU, the value of N is initialized to 128 if there is no temporary reference frame, and is initialized to 64 if there is at least one temporary reference frame. The hash hit percentage is defined as the percentage of samples in the CTU that finds a match using a hash-based search. When encoding the current CTU, if the percentage of hash matches is below 5%, N is halved.

[134] At the CU level, IBC mode is signaled with flag information (i.e., flag information indicating whether or not IBC is applied, pred_mode_ibc_flag), and it may be signaled as IBC AMVP mode or IBC skip/merge mode as follows.

[135] - IBC Skip/Merge Mode: The merge candidate index is used to indicate which of the block vectors in the list of adjacent IBC coded block candidates are used to predict the current block. The list to merge consists of spatial, HMVP, and pairwise candidates.

[136] - IBC AMVP mode: the difference in block vectors is encoded similarly to the difference in motion vectors. The block vector prediction method uses two candidates as predictors, one from the left neighbor and one from the upper neighbor (in the case of IBC encoding). When any neighboring node is not available, the default block vector is used as a predictor. The flag is signaled to indicate the block vector predictor index.

[137] FIG. 5 and fig. 6 illustrates an image encoding method based on an IBC prediction mode and an encoding equipment prediction unit implementing the image encoding method.

[138] The encoding equipment can extract the prediction mode and motion vector of the current block and generate prediction samples of the current block. The prediction mode may include at least one of the inter-prediction modes described above. Here, prediction mode determination, motion vector extraction, and prediction sampling procedure may be performed simultaneously, or one procedure may be performed before the other. For example, the encoding equipment prediction module may include a prediction mode determining module, a motion vector extracting module, and a prediction sample extracting module. The prediction mode determining unit may determine the prediction mode for the current block, the motion vector extracting unit may extract the motion vector of the current block, and the prediction sample extracting unit 183 may extract the prediction samples of the current block. As described above, the motion vector may be referred to as a "block vector". For example, the encoding hardware prediction module may search for a block similar to the current block in the reconstructed area (or in a predetermined area (search area)) through block matching (BM), and retrieve a reference block whose difference with the current block is at least a certain amount. reference value or less. The prediction module can extract a motion vector based on the displacement difference between the reference block and the current block. The encoding equipment may determine the mode applied to the current block from various prediction modes. The encoding hardware can compare RD costs based on different prediction modes and determine the optimal prediction mode for the current block.

[139] For example, when a skip mode or a merge mode is applied to a current block, the encoding equipment may configure the list of possible merge options described above and extract a reference block having a difference with respect to the current block equal to or less than a minimum or predetermined criterion. In this case, a combine candidate associated with the extracted reference block can be selected, and combine index information indicating the selected combine candidate can be generated and signaled to the decoding equipment. The motion vector of the current block may be extracted using the motion vector of the selected fusion candidate.

[140] As another example, when the (A)MVP mode is applied to the current block, the encoding equipment may configure the list of (A)MVP candidates described above and use the motion vector of the MVP candidate selected from among the candidates MVPs (motion vector predictors) included in the list of possible (A)MVP options as the MVP of the current block. In this case, for example, a motion vector indicating a reference block extracted by the above-described motion estimation can be used as a motion vector of the current block, and a possible MVP variant having a motion vector whose difference with respect to the motion vector of the current block is among the possible MVPs. options, is the smallest, may represent the selected possible MVP option. A motion vector difference (MVP), which is the difference from which the MVP is subtracted, can be extracted from the motion vector of the current block. In this case, information regarding the MVD may be signaled to the decoding equipment.

[141] The encoding equipment may extract residual samples based on the predictive samples (S510). The encoding hardware may extract residual samples by comparing the original samples of the current block with the predicted samples.

[142] The encoding equipment encodes image information including prediction information and residual information (S520). The encoding equipment may output the encoded image information in the form of a bit stream. The prediction information may include a prediction mode vector (ie, a skip flag, a combine flag, or a mode index) and information regarding a motion vector as information associated with a prediction procedure. The information regarding the motion vector may include candidate selection information (eg, a pooling index, an MVP flag, or an MVP index), which is information for extracting the motion vector. In addition, information regarding the motion vector may include information regarding the MVD described above. Information regarding the motion vector may include information indicating whether L0 prediction, L1 prediction, or bi-prediction is applied. Residual information is information regarding the residual samples. The residual information may include information regarding quantized transform coefficients for the residual samples.

[143] The output bitstream may be stored on a (digital) storage medium and delivered to the decoding equipment, or may be delivered to the decoding equipment via a network.

[144] Meanwhile, as described above, the encoding equipment can generate a reconstructed frame (including reconstructed samples and a reconstructed block) based on the reference samples and residual samples. This is because the encoding equipment must extract a prediction result identical to the prediction result performed in the decoding equipment, and therefore the encoding efficiency can be improved. Therefore, the encoding equipment can store the reconstructed frame (or reconstructed samples, reconstructed block) in a storage device and use it as a reference frame for inter-prediction.

[145] FIG. 7 and fig. 8 illustrates an image decoding method based on the IBC prediction mode and a decoding equipment prediction unit implementing the image decoding method.

[146] The decoding equipment may perform an operation corresponding to an operation performed in the encoding equipment. The decoding equipment may perform IBC prediction for the current block based on the received prediction information and extract prediction samples.

[147] Specifically, the decoding equipment may determine a prediction mode for the current block based on the received prediction information (S700). The decoding equipment may determine which inter-prediction mode is applied to the current block based on the prediction mode information in the prediction information.

[148] For example, the decoding equipment may determine whether or not the combining mode is applied to the current block, or whether or not the (A)MVP mode is determined, based on the combining flag. Alternatively, one of the various possible inter-prediction mode options may be selected based on the mode index. Options for the inter-prediction mode may include a skip mode, an aggregation mode, and/or (A)MVP mode, or may include the various inter-prediction modes described in FIG. 4.

[149] The decoding equipment extracts the motion vector of the current block based on the determined prediction mode (S710). As described above, the motion vector may be referred to as a "block vector". For example, when a skip mode or a combine mode is applied to the current block, the decoding equipment may configure the list of combine candidates as described above, and select one of the combine candidates included in the list of combine candidates. The selection may be made based on the above selection information (merging index). The motion vector of the current block may be extracted using the motion vector of the selected fusion candidate. The motion vector of the selected merge candidate can be used as the motion vector of the current block.

[150] As another example, when the (A)MVP mode is applied to the current block, the decoding equipment may configure a list of possible (A)MVP options as described above, and use the motion vector of the MVP candidate selected from among the possible MVP options included in the list of possible (A)MVP options as the MVP of the current block. The selection may be made based on the selection information (MVP flag or MVP index) described above. In this case, the MVD of the current block can be extracted based on information regarding the MVD, and the motion vector of the current block can be extracted based on the MVP and MVD of the current block. In addition, the reference frame index of the current block may be retrieved based on the reference frame index information. The frame indicated by the reference frame index in the list of reference frames for the current block may be retrieved as a reference frame referred to for inter-prediction of the current block.

[151] Meanwhile, as described above, the motion vector of the current block can be retrieved without configuring the candidate list, in which case, the motion vector of the current block can be retrieved according to the procedure disclosed in the corresponding prediction mode. In this case, configuring the list of possible options as described above may be omitted.

[152] The decoding equipment may generate predictive samples for the current block based on a motion vector for the current block (S720). The decoding equipment may extract predicted samples of the current block using samples of a reference block indicated by the motion vector of the current block in the current frame. In this case, the prediction sample filtering procedure may further be performed on all or some of the prediction samples of the current block.

[153] For example, the decoding equipment prediction module may include a prediction mode determining module, a motion vector extracting module, and a prediction sample extracting module. The prediction mode for the current block may be determined based on a prediction mode vector received from the prediction mode determining unit, the motion vector of the current block may be extracted based on information regarding the motion vector received from the motion vector extracting unit 262, and the prediction sample extracting unit may extract prediction samples. current block.

[154] The decoding equipment generates residual samples for the current block based on the received residual information (S730). The decoding equipment may generate reconstructed samples for the current block based on the prediction samples and residual samples, and generate a reconstructed frame based on them (S740). After this, the procedure for in-circuit filtration, etc. may additionally be applied to the reconstructed frame as described above.

[155] In the following, embodiments related to the merge mode and the skip mode of the IBC prediction modes are described along with the following tables. In particular, a method is shown for determining the maximum number of possible combination mode options to be retrieved in order to find the motion vector of the current block when IBC is used.

[156] The following table provides sample tile group header syntax.

[157] Tab. 1

[158] In next generation video codec technology, when the tile group type (tile_group_type) is P (unidirectional prediction) or B (bidirectional prediction), the maximum number of total possible merge mode options is specified by the syntax element six_minus_max_num_merge_cand, and the maximum number of possible options The aggregation modes for IBC can be determined using it.

[159] When the tile group type (tile_group_type) is I (internal prediction), the overall merge mode is not present, whereby identical syntax, six_minus_max_num_merge_cand, is passed to specify the maximum number of possible merge mode options for the IBC.

[160] IBC technology is a technology for performing motion compensation in a frame and may have different characteristics compared with the combining mode of existing inter-prediction, and the method for designing candidate combination modes may be different. Therefore, using the same number of possible joins may not be efficient in terms of compression performance.

[161] In one example, the tile group header syntax may be as shown in the following table.

[162] Tab. 2

[163] The semantics of the syntactic elements included in the syntax of Table 2 may be expressed, for example, as shown in the following table.

[164] Tab. 3

- six_minus_max_num_ibc_merge_cand specifies the maximum number of possible merge motion vector prediction (MVP) candidates supported in a tile group, subtracted from 6. The maximum number of possible merge MVP candidates, MaxNumibcMergeCand, is retrieved as follows:

MaxNumibcMergeCand=6-six_minus_max_num_ibc_merge_cand

The MaxNumibcMergeCand value must be between 1 and 6, inclusive.

[165] In another example, the tile group header syntax may be as shown in the following table.

[166] Tab. 4

[167] The semantics of the syntactic elements included in the syntax of Table 4 may be expressed, for example, as shown in the following table.

[168] Tab. 5

- five_minus_max_num_ibc_merge_cand specifies the maximum number of possible merge motion vector predictions (MVPs) supported in a tile group, subtracted from 5. The maximum number of possible merge MVPs, MaxNumibcMergeCand, is retrieved as follows:

MaxNumibcMergeCand=5-five_minus_max_num_ibc_merge_cand

The MaxNumibcMergeCand value must be between 1 and 5, inclusive.

[169] In another example, the tile group header syntax may be as shown in the following table.

[170] Tab. 6

[171] The semantics of the syntactic elements included in the syntax of Table 6 may be expressed, for example, as shown in the following table.

[172] Tab. 7

- four_minus_max_num_ibc_merge_cand specifies the maximum number of possible merge motion vector prediction (MVP) candidates supported in a tile group, subtracted from 4. The maximum number of possible merge MVP candidates, MaxNumibcMergeCand, is retrieved as follows:

MaxNumibcMergeCand=4-four_minus_max_num_ibc_merge_cand

The MaxNumibcMergeCand value must be between 1 and 4, inclusive.

[173] In another example, the tile group header syntax may be as shown in the following table.

[174] Tab. 8

[175] The semantics of the syntactic elements included in the syntax of Table 8 may be expressed, for example, as shown in the following table.

[176] Tab. 9

- three_minus_max_num_ibc_merge_cand specifies the maximum number of possible merge motion vector prediction (MVP) candidates supported in a tile group, subtracted from 3. The maximum number of possible merge MVP candidates, MaxNumibcMergeCand, is retrieved as follows:

MaxNumibcMergeCand=3-three_minus_max_num_ibc_merge_cand

The MaxNumibcMergeCand value must be between 1 and 3, inclusive.

[177] In another example, the tile group header syntax may be as shown in the following table.

[178] Tab. 10

[179] The semantics of the syntactic elements included in the syntax of Table 10 may be expressed, for example, as shown in the following table.

[180] Tab. eleven

- two_minus_max_num_ibc_merge_cand specifies the maximum number of possible merging motion vector prediction (MVP) candidates supported in a tile group, subtracted from 2. The maximum number of possible merging MVP candidates, MaxNumibcMergeCand, is retrieved as follows:

MaxNumibcMergeCand=2-two_minus_max_num_ibc_merge_cand

The MaxNumibcMergeCand value must be between 1 and 2, inclusive.

[181] When the above examples are described in standardization document format, they may be expressed as follows, and the details will be apparent to those skilled in the art.

[182] Tab. 12

8.6.2. Extraction process for motion vector components for IEC blocks

8.6.2.1. General information

The inputs to this process are as follows:

- location (xCb, yCb) of the luminance signal of the upper left sample of the current luminance coding block relative to the upper left sample of the luminance signal of the current frame,

- cbWidth variable indicating the width of the current encoding block in luminance signal samples,

- cbHeight variable indicating the height of the current encoding block in luma samples.

The findings from this process are:

- vector mvL of the brightness signal motion with 1/16 fractional sampling accuracy.

The motion vector mvL of the luminance signal is extracted as follows:

If merge_flag[xCb][yCb] is equal to 1, the extraction process for luma motion vectors for merge mode as specified in section 8.6.2.2 is activated with inputs as luma location (xCb, yCb), variables cbWidth and cbHeight, and output, which is the motion vector mvL of the brightness signal.

Otherwise, the following applies:

1. The mvd variable is retrieved as follows:

mvd[0]=MvdL0[xCb][yCb][0] (8-967)

mvd[1]=MvdL0[xCb][yCb][1] (8-210)

2. The extraction process for predicting luminance signal motion vectors in Section 8.6.2.6 is activated with the location (xCb, yCb) of the luminance signal encoding block, the width cbWidth of the encoding block, and the height cbHeight of the encoding block as inputs, and with the output being the MVP.

3. The motion vector mvL of the luminance signal is extracted as follows:

u[0]=(mvp[0]+mvd[0]+2 ¹⁸ )%2 ¹⁸ (8-968)

mvL[0]=(u[0]>=2 ¹⁷ )?(u[0]-2 ¹⁸ ):u[0] (8-969)

u[1]=(mvp[1]+mvd[1]+2 ¹⁸ )%2 ¹⁸ (8-970)

mvL[l]=(u[l]>=2 ¹⁷ )?(u[1]-2 ¹⁸ ):u[l] (8-971)

Note 1: The resulting values of mvL[0] and mvL[1], as stated above, must always be in the range -2 ¹⁷ to 2 ¹⁷ -1, inclusive.

The update process for the history-based motion vector predictor list, as specified in section 8.6.2.10, is activated with the luminance signal motion vector mvL.

The bitstream compliance requirement is that the motion vector mvL of the luminance signal must be subject to the following constraints:

When the retrieval process for block availability, as specified in Section 6.4.X [Ed. (BB): Process of checking the availability of neighboring blocks, TBD] is activated with the current luminance signal location (xCurr, yCurr) set to (xCb, yCb), and the neighboring luminance signal location (xCb+(mvL[0]>>4), yCb+(mvL[1]>>4)) as inputs and the output must be true.

When the retrieval process for block availability, as specified in Section 6.4.X [Ed. (BB): Process of checking the availability of neighboring blocks, TBD] is activated with the current luminance signal location (xCurr, yCurr) set to (xCb, yCb), and the neighboring luminance signal location (xCb+(mvL[0]>>4) +cbWidth-1, yCb+(mvL[1]>>4)+cbHeight-1) as inputs, and the output should be true.

[183] One or both of the following conditions must be true:

The value (mvL[0]>>4)+cbWidth is less than or equal to 0.

The value (mvL[1]>>4)+cbHeight is less than or equal to 0. The following conditions must evaluate to true:

(yCb+(mvL[1]>>4))>>CtbLog2SizeY=yCb>>CtbLog2SizeY (8-972)

(yCb+(mvL[1]>>4)+cbHeight-1)>>CtbLog2SizeY=yCb>>CtbLog2SizeY (8-973)

(xCb+(mvL[0]>>4))>>CtbLog2SizeY>=(xCb>>CtbLog2SizeY)-1 (8-974)

(xCb+(mvL[0]>>4)+cbWidth-1)>>CtbLog2SizeY<=(xCb>>CtbLog2SizeY) (8-975)

[Ed. (SL): conditions (8-218) and (8-216), possibly checked via 6.4.X]

- When (xCb+(mvL[0]>>4))>>CtbLog2SizeY is equal to (xCb>>CtbLog2SizeY)-1, the retrieval process for block availability as specified in Section 6.4.X [Ed. (BB): Process of checking the availability of neighboring blocks, TBD], is activated with the current location (xCurr, yCurr) of the luminance signal set to (xCb, yCb), and the neighboring location (((xCb+(mvL[0]>>4) +CtbSizeY)>>(CtbLog2SizeY-1))<<(CtbLog2SizeY-1), ((yCb+(mvL[1]>>4))>>(CtbLog2SizeY-1))<<(CtbLog2SizeY-1)) brightness signal as inputs, and the output must be false.

8.6.2.2. Extraction process for luminance signal motion vector for combining mode

This process is activated only when merge_flag[xCb][yPb] is equal to 1 and CuPredMode[xCb][yPb] is equal to MODE_IBC, where (xCb, yCb) indicates the top left sample of the current luminance coding block relative to the top left sample of the current luminance code frame.

The inputs to this process are as follows:

- location (xCb, yCb) of the luminance signal of the upper left sample of the current luminance coding block relative to the upper left sample of the luminance signal of the current frame,

- cbWidth variable indicating the width of the current encoding block in luminance signal samples,

- cbHeight variable indicating the height of the current encoding block in luma samples.

The findings from this process are:

- motion vectors mvL of the brightness signal with 1/16 fractional sampling accuracy.

The motion vector mvL of the luminance signal is extracted through the following ordered steps:

1. The extraction process for possible combinations from adjacent coding units, as specified in section 8.6.2.3, is activated with the location (xCb, yCb) of the luminance coding unit, the width cbWidth of the luminance coding unit, and the height cbHeight of the luminance coding unit as inputs and outputs representing the availability flags availableFlagA0, availableFlagA1, availableFlagB0, availableFlagB1 and availableFlagB2 and motion vectors mvA0, mvA1, mvB0, mvB1 and mvB2.

2. The list of possible motion vector options for merging, mergeMvCandList, is constructed as follows:

i=0

if(availableFlagA1)

mergeMvCandList [i++]=mvA1

if(availableFlagB1)

mergeMvCandList [i++]=mvB1

if(availableFlagB0) (8-976)

mergeMvCandList [i++]=mvB0

if(availableFlagA0)

mergeMvCandList [i++]=mvA0

[184] if(availableFlagB2)

mergeMvCandList [i++]=mvB2

3. The numCurrMergeCand variable is set equal to the number of possible merge options in mergeMvCandList.

4. When numCurrMergeCand is less than (MaxNumibcMergeCand-1) and NumHmvpIbcCand is greater than 0, the history-based merge candidate extraction process as specified in 8.6.2.5 is activated with mergeMvCandList and numCurrMergeCand as inputs and modified mergeMvCandList and numCurrMergeCand as outputs.

5. When numCurrMergeCand is less than MaxNumibcMergeCand and greater than 1, the extraction process for the pairwise average merge candidate specified in Section 8.6.2.4 is activated with mergeMvCandList and numCurrMergeCand as inputs, and outputs are assigned to mergeMvCandList and numCurrMergeCand.

6. The following appointments are made:

mvL[0]=mergeMvCandList[merge_idx[xCb][yCb]][0] (8-977)

mvL[1]=mergeMvCandList[merge_idx[xCb][yCb]][1] (8-978)

[185] The drawings are provided to illustrate a specific example of this document. Since the name of the specific equipment described in the drawings or the name of a particular signal/message/field is provided as an example, the technical features of this document are not limited to the specific name used in the drawings.

[186] FIG. 9 and fig. 10 schematically shows an example of a method for encoding videos/images and related components according to embodiment(s) of the present document. The method disclosed in FIG. 9 may be performed by the encoding equipment disclosed in FIG. 2. In particular, for example, S900 and S910 of FIG. 9 may be performed by the encoding equipment prediction unit 220, and S920 and S930 of FIG. 9 may be performed by the entropy encoder 240 of the encoding equipment. The method disclosed in FIG. 9 may include embodiments described above herein.

[187] Referring to FIG. 9, the encoding equipment may extract the prediction mode for the current block in the current frame as the IBC prediction mode (S900). The IBC prediction mode may include an IBC pooling mode, an IBC (A)MVP mode, an IBC HMVP mode, and a pairwise average IBC pooling mode. In one example, the IBC prediction mode may be an IBC pooling mode (combining mode for IBC).

[188] The encoding equipment may retrieve a candidate list for the IBC prediction mode (S910). The list of candidates for the IBC prediction mode may include candidate block vectors indicating a reference block in the current frame. In one example, the list of possible options for the IBC forecast mode may be a list of possible combinations for the IBC forecast mode. Table 12 described above shows mvL[0] and mvL[1] as example lists of possible options.

[189] The encoding equipment may generate index information indicating a block vector of the current block based on the candidate list for the IBC prediction mode (S920). For example, the index information may be merge_idx[xCb][yCb] included in table 12 described above.

[190] The encoding equipment can encode video/image information (S930). The image/video information may include index information indicating a block vector of the current block. The luminance signal sampling information may include, for example, prediction-related information (prediction mode information) and residual information. The forecasting-related information includes information regarding various forecasting modes (i.e., inter-prediction aggregation mode, inter-forecasting MVP mode, IBC-prediction mode (IBC-coalescing mode, IBC(A)MVP-mode, IBC HMVP-mode ) etc.). MVD information, candidate list information, information regarding the maximum number of block vectors included in the candidate list for the IBC prediction mode, information regarding the maximum number of block vectors included in the candidate list for the IBC merge mode, inter-prediction mode information regarding the maximum number of block vectors included in the candidate list, information regarding the maximum number of motion vectors included in the candidate list for the inter-prediction combining mode, information regarding block vectors (motion vectors), etc. may turn on. In addition, the image information may include flag information (ie, pred_mode_ibc_flag) indicating whether or not the IBC prediction mode is applied.

[191] In one embodiment, the image information may include a sequence parameter set (SPS), and the SPS includes an IBC enable flag indicating whether or not the IBC prediction mode is activated. For example, when the value of the IBC enable flag is 1, information regarding the maximum number of candidate block vectors included in the candidate list for the IBC prediction mode can be generated.

[192] In one embodiment, the maximum number of candidate block vectors included in the candidate list for the IBC prediction mode may be retrieved based on the difference between 6 and the information value regarding the maximum number of candidate block vectors included in the candidate list for IBC forecasting mode. For example, the maximum number of candidate block vectors included in the candidate list for the IBC prediction mode can be retrieved based on the equation included in Table 3 (MaxNumibcMergeCand=6-six_minus_max_num_ibc_merge_cand). In the equation, MaxNumibcMergeCand may represent the maximum number of candidate block vectors included in the candidate list for the IBC prediction mode, and six_minus_max_num_ibc_merge_cand may represent an information value regarding the maximum number of candidate block vectors included in the candidate list for the IBC prediction mode.

[193] In one embodiment, the image information may include information regarding the maximum number of motion vector candidates included in the candidate list for the inter-prediction mode, and information regarding the maximum number of block vector candidates included in the candidate list for the mode IBC predictions (eg, six_minus_max_num_merge_cand) may differ from the information regarding the maximum number of possible motion vectors included in the candidate list for the inter-prediction mode (eg, six_minus_max_num_ibc_merge_cand).

[194] In one embodiment, when the number of block vector candidates included in the candidate list for the IBC prediction mode is less than the maximum number and greater than 1, the process of extracting the pairwise average of the pooling candidates may be performed. Alternatively, when the number of block vector candidates included in the candidate list for the IBC prediction mode is less than the maximum number (or MaxNumMergeCand-1), and the number of history-based motion vector prediction (HMVP) candidates exceeds 0 ,the process of extracting join candidates based on the ,history can be performed.

[195] The encoded image/video information may be output in the form of a bit stream. The bit stream may be transmitted to decoding equipment via a network or (digital) storage medium.

[196] The image information may include various information according to an embodiment of the present document. For example, the image information may include at least one information or at least one syntactic element disclosed in at least one of Tables 1, 2, 4, 6, 8 and 10 described above.

[197] In one example, the encoding equipment may extract residual samples based on the predictive samples and the original samples. In this case, residual information can be extracted based on the residual samples. Reconstructed samples can be formed based on residual information. The reconstructed block and the reconstructed frame can be retrieved based on the reconstructed samples.

[198] FIG. 11 and fig. 12 schematically shows an example of an image/video decoding method and related components according to an embodiment of the present document. The method disclosed in FIG. 11 may be performed by the decoding equipment illustrated in FIG. 3. In particular, for example, S1100 according to FIG. 11 may be performed by the entropy decoder 310 of the decoding equipment, S1110-S1140 may be performed by the prediction unit 330 of the decoding equipment, and S1150 may be performed by the adder 340 of the decoding equipment. The method disclosed in FIG. 11 may include the embodiments described above herein.

[199] Referring to FIG. 11, the decoding equipment can receive/receive image/video information. For example, decoding equipment may receive/receive image/video information via a bit stream. The decoding equipment may obtain index information for extracting the block vector through the bit stream (S1100). For example, the index information may be merge_idx[xCb][yCb] included in table 12 described above. In addition, the decoding equipment may further obtain flag information (ie, pred_mode_ibc_flag) indicating whether or not the IBC prediction mode is applied via the bitstream.

[200] The image/video information may include various information according to an embodiment of the present document. For example, the image/video information may include information disclosed in at least one of the tables 1, 2, 4, 6, 8 and 10 described above.

[201] The decoding equipment may extract the prediction mode for the current block in the current frame as the IBC prediction mode (S1110). The IBC prediction mode may include an IBC pooling mode, an IBC (A)MVP mode, an IBC HMVP mode, and a pairwise average IBC pooling mode. In one example, the IBC prediction mode may be an IBC pooling mode (combining mode for IBC).

[202] The decoding equipment may retrieve a candidate list for the IBC prediction mode (S1120). The list of candidates for the IBC prediction mode may include candidate block vectors indicating a reference block in the current frame. In one example, the list of possible options for the IBC forecast mode may be a list of possible combinations for the IBC forecast mode. Table 12 described above shows mvL[0] and mvL[1] as example lists of possible options.

[203] The decoding equipment may extract a block vector for the current block based on the candidate list for the IBC prediction mode and index information (S1130).

[204] The decoding equipment may generate predictive samples of the current block based on the block vector (S1140). Predictive samples for the current block may be generated based on reference samples in the reference block indicated by the block vector. Here, the reference block may be included in the current frame.

[205] The decoding equipment may generate reconstructed samples based on the predicted samples (S1150). The decoding equipment can obtain residual information from image information and can extract residual samples based on the residual information. The decoding equipment may generate (extract) reconstructed samples based on the residual samples and the predictive samples. For example, the decoding equipment may generate (extract) reconstructed samples by adding the residual samples with the predictive samples.

[206] In one embodiment, the image information may include a sequence parameter set (SPS), and the SPS includes an IBC enable flag indicating whether or not the IBC prediction mode is activated. For example, when the value of the IBC enable flag is 1, information regarding the maximum number of candidate block vectors included in the candidate list for the IBC prediction mode may be signaled/parsed.

[207] In one embodiment, the maximum number of candidate block vectors included in the candidate list for the IBC prediction mode may be retrieved based on the difference between 6 and the information value regarding the maximum number of candidate block vectors included in the candidate list for IBC forecasting mode. For example, the maximum number of candidate block vectors included in the candidate list for the IBC prediction mode can be retrieved based on the equation included in Table 3 (MaxNumibcMergeCand=6-six_minus_max_num_ibc_merge_cand). In the equation, MaxNumibcMergeCand may represent the maximum number of candidate block vectors included in the candidate list for the IBC prediction mode, and six_minus_max_num_ibc_merge_cand may represent an information value regarding the maximum number of candidate block vectors included in the candidate list for the IBC prediction mode.

[208] In one embodiment, the image information may include information regarding the maximum number of motion vector candidates included in the candidate list for the inter-prediction mode, and information regarding the maximum number of block vector candidates included in the candidate list for the mode IBC predictions (eg, six_minus_max_num_merge_cand) may differ from the information regarding the maximum number of possible motion vectors included in the candidate list for the inter-prediction mode (eg, six_minus_max_num_ibc_merge_cand).

[209] In one embodiment, when the number of block vector candidates included in the candidate list for the IBC prediction mode is less than the maximum number and greater than 1, the process of extracting the pairwise average of the pooling candidates may be performed. Alternatively, When the number of block vector candidates included in the candidate list for the IBC prediction mode is less than the maximum number (or the maximum number of 1, MaxNumMergeCand-1), and the number of history-based motion vector prediction (HMVP) candidates exceeds 0, the process of extracting join candidates based on the history can be performed.

[210] In the above-described embodiment, the methods are described on the basis of a flowchart having a sequence of steps or blocks. The present disclosure is not limited to the order of the above steps or blocks. Some stages or blocks may occur simultaneously or in a different order from other stages or blocks as described above. Additionally, those skilled in the art will understand that the steps shown in the above flowchart are not exhaustive and other steps may be included or one or more steps in the flowchart may be deleted without affecting scope of this disclosure.

[211] The method according to the above-described embodiments of the present document may be implemented in software form, and the encoding equipment and/or decoding equipment according to the present document, for example, may be included in equipment that performs image processing for a television, a computer, a smartphone, a set-top box, display devices, etc.

[212] When the embodiments herein are implemented in software, the above-described method may be implemented as a module (process, function, etc.) that performs the above-described function. The module may be stored in a storage device and executed by a processor. The storage device may be internal or external to the processor and may be coupled to the processor through various known means. The processor may include an application specific integrated circuit (ASIC), other chipsets, logic circuits, and/or data processing devices. The storage device may include read-only memory (ROM), random access memory (RAM), flash memory, memory cards, storage media, and/or other storage devices. Thus, the embodiments described herein may be performed through implementation on a processor, microprocessor, controller, or chip. For example, the functional modules shown in each drawing may be implemented and executed on a computer, processor, microprocessor, controller, or chip. In this case, information regarding instructions or an algorithm to be implemented may be stored in a digital storage medium.

[213] In addition, the decoding equipment and encoding equipment to which the present disclosure applies may be included in multimedia broadcasting equipment, mobile communication terminal, home theater system video equipment, digital cinema system video equipment, surveillance camera, video chatting equipment, real-time communication equipment such as video communication, mobile streaming equipment, storage media, video recording camera, VoD service equipment, over-the-top (OTT) video equipment, Internet streaming service equipment, 3D video equipment, video teleconferencing equipment communications, transport user equipment (ie, vehicle user equipment, aircraft user equipment, marine user equipment, etc.) and medical video equipment, and can be used to process video and data signals. For example, over-the-top (OTT) video equipment may include a game console, Blu-ray player, Internet-enabled television, home theater system, smartphone, tablet PC, digital video recorder (DVR), etc.

[214] Additionally, the processing method to which the present document applies may be formed in the form of a program to be executed by a computer, and may be stored on a computer-readable recording medium. Multimedia data having a data structure according to the present disclosure may also be stored on computer-readable recording media. Computer readable recording media includes all types of storage devices on which data readable by a computer system is stored. Computer readable recording media, for example, may include BD, universal serial bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical storage device. In addition, computer-readable recording media include media implemented in the form of carrier waves (ie, as transmission over the Internet). In addition, the bit stream generated by the encoding method may be stored in a computer-readable recording medium or may be transmitted over wired/wireless communication networks.

[215] In addition, embodiments of the present document may be implemented with a computer program product according to program codes, and the program codes may be executed on a computer by the embodiments of this document. The program codes may be stored on a medium that can be read by a computer.

[216] FIG. 13 shows an example of a content streaming system to which the embodiments disclosed herein may be applied.

[217] Referring to FIG. 13, the content streaming system to which the embodiment(s) of this document applies may mainly include an encoding server, a streaming server, a web server, a media storage, a user device, and a media input device.

[218] The encoding server compresses content input from media input devices such as a smartphone, camera, video recorder, etc. into digital data to generate a bit stream and transmit the bit stream to the streaming server. As another example, when media input devices such as smartphones, cameras, video recorders, etc. directly generate a bitstream, the encoding server may be omitted.

[219] The bitstream may be generated by an encoding method or a bitstream generation method to which the embodiment(s) of this document applies, and the streaming server may temporarily store the bitstream while transmitting or receiving the bitstream.

[220] The streaming server transmits multimedia data to the user device based on the user's request through the web server, and the web server serves as a medium for informing the user regarding the service. When a user requests a required service from a web server, the web server delivers it to the streaming server and the streaming server transmits the media data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server serves to manage the command/response between devices in the content streaming system.

[221] The streaming server may receive content from a media storage and/or encoding server. For example, when content is received from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may maintain the bit stream for a predetermined time.

[222] Examples of a user device may include a mobile phone, a smartphone, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable media player (PMP), a navigation device, a tablet PC, tablet PCs, ultrabooks, wearable devices (such as smart watches, smart glasses, head-mounted displays), digital TVs, desktop computers, digital signage system, etc. Each server in a content streaming system may operate as a distributed server, in which case data received from each server may be distributed.

[223] Each server in a content streaming system may operate as a distributed server, in which case data received from each server may be distributed.

[224] The claims described herein may be combined in various ways. For example, the technical features of the method claims of this document can be combined and implemented as equipment, and the technical features of the equipment claims of this document can be combined and implemented as a method. In addition, the technical features of the method claim of this document and the technical features of the equipment claim can be combined for implementation as equipment, and the technical features of the method claim of this document and the technical features of the equipment claim can be combined and implemented as a way.

Claims (37)

1. A method for decoding images, carried out by means of decoding equipment, the method comprising the steps of: obtaining image information including flag information for an intra-block coding (IBC) prediction mode, index information for extracting a block vector, and residual information through a bit stream; extracting a prediction mode for the current block in the current frame as an IBC prediction mode based on the flag information; retrieving a list of possible options for the IBC prediction mode, wherein the list of possible options includes possible options of the block vectors; extracting a block vector for the current block based on the candidate list for the IBC prediction mode and the index information; generating predictive samples of the current block based on the block vector; generating residual samples of the current block based on residual information; And generate reconstructed samples based on predictive samples and residual samples, wherein the image information contains information regarding the maximum number of possible block vector variants included in the candidate list for the IBC prediction mode, and information regarding the maximum number of possible motion vector variants included in the candidate list for the inter-prediction mode, wherein the information regarding the maximum number of block vector candidates included in the candidate list for the IBC prediction mode is different from the information regarding the maximum number of motion vector candidates included in the candidate list for the inter-prediction mode, and wherein the maximum value for the maximum number of possible block vector options included in the list of possible options for the IBC prediction mode is pre-defined as 6. 2. A method for encoding images, carried out by means of encoding equipment, the method comprising the steps of: extracting a prediction mode for the current block in the current frame as a prediction mode based on intra-block coding (IBC); generating flag information for the IBC prediction mode; retrieving a list of candidates for the IBC prediction mode, wherein the list of candidates includes candidate block vectors for the reference block in the current frame; generating index information for the block vector of the current block based on the list of possible options for the IBC prediction mode; generating residual information for residual samples of the current block; And encode image information including flag information, index information and residual information, wherein the image information contains information regarding the maximum number of possible block vector options included in the candidate list for the IBC prediction mode, and information regarding the maximum number of possible motion vector options included in the candidate list for the inter-prediction mode, wherein the information regarding the maximum number of possible variants of block vectors included in the list of possible variants for the IBC-prediction mode differs from the information regarding the maximum number of possible variants of motion vectors included in the list of possible variants for the inter-prediction mode, wherein the maximum value for the maximum number of possible block vector options included in the list of possible options for the IBC prediction mode is pre-defined as 6. 3. A non-transitory, machine-readable digital storage medium storing instructions that, when executed, cause encoding equipment to carry out the steps of: extracting a prediction mode for the current block in the current frame as a prediction mode based on intra-block coding (IBC); generating flag information for the IBC prediction mode; retrieving a list of candidates for the IBC prediction mode, wherein the list of candidates includes candidate block vectors for the reference block in the current frame; generating index information for the block vector of the current block based on the list of possible options for the IBC prediction mode; generating residual information for residual samples of the current block; And encoding the image information to form a bitstream, wherein the image information includes flag information, index information and residual information, wherein the image information contains information regarding the maximum number of possible block vector variants included in the candidate list for the IBC prediction mode, and information regarding the maximum number of possible motion vector variants included in the candidate list for the inter-prediction mode, wherein the information regarding the maximum number of block vector candidates included in the candidate list for the IBC prediction mode is different from the information regarding the maximum number of motion vector candidates included in the candidate list for the inter-prediction mode, and wherein the maximum value for the maximum number of possible block vector options included in the list of possible options for the IBC prediction mode is pre-defined as 6. 4. A transmission method for transmitting data for image information, comprising the steps of: obtain a bit stream of image information, wherein the bit stream is generated based on extracting the prediction mode for the current block in the current frame as a prediction mode based on intra-block coding (IBC), generate flag information for the IBC prediction mode, extract a list of possible options for the IBC mode prediction, wherein the list of possible options includes possible options of block vectors for the reference block in the current frame, generates index information for the block vector of the current block based on the list of possible options for the IBC prediction mode, generates residual information for the residual samples of the current block and encodes image information including flag information, index information and residual information; And transmit data containing a stream of bits, wherein the image information contains information regarding the maximum number of possible block vector variants included in the candidate list for the IBC prediction mode, and information regarding the maximum number of possible motion vector variants included in the candidate list for the inter-prediction mode, wherein the information regarding the maximum number of block vector candidates included in the candidate list for the IBC prediction mode is different from the information regarding the maximum number of motion vector candidates included in the candidate list for the inter-prediction mode, and wherein the maximum value for the maximum number of possible block vector options included in the list of possible options for the IBC prediction mode is pre-defined as 6.