RU2794519C2 - Method and device for video encoding/decoding using in-block copying mode and method for bit stream transmission - Google Patents

Abstract

FIELD: encoding/decoding images.

SUBSTANCE: image encoding and decoding devices are proposed that perform: determining the prediction mode of the current block based on the prediction mode information of the current unit received from the bit stream, obtaining a block of transform coefficients of the current block from the bit stream, dequantizing the transform coefficient to form a block of dequantized transform coefficients, determining transform kernels for the block of dequantized transform coefficients; and generating a residual block for the current block by an inverse transformation on the block of dequantized transform coefficients using the transform kernel, with the prediction mode of the current block being the intra-block copy (IBC) mode.

EFFECT: increased the coding efficiency.

11 cl, 25 dwg, 2 tbl

Description

The field of technology to which the invention belongs

[1] The present disclosure relates to a method and apparatus for encoding/decoding images and a method for transmitting a bitstream, and more specifically, to a method and apparatus for encoding/decoding an image using a transform kernel, and a method for transmitting a bitstream generated thereby. .

State of the art

[2] Recently, the demand for high-quality high definition images, such as high definition (HD) images and ultra high definition (UHD) images, has been growing in various fields of technology. As the resolution and quality of image data increases, the amount of information or bits to be transmitted increases relatively compared to existing image data. An increase in the amount of information or bits to be transmitted results in an increase in transmission costs and storage costs.

[3] Accordingly, there is a need for a high-performance image compression technology for efficiently transmitting, storing, and reproducing information regarding high-quality, high-resolution images.

The essence of the invention

Technical task

[4] The purpose of the present disclosure is to provide an image encoding/decoding method and apparatus with improved encoding/decoding efficiency.

[5] Another object of the present disclosure is to provide a method and apparatus for encoding/decoding an image by changing the selection method of the transformation kernel according to the coding mode.

[6] Another purpose of the present disclosure is to provide a method for transmitting a bit stream generated by an image encoding method or apparatus according to the present disclosure.

[7] Another purpose of the present disclosure is to provide a recording medium storing a bit stream generated by an image encoding method or apparatus according to the present disclosure.

[8] Another purpose of the present disclosure is to provide a recording medium storing a bitstream received, decoded, and used in order to reconstruct an image by an image decoding apparatus according to the present disclosure.

[9] The technical problems solved by the present disclosure are not limited to the above technical problems, and other technical problems that are not described herein should become apparent to those skilled in the art from the description below.

Technical solution

[10] The image decoding method performed by the image decoding apparatus according to the embodiment for solving the above problems may include determining the prediction mode of the current block based on the prediction mode information of the current block obtained from the bit stream, obtaining a block of transformation coefficients of the current block from the stream bits, dequantizing the transform coefficient to generate a block of dequantized transform coefficients, determining a transform kernel for the block of dequantized transform coefficients, and generating a residual block for the current block by performing an inverse transform on the block of dequantized transform coefficients using the transform kernel

[11] When the prediction mode of the current block is an inter prediction mode or an intra prediction mode, a transform kernel may be determined based on the transform kernel index information obtained from the bit stream

[12] When the prediction mode of the current block is neither the inter prediction mode nor the intra prediction mode, the transform kernel index information may not be obtained from the bit stream and may be extracted as a predetermined value.

[13] When the current block prediction mode is an intra-block copy (IBC) mode that refers to a block located in the same frame as the current block and decoded before the current block, transform kernel index information may not be obtained from the bitstream. and can be retrieved as a predefined value.

[14] When the prediction mode of the current block is the intra-block copy (IBC) mode, the transform kernel index information can be extracted as a value indicating any transform kernel between DCT2 and DCT7.

[15] Determining the transform kernel for the block of dequantized transform coefficients may include extracting the transform kernel of the current block, based on the first flag information indicating whether or not the transform kernel index information of the current block encoded in intra prediction mode can be included in the bitstream. , and second flag information indicating whether or not transformation kernel index information of the current block encoded in the inter prediction mode can be included in the bitstream, and the first flag information and the second flag information can be obtained from the bitstream.

[16] When the first flag information and the second flag information have a first value indicating that the transformation kernel index information is not included in the bitstream, and the prediction mode of the current block is an intra-block copy (IBC) mode, the transformation kernel of the current block may be extracted at based on the width and height of the current block.

[17] The transformation kernel of the current block may include a horizontal transformation kernel and a vertical transformation kernel, the horizontal transformation kernel of the current block may be determined based on whether or not the width of the current block is within a predetermined range, and based on a comparison between width and height of the current box, and the vertical transformation kernel of the current box may be determined based on whether or not the height of the current box is within a predetermined range, and based on a comparison between the width and height of the current box

[18] When at least one of the first flag information and the second flag information has a second value indicating that the transformation core index information can be included in the bit stream, and the current block prediction mode is an intra-block copy (IBC) mode, the transformation core of the current block may be extracted based on the transformation kernel index information, and the transformation kernel index information may not be obtained from the bit stream and may be extracted as a predetermined value.

[19] When the transformation kernel index information is not obtained from the bitstream, the transformation kernel index information may be extracted as a predetermined value. At this time, when the prediction mode of the current block is an intra-block copy (IBC) mode that refers to a block located in the same frame as the current block and decoded before the current block, the transform kernel index information can be extracted as the first values. When the prediction mode of the current block is a prediction mode other than the IBC mode, the transform kernel index information may be extracted as a second value different from the first value.

[20] An image decoding apparatus according to an embodiment for solving the above problems includes a storage device and at least one processor. At least one processor may determine the prediction mode of the current block based on the prediction mode information of the current block obtained from the bitstream, obtain a block of transform coefficients of the current block from the bitstream, dequantize the transform coefficient to generate a block of dequantized transform coefficients, determine a transform kernel for the block dequantized transform coefficients, and generate a residual block for the current block by performing an inverse transform on the block of dequantized transform coefficients using a transform kernel.

[21] When the prediction mode of the current block is an inter prediction mode or an intra prediction mode, a transform kernel may be determined based on transform kernel index information obtained from the bit stream.

[22] In addition, the image encoding method performed by the image encoding apparatus of the embodiment for solving the above problems includes selecting a prediction mode of the current block, generating a prediction block for the current block based on the selected prediction mode, generating a residual block for the current block based on the prediction block, selecting a transform kernel for transforming the residual block based on the prediction mode, transforming the residual block based on the transform kernel to generate a transform coefficient block, quantizing the transform coefficient block to generate a block of quantized transform coefficients, and encoding the block of quantized transform coefficients, to form a bit stream.

[23] When the prediction mode of the current block is an inter prediction mode or an intra prediction mode, transformation kernel index information for the transformation kernel used to transform the residual block may be included in the bitstream.

[24] When the prediction mode of the current block is an intra-block copy (IBC) mode, transformation of the residual block may be performed based on a predetermined transformation kernel, and index information of the transformation kernel for the predetermined transformation kernel used to transform the residual block may not be included in bit stream.

[25] The first flag information indicating whether or not the transformation kernel index information of the current block encoded in the intra prediction mode can be included in the bitstream, and the second flag information indicating whether or not the transformation kernel index information of the current block encoded in the inter prediction mode, included in the bit stream, may be included in the bit stream. At this time, when at least one of the first flag information and the second flag information has a second value indicating that the transform core index information can be included in the bitstream, and the current block prediction mode is an intra-block copy (IBC) mode, the transform the residual block can be executed based on a predefined transformation kernel. In addition, the transformation kernel index information for the predetermined transformation kernel used to transform the residual block may not be included in the bitstream.

[26] When the first flag information and the second flag information have a first value indicating that the transformation core index information is not included in the bit stream, and the prediction mode of the current block is an intra-block copy (IBC) mode, the transformation of the residual block may be performed based on predefined transformation kernel. At this time, a predetermined transformation kernel may be extracted based on the width and height of the current block. In addition, the transformation kernel index information for the predetermined transformation kernel used to transform the residual block may not be included in the bitstream.

[27] In addition, the image encoding apparatus according to the embodiment, in order to solve the above problems, can transmit a bit stream generated as described above.

Benefits of the Invention

[28] According to the present disclosure, it is possible to provide an image encoding/decoding method and apparatus with improved encoding/decoding efficiency.

[29] In addition, according to the present disclosure, it is possible to provide a method and apparatus for encoding/decoding an image by changing the selection method of the transformation kernel according to the encoding mode.

[30] In addition, according to the present disclosure, it is possible to provide a method for transmitting a bit stream generated by an image encoding method or apparatus according to the present disclosure.

[31] In addition, according to the present disclosure, it is possible to provide a recording medium storing a bit stream generated by the image encoding method or apparatus according to the present disclosure.

[32] In addition, according to the present disclosure, it is possible to provide a recording medium storing a bit stream received, decoded, and used to reconstruct an image by the image decoding apparatus according to the present disclosure.

[33] Those skilled in the art should appreciate that the benefits that can be achieved through the present disclosure are not limited to those specifically described above, and other benefits of the present disclosure should be more clearly understood from the detailed description.

Brief description of the drawings

[34] FIG. 1 is a view schematically illustrating a video coding system according to an embodiment.

[35] FIG. 2 is a view schematically illustrating an image encoding apparatus according to the embodiment.

[36] FIG. 3 is a view schematically illustrating an image decoding apparatus according to the embodiment.

[37] FIG. 4-11 are views illustrating an encoding mode according to an embodiment.

[38] FIG. 12 is a flowchart illustrating a method for encoding and decoding using transformation and inverse transformation by an encoding apparatus and a decoding apparatus according to an embodiment.

[39] FIG. 13 and 14 are flowcharts illustrating a method for performing conversion and inverse conversion by an encoder and a decoder according to the embodiment.

[40] FIG. 15 is a view illustrating a transformation core according to an embodiment.

[41] FIG. 16-18 are views illustrating an embodiment of performing encoding and decoding using a transformation core by an encoding device and a decoding device.

[42] FIG. 19-24 are views illustrating another embodiment of performing encoding and decoding using a transform kernel by an encoding device and a decoding device.

[43] FIG. 25 is a view showing a content streaming system according to an embodiment.

Optimal Mode for Carrying Out the Invention

[44] Hereinafter, embodiments of the present disclosure are described in detail with reference to the accompanying drawings so that they can be easily implemented by those skilled in the art. However, the present disclosure may be implemented in any number of forms and is not limited to the embodiments described herein.

[45] In describing the present disclosure, if it is determined that a detailed description of an associated known function or construct would unduly ambiguize the scope of the present disclosure, the detailed description thereof is omitted. In the drawings, parts not related to the description of the present disclosure are omitted and like reference numbers are attached to like parts.

[46] In the present disclosure, when a component is "connected", "coupled", or "linked" to another component, this may include not only a direct connection-based relationship, but also an indirect connection-based relationship in which an intermediate component is present. In addition, when a component "includes" or "has" other components, it means that other components may be included in addition to, rather than excluding, other components, unless otherwise noted.

[47] In the present disclosure, the terms "first", "second", etc. may be used only for the purpose of distinguishing one component from other components and do not limit the order or importance of components unless otherwise noted. Accordingly, within the scope of this disclosure, the first component in one embodiment may be referred to as the "second component" in another embodiment, and similarly, the second component in one embodiment may be referred to as the "first component" in another embodiment.

[48] In the present disclosure, components that are different from each other are intended to clearly describe each feature and do not mean that the components are necessarily separated. Thus, multiple components may be integrated and implemented in a single hardware or software module, or a single component may be distributed and implemented in multiple hardware or software modules. Therefore, even if not stated otherwise, such embodiments in which components are integrated or a component is distributed are also included within the scope of the present disclosure.

[49] In the present disclosure, the components described in various embodiments do not necessarily mean essential components, and some components may be optional components. Accordingly, an embodiment consisting of a subset of the components described in the embodiment is also included within the scope of the present disclosure. In addition, embodiments including other components, in addition to the components described in various embodiments, are included within the scope of the present disclosure.

[50] The present disclosure relates to image encoding and decoding, and the terms used in the present disclosure may have a general meaning widely used in the technical field to which the present disclosure belongs, if not defined for the first time in the present disclosure.

[51] Hereinafter, the present disclosure discloses a video/image coding technology. The encoding/decoding method disclosed in the present disclosure and its embodiments is applicable to the methods disclosed in the universal video coding (VVC) standard or the next generation video/picture coding standard.

[52] The present disclosure provides various embodiments related to video/image coding, and unless otherwise indicated, the embodiments may be performed in combination with each other.

[53] In the present disclosure, "frame" generally means a unit representing one image in a particular time period, and a slice/tile is a coding unit constituting a frame, and one frame may be composed of one or more slices/ tiles. In addition, a slice/tile may include one or more coding tree units (CTUs).

[54] In the present disclosure, "pixel" or "pel" can mean the smallest unit constituting one frame (or image). In addition, "sample" can be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel/pixel value of a luma component, or only a pixel/pixel value of a chroma component.

[55] In the present disclosure, a "unit" may represent a basic unit of image processing. The unit may include at least one of a specific area of the frame and information associated with the area. The unit can be used interchangeably with terms such as "array of samples", "block" or "zone" in some cases. In general, an MxN block may include samples (or arrays of samples) or a set (or array) of transform coefficients of M columns and N rows.

[56] In the present disclosure, "current block" may mean one of "current coding block", "current coding unit", "target coding block", "target decoding block", or "target processing block". When prediction is performed, "current block" may mean "current prediction block" or "target prediction block". When transformation (inverse transformation)/quantization (dequantization) is performed, "current block" may mean "current transformation block" or "transformation target block". When filtering is performed, "current block" may mean "filtering target block".

[57] In the present disclosure, the terms "/" and "" are to be interpreted as indicating "and/or". For example, the expression "A/B" and "A, B" can mean "A and/or B". Additionally, "A/B/C" and "A/B/C" may mean "at least one of A, B and/or C".

[58] In the present disclosure, the term "or" is to be interpreted as indicating "and/or". For example, the expression "A or B" may contain 1) only "A", 2) only "B", and/or 3) "both A and B". In other words, in the present disclosure, the term "or" is to be interpreted as indicating "in addition or alternatively".

[59] Overview of the video coding system

[60] FIG. 1 is a view showing a video coding system according to an embodiment. The video encoding system according to the embodiment may include an encoding device 10 and a decoding device 20. The encoder 10 may deliver the encoded information or video and/or image data to the decoder 20 in the form of a file or streaming via a digital storage medium or a network.

[61] The encoding device 10 according to the embodiment may include a video source generator 11, an encoding module 12, and a transmitter 13. The decoding device 20 according to the embodiment may include a receiver 21, a decoding module 22, and a rendering module 23. The encoding module 12 may be referred to as a "video/image coding module", and the decoding module 22 may be referred to as a "video/image decoding module". The transmitter 13 may be included in the encoding module 12. The receiver 21 may be included in the module 22 decoding. The rendering module 23 may include a display, and the display may be configured as a separate device or an external component.

[62] The video source generator 11 can acquire video/image through a process of capturing, synthesizing, or generating video/image. The video source generator 11 may include a video/image capture device and/or a video/image generator. The video/image capture device may include, for example, one or more cameras, video/image archives including previously captured videos/images, and the like. The video/image generation apparatus may include, for example, computers, tablet computers, and smartphones, and may (electronically) generate video/images. For example, the virtual video/image may be generated via a computer or the like. In this case, the video/image capturing process may be replaced by a linked data generation process.

[63] The encoding unit 12 may encode the input video/image. The coding unit 12 may perform a series of procedures such as prediction, transformation, and quantization for compression and coding efficiency. The encoding unit 12 may output encoded data (encoded video/image information) in the form of a bit stream.

[64] The transmitter 13 may transmit the encoded information or video/image data output in the form of a bit stream to the receiver 21 of the decoding apparatus 20 via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. The transmitter 13 may include an element for generating a media file via a predetermined file format, and may include an element for transmission via a broadcast/communications network. The receiver 21 may extract/receive the bit stream from the storage medium or network and transmit the bit stream to the decoding module 22 .

[65] The decoding module 22 may decode the video/image by performing a sequence of procedures such as dequantization, inverse transformation, and prediction corresponding to the operation of the encoding module 12.

[66] The rendering unit 23 may prepare a decoded video/image by rendering. The rendered video/image can be displayed through the display.

[67] Overview of the image encoding device

[68] FIG. 2 is a view schematically showing an image encoding apparatus according to the embodiment. As shown in FIG. 2, the image encoding apparatus 100 may include an image segmentation unit 110, a subtractor 115, a transform 120, a quantizer 130, a dequantizer 140, an inverse transform 150, an adder 155, a filter 160, a memory 170, an inter prediction unit 180, an intra prediction unit 185. and an entropy encoder 190. An inter prediction unit 180 and an intra prediction unit 185 may be collectively referred to as a "prediction unit". Transformer 120, quantizer 130, dequantizer 140, and inverse transform 150 may be included in the residual processor. The residual processor may further include a subtractor 115.

[69] All or at least a portion of the plurality of components configuring the image encoding apparatus 100 may be configured by a single hardware component (eg, encoder or processor) in some embodiments. In addition, the storage device 170 may include a decoded frame buffer (DPB) and may be configured by a digital storage medium.

[70] The image segmentation unit 110 may segment an input image (either a frame or a movie frame) input to the image encoding apparatus 100 into more than one processing unit. For example, a processing unit may be referred to as a "coding unit (CU)". A coding unit may be obtained by recursively segmenting a coding tree unit (CTU) or a largest coding unit (LCU) according to a quadtree, binary tree, and ternary tree (QT/BT/TT) structure. For example, one coding unit may be segmented into multiple deeper coding units based on a quadtree structure, a binary tree structure, and/or a ternary structure. To segment a coding unit, a quadtree structure may be applied first, and subsequently a binary tree structure and/or a ternary structure may be applied. Alternatively, a binary tree structure may be applied first. The encoding procedure according to the present disclosure may be performed based on a final encoding unit that is no longer segmented. In this case, based on the coding efficiency according to the image characteristics, the largest coding unit may be used as the final coding unit, or the coding unit of greater depth obtained by segmenting the largest coding unit may be used as the final coding unit. Here, the encoding procedure may include a prediction, transformation, and restoration procedure, which is described below. As another example, the encoding procedure processing unit may be a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transformation unit may be split or segmented from the final coding unit. The prediction unit may be a selective prediction unit, and the transform unit may be a unit for extracting a transform coefficient and/or a unit for extracting a residual signal from a transform coefficient.

[71] The prediction unit (external prediction unit 180 or intra prediction unit 185) may perform prediction on a block to be processed (current block) and generate a prediction block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied based on the current block or CU. The prediction module may generate various information related to the prediction of the current block, and pass the generated information to the entropy encoder 190. The prediction information may be encoded in the entropy encoder 190 and output in the form of a bitstream.

[72] The intra prediction module 185 may predict the current block by referring to the samples in the current frame. The reference samples may be located in the current block's surroundings, or may be spaced apart according to the intra prediction mode and/or intra prediction technology. The intra prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The non-directional mode may include, for example, a DC mode and a planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the granularity of the prediction direction. However, this is just an example, more or less directional prediction modes may be used depending on the setting. The intra prediction unit 185 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

[73] The inter prediction unit 180 may derive a predicted block for the current block based on a reference block (reference sample array) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the adjacent block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include external prediction direction information (L0 prediction, L1 prediction, or bi prediction). In the case of inter prediction, an adjacent block may include a spatial adjacent block present in the current frame and a temporal adjacent block present in the reference frame. The reference frame including the reference block and the reference frame including the temporal neighbor block may be the same or different. The temporary neighbor block may be referred to as a "collocated reference block", "collocated CU (colCU)", or the like. A reference frame including a temporal adjacent block may be referred to as a "collocated frame (colPic)". For example, inter prediction module 180 may configure a motion information candidate list based on neighboring blocks and generate information indicating which candidate is used to derive the motion vector and/or the reference frame index of the current block. External prediction may be performed based on various prediction modes. For example, in the case of the skip mode and the merge mode, the inter prediction unit 180 may use the neighboring block motion information as the current block motion information. In the case of the skip mode, unlike the combine mode, the residual signal may not be transmitted. In the case of a motion vector prediction (MVP) mode, the motion vector of an adjacent block may be used as a motion vector predictor, and the motion vector of the current block may be signaled by encoding a motion vector difference and an indicator for the motion vector predictor. The motion vector difference may mean the difference between the motion vector of the current block and the motion vector predictor.

[74] The prediction module may generate a prediction signal based on various prediction methods and prediction technologies described below. For example, the prediction module may not only apply intra prediction or inter prediction, but also apply both intra prediction and inter prediction at the same time to predict the current block. A prediction method that simultaneously applies both intra prediction and inter prediction to predict the current block may be referred to as "Combined Intra-Intra Prediction (CIIP)". In addition, the prediction module may perform intra-block copy (IBC) to predict the current block. In-block copying can be used for image/video encoding of game content and the like, such as on-screen content coding (SCC). IBC is a method for predicting the current frame using a previously reconstructed reference block in the current frame at a location separated from the current block by a predetermined distance. When IBC is applied, the location of the reference block in the current frame may be encoded as a vector (block vector) corresponding to a predetermined distance. The IBC essentially performs prediction within the current frame, but may be performed similarly to outer prediction in which the reference block is retrieved within the current frame. Thus, the IBC may use at least one of the external prediction technologies described in this disclosure.

[75] The predictive signal generated by the prediction module may be used to generate a reconstructed signal or generate a residual signal. The subtractor 115 may generate a residual signal (residual block or residual sample array) by subtracting the prediction signal (predicted block or predictive sample array) output from the predictor from the input image signal (original block or original sample array). The generated residual signal may be transmitted to transducer 120.

[76] Converter 120 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technology may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loeve Transform (KLT), Graph Based Transform (GBT), or Conditionally Nonlinear Transform (CNT). Here, GBT means a transformation obtained from a graph when relationship information between pixels is represented by a graph. CNT means the transformation obtained based on the predictive signal generated using all previously reconstructed pixels. In addition, the transformation process may be applied to square pixel blocks having the same size, or may be applied to blocks having a variable size rather than square.

[77] The quantizer 130 may quantize the transform coefficients and pass them to the entropy encoder 190. The entropy encoder 190 may encode the quantized signal (information regarding the quantized transform coefficients) and output a bit stream. The information regarding the quantized transform coefficients may be referred to as "residual information". The quantizer 130 may repackage the quantized transform coefficients in block form into one-dimensional vector form based on the scan order of the coefficients, and generate information about the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form.

[78] The entropy encoder 190 may implement various coding methods such as exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like, for example. Entropy encoder 190 may encode information needed for video/image reconstruction other than quantized transform coefficients (eg, syntax element values, etc.) together or separately. Encoded information (eg, encoded video/image information) may be transmitted or stored in units of network abstraction layers (NALs) in the form of a bitstream. The video/image information may further include information regarding various parameter sets such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The signaling information, the signaling information and/or the syntax elements described in the present disclosure may be encoded via the encoding procedure described above and included in the bitstream.

[79] The bit stream may be transmitted over a network or may be stored on a digital storage medium. The network may include a broadcast network and/or a communications network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. A transmitter (not shown) transmitting a signal output from the entropy encoder 190 and/or a storage module (not shown) storing the signal may be included as an internal/external element of the encoder 100, and alternatively, a transmitter may be included in the entropy encoder 190 .

[80] The quantized transform coefficients output from quantizer 130 may be used to generate a residual signal. For example, the residual signal (residual block or residual samples) may be recovered by applying dequantization and inverse transform to the quantized transform coefficients via dequantizer 140 and inverse transform 150.

[81] The adder 155 adds the recovered residual signal to the prediction signal output from the inter prediction unit 180 or the intra prediction unit 185 so as to generate a reconstructed signal (recovered frame, reconstructed block, reconstructed sample array). If there is no remainder for a block to be processed, such as in a case in which the skip mode is applied, the predicted block may be used as the reconstructed block. The adder 155 may be referred to as a "recovery module" or "recovered block generator". The generated recovered signal may be used for intra prediction of the next block to be processed in the current frame, and may be used for inter prediction of the next frame through filtering as described below.

[82] The filter 160 can improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, filter 160 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame, and store the modified reconstructed frame in memory 170, namely the DPB of memory 170. Various filtering techniques may include, for example, filtering to deblock , sampled adaptive bias, adaptive loop filter, bilateral filter, etc. The filter 160 may generate various filtering-related information and pass the generated information to the entropy encoder 190 as described below in the description of each filtering method. Filtering related information may be encoded by entropy encoder 190 and output in the form of a bitstream.

[83] The modified reconstructed frame transmitted to the memory 170 may be used as a reference frame in the external prediction module 180. When inter-prediction is applied via the image encoding apparatus 100, prediction mismatch between the image encoding apparatus 100 and the image decoding apparatus can be eliminated, and encoding efficiency can be improved.

[84] The DPB of the memory 170 may store the modified reconstructed frame for use as a reference frame in the inter prediction module 180. The memory 170 may store block motion information from which motion information in the current frame is extracted (or encoded) and/or motion information of blocks in the frame that have already been recovered. The stored motion information may be transmitted to the inter prediction unit 180 and used as spatial neighbor block motion information or temporal neighbor block motion information. The memory 170 may store the recovered samples of the recovered blocks in the current frame and may pass the recovered samples to the intra prediction module 185 .

[85] Overview of the image decoding apparatus

[86] FIG. 3 is a view schematically showing an image decoding apparatus according to the embodiment. As shown in FIG. 3, the image decoding apparatus 200 may include an entropy decoder 210, a dequantizer 220, an inverse transform 230, an adder 235, a filter 240, a memory 250, an inter prediction unit 260, and an intra prediction unit 265. The external prediction module 260 and the intra prediction module 265 may be collectively referred to as a "prediction module". Dequantizer 220 and inverse converter 230 may be included in the residual processor.

[87] All or at least a portion of the plurality of components configuring the image decoding apparatus 200 may be configured by a hardware component (eg, decoder or processor) according to the embodiment. In addition, the storage device 250 may include a decoded frame buffer (DPB) or may be configured by a digital storage medium.

[88] The image decoding apparatus 200, which receives a bitstream including video/image information, can restore an image by performing a process corresponding to the process performed by the image encoding apparatus 100 of FIG. 2. For example, the image decoding apparatus 200 may perform decoding using a processing unit applied in the image encoding apparatus. Thus, the decoding processing unit, for example, may be an encoding unit. The coding unit may be obtained by segmenting the coding tree unit or the largest coding unit. The reconstructed image signal decoded and output by the decoding apparatus 200 can be reproduced by the playback apparatus.

[89] The image decoding apparatus 200 may receive a signal output from the image encoding apparatus of FIG. 2 in the form of a bitstream. The received signal may be decoded via entropy decoder 210. For example, entropy decoder 210 may parse the bit stream to extract information (eg, video/image information) needed for image recovery (or frame recovery). The video/image information may further include information regarding various parameter sets such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The image decoding apparatus may further decode the frame based on the parameter set information and/or general constraint information. Signaling/receiving information and/or syntax elements described in the present disclosure may be decoded by a decoding procedure and obtained from a bit stream. For example, entropy decoder 210 decodes information in the bitstream based on an encoding method such as Exponential Golomb, CAVLC, or CABAC encoding, and output syntax element values required for image reconstruction, and quantized transform coefficient values for the remainder. More specifically, the CABAC entropy decoding method can receive a bin corresponding to each syntax element in the bitstream, determine a context model using decoding target syntax element information, neighbor block decoding information and decoding target block, or symbol/bin information decoded in the previous step, and perform arithmetic decoding on the bin by predicting the occurrence probability of the bin according to the determined context model, and generating a symbol corresponding to the value of each syntax element. In this case, the CABAC entropy decoding method may update the context model by using the decoded symbol/bin information for the context model of the next symbol/bin after determining the context model. Prediction-related information from the information decoded by the entropy decoder 210 may be provided to the prediction unit (the outer prediction unit 260 and the intra prediction unit 265), and the residual value for which the entropy decoding is performed in the entropy decoder 210, i. quantized transform coefficients and related parameter information may be input to the dequantizer 220. In addition, information regarding filtering from the information decoded by the entropy decoder 210 may be provided to the filter 240. Meanwhile, a receiver (not shown) for receiving a signal output from the device image encoding device may be further configured as an internal/external element of the image decoding device 200, or the receiver may be a component of the entropy decoder 210.

[90] Meanwhile, the image decoding apparatus according to the present disclosure may be referred to as a "video/image/frame decoding apparatus". The image decoding apparatus can be classified into an information decoder (video/image/frame information decoder) and a sampling decoder (video/image/frame samples decoder). The information decoder may include an entropy decoder 210. The selective decoder may include at least one of a dequantizer 220, an inverse transform 230, an adder 235, a filter 240, a memory 250, an inter-predictor 260, or an intra-predictor 265.

[91] The dequantizer 220 may dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 220 may recompose the quantized transform coefficients in the form of a two-dimensional block. In this case, re-arranging may be performed based on the scan order of the coefficients performed in the image encoding apparatus. The dequantizer 220 may perform dequantization on the quantized transform coefficients by using a quantization parameter (eg, quantization step size information) and obtain the transform coefficients.

[92] The inverse transform 230 may inversely transform the transform coefficients to obtain a residual signal (residual block, array of residual samples).

[93] The prediction module may perform prediction for the current block and may generate a predicted block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied to the current block based on the prediction information output from the entropy decoder 210, and may determine a specific intra/inter prediction mode.

[94] Identical to that described in the prediction module of the image encoding apparatus 100, the prediction module can generate a prediction signal based on various prediction methods (technologies) which are described below.

[95] For example, the prediction unit may apply intra prediction or inter prediction, or simultaneously apply intra prediction and inter prediction to predict one block. This may be referred to as "combined external and internal prediction (CIIIP)". In addition, the prediction module may perform intra-block copy (IBC) to predict the block. In-block copying can be used to encode image/video content such as a game, for example, to perform screen content coding (SCC). The IBC may be performed similarly to inter prediction in that the prediction is performed in the current frame, but the reference block is retrieved from the current frame. Thus, the IBC may use at least one of the external prediction technologies described herein.

[96] The intra prediction module 265 may predict the current block with reference to the samples in the current frame. The description of the intra prediction module 185 applies equally to the intra prediction module 265 . The referenced samples may be located in the neighborhood (nearby) or spaced apart from the current block. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit 265 may determine the prediction mode applied to the current block using the prediction mode applied to the adjacent block.

[97] The inter prediction unit 260 may derive a predicted block for the current block based on a reference block (reference sample array) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the adjacent block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include external prediction direction information (L0 prediction, L1 prediction, or bi prediction). In the case of inter prediction, an adjacent block may include a spatial adjacent block present in the current frame and a temporal adjacent block present in the reference frame. For example, inter prediction module 260 may configure a motion information candidate list based on neighboring blocks, and derive the current block's motion vector and/or reference frame index based on the received candidate selection information. Intra prediction may be performed based on various prediction modes, and the prediction information may include information indicative of the inter prediction mode for the current block.

[98] The adder 235 may generate a reconstructed signal (recovered frame, reconstructed block, reconstructed sample array) by summing the obtained residual signal with the predictive signal (predicted block, predicted sample array) output from the prediction module (including the outer prediction module 260 and /or internal prediction module 265). If there is no remainder for the block to be processed, for example, when the skip mode is applied, the predicted block may be used as the reconstructed block. The description of adder 155 is equally applicable to adder 235. Adder 235 may be referred to as a "recovery module" or "recovered block generator". The generated recovered signal may be used for intra prediction of the next block to be processed in the current frame, and may be used for inter prediction of the next frame through filtering as described below.

[99] The filter 240 can improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, filter 240 may generate a modified reconstructed frame by applying various filtering methods to the reconstructed frame and store the modified reconstructed frame in memory 250, namely the DPB of storage 250. Various filtering methods may include, for example, filtering to deblock , sampled adaptive bias, adaptive loop filter, bilateral filter, etc.

[100] The (modified) reconstructed frame stored in the DPB of the storage device 250 may be used as a reference frame in the inter prediction module 260 . The memory 250 may store block motion information from which motion information in the current frame is extracted (or decoded) and/or motion information of blocks in the frame that have already been recovered. The stored motion information may be transmitted to the inter prediction unit 260 so that it is used as spatial neighbor block motion information or temporal neighbor block motion information. The memory 250 may store the recovered samples of the recovered blocks in the current frame and pass the recovered samples to the intra prediction module 265 .

[101] In the present disclosure, the embodiments described in the filter 160, the inter prediction unit 180, and the intra prediction unit 185 of the image encoding apparatus 100 may be equally or appropriately applied to the filter 240, the inter prediction unit 260, and the intra prediction unit 265 of the image decoding apparatus 200, respectively, and the embodiments described in the filter 240, the inter prediction unit 260 and the intra prediction unit 265 of the image decoding apparatus 200 are equally or appropriately applied to the filter 160, the inter prediction unit 180, and the intra prediction unit 185 of the device 100 image encoding, respectively.

[102] Intra prediction overview

[103] Hereinafter, this document describes an intra prediction method according to an embodiment. The intra prediction may indicate a prediction that generates prediction samples for the current block based on the reference samples in the frame to which the current block belongs (hereinafter referred to as the "current frame"). When intra prediction is applied to the current block, adjacent reference samples to be used for intra prediction of the current block may be extracted. Adjacent reference samples of the current block may include the sample adjacent to the left boundary of the current block having size nWxnH and the sum of the 2xnH samples adjacent to the lower left, the samples adjacent to the upper boundary of the current block and the sum of the 2xnW samples adjacent to the right the top part, and one selection bordering the left top part relative to the current block. Alternatively, the adjacent reference samples of the current block may include a plurality of columns of top neighbor samples and a plurality of rows of left neighbor samples. In addition, adjacent reference samples of the current block may include the sum of nH samples adjacent to the right edge of the current block having size nWxnH, the sum of nW samples adjacent to the bottom edge of the current block, and one sample bordering the bottom right of the current block. Meanwhile, when the ISP, which is described below, is applied, adjacent reference samples may be extracted in units of sub-segments.

[104] On the other hand, some adjacent reference samples of the current block have not yet been decoded or may not be available. In this case, the decoder may construct adjacent reference samples to be used for prediction by substituting the available samples for the unavailable samples. Alternatively, adjacent reference samples to be used for prediction may be constructed using an interpolation of the available samples.

[105] When adjacent reference samples are extracted, (i) a predictive sample may be derived based on an average or an interpolation of adjacent reference samples of the current block, and (ii) a predictive sample may be derived based on a reference sample present in a particular direction (prediction) relative to the predictive sample from neighboring reference samples of the current block. Case (i) may be referred to as "non-directional mode" or "non-angular mode", and case (ii) may be referred to as "directional mode" or "angular mode". In addition, the prediction sample may be generated through interpolation with a second neighbor sample and a first neighbor sample located in the opposite direction of the prediction direction of the intra prediction mode of the current block based on the prediction sample of the current block from the neighbor reference samples. The above case may be referred to as "linear interpolation intra prediction (LIP)". In addition, the chrominance signal prediction samples may be generated based on the luminance signal samples using a linear model. The above case may be referred to as "LM mode". In addition, the temporal prediction sample of the current block may be derived based on the filtered neighbor reference samples, and the prediction sample of the current block may be derived by weighting the temporal prediction sample and at least one reference sample derived according to the intra prediction mode from the existing neighbor reference samples, those. unfiltered neighboring reference samples. The above case may be referred to as "position dependent intra prediction (PDPC)". In addition, the reference sample line with the highest prediction accuracy may be selected from a plurality of adjacent reference sample lines of the current block to extract the prediction sample using the reference sample located in the prediction direction in the corresponding line, and, at this time, intra prediction coding may be performed by indications (transmissions in service signals) of the reference sample line used for the decoding device. The above case may be referred to as "multiple reference line (MRL) intra prediction" or "MRL intra prediction". In addition, the current block may be divided into vertical or horizontal subsegments to perform intra prediction based on the same intra prediction mode, and adjacent reference samples may be extracted and used in units of subsegments. Thus, in this case, the intra prediction mode for the current block is identically applied to subsegments, and adjacent reference samples are extracted and used in subsegment units, thereby improving intra prediction performance. Such a prediction method may be referred to as "intra-sub-segment prediction (ISP) or ISP-based". In addition, when the direction of prediction based on the prediction sample indicates the space between adjacent reference samples, i. e. when the prediction direction indicates a fractional sample position, the value of the prediction sample can be obtained through interpolation of a plurality of reference samples located near the prediction direction (near the fractional sample position). The above-described intra prediction methods may be referred to as "intra prediction type" to be distinguished from the intra prediction mode. In addition, after the prediction signal for the downsampled pixel set of the current block is generated using the reconstructed neighboring pixels located at the left and the top of the current block, the generated prediction signal and the neighboring sample value can be interpolated in the vertical and horizontal directions to generate a prediction signal having the original size, thereby applying matrix weighted intra prediction (MIP) to perform intra prediction of the current block.

[106] The type of intra prediction may be referred to by various terms such as an intra prediction scheme or an additional intra prediction mode. For example, the intra prediction type (or optional intra prediction mode) may include at least one of LIP, PDPC, MRL, ISP, or MIP. Information regarding the type of intra prediction may be encoded by an encoder, included in a bitstream, and signaled to a decoder. The information regarding the type of intra prediction may be implemented in various forms, for example, as flag information indicating whether or not to apply each type of intra prediction, or as index information indicating one of several types of intra prediction.

[107] Meanwhile, if necessary, post-filtering may be performed on the extracted predictive sample. Specifically, the intra prediction procedure may include an intra prediction mode/type determination step, an adjacent reference sample extraction step, and an intra prediction mode/type prediction step. In addition, if necessary, post-filtering can be performed on the extracted predictive sample.

[108] Hereinafter, this document describes a video/image coding method based on intra prediction. First, the encoder performs intra prediction with respect to the current block. The encoder may extract the intra prediction mode/type for the current block, extract adjacent reference samples of the current block, and generate prediction samples in the current block based on the intra prediction mode/type and adjacent reference samples. Here, the procedures for determining the mode/type of intra prediction, extracting adjacent reference samples, and generating prediction samples may be simultaneously performed, or any procedure may be performed before other procedures. Meanwhile, when the predictive sample filtering procedure described below is executed, the intra prediction unit 185 may further include a filter of the predictive samples. The encoder may determine the mode/type applied to the current block from a plurality of intra prediction modes/types. The encoder can compare the rate-dependent distortion cost function (RD) for intra prediction modes/types and determine the optimal intra prediction mode/type for the current block.

[109] Meanwhile, the encoder may perform a filtering procedure on the predictive samples. Filtering predictive samples may be referred to as "post-filtering". Through the prediction samples filtering procedure, some or all of the prediction samples may be filtered. In some cases, the filtering procedure for predictive samples may be omitted.

[110] Next, the encoder may generate residual samples for the current block based on the predictive samples. The encoder may compare the predictive samples in the original samples of the current block on a phase basis and extract the residual samples.

[111] Next, the encoding apparatus may encode image information including information regarding intra prediction (prediction information) and residual information regarding residual samples. The prediction information may include intra prediction mode information and intra prediction type information. The encoding device may output the encoded image information in the form of a bit stream. The output bit stream may be transmitted to the decoder via a storage medium or a network.

[112] The residual information may include a residual encoding syntax, which is described below. The encoder may transform/quantize the residual samples and extract the quantized transform coefficients. The residual information may include information regarding the quantized transform coefficients.

[113] Meanwhile, as described above, the image encoding apparatus may generate a reconstructed frame (including reconstructed samples and a reconstructed block). To this end, the image coding apparatus may perform inverse quantization/inverse transformation with respect to the quantized transform coefficients and extract (modified) residual samples. The reason for transforming/quantizing the residual samples and then performing inverse quantization/inverse transform is to extract identical residual samples as residual samples extracted by the decoder as described above. The encoder may generate a reconstructed block including reconstructed samples for the current block based on the predictive samples and the (modified) residual samples. Based on the reconstructed block, a reconstructed frame for the current frame may be generated. As described above, the in-loop filtering procedure may be applied to the reconstructed frame.

[114] Hereinafter, this document describes a video/image coding method based on intra prediction. The decoding device may perform an operation corresponding to the operation performed by the encoding device.

[115] First, the decoding apparatus may derive the intra prediction mode/type for the current block based on the received prediction information (intra prediction mode/type information). The decoding apparatus may extract adjacent reference samples of the current block. based on the intra prediction mode/type and adjacent reference samples.In this case, the image decoding apparatus may perform a prediction sample filtering procedure.Filtering of the prediction samples may be referred to as "post-filtering."Through the prediction sample filtering procedure, some or all of the prediction samples may be filtered.In some cases, the procedure for filtering predictive samples may be omitted.

[116] The image decoding apparatus may generate residual samples for the current block based on the received residual information (S640). The decoder may generate reconstructed samples for the current block based on the predictive samples and the residual samples, and extract a reconstructed block including the reconstructed samples. Based on the reconstructed block, a reconstructed frame for the current frame may be generated. The in-loop filtering procedure is additionally applicable to the reconstructed frame.

[117] The intra prediction mode information may include, for example, flag information (eg, intra_luma_mpm_flag) indicating whether or not the most likely mode (MPM) or remaining mode is applied to the current block, and when the MPM is applied to the current block, the information the prediction mode may further include index information (eg, intra_luma_mpm_idx) indicating one of the possible intra prediction modes (MPM candidates). Intra prediction mode candidates (MPM candidates) may configure an MPM candidate list or an MPM list. For example, the list of MPM candidates may include an adjacent block intra prediction mode or a preset base intra prediction mode. In addition, when MPM is not applied to the current block, the intra prediction mode information may further include remaining mode information (e.g., intra_luma_mpm_remainder) indicating one of the remaining intra prediction modes except for intra prediction mode options (MPM options) . The decoding apparatus may determine the intra prediction mode of the current block based on the intra prediction mode information.

[118] In addition, the intra-prediction type information may be implemented in various forms. For example, the intra prediction type information may include intra prediction type index information indicating one of the intra prediction types. As another example, the intra prediction type information may include at least one of reference sample line information (e.g., intra_luma_ref_idx) indicating whether or not the MRL should be applied to the current block, and if so, which reference sample line. is used, ISP flag information (e.g., intra_subpartitions_mode_flag) indicating whether or not ISP should be applied to the current block, ISP type information (e.g., intra_subpartitions_split_flag)) indicating the sub-segment splitting type when applying ISP, flag information indicating whether whether or not to apply PDPC, or flag information indicating whether or not to apply LIP.

[119] The intra prediction mode information and/or the intra prediction type information may be encoded/decoded via the coding method described in the present disclosure. For example, the intra prediction mode information and/or the intra prediction type information may be coded/decoded by entropy coding (eg, CABAC, CAVLC) based on a truncated binary code (Rice).

[120] Later in this document, the intra prediction mode is described in more detail. Fig. 4 shows the direction of intra prediction according to the embodiment. To capture any edge direction represented in natural video, as shown in FIG. 8a, the intra prediction mode may include two non-directional intra prediction modes and 65 directional intra prediction modes. The non-directional intra prediction modes may include a planar intra prediction mode and a DC intra prediction mode, and the directional intra prediction modes may include the second to 66th intra prediction modes.

[121] Meanwhile, the intra prediction mode may further include a cross-component linear model (CCLM) based mode for chrominance samples in addition to the above-described intra prediction modes. The CCLM mode may be partitioned into L_CCLM, T_CCLM, LT_CCLM according to whether or not left samples, top samples, or both are considered to extract the LM parameter, and can only be applied to the chrominance signal component. For example, the intra prediction mode may be indexed as shown in the following table.

[122] Tab. 1

Internal Prediction Mode Associated name 0 INTRA_PLANAR 1 INTRA_DC 2...66 INTRA_ANGULAR2...INTRA_ANGULAR66 81...83 INTRA_LT_CCLM, INTRA_L_CCLM, INTRA_T_CCLM

[123] FIG. 5 shows the direction of intra prediction according to another embodiment. Here, the dotted line direction shows the wide-angle mode applied to the non-square block only. As shown in FIG. 5, in order to capture any edge direction represented in natural video, the intra prediction mode according to the embodiment may include two non-directional intra prediction modes and 93 directional intra prediction modes. The non-directional intra prediction modes may include a planar prediction mode and a DC prediction mode, and the directional intra prediction modes may include the second-80th and -1st--14th intra prediction modes, as indicated by an arrow along fig. 5. The planar prediction mode may be denoted by INTRA_PLANAR, and the DC prediction mode may be denoted by INTRA_DC. In addition, the directional intra prediction mode may be denoted by INTRA_ANGULAR-14 through INTRA_ANGULAR-1 and INTRA_ANGULAR2 through INTRA_ANGULAR80.

[124] Meanwhile, the intra prediction type (or additional intra prediction mode) may include at least one of LIP, PDPC, MRL, ISP, or MIP. The intra prediction type may be indicated based on the intra prediction type information, and the intra prediction type information may be implemented in various forms. For example, the intra prediction type information may include intra prediction type index information indicating one of the intra prediction types. As another example, the intra prediction type information may include at least one of reference sample line information (eg, intra_luma_ref_idx) indicating whether or not the MRL should be applied to the current block, and if so, which reference sample line. is used, ISP flag information (for example, intra_subpartitions_mode_flag) indicating whether or not ISP should be applied to the current block, ISP type information (for example, intra_subpartitions_split_flag) indicating the type of sub-partitions split when applying ISP, flag information indicating whether or not do not apply PDPC, flag information indicating whether or not to apply LIP, or MIP flag information indicating whether or not to apply MIP.

[125] External Prediction Overview

[126] Hereinafter, an external prediction method according to an embodiment is described. The predictor unit of the encoder and the decoder may perform inter-prediction in units of blocks to extract a predictive sample. Intra-prediction may mean that a prediction derived in a manner that depends on data elements (eg, sample values, motion information, etc.) of frame(s) other than the current frame. When inter-prediction is applied to a current block, a predicted block (predictive sample array) for the current block may be derived based on a reference block (reference sample array) indicated by a motion vector on a reference frame indicated by a reference frame index. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information of the current block may be predicted in units of blocks, subblocks, or samples based on the correlation of the motion information between the adjacent block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction type information (L0 prediction, L1 prediction, bi-prediction, etc.). In the case of inter prediction, an adjacent block may include a spatial adjacent block present in the current frame and a temporal adjacent block present in the reference frame. The reference frame including the reference block and the reference frame including the temporal neighbor block may be the same or different. A temporal neighbor block may be referred to as a "collocated reference block", a "collocated CU (colCU)", and a reference frame including the temporal adjacent block may be referred to as a "collocated frame (colPic)". For example, a list of motion information candidates may be constructed based on neighboring blocks with respect to the current block, and flag or index information indicating which candidate is selected (used) may be signaled so as to derive the current block's motion vector and/ or the index of the reference frame. External prediction may be performed based on various prediction modes. For example, in the case of the skip mode and the merge mode, the motion information of the current block may be equal to the motion information of the selected neighboring block. In the case of the skip mode, the residual signal may not be transmitted, in contrast to the combine mode. In the case of a motion information prediction (MVP) mode, a motion vector of a selected neighbor block may be used as a motion vector predictor, and the motion vector difference may be signaled. In this case, the motion vector of the current block may be extracted using the sum of the motion vector predictor and the motion vector difference.

[127] The motion information may include L0 motion information and/or L1 motion information according to the type of external prediction (L0 prediction, L1 prediction, bi-prediction, etc.). The motion vector in the L0 direction may be referred to as the "L0 motion vector" or "MVL0", and the motion vector in the L1 direction may be referred to as the "L1 motion vector" or "MVL1". Prediction based on the L0 motion vector may be referred to as "L0 prediction", prediction based on the L1 motion vector may be referred to as "L1 prediction", and prediction based on both the L0 motion vector and the L1 motion vector may be referred to as "biprediction". ". Here, the L0 motion vector may indicate a motion vector associated with the L0 list of reference frames (L0), and the L1 motion vector may indicate a motion vector associated with the L1 list of reference frames (L1). The reference frame list L0 may include frames before the current frame in the output order as reference frames, and the reference frame list L1 may include frames after the current frame in the output order. Previous frames may be referred to as "forward (reference) frames", and subsequent frames may be referred to as "reverse (reference) frames". The reference frame list L0 may further include frames after the current frame in output order as reference frames. In this case, in the reference frame list L0, previous frames may be indexed first, and subsequent frames may then be indexed. The reference frame list L1 may further include frames before the current frame in output order as reference frames. In this case, in the reference frame list L1, subsequent frames may be indexed first, and previous frames may then be indexed. Here, the output order may correspond to the order of the numbers in the sequence of frames (POC).

[128] Hereinafter, this document describes a video/image coding procedure based on inter prediction and an inter prediction unit in an encoding apparatus with reference to FIG. 6 and 7.

[129] First, the encoder may perform inter prediction with respect to the current block (S11). The image encoding apparatus may extract the inter-prediction mode and motion information of the current block, and generate prediction samples of the current block. Here, determination of the inter-prediction mode, motion information extraction, and prediction sampling procedures may be simultaneously performed, or any of these may be performed before other procedures. For example, the encoder inter-prediction unit 180 may include a prediction mode determination unit 181, a motion information extraction unit 182, and a prediction sample extraction unit 183. The prediction mode determining unit 181 may determine the prediction mode of the current block, the motion information extractor 182 may extract the motion information of the current block, and the prediction sample extractor 183 may extract the prediction samples of the current block. For example, the encoder inter prediction unit 180 may search for a block similar to the current block in a predetermined area (search area) of reference frames through motion estimation, and retrieve a reference block whose difference relative to the current block is equal to or less than a predetermined criterion or minimum. Based on this, a reference frame index indicating a reference frame in which the reference block is located may be extracted, and a motion vector may be extracted based on a position difference between the reference block and the current block. The encoder may determine the mode applied to the current block from various prediction modes. The encoder can compare the RD costs for different prediction modes and determine the optimal prediction mode of the current block.

[130] For example, when the skip mode or merge mode is applied to the current block, the encoder 100 may construct a list of merge options and extract a reference block whose difference with respect to the current block is equal to or less than a predetermined criterion or minimum from the reference blocks indicated by possible joins included in the list of possible joins. In this case, the combining candidate associated with the extracted reference block may be selected, and combining index information indicating the selected combining candidate may be generated and signaled to the decoder. The motion information of the current block may be extracted using the motion information of the selected join candidate.

[131] As another example, when the (A)MVP mode is applied to the current block, the encoder may construct the (A) MVP candidate list and extract the motion vector of the MVP candidate selected from among the MVP candidates included in (a) a list of possible MVP options. In this case, for example, a motion vector indicating a reference block derived by the above-described motion estimation may be used as the motion vector of the current block, and the MVP candidate with the motion vector having the smallest difference with respect to the motion vector of the current block among the candidate MVP candidates , may represent the selected MVP candidate. A motion vector difference (MVD), which is a difference obtained by subtracting the MVP from the motion vector of the current block, can be extracted. In this case, information regarding the MVD may be signaled to the decoder. In addition, when applying the (A)MVP mode, the index value of the reference frame may be constructed as index information of the reference frame and separately signaled to the decoder.

[132] The encoder may derive residual samples based on the predictive samples (S12). The encoder may extract the residual samples through a comparison between the original samples of the current block and the predictive samples.

[133] The encoding device may encode image information including prediction information and residual information (S13). The encoding device may output the encoded image information in the form of a bitstream. The prediction information may include prediction mode information (eg, skip flag, merge flag, or mode index, etc.) and motion information as information associated with the prediction procedure. The information on the motion information may include candidate selection information (eg, merge index, MVP flag, or MVP index), which is information for extracting a motion vector. In addition, information regarding motion information may include information regarding the above-described MVD and/or reference frame index information. In addition, the information on the motion information may include information indicating whether L0 prediction, L1 prediction, or bi-prediction should be applied. The residual information is information about the residual samples. The residual information may include information regarding quantized transform coefficients for the residual samples.

[134] The output bitstream may be stored on a (digital) storage medium and transmitted to the decoder, or may be transmitted to the decoder via a network.

[135] As described above, the encoder may generate a reconstructed frame (including the reconstructed samples and the reconstructed block) based on the reference samples and the residual samples. This is for extracting, by the encoding apparatus, a prediction result identical to that performed by the decoding apparatus, thereby improving encoding efficiency. Accordingly, the encoder may store the reconstructed frame (or the reconstructed samples and the reconstructed block) in a memory and use it as a reference frame for inter prediction. As described above, the in-loop filtering procedure may additionally be applied to the reconstructed frame.

[136] Hereinafter, this document describes a video/image decoding procedure based on inter prediction and an inter prediction unit in a decoding apparatus with reference to FIG. 8 and 9. The decoder may perform an operation corresponding to the operation performed by the encoder. The decoding apparatus may perform prediction on the current block based on the received prediction information and extract prediction samples.

[137] In particular, the decoding apparatus may determine the prediction mode of the current block based on the received prediction information (S21). The image decoding apparatus can determine which inter prediction mode is applied to the current block based on the prediction mode information in the prediction information.

[138] For example, it may be determined whether the merge mode or (A)MVP mode is applied to the current block based on the merge flag. Alternatively, one of the various possible inter prediction modes may be selected based on the mode index. Possible inter prediction mode options may include skip mode, merge mode, and/or (A)MVP mode, or may include various inter prediction modes, which are described below.

[139] The decoding apparatus may extract motion information of the current block based on the determined inter prediction mode (S22). For example, when the skip mode or merge mode is applied to the current block, the decoding device may construct a merge candidate list, which is described below, and select one of the merge candidates included in the merge candidate list. The selection may be performed based on the above-described candidate selection information (combination index). The motion information of the current block may be extracted using the motion information of the selected join candidate. The motion information of the selected join candidate may be used as the motion information of the current block.

[140] As another example, when the (A)MVP mode is applied to the current block, the decoder may construct a list of (A)MVP candidates and use the motion vector of the MVP candidate selected from among the MVP candidates included to the list of possible (A)MVP options, as the MVP of the current block. The selection may be performed based on the candidate selection information (MVP flag or MVP index) described above. In this case, the MVD of the current block may be derived based on information regarding the MVD, and the motion vector of the current block may be derived based on the MVP and MVD of the current block. In addition, the index of the key frame of the current block may be derived based on the information of the index of the key frame. The frame indicated by the index of the reference frame in the reference frame list of the current block may be retrieved as the reference frame referenced for external prediction of the current block.

[141] Meanwhile, as described below, motion information of the current block can be extracted without constructing candidate lists, and in this case, motion information of the current block can be extracted according to the disclosed procedure in the prediction mode described below. In this case, the above construction of lists of options may be omitted.

[142] The image decoding apparatus may generate prediction samples of the current block based on motion information of the current block (S23). In this case, the reference frame may be derived based on the reference frame index of the current block, and the prediction samples of the current block may be derived using the reference block samples indicated by the motion vector of the current block on the reference frame. In this case, as described below, in some cases, the prediction sample filtering procedure may further be performed on all or some of the prediction samples of the current block.

[143] For example, the inter prediction unit 260 of the decoding apparatus 200 may include a prediction mode determination unit 261, a motion information extraction unit 262, and a prediction sample extraction unit 263. The prediction mode determining unit 181 may determine the prediction mode for the current block based on the received prediction mode information, the motion information extraction unit 182 may extract the motion information (motion vector and/or reference frame index) for the current block based on the information for the received motion information, and the prediction sample extraction module 183 may extract the prediction samples of the current block.

[144] The decoding apparatus may generate residual samples of the current block based on the received residual information (S24). The decoding apparatus may generate reconstructed samples of the current block based on the predictive samples and residual samples, and generate a reconstructed frame based on this (S25). Thereafter, the in-loop filtering procedure is applicable to the reconstructed frame.

[145] As described above, the inter prediction procedure may include a step of determining an inter prediction mode, a step of extracting motion information according to the determined prediction mode, and a step of performing prediction (predictive sampling) based on the extracted motion information. The inter prediction procedure may be performed by an encoder and a decoder as described above.

[146] Intra-Block Copy (IBC) Prediction General Representation

[147] The decoding apparatus according to the embodiment can perform an intra-block copy (IBC) prediction mode to decode the current block using reference block information of the current frame in which the current block is located to decode the current block, which is the block to be decoded. . For example, the decoding device may decode the current block using the decoding information of the reference block selected from the reconstructed area of the current frame. Since only the reconstructed area in the predetermined area including the current CTU is used as the reference area, it is possible to reduce the memory consumption and reduce the complexity of the decoding apparatus. Thus, the IBC mode can be implemented with a hardware configuration using an on-chip memory.

[148] In the IBC mode, prediction is essentially performed in the current frame, but may be performed similarly to inter prediction in which the reference block is retrieved in the current frame. For example, IBC may be performed using at least one of the external prediction technologies described above. For example, the decoding apparatus may use the motion vector extraction method described above to perform the IBC mode. At least one of the above external prediction technologies may be modified and used with IBC prediction in mind, as described below. In an embodiment, the IBC may be referred to as "Current Frame Reference (CPR)" in that it refers to the current frame.

[149] To perform the IBC mode, the encoder may extract the optimal motion vector for the current block (eg, CU) by performing block matching. The extracted motion vector may be signaled to the decoder via a bit stream using a method similar to signaling the motion vectors in the above-described inter prediction. The decoder may derive a reference block for the current block in the current frame via a signaled motion vector, and extract a prediction signal (predicted block or prediction samples) for the current block through it. Here, a block vector (or motion vector) may represent an offset from the current block to a reference block located in an already reconstructed area in the current frame. Accordingly, a block vector (or motion vector) may be referred to as a "displacement vector". Later in this document, in IBC, the motion vector may correspond to a block vector or a displacement vector. The motion vector of the current block may include a motion vector for a luminance signal component (luminance motion vector) or a motion vector for a chrominance signal component (chroma motion vector). For example, the luminance motion vector for an IBC-coded CU may be an integer unit of samples (ie, integer precision). The chrominance motion vector may also be truncated in units of integer samples. As described above, the IBC may use at least one of the inter prediction technologies and, for example, when the IBC is applied as an AMVR, the 1-pellet and 4-pellet motion vector accuracy may be switched.

[150] At the CU level, the IBC mode may be signaled using a flag, or may be signaled in the IBC AMVP mode or the IBC skip/combine mode as follows.

[151] IBC Skip/Merge Mode: A merge candidate index may be used to indicate which block vector is used to predict the current block in a list consisting of adjacent candidate IBC coding blocks. The pooling list may consist of spatial, HMVP, and pairwise candidate options.

[152] IBC AMVP mode: The block vector difference may be encoded identically to motion vector difference coding. The block vector prediction method uses two options as a predictor. If encoding is done in IBC mode, one is for the left neighbor and the other is for the top neighbor. If at least one neighbor is actually available, the underlying block vector may be used as a predictor. A flag may be signaled to indicate a block vector predictor index.

[153] Hereinafter, a video/image coding procedure based on the IBC mode and a predictor in an encoder will be described with reference to FIG. 10.

[154] The encoder may perform IBC prediction (IBC based prediction) for the current block (S31). The encoder may extract the prediction mode and motion vector of the current block and generate prediction samples of the current block. The prediction mode may include at least one of the inter prediction modes described above. Here, the prediction mode determination, motion vector extraction, and prediction sampling procedures may be performed simultaneously, or any procedure may be performed before other procedures. For example, the prediction module of the image encoding apparatus may include a prediction mode determination module, a motion vector extraction module, and a prediction sample extraction module. The prediction mode determining module may determine the prediction mode of the current block, the motion vector extractor may extract the motion vector of the current block, and the prediction sample extractor may extract the prediction samples of the current block. For example, the encoder predictor may search for a block similar to the current block in the reconstructed area (or a specific area (search area) of the recovered area) of the current frame by block-by-block matching (BM), and extract a reference block whose difference relative to the current block is or less than a certain criterion or minimum. The motion vector may be derived based on the offset difference between the reference block and the current block. The encoder may determine the mode applied to the current block from various prediction modes. The encoder can compare RD costs based on different prediction modes and determine the optimal prediction mode for the current block.

[155] For example, when applying the skip mode or merge mode to the current block, the encoder may construct the above-described merge candidate list and extract a reference block whose difference relative to the current block is equal to or less than a certain criterion or minimum from the reference blocks indicated by the possible merges. join options included in the list of possible join options. In this case, the combining candidate associated with the extracted reference block may be selected, and combining index information indicating the selected combining candidate may be generated and signaled to the decoder. Using the motion vector of the selected join candidate, the motion vector of the current block can be extracted.

[156] As another example, when applying the (A)MVP mode to the current block, the encoder may construct the above-described list of (A)MVP candidates and use the motion vector of the MVP candidate selected from among the MVP candidates, included in the list of possible (A)MVP options, as the MVP of the current block. In this case, for example, a motion vector indicating a reference block derived by the above-described motion estimation may be used as the motion vector of the current block, and the MVP candidate having the smallest difference with respect to the motion vector of the current block among the candidate MVP candidates may become the selected MVP option is possible. A motion vector difference (MVD), which is obtained by subtracting the MVP from the motion vector of the current block, can be extracted. In this case, information regarding the MVD may be signaled to the decoder.

[157] The encoder may derive residual samples based on the predictive samples (S32). The encoder may extract the residual samples through a comparison between the original samples of the current block and the predictive samples.

[158] The encoding device may encode image information including prediction information and residual information (S33). The encoding device may output the encoded image information in the form of a bit stream. The prediction information may include information on prediction mode information (eg, skip flag, merge flag, or mode index) and motion information as information associated with the prediction procedure. The motion vector information may include candidate selection information (eg, merge index, MVP flag, or MVP index), which is information for extracting the motion vector. In addition, the information regarding the motion vector may include information regarding the above-described MVD. In addition, the motion vector information may include information indicating whether L0 prediction, L1 prediction, or bi-prediction should be applied. The residual information is information about the residual samples. The residual information may include information regarding quantized transform coefficients for the residual samples.

[159] The output bitstream may be stored on a (digital) storage medium and transmitted to the decoder, or may be transmitted to the decoder via a network.

[160] Meanwhile, as described above, the encoder may generate a reconstructed frame (including the reconstructed samples and the reconstructed block) based on the reference samples and the residual samples. This is for extracting, by the encoding apparatus, a prediction result identical to that performed by the decoding apparatus, thereby improving encoding efficiency. Accordingly, the encoder may store the reconstructed frame (or the reconstructed samples and the reconstructed block) in a memory and use it as a reference frame for inter prediction. The in-loop filtering procedure may additionally be applied to the reconstructed frame as described above.

[161] Hereinafter, this document describes a procedure for decoding video/images based on IBC and a predictor in a decoding apparatus with reference to FIG. 11. The decoding device can perform an operation corresponding to the operation performed by the encoding device. The decoding apparatus may perform IBC prediction on the current block based on the received prediction information and extract prediction samples.

[162] In particular, the decoding apparatus may determine the prediction mode for the current block based on the received prediction information (S41). The decoding apparatus can determine which inter prediction mode is applied to the current block based on the prediction mode information in the prediction information.

[163] For example, it may be determined whether the merge mode or (A)MVP mode should be applied to the current block based on the merge flag. Alternatively, one of the various possible inter prediction modes may be selected based on the mode index. Possible inter prediction modes may include skip mode, merge mode, and/or (A)MVP mode, or may include various inter prediction modes described below.

[164] The encoding device may derive a motion vector of the current block based on the determined prediction mode (S42). For example, when applying the skip mode or the combining mode to the current block, the decoding device may construct the above-described combining candidate list and select one of the combining modes included in the combining candidate list. The selection may be performed based on the above-described candidate selection information (combination index). The motion vector of the current block may be extracted using the motion vector of the selected join candidate. The motion vector of the selected join candidate may be used as the motion vector of the current block.

[165] As an example, when applying the (A)MVP mode to the current block, the decoder may construct a list of (A)MVP candidates and use the motion vector of the MVP candidate selected from among the MVP candidates included in a list of possible (A)MVP options, as the MVP of the current block. The selection may be performed based on the candidate selection information (MVP flag or MVP index) described above. In this case, the MVD of the current block may be derived based on information regarding the MVD, and the motion vector of the current block may be derived based on the MVP and MVD of the current block.

[166] Meanwhile, as described above, the motion information of the current block can be extracted without configuring the candidate list, in which case the motion vector of the current block can be extracted according to the procedure disclosed in the corresponding prediction mode. In this case, the above construction of lists of options may be omitted.

[167] The decoding apparatus may generate predictive samples for the current block based on the motion vector for the current block (S720). The prediction samples of the current block may be derived using the reference block samples indicated by the motion vector of the current block in the current frame. In this case, the filtering procedure of the prediction samples for all or some of the prediction samples of the current block may further be performed.

[168] For example, the prediction module of the decoding device may include a prediction mode determination module, a motion vector extraction module, and a prediction sample extraction module, the prediction mode determination module may determine the prediction mode for the current block based on the received prediction mode information, the motion vector extraction module may extract the motion vector of the current block based on the received motion vector information, and the prediction sample extractor may extract the prediction samples of the current block.

[169] The image decoding apparatus may generate residual samples of the current block based on the received residual information (S44). The image decoding apparatus may generate reconstructed samples for the current block based on the predictive samples and the residual samples, and generate a reconstructed frame based on them (S45). Thereafter, the in-loop filtering procedure may further be applied to the reconstructed frame.

[170] General representation of signaling transformation information

[171] In an embodiment, information for transforming/inverting the current block may be signaled from an encoder to a decoder via a bitstream. Referring to FIG. 12, as described above, the coding device can generate a prediction block, which is a block of prediction samples predicted through prediction in the inter-, intra-, and IBC mode (S110), and extract the residual block(s) for the block to be encoded, based on the prediction block (S120). The encoder may extract a transform block consisting of the quantized transform coefficients by applying transform and quantization to the extracted residual block (S130). The encoder may generate prediction mode information indicating prediction modes (internal, intra, and IBC modes) used to encode the current block, transform information used to extract the transform block, and encoding information for quantized transform coefficients, constructing a conversion block as a bitstream (S140). Then, the encoder may transmit the generated bit stream to the decoder (S150).

[172] The decoding apparatus may receive the bit stream (S160) and obtain prediction mode information, transform information, and encoding information for the quantized transform coefficients from the bit stream. The decoding apparatus may decode the encoding information for the transform coefficients and extract a transform block consisting of the quantized transform coefficients (S170). The decoding apparatus may extract the residual block through dequantization/inverse transform based on the transform block (S180). Then, the decoding apparatus may generate a prediction block based on the prediction information obtained from the bit stream (S190). Then, the decoding apparatus may generate a reconstructed block for the current block based on the prediction block and the residual block (S195). Meanwhile, at least one of quantization/dequantization and/or transformation/inverse transformation may be omitted.

преобразования, чтобы сформировать выборки y[i] остаточных блоков для блока x[j] преобразования. Здесь, j может иметь значение в 0 - nonZeroS-1; nonZeroS указывает горизонтальный размер выборки ненулевых квантованных коэффициентов преобразования; i может иметь размер в 0 - nTbS-1; nTbS указывает горизонтальный размер выборки остаточных выборок.[173] The encoding device and the decoding device according to the embodiment may select a transformation kernel to perform transformation (inverse transformation). The transformation kernel may be referred to by various terms such as transformation matrix, transformation type, or transformation function. An encoder may generate a transformation block by transforming a residual block based on a selected transformation kernel, and a decoding device may generate a residual block by inversely transforming a transformation block based on a selected transformation kernel. The following equation shows an embodiment in which the decoding device applies the matrix

transforms to form residual block samples y[i] for transform block x[j]. Here, j can have a value of 0 - nonZeroS-1; nonZeroS indicates the horizontal sample size of non-zero quantized transform coefficients; i can have a size of 0 - nTbS-1; nTbS indicates the horizontal sample size of the residual samples.

[174] Equation 1

, при i=0...nTbS-1y[i]=

, for i=0...nTbS-1

[175] The transformation kernel used when the decoding device performs the inverse transformation may be inversely related to the transformation kernel used when the encoding device performs the transformation. For example, when the transformation kernel consists of a transformation matrix, the transformation matrix used when the decoding device performs the inverse transformation may be the inverse of the transformation matrix used when the encoding device performs the transformation. When the transformation core is composed of a transformation function, the transformation function used when the decoding device performs inverse transformation may be the inverse function of the transformation function used when the encoding device performs transformation.

[176] An encoder and a decoder according to an embodiment may perform multiple transform selection (MTS) to select a core to transform (inverse transform) the current block from a plurality of transform cores. This is described in more detail below with reference to FIG. 13 and 14. The encoder or decoder may select any transform kernel from the plurality of transform kernels to perform the transform or inverse transform. In an embodiment, the coding device may apply a plurality of transform kernels to the residual block and then select a transform kernel with the highest transform efficiency as a transform kernel for transforming the residual block (S131). Here, the conversion kernel with the highest conversion efficiency may be selected for RD cost comparison. In addition, the encoding device may generate a transformation block as a result of applying the selected transformation kernel to the residual block (S132). Finally, the encoding device may output transformation information for signaling the transformation core used for transformation to the decoding device and information regarding the transformation block as a bitstream (S133). Here, the transformation information may include at least one of information indicating whether or not the MTS is executed according to the prediction mode, information indicating the core applied to the MTS, or information indicating whether or not this information is included in the bitstream.

[177] In response to this, the decoding apparatus can obtain transformation information and information regarding the transformation block from the bitstream to perform inverse transformation of the transformation block, and generate a residual block corresponding to it (S181). In addition, the decoding apparatus may select a transformation core for inverse transformation of the current transformation block based on the transformation information (S182). In addition, the decoding apparatus may generate a residual block for the current transformation block using the selected transformation kernel (S183).

[178] As described above, since any transform kernel is selected from a plurality of transform kernels according to the transform efficiency and used, the encoding device can signal information regarding the transformation kernel used for transformation to the decoding device, and the decoding device can perform inverse transformation with using a transformation kernel corresponding to the transformation kernel used for transformation by the encoding device.

[179] Meanwhile, to perform the MTS, the transformation core may be determined by selecting some of the plurality of sets of transformation cores. For example, a set of transformation kernels may consist of a combination of a vertical transformation kernel and a horizontal transformation kernel. In addition, as index information for selecting a set of transformation cores, MTS index information, which is index information of a transformation core, can be used.

[180] Accordingly, the encoding device according to the embodiment can signal the MTS index information to the decoding device as information for enabling the decoding device to select a set of transformation cores. For example, the MTS index information may be generated by the encoder and signaled to the decoder to indicate one of the sets of transform kernels. In an embodiment, a set of transform kernels may be signaled as index information indicating a vertical transform kernel and index information indicating a horizontal transform kernel. The following table shows an embodiment of a set of transform kernels according to the value of the MTS index information.

[181] Tab. 2

tu_mts_idx[x0][y0] 0 1 2 3 4 trTypeHor 0 1 2 1 2 trTypeVer 0 1 1 2 2

[182] Here, trTypeHor may indicate a horizontal transformation kernel, and trTypeVer may indicate a vertical transformation kernel. In an embodiment, a value of 0 in trTypeHor and trTypeVer according to the above table may indicate DCT2, a value of 1 in trTypeHor and trTypeVer may indicate DST7, and a value of 2 in trTypeHor and trTypeVer may indicate DCT8. However, this is an example, different DCTs or DSTs can be converted to the values indicated by trTypeHor and trTypeVer. Fig. 15 shows an example of basis functions for DCT2, DCT8 and DST7.

[183] Embodiment 1

[184] Hereinafter, this document describes the signaling of the MTS indexes of the encoder according to the embodiment and the demapping process by the decoder, respectively, with reference to FIG. 16-18. Fig. 16 is a flowchart illustrating a method for encoding a current block using an MTS by an encoding device according to an embodiment. The encoder according to the embodiment may determine whether or not the MTS index signaling is transmitted via the bit stream to encode the current block (S210). For example, the encoder may determine that transmission in the MTS-index signaling is possible to encode the current block when the flag information included in the Sequence Parameter Set (SPS) indicates whether or not transmission in the MTS-index signaling is possible. index. For example, the set of sequence parameters may include first flag information indicating whether or not tu_mts_idx may be included in the bitstream when the current coding mode is an external mode, and second flag information indicating whether or not tu_mts_idx may be included in the bitstream. when the current encoding mode is the intra mode. Therefore, the encoder can determine whether or not transmission in the MTS index signaling is possible according to the encoding mode of the current block.

[185] Here, the first flag may be sps_explicit_mts_intra_enabled_flag, which is a parameter indicating whether or not tu_mts_idx is present in the syntax of intra coding units. For example, when tu_mts_idx is present in the syntax of intra coding units, sps_explicit_mts_intra_enabled_flag may have a value of 1. Meanwhile, when tu_mts_idx is not present in the syntax of intra coding units, sps_explicit_mts_intra_enabled_flag may have a value of 0.

[186] The second flag may be sps_explicit_mts_inter_enabled_flag, which is a parameter indicating whether or not tu_mts_idx is present in the syntax of the outer coding units. When tu_mts_idx is present in the syntax of outer encoding units, sps_explicit_mts_intra_enabled_flag may have a value of 1. When tu_mts_idx is not present in the syntax of outer encoding units, sps_explicit_mts_intra_enabled_flag may have a value of 0.

[187] Then, the encoder may perform transformation of the current block for each of the plurality of transformation cores when transmission in the MTS index signaling is possible (S221). For example, the encoder may generate a transformation block by transforming the residual block for the current block for each of the plurality of cores.

[188] Then, the encoder may determine whether or not to apply a transformation to perform encoding of the current block based on the result of performing the transformation of the current block for each of the plurality of cores (S222). For example, the encoder may determine whether or not to apply a transform to perform encoding by comparing the encoding efficiency in case no transform is performed and in case in which transform is performed for each transform kernel.

[189] Then, when signaling of the MTS index is possible and the current block is encoded based on the transform, the encoder outputs the MTS index of the transform core used for the transform as the value of tu_mts_idx as the bitstream, and outputs 0 to as a value of transform_skip_flag, which is a parameter indicating whether or not a transformation is skipped, as a bitstream (S223). Therefore, the encoder can signal information regarding the transformation kernel used for transformation to the decoder.

[190] Meanwhile, when the current block is not encoded based on the transform, the encoding device may not output the value of tu_mts_idx as the bitstream, and may output 1 as the value of the transform_skip_flag, which is a parameter indicating whether or not a transform is skipped, as bitstream (S224). Therefore, the encoder can signal, to the decoder, information indicating that the transformation of the current block is not performed and skipped.

[191] Referring to step S210 again, when transmission in the MTS index signaling is impossible, the encoding device may perform transformation of the current block using a predetermined transformation kernel (S231). For example, the encoder may generate a transformation block by transforming the residual block for the current block using a predetermined transformation kernel. For example, the encoder may use DCT2 as the horizontal transform kernel and use DCT2 as the vertical transform kernel. Alternatively, the encoder may use DST7 as the horizontal transform kernel and use DST7 as the vertical transform kernel. Alternatively, the encoder may use DCT8 as the horizontal transform kernel and use DCT8 as the vertical transform kernel.

[192] Then, the encoding device may determine whether or not to apply a transformation to perform encoding of the current block based on the result of performing the transformation of the current block using a predetermined transformation kernel (S232). For example, the encoding device may determine whether or not to apply a transformation to perform encoding by comparing the encoding efficiency in case no transformation is performed and in case transformation is performed.

[193] Then, when the current block is encoded based on the transform, the encoding device may not output the value of tu_mts_idx as the bitstream, and may output 0 as the value of transform_skip_flag, which is a parameter indicating whether or not the transformation is skipped, as the bitstream (S233). Therefore, the encoding device can signal information about the transformation kernel to the decoding device by causing the decoding device to retrieve information about the transformation kernel used for transformation.

[194] Meanwhile, when the current block is not encoded based on the transform, the encoding device may not output the value of tu_mts_idx as the bitstream, and may output 1 as the value of the transform_skip_flag, which is a parameter indicating whether or not a transform is skipped, as bitstream (S234). Therefore, the encoder can signal, to the decoder, information indicating that the transformation of the current block is not performed and skipped.

[195] FIG. 17 illustrates a method for decoding a current block using an MTS by a decoder according to an embodiment. The decoding apparatus according to the embodiment can determine whether or not the coding mode of the current block is an intra mode, and whether or not an additional condition is satisfied (S310). Here, a further condition may be whether or not the first flag indicates whether or not tu_mts_idx is included in the bitstream when the current coding mode is an outer mode. For example, when the coding mode of the current block is not an intra mode, and the first flag indicates whether or not tu_mts_idx is included in the bitstream when the current coding mode is an inter mode, the decoding apparatus can obtain tu_mts_idx from the bitstream (S330).

[196] Otherwise, the decoding apparatus according to the embodiment determines whether or not the coding mode of the current block is an intra mode, whether or not the additional condition is satisfied (S320). Here, a further condition may be whether or not the second flag obtained from the bitstream indicates whether or not tu_mts_idx is included in the bitstream when the current coding mode is an intra mode. For example, when the coding mode of the current block is an intra mode, and the second flag indicates whether or not tu_mts_idx is included in the bitstream when the current coding mode is an intra mode, the decoding apparatus can obtain tu_mts_idx from the bitstream (S330).

[197] Meanwhile, when the condition of step S320 is not satisfied, the decoding apparatus may not obtain the tu_mts_idx of the current block (S340). In addition, the decoder may not obtain the tu_mts_idx of the current block when the value of the transform_skip_flag obtained from the bitstream indicates that no transformation of the current block is performed.

[198] Here, the order of steps S210 and S320 may be changed. For example, the encoder may determine whether or not the encoding mode of the current block is an intra mode and the additional condition is satisfied (S320), and then may determine whether or not the encoding mode of the current block is an external mode and the additional condition is satisfied (S310 ) according to the result.

[199] The decoding device may determine whether a predetermined transformation kernel is used or a transformation kernel is selected according to tu_mts_idx to encode the current block. For example, the decoding device may determine that a predetermined transformation kernel is used when the value of sps_explicit_mts_intra_enabled_flag is 0, the value of sps_explicit_mts_inter_enabled_flag is 0, and the encoding mode of the current block is an intra mode. Meanwhile, when such conditions are not satisfied, the decoding apparatus may determine that the transformation core is selected according to tu_mts_idx.

[200] When the current block is decoded using a predetermined transformation kernel, the decoding device may generate a residual block for the current block by inversely transforming the transformation block of the current block using the predetermined transformation kernel regardless of the value of tu_mts_idx. For example, the decoding device may use DCT2 as the horizontal transform kernel and use DCT2 as the vertical transform kernel. Alternatively, the decoding device may use DST7 as a horizontal transform kernel and use DST7 as a vertical transform kernel. Alternatively, the decoder may use DCT8 as the horizontal transform kernel and use DCT8 as the vertical transform kernel.

[201] Meanwhile, when the current block is decoded according to tu_mts_idx, the decoder may generate a residual block for the current block by inverting the transformation block of the current block using a transformation kernel selected according to Table 2 and the value of tu_mts_idx obtained from the bitstream. Fig. 18 illustrates part of a syntax structure for obtaining TU encoding information by parsing, by a decoder, a bitstream generated by encoding an image by an encoder according to an embodiment, in the examples of FIG. 16 and 17.

[202] As described above, the encoder may include in the bitstream tu_mts_idx, which is an MTS index parameter indicating the horizontal and vertical transform kernels applied to the luma transform block of the current coding unit; tu_mts_idx may indicate a value for setting the value of trTypeHor and trTypeVer as described above with reference to Table 2.

[203] The encoder may signal the tu_mts_idx to the decoder via the bitstream only under a specific condition. For example, the encoder may determine whether or not tu_mts_idx is signaled based on the prediction mode of the current block and the values of the first flag and the second flag. Here, the first flag may be sps_explicit_mts_intra_enabled_flag, which is a parameter indicating that tu_mts_idx is present in the syntax of the intra coding units. In addition, the second flag may be sps_explicit_mts_inter_enabled_flag, which is a parameter indicating whether or not tu_mts_idx is present in the syntax of the outer coding units.

[204] The encoder may include information indicating whether or not sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are transmitted to the bitstream to reduce the number of bits of the bitstream. For example, the encoder may include third flag information in the bitstream as a parameter indicating whether or not two flags are included in the bitstream. For example, the third flag may be sps_mts_enabled_flag.

[205] For example, sps_mts_enabled_flag may indicate whether or not sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are present in the RBSP syntax of the sequence parameter set. For example, the value of sps_mts_enabled_flag may be set to 1 to indicate that sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are present in the RBSP syntax of the sequence parameter set. Meanwhile, the value of sps_mts_enabled_flag may be set to 0 to indicate that sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are not present in the RBSP syntax of the sequence parameter set.

[206] In an embodiment, the decoding device may set the value of sps_explicit_mts_intra_enabled_flag to 0 when sps_explicit_mts_intra_enabled_flag is not obtained from the bit stream. In addition, when sps_explicit_mts_intra_enabled_flag is not obtained from the bitstream, the value of sps_explicit_mts_intra_enabled_flag may be set to 0.

[207] This is again described next with reference to FIG. 18. According to the third IF statement 300 shown in FIG. 18 for the transformation unit syntax, when the prediction mode of the current block is not an intra mode and tu_mts_idx may be present in the syntax of outer coding units, or when the prediction mode of the current block is an intra mode and tu_mts_idx may be present in the syntax of intra coding units, the decoder may receive the tu_mts_idx parameter from the bitstream.

[208] Meanwhile, when the third IF statement 300 is not true, the decoder does not receive the tu_mts_idx parameter from the bit stream. In this case, the decoding device may set a predetermined set as the value of tu_mts_idx. For example, when tu_mts_idx is not obtained from the bitstream, the decoder may set the value of tu_mts_idx to 0. Accordingly, the decoder may set the value of trTypeHor and trTypeVer to 0, and determine that DCT2 is used as the transformation core for the current block according to trTypeHor and trTypeVer, as described above with reference to table 2.

[209] Embodiment 2

[210] Meanwhile, when an image is encoded by performing MTS as described above, compression efficiency is improved in that an optimal transform kernel is found and applied to a transform block. However, when the MTS is performed, since the transformation kernel is determined for each target block, the coding complexity may increase significantly. Therefore, the encoding apparatus according to the embodiment can perform transformation by selecting any transformation kernel from a plurality of transformation kernels, and output an MTS index indicating the selected transformation kernel as a bitstream when the encoding mode of the current block is an outer or inner mode, but can perform converting using a predetermined transform kernel when the coding mode of the current block is the IBC mode, thereby reducing the coding complexity. In addition, in this case, by using a predetermined transformation kernel, the MTS index may not be output as a bitstream.

[211] Hereinafter, this document describes the signaling of the MTS indexes of the encoder according to the embodiment and the demapping process by the decoder, respectively, with reference to FIG. 19-22. Fig. 19 is a flowchart illustrating the operation of an encoding apparatus according to an embodiment. First, the operation of the encoding device will be described with reference to FIG. 19. In FIG. 19, steps S410-S424 and S434-S436 correspond to steps S210-S224 and S232-S234 in FIG. 16, and therefore their detailed description is omitted. Hereinafter in this document, the focus is on steps S431-S433 of FIG. 19 which are different from FIG. 16.

[212] The encoder according to the embodiment can determine whether or not the encoding mode of the current block is IBC mode (S431) when no transmission in the MTS index signaling is performed (S410). The encoding device can generate a transformation block by performing transformation of the current residual block using the first transformation kernel when the encoding mode of the current block is IBC mode (S432). In an embodiment, the encoder may not perform transformation by selecting an optimal transformation kernel from a plurality of transformation kernels, but may perform transformation using a predetermined first transformation kernel. For example, the encoder may use DCT2 as the horizontal transform kernel and use DCT2 as the vertical transform kernel. Alternatively, the encoder may use DST7 as the horizontal transform kernel and use DST7 as the vertical transform kernel. Alternatively, the encoder may use DCT8 as the horizontal transform kernel and use DCT8 as the vertical transform kernel.

[213] Meanwhile, the encoding device can generate a transformation block by performing transformation of the current residual block using the second transformation kernel (S433) when the encoding mode of the current block is not the IBC mode. In an embodiment, when the encoding mode of the current block is an outer or inner mode, the encoding device may perform transformation using a predetermined second transformation kernel. For example, an encoder may use DCT2 as a horizontal transform kernel and use DCT2 as a vertical transform kernel. Alternatively, the encoder may use DST7 as the horizontal transform kernel and use DST7 as the vertical transform kernel. Alternatively, the encoder may use DCT8 as the horizontal transform kernel and use DCT8 as the vertical transform kernel.

[214] The encoding device according to the embodiment can preset the first transformation core and the second transformation core as specific transformation cores according to the transformation characteristics of the residual block generated in the IBC mode and the transformation characteristics of the residual block generated in the mode other than the IBC mode. (for example, in external or internal mode). The encoder according to the embodiment can equally define the first transformation kernel and the second transformation kernel. For example, the encoder may use DCT2 as a horizontal transform kernel and use DCT2 as a vertical transform kernel for both the second transform kernel and the second transform kernel.

[215] In another embodiment, the encoding device may define the first transformation kernel and the second transformation kernel differently. For example, the encoder may use DST7 as the horizontal transform kernel and use DST7 as the vertical transform kernel for the first transform kernel for IBC mode and use DCT2 as the horizontal transform kernel and use DCT2 as the vertical transform kernel for the second transform kernel for IBC mode. prediction other than the IBC mode. Then, the encoding device may determine whether or not to apply a transformation to encode the current block based on the result of performing the transformation of the current block using the predetermined transformation kernel as described above (S434), and perform processing accordingly. For example, when the current block is encoded based on the transform, the encoding device may not output the value of tu_mts_idx as a bitstream, and may output, as a bitstream, 0 as a value of transform_skip_flag, which is a parameter indicating whether or not a transformation is skipped (S435 ). Meanwhile, when the current block is not encoded based on the transform, the encoding device may not output the value of tu_mts_idx as the bitstream, and may output, in the bitstream, 1 as the value of the transform_skip_flag, which is a parameter indicating whether or not a transform is skipped ( S436).

[216] FIG. 20 illustrates a method for determining an MTS index by a decoder according to an embodiment. The decoding apparatus according to the embodiment can determine whether or not the coding mode of the current block is an inter mode, and the additional condition is satisfied (S510). Here, a further condition may be whether or not the first flag obtained from the bitstream indicates whether or not tu_mts_idx is included in the bitstream when the current coding mode is an outer mode. For example, when the coding mode of the current block is an inter mode, and the first flag indicates whether or not tu_mts_idx is included in the bitstream when the current coding mode is an inter mode, the decoding device can obtain tu_mts_idx from the bitstream (S520).

[217] Otherwise, the decoding apparatus according to the embodiment can determine whether or not the coding mode of the current block is an intra mode, and whether or not an additional condition is satisfied (S530). Here, a further condition may be whether or not the second flag obtained from the bitstream indicates whether or not tu_mts_idx is included in the bitstream when the current coding mode is an intra mode. For example, when the coding mode of the current block is an intra mode, and the first flag indicates whether or not tu_mts_idx is included in the bitstream when the current coding mode is an intra mode, the decoding device can obtain tu_mts_idx from the bitstream (S520).

[218] Meanwhile, otherwise, the decoding apparatus may extract the tu_mts_idx of the current block without obtaining it from the bitstream (S540). Meanwhile, as described with reference to FIG. 17, determinations as to whether or not the conditions in steps S510 and S520 are satisfied may be performed in a different order.

[219] Hereinafter, this document describes a method for extracting the tu_mts_idx of the current block by the decoder without obtaining it from the bitstream in more detail with reference to FIG. 21. The decoding apparatus according to the embodiment may determine whether or not the coding mode of the current block is the IBC mode (S541). The decoding apparatus according to the embodiment may set tu_mts_idx to the first value without obtaining tu_mts_idx from the bitstream when the coding mode of the current block is IBC mode (S542). The decoding apparatus may set tu_mts_idx to the second value without obtaining tu_mts_idx from the bitstream when the coding mode of the current block is not the IBC mode (S543).

[220] In the case of the example of Table 2, the decoding device may set the value of tu_mts_idx to a predetermined value of values 0-4. In the embodiment, when the encoder uses DCT2 as the horizontal transform kernel and uses DCT2 as the vertical transform kernel both in the case where the prediction mode of the current block is IBC mode and in the case where the prediction mode is not IBC mode , the decoder may set the value of tu_mts_idx to 0 regardless of whether or not the coding mode of the current mode is the IBC mode.

[221] In another embodiment, when the encoder uses DST7 as the horizontal transform kernel, and uses DST7 as the vertical transform kernel in case the prediction mode of the current block is IBC mode, and uses DCT2 as the horizontal transform kernel, and uses DCT2 as a vertical transform kernel, in case the current block prediction mode is not IBC mode, the decoding apparatus may set tu_mts_idx to 0 regardless of whether or not the current mode prediction mode is IBC mode.

[222] FIG. 22 illustrates part of a syntax structure for obtaining TU encoding information by parsing, by a decoder, a bitstream generated by encoding an image by an encoder according to an embodiment, in an execution of an encoder and a decoder.

[223] The third IF statement 400 of FIG. 22 indicates that the tu_mts_idx parameter is included in the bit stream when the current block prediction mode is an inter mode and tu_mts_idx may be present in the outer coding units syntax, or when the current block prediction mode is an intra mode and tu_mts_idx may be present in the units syntax internal coding. Therefore, the encoder can output the value of tu_mts_idx as a bitstream when the above conditions are satisfied. The decoder may obtain the value of tu_mts_idx from the bit stream when the above conditions are satisfied.

[224] Meanwhile, when the third IF statement 400 is not satisfied, the encoder does not output the value of tu_mts_idx as a bitstream. In addition, the decoder may not obtain the value of tu_mts_idx from the bitstream. In this case, the decoding device may set a predetermined value as the value of tu_mts_idx. For example, when tu_mts_idx is not obtained from the bitstream, the decoder may set the value of tu_mts_idx to 0. Accordingly, the decoder may set the values of trTypeHor and trTypeVer to 0, and determine that DCT2 is used as the transformation core for the current block according to trTypeHor and trTypeVer as described above with reference to Table 2.

[225] Meanwhile, when the coding mode of the current block is IBC, the decoding device may set the value of tu_mts_idx to 1. Accordingly, the decoding device may set the values of trTypeHor and trTypeVer to 1 and determine that DST7 is used as the transformation core for the current block , according to trTypeHor and trTypeVer, as described above with reference to Table 2.

[226] Hereinafter, a method for selecting a transform kernel based on an MTS index by a decoding apparatus according to an embodiment is described. Fig. 23 and 24 are a flowchart illustrating a transform kernel selection method by a decoding apparatus according to an embodiment. The decoding apparatus according to the embodiment may determine the transformation kernel depending on whether or not to extract the transformation kernel index.

[227] Hereinafter, this document describes a method for determining whether or not to extract a transformation kernel index by a decoding apparatus, with reference to FIG. 23. First, the decoding apparatus according to the embodiment may determine whether or not to extract the MTS index according to the value of sps_mts_enabled_flag (S610). The decoder according to an embodiment may determine that the MTS index is not retrieved when the value of sps_mts_enabled_flag is 0. Therefore, the decoder may set the implicitMtsEnabled value, which is a parameter indicating whether or not to retrieve the MTS index, to 0 ( S620).

[228] When the value of sps_mts_enabled_flag is 1, the decoding device according to the embodiment may determine additional conditions to determine whether or not to extract the MTS index (S630). For example, the decoder may determine whether or not the value of sps_explicit_mts_intra_enabled_flag is 0, whether or not the value of sps_explicit_mts_inter_enabled_flag is 0, and whether or not the coding mode of the current block is IBC mode (S630). When the additional conditions are satisfied, the decoding device may set the implicitMtsEnabled value to 1 (S640). Meanwhile, when the additional conditions are satisfied, the decoding apparatus may set the implicitMtsEnabled value to 0 (S650).

[229] Hereinafter, this document describes a method for determining a transformation kernel by a decoding device depending on whether or not to extract a transformation kernel index, with reference to FIG. 24.

[230] The decoding device according to the embodiment may determine whether or not to extract the MTS index, depending on whether or not the implicitMtsEnabled value is 1 (S710). When the value of implicitMtsEnabled is not equal to 1, the decoding apparatus may determine that a transformation kernel index value is not obtained, and determine the horizontal transformation kernel and the vertical transformation kernel by setting trTypeHor and trTypeVer according to the value of tu_mts_idx (S720).

[231] Meanwhile, when the implicitMtsEnabled value is 1, the decoding apparatus may determine the transformation core by determining additional conditions (S730). For example, the decoder may choose how to determine the transformation kernel, depending on whether the implicitMtsEnabled value is 1 or not, whether the sps_explicit_mts_intra_enabled_flag value is 0 or not, whether the sps_explicit_mts_inter_enabled_flag value is 0 or not, and whether or not the encoding mode of the current block is the IBC mode (S730).

[232] When the additional conditions are satisfied, the decoding apparatus may determine the horizontal transformation kernel and the vertical transformation kernel by extracting trTypeHor and trTypeVer according to the block width and height (S740). For example, when the width of the current block is greater than or equal to 4, the width of the current block is less than or equal to 16, and the height of the current block is greater than or equal to the width of the current block, the decoding apparatus may determine a horizontal transformation kernel by setting trTypeHor to 1. Meanwhile, otherwise, the decoding apparatus may determine the horizontal transformation kernel by setting trTypeHor to 0.

[233] In addition, when the height of the current block is greater than or equal to 4, the height of the current block is less than or equal to 16, and the width of the current block is greater than or equal to the height of the current block, the decoding apparatus may determine the horizontal transformation kernel by setting trTypeHor to 1. Meanwhile, otherwise, the decoder may determine the horizontal transformation kernel by setting trTypeHor to 0.

[234] For example, the decoding device may extract trTypeHor and trTypeVer according to the following equations. Here, nTbW denotes the width of the current box, and nTbH denotes the height of the current box.

[235] Equation 2

[236] trTypeHor=(nTbW>=4 andand nTbW<=16 andand nTbW<=nTbH)?1:0

[237] Equation 3

[238] trTypeVer=(nTbH>=4 andand nTbH<=16 andand nTbH<=nTbW)?1:0

[239] According to the above equations and Table 2 above, when the trTypeHor value of the current block is 1, the decoding device can use DST7 as the horizontal transformation kernel, and when the trTypeVer value is 1, the decoding device can use DST7 as the vertical transformation kernel. In addition, when the trTypeHor value of the current block is 0, the decoder may use DCT2 as the horizontal transform kernel, and when the trTypeVer value is 0, the decoder may use DCT2 as the vertical transform kernel. By extracting the horizontal transform kernel and the vertical transform kernel for the current block, coding efficiency can be improved.

[240] Meanwhile, when the conditions of step S730 are not satisfied, the decoding apparatus may determine that the transformation kernel index value is not obtained, and determine the horizontal transformation kernel and the vertical transformation kernel by setting trTypeHor and trTypeVer according to the value of tu_mts_idx (S720).

[241] Although the exemplary methods of the present disclosure described above are presented as a sequence of operations for clarity of description, it is not intended to limit the order in which the steps are performed, and the steps may be performed simultaneously or in a different order as needed. In order to implement the method according to the present disclosure, the described steps may further include other steps, may include the remaining steps except for some steps, or may include other additional steps except for some steps.

[242] In the present disclosure, an image encoding device or an image decoding device that performs a predetermined operation (step) may perform an operation (step) of confirming a condition or situation of performing a corresponding operation (step). For example, if it is described that the predetermined operation is performed when the predetermined condition is satisfied, the image encoding apparatus or the image decoding apparatus may perform the predetermined operation after determining whether or not the predetermined condition is satisfied.

[243] The various embodiments of the present disclosure are not a list of all possible combinations and are intended to describe salient aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more of them.

[244] Various embodiments of the present disclosure may be implemented in hardware, firmware, software, or a combination of the above. If the present disclosure is implemented in hardware, the present disclosure may be implemented using Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGA), general processors, controllers, microcontrollers, microprocessors, etc.

[245] In addition, the image decoding apparatus and the image encoding apparatus to which embodiments of the present disclosure apply may be included in a multimedia broadcast transceiver, a mobile communication terminal, a home theater system video apparatus, a digital cinema system video apparatus, a surveillance camera, video chatting device, real-time communication device, for example, for video communication, mobile streaming device, storage media, video recording camera, video-on-demand (VoD) service delivery device, OTT video device ( video over networks), an Internet streaming service provider, a 3D video device, a video telephony system video device, a medical video device, and the like. and can be used to process video signals or data signals. For example, OTT video devices may include a game console, a Blu-ray player, an Internet access TV, a home theater system, a smartphone, a tablet PC, a digital video recorder (DVR), and the like.

[246] FIG. 25 is a view illustrating a content streaming system to which an embodiment of the present disclosure is applicable.

[247] Referring to FIG. 25, the content streaming system to which the embodiment(s) of the present document is applied may mainly include an encoding server, a streaming server, a web server, a media storage, a user device, and a media input device.

[248] The encoding server compresses content input from media input devices such as a smartphone, camera, camcorder, etc. into digital data to form a bit stream, and transmits the bit stream to the stream server. As another example, when media input devices such as smartphones, cameras, camcorder recorders, etc. directly generate a bitstream, the encoding server may be omitted.

[249] The bitstream may be generated by an image encoding method or an image encoding apparatus to which an embodiment of the present disclosure applies, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[250] The streaming server transmits media data to the user device based on the user's request through the web server, and the web server serves as a medium for informing the user about the service. When a user requests a desired service from a web server, the web server may deliver it to the streaming server, and the streaming server may transmit media data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server serves to manage the command/response between devices in the content streaming system.

[251] The streaming server may receive content from a media store and/or an encoding server. For example, when content is received from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may keep the bitstream for a predetermined time.

[252] Examples of a user device may include a mobile phone, smartphone, laptop computer, digital broadcast terminal, personal digital assistant (PDA), portable media player (PMP), navigation device, stylus tablet PC, tablet PCs, ultrabooks, wearable devices (e.g., smart watches, smart glasses, head-mounted displays), digital TVs, desktop computers, digital signage system, etc.

[253] Each server in the content streaming system may operate as a distributed server, in which case the data received from each server may be distributed.

[254] The operation according to the methods of the above-described embodiments may be performed on the device or computer by means of instructions (e.g., operating systems, applications, firmware, programs, etc.) executing on the device, or software executing on the device or computer. . Such software or instructions executable on a device may be recorded on a non-volatile computer-readable medium executable through a component of the device or computer. The computer-readable recording medium may include stand-alone or a combination of program instructions, data files, data structures, and so on. The program instructions recorded on a computer-readable recording medium may be specially designed and constructed for the present disclosure, or may be known to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic recording media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROM or DVD-ROM; magneto-optical media such as floppy disks; and hardware devices such as Read Only Memory (ROM), Random Access Memory (RAM), flash memory, etc., which are specifically designed to store and implement a program instruction. Examples of program instructions include not only mechanical language code formatted by a compiler, but also high-level language code that can be implemented by a computer using an interpreter. Hardware devices may be configured to operate as one or more software modules, or vice versa, in order to conduct processes according to the present disclosure.

[255] Although a specific technical idea for encoding and decoding a video or image is described by way of limited embodiments and drawings, they are provided only to help a more general understanding of the disclosure, and the present disclosure is not limited to the above embodiments. Those skilled in the art to which the present disclosure relates should appreciate that various modifications and changes may be made with respect to the above description.

[256] Therefore, the technical idea described in the present disclosure should not be limited to the above-described embodiments, and not only the claims described below, but also all modifications that are the same or equivalent to the claims, should fall within the scope of the disclosure. .

Industrial Applicability

[257] Embodiments of the present disclosure may be used to encode or decode an image.

1. An image decoding method performed by an image decoding device, wherein the image decoding method comprises the steps of:

determining a prediction mode of the current block based on the prediction mode information of the current block obtained from the bit stream;

- receive a block of transformation coefficients of the current block from the bit stream;

dequantizing the transform coefficient to form a block of dequantized transform coefficients;

- determine the transformation kernel for the block of dequantized transformation coefficients; And

- forming a residual block for the current block by performing an inverse transform on the block of dequantized transform coefficients using a transform kernel,

wherein, on the basis that the prediction mode of the current block is an inter prediction mode or an intra prediction mode, a transform kernel is determined based on the transform kernel index information obtained from the bit stream.

2. Method for decoding images according to claim 1,

- in which, on the basis that the prediction mode of the current block is neither the inter prediction mode nor the intra prediction mode, the transform kernel index information is not obtained from the bit stream and extracted as a predetermined value.

3. Method for decoding images according to claim 1,

- in which, on the basis that the prediction mode of the current block is an intra-block copy (IBC) mode that refers to a block located in the same frame as the current block and decoded before the current block, the transform kernel index information is not obtained from the bitstream and retrieved as a predefined value.

4. Method for decoding images according to claim 2,

- in which, on the basis that the prediction mode of the current block is an intra-block copy (IBC) mode, the transformation kernel index information is extracted as a value indicating the transformation kernel from DCT2 and DCT7.

5. Method for decoding images according to claim 1,

wherein determining the transform kernel for the block of dequantized transform coefficients comprises extracting the transform kernel of the current block based on the first flag information indicating whether or not the transform kernel index information of the current block encoded in intra prediction mode can be included in the bit stream. , and second flag information indicating whether or not transformation kernel index information of the current block encoded in the inter prediction mode can be included in the bitstream, and

wherein the first flag information and the second flag information are obtained from the bit stream.

6. Method for decoding images according to claim 5,

- in which, based on the fact that the first flag information and the second flag information have a first value indicating that the transformation core index information is not included in the bit stream, and the prediction mode of the current block is an intra-block copy (IBC) mode, the transformation core of the current block is retrieved based on the width and height of the current block.

7. Method for decoding images according to claim 6,

- in which the transformation kernel of the current block includes a horizontal transformation kernel and a vertical transformation kernel,

wherein the horizontal transformation kernel of the current box is determined based on whether or not the width of the current box is within a predetermined range, and based on a comparison between the width and height of the current box, and

wherein the vertical transformation kernel of the current box is determined based on whether or not the height of the current box is within a predetermined range, and based on a comparison between the width and height of the current box.

8. Method for decoding images according to claim 5,

- in which, on the basis that at least one of the first flag information and the second flag information has a second value indicating that the transform kernel index information can be included in the bitstream, and the prediction mode of the current block is an intra-block copy (IBC) ):

- the transformation kernel of the current block is extracted based on the transformation kernel index information, and

- transformation core index information is not obtained from the bit stream and is extracted as a predetermined value.

9. Method for decoding images according to claim 1,

- in which, on the basis that the transformation kernel index information is not obtained from the bitstream, the transformation kernel index information is extracted as a predetermined value,

whereby, based on the fact that the prediction mode of the current block is an intra-block copy (IBC) mode that refers to a block located in the same frame as the current block and decoded before the current block, the transform kernel index information is extracted into as the first value, and

wherein, on the basis that the prediction mode of the current block is a prediction mode other than the IBC mode, the transformation kernel index information is extracted as a second value different from the first value.

10. An image decoding device, comprising:

- Memory device; And

- at least one processor,

wherein at least one processor is configured to:

- determine the prediction mode of the current block based on the prediction mode information of the current block obtained from the bitstream,

- get a block of transformation coefficients of the current block from the bitstream,

- dequantize the transform coefficient to form a block of dequantized transform coefficients,

- determine the transform kernel for the block of dequantized transform coefficients, and

- generate a residual block for the current block by performing an inverse transform on the block of dequantized transform coefficients using a transform kernel,

wherein, on the basis that the prediction mode of the current block is an inter prediction mode or an intra prediction mode, a transform kernel is determined based on the transform kernel index information obtained from the bitstream.

11. An image encoding method carried out by an image encoding device, wherein the image encoding method comprises the steps of:

- select the mode of prediction of the current block;

- forming a prediction block for the current block based on the selected prediction mode;

- forming a residual block for the current block based on the prediction block;

selecting a transform kernel to transform the residual block based on the prediction mode;

transforming the residual block based on the transform kernel to form a block of transform coefficients;

- quantizing the block of transform coefficients to form a block of quantized transform coefficients; And

- encode a block of quantized transform coefficients to form a bitstream,

Wherein, on the basis that the prediction mode of the current block is an inter prediction mode or an intra prediction mode, transform kernel index information for a transform kernel used to transform the residual block is included in the bit stream.

12. The method of encoding images according to claim 11,

- in which, based on the fact that the prediction mode of the current block is an intra-block copy (IBC) mode:

- the transformation of the residual block is performed based on a predefined transformation kernel, and

- Transform kernel index information for a predetermined transform kernel used to transform the residual block is not included in the bitstream.

13. The method of encoding images according to claim 11,

- in which the first flag information indicating whether or not the transformation kernel index information of the current block coded in the intra prediction mode can be included in the bitstream, and the second flag information indicating whether or not the transformation kernel index information of the current block encoded in inter prediction mode, included in the bitstream, included in the bitstream,

wherein at least one of the first flag information and the second flag information has a second value indicating that the transform kernel index information can be included in the bitstream, and the current block prediction mode is an intra-block copy (IBC) ):

- the transformation of the residual block is performed based on a predefined transformation kernel, and

- Transform kernel index information for a predetermined transform kernel used to transform the residual block is not included in the bitstream.

14. The method of encoding images according to claim 13,

- in which, on the basis that the first flag information and the second flag information have a first value, indicating that the transformation kernel index information is not included in the bitstream, and the current block prediction mode is an intra-block copy (IBC) mode:

- the transformation of the residual block is performed on the basis of a predefined transformation kernel,

- a predefined transform kernel is retrieved based on the width and height of the current block, and

- Transform kernel index information for a predetermined transform kernel used to transform the residual block is not included in the bitstream.

15. A method for transmitting a bit stream generated by the image encoding method of claim 11.

Claims (68)

1. An image decoding device comprising storage device and at least one processor wherein said at least one processor is configured to: determining a prediction mode of the current block based on the prediction mode information of the current block obtained from the bitstream; obtaining a block of transform coefficients of the current block from the bitstream; dequantizing the transform coefficient to generate a block of dequantized transform coefficients; determining a transform kernel for a block of dequantized transform coefficients; And generating a residual block for the current block by performing an inverse transform on the block of dequantized transform coefficients based on the determined transform kernel, wherein the transformation kernel is determined based on the transformation kernel index information, transformation kernel index information is obtained from the bit stream based on whether the current block's prediction mode is an inter prediction mode or an intra prediction mode, and the transformation kernel index information is defined as a value indicating a predetermined transformation kernel without obtaining it from the bitstream based on the fact that the prediction mode of the current block is an intra-block copy (IBC) mode. 2. The image decoding device according to claim 1, in which the predefined transformation kernel is one of DCT2 and DCT7. 3. The image decoding device according to claim 1, wherein determining the transform kernel for the block of dequantized transform coefficients comprises extracting the transform kernel of the current block based on the first flag information indicating whether or not the transform kernel index information of the current block encoded in intra prediction mode can be included in the bitstream, and second flag information indicating whether or not transformation kernel index information of the current block encoded in the inter prediction mode can be included in the bitstream, and wherein the first flag information and the second flag information are obtained from the bitstream. 4. The image decoding device according to claim 3, wherein, on the basis that the first flag information and the second flag information have a first value indicating that the transformation kernel index information is not included in the bit stream, and the prediction mode of the current block is the IBC mode, the transformation kernel of the current block is derived based on the width and height of the current block. 5. The image decoding device according to claim 4, in which the transformation kernel of the current block includes a horizontal transformation kernel and a vertical transformation kernel, wherein the horizontal transformation kernel of the current box is determined based on whether or not the width of the current box is within a predetermined range, and a comparison between the width and height of the current box, and wherein the vertical transformation kernel of the current box is determined based on whether or not the height of the current box is within a predetermined range, and a comparison between the width and the height of the current box. 6. The image decoding device according to claim 3, in which, based on at least one of the first flag information or the second flag information has a second value indicating that the transform kernel index information can be included in the bitstream, and the prediction mode of the current block is the IBC mode, the transformation kernel of the current block is retrieved based on the transformation kernel index information, and the transformation kernel index information is not obtained from the bitstream, but is extracted as a predetermined value. 7. The image decoding device according to claim 1, in which, based on the fact that the transformation core index information is not obtained from the bit stream, the transformation core index information is extracted as a predetermined value, wherein, on the basis that the prediction mode of the current block is an IBC mode that refers to a block located in the same frame as the current block and decoded before the current block, the transform kernel index information is extracted as the first value, and wherein, on the basis that the prediction mode of the current block is a prediction mode other than the IBC mode, the transformation kernel index information is extracted as a second value different from the first value. 8. An image encoding device comprising storage device and at least one processor wherein said at least one processor is configured to: selecting a prediction mode for the current block; generating a prediction block for the current block based on the selected prediction mode; generating a residual block for the current block based on the prediction block; selecting a transform kernel to transform the residual block based on the prediction mode; transforming the residual block based on the selected transform kernel to generate a block of transform coefficients; quantizing the block of transform coefficients to generate a block of quantized transform coefficients; And encoding a block of quantized transform coefficients to form a bitstream, wherein the transform kernel index information of the transform kernel used to transform the residual block is encoded into the bitstream based on whether the prediction mode of the current block is an inter prediction mode or an intra prediction mode, and wherein the transformation kernel index information is defined as a value indicating a predetermined transformation kernel without encoding it into a bitstream based on the prediction mode of the current block being an intra-block copy (IBC) mode. 9. The image encoding device according to claim 8, in which the first flag information indicating whether or not the transformation kernel index information of the current block encoded in the intra prediction mode can be included in the bit stream, and the second flag information indicating whether or not the transformation kernel index information of the current block encoded in inter prediction mode, included in the bitstream, included in the bitstream, in which, based on at least one of the first flag information or the second flag information has a second value indicating that the transform kernel index information can be included in the bitstream, and the prediction mode of the current block is the IBC mode, the transformation of the residual block is performed based on a predefined transformation kernel, and the transformation kernel index information for the predetermined transformation kernel used to transform the residual block is not included in the bitstream. 10. The image encoding device according to claim 9, wherein, on the basis that the first flag information and the second flag information have a first value, indicating that the transform kernel index information is not included in the bitstream, and the prediction mode of the current block is the IBC mode: the transformation of the residual block is performed based on a predefined transformation kernel, a predefined transform kernel is retrieved based on the width and height of the current block, and the transformation kernel index information for the predetermined transformation kernel used to transform the residual block is not included in the bitstream. 11. An apparatus for transmitting a bit stream generated by an image encoding method performed by an image encoding apparatus, comprising storage device and at least one processor wherein at least one processor controls the device for transmitting the bit stream, wherein the image encoding method comprises: selection of the prediction mode of the current block; generating a prediction block for the current block based on the selected prediction mode; generating a residual block for the current block based on the prediction block; selecting a transform kernel for transforming the residual block based on the prediction mode; transforming the residual block based on the selected transform kernel to generate a block of transform coefficients; quantizing the block of transform coefficients to generate a block of quantized transform coefficients; And encoding a block of quantized transform coefficients to form a bit stream, wherein the transform kernel index information of the transform kernel used to transform the residual block is encoded into the bitstream based on whether the prediction mode of the current block is an inter prediction mode or an intra prediction mode, and wherein the transformation kernel index information is defined as a value indicating a predetermined transformation kernel without encoding it into a bitstream based on the prediction mode of the current block being an intra-block copy (IBC) mode.

Images