RU2795932C1 - Image encoding method based on transformation and device for its implementation - Google Patents

Abstract

FIELD: imagine encoding method.

SUBSTANCE: invention relates in particular to a method and a device for encoding an image based on transformation in an image coding system. According to the decoding method, the transformation coefficients for the current block are derived based on the residual information, determining whether there is a significant coefficient in the second area other than the first top-left area of the current block. The multiple transform selection index (MTS) is obtained by parsing the bitstream to identify if the significant coefficient is not in the second region. The residual sample is deduced by applying the MTS core derived based on the MTS index to the transform coefficients of the first region.

EFFECT: increase of the efficiency of video encoding.

12 cl, 23 dwg, 20 tbl

Description

TECHNOLOGY area

[1] The present disclosure relates to an image encoding method, and more specifically, to a method and apparatus for encoding an image based on a transform in an image encoding system.

Related technical field

[2] At present, the demand for images/videos of high definition and high quality, such as images/videos of ultra high definition (UHD) 4K or 8K or higher, has increased in various fields. Since the image/video data is of high resolution and high quality, the amount of information or the number of bits to be transmitted is increased compared to conventional image data. Therefore, when image data is transmitted using media such as a conventional wired/wireless broadband line, or image/video data is stored using an existing storage medium, its transmission cost and storage cost increase.

[3] In addition, there is currently an increasing interest and demand for immersive media such as virtual reality (VR) and artificial reality (AR) content or holograms or the like, broadcasting of images/videos having image features other than signs of a real image, such as a game image.

[4] Accordingly, there is a need for a high-performance image/video compression technology for efficiently compressing, transmitting or storing and reproducing high-definition and high-quality image/video information having various features as described above.

Brief description of the essence of the invention

[5] The technical aspect of the present disclosure is to provide a method and apparatus for improving image encoding efficiency.

[6] Another technical aspect of the present disclosure is to provide a method and apparatus for improving the coding efficiency of a transform index.

[7] Another technical aspect of the present disclosure is to provide a method and apparatus for encoding using LFNST and MTS.

[8] Another technical aspect of the present disclosure is to provide a picture encoding method and apparatus for signaling an LFNST index and an MTS index.

[9] According to an embodiment of the present disclosure, an image decoding method performed by a decoding apparatus is provided. The method may include: deriving transform coefficients for the current block based on the residual information; determining whether a significant coefficient is present in a second region other than the first top-left region of the current block; obtaining, by parsing, an MTS index from the bitstream when a significant coefficient is not present in the second region, deriving residual samples by applying the MTS kernel derived based on the MTS index to the transform coefficients of the first region.

[10] The first area is the top-left area 16×16 of the current block.

[11] Determining whether a significant coefficient is present in the second region comprises deriving a variable value indicating whether a significant coefficient is present in the second region in the residual coding level decoding process, the value of the variable is initially set to 1, and when the significant coefficient is present in the second region , the variable value is changed to 0 and the MTS index is not parsed.

[12] The MTS index is analyzed at the coding unit level.

[13] The image decoding method further comprises parsing with respect to the LFNST index indicating the LFNST core applied to the current block, and the LFNST index and the MTS index are signaled at the coding unit level, and the MTS index is signaled immediately after the signaling of the LFNST index.

[14] The three types of the current block are dual luma tree or single tree, and when the value of the LFNST index is 0, the MTS index is signaled.

[15] According to another embodiment of the present disclosure, an image encoding method performed by an encoding apparatus is provided. The method may include: deriving residual samples for the current block based on the prediction samples; deriving transform coefficients for the current block based on the MTS for the residual samples; zeroing the second area of the current block other than the first area from the top-left of the current block; and encoding the residual information output by quantizing the transform coefficients and an MTS index indicating an MTS core.

[16] According to still another embodiment of the present disclosure, a digital storage medium that stores image data including encoded image information and a bitstream generated in accordance with an image encoding method performed by an encoding device can be provided.

[17] According to another embodiment of the present disclosure, a digital storage medium that stores image data including encoded image information and a bitstream can be provided for causing a decoding apparatus to perform an image decoding method.

[18] According to the present disclosure, the overall image/video compression efficiency can be improved.

[19] According to the present disclosure, it is possible to improve the encoding efficiency of the transform index.

[20] A technical aspect of the present disclosure may provide a method and apparatus for encoding using LFNST and MTS.

[21] A technical aspect of the present disclosure may provide a method and apparatus for signaling an LFNST index and an MTS index.

[22] The results that can be obtained using specific examples of the present disclosure are not limited to the results listed above. For example, various technical results may be achieved that a person skilled in the art will be able to understand or obtain based on the present disclosure. Accordingly, the specific results of the present disclosure are not limited to those expressly described in the present disclosure and may include various results that can be understood or obtained using the technical features of the present disclosure.

Brief description of the drawings

[23] FIG. 1 schematically illustrates an example of a video/image coding system in which the present disclosure is applicable.

[24] FIG. 2 is a diagram schematically illustrating the configuration of a video/image coding apparatus to which the present disclosure is applicable.

[25] FIG. 3 is a diagram schematically illustrating the configuration of a video/image decoding apparatus to which the present disclosure is applicable.

[26] FIG. 4 schematically illustrates a multiple conversion scheme according to an embodiment of the present document.

[27] FIG. 5 schematically shows intra-directional modes of 65 prediction directions.

[28] FIG. 6 is a diagram for explaining RST according to an embodiment of the present disclosure.

[29] FIG. 7 is a diagram illustrating the layout sequence of the direct primary transform output into a one-dimensional vector according to an example.

[30] FIG. 8 is a diagram illustrating the layout sequence of direct secondary transform output into a two-dimensional block according to an example.

[31] FIG. 9 is a diagram illustrating wide-angle intra prediction modes according to an embodiment of the present document.

[32] FIG. 10 is a diagram illustrating a block shape to which LFNST is applied.

[33] FIG. 11 is a diagram illustrating the layout of the direct LFNST output according to the example.

[34] FIG. 12 is a diagram illustrating that the number of output data for direct LFNST is limited to a maximum of 16 according to an example.

[35] FIG. 13 is a diagram illustrating zeroing in a block to which 4×4 LFNST is applied according to an example.

[36] FIG. 14 is a diagram illustrating zeroing in a block to which 8×8 LFNST is applied according to an example.

[37] FIG. 15 is a diagram illustrating zeroing in a block to which 8×8 LFNST is applied according to another example.

[38] FIG. 16 is a diagram illustrating an example of a subblock into which one coding block is divided.

[39] FIG. 17 is a diagram illustrating another example of a subblock into which one coding block is divided.

[40] FIG. 18 is a diagram illustrating the symmetry of an M×2 (M×1) block and a 2×M (1×M) block according to an example.

[41] FIG. 19 is a diagram illustrating an example of transposing a 2×M block according to an example.

[42] FIG. 20 shows the scanning order for an 8x2 or 2x8 area according to an example.

[43] FIG. 21 is a flowchart for explaining an image decoding method according to an example.

[44] FIG. 22 is a flowchart for explaining an image encoding method according to an example.

[45] FIG. 23 illustrates the structure of a content streaming system to which the present disclosure applies.

Description of Exemplary Embodiments

[46] Although the present document is susceptible to various modifications and includes various embodiments, its specific embodiments shown in the drawings by way of example will be described in detail in the following. However, this is not intended to limit the present disclosure to the specific embodiments described herein. The terminology used here is only intended to describe specific embodiments, and is not intended to limit the technical idea of the present disclosure. Singular forms may include plural forms, unless the context clearly indicates otherwise. Terms such as "comprise" and "have" are intended to indicate that the features, numbers, steps, operations, elements, components, or combinations thereof used in the following description exist, and thus should not be construed as that the possibility of the existence or addition of one or more other features, numbers, steps, operations, elements, components, or a combination thereof is excluded in advance.

[47] Meanwhile, each component in the drawings described herein is illustrated independently for convenience of description regarding various characteristic functions, however, this does not mean that each component is implemented as separate hardware or software. For example, two or more of these components may be combined to form one component, and any single component may be subdivided into multiple components. Embodiments in which components are combined and/or separated will be included within the scope of the patent rights of this disclosure unless they deviate from the spirit of this disclosure.

[48] Next, the preferred embodiments of the present disclosure will be explained in more detail with reference to the accompanying drawings. Also, like reference numerals are used for like components in the drawings, and repeated descriptions of like components will be omitted.

[49] This document relates to video/image coding. For example, the method/example disclosed herein may refer to the VVC (Versatile Video Coding) standard (ITU-T Rec. H.266), the next generation video/image coding standard after VVC, or other standards related to video coding. (eg, HEVC (High Efficiency Video Coding) standard (ITU-T Rec. H.265), EVC (Essential Video Coding) AVS2 standard, etc.).

[50] Herein, various embodiments relating to video/image coding may be provided, and the embodiments may also be combined with each other and executed.

[51] As used herein, video may refer to a set of a series of images over time. A picture (picture) usually means a unit representing an image in a specific time zone, and a slice (slice)/tile is a unit constituting a part of a picture. A slice/tile may include one or more coding tree units (CTUs). One picture may consist of one or more clippings/tiles. One picture can consist of one or more groups of tiles. One group of tiles may include one or more tiles.

[52] A pixel or pel can mean the smallest unit that makes up one picture (or image). Also, ‘sampling’ (‘discrete sample’) can be used as a term corresponding to a pixel. The sample may generally represent a pixel or pixel value, and may represent only a luminance component pixel/pixel value, or only a chrominance component pixel/pixel value. Alternatively, the sample may refer to a pixel value in the spatial domain, or when that pixel value is converted to the frequency domain, it may refer to a transform factor in the frequency domain.

[53] The unit may represent a basic unit of image processing. The item may include at least one of a specific area and information related to the area. One unit may include one luminance block and two chrominance blocks (eg, cb, cr). A unit and a term such as block, area, or the like may be used interchangeably as the case may be. In general, an M×N block may include a set (or array) of samples (or arrays of samples) or transform coefficients, consisting of M columns and N rows.

[54] In this document, the characters "/" or "," shall be interpreted as indicating "and/or". For example, the expression "A/B" can mean "A and/or B". Additionally, "A, B" may mean "A and/or B". In addition, "A/B/C" may mean "at least one of A, B and/or C". Also, "A/B/C" can mean "at least one of A, B and/or C".

[55] In addition, in this document, the term "or" should be interpreted as indicating "and/or". For example, the expression "A or B" may include 1) only A, 2) only B, and/or 3) both A and B. In other words, the term "or" in this document is to be interpreted as indicating "in addition or alternatively" .

[56] In the present disclosure, "at least one of A and B" can mean "only A", "only B", or "both A and B". Also, in the present disclosure, the expression "at least one of A or B" or "at least one of A and/or B" can be interpreted as "at least one of A and B".

[57] In addition, in the present disclosure, at least one of A, B, and C" can mean "A only", "B only", "C only", or "any combination of A, B and C". Also, "at least one of A, B or C" or "at least one of A, B and/or C" can mean "at least one of A, B and C".

[58] In addition, parentheses used in this disclosure may mean "for example". Specifically, when "prediction (intra-prediction)" is indicated, it may mean that "intra-prediction" is offered as an example of "prediction". In other words, "prediction" in the present disclosure is not limited to "intra-prediction", and "intra-prediction" may be offered as an example of "prediction". Also, when "prediction (ie, intra-prediction)" is indicated, it also means that "intra-prediction" is offered as an example of "prediction".

[59] The technical features described separately in one drawing in the present disclosure may be implemented separately or may be implemented simultaneously.

[60] FIG. 1 schematically illustrates an example of a video/image coding system in which the present disclosure is applicable.

[61] With reference to FIG. 1, the video/image coding system may include a first device (source device) and a second device (receiver device). The source device may deliver the encoded information or video/image data in the form of a file or streaming (streaming) to the receiving device via a digital storage medium or a network.

[62] The source device may include a video source, an encoder, and a transmitter. The receiving device may include a receiver, a decoding device, and a rendering device. The encoding device may be referred to as a video/image encoding device, and the decoding device may be referred to as a video/image decoding device. The transmitter may be included in the encoder. The receiver may be included in the decoding device. The rendering device may include a display, and the display may be configured as a separate device or an external component.

[63] A video source may acquire a video/image through a video/image capture, synthesis, or generation process. The video source may include a video/image capture device and/or a video/image generation device. The video/image capture device may include, for example, one or more cameras, video/image archives including previously captured videos/images, and the like. The video/image generating device may include, for example, a computer, tablet, and smartphone, and may (electronically) generate video/images. For example, the virtual video/image may be generated via a computer or the like. In this case, the video/image capture process can be replaced with a related data generation process.

[64] The encoding device may encode the input video/image. The encoding device may perform a series of procedures such as prediction, transformation, and quantization for compression and encoding efficiency. The encoded data (encoded video/image information) may be output in the form of a bitstream.

[65] The transmitter may transmit the encoded information or video/image data output in the form of a bitstream to the receiver of the receiving device via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. The transmitter may include an element for generating a media file by a predefined file format and may include an element for transmission over a broadcast/communications network. The receiver may receive/retrieve the bitstream and transmit the received/extracted bitstream to a decoder.

[66] The decoding device can decode the video/image by performing a series of procedures such as dequantization, inverse transformation, and prediction corresponding to the operation of the encoding device.

[67] The rendering device may render the decoded video/image. The rendered video/image can be played through the display.

[68] FIG. 2 is a diagram schematically illustrating the configuration of a video/image coding apparatus in which embodiments of the present document may be applied. Further, what is referred to as a video encoding device may include an image encoding device.

[69] With reference to FIG. 2, encoder 200 includes an image splitter 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. Predictor 220 may include an inter predictor 221 and an intra predictor 222. The processor Residual processor 230 may include a transform 232, a quantizer 233, a dequantizer 234, and an inverse transform 235. Residual processor 230 may further include a subtractor 231. Adder 250 may be referred to as a reconstructor or reconstructed block generator. The image splitter 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 that have been described above may be formed by one or more hardware components (e.g., encoder chipsets or processors) according to an embodiment. In addition, memory 270 may include a decoded picture buffer (DPB) or may be configured with a digital storage medium. The hardware component may further include memory 270 as an internal/external component.

[70] The image splitter 210 may split the input image (or picture or frame) input to the encoding device 200 into one or more processing units. For example, a processing unit may be referred to as a coding unit (CU). In this case, starting from a coding tree unit (CTU) or a largest coding unit (LCU), a coding unit may be recursively partitioned according to a quadtree/binary tree/ternary tree (QTBTTT) structure. For example, one coding unit may be split into multiple deeper coding units based on a quadtree structure, a binary tree structure, and/or a ternary tree structure. In this case, for example, a quadtree structure may be applied first, and a binary tree structure and/or a ternary tree structure may be applied later. Alternatively, the binary tree structure may be applied first. An encoding procedure according to the present disclosure may be performed based on a final coding unit that is no longer split. In this case, the largest coding unit can be directly used as the final coding unit based on the coding efficiency according to the image characteristic. Alternatively, the coding unit may be recursively broken down into deeper coding units, as needed, such that the optimum size coding unit may be used as the final coding unit. Here, the encoding procedure may include a prediction, transformation, and recovery procedure to be described later. As an example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit may be separated or split from the aforementioned final coding unit. The prediction unit may be a sample prediction unit, and the transform unit may be a unit for deriving a transform coefficient and/or a unit for deriving a residual signal from a transform coefficient.

[71] The unit may be used interchangeably with terms such as block, area, or the like, as appropriate. In general, an M×N block may represent a set of samples or transform coefficients consisting of M columns and N rows. The sample may generally represent a pixel or pixel value, may represent only a luminance component pixel/pixel value, or represent only a chrominance component pixel/pixel value. Sampling can be used as a term corresponding to a single picture (or image) for a pixel or pel.

[72] The subtractor 231 subtracts the prediction signal (predicted block, prediction sample array) output from the inter predictor 221 or intra predictor 222 from the input image signal (original block, original sample array) to generate a residual signal (residual block, residual array of samples), and the generated residual signal is provided to the transformer 232. The predictor may perform prediction on a target processing block (hereinafter referred to as the current block), and may generate a predicted block including the prediction samples for the current block. The predictor may determine whether intra-prediction or inter-prediction is applied on the current block or on a CU basis. As discussed further in the description of each prediction mode, the predictor may generate various prediction-related information, such as prediction mode information, and pass the generated information to entropy encoder 240. The prediction information may be encoded in entropy encoder 240 and output in the form of a bitstream.

[73] Intra-predictor 222 may predict the current block by referring to the samples in the current picture. These samples may be adjacent or spaced apart from the current block according to the prediction mode. In intra-prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. Non-directional modes may include, for example, DC mode and planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the granularity of the prediction direction. However, this is only an example, and more or less directional prediction modes may be used depending on the setting. The intra-predictor 222 may determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[74] The inter predictor 221 may output a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector in the reference picture. Here, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted on a block, a sub-block, or a sample based on the correlation of the motion information between the adjacent block and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block existing in the current picture and a temporal neighbor block existing in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighbor block may be the same or different. The temporal neighbor block may be referred to as a co-located reference block, co-located CU (colCU) or the like, and a reference picture including the temporal neighbor block may be referred to as a colPic. For example, the inter predictor 221 may configure a list of motion information candidates based on adjacent blocks and generate information indicating which candidate is used to derive a motion vector and/or a reference picture index of the current block. Inter-prediction may be performed based on various prediction modes. For example, in the case of the skip mode and the combine mode, the inter predictor 221 may use the motion information of the neighboring block as the motion information of the current block. In the skip mode, unlike the combine mode, the residual signal may not be transmitted. In the case of a motion vector prediction (MVP) mode, the motion vector of an adjacent block may be used as a motion vector predictor, and the motion vector of the current block may be indicated by motion vector difference signaling.

[75] Predictor 220 may generate a prediction signal based on various prediction methods described below. For example, the predictor may not only apply intra-prediction or inter-prediction for prediction on one block, but may also apply intra-prediction and inter-prediction at the same time. This may be referred to as combined inter- and intra-prediction (CIIP). In addition, the predictor may be based on an intra-block copy (IBC) prediction mode or a palette mode to perform prediction on a block. An IBC prediction mode or a palette mode may be used for encoding image/video game content or the like, for example, screen content coding (SCC). Although IBC basically performs prediction in the current block, it can be performed similarly to inter-prediction in that it outputs a reference block in the current block. That is, the IBC may use at least one of the inter prediction methods described in this disclosure.

[76] The prediction signal generated by inter-predictor 221 and/or intra-predictor 222 may be used to generate a recovered signal or to generate a residual signal. Transformer 232 may generate transform coefficients by applying a transform method to the residual signal. For example, the transform method may include at least one of discrete cosine transform (DCT), discrete sine transform (DST), Karhunen-Loeve transform (KLT), graph-based transform (GBT), or conditionally non-linear transform (CNT) . Here, GBT means a transformation obtained from a graph when relationship information between pixels is represented by a graph. CNT refers to a transform derived from a prediction signal generated using all previously reconstructed pixels. Also, the transformation process may be applied to square blocks of pixels having the same size, or may be applied to blocks having a variable size other than square.

[77] The quantizer 233 may quantize the transform coefficients and pass them to the entropy encoder 240, and the entropy encoder 240 may encode the quantized signal (quantized transform coefficient information) and output the encoded signal in a bitstream. Information about the quantized transform coefficients may be referred to as residual information. The quantizer 233 may reorder the quantized block-type transform coefficients into a one-dimensional vector form based on the scan order of the coefficients, and generate information about the quantized transform coefficients based on the quantized transform coefficients in the form of a one-dimensional vector. Entropy encoder 240 may perform various coding techniques such as exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like, for example. Entropy encoder 240 may encode information needed for video/image reconstruction other than quantized transform coefficients (eg, syntax element values, etc.) together or separately. Encoded information (eg, encoded video/image information) may be transmitted or stored in units of a network abstraction layer (NAL) in the form of a bitstream. The video/image information may further include information about various parameter sets such as adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), video parameter set (VPS), and the like. In addition, the video/image information may further include general restriction information. In the present disclosure, information and/or syntax elements transmitted/signaled from an encoder to a decoder may be included in video/image information. The video/image information may be encoded by the above-described encoding procedure and included in the bitstream. The bitstream may be transmitted over a network or may be stored in a digital storage medium. The network may include a broadcast network, a communications network, and/or the like, and a digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. A transmitter (not shown), a transmitting signal outputted from the entropy encoder 240, and/or a storage module (not shown) storing the signal may be configured as an internal/external element of the encoder 200, or a transmitter may be included in the entropy encoder 240.

[78] The quantized transform coefficients output from the quantizer 233 may be used to generate a prediction signal. For example, the residual signal (residual block or residual samples) may be recovered by applying dequantization and inverse transform to the quantized transform coefficients by dequantizer 234 and inverse transform 235. Summer 250 adds the recovered residual signal to the prediction signal output from inter-predictor 221 or intra-predictor 222 so that a reconstructed signal (reconstructed picture, reconstructed block, reconstructed array of samples) can be generated. If there is no residual for the target processing block, as in the case where the skip mode is applied, the predicted block may be used as the reconstructed block. Adder 250 may be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal may be used to intra-predict the next block to be processed in the current picture and, as described below, may be used to inter-predict the next picture by filtering.

[79] Meanwhile, luma mapping with chrominance scaling (LMCS) may be applied in the encoding and/or picture reconstruction process.

[80] Filter 260 can improve subjective/objective video quality by applying filtering to the reconstructed signal. For example, filter 260 may generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and may store the modified reconstructed picture in memory 270, specifically in DPB memory 270. Various filtering methods may include, for example, deblocking filtering, adaptive offset sampling, adaptive loop filter, bidirectional filter, and the like. The filter 260 may generate various filtering-related information and pass the generated information to the entropy encoder 240, as described later in the description of each filtering method. The filtering information may be encoded by entropy encoder 240 and output in the form of a bitstream.

[81] The modified reconstructed picture that has been transferred to the memory 270 can be used as a reference picture in the inter predictor 221. Through this, the encoding device can prevent prediction inconsistency in the encoding device 200 and the decoding device when inter prediction is applied, and can also improve coding efficiency.

[82] The DPB of the memory 270 may store the modified reconstructed picture for use as a reference picture in the inter predictor 221. The memory 270 may store the motion information of the block in the current picture from which motion information has been derived (or encoded) and/or movement of blocks in the already restored picture. The stored motion information may be transmitted to the inter predictor 221 for use as neighboring block motion information or temporary neighboring block motion information. Memory 270 may store reconstructed samples of reconstructed blocks in the current picture and may provide them to intra-predictor 222.

[83] FIG. 3 is a diagram schematically illustrating the configuration of a video/image decoding apparatus to which the present disclosure may be applied.

[84] With reference to FIG. 3, decoder 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. Predictor 330 may include an inter predictor 331 and an intra predictor 332. Residual processor 220 may include dequantizer 321 and inverse transform 321. The entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 that have been described above may be configured by one or more hardware components (e.g., decoder chipsets or processors) in accordance with embodiment. In addition, memory 360 may include a decoded picture buffer (DPB) or may be configured with a digital storage medium. The hardware component may further include memory 360 as an internal/external component.

[85] When a bitstream including video/image information is input, the decoding apparatus 300 may reconstruct the image according to the process by which the video/image information was processed in the encoding apparatus of FIG. 2. For example, the decoding apparatus 300 may derive ones/blocks based on information related to the block split obtained from the bitstream. The decoding device 300 may perform decoding using a processing unit applied in the encoding device. Therefore, a decoding processing unit may be, for example, an coding unit that may be partitioned according to a quadtree structure, a binary tree structure, and/or a ternary tree structure from a coding tree unit or a largest coding unit. One or more transformation units may be derived from a coding unit. And, the reconstructed image signal decoded and output by the decoding apparatus 300 can be reproduced by the playback apparatus.

[86] The decoder 300 may receive a signal output from the encoder of FIG. 2 in the form of a bitstream, and the received signal may be decoded by the entropy decoder 310. For example, the entropy decoder 310 may perform parsing of the bitstream to derive information (eg, video/image information) necessary for image reconstruction (or picture reconstruction). The video/image information may further include information about various parameter sets such as adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), video parameter set (VPS), and the like. In addition, the video/image information may further include general restriction information. The decoding device may further decode the picture based on the parameter set information and/or general constraint information. The signaled/received information and/or syntax elements described later in this disclosure may be decoded by a decoding procedure and obtained from a bitstream. For example, entropy decoder 310 may decode information in the bitstream based on an encoding method such as exponential Golomb coding, CAVLC, or CABAC, and may output a syntax element value required for image reconstruction and quantized transform coefficient values for the remainder. More specifically, the CABAC entropy decoding method can receive a bin corresponding to each syntax element in the bitstream, determine the context model using the decoding target syntax element information, the decoding information of the neighboring and target decoding blocks, or the symbol/bin information decoded in the previous stage, predict the probability generate a bean according to the determined context model, and perform arithmetic decoding of the bean to generate a character corresponding to the value of each syntax element. Here, the CABAC entropy decoding method can update the context model using the decoded symbol/bin information for the context model of the next symbol/bin after determining the context model. The prediction information among the information decoded by the entropy decoder 310 may be provided to a predictor (inter-predictor 332 and intra-predictor 331) and residual values, i.e., quantized transform coefficients on which entropy decoding was performed in entropy decoder 310, and the associated parameter information may be input to the remainder processor 320 . Residual processor 320 may output a residual signal (residual block, residual samples, residual array of samples). In addition, filtering information among the information decoded by the entropy decoder 310 may be provided to the filter 350. Meanwhile, a receiver (not shown) that receives a signal output from the encoder may be further configured as an internal/external member of the device 300 decoding device, or the receiver may be a component of the entropy decoder 310. Meanwhile, the decoding apparatus according to the present disclosure may be referred to as a video/image/picture decoding apparatus, and the decoding apparatus may be classified into an information decoder (video/image/picture information decoder) and a decoder sampling (video/image/picture sampling decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverse transform 322, an adder 340, a filter 350, a memory 360, an inter predictor 332, and an intra predictor 331.

[87] The dequantizer 321 may output transform coefficients by dequantizing the quantized transform coefficients. The dequantizer 321 may reorder the quantized transform coefficients in the form of a two-dimensional block. In this case, the reordering may be performed based on the coefficient scan order performed in the encoder. The dequantizer 321 may perform dequantization on the quantized transform coefficients using a quantization parameter (eg, quantization step size information) and obtain the transform coefficients.

[88] The detransformer 222 obtains the residual signal (residual block, residual array of samples) by inverse transforming the transform coefficients.

[89] The predictor may perform prediction on the current block and generate a predicted block including the prediction samples for the current block. The predictor may determine whether intra-prediction or inter-prediction is applied to the current block based on the prediction information output from the entropy decoder 310, and may determine, in particular, the intra/inter prediction mode.

[90] The predictor may generate a prediction signal based on various prediction methods. For example, the predictor may apply intra-prediction or inter-prediction on the same block, and may also apply intra-prediction and inter-prediction at the same time. This may be referred to as combined inter- and intra-prediction (CIIP). In addition, the predictor can perform intra-block copy (IBC) for on-block prediction. The intra-block copy mode may be used for encoding image/video game content or the like, such as screen content coding (SCC). Although IBC basically performs prediction in the current block, it can be performed similarly to inter-prediction in that it outputs a reference block in the current block. That is, the IBC may use at least one of the inter prediction methods described in this disclosure.

[91] The intra-predictor 331 may predict the current block by referring to the samples in the current picture. These samples may be adjacent or spaced apart from the current block according to the prediction mode. In intra-prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The intra predictor 331 may determine the prediction mode applied to the current block using the prediction mode applied to the neighboring block.

[92] The inter predictor 332 may output a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector in the reference picture. Meanwhile, in order to reduce the amount of motion information transmitted in the inter-prediction mode, motion information may be predicted on a block, sub-block, or sample based on a correlation of motion information between an adjacent block and a current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block existing in the current picture and a temporal neighbor block existing in the reference picture. For example, the inter predictor 332 may configure a list of motion information candidates based on neighboring blocks, and derive a motion vector and/or a reference picture index of the current block based on the received candidate selection information. Inter prediction may be performed based on various prediction modes, and the prediction information may include information indicating the inter prediction mode for the current block.

[93] The adder 340 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed array of samples) by summing the obtained residual signal with the prediction signal (predicted block, predicted array of samples) output from the predictor 330. If there is no residual for the target processing block, as in the case where the skip mode is applied, the predicted block may be used as the reconstructed block.

[94] The adder 340 may be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal may be used to intra-predict the next target processing block in the current block and, as described below, may be filtered out or used to inter-predict the next picture.

[95] Meanwhile, in the picture decoding process, luminance chrominance scaling (LMCS) mapping may be applied.

[96] Filter 350 can improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, filter 350 may generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and may pass the modified reconstructed picture to memory 260, specifically to DPB memory 360. Various filtering methods may include, for example, deblocking filtering, adaptive sample bias, adaptive loop filter, bidirectional filter, and the like.

[97] The (modified) reconstructed picture that has been stored in the DPB memory 360 may be used as a reference picture in the inter predictor 332. The memory 360 may store motion information of a block in the current picture from which the motion information is derived (or decoded), and /or information about the movement of blocks in the already restored picture. The stored motion information may be transmitted to the inter predictor 332 for use as neighbor block motion information or temporary neighbor block motion information. Memory 360 may store reconstructed samples of reconstructed blocks in the current picture and pass them to intra-predictor 331.

[98] In this specification, the examples described for predictor 330, dequantizer 321, inverse transform 322, and filter 350 of decoder 300 may be similarly or respectively applied to predictor 220, dequantizer 234, inverse transform 235, and filter 260 of encoder 200, respectively. .

[99] As described above, the prediction is performed to improve the compression efficiency when performing video encoding. By doing this, a predicted block including the prediction samples for the current block, that is, the coding target block, can be generated. Here, the predicted block includes prediction samples in the spatial domain (or pixel domain). The predicted block can be output in the same way in the encoding device and the decoding device, and the encoding device can improve the image encoding efficiency by signaling to the decoding device not the original sample value of the original block itself, but the residual information (residual information) between the original block and the predicted block. The decoding apparatus may derive a residual block including the residual samples based on the residual information, generate a reconstructed block including the reconstructed samples by summing the residual block and the predicted block, and generate a reconstructed picture including the reconstructed blocks.

[100] Residual information may be generated through transformation and quantization procedures. For example, the coding device can output a residual block between the original block and the predicted block, output transform coefficients by performing a transform procedure on the residual samples (residual array of samples) included in the residual block, and output quantized transform coefficients by performing a quantization procedure on the transform coefficients, so that it can signal the associated residual information to a decoder (via a bitstream). Here, the residual information may include value information, location information, a transform scheme, a transform kernel, a quantization parameter or the like of quantized transform coefficients. The decoder may perform a dequantization/inverse transform procedure and output residual samples (or residual block of samples) based on the residual information. The decoder may generate a reconstructed block based on the predicted block and the residual block. An encoder may output a residual block by dequantizing/inverse transforming the quantized transform coefficients for a reference for inter-prediction of the next picture, and may generate a reconstructed picture based on this.

[101] FIG. 4 schematically illustrates a multiple transform method according to an embodiment of the present disclosure.

[102] With reference to FIG. 4, the transducer may correspond to the transducer in the encoder of FIG. 2, and the inverse transform may correspond to the inverse transform in the encoder of FIG. 2 or the inverse converter in the decoder according to FIG. 3.

[103] The converter can output (primary) transform coefficients by performing primary transform based on the residual samples (residual sample array) in the residual block (S410). This primary transformation may be referred to as the main (basic) transformation. Here, the primary transform may be based on the multiple transform selection (MTS), and when the multiple transform is applied as the primary transform, it may be referred to as the multiple primary transform.

[104] The multiple fundamental transform may represent a transform method further using a discrete cosine transform (DCT) of type 2 and a discrete sine transform (DST) of type 7, a DCT of type 8, and/or a DST of type 1. That is, the multiple fundamental transform may represent a method transforms for converting a residual signal (or residual block) of the spatial domain into transform coefficients (or primary transform coefficients) of the frequency domain based on a plurality of transform kernels selected from DCT type 2, DST type 7, DCT type 8, and DST type 1. Here, the coefficients of the primary transform may be referred to as temporal transform coefficients from the viewpoint of the transducer.

[105] In other words, when the conventional transform method is applied, the transform coefficients can be generated by applying a spatial-to-frequency domain transform to the residual signal (or residual block) based on DCT type 2. In contrast, when multiple base transform is applied, transform coefficients (or primary transform coefficients) may be generated by applying a spatial-to-frequency domain transform to the residual signal (or residual block) based on DCT type 2, DST type 7, DCT type 8, and/or DST type 1. Here, DCT type 2, DST type 7, DCT type 8, and DST type 1 may be referred to as transformation types, transformation kernels, or transformation bases. These types of DCT/DST transformations can be defined based on basis functions.

[106] When multiple base transformation is performed, the vertical transformation kernel and the horizontal transformation kernel for the target block may be selected from the transformation kernels, the vertical transformation may be performed on the target block based on the vertical transformation kernel, and the horizontal transformation may be performed on the target block based on the kernel horizontal transformation. Here, a horizontal transform may indicate a transform on the horizontal components of the target box, and a vertical transform may indicate a transform on the vertical components of the target box. The vertical transform kernel/horizontal transform kernel may be adaptively determined based on the prediction mode and/or transform index for the target block (CU or sub-block) including the residual block.

[107] In addition, according to an example, if the primary transformation is performed by applying the MTS, the mapping relationship for the transformation kernels can be set by setting special basis functions to predefined values and combining the basis functions to apply in vertical transformation or horizontal transformation. For example, when a horizontal transformation kernel is expressed as trTypeHor and a vertical direction transformation kernel is expressed as trTypeVer, a value of 0 for trTypeHor or trTypeVer may be set to DCT2, a value of 1 for trTypeHor or trTypeVer may be set to DST7, and a value of 2 for trTypeHor or trTypeVer may be set to DST7. be set to DCT8.

[108] In this case, the MTS index information may be encoded and signaled to a decoder to indicate any one of a plurality of sets of transformation cores. For example, MTS index 0 may indicate that both trTypeHor and trTypeVer are 0, MTS index 1 may indicate that both trTypeHor and trTypeVer are 1, MTS index 2 may indicate that trTypeHor is 2, and trTypeVer is 1, MTS index 3 may indicate that trTypeHor is 1 and trTypeVer is 2, and MTS index 4 may indicate that both trTypeHor and trTypeVer are 2.

[109] In one example, sets of transformation cores according to MTS index information are shown in the following table.

[110] [Table 1]

[112] The converter can output modified (secondary) transform coefficients by performing secondary transform based on the (primary) transform coefficients (S420). The primary transform is a transform from the spatial domain to the frequency domain, and the secondary transform refers to a transform into a more compressed expression using the correlation existing between the coefficients of the (primary) transform. The secondary transformation may include an inseparable transformation. In this case, the secondary transform may be referred to as a non-separable secondary transform (NSST) or a mode-dependent non-separable secondary transform (MDNSST). The non-separable secondary transform may represent a transform that generates modified transform coefficients (or secondary transform coefficients) for the residual signal by secondary transform, based on the non-separable transform matrix, of the (primary) transform coefficients derived by the primary transform. Here, the vertical transformation and the horizontal transformation cannot be applied separately (or the horizontal and vertical transformations cannot be applied independently) to the (primary) transformation coefficients, but the transformations can be applied simultaneously based on the nonseparable transformation matrix. In other words, the inseparable secondary transform may represent a transform method that is not separately applied in the vertical direction and the horizontal direction for the (primary) transform coefficients, and, for example, two-dimensional signals (transform coefficients) are reordered into a one-dimensional signal by some certain direction (for example, the direction "row first" or "column first" direction), and then the modified transform coefficients (or secondary transform coefficients) are generated based on the nonseparable transform matrix. For example, according to the row first order, M×N blocks are arranged in a line in the order of first row, second row, ... and Nth row. According to the column first order, the M×N blocks are arranged in a line in the order of the first column, the second column, ... and the Nth column. An inseparable secondary transform may be applied to the top-left region of a block configured with (primary) transform coefficients (hereinafter, may be referred to as a transform coefficient block). For example, if the width (W) and height (H) of the transform coefficient block are equal to or greater than 8, then an 8×8 inseparable secondary transform may be applied to the top-left region of the 8×8 transform coefficient block. In addition, if the width (W) and height (H) of the transform coefficient block are equal to or greater than 4, and the width (W) or height (H) of the transform coefficient block is less than 8, then a 4x4 inseparable secondary transform may be applied. to the top-left area min(8,W) × min(8,H) of the block of transform coefficients. However, the embodiment is not limited to this, and for example, even if one of the conditions that the width (W) or height (H) of the transform coefficient block is equal to or greater than 4 is satisfied, then a 4×4 inseparable secondary transform can be applied to the upper- left area min(8,W) × min(8,H) of the transform coefficient block.

[112] Specifically, for example, if a 4×4 input block is used, then the non-separable secondary transform can be performed as follows.

[113] Input block X 4×4 can be represented as follows.

[114] [Equation 1]

может быть представлен, как указано ниже.[115] If X is represented as a vector, then the vector

can be presented as below.

[116] [Equation 2]

является одномерным вектором, полученным переупорядочением двумерного блока X в Уравнении 1 в соответствии с порядком сначала по строке.[117] In Equation 2, the vector

is a one-dimensional vector obtained by rearranging the two-dimensional block X in Equation 1 according to row first order.

[118] In this case, the secondary inseparable transformation may be calculated as follows.

[119] [Equation 3]

представляет вектор коэффициентов преобразования, и T представляет матрицу 16×16 (неразделимого) преобразования.[120] In this equation,

represents a vector of transform coefficients, and T represents a 16×16 (inseparable) transform matrix.

, и

может быть переупорядочен в блок 4×4 с помощью порядка сканирования (горизонтальное, вертикальное, диагональное и т.п.). Однако вышеописанное вычисление является примером, и может использоваться преобразование гиперкуба Гивенса (HyGT) или тому подобное для вычисления неразделимого вторичного преобразования, чтобы уменьшить вычислительную сложность неразделимого вторичного преобразования.[121] Through Equation 3 above, a 16×1 vector of transform coefficients can be derived

, And

can be reordered into a 4×4 block using the scan order (horizontal, vertical, diagonal, etc.). However, the above calculation is an example, and the Givens Hypercube Transform (HyGT) or the like can be used to calculate the non-separable secondary transform to reduce the computational complexity of the non-separable secondary transform.

[122] Meanwhile, in the inseparable secondary transformation, the transformation core (transformation base, transformation type) may be selected as mode dependent. In this case, the mode may include an intra-prediction mode and/or an inter-prediction mode.

[123] As described above, the non-separable secondary transform may be performed based on the 8×8 transform or the 4×4 transform determined based on the width (W) and height (H) of the transform coefficient block. The 8×8 transform refers to a transform that is applicable to the 8×8 region included in the transform coefficient block when both W and H are equal to or greater than 8, and the 8×8 region may be the top-left region of 8×8 in the block of transformation coefficients. Similarly, the 4×4 transform refers to the transform that is applicable to the 4×4 region included in the transform coefficient block when both W and H are equal to or greater than 4, and the 4×4 region may be the top-left region 4 ×4 in the transform coefficient block. For example, an 8x8 matrix of a transform kernel may be a 64x64/16x64 matrix, and a 4x4 matrix of a transform kernel may be a 16x16/8x16 matrix.

[124] Here, in order to select a mode-dependent transform core, two non-separable secondary transform kernels per transform set for the non-separable secondary transform can be configured for both 8×8 transform and 4×4 transform, and there can be four transform sets. That is, four transform sets may be configured for 8×8 transform, and four transform sets may be configured for 4×4 transform. In this case, each of the four 8x8 transform transform sets may include two 8×8 transform kernels, and each of the four 4×4 transform transform sets may include two 4×4 transform kernels.

[125] However, since the transform size, i.e., the size of the area to which the transform is applied, may have a size other than 8x8 or 4x4, for example, the number of sets may be n, and the number of transform cores in each set can be k.

[126] A transform set may be referred to as an NSST set or an LFNST set. A specific set among the transform sets may be selected, for example, based on the intra-prediction mode of the current block (CU or sub-block). The low frequency non-separable transform (LFNST) may be an example of a shorthand non-separable transform to be described later, and represents a non-separable transform for the low frequency component.

[127] For reference, for example, an intra-prediction mode may include two non-directional (or non-angular) intra-prediction modes and 65 directional (or angular) intra-prediction modes. Non-directional intra prediction modes may include planar intra prediction mode #0 and DC intra prediction mode #1, and directional intra prediction modes may include 65 intra prediction modes #2 to 66. However, this is an example. , and this document can be applied even if the number of intra-prediction modes is different. Meanwhile, in some cases, the intra prediction mode No. 67 may be further used, and the intra prediction mode No. 67 may represent the linear model (LM) mode.

[128] FIG. 5 shows by way of example the intra-directional modes of 65 prediction directions.

[129] With reference to FIG. 5, based on the intra prediction mode 34 having a left-up diagonal prediction direction, the intra prediction modes can be divided into intra prediction modes having a horizontal directionality and intra prediction modes having a vertical directionality. In FIG. 5, H and V denote horizontal directivity and vertical directivity, respectively, and numbers from −32 to 32 indicate offsets in units of 1/32 along the discrete grid position. These numbers may represent an offset for the mode index value. Intra prediction modes 2 to 33 are horizontal, and intra prediction modes 34 to 66 are vertical. Strictly speaking, the intra-prediction mode 34 can be considered as neither horizontal nor vertical, but can be classified as belonging to the horizontal orientation when defining the set of secondary transform transformations. This is because the input data is transposed to be used for the vertical direction mode symmetrically based on the intra prediction mode 34, and the input data alignment method for the horizontal mode is used for the intra prediction mode 34 . Transposing the input data means that the rows and columns of the two-dimensional block of M×N data are switched to N×M data. The intra prediction mode 18 and the intra prediction mode 50 may represent the horizontal intra prediction mode and the vertical intra prediction mode, respectively, and the intra prediction mode 2 may be referred to as the diagonal right-up intra prediction mode, since the intra prediction mode 2 -prediction has a left reference pixel and performs prediction in the right-up direction. Similarly, the intra-prediction mode 34 may be referred to as the diagonal right-down intra-prediction mode, and the intra-prediction mode 66 may be referred to as the diagonal left-down intra-prediction mode.

[130] According to an example, four sets of transformations according to the intra prediction mode may be displayed, for example, as shown in the following table.

[131] [Table 2]

[132] As shown in Table 2, any one of the four transform sets, ie, lfnstTrSetIdx, can be mapped to any of the four indices, ie, 0 to 3, according to the intra-prediction mode.

[133] When it is determined that a specific set is used for a non-separable transform, one of the k transform kernels in the specific set may be selected with the non-separable secondary transform index. The encoder may output a non-separable secondary transform index indicating a specific transform core based on a rate-distortion (RD) test, and may signal the non-separable secondary transform index to the decoding apparatus. The decoder may select one of the k transform cores in a particular set based on the non-separable secondary transform index. For example, lfnst index value 0 may refer to the first non-separable subtransform kernel, lfnst index value 1 may refer to the second nonseparable secondary transform kernel, and lfnst index value 2 may refer to the third nonseparable secondary transform kernel. Alternatively, lfnst index value 0 may indicate that the first nonseparable secondary transform is not applied to the target block, and lfnst index values 1 to 3 may indicate three transformation kernels.

[134] The transformer may perform an inseparable secondary transform based on the selected transform kernel and may obtain modified (secondary) transform coefficients. As described above, the modified transform coefficients may be output as transform coefficients quantized with a quantizer, and may be encoded and signaled to a decoder and transmitted to a dequantizer/inverse converter at an encoder.

[135] Meanwhile, as described above, if the secondary transform is omitted, the (primary) transform coefficients that are the output of the primary (separable) transform may be output as transform coefficients quantized by the quantizer as described above, and may be encoded and be signaled to the decoder and transmitted to the dequantizer/inverter in the encoder.

[136] The reverse converter can perform a number of procedures in reverse order with respect to how they were performed in the above converter. The detransformer may receive (dequantized) transform coefficients and output the (primary) transform coefficients by performing a secondary (inverse) transform (S450), and may obtain a residual block (residual samples) by performing a primary (inverse) transform on the (primary) transform coefficients (S460 ). In this regard, the primary transform coefficients may be referred to as modified transform coefficients in terms of the inverse transform. As described above, the encoding device and the decoding device may generate a reconstructed block based on the residual block and the predicted block, and may generate a reconstructed picture based on the reconstructed block.

[137] The decoding apparatus may further include a secondary inverse transform application determiner (or element for determining whether the secondary inverse transform should be applied) and a secondary inverse transform determiner (or element for determining the secondary inverse transform). The secondary inverse transform application determiner may determine whether the secondary inverse transform should be applied. For example, the secondary inverse transform may be NSST, RST, or LFNST, and the secondary inverse transform application determiner may determine whether the secondary inverse transform should be applied based on the secondary transform flag obtained by bitstream parsing. In another example, the secondary inverse transform application determiner may determine whether the secondary inverse transform should be applied based on the transform coefficient of the residual block.

[138] The secondary inverse transform determiner may determine the secondary inverse transform. In this case, the secondary inverse transform determiner may determine the secondary inverse transform applied to the current block based on the LFNST (NSST or RST) transform set specified according to the intra-prediction mode. In an embodiment, the method for determining the secondary transform may be determined depending on the method for determining the primary transform. Various combinations of primary transforms and secondary transforms may be determined according to the intra-prediction mode. In addition, in the example, the secondary inverse transform determiner may determine the area to which the secondary inverse transform is applied based on the size of the current block.

[139] Meanwhile, as described above, if the secondary (inverse) transform is omitted, (dequantized) transform coefficients can be received, a primary (separable) inverse transform can be performed, and a residual block (residual samples) can be obtained. As described above, the encoding device and the decoding device may generate a reconstructed block based on the residual block and the predicted block, and may generate a reconstructed picture based on the reconstructed block.

[140] Meanwhile, in the present disclosure, a reduced secondary transform (RST) in which the size of a transform matrix (kernel) is reduced can be applied in the NSST concept to reduce the amount of computation and memory required for an inseparable secondary transform.

[141] Meanwhile, the transformation kernel, the transformation matrix, and the coefficient constituting the transformation kernel matrix, that is, the kernel coefficient or the matrix coefficient described in the present disclosure, can be expressed in 8 bits. This may be a condition for implementation in the decoder and the encoder, and may reduce the amount of memory required to store the transformation kernel, while reducing the performance that can reasonably be tolerated compared to the existing cases of 9 bits or 10 bits. Additionally, an 8-bit kernel matrix expression may allow the use of a small multiplier and may be more suitable for single instruction stream, multiple data stream (SIMD) architecture instructions used for optimal software implementation.

[142] In this specification, the term "RST" may mean a transformation that is performed on the residual samples for a target block based on a transformation matrix that is reduced in size according to a reduction factor. In case of performing a shorthand transformation, the amount of computation required for the transformation can be reduced due to the reduction in the size of the transformation matrix. That is, RST can be used to solve the problem of computational complexity that occurs when an inseparable transformation or a transformation of a large block size occurs.

[143] RST may be defined by various terms such as reduced transformation, reduced secondary transformation, reduction transformation, simplified transformation, simple transformation, and the like, and the naming of RST is not limited to the examples listed. Alternatively, since RST is mainly performed in the low frequency region including a non-zero coefficient in the transform block, it may be referred to as a low frequency inseparable transform (LFNST). The transform index may be referred to as the LFNST index.

[144] Meanwhile, when the secondary inverse transform is performed based on the RST, the inverse transform 235 of the encoder 200 and the inverse transform 322 of the decoder 300 may include an inverse shortened secondary transform that outputs modified transform coefficients based on the inverse RST transform coefficients, and an inverse primer that outputs residual samples for the target block based on an inverse primer of the modified transform coefficients. The inverse primary transformation refers to the inverse transformation of the primary transformation applied to the remainder. In the present disclosure, deriving a transform coefficient based on a transform may refer to deriving a transform coefficient by applying a transform.

[145] FIG. 6 is a diagram illustrating RST according to an embodiment of the present disclosure.

[146] In the present disclosure, a "target block" may refer to a current block to be encoded, a residual block, or a transform block.

[147] In RST, according to the example, an N-dimensional vector can be mapped to an R-dimensional vector located in another space, so that a shortened transformation matrix can be determined, where R is less than N. N can mean the square of the block side length to which the transform is applied, or the total number of transform coefficients corresponding to the block to which the transform is applied, and the abbreviated factor can mean an R/N value. The reduced factor (ratio) may be referred to as a reduced factor, reduction factor, simplified factor, simple factor, or defined by various other terms. Meanwhile, R may be referred to as a reduced rate, but depending on the circumstances, the reduced rate may mean R. In addition, depending on the circumstances, the reduced factor may mean the value of N/R.

[148] In an example, a pruning factor or a pruning factor may be signaled by a bit stream, but the example is not limited to this. For example, a predetermined value for the reduced factor or reduced factor may be stored in each of the encoder 200 and the decoder 300, in which case, the reduced factor or reduced factor may not be signaled separately.

[149] The size of the reduced transformation matrix according to the example may be R×N less than N×N, the size of the conventional transformation matrix, and may be determined as in Equation 4 below.

[150] [Equation 4]

[151] The matrix T in the reduced transform block shown in FIG. 6(a) may mean the T _R × _N matrix of Equation 4. As shown in FIG. 6(a), when the reduced transform matrix T _R × _N is multiplied by the residual samples for the target block, the transform coefficients for the target block can be output.

[152] In the example, if the size of the block to which the transform is applied is 8x8 and R=16 (i.e., R/N=16/64=1/4), then RST in accordance with FIG. 6(a) can be expressed as a matrix operation as shown in Equation 5 below. In this case, the memory and multiplication calculation can be reduced to about 1/4 by the reduced factor.

[153] In the present disclosure, a matrix operation can be understood as an operation of multiplying a column vector by a matrix to the left of the column vector to obtain a column vector.

[154] [Equation 5]

[155] In Equation 5, r ₁ to r ₆₄ may represent residual samples for the target block, and may specifically be transform coefficients generated by applying a primary transform. As a result of calculating Equation 5, the transform coefficients ci for the target block can be derived, and the derivation process ci can be as in Equation 6.

[156] [Equation 6]

[157] As a result of calculating Equation 6, the transform coefficients c ₁ to c _R for the target block can be derived. That is, when R=16, transform coefficients c ₁ to c ₁₆ for the target block can be output. If, instead of RST, a regular transform is applied, and a 64×64 (N×N) transform matrix is multiplied by 64×1 (N×1) residual samples, then only 16 (R) transform coefficients are output for the target block, as applied RST, although 64 (N) transform coefficients are derived for the target block. Since the total number of transform coefficients for the target block is reduced from N to R, the amount of data transmitted by the encoder 200 to the decoder 300 is reduced, so that the transmission efficiency between the encoder 200 and the decoder 300 can be improved.

[158] When considered from the viewpoint of the size of the transformation matrix, the size of the regular transformation matrix is 64×64 (N×N), but the size of the reduced transformation matrix is reduced to 16×64 (R×N), so the memory usage in the case of RST execution may be reduced in terms of R/N compared to the case of regular conversion. In addition, compared with the number of N×N multiplication computations in the case of using a regular transformation matrix, the use of a reduced transformation matrix can reduce the number of multiplication computations with respect to R/N (R×N).

[159] In an example, the transformer 232 of the encoder 200 may derive transform coefficients for a target block by performing a primary transform and an RST-based secondary transform on the residual samples for the target block. These transform coefficients may be provided to an inverse transform of the decoder 300, and the inverse transform 322 of the decoder 300 may output modified transform coefficients based on an inverse reduced secondary transform (RST) for the transform coefficients, and may output the residual samples for the target block based on the inverse primary transform for modified conversion coefficients.

[160] The size of the T _R × _N matrix of the inverse RST according to the example is N × R smaller than the size of the N × N matrix of the regular inverse transformation, and is in transposition relationship with the reduced transformation matrix T _R × _N shown in Equation 4.

[161] The matrix T ^t in the reduced inverse transform block shown in FIG. 6(b) may mean an inverse RST matrix T _R × _N ^T (superscript T means transposition). When the inverse RST matrix T _R × _N ^T is multiplied by the transform coefficients for the target block, as shown in FIG. 6(b), modified transform coefficients for the target block or residual samples for the current block may be output. The matrix T _R × _N ^T of the inverse RST can be expressed as (T _R × _N ^T ) _NxR .

[162] More specifically, when the inverse RST is applied as a secondary inverse transform, the modified transform coefficients for the target block may be output when the inverse RST matrix T _R × _N ^T is multiplied by the transform coefficients for the target block. Meanwhile, the inverse RST may be applied as an inverse primary transform, in which case, residual samples for the target block may be output when the inverse RST matrix T _R × _N ^T is multiplied by the transform coefficients for the target block.

[163] In an example, if the size of the block to which the inverse transform is applied is 8x8 and R=16 (i.e., R/N=16/64=1/4), then RST according to FIG. 6(b) can be expressed as a matrix operation as shown in Equation 7 below.

[164] [Equation 7]

[165] In Equation 7, c ₁ to c ₁₆ may represent transform coefficients for the target block. As a result of evaluating Equation 7, r _i representing modified transform coefficients for the target block or residual samples for the target block can be derived, and the derivation process of r _i can be as in Equation 8.

[166] [Equation 8]

[167] As a result of evaluating Equation 8, r ₁ to r _N representing modified transform coefficients for the target block or residual samples for the target block can be output. When considered from the point of view of the size of the inverse transform matrix, the size of the regular inverse transform matrix is 64×64 (N×N), but the size of the reduced inverse transform matrix is reduced to 64×16 (R×N), so that the memory usage in case of performing the inverse RST can be reduced in terms of R/N compared to the case of regular inverse transformation. Additionally, compared to the number of N×N multiplication computations in the case of using a regular inverse transform matrix, the use of a shortened inverse transform matrix can reduce the number of multiplication computations in relation to R/N (N×R).

[168] The transform set configuration shown in Table 2 may be applied to 8x8 RST. That is, 8×8 RST can be applied according to the transform set in Table 2. Since one transform set includes two or three transforms (kernels) according to the intra-prediction mode, it can be configured to select one of up to four transformations, including the case where no secondary transformation is applied. In a transform where no secondary transform is applied, this may be taken into account to apply the identity matrix. Assuming that indices 0, 1, 2, and 3 are respectively assigned to the four transforms (for example, index 0 can be allocated for the case where the identity matrix is applied, i.e., the case when no secondary transformation is applied), the transform index or the lfnst index as of a syntax element may be signaled for each block of transform coefficients, thereby indicating the transform to be applied. That is, for an 8x8 top-left block, using the transform index, you can specify 8x8 RST in the RST configuration, or specify 8x8 lfnst when LFNST is applied. 8×8 lfnst and 8×8 RST refer to transforms applicable to the 8×8 region included in the transform coefficient block when W and H of the target block to be transformed are equal to or greater than 8, and the 8×8 region may be top-left 8×8 area in the transform coefficients block. Similarly, 4×4 lfnst and 4×4 RST refer to transforms applicable to a 4×4 region included in a transform coefficient block when W and H of the target block are equal to or greater than 4, and the 4×4 region may be upper- left 4x4 area in the transform coefficients block.

[169] According to an embodiment of the present disclosure, for transformation in the encoding process, only 48 data pieces can be selected, and a maximum 16x48 transformation kernel matrix can be applied to them, instead of applying a 16x64 transformation kernel matrix to 64 data pieces , forming an 8×8 region. Here, "maximum" means that m has a maximum value of 16 in the m×48 matrix of the transformation kernel for generating m coefficients. That is, when RST is performed by applying an m×48 matrix of a transform kernel (m≤16) to an 8×8 region, 48 pieces of data are input and m coefficients are generated. When m is 16, 48 pieces of data are input and 16 coefficients are generated. That is, assuming that 48 pieces of data form a 48×1 vector, the 16×48 matrix and the 48×1 vector are multiplied sequentially, thereby generating a 16×1 vector. Here, 48 pieces of data forming an 8x8 area can be properly arranged to form a 48x1 vector. For example, a 48×1 vector may be generated based on 48 pieces of data constituting a region, excluding the bottom-right 4×4 region among the 8×8 regions. Here, when the matrix operation is performed by applying a maximum 16×48 matrix of the transform kernel, 16 modified transform coefficients are generated, and the 16 modified transform coefficients can be ordered in the upper-left region 4×4 according to the scan order, and the upper-right region 4 x4 and bottom-left area 4x4 can be filled with zeros.

[170] For the inverse transformation in the decoding process, the transposed matrix of the above-described transformation kernel matrix can be used. That is, when the inverse RST or LFNST is performed in the process of inverse transformation by the decoding device, the input coefficient data to which the inverse RST is applied is configured into a one-dimensional vector according to a predetermined order, and the modified coefficient vector obtained by multiplying the one-dimensional vector and the corresponding inverse matrix The RST to the left of the one-dimensional vector may be arranged into a two-dimensional block according to a predefined stacking order.

[171] In summary, in the transform process, when RST or LFNST is applied to the 8x8 region, a matrix operation is applied to 48 transform coefficients in the top-left, top-right, and bottom-left regions of the 8x8 region, excluding the bottom-right a region among transform coefficients in an 8×8 region and a 16×48 matrix of the transform kernel. For a matrix operation, 48 transform coefficients are entered in a one-dimensional array. When a matrix operation is performed, 16 modified transform coefficients are output, and the modified transform coefficients may be ordered in the upper-left region in an 8×8 region.

[172] In contrast, in the inverse transform process, when the inverse RST or LFNST is applied to the 8×8 region, 16 transform coefficients corresponding to the top-left region in the 8×8 region among the transform coefficients in the 8×8 region may be input in a one-dimensional array in accordance with the scan order and may be subjected to a matrix operation with a matrix of 48×16 transformation kernels. That is, the matrix operation can be expressed as (48×16 matrix)*(16×1 vector of transform coefficients)=(48×1 vector of modified transform coefficients). Here, an n×1 vector may be interpreted to have the same value as an n×1 matrix, and may thus be expressed as an n×1 column vector. Also, * means matrix multiplication. When the matrix operation is performed, 48 modified transform coefficients can be output, and 48 modified transform coefficients can be arranged in the top-left, top-right, and bottom-left regions in an 8×8 area excluding the bottom-right region.

[173] When the secondary inverse transform is based on RST, the inverse transform 235 of the encoder 200 and the inverse transform 322 of the decoder 300 may include an inverse shortened secondary transform for deriving modified transform coefficients based on the inverse RST on the transform coefficients, and an inverse primary transform for deriving residual samples for the target block based on the inverse primary transform on the modified transform coefficients. The inverse primary transformation refers to the inverse transformation of the primary transformation applied to the remainder. In the present disclosure, deriving a transform coefficient based on a transform may refer to deriving a transform coefficient by applying a transform.

[174] The above-described unseparated transform, LFNST, will be described in detail as follows. The LFNST may include forward transformation by the encoder and inverse transformation by the decoder.

[175] The encoding device takes the result (or part of the result) output after applying the primary (main) transform as an input, and applies the direct secondary transform (secondary transform).

[176] [Equation 9]

[177] In Equation 9, x and y are inputs and outputs of the secondary transform, respectively, and G is a matrix representing the secondary transform, and the transform basis vectors are formed by column vectors. In the case of reverse LFNST, when the dimension of the transformation matrix G is expressed as [number of rows × number of columns], in the case of forward LFNST, the transposition of the matrix G becomes the dimension of G ^T .

[178] For the inverse LFNST, the dimensions of the matrix G are [48x16], [48x8], [16x16], [16x8], and the [48x8] matrix and the [16x8] matrix are partial matrices obtained by discretizing the 8 basis transformation vectors on the left of the [48×16] matrix and the [16×16] matrix, respectively.

[179] On the other hand, for direct LFNST, the dimensions of the matrix G ^T are [16×48], [8×48], [16×16], [8×16], and the matrix [8×48] and the matrix [ 8x16] are partial matrices obtained by discretizing the 8 basis transform vectors from above the [16x48] matrix and the [16x16] matrix, respectively.

[180] Therefore, in the case of direct LFNST, vector [48×1] or vector [16×1] is possible as input x, and vector [16×1] or vector [8×1] is possible as output y. In video encoding and decoding, the output of a direct primary transform is two-dimensional (2D) data, so in order to generate a [48x1] vector or a [16x1] vector as an x input, a one-dimensional vector must be generated by properly ordering the 2D data, which is direct conversion output.

[181] FIG. 7 is a diagram illustrating the layout sequence of the direct primary transform output into a one-dimensional vector according to an example. The left diagrams (a) and (b) in FIG. 7 shows the sequence for generating the [48×1] vector, and the right diagrams (a) and (b) in FIG. 7 shows the sequence for generating the [16×1] vector. In the case of LFNST, a one-dimensional vector x can be obtained by sequentially stacking 2D data in the same order as in (a) and (b) in FIG. 7.

[182] The layout direction of the direct primary transform output may be determined according to the intra-prediction mode of the current block. For example, when the intra-prediction mode of the current block is in the horizontal direction with respect to the diagonal direction, the direct primary transform output may be arranged in the order of FIG. 7(a), and when the intra-prediction mode of the current block is in the vertical direction with respect to the diagonal direction, the direct primary transform output can be arranged in the order of FIG. 7(b).

[183] According to an example, a build order different from the build orders in FIG. 7(a) and (b) and to obtain the same result (vector y) as obtained by applying the stacking orders in FIG. 7(a) and (b), the column vectors of the matrix G may be reordered according to the layout order. That is, one can reorder the column vectors of G such that each element that makes up vector x is always multiplied by the same transformation basis vector.

[184] Since the output y obtained using Equation 9 is a one-dimensional vector, when two-dimensional data is required as input in the process of using the result of the direct secondary transform as input, for example, in the process of performing quantization or residual encoding, the output vector y in Equation 9 should be properly re-linked as 2D data.

[185] FIG. 8 is a diagram illustrating the layout sequence of the direct secondary transform output into a two-dimensional vector according to an example.

[186] In the case of LFNST, output values may be arranged into a 2D block according to a predefined scan order. Fig. 8(a) shows that when the output y is a [16x1] vector, the output values are arranged at 16 2D block positions according to the diagonal scan order. Fig. 8(b) shows that when the output y is the [8×1] vector, the output values are arranged at 8 positions of the 2D block according to the diagonal scan order, and the remaining 8 positions are filled with zeros. X in Fig. 8(b) indicates that this position is filled with zero.

[187] According to another example, since the order in which the output vector y is processed when performing quantization or residual encoding can be preset, the output vector y may not be arranged into a 2D block as shown in FIG. 8. However, in the case of residual coding, data coding may be performed in 2D block units (for example, 4×4) such as CG (coefficient group), in which case, the data is arranged according to a specific order, such as diagonal order. scanning in Fig. 8.

[188] Meanwhile, the decoding apparatus can configure the one-dimensional input vector y by arranging the two-dimensional data output by the dequantization process or the like according to a preset scan order for inverse transformation. The input vector y can be output as the output vector x through the following equation.

[189] [Equation 10]

x=Gy

[190] In the case of inverse LFNST, the output vector x may be derived by multiplying the input vector y, which is a [16x1] vector or an [8x1] vector, by a matrix G. For inverse LFNST, the output vector x may be either vector [48x1] or vector [16x1].

[191] The output vector x is arranged into a two-dimensional block according to the order shown in FIG. 7 and is assembled as 2D data, and this 2D data becomes the input (or part of the input) of the inverse primary transform.

[192] Accordingly, the inverse secondary transformation is generally the opposite process of the direct secondary transformation, and in the case of the reverse transformation, unlike the forward direction, the inverse secondary transformation is applied first, and then the inverse primary transformation is applied.

[193] In the inverse LFNST, one of 8 [48×16] matrices and 8 [16×16] matrices may be selected as the transformation matrix G. Whether to use a [48x16] matrix or a [16x16] matrix depends on the block size and shape.

[194] In addition, 8 matrices can be derived from four transformation sets as shown in Table 2 above, and each transformation set can be composed of two matrices. Which transform set to use among the 4 transform sets is determined according to the intra prediction mode, and more specifically, the transform set is determined based on the wide angle intra prediction (WAIP) enhanced intra prediction mode value. Which matrix to choose from the two matrices that make up the selected transform set is output by index signaling. More specifically, 0, 1 and 2 are possible as the transmitted index value, 0 may indicate that LFNST is not applied, and 1 and 2 may indicate any one of the two transformation matrices constituting the transformation set selected based on the intra prediction mode value.

[195] FIG. 9 is a diagram illustrating wide-angle intra prediction modes according to an embodiment of the present document.

[196] The normal intra prediction mode value may be 0 to 66 and 81 to 83, and the intra prediction mode value extended due to WAIP may be -14 to 83, as shown. Values from 81 to 83 indicate the CCLM (linear cross component model) mode, and values from -14 to -1 and values from 67 to 80 indicate the intra-prediction mode enhanced by the application of WAIP.

[197] When the width of the current prediction block is larger than the height, the top reference pixels are generally closer to positions within the predicted block. Therefore, it would be more accurate to predict in the bottom-left direction than in the top-right direction. In contrast, when the block height is greater than the width, the left reference pixels are generally closer to positions within the predicted block. Therefore, it may be more accurate to predict in the up-right direction than in the down-left direction. Therefore, it may be preferable to apply remapping, i. e. modifying the mode index to the wide-angle intra-prediction mode index.

[198] When wide-angle intra-prediction is applied, information about an existing intra-prediction may be signaled, and after the information is parsed, the information may be re-mapped to a wide-angle intra-prediction mode index. Therefore, the total number of intra prediction modes for a particular block (eg, a non-square block of a particular size) may not change, and the total number of intra prediction modes is 67, and the coding of the intra prediction mode for a particular block may not change.

[199] Table 3 below shows the process of deriving the modified intra mode by re-mapping the intra prediction mode to the wide-angle intra prediction mode.

[200]

[Table 3] The inputs to this process are:
- variable predModeIntra specifying the intra-prediction mode,
- variable nTbW specifying the width of the transform block,
- variable nTbH specifying the height of the transform block,
- cIdx variable specifying the color component of the current block.
The output of this process is a modified intra prediction mode, predModeIntra.
The variables nW and nH are output as follows:
- If IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT or cIdx is not equal to 0, then the following applies:
nW=nTbW
nH=nTbH
- Otherwise (IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT and cIdx is equal to 0), then the following applies:
nW=nCbW
nH=nCbH
The whRatio variable is set to Abs(Log2(nW/nH)).
For non-square blocks (nW is not equal to nH), the intra-prediction mode predModeIntra is modified as follows:
- If all of the following conditions are true, then predModeIntra is set to (predModeIntra+65),
-- nW is greater than nH
-- predModeIntra is greater than or equal to 2
-- predModeIntra less than (whRatio > 1)?(8+2* whRatio) : 8
- Otherwise, if all of the following conditions are true, then predModeIntra is set to (predModeIntra - 67),
-- nH is greater than nW
-- predModeIntra is less than or equal to 66
-- predModeIntra greater than (whRatio > 1)?(60-2* whRatio): 60

[201] In Table 3, the value of the extended intra-prediction mode is finally stored in the variable predModeIntra, and ISP_NO_SPLIT indicates that the CU is not subdivided by the intra-subdivision (ISP) method currently adopted in the VVC standard, and the value is 0 , 1 and 2 of the variable cIdx indicate the case of the luminance components, Cb and Cr, respectively. The Log2 function shown in Table 3 returns log base 2, and the Abs function returns the absolute value.

[202] Variable predModeIntra indicating the mode of intra-prediction, and the height and width of the transform block, etc. are used as input values of the wide-angle intra prediction mode display process, and the output value is the modified intra prediction mode predModeIntra. The height and width of the transform block or encoding block may be the height and width of the current block to re-display the intra-prediction mode. In this case, the variable whRatio, which reflects the ratio of width to height, can be set to Abs(Log2(nW/nH)).

[203] For a non-square block, the intra-prediction mode can be divided into two cases and modified.

[204] First, if all conditions (1)~(3) are satisfied, (1) the width of the current block is greater than the height, (2) the intra-prediction mode before modification is equal to or greater than 2, (3) the mode intra-prediction is less than the value derived from (8+2*whRatio) when whRatio is greater than 1 and less than 8 when whRatio is less than or equal to 1 [predModeIntra less than (whRatio>1)? (8+2*whRatio):8], the intra prediction mode is set to 65 more than the intra prediction mode [predModeIntra is set to (predModeIntra+65)].

[205] If contrary to the above, that is, the following conditions (1)~(3) are satisfied: (1) the height of the current block is greater than the width, (2) the intra-prediction mode before modification is less than or equal to 66, (3 ) intra prediction mode is greater than the value derived from (60-2*whRatio) when whRatio is greater than 1 and greater than 60 when whRatio is less than or equal to 1 [predModeIntra greater than (whRatio>1 )? (60-2*whRatio):60], the intra-prediction mode is set to 67 less than the intra-prediction mode [predModeIntra is set to (predModeIntra-67)].

[206] Table 2 above shows how the transform set is selected based on the intra-prediction mode value extended by WAIP in LFNST. As shown in FIG. 9, modes 14 to 33 and modes 35 to 80 are symmetrical about the prediction direction around mode 34. For example, mode 14 and mode 54 are symmetrical about the direction corresponding to mode 34. Therefore, the same set of transformations is applied to modes located in mutually symmetrical directions. , and this symmetry is also reflected in Table 2.

[207] Meanwhile, it is assumed that the direct LFNST input for mode 54 is symmetrical with the direct LFNST input for mode 14. For example, for mode 14 and mode 54, 2D data is reordered into 1D data according to the stacking order shown in FIG. . 7(a) and in FIG. 7(b), respectively. In addition, it can be seen that the patterns in the order shown in FIG. 7(a) and in FIG. 7(b) are symmetrical with respect to the direction (diagonal direction) indicated by mode 34.

[208] Meanwhile, as described above, which transformation matrix of the [48x16] matrix and the [16x16] matrix is applied to the LFNST is determined by the size and shape of the transformation target block.

[209] FIG. 10 is a diagram illustrating a block shape to which LFNST is applied. Fig. 10(a) shows 4x4 blocks, (b) shows 4x8 and 8x4 blocks, (c) shows 4xN blocks or Nx4 blocks in which N is 16 or more, (d) shows blocks 8×8, (e) shows M×N blocks where M≥8, N≥8 and N>8 or M>8.

[210] FIG. 10, bold-bordered boxes indicate areas to which the LFNST is applied. For blocks according to Fig. 10(a) and (b), LFNST is applied to the top-left 4×4 region, and for the block of FIG. 10(c), LFNST is applied individually to two top-left 4x4 regions that are adjacent. In FIG. 10(a), (b) and (c), since the LFNST is applied in units of 4x4 areas, this LFNST will be referred to as "4x4 LFNST" hereinafter. As an appropriate transformation matrix, a [16×16] matrix or a [16×8] matrix may be applied based on the dimension of the matrix G in Equations 9 and 10.

[211] More specifically, the [16x8] matrix is applied to a 4x4 unit (4x4 TU or 4x4 CU) of FIG. 10(a), and a [16×16] matrix is applied to the blocks in FIG. 10(b) and (c). This is intended to adjust the worst case computational complexity to 8 multiplications per sample.

[212] Referring to FIG. 10(d) and (e), the LFNST is applied to the top-left region of 8×8, and this LFNST is hereinafter referred to as "8×8 LFNST". As the corresponding transformation matrix, a [48x16] matrix or a [48x8] matrix may be used. In the case of direct LFNST, since the vector [48×1] (vector x in Equation 9) is input as input, the values of all samples of the top-left 8×8 region are not used as input values of direct LFNST. That is, as can be seen in the left arrangement in FIG. 7(a) or the left arrangement in FIG. 7(b), the [48x1] vector can be generated based on the samples belonging to the remaining 3 4x4 blocks, leaving the lower right 4x4 block as it is.

[213] The [48x8] matrix may be applied to an 8x8 block (8x8 TU or 8x8 CU) in FIG. 10(d), and the [48×16] matrix may be applied to the 8×8 block in FIG. 10(e). It is also intended to adjust the worst case computational complexity to 8 multiplications per sample.

[214] Depending on the shape of the block, when the corresponding direct LFNST (4×4 LFNST or 8×8 LFNST) is applied, 8 or 16 outputs are generated (vector y in Equation 9, vector [8×1] or [16×1 ]). In forward LFNST, the number of outputs is equal to or less than the number of inputs, due to the characteristics of the matrix G ^T .

[215] FIG. 11 is a diagram illustrating the layout of the direct LFNST output according to the example, and shows a block in which the direct LFNST output is arranged according to the block shape.

[216] The shaded area of the top-left block shown in FIG. 11 corresponds to the area where the direct LFNST output is located, the positions labeled 0 indicate samples filled with 0 values, and the remaining area represents areas not modified by the direct LFNST. In the region not modified by LFNST, the output of the direct primary transform remains unchanged.

[217] As described above, since the dimension of the applied transformation matrix varies according to the shape of the block, the number of output data also varies. As shown in FIG. 11, the direct LFNST output may not completely fill the top-left 4x4 block. In the case of FIG. 11(a) and (d), the [16x8] matrix and the [48x8] matrix are applied to the block indicated by the thick line or the partial area within the block, respectively, and the [8x1] vector is generated as the output of the direct LFNST. That is, according to the scanning order shown in FIG. 8(b), only 8 output data can be filled as shown in FIG. 11(a) and (d), and 0 can be filled in the remaining 8 positions. In the case of the applied LFNST block according to FIG. 10(d) as shown in FIG. 11(d), the two top-right and bottom-left 4x4 blocks of the top-left 4x4 block are also filled with 0 values.

[218] As described above, mainly by signaling the LFNST index, it is specified whether the LFNST should be applied and the transformation matrix to be applied. As shown in FIG. 11, when the LFNST is applied, since the number of output data of the direct LFNST may be equal to or less than the number of input data, a null-filled area occurs as follows.

[219] 1) As shown in FIG. 11(a), samples from the 8th and subsequent positions in the scan order in the top-left 4×4 block, that is, samples from the 9th to the 16th.

[220] 2) As shown in FIG. 11(d) and (e), when a [16x48] matrix or an [8x48] matrix is applied, two 4x4 blocks adjacent to the top-left 4x4 block, or the second and third 4x4 blocks in scan order.

[221] Therefore, if non-zero data exists when checking areas 1) and 2), then it is clear that LFNST is not applied, so that the corresponding LFNST index signaling can be omitted.

[222] As an example, for example, in the case of the LFNST adopted in the VVC standard, since the LFNST index signaling is performed after the coding of the remainder, the encoder can know whether there are non-zero data (significant coefficients) for all positions in the TU or CU by remainder coding. Accordingly, the encoder may determine whether to perform signaling on the LFNST index based on the existence of non-zero data, and the decoder may determine whether the LFNST index is parsed. When non-null data does not exist in the area specified in 1) and 2 above), LFNST index signaling is performed.

[223] Since a truncated unary code is used as a binarization method for the LFNST index, the LFNST index consists of up to two bins, and 0, 10, and 11 are assigned as binary codes for the possible values 0, 1, and 2 of the LFNST index, respectively. In the case of the LFNST currently adopted for VVC, CABAC context-based coding is applied to the first bin (regular coding) and bypass coding is applied to the second bin. The total number of contexts for the first bin is 2, when (DCT-2, DCT-2) is applied as a primary transform pair for the horizontal and vertical directions, and the luma component and the chrominance component are dual-tree encoded, one context is extracted and the other context is applied for the remaining cases. The LFNST index encoding is shown in the table as follows.

[224] [Table 4]

[225] Meanwhile, for the received LFNST, the following simplification methods can be applied.

[226] (i) According to the example, the number of output data for direct LFNST may be limited to a maximum of 16.

[227] In the case of FIG. 10(c), a 4x4 LFNST may be applied to two 4x4 regions adjacent top-left, respectively, and in this case, a maximum of 32 LFNSTs of output data may be generated. When the number of output data for forward LFNST is limited to a maximum of 16, in the case of 4×N/N×4 (N≥16) blocks (TU or CU), 4×4 LFNST is applied to only one 4×4 top-left area, LFNST can only be applied once to all blocks in FIG. 10. With this, the implementation of image encoding can be simplified.

[228] FIG. 12 shows that the number of output data for direct LFNST is limited to a maximum of 16 in accordance with the example. As shown in FIG. 12, when the LFNST is applied to the top-left 4×4 region in a 4×N block or an N×4 block in which N is 16 or more, 16 parts become the direct LFNST output.

[229] (ii) According to an example, nulling may additionally be applied to an area for which no LFNST is applied. In the present document, nulling may mean filling the values of all positions belonging to a particular region with the value 0. That is, nulling may be applied to a region that does not change due to LFNST and retains the result of a direct primary transformation. As described above, since the LFNST is divided into 4x4 LFNST and 8x8 LFNST, the zeroing can be divided into two types ((ii)-(A) and (ii)-(B)) as follows.

[230] (ii)-(A) When 4×4 LFNST is applied, an area to which 4×4 LFNST is not applied may be set to zero. Fig. 13 is a diagram illustrating zeroing in a block to which 4×4 LFNST is applied according to an example.

[231] As shown in FIG. 13 with respect to the block to which the 4×4 LFNST is applied, that is, for all blocks in FIG. 11(a), (b) and (c), the entire area to which LFNST is not applied may be filled with zeros.

[232] On the other hand, FIG. 13(d) shows that when the maximum value of the forward LFNST output data number is limited to 16 as shown in FIG. 12, zeroing is performed on the remaining blocks to which 4×4 LFNST is not applied.

[233] ii)-(B) When 8×8 LFNST is applied, an area to which 8×8 LFNST is not applied may be set to zero. Fig. 14 is a diagram illustrating zeroing in a block to which 8×8 LFNST is applied according to an example.

[234] As shown in FIG. 14 with respect to the block to which 8x8 LFNST is applied, that is, for all blocks in FIG. 11(d) and (e), the entire area to which LFNST does not apply may be filled with zeros.

[235] iii) Due to the zeroing presented in (ii) above, the area filled with zeros may not be the same when LFNST is applied. Accordingly, it can be checked whether non-zero data exists according to the nulling proposed in (ii) in a wider area than in the case of LFNST according to FIG. eleven.

[236] For example, when (ii)-(B) is applied, after checking whether non-null data exists, where the area is filled with null values in FIG. 11(d) and (e) in addition to the area filled with 0 additionally in FIG. 14, signaling for the LFNST index can only be performed when non-zero data does not exist.

[237] Of course, even if the nulling proposed in (ii) is applied, it is possible to check whether non-zero data exists in the same way as the existing LFNST index signaling. That is, after checking whether non-zero data exists in the block filled with zeros in FIG. 11, LFNST index signaling may be applied. In this case, only the encoder performs nulling, and the decoder assumes no nulling, that is, by only checking if non-zero data exists only in the area explicitly marked as 0 in FIG. 11 may perform parsing on the LFNST index.

[238] Alternatively, according to another example, zeroing may be performed as shown in FIG. 15. FIG. 15 is a diagram illustrating zeroing in a block to which 8×8 LFNST is applied according to another example.

[239] As shown in FIG. 13 and 14, nulling may be applied to all areas other than the area to which the LFNST is applied, or nulling may be applied to only a partial area, as shown in FIG. 15. Zeroing is only applied to areas other than the 8x8 top-left area of FIG. 15, zeroing cannot be applied to the bottom-right 4x4 block within the top-left 8x8 area.

[240] Various embodiments can be obtained that use combinations of simplification methods ((i), (ii)-(A), (ii)-(B), (iii)) for LFNST. Of course, combinations of the above simplification methods are not limited to the following embodiments, and any combination can be applied to LFNST.

[241] Embodiment

[242] - Limit the number of outputs for direct LFNST to a maximum of 16 → (i)

[243] - When 4x4 LFNST is applied, all areas to which 4x4 LFNST is not applied are set to zero → (ii)-(A)

[244] - When 8x8 LFNST is applied, all areas to which 8x8 LFNST is not applied are set to zero → (ii)-(B)

[245] - After checking if non-null data exists, also existing areas filled with zero values, and areas filled with zeros due to additional zeros ((ii)-(A), (ii)-(B)), the LFNST index is signaled, only when non-null data doesn't exist → (iii)

[246] In the case of an embodiment where LFNST is applied, the area in which non-zero output data can exist is limited within the top-left 4x4 area. More specifically, in the case of FIG. 13(a) and FIG. 14(a), the 8th position in the scan order is the last position where non-zero data may exist. In the case of FIG. 13(b) and (c) and FIG. 14(b), the 16th position in the scan order (i.e., the position of the top-right edge of the top-left 4x4 block) is the last position where data other than 0 can exist.

[247] Therefore, when LFNST is applied, after checking whether non-zero data exists at a position where the remainder encoding process is not allowed (at a position beyond the last position), it can be determined whether the LFNST index is signaled.

[248] In the case of the nulling method proposed in (ii), since the number of data ultimately generated when both the primary transform and the LFNST are applied, the amount of computation required to perform the entire transform process can be reduced. That is, when LFNST is applied, since nulling is applied to direct primary transform output existing in a region in which LFNST is not applied, it is not necessary to generate data for the region that becomes nulled at the time the direct primary transform is performed. Accordingly, it is possible to reduce the amount of computation required to generate the corresponding data. Additional results of the zeroing method proposed in (ii) are formulated as follows.

[249] First, as described above, the amount of computation required to perform the entire conversion process is reduced.

[250] In particular, when (ii)-(B) is applied, the worst-case calculation amount is reduced so that the conversion process can be simplified. In other words, in general, a large amount of computation is required to perform a large-scale primary transformation. By applying (ii)-(B), the number of data output as a result of executing direct LFNST can be reduced to 16 or less. In addition, as the size of the entire block (TU or CU) increases, the effect of reducing the conversion operation amount is further increased.

[251] Second, the amount of computation required for the entire conversion process can be reduced, thereby reducing the power consumption required to perform the conversion.

[252] Third, the waiting time associated with the conversion process is reduced.

[253] A secondary transform, such as LFNST, adds a computational burden to an existing primary transform, thereby increasing the overall latency associated with performing the transform. Particularly in the case of intra-prediction, since the reconstructed neighboring block data is used in the prediction process, during encoding, an increase in latency due to secondary transformation results in an increase in the latency before restoration. This can lead to an increase in the overall latency of intra-prediction coding.

[254] However, if the nulling proposed in (ii) is applied, the delay time for performing the primary transformation can be markedly reduced; when LFNST is applied, the delay time for the entire transformation is maintained or reduced, so that the encoder can be implemented more simply.

[255] Meanwhile, in conventional intra-prediction, the target coding unit is considered as one coding unit, and coding is performed without splitting it. However, ISP (Intra-Sub-Splitting) encoding refers to performing intra-prediction encoding with a target encoding block divided in the horizontal direction or the vertical direction. In this case, the reconstructed block may be generated by performing encoding/decoding in units of divided blocks, and the reconstructed block may be used as a reference block of the next divided block. According to an example, in ISP coding, one coding block may be split into two or four subblocks and encoded, and in ISP, intra-prediction is performed on one subblock with reference to the reconstructed pixel value of the subblock adjacent to its left or top side. Further, the term "encoding" may be used as a concept including both encoding performed by an encoding device and decoding performed by a decoding device.

[256] Table 5 shows an example of sub-blocks divided according to block sizes when ISP is applied, and sub-partitions divided according to ISP may be referred to as transform units (TUs).

[257]

[Table 5] Unit Size (CU) Number of splits 4×4 Not available 4×8, 8×4 2 All other cases 4

[258] The ISP splits the block predicted as intra luma prediction into two or four subdivisions in the vertical direction or the horizontal direction according to the block size. For example, the minimum block size to which an ISP can be applied is 4x8 or 8x4. If the block size is larger than 4x8 or 8x4, the block is split into four subdivisions.

[259] FIG. 16 and 17 show an example of a sub-block into which one coding block is partitioned, and more specifically, FIG. 16 is an example of partitioning for the case in which the coding block (width (W) × height (H)) is a 4×8 block or an 8×4 block, and FIG. 17 shows a partitioning example for a case in which a coding block is neither a 4x8 block, nor an 8x4 block, nor a 4x4 block.

[260] When ISP is applied, sub-blocks are sequentially encoded according to a partition type such as horizontally or vertically, left-to-right, or top-to-bottom, and encoding for the next sub-block may be performed after execution up to the reconstruction process through inverse transform and intra- prediction for one subblock. For the leftmost or topmost sub-block, as in the conventional intra-prediction method, the reconstructed pixel of the coding block that has already been coded is referred to. Additionally, if the previous sub-block is not adjacent to each side of the inner sub-block that follows it, to derive reference pixels adjacent to the corresponding side, as in the conventional intra-prediction method, the reconstructed pixel of the already encoded adjacent coding block is referred to.

[261] In the ISP coding mode, all sub-blocks may be encoded using the same intra-prediction mode, and a flag indicating whether or not to use ISP coding and a flag indicating which direction (horizontal or vertical) may be signaled. partitioning must be done. As in FIG. 16 and 17, the number of sub-blocks may be set to 2 or 4 depending on the shape of the block, and when the size (width×height) of one sub-block is less than 16, splitting may not be allowed for the respective sub-blocks, or the application of the ISP coding itself may be restricted. .

[262] Meanwhile, in the case of the ISP prediction mode, one coding unit is divided into two or four partition blocks, that is, a subblock, and predicted, and the same intra-prediction mode is applied to the two or four partition blocks thus divided.

[263] As described above, as a horizontal direction (if an M×N coding unit having a horizontal length and a vertical length M and N, respectively, is divided in the horizontal direction, it is divided into M×(N/2) blocks when divided by two and into M×(N/4) blocks when divided by four) and the vertical direction (if an M×N coding unit is divided in the vertical direction, it is divided into (M/2)×N blocks when divided by two and divided by blocks (M/4)×N when divided by four) are possible as splitting directions. When partitioning in the horizontal direction, partition blocks are encoded in order from top to bottom, and when partitioned in the vertical direction, partition blocks are encoded in order from left to right. The current encoded split block may be predicted with reference to the restored pixel values of the upper (left) split block in the case of splitting in the horizontal (vertical) direction.

[264] The transform may be applied to the residual signal generated by the ISP prediction method in units of partition blocks. MTS (Multiple Transform Selection) technology, based on the combination of DST-7/DCT-8 as well as the existing DCT-2, can be applied to primary transform (main transform or primary transform) based on forward direction, and LFNST (Low Frequency Inseparable Transform) may be applied to a transform coefficient generated according to the primary transform to generate a final modified transform coefficient.

[265] That is, LFNST can also be applied to split blocks separated by applying the ISP transform mode, and the same intra-prediction mode is applied to split transform blocks as described above. Accordingly, when selecting an LFNST set derived based on the intra-prediction mode, the derived LFNST set can be applied to all partition blocks. That is, the same intra-prediction mode is applied to all partition blocks, yet the same set of LFNSTs can be applied to all partition blocks.

[266] Meanwhile, according to an example, LFNST can only be applied to transform blocks having horizontal and vertical lengths of 4 or more. Therefore, when the horizontal or vertical length of the partition block divided according to the ISP prediction method is less than 4, LFNST is not applied and the LFNST index is not signaled. Accordingly, when LFNST is applied to each partition block, the corresponding partition block can be considered as one transform block. Of course, when the ISP prediction method is not applied, LFNST may be applied to the coding block.

[267] The application of LFNST to each partition block is described in detail as follows.

[268] As an example, after applying a direct LFNST to an individual partition block, and after only 16 coefficients (8 or 16) remain in the top-left 4x4 region according to the scan order of the transform coefficients, padding zeroing may be applied. all remaining positions and areas with a value of 0.

[269] Alternatively, according to an example, when the length of one side of the split box is 4, LFNST is only applied to the top-left region of 4×4, and when the length of all sides of the split box, that is, width and height, is 8 or more , LFNST can be applied to the remaining 48 coefficients except for the 4x4 bottom-right region in the 8x8 top-left region.

[270] Alternatively, according to the example, in order to adjust the worst-case computational complexity to 8 multiplications per sample, when each partition block is 4x4 or 8x8, only 8 transform coefficients can be derived after applying direct LFNST. That is, if the partition block is 4×4, an 8×16 matrix can be applied as a transformation matrix, and if the partition block is 8×8, an 8×48 matrix can be applied as a transformation matrix.

[271] Meanwhile, in the current VVC standard, LFNST index signaling is performed in coding units. Accordingly, when the ISP prediction mode is used and LFNST is applied to all partition blocks, then the same LFNST index value can be applied to the corresponding partition blocks. That is, when an LFNST index value is transmitted at the coding unit level, the corresponding LFNST index can be applied to all partition blocks in the coding unit. As described above, the LFNST index value may have the values 0, 1, and 2; 0 indicates the case in which LFNST is not applied, 1 and 2 indicate two transformation matrices present in the same LFNST set when LFNST is applied.

[272] As described above, the LFNST set is determined by the intra prediction mode, and since all partition blocks in a coding unit are predicted in the same intra prediction mode in the case of the ISP prediction mode, the partition blocks can refer to the same LFNST set.

[273] As another example, LFNST index signaling is still performed in coding units, but in the case of ISP prediction mode, without determining whether or not to apply LFNST evenly to all partition blocks, whether to apply the LFNST index value signaled at the level coding units, to each split block or not to apply LFNST, may be determined by a separate condition. Here, a single condition may be signaled in the form of a flag for each partition by a bitstream, and when the flag value is 1, the LFNST index value signaled at the coding unit level may be applied, and when the flag value is 0, LFNST may not be applied.

[274] Meanwhile, in a coding unit to which the ISP mode is applied, an example of applying LFNST when the length of one side of the partition block is less than 4 is discussed as follows.

[275] First, when the partition block size is N×2 (2×N), LFNST may be applied to the top-left region M×2 (2×M) (here, M≤N). For example, when M=8, the corresponding top-left region becomes 8×2 (2×8), and thus, the region in which 16 residual signals exist can be an input to the forward LFNST, and the forward transform matrix R× can be applied. 16 (R≤16).

[276] Here, the direct LFNST matrix may not be the matrix included in the current VVC standard, but a separate additional matrix. Moreover, for worst-case complex control, an 8x16 matrix obtained by sampling only the top 8 row vectors of a 16x16 matrix can be used for transformation. The method for adjusting the complexity will be described in detail later.

[277] Second, when the partition block size is N×1 (1×N), LFNST may be applied to the top-left region M×1 (1×M) (here, M≤N). For example, when M=16, the corresponding upper-left area becomes 16×1 (1×16), and thus, the region in which 16 residual signals exist can be the input of the direct LFNST, and the matrix R×16 (R ≤16) direct conversion.

[278] Here, the matrix of the corresponding direct LFNST may not be the matrix included in the current VVC standard, but a separate additional matrix. In addition, to control the worst-case complexity, an 8x16 matrix obtained by sampling only the top 8 row vectors of a 16x16 matrix can be used for the transformation. The complexity control method will be described in more detail later.

[279] The first embodiment and the second embodiment may be applied simultaneously, or any one of the two embodiments may be applied. In particular, in the case of the second embodiment, since a one-dimensional transformation is considered in the LFNST, it was found through experiments that the improvement in compression performance that could be obtained from the existing LFNST was not relatively large compared to the cost of signaling the LFNST index. However, in the case of the first embodiment, an improvement in compression performance similar to that obtained from conventional LFNST was observed, that is, in the case of ISP, it can be verified by experiments that applying LFNST for 2xN and Nx2 contributes to the actual compression performance.

[280] In LFNST in the current VVC, symmetry between inter-prediction modes is applied. The same LFNST set is applied to the two directional modes centered on mode 34 (prediction in the bottom-right 45-degree diagonal direction), and, for example, the same LFNST set is applied to mode 18 (horizontal direction prediction mode) and mode 50 (vertical direction prediction mode). However, in modes 35-66, when direct LFNST is applied, the input data is transposed and then LFNST is applied.

[281] Meanwhile, VVC supports a wide-angle intra-prediction (WAIP) mode, and the LFNST set is derived based on the modified intra-prediction mode considering the WAIP mode. For modes extended by WAIP, the set of LFNSTs is also determined using symmetry, as in the general directional intra-prediction mode. For example, mode -1 is symmetrical to mode 67 so that the same LFNST set applies, and mode -14 is symmetrical to mode 80 so that the same LFNST set applies. Modes 67-80 apply the LFNST transformation after transposing the input before applying the direct LFNST.

[282] In the case of the LFNST applied to the top-left M×2 (M×1) block, the above-described symmetry for the LFNST cannot be applied because the block to which the LFNST is applied is non-square. Therefore, instead of applying symmetry based on the intra-prediction mode, as in LFNST in Table 2, symmetry between an M×2 (M×1) block and a 2×M (1×M) block can be applied.

[283] FIG. 18 is a diagram illustrating the symmetry between an M×2 (M×1) block and a 2×M (1×M) block according to an example.

[284] As shown in FIG. 16, mode 2 in an Mx2 (Mx1) block can be considered as symmetrical to mode 66 in a 2xM (1xM) block, and thus the same LFNST set can be applied to a 2xM (1 ×M) and block M×2 (M×1).

[285] Here, in order to apply the LFNST set that has been applied to the M×2 (M×1) block to the 2×M (1×M) block, the LFNST set is selected based on mode 2 instead of mode 66. That is, before applying direct LFNST, you can apply LFNST after transposing the 2xM block input (1xM).

[286] FIG. 19 is a diagram illustrating an example of transposing a 2xM block according to an example.

[287] FIG. 19A is a diagram illustrating that LFNST can be applied by reading input data in column first order with respect to a 2xM block, and FIG. 19B is a diagram illustrating that the LFNST is applied by reading the input data in row first order with respect to an M×2 (M×1) block. The method of applying LFNST to a top-left M×2 (M×1) block or a 2×M (M×1) block is summarized as follows.

[288] 1. First, as shown in FIG. 19A and 19B, the input data is ordered to form a forward LFNST input vector. For example, with reference to FIG. 18, for the M×2 block predicted in mode 2, the order of FIG. 19(b), and for the 2xM block predicted in mode 66, after ordering the input data according to the order shown in FIG. 19(a), the LFNST set for mode 2 may be used.

[289] 2. For an M×2 (M×1) block, the LFNST set is determined based on the modified WAIP intra-prediction mode. As described above, a predetermined mapping relationship is set between the intra prediction mode and the LFNST set, and may be represented by a mapping table as shown in Table 2.

[290] For a 2×M (1×M) block, after obtaining a symmetric mode with respect to the prediction mode (mode 34 in the case of the VVC standard) in a diagonal direction of 45 degrees to the right down from the WAIP-modified intra-prediction mode, the LFNST set is determined based on the appropriate symmetrical mode and mapping table. The symmetrical mode (y) with respect to mode 34 can be derived using the following equation. The mapping table is described in more detail below.

[291] [Equation 11]

if 2≤x≤66, y=68-x,

otherwise (x≤-1 or x≥67), y=66-x

[292] 3. When direct LFNST is applied, the transform coefficient can be derived by multiplying the input prepared by process 1 by the LFNST kernel. The LFNST core may be selected from the LFNST set defined in process 2 and a predefined LFNST index.

[293] For example, if M=8 and a 16x16 matrix is applied as the LFNST kernel, then 16 transform coefficients can be generated by multiplying the matrix by 16 inputs. The generated transform coefficients may be ordered in the top-left 8×2 or 2×8 region according to the scan order used in the VVC standard.

[294] FIG. 20 shows the scanning order for an 8x2 or 2x8 area according to an example.

[295] All areas other than the top-left area of 8x2 or 2x8 may be filled with zero values (set to zero), or an existing transform coefficient to which the primary transform has been applied may be maintained as it is. The predefined LFNST index can be one of the values (0, 1, 2) of the LFNST index, which is checked when calculating the cost according to the RD criterion when the value of the LFNST index changes during the encoding process.

[296] For a configuration that results in a worst-case computational complexity below a certain level (e.g., 8 multiplications per sample), for example, after generating only eight transform coefficients by multiplying an 8x16 matrix obtained by taking only the top eight rows of a 16x16 matrix , the eight transform coefficients may be ordered according to the scan order, as shown in FIG. 20 and zeroing may be applied to the remaining coefficient regions. Worst case complexity control will be described later.

[297] 4. When applying the inverse LFNST, a predetermined number (eg, 16) of transform coefficients is set as the input vector, and after selecting the LFNST kernel (eg, a 16×16 matrix) derived from the LFNST set obtained from process 2 and the LFNST index obtained by parsing, the output vector can be derived by multiplying the LFNST kernel by the input vector.

[298] In the case of an M×2 (M×1) block, the output vector may be ordered according to row first order, as shown in FIG. 19B, and in the case of a 2×M (1×M) block, the output vector may be ordered according to column first order as shown in FIG. 19A.

[299] For the remaining area, except for the area where the corresponding output vector is located in the top-left area Mx2 (Mx1) or 2xM (Mx2), and the area other than the top-left area Mx2 (M×1) or 2×M (M×2) in the partition block, all of which may be configured to be zeroed (zeroed out) or to keep the transform coefficient recovered by the residual encoding and dequantization processes as it is.

[300] As in clause 3, when generating the input vector, the input data may be ordered according to the scan order of FIG. 20, and the input vector may be generated with a reduced number of inputs (eg, 8 instead of 16) to adjust the worst case computational complexity to a certain level or less.

[301] For example, when using 8 inputs and when M=8, only the left 16x8 matrix can be taken from the corresponding 16x16 matrix, and multiplied, thereby obtaining 16 outputs. Worst case complexity control will be described later.

[302] While the above embodiment represents a case in which symmetry is applied between an M×2 (M×1) block and a 2×M (1×M) block when applying LFNST, different sets of LFNSTs may be applied to two block shapes , respectively, in accordance with another example.

[303] Next, various configuration examples of the LFNST set for the ISP mode and the mapping method using the intra prediction mode will be described.

[304] In the case of the ISP mode, the configuration of the LFNST set may be different from the configuration of the existing LFNST set. In other words, different kernels from existing LFNST kernels may be used, and a mapping table different from the mapping table between the intra prediction mode index applied to the current VVC standard and the LFNST set may be used. The mapping table applied to the current VVC standard can be shown in Table 2.

[305] In Table 2, the value of preModeIntra means the intra prediction mode value changed considering WAIP, and the value of lfnstTrSetIdx is an index value indicating a specific LFNST set. Each LFNST set is configured with two LFNST cores.

[306] When the ISP prediction mode is applied, and when the horizontal length and vertical length of each partition block is equal to or greater than 4, the same kernels as the LFNST kernels used in the current VVC standard can be applied, and a mapping table can also be applied. as it is. Of course, it is also possible to use a different mapping table and LFNST kernels than the current VVC standard.

[307] When the ISP prediction mode is applied and when the horizontal length or vertical length of each partition block is less than 4, the LFNST table and kernels different from the current VVC standard may be applied. Next, Tables 7-9 show mapping tables between the intra prediction mode value (the intra prediction mode value modified considering WAIP) and the LFNST set that can be applied to an M×2 block (M×1) or a 2×M block ( 1×M).

[308] [Table 6]

[309] [Table 7]

[310] [Table 8]

[311] The first mapping table of Table 6 includes seven sets of LFNSTs, and the mapping table of Table 7 includes four sets of LFNSTs, and the mapping table of Table 8 includes two sets of LFNSTs. As another example, when configured with a single LFNST set, the value of lfnstTrSetIdx may be fixed to 0 relative to the value of preModeIntra.

[312] Next, a method for maintaining the worst case computational complexity when LFNST is applied to the ISP mode will be described.

[313] In the case of the ISP mode, in order to keep the number of multiplications per sample (or factor or position) at a certain value or less when LFNST is applied, the application of LFNST may be restricted. Depending on the partition block size, the number of multiplications per sample (or factor or position) can be kept at 8 or less by applying LFNST as follows.

[314] 1. When the horizontal length and vertical length of the partition block are equal to or greater than 4, the same method as the worst-case calculation complexity adjustment method for LFNST in the current VVC standard can be applied.

[315] That is, when the partition block is a 4×4 block, instead of a 16×16 matrix, in the forward direction, an 8×16 matrix obtained by sampling the top 8 rows from a 16×16 matrix can be applied, and in the reverse direction, a 16×8 matrix obtained by discretizing the left 8 columns from a 16×16 matrix can be used. Additionally, when the partition block is an 8x8 block, forward instead of a 16x48 matrix, an 8x48 matrix obtained by sampling the top 8 rows from a 16x48 matrix can be used, and backwards instead of a 48x16 matrix , a 48x8 matrix obtained by discretizing the left 8 columns from a 48x16 matrix can be used.

[316] In the case of a 4xN or Nx4 block (N>4), when direct transformation is performed, the 16 coefficients generated after applying the 16x16 matrix only to the top-left 4x4 block are ordered in the top-left region 4x4, and other areas can be filled with 0 values. Additionally, when performing an inverse transform, the 16 coefficients located in the top-left 4x4 block can be ordered in scan order to configure an input vector, and then 16 output data can be generated by multiplying by a 16×16 matrix. The generated output may be ordered in the top-left 4×4 region, and the remaining regions except for the top-left 4×4 region may be filled with zeros.

[317] In the case of an 8×N or N×8 (N>8) block, when direct transformation is performed, the 16 coefficients generated after applying the 16×48 matrix only to the ROI region in the top-left 8×8 block (the remaining regions, excluding the bottom-right 4x4 block from the top-left 8x8 block) may be ordered in the top-left 4x4 region, and the other regions may be filled with 0 values. in the top-left 4x4 block may be ordered in scan order to configure an input vector, and then 48 outputs may be generated by multiplying by a 48x16 matrix. The generated output may be filled in the ROI area and other areas may be filled with 0 values.

[318] 2. When the partition block size is N×2 or 2×N and LFNST is applied to the top-left region M×2 or 2×M (M≤N), a matrix to which discretization has been applied according to the value N.

[319] When M=8, for a partition block N=8, that is, an 8x2 or 2x8 block, in the case of direct transformation, instead of a 16x16 matrix, an 8x16 matrix obtained by sampling the top 8 rows can be used from a 16×16 matrix, and in the case of the inverse transformation, instead of a 16×16 matrix, a 16×8 matrix obtained by discretizing the left 8 columns from a 16×16 matrix can be used.

[320] When N is greater than 8, in the case of direct conversion, the 16 output data generated after applying the 16×16 matrix to the top-left 8×2 or 2×8 block may be ordered in the top-left 8×2 block or 2x8, and the remaining areas may be filled with 0 values. When performing the inverse transformation, the 16 coefficients located in the top-left 8x2 or 2x8 block may be ordered in scan order to configure the input vector, and then 16 output data can be generated by multiplying by a 16×16 matrix. The generated output can be ordered in the top-left 8x2 or 2x8 block, and all other areas can be filled with 0 values.

[321] 3. When the partition block size is N×1 or 1×N and LFNST is applied to the top-left region M×1 or 1×M (M≤N), a matrix to which discretization has been applied according to N value.

[322] When M=16, for a partition block N=16, that is, a 16x1 or 1x16 block, in the case of direct transformation, instead of a 16x16 matrix, an 8x16 matrix obtained by sampling the top 8 rows can be used from a 16×16 matrix, and in the case of the inverse transformation, instead of a 16×16 matrix, a 16×8 matrix obtained by discretizing the left 8 columns from a 16×16 matrix can be used.

[323] When N is greater than 16, in the case of direct conversion, 16 output data generated after applying a 16×16 matrix to a 16×1 or 1×16 top-left block may be ordered in a 16×1 top-left block or 1x16, and the remaining areas can be filled with 0 values. When performing the inverse transformation, the 16 coefficients located in the top-left block of 16x1 or 1x16 can be ordered in scan order to configure the input vector, and then 16 output data can be generated by multiplying by a 16×16 matrix. The generated output can be ordered in the top-left 16x1 or 1x16 block, and all other areas can be filled with 0 values.

[324] As another example, to keep the number of multiplications per sample (or coefficient or position) at some value or less, the number of multiplications per sample (or coefficient or position) based on the size of the ISP coding unit, and not ISP split block size may be supported at 8 or less. If there is only one block among the ISP split blocks that satisfies the condition under which the LFNST is applied, the LFNST worst case complexity calculation may be applied based on the corresponding coding unit size rather than the split block size. For example, when a luma coding block for a certain coding unit is partitioned into 4 partition blocks of 4×4 and encoded by ISP, and when a non-zero transform coefficient exists for two partition blocks among them, the other two partition blocks can be respectively set to generate 16 coefficients conversions instead of 8 (encoder based).

[325] Next, a method for signaling the LFNST index in the case of the ISP mode will be described.

[326] As described above, the LFNST index may have the values 0, 1, and 2, where 0 indicates that no LFNST is applied, and 1 and 2, respectively, indicate any one of the two LFNST kernel matrices included in the selected LFNST set. The LFNST is applied based on the LFNST kernel matrix selected by the LFNST index. The method of passing the LFNST index in the current VVC standard will be described as follows.

[327] 1. An LFNST index may be transmitted once per coding unit (CU), and in the case of a dual tree, individual LFNST indices may be signaled for a luminance block and a chrominance block, respectively.

[328] 2. When the LFNST index is not signaled, the value of the LFNST index is assumed to be the default value of 0. The case where the value of the LFNST index is assumed to be 0 is as follows.

[329] A. In the case of a mode in which no transform is applied (eg, skip transform, BDPCM, lossless coding, etc.)

[330] B. When the primary transform is not DCT-2 (DST7 or DCT8), that is, when the horizontal direction transform or the vertical direction transform is not DCT-2

[331] C. When the horizontal length or vertical length for the luminance block of the coding unit exceeds the size of the maximum luminance transform to be converted, for example, when the size of the maximum luminance transform to be converted is 64, and when the size for the luminance block of the coding block is 128×16, LFNST cannot apply.

[332] In the case of a dual tree, it is determined whether or not the size of the maximum luminance transform for each of the coding unit for the luma component and the coding unit for the chrominance component is exceeded. That is, for the luma block, it is checked whether the maximum luminance transform transform size is exceeded, and for the chrominance block, it is checked whether the horizontal/vertical length of the corresponding luma block for the color format and the maximum luminance transform transform size are exceeded. For example, when the color format is 4:2:0, the horizontal/vertical length of the corresponding luma block is equal to twice the length of the corresponding chroma block, and the transform size of the corresponding luma block is equal to twice the size of the corresponding chrominance block. As another example, when the color format is 4:4:4, the horizontal/vertical length and transform size of the corresponding luma block are the same as the dimensions of the corresponding chrominance block.

[333] A 64-length transform or a 32-length transform may mean a transform applicable to a width or height having a length of 64 or 32, respectively, and "transform size" may mean 64 or 32 as the corresponding length.

[334] In the case of a single tree, after checking whether or not the horizontal length or vertical length of the luminance block exceeds the maximum size of the luminance transform block to be converted, if it does, then the LFNST index signaling may be omitted.

[335] D. The LFNST index may be transmitted only when the horizontal length and vertical length of the coding unit are equal to or greater than 4.

[336] In the case of a dual tree, the LFNST index can only be signaled when both the horizontal and vertical lengths for the corresponding component (i.e., luminance or chrominance component) are equal to or greater than 4.

[337] In the case of a single tree, the LFNST index may be signaled when both the horizontal and vertical lengths for the luma component are equal to or greater than 4.

[338] E. If the position of the last non-zero coefficient is not a DC position (top-left block position), and if the position of the last non-zero coefficient is not a DC position, in the case of a dual-type tree luma block, then the LFNST index is transmitted. In the case of a tree chromaticity block of dual type, if either one of the position of the last non-zero coefficient for Cb and the position of the last non-zero coefficient for Cr is not a DC position, the corresponding LNFST index is transmitted.

[339] In the case of a single type tree, if the position of the last non-zero coefficient of any one of the luma component, the Cb component, and the Cr component is not a DC position, the LFNST index is transmitted.

[340] Here, if a Coded Block Flag (CBF) value indicating whether or not a transform coefficient exists for one transform block is 0, the position of the last non-zero coefficient for the corresponding transform block is not checked to determine whether or not LFNST is signaled. That is, when the value of the corresponding CBF is 0, since no transformation is applied to the corresponding block, the position of the last non-zero coefficient may be ignored when checking the condition for signaling the LFNST index.

[341] For example, 1) in the case of a dual type tree and a luminance component, if the value of the corresponding CBF is 0, the LFNST index is not signaled, 2) in the case of a dual type tree and a chrominance component, if the CBF value of Cb is 0 and the CBF value of Cr is 1, then only the position of the last non-zero coefficient for Cr is checked, and the corresponding LFNST index is transmitted, 3) in the case of a single type tree, the position of the last non-zero coefficient is checked only for components having a CBF value of 1 for each of luma, Cb and cr.

[342] F. When it is confirmed that the transform coefficient exists in a position other than the position where the transform coefficient LFNST may exist, the LFNST index signaling may be omitted. In the case of a 4×4 transform block and an 8×8 transform block, LFNST transform coefficients may exist eight positions from the DC position according to the scan order of the transform coefficients in the VVC standard, and the remaining positions are filled with zeros. Additionally, when the 4×4 transform block and the 8×8 transform block are not present, the LFNST transform coefficients may exist sixteen positions from the DC position according to the scan order of the transform coefficients in the VVC standard, and the remaining positions are filled with zeros.

[343] Accordingly, if non-zero transform coefficients exist in an area to be filled with a value of zero after the residual encoding advances, the LFNST index signaling may be omitted.

[344] Meanwhile, the ISP mode may also be applied to the luminance block only, or may be applied to both the luminance block and the chrominance block. As described above, when ISP prediction is applied, the corresponding coding unit may be divided into two or four partition blocks and predicted, and a transform may be applied to each of the partition blocks. Therefore, when specifying the condition for signaling the LFNST index in coding units, it is necessary to take into account the fact that LFNST can be applied to the corresponding partition blocks. In addition, when the ISP prediction mode is applied only to a specific component (eg, luminance block), the LFNST index must be signaled in view of the fact that only this component is divided into partition blocks. The LFNST signaling methods available in the ISP mode are summarized as follows.

[345] 1. The LFNST index may be transmitted once per coding unit (CU), and in the case of a dual tree, separate LFNST indices may be signaled for the luma block and the chrominance block, respectively.

[346] 2. When the LFNST index is not signaled, the value of the LFNST index is assumed to be the default value of 0. The case where the value of the LFNST index is assumed to be 0 is as follows.

[347] A. In the case of a mode in which no transform is applied (eg, skip transform, BDPCM, lossless coding, etc.)

[348] B. When the horizontal length or vertical length for the luminance block of the coding unit exceeds the size of the maximum luminance transform to be converted, for example, when the size of the maximum luminance transform to be converted is 64, and when the size for the luminance block of the coding block is 128×16, LFNST cannot apply.

[349] Whether or not the LFNST index should be signaled may be determined based on the split block size instead of the coding unit. That is, if the horizontal or vertical split block length for the corresponding luminance block exceeds the size of the maximum luminance transform to be converted, the LFNST index signaling may be omitted, and the LFNST index value may be assumed to be 0.

[350] In the case of a dual tree, it is determined whether or not the size of the maximum luminance transform for each of the coding unit or partition block for the luma component and the coding unit or partition block for the chrominance component is exceeded. That is, if the horizontal and vertical lengths of the coding unit or partition block for luma are compared with the maximum luminance transform size, respectively, and at least one of them is greater than the maximum luma transform size, then LFNST does not apply, and in the case of a coding unit or the split block for chroma, the horizontal/vertical length of the corresponding luminance block for the color format, and the size of the maximum convertible luminance transform are compared. For example, when the color format is 4:2:0, the horizontal/vertical length of the corresponding luma block is equal to twice the length of the corresponding chroma block, and the transform size of the corresponding luma block is equal to twice the size of the corresponding chrominance block. As another example, when the color format is 4:4:4, the horizontal/vertical length and transform size of the corresponding luma block are the same as those of the corresponding chrominance block.

[351] In the case of a single tree, after checking whether or not the horizontal length or vertical length for the luminance block (coding unit or split block) exceeds the maximum size of the luminance transform block to be converted, if it does, then the LFNST index signaling may be omitted.

[352] C. If the LFNST included in the current VVC standard is used, the LFNST index can only be transmitted when both the horizontal length and the vertical length of the partition block are equal to or greater than 4.

[353] If LFNST is applied to a 2×M (1×M) or M×2 (M×1) block in addition to the LFNST included in the current VVC standard, the LFNST index may be transmitted only when the split block size is equal to or greater than than a 2×M (1×M) or M×2 (M×1) block. Here, the expression "block PxQ is equal to or greater than block R×S" means that P≥R and Q≥S.

[354] Thus, the LFNST index can only be transmitted when the partition block is equal to or greater than the minimum size to which the LFNST applies. In the case of a dual tree, the LFNST index can only be signaled when the partition block for the luma or chrominance component is equal to or greater than the minimum size to which the LFNST applies. In the case of a single tree, the LFNST index may only be signaled when the split box for the luminance component is equal to or greater than the minimum size to which the LFNST applies.

[355] In this document, the expression "M×N block is greater than or equal to K×L block" means that M is greater than or equal to K and N is greater than or equal to L. The expression "M×N block is greater than K×L block" means that M is greater than or equal to K and N is greater than or equal to L, and that M is greater than K or N is greater than L. The expression "M×N block is less than or equal to K×L block" means that M is less than or is equal to K and N is less than or equal to L, while "block M×N is less than block K×L" means that M is less than or equal to K and N is less than or equal to L, and that M is less than K, or N is less than L.

[356] D. If the position of the last non-zero coefficient is not a DC position (top-left block position) and if the position of the last non-zero coefficient is not a DC position in any one of all partition blocks, in the case of a dual-type tree luma block, then the index LFNST is transmitted. In the case of a dual type tree and a chrominance block, if at least one position of the last non-zero coefficient of all partition blocks for Cb (if the ISP mode is not applied to the chrominance component, the number of partition blocks is considered to be one) and the position of the last non-zero coefficient of all partition blocks for Cr (if the ISP mode is not applied to the chrominance component, the number of partition blocks is considered to be one) is not a DC position, the corresponding LNFST index may be transmitted.

[357] In the case of a single type tree, if the position of the last non-zero coefficient of any one of all partition blocks for the luminance component, the Cb component, and the Cr component is not a DC position, the corresponding LFNST index may be transmitted.

[358] Here, if a Coded Block Flag (CBF) value indicating whether a transform coefficient exists for each split block is 0, the position of the last non-zero coefficient for the corresponding split block is not checked to determine whether or not the LFNST index is signaled. That is, when the corresponding CBF value is 0, since no transformation is applied to the corresponding block, the position of the last non-zero coefficient for the corresponding split block is not taken into account when checking the condition for signaling the LFNST index.

[359] For example, 1) in the case of a dual type tree and a luminance component, if the corresponding CBF value for each split box is 0, the split box is excluded when determining whether or not to signal the LFNST index, 2) in the case of a dual type tree and a component chromaticity, if the CBF value of Cb is 0 and the CBF value of Cr is 1 for each partition block, then only the position of the last non-zero coefficient for Cr is checked to determine whether or not the LFNST index should be signaled, 3) in the case of a single type tree, it can be determined whether or not the LFNST index should be signaled by checking the position of the last non-zero coefficient only for blocks having a CBF value of 1 for all luma component, Cb component, and Cr component split blocks.

[360] In the case of the ISP mode, the image information may also be configured such that the position of the last non-zero coefficient is not checked, and an embodiment of this is as follows.

[361] i. In the case of ISP mode, LFNST index signaling may be enabled without checking the position of the last non-zero coefficient for the luminance block and the chrominance block. That is, even if the position of the last non-zero coefficient for all partition blocks is a DC position or the corresponding CBF value is 0, LFNST index signaling may be enabled.

[362] ii. In the case of the ISP mode, checking the position of the last non-zero coefficient only for the luminance block may be omitted, and in the case of the chrominance block, checking the position of the last non-zero coefficient may be performed in the manner described above. For example, in the case of a dual type tree and luma block, LFNST index signaling is allowed without checking the position of the last non-zero coefficient, and in the case of a dual type tree and chrominance block, whether the corresponding LFNST index is signaled or not can be checked by checking whether or not a DC exists. position for the position of the last non-zero coefficient, in the manner described above.

[363] iii. In the case of ISP mode and a single type tree, method i or ii may be applied. That is, in the case of the ISP mode and when option i is applied to a single type tree, it is possible to omit checking the position of the last non-zero coefficient for both the luminance block and the chrominance block and enable LFNST index signaling. Alternatively, when applying option ii, for luma splitters, the check for the position of the last non-zero coefficient is omitted, and for splitters for the chrominance component (if ISP is not applied for the chroma component, the number of splitters can be considered equal to 1), the position of the last non-zero coefficient is checked in the manner described above, thereby determining whether or not the LFNST index should be signaled.

[364] E. When it is confirmed that the transform coefficient exists in a position other than the position where the transform coefficient LFNST may exist even for one tiling block among all tiling blocks, the LFNST index signaling may be omitted.

[365] For example, in the case of a 4×4 partition block and an 8×8 partition block, LFNST transform coefficients may exist eight positions from the DC position in accordance with the scan order of the transform coefficients in the VVC standard, and the remaining positions are filled with zeros. Additionally, if it is equal to or greater than 4×4 and is neither a 4×4 partition block nor an 8×8 partition block, then the LFNST transform coefficients may exist 16 positions from the DC position according to the scan order of the transform coefficients in VVC standard, and all other positions are filled with zeros.

[366] Accordingly, if non-zero transform coefficients exist in an area to be filled with a zero value after the residual encoding advances, the LFNST index signaling may be omitted.

[367] If the LFNST can be applied even when the partition block is 2xM (1xM) or Mx2 (Mx1), the area in which the LFNST transform coefficients can be located can be indicated as follows. If an area other than the area where the transform coefficients may be located may be filled with 0, and if a non-zero transform coefficient exists in the region that should be filled with 0 when LFNST is assumed to be applied, the LFNST index signaling may be omitted.

[368] i. If LFNST can be applied to a 2xM or Mx2 block and M=8, only 8 LFNST transform coefficients can be generated for a 2x8 or 8x2 partition block. When the transform coefficients are ordered in scan order, as shown in FIG. 20, the 8 transform coefficients are ordered in scan order from the DC position, and the remaining 8 positions may be filled with zeros.

[369] For a 2xN or Nx2 (N>8) partition block, 16 LFNST transform coefficients can be generated, and when the transform coefficients are ordered in scan order as shown in FIG. 20, the 16 transform coefficients are ordered in scan order from the DC position, and the rest of the area may be filled with zeros. That is, in a 2×N or N×2 partition block (N>8), an area other than the top-left 2×8 or 8×2 block may be filled with zeros. Even for a 2×8 or 8×2 partition block, 16 transform coefficients may be generated instead of 8 LFNST transform coefficients, in which case, the area to be filled with zeros does not occur. As described above, when LFNST is applied, and when a non-zero transform coefficient is found to exist in a region determined to be filled with 0 in at least one partition block, the LFNST index signaling may be omitted, and the LFNST index may be assumed to be 0.

[370] ii. If LFNST can be applied to a 1×M or M×1 block, and M=16, only 8 LFNST transform coefficients can be generated for a 1×16 or 16×1 partition block. When the transform coefficients are ordered in left-to-right or top-to-bottom scan order, the 8 transform coefficients are ordered in the corresponding scan order from the DC position, and the remaining 8 positions may be filled with zeros.

[371] For a 1xN or Nx1 partition block (N>16), 16 LFNST transform coefficients can be generated, and when the transform coefficients are ordered in left-to-right or top-to-bottom scan order, the 16 transform coefficients are ordered in the corresponding scan order from DC position, and the rest of the area can be filled with zeros. That is, in a 1×N or N×1 (N>16) partition block, an area other than the top-left 1×16 or 16×1 block may be filled with zeros.

[372] Even for a partition block of 1×16 or 16×1, 16 transform coefficients may be generated instead of 8 LFNST transform coefficients, in which case, the area to be filled with zeros does not occur. As described above, when LFNST is applied, and when a non-zero transform coefficient is found to exist in a region determined to be filled with 0 in at least one partition block, the LFNST index signaling may be omitted, and the LFNST index may be assumed to be 0.

[373] Meanwhile, in the case of the ISP mode, the length condition is independently considered for the horizontal direction and the vertical direction, and DST-7 is applied instead of DCT-2 without signaling for the MTS index. It is determined whether or not the horizontal or vertical length is greater than or equal to 4 and less than or equal to 16, and a primary transformation kernel is determined according to the result of the determination. Accordingly, in the case of the ISP mode where LFNST can be applied, the following configuration of the transform combination is possible.

[374] 1. When the LFNST index is equal to 0 (including the case in which the LFNST index is assumed to be 0), it is possible to follow the primary conversion decision during ISP included in the current VVC standard. In other words, it can be checked whether or not the length condition (equal to or greater than 4, or equal to or less than 16) is satisfied independently for the horizontal and vertical directions, respectively, and if it is satisfied, then DST-7 can be applied instead of DCT -2 for the primary conversion, and if it is not satisfied then DCT-2 may be applied.

[375] 2. For the case in which the LFNST index is greater than 0, the following two configurations are possible as the primary conversion.

[376] A. DCT-2 can be used for both horizontal and vertical directions.

[377] B. The primary conversion decision during ISP included in the current VVC standard can be followed. In other words, it can be checked whether or not the length condition (equal to or greater than 4, or equal to or less than 16) is satisfied independently for the horizontal and vertical directions, respectively, and if it is satisfied, then DST-7 can be applied instead of DCT -2, and if it is not satisfied, then DCT-2 may be applied.

[378] In the ISP mode, the image information may be configured such that the LFNST index is transmitted for each split block rather than for each coding unit. In this case, in the above-described LFNST index signaling method, only one partition block may be considered to exist in the unit in which the LFNST index is transmitted, and it may be determined whether or not the LFNST index should be signaled.

[379] Meanwhile, the signaling order of the LFNST index and the MTS index will be described below.

[380] According to an example, the LFNST index signaled in the residual coding may be encoded after the coding position for the position of the last non-zero coefficient, and the MTS index may be encoded immediately after the LFNST index. In case of this configuration, the LFNST index can be signaled for each transformation unit. Alternatively, even if not signaled in the residual encoding, the LFNST index may be encoded after the encoding for the position of the last significant coefficient, and the MTS index may be encoded after the LFNST index.

[381] The syntax for encoding the remainder according to the example is as follows.

[382] [Table 9]

[383]

[384] The values of the main variables shown in Table 9 are as follows.

[385] 1. cbWidth, cbHeight: width and height of the current encoding block

[386] 2. log2TbWidth, log2TbHeight: The base 2 logarithm value of the width and height of the current transform block, it can be reduced, by reflecting null, to the top-left region where a non-zero coefficient may exist.

[387] 3. sps_lfnst_enabled_flag: a flag indicating whether or not LFNST is enabled, if the flag value is 0, it indicates that LFNST is not enabled, and if the flag value is 1, it indicates that LFNST is enabled. This is defined in the Sequence Parameter Set (SPS).

[388] 4. CuPredMode[chType][x0][y0]: prediction mode corresponding to the variable chType and position (x0, y0), chType may have values of 0 and 1, where 0 indicates a luma component and 1 indicates a chrominance component. The position (x0, y0) indicates the position in the picture, and MODE_INTRA (intra-prediction) and MODE_INTER (inter-prediction) are possible as the value of CuPredMode[chType][x0][y0].

[389] 5. IntraSubPartitionsSplit[x0][y0]: The contents of location (x0, y0) are the same as in clause 4. This indicates which ISP split is applied at location (x0, y0), ISP_NO_SPLIT indicates that the coding unit corresponding to the position (x0, y0) is not divided into partition blocks.

[390] 6. intra_mip_flag[x0][y0]: The position contents of (x0, y0) are the same as in item 4 above. intra_mip_flag is a flag indicating whether or not a matrix-based prediction (MIP) mode is applied. If the flag value is 0, this indicates that MIP is not allowed, and if the flag value is 1, it indicates that MIP is allowed.

[391] 7. cIdx: A value of 0 indicates luminance, and values of 1 and 2 indicate Cb and Cr, which are respectively the chrominance components.

[392] 8. treeType: indicates single tree and dual tree, etc. (SINGLE_TREE: single tree, DUAL_TREE_LUMA: dual tree for luma components, DUAL_TREE_CHROMA: dual tree for chroma component).

[393] 9. tu_cbf_cb[x0][y0]: The position contents (x0, y0) are the same as in clause 4. This indicates a coded block flag (CBF) for the Cb component. If its value is 0, it means that there are no non-zero coefficients in the corresponding transformation unit for the Cb component, and if its value is 1, it indicates that non-zero coefficients are present in the corresponding transformation unit for the Cb component.

[394] 10. lastSubBlock: indicates the position in the scan order of the sub-block (coefficient group (CG)) in which the last non-zero coefficient is located. 0 indicates the subbox in which the DC component is included, and if the value is greater than 0, it is not the subbox in which the DC component is included.

[395] 11. lastScanPos: indicates the position where the last significant coefficient is in the scan order within one sub-block. If one subblock includes 16 positions, values from 0 to 15 are possible.

[396] 12. lfnst_idx[x0][y0]: LFNST index syntax element to be parsed. If it is not parsed, it is assumed to be 0. That is, the default value is set to 0, indicating that LFNST is not applied.

[397] 13. LastSignificantCoeffX, LastSignificantCoeffY: indicate the x and y coordinates where the last significant coefficient is located in the transform block. x-coordinate starts at 0 and increases from left to right, and y-coordinate starts at 0 and increases from top to bottom. If the values of both variables are 0, this means that the last significant coefficient is located in DC.

[398] 14. cu_sbt_flag: A flag indicating whether or not the sub-block transform (SBT) included in the current VVC standard is allowed or not. If the flag value is 0, this indicates that SBT is not allowed, and if the flag value is 1, this indicates that SBT is allowed.

[399] 15. sps_explicit_mts_inter_enabled_flag, sps_explicit_mts_intra_enabled_flag: flags indicating whether or not explicit MTS applies to inter-CU and intra-CU, respectively. If the value of the corresponding flag is 0, this indicates that the MTS is not allowed for the inter-CU or intra-CU, and if the value of the corresponding flag is 1, this indicates that the MTS is allowed.

[400] 16. tu_mts_idx[x0][y0]: MTS index syntax element to be parsed. If it is not parsed, it is assumed to be 0. That is, the default value is set to 0, indicating that DCT-2 is allowed for both horizontal and vertical directions.

[401] As shown in Table 9, in the case of a single tree, it is possible to determine whether or not the LFNST index should be signaled using only the position condition of the last significant coefficient for luminance. That is, if the position of the last significant coefficient is not DC and the last significant coefficient exists in the top-left subblock (CG), for example, a 4x4 block, then the LFNST index is signaled. In this case, in the case of a 4×4 transform block and an 8×8 transform block, the LFNST index is only signaled when the last significant coefficient exists at positions 0-7 in the top-left sub-block.

[402] In the case of a dual tree, the LFNST index is signaled independently of each of luma and chrominance, and in the case of chrominance, the LFNST index can be signaled by applying the last significant coefficient position condition to the Cb component only. The corresponding condition may not be checked for the Cr component, and if the CBF value for Cb is 0, the LFNST index may be signaled by applying the position condition of the last significant coefficient to the Cr component.

[403] 'Min(log2TbWidth, log2TbHeight)>=2' in Table 9 may be expressed as "Min(tbWidth, tbHeight)>=4", and 'Min(log2TbWidth, log2TbHeight)>=4' may be expressed as "Min( tbWidth, tbHeight)>= 16”.

[404] In Table 9, log2ZoTbWidth and log2ZoTbHeight mean logarithm values with bases of width and height equal to 2 (base 2) for the top-left region where the last significant coefficient can exist, when set to zero, respectively.

[405] As shown in Table 9, the values of log2ZoTbWidth and log2ZoTbHeight can be updated in two places. The first is before the value of the MTS index or LFNST index is parsed, and the second is after the MTS index is parsed.

[406] The first update is performed before parsing the MTS index value (tu_mts_idx[x0][y0]), so that log2ZoTbWidth and log2ZoTbHeight can be set regardless of the MTS index value.

[407] After the MTS index is parsed, log2ZoTbWidth and log2ZoTbHeigh are set for the MTS index greater than 0 (DST-7/DCT-8 combination). When DST-7/DCT-8 is independently applied in each of the horizontal and vertical directions in the primary transform, there can be up to 16 significant coefficients per row or column in each direction. That is, after applying DST-7/DCT-8 with a length of 32 or more, up to 16 transform coefficients can be output for each row or column on the left or top. Accordingly, in the 2D block, when DST-7/DCT-8 is applied in both the horizontal direction and the vertical direction, significant coefficients can only exist in the top-left region of 16×16.

[408] In addition, when DCT-2 is independently applied in each of the horizontal and vertical directions in the current primary transform, there may be up to 32 significant coefficients per row or column in each direction. That is, when using a DCT-2 with a length of 64 or more, up to 32 transform coefficients can be output for each row or column on the left or top. Accordingly, in the 2D block, when DCT-2 is applied in both the horizontal direction and the vertical direction, significant coefficients may only exist in the top-left region of 32×32.

[409] In addition, when DST-7/DCT-8 is applied on one side and DCT-2 is applied on the other side to the horizontal and vertical directions, 16 significant coefficients may exist in the first direction, and 32 significant coefficients may exist in the latter. direction. For example, in the case of a 64×8 transform block, if DCT-2 is applied in the horizontal direction and DST-7 is applied in the vertical direction (a situation may arise where an implicit MTS is applied), a significant coefficient may exist in the region up to the top-left region 32 ×8.

[410] If, as shown in Table 9, log2ZoTbWidth and log2ZoTbHeight are updated in two places, that is, before parsing the MTS index, the last_sig_coeff_x_prefix and last_sig_coeff_y_prefix ranges can be determined by log2ZoTbWidth and log2ZoTbHeight, as shown in the table below.

[411]

[Table 10] 7.4.9.11 Remainder encoding semantics
…
last_sig_coeff_x_prefix specifies the prefix of the column position of the last significant coefficient in the scan order in the transform block. Last_sig_coeff_x_prefix values must be in the range 0 to (log2ZoTbWidth << 1)-1), inclusive.
When last_sig_coeff_x_prefix is missing, it is assumed to be 0.
last_sig_coeff_y_prefix specifies the string position prefix of the last significant coefficient in the scan order in the transform block. Last_sig_coeff_y_prefix values must be in the range 0 to (log2ZoTbHeight << 1)-1), inclusive.
When last_sig_coeff_y_prefix is absent, it is assumed to be 0.
…

[412] Additionally, in this case, the maximum values of last_sig_coeff_x_prefix and last_sig_coeff_y_prefix can be set by mapping log2ZoTbWidth and log2ZoTbHeight values during binarization for last_sig_coeff_x_prefix and last_sig_coeff_y_prefix.

[413] [Table 11]

[414] According to an example, in the case where the ISP and LFNST mode is applied, when the Table 9 signaling is applied, the specification text can be configured as shown in Table 12. Compared with Table 9, the LFNST index signaling condition is only in the case except the ISP mode (IntraSubPartitionsSplit[x0][y0]==ISP_NO_SPLIT in Table 9) is removed.

[415] In a single tree, when the LFNST index transmitted for the luma component (cIdx=0) is reused for the chrominance component, the LFNST index transmitted for the first ISP split block in which a significant coefficient exists can be applied to the transform block. Alternatively, even in a single tree, the LFNST index may be signaled for the chrominance component separately from the index for the luma component. The description of the variables in Table 12 is the same as in Table 9.

[416] [Table 12]

[417] According to another example, in Table 12, when the last significant coefficient is allowed to be in the DC position for the ISP, the condition for parsing with respect to the LFNST index can be changed as follows.

[418] [Table 13]

[419] According to an example, the LFNST index and/or the MTS index may be signaled at the coding unit level. As described above, the LFNST index may have three values 0, 1 and 2, where 0 indicates that no LFNST is applied and 1 and 2 respectively indicate the first candidate and the second candidate of the two LFNST core candidates included in the selected LFNST set. The LFNST index is encoded by truncated unitary binarization, and the values 0, 1, and 2 may be encoded as bin strings 0, 10, and 11, respectively.

[420] According to an example, LFNST can only be applied when DCT-2 is applied in both the horizontal direction and the vertical direction in primary conversion. Therefore, if the MTS index is signaled after the LFNST index is signaled, the MTS index can be signaled only when the LFNST index is 0, and primary transformation can be performed by applying DCT-2 in both the horizontal direction and the vertical direction without signaling the MTS index when LFNST index is not equal to 0.

[421] The MTS index may have the values 0, 1, 2, 3, and 4, where 0, 1, 2, 3, and 4 may indicate that DCT-2/DCT-2, DST-7/DST-7, DCT- 8/DST-7, DST-7/DCT-8, DCT-8/DCT-8 are applied in horizontal and vertical directions, respectively. In addition, the MTS index may be encoded with truncated unitary binarization and the values 0, 1, 2, 3, and 4 may be encoded as bin strings 0, 10, 110, 1110, and 1111, respectively.

[422] The LFNST index and the MTS index may be signaled at the coding unit level, and the MTS index may then be encoded after the LFNST index at the coding unit level. The coding unit syntax table for this is as follows.

[423] [Table 14]

[424] Comparing Table 14 with Table 13, a condition for checking whether the value of tu_mts_idx[x0][y0] is 0 (i.e., checking whether DCT-2 is applied in the horizontal direction and in the vertical direction), in the condition for lfnst_idx[x0][y0] alarm changed to condition to check if transform_skip_flag[x0][y0] is 0 (!transform_skip_flag[x0][y0]). transform_skip_flag[x0][y0] indicates whether the coding unit is encoded in a transform-skip mode in which a transform is skipped and the flag is signaled before the MTS index and the LFNST index. That is, since lfnst_idx[x0][y0] is signaled before the value of tu_mtx_idx[x0][y0] is signaled, only the condition for the value of transform_skip_flag[x0][y0] can be checked.

[425] As shown in Table 14, a set of conditions is checked when encoding tu_mts_idx[x0][y0], and tu_mts_idx[x0][y0] is signaled only when the value of lfnst_idx[x0][y0] is equal to 0, as described above.

[426] tu_cbf_LUMA[x0][y0] is a flag indicating whether there is a significant coefficient for the luma component, and cbWidth and cbHeight indicate the width and height of the coding unit for the luminance component, respectively.

[427] In Table 14, (IntraSubPartitionsSplit[x0][y0]== ISP_NO_SPLIT) indicates that ISP mode is not applied, and (!cu_sbt_flag) indicates that SBT is not applied.

[428] According to Table 14, when both the width and height of the coding unit for the luminance component are 32 or less, tu_mts_idx[x0][y0] is signaled, that is, whether MTS is applied is determined by the width and height of the coding unit for the component brightness.

[429] According to another example, when tiling of a transform unit (TU) occurs (for example, when the maximum transform size is set to 32, a 64×64 coding unit is divided into four 32×32 transform blocks and encoded), the MTS index may be signaled based on the size of each transform block. For example, when both the width and height of a transform block are 32 or less, the same MTS index value can be applied to all transform blocks in a coding unit, thereby applying the same primary transform. Also, when a transform block is tiled, the value of tu_cbf_LUMA[x0][y0] in Table 14 may be the CBF value for the top-left transform block, or may be set to 1 when the CBF value for even one transform block out of all conversion blocks is 1.

[430] According to an example, when the ISP mode is applied to the current block, LFNST may be applied, in which case Table 14 may be changed as shown in Table 15.

[431] [Table 15]

[432] As shown in Table 15, even in ISP mode, (IntraSubPartitionsSplitType!=ISP_NO_SPLIT)lfnst_idx[x0][y0] can be configured to be signaled and the same LFNST index value can be applied to all ISP split blocks.

[433] Additionally, as shown in Table 15, since tu_mts_idx[x0][y0] can only be signaled in a mode excluding the ISP mode, the MTS index encoding part is the same as in Table 14.

[434] As shown in Table 14 and Table 15, when the MTS index is signaled immediately after the LFNST index, the primary conversion information cannot be known when the residual encoding is performed. That is, the MTS index is signaled after the coding of the remainder. Accordingly, in the remainder coding portion, the portion on which nulling is performed leaving only 16 coefficients for DST-7 or DCT-8 having a length of 32 may vary as shown below in Table 16.

[435]

[Table 16] residual_coding( x0, y0, log2TbWidth, log2TbHeight, cIdx ) { Descriptor if( ( sps_mts_enabled_flag && cu_sbt_flag && log2TbWidth < 6 && log2TbHeight < 6 ) && cIdx == 0 && log2TbWidth > 4 ) log2ZoTbWidth=4 Else log2ZoTbWidth=Min( log2TbWidth, 5) MaxCcbs=2 * ( 1 << log2TbWidth ) * ( 1<< log2TbHeight ) if( ( sps_mts_enabled_flag && cu_sbt_flag && log2TbWidth < 6 && log2TbHeight < 6 ) && cIdx == 0 && log2TbHeight > 4 ) log2ZoTbHeight=4 Else log2ZoTbHeight=Min( log2TbHeight, 5)

[436]

if( log2TbWidth > 0 ) last_sig_coeff_x_prefix ae(v) if( log2TbHeight > 0 ) last_sig_coeff_y_prefix ae(v) if( last_sig_coeff_x_prefix > 3 ) last_sig_coeff_x_suffix ae(v) if( last_sig_coeff_y_prefix > 3 ) last_sig_coeff_y_suffix ae(v) log2TbWidth=log2ZoTbWidth log2TbHeight=log2ZoTbHeight remBinsPass1=( ( 1 << ( log2TbWidth+log2TbHeight ) ) * 7 ) >> 2 log2SbW=( Min( log2TbWidth, log2TbHeight ) < 2 ? 1 : 2 ) log2SbH=log2SbW if( log2TbWidth+log2TbHeight > 3 ) { if( log2TbWidth < 2 ) { log2SbW=log2TbWidth log2SbH=4 - log2SbW } else if( log2TbHeight < 2 ) { log2SbH=log2TbHeight log2SbW=4 - log2SbH } } numSbCoeff=1 << ( log2SbW+log2SbH ) lastScanPos=numSbCoeff lastSubBlock=( 1 << ( log2TbWidth+log2TbHeight - ( log2SbW+log2SbH ) ) ) - 1 do { if( lastScanPos== 0 ) { lastScanPos=numSbCoeff lastSubBlock-- } lastScanPos-- xS=DiagScanOrder[ log2TbWidth - log2SbW ][ log2TbHeight - log2SbH ] [ lastSubBlock ][ 0 ] yS=DiagScanOrder[ log2TbWidth - log2SbW ][ log2TbHeight - log2SbH ] [ lastSubBlock ][ 1 ] xC=( xS << log2SbW ) + DiagScanOrder[ log2SbW ][ log2SbH ][ lastScanPos ][ 0 ] yC=( yS << log2SbH ) + DiagScanOrder[ log2SbW ][ log2SbH ][ lastScanPos ][ 1 ] } while( ( xC != LastSignificantCoeffX ) || ( yC != LastSignificantCoeffY ) ) if( lastSubBlock== 0 &&log2TbWidth >= 2 && log2TbHeight >= 2 && !transform_skip_flag[ x0 ][ y0 ] && lastScanPos > 0 ) LfnstDcOnly= 0 if( ( lastSubBlock > 0 && log2TbWidth >= 2 &&log2TbHeight >= 2 ) || ( lastScanPos > 7 && ( log2TbWidth== 2 || log2TbWidth== 3 ) && log2TbWidth == log2TbHeight ) ) LfnstZeroOutSigCoeffFlag=0

[437] As shown in Table 16, in the process of determining

log2ZoTbWidth and log2ZoTbHeight (where log2ZoTbWidth and log2ZoTbHeight respectively denote the base 2 logarithms of the width and height of the top-left region remaining after zeroing), the check for the value of tu_mts_idx[x0][y0] may be omitted.

[438] The binarization of last_sig_coeff_x_prefix and last_sig_coeff_y_prefix in Table 16 may be determined based on log2ZoTbWidth and log2ZoTbHeight, as shown in Table 11.

[439] Additionally, as shown in Table 16, the sps_mts_enable_flag check condition may be added when determining log2ZoTbWidth and log2ZoTbHeight in the residual encoding.

[440] The TR in Table 11 indicates a truncated Rice binarization method, and the last significant coefficient information may be binarized based on cMax and cRiceParam defined in Table 11 in accordance with the method described in the following table.

[441] According to an example, when information about the position of the last significant luma transform block coefficient is recorded in the residual encoding process, the MTS index may be signaled as shown in Table 17.

[442] [Table 17]

[443] In Table 17, LumaLastSignificantCoeffX and LumaLastSignificantCoeffY indicate the X coordinate and Y coordinate of the position of the last significant coefficient for the luminance transform block, respectively. The condition that LumaLastSignificantCoeffX and LumaLastSignificantCoeffY must be less than 16 has been added to Table 17. When either one is 16 or more, DCT-2 is applied in both the horizontal and vertical directions so that signaling for tu_mts_idx[x0][y0] is omitted and DCT-2 can be understood to apply to both the horizontal and vertical directions.

[444] When both LumaLastSignificantCoeffX and LumaLastSignificantCoeffY are less than 16, this means that the last significant coefficient exists in the top-left region of 16x16. Also, it indicates that when a DST-7 or DCT-8 of length 32 is applied in the current VVC standard, there is a possibility that nulling is applied, which leaves only 16 transform coefficients on the left or top. Accordingly, tu_mts_idx[x0][y0] may be signaled to indicate the transform kernel used for the primary transform.

[445] In another example, the coding unit syntax table and the remainder coding syntax table are as follows.

[446] [Table 18]

[447] [Table 19]

[448] In Table 18, MtsZeroOutSigCoeffFlag is initially set to 1, and this value may change in the residual coding in Table 19. The value of the MtsZeroOutSigCoeffFlag variable changes from 1 to 0 when a significant coefficient exists in the region (LastSignificantCoeffX > 15 || LastSignificantCoeffY > 15) , to be filled with 0 by zeroing, in which case the MTS index is not signaled, as shown in Table 19.

[449] Meanwhile, the conversion unit level syntax table according to the example is shown as follows.

[450] [Table 20]

[451] As shown in Table 20, MtsZeroOutSigCoeffFlag may be set to 1 when tu_cbf_LUMA[x0][y0] is equal to 1, and when tu_cbf_LUMA[x0][y0] is equal to 0, the existing value of MtsZeroOutSigCoeffFlag may be maintained. Accordingly, when tu_cbf_LUMA[x0][y0] is equal to 0 and the value of MtsZeroOutSigCoeffFlag is kept at 0, the mts_idx[x0][y0] encoding may be omitted. That is, if the CBF value of the luma component is 0 because the MTS index is insignificant since no transformation is applied, the encoding of the MTS index may be omitted.

[452] The following drawings have been prepared to explain a specific example of this specification. Since the names of specific devices described in the drawings or the names of specific signals/messages/fields are given as an example, the technical features of this specification are not limited to the specific names used in the following drawings.

[453] FIG. 21 is a flowchart showing the operation of a video decoding apparatus according to an embodiment of the present document.

[454] Each step disclosed in FIG. 21 is based on some of the content described above in FIG. 4-20. Therefore, the explanation for the specific content repeating the content described above in FIG. 3-20, will be omitted or given briefly.

[455] According to an embodiment, the decoding apparatus 300 may receive residual information from the bitstream (S2110).

[456] More specifically, the decoding apparatus 300 may decode quantized transform coefficient information for the current block from the bitstream, and may derive quantized transform coefficients for the target block based on the quantized transform coefficient information for the current block. Information about the quantized transform coefficients for the target block may be included in a sequence parameter set (SPS) or a slice header, and may include at least one of whether or not a shortened transform (RST) is applied, reduced coefficient information, minimum transform size information to which the reduced transform is applied, information about the maximum transform size to which the reduced transform is applied, and transform index information indicating any one of the simplified inverse transform size and the transform kernel matrix included in the transform set.

[457] In addition, the decoding apparatus may further receive information about the intra-prediction mode for the current block and information about whether or not ISP is applied to the current block. The decoder can output whether or not the current block is divided into a predetermined number of subdivision transform blocks by applying and parsing flag information indicating whether or not to apply ISP encoding or ISP mode. Here, the current block may be a coding block. Further, the decoding apparatus may output the size and number of divided subdivision blocks by flag information indicating in which direction the current block will be divided.

[458] The decoding apparatus 300 can output transform coefficients by dequantizing the residual information about the current block, that is, the quantized transform coefficients (S2120).

[459] The derived transform coefficients may be ordered according to the reverse diagonal scan order in units of 4×4 blocks, and the transform coefficients within the 4×4 block may also be ordered according to the reverse diagonal scan order. That is, the transform coefficients on which dequantization has been performed may be ordered according to the reverse scan order used in a video codec, such as VVC or HEVC.

[460] The transform coefficient derived from this residual information may be a dequantized transform coefficient as described above, and may also be a quantized transform coefficient. That is, the transform coefficient may be any data capable of checking whether or not it is non-zero data in the current block, regardless of whether it is quantized or not.

[461] The decoder may derive residual samples by applying an inverse transform to the quantized transform coefficients.

[462] As described above, the decoder can output residual samples by applying LFNST, which is a non-separable transform, or MTS, which is a separable transform, and such transforms can be performed based on the LFNST index indicating the core of the LFNST, that is, the matrix LFNST, and an MTS index indicating the MTS core, respectively.

[463] The decoding device may receive and parse at least one of the LFNST index or MTS index at the coding unit level, and may parse the LFNST index indicating the LFNST core before, that is, immediately before, the index MTS that specifies the MTS core.

[464] The MTS index may be parsed according to a specific condition for the LFNST index, for example, when the LFNST index is 0. Also, according to an example, when the tree type of the current block is a dual luma tree or a single tree, and the value of the LFNST index is 0, parsing can be performed against the MTS index.

[465] When the value of the LFNST index is 0, the residual encoding layer may include a syntax for the position information of the last significant coefficient, and the LFNST index may be obtained by parsing after parsing on the position information of the last significant coefficient.

[466] According to an example, when the current block is a luminance block and the LFNST index is 0, parsing can be performed on the MTS index. That is, when the current block is a luminance block, and if the LFNST index is greater than 0, parsing with respect to the MTS index may not be performed.

[467] According to an example when the tree type of the current block is a dual tree, an LFNST index for each of the luminance block and the chrominance block can be obtained by parsing.

[468] Meanwhile, when deriving the transform coefficient, the width and height for the top-left region in which the last significant coefficient may exist in the current block may be deduced by zeroing, and the width and height for the top-left region may be deduced before parsing regarding the MTS index.

[469] Meanwhile, the position of the last significant coefficient may be output by the width and height for the top-left region, and the position information of the last significant coefficient may be binarized based on the width and height for the top-left region.

[470] Alternatively, according to an example where the tree type of the current block is a single tree type, the decoder may perform residual encoding on the luma block and chrominance block of the current block, and may then parse with respect to the LFNST index.

[471] When parsing is performed on the LFNST index at the coding unit level after performing residual encoding not at the transform block level or residual coding level, the LFNST index representing the positions of the overall transform coefficients for the luma block and the chrominance block, and nulling information can be received, involved in the conversion process, and not the conversion factor information on either of the luminance block or the chrominance block.

[472] When the tree type of the current block is a dual tree type and the chrominance component is encoded, the decoding device may perform residual encoding on the Cb component and the Cr component of the chrominance block, and then may perform parsing on the LFNST index.

[473] When parsing is performed on the LFNST index at the coding unit level after performing residual encoding not at the transform block level or residual coding level, the LFNST index representing the positions of the overall transform coefficients for the Cb component and the Cr component of the chrominance block, and zeroing information is received involved in the transform process, and not the transform coefficient information on any one of the Cb component and the Cr component of the chrominance block.

[474] When the current block is split into a plurality of subdivision blocks, the decoding device may perform residual encoding over the plurality of subdivision blocks, and may then perform parsing with respect to the LFNST index.

[475] Similar to the above description, when parsing is performed on the LFNST index at the coding unit level after performing residual encoding not at the transform block level or residual coding level, an LFNST index representing the positions of the overall transform coefficients for all subdivision blocks can be received, and the zeroing information involved in the transformation process, and not the transformation coefficient information on some or individual sub-division blocks.

[476] According to an example, when the current block is partitioned into a plurality of subdivision blocks, parsing with respect to the LFNST index may be performed regardless of whether or not a transform coefficient exists in a region excluding the DC position of each of the plurality of subdivision blocks. That is, if the last significant coefficient is only allowed to be in the DC position for all subdivision blocks when ISP is applied to the current block, LFNST index signaling may be allowed.

[477] In the residual encoding process, the decoding apparatus may output a first variable indicating whether a transform coefficient exists in a region excluding the DC position of the current block, and may output a second variable indicating whether a transform coefficient exists in a second region excluding the top-left the first region of the current block or subdivision block into which the current block is split.

[478] The decoding device may parse with respect to the LFNST index when a transform coefficient exists in a region excluding the DC position and a transform coefficient does not exist in the second region.

[479] Specifically, in order to determine whether to parse on the LFNST index, the decoding device may output a first variable indicating whether a transform coefficient, i.e., a significant coefficient, exists in a region excluding the DC position of the current block.

[480] The first variable may be an LfnstDcOnly variable, which may be deduced during the encoding of the remainder. The first variable can be output as 0 when the index of the subblock including the last significant coefficient in the current block is 0 and the position of the last significant coefficient in the subblock is greater than 0, and parsing can be performed against the LFNST index when the first variable is 0 A subblock refers to a 4×4 block used as a coding unit in residual coding, and may also be referred to as a coefficient group (CG). A sub-block index of 0 indicates a top-left 4×4 sub-block.

[481] The first variable may be initially set to 1 and may be stored as 1 or may change to 0 depending on whether there is a significant coefficient in the region excluding the DC position.

[482] The variable LfnstDcOnly indicates whether a non-zero coefficient exists at a non-DC component position for at least one transform block within one coding unit, may be equal to 0 when a non-zero coefficient exists at a non-DC component position for at least one transform block within one coding unit, and may be equal to 1 when a non-zero coefficient does not exist at the position of the non-DC component for all transform blocks within one coding unit.

[483] The decoding device may output a second variable indicating whether there is a significant coefficient in the second region excluding the top-left first region of the current block, thereby checking whether zeroing has been performed on the second region.

[484] The second variable may be the LfnstZeroOutSigCoeffFlag variable, which may indicate that zeroing is performed when the LFNST is applied. The second variable may be initially set to 1 and may be changed to 0 when a significant coefficient exists in the second region.

[485] The variable LfnstZeroOutSigCoeffFlag may be output as 0 when the index of the subblock in which the last non-zero coefficient exists is greater than 0 and the width and height of the transform block are equal to or greater than 4, or when the position of the last non-zero coefficient in the subblock is where there is a last non-zero coefficient greater than 7 and the transform block size is 4×4 or 8×8. A subblock refers to a 4x4 block used as a coding unit in residual coding, and may also be referred to as a coefficient group (CG). A sub-block index of 0 indicates a top-left 4×4 sub-block.

[486] That is, when a non-zero coefficient is output in an area excluding the top-left region where the LFNST transform coefficient may exist in the transform block, or the non-zero coefficient exists outside the eighth position in the scan order for the 4×4 block and the 8×8 block, the variable LfnstZeroOutSigCoeffFlag is set to 0.

[487] According to an example when ISP is applied to a coding unit, if it is identified that a transform coefficient exists at a position, excluding the position where the LFNST transform coefficient may exist even for one subdivision block among all subdivision blocks, the LFNST index signaling may be omitted. That is, when zeroing is not performed in one subdivision block and a significant coefficient exists in the second region, the LFNST index is not signaled.

[488] The first region may be inferred based on the size of the current block.

[489] For example, when the size of the current block is 4×4 or 8×8, the first area may be from the top-left of the current block to the position of the eighth sample in the scanning direction. When the current block is split and the subdivision block size is 4×4 or 8×8, the first region may be from the top-left of the subdivision block to the position of the eighth sample in the scan direction.

[490] When the size of the current block is 4x4 or 8x8, eight pieces of data are output by direct LFNST, and therefore, the eight transform coefficients received by the decoder can be ordered from the top-left of the current block to the position of the eighth sample in scanning direction as shown in Fig. 13(a) and FIG. 14(a).

[491] When the size of the current block is not 4×4 or 8×8, the first region may be a top-left 4×4 region of the current block. When the size of the current block is not 4x4 or 8x8, 16 pieces of data are output by forward LFNST, and therefore the 16 transform coefficients received by the decoder can be ordered in the top-left 4x4 region of the current block, as shown in fig. 13(b)-(d) and FIG. 14(b).

[492] The transform coefficients that may be ordered in the first region may be ordered according to the diagonal scanning direction, as shown in FIG. 8.

[493] As described above, when the current block is divided into subdivision blocks, if the transform coefficient does not exist in all separate areas for the plurality of subdivision blocks, the decoding device may perform parsing on the LFNST index. When a transform coefficient exists in the second region for any one subdivision block, no parsing is performed on the LFNST index.

[494] As described above, LFNST may be applied to a subdivision block having a width and height of 4 or more, and an LFNST index for the current block, which is a coding block, may be applied to a plurality of subdivision blocks.

[495] Because the null displayed on the LFNST (including all nulls affected by the application of the LFNST) is applied to the subdivision box as it is, the first region is equally applied to the subdivision box. That is, when the divided subdivision block is a 4x4 block or an 8x8 block, LFNST can be applied from the top-left of the subdivision block to the eighth transformation factor in the scan direction, and when the subdivision block is not a 4x4 block or an 8x8 block , LFNST can be applied to the transform coefficients in the top-left region of the 4x4 subdivision block.

[496] In addition, in the residual encoding process, the decoding apparatus may determine whether a transform coefficient is present in an area other than the top-left region 16×16 of the current block (S2130), and may parse with respect to the MTS index when the coefficient no transformation exists in an area other than the 16x16 area (S2140).

[497] To this end, the decoding apparatus may output a third variable indicating whether a transform coefficient exists in a region other than the top-left region of 16×16.

[498] The third variable may be the MtsZeroOutSigCoeffFlag variable, which may indicate that zeroing is performed when the MTS is applied. The variable MtsZeroOutSigCoeffFlag indicates whether there is a transform coefficient in the top-left region where the last significant coefficient may exist by zeroing after the execution of the MTS, that is, the region excluding the 16x16 top-left region, and the value of the third variable may initially be set to 1 and may change from 1 to 0 when the transform coefficient exists in an area other than the top-left area of 16×16. When the value of the third variable is 0, the MTS index is not signaled.

[499] The decoder may output residual samples by applying at least one of LFNST performed based on the LFNST index or MTS performed based on the MTS index, and according to an example, the residual samples may be output by applying the MTS core outputted to based on the MTS index, to 16×16 top-left transform coefficients (S2150).

[500] Next, the decoding apparatus 300 may generate reconstructed samples based on the residual samples for the current block and the prediction samples for the current block (S2160).

[501] The following drawings have been prepared to explain a specific example of the present description. Since the names of specific devices described in the drawings or the names of specific signals/messages/fields are given as an example, the technical features of the present description are not limited to the specific names used in the following drawings.

[502] FIG. 22 is a flowchart showing the operation of a video encoding apparatus according to an embodiment of the present document.

[503] Each step disclosed in FIG. 22 is based on some of the content described above with reference to FIG. 4-20. Therefore, the explanation for the specific content repeating the content described above with reference to FIG. 2 and 4-20 will be omitted or given briefly.

[504] The encoder 200 according to the embodiment may output prediction samples for the current block based on the intra prediction mode applied to the current block (S2210).

[505] The encoder may perform prediction for each subblock of the transform partition in the case where ISP is applied to the current block.

[506] The encoding device may determine whether or not to apply the ISP encoding or the ISP mode to the current block, that is, the encoding block, and according to the result of the determination, it is possible to determine in which direction the current block is to be divided and output the size and number subblocks to be split.

[507] The encoder 200 may output residual samples for the current block based on the prediction samples (S2220).

[508] The encoder 200 may derive transform coefficients for the current block by applying at least one of the LFNST or MTS to the residual samples, and may order the transform coefficients according to a predetermined scan order, according to an example, the transform coefficients for the current block may be output based on MTS for residual samples (S2230).

[509] Primary transform may be performed by a plurality of transform kernels such as MTS, in which case, the transform kernel may be selected based on the intra-prediction mode.

[510] After deriving the transform coefficients by applying the MTS, the encoder may zero out the remaining areas of the current block except for a specific top-left area of the current block, for example, a 16x16 area (S2240).

[511] According to an example, the encoder may perform nulling when the MTS is applied in the primary transform of the current block. The encoder may perform zero-padding of an area except for the top-left 16x16 region of the current block or sub-block of the split with zeros, and may encode the MTS index according to a third variable indicating whether a transform coefficient exists in the zero-out area.

[512] The third variable may be the variable MtsZeroOutSigCoeffFlag, which may indicate that zeroing is performed when the MTS is applied. The MtsZeroOutSigCoeffFlag variable indicates whether a transform coefficient exists in the top-left region where the last significant coefficient may exist by zeroing after the execution of the MTS, that is, the region excluding the 16x16 top-left region, and the value of the third variable may initially be set to 1 and may change from 1 to 0 when the transform coefficient exists in an area other than the top-left area of 16×16. When the value of the third variable is 0, the MTS index is not encoded or signaled.

[513] According to an example, the encoder may output the position of the last significant coefficient based on the width and height of the top-left region, and may binarize the position information of the last significant coefficient.

[514] According to an example, the width and height of the top-left region may be output before signaling the MTS index.

[515] Also, the encoder 200 may determine whether or not to perform a secondary transform, or inseparable transform, specifically LFNST, on the transform coefficients for the current block, and may apply the LFNST to the transform coefficients to derive modified transform coefficients.

[516] LFNST is a non-separable transform that applies a transform without splitting the coefficients in a particular direction, as opposed to a primary transform that splits and transforms in the vertical or horizontal direction the coefficients that are the targets of the transform. Such a non-separable transform may be a low-frequency non-separable transform that applies the transform only to the low frequency region and not to the entire target block that is the target of the transform.

[517] When the ISP is applied to the current block, the encoder may determine whether or not the LFNST can be applied to the height and width of the split sub-block of the partition.

[518] The encoder may determine whether or not the LFNST can be applied to the height and width of the split subblock of the partition. In this case, the decoder may parse the LFNST index when the height and width of the split subblock are equal to or greater than 4.

[519] The encoding device can configure and output image information such that at least one of the LFNST index and the MTS index is signaled at the coding unit level, and the MTS index is signaled immediately after the signaling of the LFNST index, and the remainder output by quantizing the transform coefficient information, may be encoded (S2250).

[520] The MTS index may be encoded according to a specific condition for the LFNST index, for example, when the LFNST index is 0. Also, according to an example, when the tree type of the current block is a dual luminance tree or a single tree and the value of the LFNST index is 0, the MTS index may be encoded.

[521] According to an example, when the current block is a luma block and the LFNST index indicates 0, the encoder may encode the MTS index.

[522] According to an example, when the tree type of the current block is a dual tree, the encoder may encode an LFNST index for each of the luma block and the chrominance block.

[523] Alternatively, according to an example where the tree type of the current block is a single tree type, the encoder may derive all transform coefficients for the luminance block and the chrominance block of the current block, and may then encode the LFNST index at the coding unit level.

[524] When the LFNST index is encoded at the coding unit level after all transform coefficients are derived, and not at the transform block level or residual coding level, the LFNST index showing the positions of the overall transform coefficients for the luma block and the chrominance block, and the nulling information affected in the process transforms, rather than transform coefficient information per luminance block or chrominance block, may be encoded.

[525] When the tree type of the current block is a dual tree type and a chrominance component is encoded, the encoder may output all transform coefficients of the Cb component and the Cr component of the chrominance block, and may then encode the LFNST index at the coding unit level.

[526] When the LFNST index is encoded at the coding unit level after deriving all transform coefficients not at the transform block level or at the residual coding level, the LFNST index representing the positions of the overall transform coefficients for the Cb component and the Cr component of the chrominance block, and the nulling information affected in transformation process, rather than the transformation coefficient information on any one of the Cb component and the Cr component of the chrominance block, may be encoded.

[527] When the current block is split into a plurality of subdivision blocks, the encoder may output all transform coefficients for the plurality of subdivision blocks, and may then encode the LFNST index at the coding unit level.

[528] Similar to the above description, when the LFNST index is parsed at the coding unit level after deriving all transform coefficients not at the transform block level or at the residual coding level, the LFNST index showing the positions of the full transform coefficients for all subdivision blocks, and the nulling information affected in the process of transformation, and not the information of the transformation coefficients on some or individual subdivision blocks, may be encoded.

[529] When the current block is split into a plurality of subdivision blocks, the encoder may encode the LFNST index regardless of whether a transform coefficient exists in the area, excluding the DC position of each of the plurality of subdivision blocks. That is, if the last significant coefficient is only allowed to be in the DC position for all subdivision blocks when ISP is applied to the current block, LFNST index signaling can be enabled.

[530] In the transform coefficient derivation process, the encoder may output a first variable indicating whether a transform coefficient exists in a region excluding the DC position of the current block, and a second variable indicating whether a transform coefficient exists in a second region excluding the top-left first the area of the current block or subdivision block into which the current block is split.

[531] The encoder may encode the LFNST index when a transform coefficient exists in a region excluding the DC position, and the transform coefficient does not exist in the second region.

[532] Specifically, the first variable may be an LfnstDcOnly variable and may be output as 0 when the subblock index including the last significant coefficient in the current block is 0 and the position of the last significant coefficient in the subblock is greater than 0, and the LFNST index may be encoded when the first variable is 0.

[533] The first variable may be initially set to 1 and may be stored as 1 or may change to 0 depending on whether there is a significant coefficient in the region excluding the DC position.

[534] The variable LfnstDcOnly indicates whether a non-zero coefficient exists at a non-DC component position for at least one transform block within one coding unit, may be equal to 0 when a non-zero coefficient exists at a non-DC component position for at least one transform block within one coding unit, and may be equal to 1 when a non-zero coefficient does not exist at the position of the non-DC component for all transform blocks within one coding unit.

[535] The encoder may zero out the second region of the current block where no modified transform coefficients exist after performing the LFNST, and may output a second variable indicating whether a transform coefficient exists in the second region.

[536] As shown in FIG. 13 and FIG. 14, the remaining area of the current block where no modified transform coefficients exist can be set to zero. Because of this zeroing, it is possible to reduce the amount of computation required to perform the entire conversion process and reduce the amount of operations required for the entire conversion process, thus reducing the power consumption required to perform the conversion. In addition, the waiting caused by the conversion process can be shortened, thereby improving the image encoding efficiency.

[537] The second variable may be the variable LfnstZeroOutSigCoeffFlag, which may indicate that zeroing is performed when the LFNST is applied. The second variable may be initially set to 1 and may change to 0 when a significant coefficient exists in the second region.

[538] The variable LfnstZeroOutSigCoeffFlag may be output as 0 when the index of the subblock in which the last non-zero coefficient exists is greater than 0 and both the width and height of the transform block are equal to or greater than 4, or when the position of the last non-zero coefficient in the subblock in which the last non-zero coefficient exists is greater than 7, and the transform block size is 4x4 or 8x8.

[539] That is, when a non-zero coefficient is output in an area excluding the top-left region where the LFNST transform coefficient may exist in the transform block, or the non-zero coefficient exists outside the eighth position in the scan order for the 4×4 block and the 8×8 block, the variable LfnstZeroOutSigCoeffFlag is set to 0.

[540] The description of the first area and the description of nulling in the case where the ISP is applied are essentially the same as the descriptions given in the decoding method, and thus the redundant description will be omitted.

[541] Additionally, the encoding device can output quantized transform coefficients by performing quantization based on transform coefficients or modified transform coefficients for the current block, and can encode and output image information including information about the quantized transform coefficients.

[542] An encoder may generate residual information including information about quantized transform coefficients. The residual information may include the transformation-related information/syntax element described above. The encoding device may encode the image/video information including the residual information, and output the encoded image/video information in the form of a bitstream.

[543] More specifically, the encoding device may generate information about the quantized transform coefficients and encode information about the generated quantized transform coefficients.

[544] In the present disclosure, at least one of quantization/dequantization and/or transformation/inverse transformation may be omitted. When quantization/dequantization is omitted, the quantized transform coefficient may be referred to as a transform coefficient. When the transformation/inverse transformation is omitted, the transformation coefficient may be referred to as a coefficient or residual coefficient, or may still be referred to as a transformation coefficient for consistency purposes.

[545] In addition, in the present disclosure, a quantized transform coefficient and a transform coefficient may be referred to as a transform coefficient and a scaled transform coefficient, respectively. In this case, the residual information may include information on the transform coefficient(s), and the information on the transform coefficient(s) may be signaled by a residual encoding syntax. The transform coefficients may be derived based on the residual information (or information about the transform coefficient(s)), and the scaled transform coefficients may be derived by inverting (scaling) the transform coefficients. The residual samples may be derived based on the inverse transform (transform) of the scaled transform coefficients. These details may also be applied/expressed in other parts of this disclosure.

[546] In the above embodiments, the methods are explained based on flowcharts, using a sequence of steps or blocks, but the present disclosure is not limited to this order of steps or blocks, and a certain step may be performed in an order other than that described above, or at the same time as another step. Additionally, one skilled in the art will appreciate that the steps shown in the flowchart above are not exclusive, that another step may be included, or that one or more steps in the flowchart may be deleted without affecting to the scope of this disclosure.

[547] The above-described methods in accordance with the present disclosure may be implemented in the form of software, and the encoding device and/or decoding device in accordance with the present disclosure may be included in an image processing device such as a television, computer, smartphone, set-top box. , display device, etc.

[548] When embodiments of the present disclosure are implemented in software, the above described methods may be implemented as modules (processes, functions, etc.) to perform the above described functions. Modules may be stored in memory and may be executed by a processor. The memory may be internal or external to the processor and may be coupled to the processor by various well known means. The processor may include an application specific integrated circuit (ASIC), other chipsets, logic circuitry, and/or a data processing device. The memory may include Read Only Memory (ROM), Random Access Memory (RAM), flash memory, a memory card, a storage medium, and/or other storage device. That is, the embodiments described in this disclosure may be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional blocks shown in each figure may be implemented and executed on a computer, processor, microprocessor, controller, or chip.

[549] In addition, the decoding device and the encoding device to which the present disclosure applies can be included in a multimedia broadcast transceiver, a mobile communication terminal, a home theater device, a digital cinema device, a surveillance camera, a video talking device, a communication device in such as video communication, mobile streaming device, storage media, camera recorder, video-on-demand (VoD) service device, direct-to-content (OTT) video delivery device, Internet streaming service device, 3D device ) video, video teleconferencing device and medical video device, and can be used to process video signal and data signal. For example, an OTT video device may include a game console, a blue-ray player, an Internet access TV, a home theater system, a smartphone, a tablet PC, a digital video recorder (DVR), and the like.

[550] In addition, the processing method to which the present disclosure applies may be in the form of a computer-executable program and may be stored in a computer-readable recording medium. Multimedia data having a data structure in accordance with the present disclosure may also be stored in computer-readable recording media. Computer-readable recording media includes all types of storage devices and distributed storage devices that store computer-readable data. Computer-readable recording media may include, for example, a Blu-ray Disc (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical storage device data. Further, the computer-readable recording medium includes media implemented in the form of a carrier wave (eg, transmission over the Internet). In addition, the bitstream generated by the encoding method may be stored in a computer-readable recording medium, or may be transmitted over wired or wireless communication networks. Additionally, embodiments of the present disclosure may be implemented as a computer program product by means of program codes, and the program codes may be executed on a computer by means of embodiments of the present disclosure. The program codes may be stored on a computer-readable medium.

[551] FIG. 23 shows an example of a content streaming system in which the present disclosure applies.

[552] A content streaming system to which the present disclosure applies may broadly include an encoding server, a streaming server, a web server, a media storage, a user equipment, and a media input device.

[553] The encoding server functions to compress digital data of content input from multimedia input devices such as a smartphone, a camera, a camera recorder (camcorder), etc. into digital data to generate a bitstream and transmit it to a streaming server . As another example, when a media input device such as a smartphone, camera, recorder camera, etc. directly generates a bitstream, the encoding server may be omitted. The bitstream may be generated by an encoding method or a bitstream generation method in which the present disclosure applies, and a streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[554] The streaming server transmits media data to the user equipment based on the user's request through a web server that functions as a tool that informs the user about the service. When the user requests the desired service, the web server forwards the request to the streaming server, and the streaming server transmits the media data to the user. In this regard, the content streaming system may include a separate control server, in which case, the control server functions to manage commands/responses between respective equipments in the content streaming system.

[555] The streaming server may receive content from a media store and/or an encoding server. For example, when content is received from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may store the bitstream for a predetermined period of time.

[556] For example, user equipment may include a mobile phone, smartphone, laptop, digital broadcast terminal, personal digital assistant (PDA), portable multimedia player (PMP), navigator, thin PC, tablet PC, ultrabook, wearable device (e.g. , watch-type terminal (smart watch), glasses-type terminal (smart glasses), head-mounted display (HMD)), digital TV, desktop computer, digital pointer, and the like. Each of the servers in the content streaming system may operate as a distributed server, in which case, the data received by each server may be processed in a distributed manner.

[557] The claims disclosed herein may be combined in various ways. For example, the technical features of the method claims in the present disclosure may be combined to be implemented or performed in a device, and the technical features of the device claims may be combined to be implemented or performed in a method. In addition, the technical features of the method claims and the device claims can be combined to be implemented or performed in the device, and the technical features of the method claims and the device claims can be combined to be implemented or performed in the method.

Claims (25)

1. An image decoding method performed by a decoding device, the method comprising: obtaining residual information from the bitstream; deriving transform coefficients for the current block based on the residual information; determining whether a significant coefficient is present in a second region other than the first top-left region of the current block; deriving, by parsing, a multiple transform selection (MTS) index from the bitstream based on the fact that a significant coefficient is not present in the second region; deriving residual samples by applying the MTS kernel derived from the MTS index to the transform coefficients of the first region; And generating a reconstructed image based on residual discrete samples. 2. The image decoding method according to claim 1, wherein the first region is a top-left 16×16 region of the current block. 3. The image decoding method according to claim 1, wherein determining whether a significant coefficient is present in the second region comprises deriving a variable value related to whether a significant coefficient is present in the second region during the residual coding level decoding process, wherein the value variable is initially set to 1, and on the basis that a significant coefficient is present in the second region, the value of the variable is changed to 0, and no parsing is performed on the MTS index. 4. The image decoding method of claim 3, wherein parsing with respect to the MTS index is performed at the coding unit level. 5. The image decoding method of claim 1, further comprising parsing with respect to a low frequency inseparable transform (LFNST) index associated with an LFNST core applied to the current block, wherein the LFNST index and the MTS index are signaled at the coding unit level, and the MTS index is signaled immediately after signaling the LFNST index. 6. The image decoding method according to claim 5, wherein the tree type of the current block is a dual luma tree or a single tree, and based on the value of the LFNST index being 0, the MTS index is signaled. 7. An image encoding method performed by an image encoding device, the method comprising: outputting discrete prediction samples for the current block; deriving residual samples for the current block based on the prediction samples; deriving transform coefficients for the current block based on multiple transform selection (MTS) for residual discrete samples; resetting the second area of the current block, other than the first area from the top-left of the current block; And encoding the residual information derived by quantizing the transform coefficients and the MTS index associated with the MTS core. 8. The image encoding method according to claim 7, wherein the first region is a top-left 16×16 region of the current block. 9. The image encoding method according to claim 7, wherein the MTS index is signaled at the coding unit level. 10. The image encoding method of claim 9, further comprising signaling a low frequency inseparable transform (LFNST) index associated with the LFNST core applied to the current block, wherein the MTS index is signaled immediately after the LFNST index signaling. 11. The image encoding method of claim 10, wherein the tree type of the current block is a dual luma tree or a single tree, and based on the value of the LFNST index being 0, the MTS index is signaled. 12. A method for transmitting data for image information by means of a transmitter, comprising: receiving, by a transmitter, a bitstream of image information including residual information from an image encoding apparatus, wherein the residual information is generated by deriving prediction samples for the current block, deriving residual samples for the current block based on the prediction samples, deriving transform coefficients for the current block based on the multiple transform selection (MTS) for residual discrete samples, zeroing out the second region of the current block other than the first top-left region of the current block, and encoding the residual information derived by quantization of the transform coefficients, and the MTS index associated with the core MTS to generate a bitstream; And transmitting, by means of the transmitter, data containing the image information bitstream to the decoding apparatus.

Images