RU2811986C2 - Image encoding method based on transformation and device for its implementation - Google Patents

Abstract

FIELD: image encoding.

SUBSTANCE: modified transform factors are derived from the transform factors by applying a low frequency non-separable transform (LFNST). The transform factors or modified transform factors are quantized, and an LFNST index is encoded indicating that LFNST is applied to the current block. Quantization is performed based on a predefined scaling matrix or a predefined constant value. Whether to apply a scaling matrix is determined based on the index LFNST and the tree type of the current block, which is one of three predefined tree types including a single tree, a dual luma tree, a dual chroma tree. When the tree type of the current block is a single tree and the current block is a chroma component, it is determined that the scaling matrix applies to the current block. When the tree type of the current block is a dual chroma tree, it is determined that the predetermined constant value is applied to the current block.

EFFECT: improved efficiency of image encoding.

13 cl, 16 dwg, 13 tbl

Description

TECHNICAL AREA

[1] The present disclosure relates to an image encoding method and, in particular, to a method and apparatus for encoding an image based on a transform in an image encoding system.

Related technical field

[2] Nowadays, the need for high-definition and high-quality images/videos, such as ultra-high-definition (UHD) 4K or 8K or higher images/videos, has increased in various fields. Since image/video data is high resolution and high quality, the amount of information or number of bits transmitted is increased compared to normal image data. Therefore, when image data is transmitted using media such as a conventional wired/wireless broadband line, or image/video data is stored using an existing storage medium, transmission costs and storage costs increase.

[3] In addition, currently there is an increasing interest and need for immersive media, such as content or hologram of virtual reality (VR) and artificial reality (AR) or the like, and the broadcast of images/videos having image characteristics other than features of real images, such as a game image.

[4] Accordingly, there is a need for a highly efficient image/video compression method for efficiently compressing, transmitting or storing and reproducing high resolution and high quality image/video information having various features as described above.

Brief description of the invention

[5] A technical aspect of the present disclosure is to provide a method and apparatus for improving image encoding efficiency.

[6] Another technical aspect of the present disclosure is to provide a method and apparatus for increasing quantization efficiency.

[7] Another technical aspect of the present disclosure is to provide a method and apparatus for improving the quantization efficiency of a chrominance component in a single type tree.

[8] According to an embodiment of the present disclosure, an image decoding method performed by a decoding apparatus is provided. The method may include: receiving residual information via a bitstream; deriving transformation coefficients for the current block by performing dequantization based on the residual information; and deriving modified transform coefficients by applying LFNST to the transform coefficients, wherein dequantization may be performed based on a predetermined scaling list, and whether the scaling list should be applied may be inferred based on whether LFNST is applied and the tree type of the current block.

[9] When the tree type of the current block is single tree and the current block is a luma component, the scaling list may not be applied, and when the tree type of the current block is single tree and the current block is a chroma component, the scaling list may be applied.

[10] Additionally, flag information indicating whether a scaling list is available when LFNST is executed may be received.

[11] When the flag information indicates that the scaling list is not available and the LFNST index is greater than 0, the scaling list cannot be applied to the luma component.

[12] When the flag information indicates that the scaling list is not available and the LFNST index is greater than 0, if the tree type of the current block is a dual chroma tree, the scaling list may not be applied to the chroma component.

[13] When the flag information indicates that the scaling list is not available and the LFNST index is greater than 0, if the tree type of the current block is a dual luma tree, the scaling list may not be applied to the luma component.

[14] The current block may include a transform block.

[15] According to another embodiment of the present disclosure, an image encoding method performed by an encoding apparatus is provided. The method may include: outputting prediction samples for the current block; deriving residual samples for the current block based on the prediction samples; deriving transformation coefficients for the current block based on the primary transformation of the residual samples; deriving modified conversion factors from conversion factors by applying LFNST; and quantizing the transform coefficients or modified transform coefficients, wherein the quantization may be performed based on a predetermined scaling list, and whether the scaling list should be applied may be inferred based on whether LFNST is applied and the tree type of the current block.

[16] According to yet another embodiment of the present disclosure, a digital storage medium may be provided that stores image data including encoded image information and a bit stream generated in accordance with an image encoding method performed by an encoding apparatus.

[17] According to yet another embodiment of the present disclosure, a digital storage medium may be provided that stores image data including encoded image information and a bit stream to cause a decoding device to perform an image decoding method.

[18] According to the present disclosure, it becomes possible to improve the overall efficiency of image/video compression.

[19] According to the present disclosure, it becomes possible to improve the quantization efficiency.

[20] According to the present disclosure, it becomes possible to improve the quantization efficiency of the chroma component in a single type tree.

[21] The results that can be obtained from the specific examples of this disclosure are not limited to the results listed above. For example, various technical results may be achieved that one skilled in the art will be able to understand or obtain based on the present disclosure. Accordingly, the specific results of the present disclosure are not limited to those expressly described in the present disclosure, and may include various results that can be understood or obtained by the technical features of the present disclosure.

Brief description of drawings

[22] FIG. 1 schematically illustrates an example of a video/image coding system to which the present disclosure is applicable.

[23] FIG. 2 is a diagram schematically illustrating the configuration of a video/image encoding apparatus to which the present disclosure is applicable.

[24] FIG. 3 is a diagram schematically illustrating the configuration of a video/image decoding apparatus to which the present disclosure is applicable.

[25] FIG. 4 illustrates the structure of a content streaming system to which the present disclosure is applied.

[26] FIG. 5 schematically illustrates a multiple conversion technology in accordance with an embodiment of the present disclosure.

[27] FIG. 6 schematically shows intra-directional modes of 65 prediction directions.

[28] FIG. 7 is a diagram for explaining RST in accordance with an embodiment of the present disclosure.

[29] FIG. 8 is a diagram illustrating a layout sequence of direct primary transform output data into a one-dimensional vector according to an example.

[30] FIG. 9 is a diagram illustrating a layout sequence of direct secondary transform output data into a two-dimensional block according to an example.

[31] FIG. 10 is a diagram illustrating wide-angle intra-prediction modes in accordance with an embodiment of the present document.

[32] FIG. 11 is a diagram illustrating a block shape to which LFNST is applied.

[33] FIG. 12 is a diagram illustrating the layout of forward LFNST output according to an example.

[34] FIG. 13 illustrates nulling in a block to which a 4x4 LFNST is applied according to the example.

[35] FIG. 14 illustrates nulling in a block to which 8x8 LFNST is applied according to the example.

[36] FIG. 15 is a flowchart illustrating operation of a video decoding apparatus according to an embodiment of the present disclosure.

[37] FIG. 16 is a flowchart illustrating the operation of a video encoding apparatus in accordance with an embodiment of the present disclosure.

Description of Exemplary Embodiments

[38] Although the present document is subject to various modifications and includes various embodiments, specific embodiments thereof, shown in the drawings by way of example, will now be described in detail. However, this is not intended to limit the present disclosure to the specific embodiments described herein. The terminology used herein is intended to describe specific embodiments only and is not intended to limit the technical intent of the present disclosure. Singular forms may include plural forms unless the context clearly indicates otherwise. Terms such as "include" and "have" are intended to indicate that the features, numbers, steps, operations, elements, components, or combinations thereof used in the following description exist, and thus are not to be construed as such. that the possibility of the existence or addition of one or more other features, numbers, steps, operations, elements, components or a combination thereof is excluded in advance.

[39] Meanwhile, each component in the drawings described herein is illustrated independently for convenience of description regarding various characteristic functions, however, this does not mean that each component is implemented as separate hardware or software. For example, two or more of these components may be combined to form a single component, and any single component may be divided into multiple components. Embodiments in which components are combined and/or separated will be included within the scope of the patent rights of this disclosure if they do not depart from the spirit of this disclosure.

[40] Next, preferred embodiments of the present disclosure will be explained in more detail with reference to the accompanying drawings. In addition, like reference numerals are used for like components in the drawings, and repeated descriptions of like components will be omitted.

[41] This document relates to video/image coding. For example, the method/example disclosed herein may relate to the VVC (Video Video Coding) standard (ITU-T Rec. H.266), the next generation video/image coding standard after VVC, or other standards related to video coding (eg HEVC (High Efficiency Video Coding) standard (ITU-T Rec. H.265), EVC (Essential Video Coding) standard, AVS2 standard, etc.).

[42] Herein, various embodiments related to video/image encoding may be provided, and unless otherwise specified, embodiments may also be combined with each other and executed.

[43] As used herein, video may refer to a series of images over time. A picture usually means a unit representing an image in a particular time zone, and a slice/tile is a unit that makes up a part of the picture. The clipping/tile may include one or more coding tree units (CTUs). One picture may consist of one or more cutouts/mosaic elements. One picture can consist of one or more groups of mosaic elements. One tile group may include one or more tiles.

[44] Pixel or pel can mean the smallest unit that makes up one picture (frame) (or image). Also, 'sample' can be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a luma component pixel/pixel value or only a chrominance component pixel/pixel value. Alternatively, the sample may refer to a pixel value in the spatial domain, or when that pixel value is converted to the frequency domain, it may refer to a frequency domain conversion factor.

[45] The unit may represent a basic image processing unit. The unit may include at least one of a specific area and information related to the area. One unit may include one luma block and two chrominance blocks (eg, cb, cr). A unit and a term such as a block, area or the like may be used interchangeably as appropriate. In general, an M×N block may include a set (or array) of samples (or arrays of samples) or transform coefficients consisting of M columns and N rows.

[46] In this document, the characters "/" or "," are to be interpreted as indicating "and/or". For example, the expression "A/B" could mean "A and/or B". Additionally, "A, B" may mean "A and/or B". Additionally, "A/B/C" may mean "at least one of A, B, and/or C." Also, "A/B/C" may mean "at least one of A, B and/or C".

[47] Additionally, as used herein, the term “or” should be interpreted to indicate “and/or.” For example, the expression "A or B" may include 1) A only, 2) B only, and/or 3) both A and B. In other words, the term "or" as used herein is to be interpreted as indicating "in addition or alternatively" .

[48] In the present disclosure, “at least one of A and B” may mean “A only,” “B only,” or “both A and B.” Also, in the present disclosure, the expression “at least one of A or B” or “at least one of A and/or B” can be interpreted as “at least one of A and B”.

[49] Additionally, in the present disclosure, at least one of "A, B, and C" may mean "A only," "B only," "C only," or "any combination of A, B, and C." Also, "at least one of A, B or C" or "at least one of A, B and/or C" may mean "at least one of A, B and C".

[50] In addition, parentheses used in this disclosure may mean "for example." Specifically, when "prediction (intra-prediction)" is specified, it may mean that "intra-prediction" is proposed as an example of "prediction". In other words, “prediction” in the present disclosure is not limited to “intra-prediction”, and “intra-prediction” may be offered as an example of “prediction”. Also, when "prediction (i.e., intra-prediction)" is specified, it also means that "intra-prediction" is proposed as an example of "prediction".

[51] Technical features separately described in a single drawing in the present disclosure may be implemented separately or may be implemented simultaneously.

[52] FIG. 1 schematically illustrates an example of a video/image coding system to which the present disclosure is applicable.

[53] With reference to FIG. 1, a video/image encoding system may include a first device (a source device) and a second device (a receiving device). The source device may deliver encoded information or video/image data in the form of a file or streaming (streaming) to the receiving device via a digital storage medium or network.

[54] The source device may include a video source, an encoder, and a transmitter. The receiving device may include a receiver, a decoding device, and a rendering device. The encoding device may be called a video/image encoding device, and the decoding device may be called a video/image decoding device. The transmitter may be included in the encoding device. The receiver may be included in a decoding device. The imaging device may include a display, and the display may be configured as a separate device or an external component.

[55] The video source may obtain video/image through a video/image capture, synthesis, or generation process. The video source may include a video/image capturing device and/or a video/image generation device. The video/image capturing device may include, for example, one or more cameras, video/image archives including previously captured video/images, and the like. The video/image generating device may include, for example, a computer, tablet and smartphone and may (electronically) generate video/images. For example, the virtual video/image may be generated through a computer or the like. In this case, the video/image capture process can be replaced by the associated data generation process.

[56] The encoding device can encode the input video/image. The encoder may perform a series of procedures such as prediction, transform and quantization for compression and encoding efficiency. The encoded data (encoded video/image information) may be output in the form of a bit stream.

[57] A transmitter may transmit encoded information or video/image data output in the form of a bit stream to a receiver of a receiving device via a digital storage medium or network in the form of a file or stream. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD and the like. The transmitter may include an element for generating a media file through a predefined file format and may include an element for transmission over a broadcast/communications network. The receiver may receive/extract the bit stream and transmit the received/extracted bit stream to a decoding device.

[58] A decoding apparatus may decode a video/image by performing a series of procedures such as dequantization, deconversion, and prediction corresponding to an operation of the encoding apparatus.

[59] The renderer can render the decoded video/image. The rendered video/image can be displayed via the display.

[60] FIG. 2 is a diagram schematically illustrating the configuration of a video/image encoding apparatus to which embodiments of the present disclosure may be applied. Further, what is referred to as a video encoding device may include an image encoding device.

[61] With reference to FIG. 2, the encoding device 200 includes an image partitioning unit 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-predictor 221 and an intra-predictor 222. The processor The remainder 230 may include a converter 232, a quantizer 233, a dequantizer 234, and an inverse converter 235. The remainder processor 230 may further include a subtractor 231. The adder 250 may be referred to as a reconstructor or reconstructed block generator. The image partitioning unit 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260, which were described above, may be constituted by one or more hardware components (eg, chipsets or encoder processors) in accordance with an embodiment. In addition, memory 270 may include a decoded picture buffer (DPB) or may be formed by a digital storage medium. The hardware component may further include memory 270 as an internal/external component.

[62] The image splitter 210 may split an input image (or picture or frame) input to the encoder 200 into one or more processing units. For example, a processing unit may be called a coding unit (CU). In this case, starting from a coding tree unit (CTU) or largest coding unit (LCU), the coding unit can be recursively split according to a quadtree/binary tree/ternary tree (QTBTTT) structure. For example, one encoding unit may be partitioned into multiple encoding units of greater depth based on a quadtree structure, a binary tree structure, and/or a ternary tree structure. In this case, for example, the quadtree structure may be applied first, and the binary tree structure and/or ternary tree structure may be applied later. Alternatively, the binary tree structure may be applied first. The encoding procedure in accordance with the present disclosure may be performed based on a final encoding unit that is no longer split. In this case, the largest encoding unit can be used directly as the final encoding unit based on the encoding efficiency according to the characteristic of the image. Alternatively, a coding unit can be recursively split into coding units of greater depth as needed, so that an optimally sized coding unit can be used as the final coding unit. Here, the encoding procedure may include procedures such as prediction, transformation and reconstruction, which will be described later. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit may be separated or split from the above-described final encoding unit. The prediction unit may be a sample prediction unit, and the transformation unit may be a unit for deriving a transform coefficient and/or a unit for deriving a residual signal from the transform coefficient.

[63] Unit may be used interchangeably with terms such as block, area or the like, as appropriate. In general, an M×N block can represent a set of samples or transform coefficients consisting of M columns and N rows. The sample may generally represent a pixel or pixel value, may represent only a luma component pixel/pixel value, or may represent only a chrominance component pixel/pixel value. A sample can be used as a term corresponding to a pixel or pel of a single picture (or image).

[64] The subtractor 231 subtracts the prediction signal (predicted block, prediction sample array) output from the predictor 220 from the input image signal (original block, original sample array) to generate a residual signal (residual block, residual sample array), and the generated the residual signal is transmitted to converter 232. Predictor 220 may perform prediction on a target processing block (hereinafter referred to as a current block) and may generate a predicted block including prediction samples for the current block. Predictor 220 may determine whether intra-prediction or inter-prediction is applied on the current block or on a CU basis. As discussed below in the description of each prediction mode, the predictor may generate various prediction-related information, such as prediction mode information, and transmit the generated information to the entropy encoder 240. The prediction information may be encoded in the entropy encoder 240 and output in the form of a bit stream.

[65] Intra predictor 222 may predict the current block by referring to samples in the current picture. These samples may be located adjacent to or spaced apart from the current block in accordance with the prediction mode. In intra-prediction, prediction modes may include a plurality of undirected modes and a plurality of directed modes. Non-directional modes may include, for example, DC mode and planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the granularity of the prediction direction. However, this is just an example, and more or less directional prediction modes may be used depending on the setting. Intra predictor 222 may determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[66] The inter-predictor 221 may output a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector in the reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter-prediction mode, motion information may be predicted on a block, sub-block, or sample basis based on the correlation of motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter-prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block existing in the current picture and a temporal neighbor block existing in the reference picture. The reference picture including the reference block and the reference picture including the temporary adjacent block may be the same or different. A temporary neighbor block may be called a co-located reference block, a co-located CU (colCU), etc., and a reference picture including a temporary neighboring block may be called a co-located picture (colPic). For example, inter-predictor 221 may configure a list of potential motion information candidates based on neighboring blocks and generate information indicating which candidate is used to derive the motion vector and/or reference picture index of the current block. Inter-prediction can be performed based on various prediction modes. For example, in the case of the skip mode and the merge mode, the inter-predictor 221 may use the motion information of a neighboring block as the motion information of the current block. In skip mode, unlike combine mode, the residual signal may not be transmitted. In the case of a motion vector prediction (MVP) mode, the motion vector of an adjacent block can be used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling a motion vector difference.

[67] Predictor 220 may generate a prediction signal based on various prediction methods described below. For example, a predictor may apply intra-prediction or inter-prediction to make a prediction on one block, and may also apply intra-prediction and inter-prediction simultaneously. This may be called combined inter- and intra-prediction (CIIP). In addition, the predictor can be based on intra-block copy (IBC) prediction mode or palette mode to perform prediction on a block. The IBC prediction mode or palette mode can be used for encoding image/video game content or the like, such as screen content encoding (SCC). Although IBC primarily performs prediction in the current block, it can perform similar to inter-prediction in that it outputs a reference block in the current block. That is, the IBC may use at least one of the inter-prediction methods described in the present disclosure.

[68] The prediction signal generated by inter-predictor 221 and/or intra-predictor 222 can be used to generate a reconstructed signal or to generate a residual signal. Converter 232 may generate transform coefficients by applying a transform method to the residual signal. For example, the transform method may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). . Here, GBT means the transformation obtained from a graph when the relation information between pixels is represented by the graph. CNT refers to the transformation obtained from the prediction signal generated using all the previously reconstructed pixels. In addition, the transformation process can be applied to square blocks of pixels that are the same size, or can be applied to blocks that are variable in size rather than square.

[69] The quantizer 233 may quantize the transform coefficients and transmit them to the entropy encoder 240, and the entropy encoder 240 may encode the quantized signal (information about the quantized transform coefficients) and output the encoded signal in a bit stream. Information about the quantized transform coefficients may be called residual information. Quantizer 233 may reorder the quantized block type transform coefficients into one-dimensional vector form based on the scanning order of the coefficients, and generate information about the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form. Entropy encoder 240 may perform various encoding techniques, such as, for example, exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like. Entropy encoder 240 may encode information needed for video/image reconstruction other than quantized transform coefficients (eg, syntax element values, etc.) together or separately. Encoded information (eg, encoded video/image information) may be transmitted or stored in Network Abstraction Layer (NAL) units in the form of a bitstream. The video/image information may further include information about various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), a video parameter set (VPS), and the like. In addition, the video/image information may further include general restriction information. In the present disclosure, information and/or syntax elements transmitted/signaled from an encoding device to a decoding device may be included in video/image information. The video/image information may be encoded by the above-described encoding procedure and included in the bitstream. The bitstream may be transmitted over a network or may be stored in a digital storage medium. Here, the network may include a broadcast network, a communications network and/or the like, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD and the like . A transmitter (not shown) transmitting the signal output from the entropy encoder 240 and/or a storage module (not shown) storing the signal may be configured as an internal/external element of the encoding device 200, or the transmitter may be included in the entropy encoder 240.

[70] The quantized transform coefficients output from quantizer 233 can be used to generate a prediction signal. For example, a residual signal (residual block or residual samples) may be recovered by applying dequantization and inverse transform to the quantized transform coefficients by dequantizer 234 and inverse transformer 235. Adder 250 adds the reconstructed residual signal with the prediction signal output from inter-predictor 221 or intra-predictor 222 so that a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) can be generated. If there is no remainder for the target processing block, as in the case where the skip mode is applied, the predicted block can be used as a reconstructed block. The adder 250 may be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used to intra-predict the next processing target block in the current picture and, as described below, can be used to inter-predict the next picture through filtering.

[71] Meanwhile, luminance chroma scaling (LMCS) mapping can be used in the process of encoding and/or restoring a picture.

[72] Filter 260 may improve subjective/objective video quality by applying filtering to the reconstructed signal. For example, filter 260 may generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and may store the modified reconstructed picture in memory 270, specifically in DPB memory 270. Various filtering methods may include, for example, deblocking filtering, adaptive sampling bias, adaptive loop filter, bidirectional filter and the like. Filter 260 may generate various filtering-related information and transmit the generated information to entropy encoder 240, as described later in the description of each filtering method. The filtering information may be encoded by entropy encoder 240 and output in the form of a bitstream.

[73] The modified reconstructed picture that has been transferred to the memory 270 can be used as a reference picture in the inter-predictor 221. Through this, the encoding device can prevent prediction inconsistency in the encoding device 200 and the decoding device when inter-prediction is applied, and can also improve coding efficiency.

[74] DPB memory 270 may store a modified reconstructed picture for use as a reference picture in inter-predictor 221. Memory 270 may store motion information of a block in the current picture from which motion information was derived (or encoded) and/or information movement of blocks in the already restored picture. The stored motion information may be transmitted to the inter-predictor 221 for use as neighbor motion information or temporary neighbor motion information. Memory 270 may store reconstructed samples of reconstructed blocks in the current picture and may transmit them to intra-predictor 222.

[75] FIG. 3 is a diagram schematically illustrating the configuration of a video/image decoding apparatus to which the present disclosure can be applied.

[76] With reference to FIG. 3, decoding apparatus 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-predictor 331 and an intra-predictor 332. A residual processor 220 may include includes a dequantizer 321 and an inverse converter 321. The entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 that were described above may be configured by one or more hardware components (e.g., decoder chipsets or processors) in accordance with embodiment. In addition, memory 360 may include a decoded picture buffer (DPB) or may be configured with a digital storage medium. The hardware component may further include memory 360 as an internal/external component.

[77] When a bitstream including video/image information is input, the decoding apparatus 300 can reconstruct the image according to the process by which the video/image information was processed in the encoding apparatus according to FIG. 2. For example, the decoding apparatus 300 may output units/blocks based on the block division related information obtained from the bitstream. The decoding apparatus 300 may perform decoding using a processing unit applied in the encoding apparatus. Therefore, the decoding processing unit may be, for example, an encoding unit, which may be divided according to a quadtree structure, a binary tree structure, and/or a ternary tree structure from an encoding tree unit or a largest encoding unit. One or more transformation units may be derived from a coding unit. And the reconstructed image signal decoded and output by the decoding device 300 can be reproduced by the playback device.

[78] The decoding apparatus 300 may receive a signal output from the encoding apparatus according to FIG. 2 in the form of a bit stream, and the received signal may be decoded by entropy decoder 310. For example, entropy decoder 310 may parse the bit stream to output information (eg, video/image information) necessary for image reconstruction (or picture reconstruction). The video/image information may further include information about various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), a video parameter set (VPS), and the like. In addition, the video/image information may further include general restriction information. The decoding apparatus may further decode the picture based on the parameter set information and/or general constraint information. The signaled/received information and/or syntax elements described later in the present disclosure may be decoded through a decoding procedure and obtained from the bitstream. For example, entropy decoder 310 may decode information in the bit stream based on an encoding method such as exponential Golomb coding, CAVLC, or CABAC, and may output the syntax element value required to reconstruct the image and the quantized values of the transform coefficients for the remainder. More specifically, the CABAC entropy decoding method can take a binary element (bin) corresponding to each syntax element in the bitstream, determine a context model using the information of the target decoding syntax element and the decoding information of the adjacent and target decoding blocks or the information of the symbol/bin decoded on the previous one. stage, predict the probability of generating a bin according to a certain context model, and perform arithmetic decoding of the bin to generate a symbol corresponding to the meaning of each syntactic element. Here, the CABAC entropy decoding method can update the context model using the information of the decoded symbol/bin for the context model of the next symbol/bin after determining the context model. Prediction information among the information decoded by the entropy decoder 310 may be provided to the predictor (inter-predictor 332 and intra-predictor 331), and residual values, that is, quantized transform coefficients on which entropy decoding was performed in the entropy decoder 310, and associated parameter information may be input to residual processor 320. Residual processor 320 may output a residual signal (residual block, residual samples, residual sample array). In addition, filtering information among the information decoded by the entropy decoder 310 may be provided to the filter 350. Meanwhile, a receiver (not shown) that receives a signal output from the encoding device may be further configured as an internal/external element of the device 300 decoding device, or the receiver may be a component of the entropy decoder 310. Meanwhile, the decoding device according to the present disclosure may be called a video/image/picture decoding device, and the decoding device may be classified into an information decoder (video/image/picture information decoder) and a decoder sampling (video/image/picture sampling decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverse converter 322, an adder 340, a filter 350, a memory 360, an inter-predictor 332, and an intra-predictor 331.

[79] Dequantizer 321 may output transform coefficients by dequantizing the quantized transform coefficients. The dequantizer 321 may reorder the quantized transform coefficients in the form of a two-dimensional block. In this case, reordering may be performed based on the order of coefficient scanning performed in the encoder. The dequantizer 321 may perform dequantization on the quantized transform coefficients using a quantization parameter (eg, quantization step size information) and obtain the transform coefficients.

[80] The dequantizer 322 obtains the residual signal (residual block, residual sample array) by inversely transforming the transform coefficients.

[81] The predictor may perform prediction on the current block and generate a predicted block including prediction samples for the current block. The predictor can determine whether intra-prediction or inter-prediction is applied to the current block based on the prediction information output from the entropy decoder 310, and can determine the intra/inter-prediction mode in particular.

[82] The predictor can generate a prediction signal based on various prediction methods. For example, a predictor may apply intra-prediction or inter-prediction on one block, and may also apply intra-prediction and inter-prediction simultaneously. This may be called combined inter- and intra-prediction (CIIP). In addition, the predictor can perform intra-block copying (IBC) to make predictions on a block. The intra-block copy mode can be used for encoding image/video game content or the like, such as screen content encoding (SCC). Although IBC primarily performs prediction in the current block, it can perform similar to inter-prediction in that it outputs a reference block in the current block. That is, the IBC may use at least one of the inter-prediction methods described in the present disclosure.

[83] Intra predictor 331 may predict the current block by referring to samples in the current picture. These samples may be located adjacent to or spaced apart from the current block in accordance with the prediction mode. In intra-prediction, prediction modes may include a plurality of undirected modes and a plurality of directed modes. Intra predictor 331 may determine the prediction mode applied to the current block using the prediction mode applied to the adjacent block.

[84] Inter-predictor 332 may output a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector in the reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter-prediction mode, motion information may be predicted on a block, sub-block, or sample basis based on the correlation of motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter-prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block existing in the current picture and a temporal neighbor block existing in the reference picture. For example, inter-predictor 332 may configure a motion information candidate list based on neighboring blocks and output a motion vector and/or reference picture index of the current block based on the received candidate selection information. Inter-prediction may be performed based on various prediction modes, and the prediction information may include information indicating an inter-prediction mode for the current block.

[85] The adder 340 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the resulting residual signal with a prediction signal (predicted block, predicted array of samples) output from the predictor 330. If there is no residual for the target processing block, as in the case when the skip mode is applied, the predicted block can be used as a reconstructed block.

[86] The adder 340 may be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used to intra-predict the next processing target block in the current block and, as described below, can be output by filtering or used to inter-predict the next picture.

[87] Meanwhile, luma chroma scaling (LMCS) mapping can be applied in the picture decoding process.

[88] Filter 350 may improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, filter 350 may generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and may transfer the modified reconstructed picture to memory 260, specifically to a DPB of memory 360. Various filtering methods may include, for example, deblocking filtering, adaptive sample bias, adaptive loop filter, bidirectional filter and the like.

[89] The (modified) reconstructed picture that has been stored in the DPB memory 360 can be used as a reference picture in the inter-predictor 332. The memory 360 can store motion information of a block in the current picture from which motion information is derived (or decoded), and /or information about the movement of blocks in an already restored picture. The stored motion information may be transmitted to the inter-predictor 332 for use as neighbor motion information or temporary neighbor motion information. Memory 360 may store reconstructed samples of reconstructed blocks in the current picture and transmit them to intra-predictor 331.

[90] Herein, the examples described for predictor 330, dequantizer 321, inverse converter 322, and filter 350 of decoder 300 may be similarly or correspondingly applied to predictor 220, dequantizer 234, inverse converter 235, and filter 260 of encoder 200, respectively.

[91] As described above, prediction is performed to improve compression efficiency when performing video encoding. By this, a predicted block can be generated including prediction samples for the current block, that is, a target encoding block. Here, the predicted block includes prediction samples in a spatial domain (or pixel domain). The predicted block may be output equally in the encoding apparatus and the decoding apparatus, and the encoding apparatus may improve the efficiency of image encoding by signaling to the decoding apparatus not the original sample value of the original block itself, but the residual information (residual information) between the original block and the predicted block. The decoding apparatus can output a residual block including residual samples based on the residual information, generate a reconstructed block including the reconstructed samples by summing the residual block and the predicted block, and generate a reconstructed picture including the reconstructed blocks.

[92] Residual information can be generated through transformation and quantization procedures. For example, the encoding device may output a residual block between an original block and a predicted block, output transform coefficients by performing a transform procedure on the residual samples (residual array of samples) included in the residual block, and output quantized transform coefficients by performing a quantization procedure on the transform coefficients, such as that it can signal associated residual information to a decoder (via a bit stream). Here, the residual information may include value information, location information, a transformation method, a transformation kernel, a quantization parameter, or the like of quantized transformation coefficients. The decoding apparatus may perform a dequantization/deconversion procedure and output residual samples (or a residual block of samples) based on the residual information. The decoding apparatus may generate a reconstructed block based on the predicted block and the residual block. The encoding apparatus may output a residual block by dequantizing/deconverting the quantized transform coefficients for the next picture inter-prediction reference and may generate a reconstructed picture based on this.

[93] FIG. 4 shows the structure of a content streaming system to which the present disclosure is applicable.

[94] A content streaming system to which the present disclosure applies may broadly include an encoding server, a streaming server, a web server, a media storage device, a user equipment, and a media input device.

[95] An encoding server functions to compress digital data of content input from multimedia input devices such as a smartphone, camera, DVR, etc. into digital data to generate a bitstream and transmit it to the streaming device. server. As another example, when a multimedia input device such as a smartphone, camera, video recorder, etc. directly generates a bitstream, the encoding server may be omitted. The bitstream may be generated by an encoding method or a bitstream generation method to which the present disclosure is applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[96] The streaming server transmits multimedia data to the user equipment based on the user's request through a web server, which functions as a tool that informs the user about the service. When a user requests a desired service, the web server forwards the request to the streaming server, and the streaming server transmits the media data to the user. In this regard, the content streaming system may include a separate control server, in which case, the control server functions to manage commands/responses between the respective equipments in the content streaming system.

[97] The streaming server may receive content from a media storage and/or encoding server. For example, when content is received from an encoding server, the content may be received in real time. In this case, to provide a smooth streaming service, the streaming server may store the bitstream for a predetermined period of time.

[98] For example, user equipment may include a mobile phone, smartphone, laptop, digital broadcast terminal, personal digital assistant (PDA), portable multimedia player (PMP), navigator, slim PC, tablet PC, ultrabook, wearable device (eg , watch type terminal (smart watch), glasses type terminal (smart glasses), head mounted display (HMD)), digital TV, desktop computer, digital pointer and the like. Each of the servers in a content streaming system can operate as a distributed server, in which case, the data received by each server can be processed in a distributed manner.

[99] FIG. 5 schematically illustrates a multiple conversion technology in accordance with an embodiment of the present disclosure.

[100] With reference to FIG. 5, the converter may correspond to a converter in the encoding device according to FIG. 2, and the inverse converter may correspond to the inverse converter in the encoding device according to FIG. 2 or an inverse converter in the decoding device according to FIG. 3.

[101] The converter can output the (primary) conversion coefficients by performing a primary conversion based on the residual samples (residual sample array) in the residual block (S510). This primary transformation may be referred to as the main (base) transformation. Here, the primary transform may be based on the multiple transform selection (MTS), and when the multiple transform is applied as the primary transform, it may be referred to as the multiple primary transform.

[102] The multiple base transform may represent a transform method further using discrete cosine transform (DCT) type 2 and discrete sine transform (DST) type 7, DCT type 8 and/or DST type 1. That is, the multiple base transform may represent a method transform to transform the spatial domain residual signal (or residual block) into frequency domain transform coefficients (or primary transform coefficients) based on a plurality of transform kernels selected from DCT type 2, DST type 7, DCT type 8 and DST type 1. Here, the coefficients The primary conversion may be referred to as the temporary conversion coefficients from the converter's point of view.

[103] In other words, when a conventional transform method is applied, transform coefficients can be generated by applying a spatial domain to frequency domain transform to a residual signal (or residual block) based on DCT type 2. In contrast, when a multiple fundamental transform is applied, transform coefficients (or primary transform coefficients) can be generated by applying a spatial domain to frequency domain transform to the residual signal (or residual block) based on DCT type 2, DST type 7, DCT type 8 and/or DST type 1. Here, DCT Type 2, DST Type 7, DCT Type 8, and DST Type 1 may be referred to as transform types, transform kernels, or transform bases. These DCT/DST transformation types can be determined based on basis functions.

[104] When multiple core transform is performed, a vertical transform kernel and a horizontal transform kernel for a target block can be selected from the transform kernels, a vertical transform can be performed on a target block based on a vertical transform kernel, and a horizontal transform can be performed on a target block based on a kernel horizontal transformation. Here, a horizontal transform may indicate a transform on the horizontal components of the target block, and a vertical transform may indicate a transform on the vertical components of the target block. The vertical transform kernel/horizontal transform kernel may be adaptively determined based on the prediction mode and/or transform index for the target block (CU or subblock) including the residual block.

[105] In addition, according to the example, if the primary transformation is performed by applying MTS, the mapping relation for the transformation kernels can be set by setting special basis functions to predefined values and combining the basis functions for use in the vertical transformation or horizontal transformation. For example, when the horizontal transformation kernel is expressed as trTypeHor and the vertical direction transformation kernel is expressed as trTypeVer, the value 0 for trTypeHor or trTypeVer can be set to DCT2, the value 1 for trTypeHor or trTypeVer can be set to DST7, and the value 2 for trTypeHor or trTypeVer can be installed in DCT8.

[106] In this case, the MTS index information may be encoded and signaled to the decoder to indicate any one of a plurality of sets of transform cores. For example, MTS index 0 may indicate that both trTypeHor and trTypeVer are 0, MTS 1 may indicate that both trTypeHor and trTypeVer are 1, MTS index 2 may indicate that trTypeHor is 2 and trTypeVer is 1, MTS index 3 may indicate that trTypeHor is 1 and trTypeVer is 2, and MTS index 4 may indicate that both trTypeHor and trTypeVer are 2.

[107] In one example, sets of transformation kernels according to MTS index information are shown in the following table.

[108] [Table 1]

[109] The converter can output modified (secondary) conversion coefficients by performing secondary conversion based on the (primary) conversion coefficients (S520). The primary transform is a conversion from the spatial domain to the frequency domain, and the secondary transform refers to the conversion to a more compressed expression using the correlation existing between the (primary) transform coefficients. The secondary transformation may include an inseparable transformation. In this case, the secondary transform may be called a non-separable secondary transform (NSST) or a mode-dependent non-separable secondary transform (MDNSST). The inseparable secondary transform may be a transform that generates modified transform coefficients (or secondary transform coefficients) for the residual signal by a secondary transform, based on the inseparable transform matrix of the (primary) transform coefficients derived by the primary transform. However, vertical transformation and horizontal transformation cannot be applied separately (or horizontal and vertical transformations cannot be applied independently) to the (primary) transformation coefficients, but transformations can be applied simultaneously based on the inseparable transformation matrix. In other words, the inseparable secondary transform may represent a transform method that is not separately applied in the vertical direction and horizontal direction for the (primary) transform coefficients, and, for example, the two-dimensional signals (transform coefficients) are reordered into a one-dimensional signal through some specific direction (for example, the direction row-first or column-first directions), and then modified transform coefficients (or secondary transform coefficients) are generated based on the inseparable transform matrix. For example, according to row first order, M×N blocks are arranged in a line in the order of first row, second row, ... and Nth row. According to column first order, M×N blocks are arranged in a line in the order of first column, second column, ... and Nth column. An inseparable secondary transform may be applied to the top-left region of a block configured with (primary) transform coefficients (hereinafter may be referred to as a transform coefficient block). For example, if the width (W) and height (H) of the transform coefficient block are equal to or greater than 8, then an inseparable secondary 8×8 transform may be applied to the top-left 8×8 region of the transform coefficient block. Additionally, if the width (W) and height (H) of the transform coefficient block are equal to or greater than 4, and the width (W) or height (H) of the transform coefficient block is less than 8, then an inseparable 4x4 secondary transform can be applied to the top-left region min(8,W) × min(8,H) of the transform coefficient block. However, the embodiment is not limited to this, and for example, even if only the condition that the width (W) or height (H) of the transform coefficient block is equal to or greater than 4 is satisfied, then an inseparable 4x4 secondary transform can be applied to the top-left area min(8,W) × min(8,H) of the block of transformation coefficients.

[110] Specifically, for example, if a 4x4 input block is used, the non-separable secondary transform can be performed as follows.

[111] The X 4x4 input block can be represented as follows.

[112] [Equation 1]

[113] If X is represented in vector form, then the vector can be presented as below.

[114] [Equation 2]

[115] In Equation 2, the vector is the one-dimensional vector obtained by reordering the two-dimensional block X in Equation 1 according to row-first order.

[116] In this case, the secondary inseparable transform can be calculated as follows.

[117] [Equation 3]

[118] In this equation, represents a vector of transform coefficients, and T represents a 16x16 (non-separable) transform matrix.

[119] Through the above Equation 3, a 16×1 vector of transform coefficients can be derived , And can be rearranged into a 4x4 block using scan order (horizontal, vertical, diagonal, etc.). However, the above calculation is an example, and a Givens hypercube transform (HyGT) or the like can be used to calculate the inseparable secondary transform to reduce the computational complexity of the inseparable secondary transform.

[120] Meanwhile, in the non-separable secondary transformation, the transformation kernel (transformation base, transformation type) can be selected as mode-dependent. In this case, the mode may include an intra-prediction mode and/or an inter-prediction mode.

[121] As described above, the non-separable secondary transform may be performed based on an 8×8 transform or a 4×4 transform determined based on the width (W) and height (H) of the transform coefficient block. 8×8 transform refers to a transform that is applicable to an 8×8 region included in a block of transform coefficients when both W and H are equal to or greater than 8, and the 8×8 region may be a top-left 8×8 region in the block of conversion coefficients. Similarly, a 4x4 transform refers to a transform that is applicable to a 4x4 region included in a block of transform coefficients when both W and H are equal to or greater than 4, and the 4x4 region may be a top-left region of 4 ×4 in the conversion coefficient block. For example, an 8×8 matrix of a transform kernel may be a 64×64/16×64 matrix, and a 4×4 matrix of a transform kernel may be a 16×16/8×16 matrix.

[122] Here, to select a mode-dependent transform kernel, two non-separable secondary transform kernels per transform set for the non-separable secondary transform can be configured for both the 8×8 transform and the 4×4 transform, and there may be four transform sets. That is, four transform sets can be configured for an 8x8 transform, and four transform sets can be configured for a 4x4 transform. In this case, each of the four transform sets for the 8×8 transform may include two 8×8 transform kernels, and each of the four transform sets for the 4×4 transform may include two 4×4 transform kernels.

[123] However, since the transformation size, that is, the size of the region to which the transformation is applied, may be of a size other than 8×8 or 4×4, for example, the number of sets may be n, and the number of transformation cores in each set may be k.

[124] A set of transformations may be referred to as an NSST set or a LFNST set. A particular set among the transformation sets may be selected, for example, based on the intra-prediction mode of the current block (CU or subblock). The Low Frequency Nonseparable Transform (LFNST) may be an example of a shortened nonseparable transform that will be described later, and represents the nonseparable transform for the low frequency component.

[125] For reference, for example, an intra-prediction mode may include two non-directional (or non-angular) intra-prediction modes and 65 directional (or angular) intra-prediction modes. The non-directional intra-prediction modes may include planar intra-prediction mode No. 0 and DC intra-prediction mode No. 1, and the directional intra-prediction modes may include 65 intra-prediction modes No. 2 to 66. However, this is an example , and this document can be applied even if the number of intra-prediction modes differs. Meanwhile, in some cases, the intra-prediction mode No. 67 can be further used, and the intra-prediction mode No. 67 can represent the linear model (LM) mode.

[126] FIG. 6 shows as an example the intra-directional modes of 65 prediction directions.

[127] With reference to FIG. 6, based on the intra-prediction mode 34 having a left-up diagonal prediction direction, the intra-prediction modes can be divided into intra-prediction modes having a horizontal direction and intra-prediction modes having a vertical direction. In fig. 6, H and V denote horizontal directivity and vertical directivity, respectively, and the numbers -32 to 32 indicate offsets in units of 1/32 at a discrete grid position. These numbers can represent an offset for the mode index value. Intra-prediction modes 2 to 33 have a horizontal focus, and intra-prediction modes 34 to 66 have a vertical focus. Strictly speaking, the intra-prediction mode 34 may be considered to be neither horizontal nor vertical, but may be classified as being of a horizontal orientation when defining the secondary transform transform set. This is because the input data is transposed to be used for the vertical direction mode symmetrically based on the intra-prediction mode 34, and the input data alignment method for the horizontal mode is used for the intra-prediction mode 34. Transposing the input data means that the rows and columns of a two-dimensional block of M×N data are switched to N×M data. The intra-prediction mode 18 and the intra-prediction mode 50 may represent the horizontal intra-prediction mode and the vertical intra-prediction mode, respectively, and the intra-prediction mode 2 may be referred to as the diagonal right-up intra-prediction mode since the intra-prediction mode 2 -prediction has a left reference pixel and performs prediction in a right-up direction. Likewise, intra-prediction mode 34 may be referred to as a right-down diagonal intra-prediction mode, and intra-prediction mode 66 may be referred to as a left-down diagonal intra-prediction mode.

[128] According to the example, four sets of transformations in accordance with the intra-prediction mode can be displayed, for example, as shown in the following table.

[129] [Table 2]

[130] As shown in Table 2, any one of the four sets of transformations, that is, lfnstTrSetIdx, can be mapped to any of the four indices, that is, from 0 to 3, in accordance with the intra-prediction mode.

[131] When it is determined that a particular set is used for an inseparable transformation, one of the k transformation kernels in the particular set can be selected using the inseparable secondary transformation index. The encoding device may output a non-separable secondary transform index indicating a particular transform kernel based on a rate-distortion (RD) test, and may signal the non-separable secondary transform index to the decoding device. The decoder may select one of the k transform kernels in a particular set based on the inseparable secondary transform index. For example, lfnst index value 0 may refer to a first non-separable secondary transform kernel, lfnst index value 1 may refer to a second non-separable secondary transform kernel, and lfnst index value 2 may refer to a third non-separable secondary transform kernel. Alternatively, an lfnst index value of 0 may indicate that the first non-separable secondary transform is not applied to the target block, and lfnst index values 1 to 3 may indicate three transform cores.

[132] The converter may perform an inseparable secondary transform based on the selected transform kernel and may obtain modified (secondary) transform coefficients. As described above, the modified transform coefficients can be output as transform coefficients quantized by a quantizer, and can be encoded and signaled to a decoding device and transmitted to a dequantizer/inverse converter in the encoding device.

[133] Meanwhile, as described above, if the secondary transform is omitted, the (primary) transform coefficients that are the output of the primary (separable) transform can be output as transform coefficients quantized by a quantizer as described above, and can be encoded and signaled to the decoder and transmitted to the dequantizer/inverter in the encoder.

[134] The inverse converter can perform a number of procedures in the reverse order of how they were performed in the above-described converter. The inverse converter can receive (dequantized) transform coefficients and output (primary) transform coefficients by performing a secondary (inverse) transform (S550) and can obtain a residual block (residual samples) by performing a primary (inverse) transform on the (primary) transform coefficients (S560 ). In this regard, the primary conversion coefficients may be called modified conversion coefficients from the point of view of the inverse converter. As described above, the encoding device and the decoding device can generate a reconstructed block based on the residual block and the predicted block, and can generate a reconstructed picture based on the reconstructed block.

[135] The decoding device may further include a secondary inverse transform application determiner (or an element for determining whether the secondary inverse transform should be applied) and a secondary inverse transform determination (or an element for determining whether the secondary inverse transform is applied). The secondary inverse transform application determiner may determine whether the secondary inverse transform should be applied. For example, the secondary inverse transform may be NSST, RST, or LFNST, and the secondary inverse transform application determiner may determine whether to apply the secondary inverse transform based on the secondary transform flag obtained by parsing the bitstream. In another example, the secondary inverse transform application determiner may determine whether to apply the secondary inverse transform based on the transform coefficient of the residual block.

[136] The secondary inverse transform determinant may determine the secondary inverse transform. In this case, the secondary inverse transform determiner may determine the secondary inverse transform applied to the current block based on the LFNST transform set (NSST or RST) specified in accordance with the intra-prediction mode. In an embodiment, the method for determining the secondary transformation may be determined depending on the method for determining the primary transformation. Various combinations of primary transforms and secondary transforms may be determined in accordance with the intra-prediction mode. Additionally, in the example, the secondary inverse transform determiner may determine the region to which the secondary inverse transform is applied based on the size of the current block.

[137] Meanwhile, as described above, if the secondary (inverse) transform is omitted, the (dequantized) transform coefficients can be received, the primary (separable) inverse transform can be performed, and the residual block(s) can be obtained. As described above, the encoding device and the decoding device can generate a reconstructed block based on the residual block and the predicted block, and can generate a reconstructed picture based on the reconstructed block.

[138] Meanwhile, in the present disclosure, the reduced secondary transform (RST), in which the size of the transformation matrix (kernel) is reduced, can be applied in the NSST concept to reduce the amount of computation and memory required for the non-separable secondary transform.

[139] Meanwhile, the transformation kernel, the transformation matrix, and the coefficient forming the transformation kernel matrix, that is, the kernel coefficient or the matrix coefficient described in the present disclosure, can be expressed in 8 bits. This may be a requirement for implementation in the decoding apparatus and the encoding apparatus, and may reduce the amount of memory required to store the conversion kernel, at a performance penalty that may be rationally tolerated, compared to the existing 9-bit or 10-bit cases. Additionally, expressing the kernel matrix in 8 bits may allow the use of a small multiplier and may be more suitable for Single Instruction Multiple Data (SIMD) architecture instructions used for optimal software implementation.

[140] In this specification, the term "RST" may mean a transform that is performed on the residual samples for a target block based on a transform matrix that is reduced in size according to a reduction factor. In the case of performing a reduced transformation, the amount of computation required for the transformation can be reduced due to the reduction in the size of the transformation matrix. That is, RST can be used to solve the problem of computational complexity encountered in non-separable transformation or large block size transformation.

[141] RST may be defined by various terms such as shorthand transform, shortened secondary transform, shorthand transform, simplified transform, simple transform, and the like, and the name RST is not limited to these examples. Alternatively, since RST is primarily performed in the low-frequency region involving a non-zero coefficient in the transform block, it may be referred to as low-frequency non-separable transform (LFNST). The transformation index may be referred to as the LFNST index.

[142] Meanwhile, when the secondary inverse transform is performed based on the RST, the inverse transformer 235 of the encoding device 200 and the inverse transformer 322 of the decoding device 300 may include an inverse shortened secondary transformer that outputs modified transform coefficients based on the inverse RST transform coefficients, and an inverse primary converter that outputs residual samples for the target block based on the inverse primary transform of the modified transform coefficients. The inverse of the primary transformation refers to the inverse of the primary transformation applied to the remainder. In the present disclosure, deriving a transformation coefficient based on a transformation may refer to deriving a transformation coefficient by applying a transformation.

[143] FIG. 7 is a diagram illustrating an RST in accordance with an embodiment of the present disclosure.

[144] In the present disclosure, a "target block" may refer to a current block to be encoded, a residual block, or a transform block.

[145] In RST, according to the example, an N-dimensional vector can be mapped to an R-dimensional vector located in another space, so that a reduced transformation matrix can be defined, where R is less than N. N can mean the square of the side length of the block , to which the transformation is applied, or the total number of transformation factors corresponding to the block to which the transformation is applied, and the shortened factor may indicate the R/N value. A shortened ratio may be referred to as a shortened ratio, a shortened ratio, a simplified ratio, a simple ratio, or various other terms. Meanwhile, R may be referred to as a short ratio, but depending on the circumstances, the short ratio may mean R. Moreover, depending on the circumstances, the short ratio may mean N/R value.

[146] In an example, a shortened indicator or a shortened coefficient may be signaled by a bit stream, but the example is not limited to this. For example, a predetermined value for a shortened index or shortened coefficient may be stored in each of the encoding apparatus 200 and the decoding apparatus 300, in which case, the shortened index or shortened coefficient may not be signaled separately.

[147] The size of the reduced transform matrix according to the example may be R×N smaller than the N×N size of the normal transform matrix, and may be determined as in Equation 4 below.

[148] [Equation 4]

[149] The matrix T in the reduced transform block shown in FIG. 7(a) may mean the T _R × _N matrix of Equation 4. As shown in FIG. 7(a), when the reduced transform matrix T _R × _N is multiplied by the residual samples for the target block, the transform coefficients for the target block can be output.

[150] In the example, if the block size to which the transformation is applied is 8×8 and R=16 (ie, R/N=16/64=1/4), then RST according to FIG. 7(a) can be expressed as a matrix operation as shown in Equation 5 below. In this case, the memory and multiplication calculation can be reduced to about 1/4 by means of a shortening factor.

[151] In the present disclosure, a matrix operation can be understood as the operation of multiplying a column vector by a matrix located to the left of the column vector to obtain a column vector.

[152] [Equation 5]

[153] In Equation 5, r ₁ to r ₆₄ may represent residual samples for the target block and may specifically be transform coefficients generated by applying the primary transform. As a result of computing Equation 5, the conversion coefficients c _i for the target block can be derived, and the derivation process c _i can be the same as in Equation 6.

[154] [Equation 6]

[155] As a result of calculating Equation 6, the conversion coefficients from ₁ to c _R for the target block can be derived. That is, when R=16, the conversion coefficients from ₁ to c ₁₆ for the target block can be derived. If, instead of RST, a regular transform is applied and a 64x64 (NxN) transformation matrix is multiplied by the 64x1 (Nx1) residual samples, then only 16 (R) transform coefficients are output for the target block since RST was applied. although 64(N) transform coefficients are derived for the target block. Since the total number of transform coefficients for the target block is reduced from N to R, the amount of data transmitted by the encoding device 200 to the decoding device 300 is reduced, so that the transmission efficiency between the encoding device 200 and the decoding device 300 can be improved.

[156] When considered in terms of the size of the transformation matrix, the size of the regular transformation matrix is 64x64 (NxN), but the size of the reduced transformation matrix is reduced to 16x64 (RxN), so the memory usage in case of performing RST may be reduced in R/N ratio compared to the case of performing a regular conversion. In addition, compared with the number of N×N multiplication calculations in the case of using a regular transformation matrix, using a reduced transformation matrix can reduce the number of multiplication calculations in the ratio R/N (R×N).

[157] In an example, transformer 232 of encoder 200 may output transform coefficients for a target block by performing a primary transform and an RST-based secondary transform on the residual samples for the target block. These transform coefficients may be transmitted to the inverse transformer of the decoding apparatus 300, and the inverse transformer 322 of the decoding apparatus 300 may output modified transform coefficients based on the inverse reduced secondary transform (RST) for the transform coefficients, and may output residual samples for the target block based on the inverse primary transform for modified conversion factors.

[158] The size of the inverse RST matrix T _N × _R according to the example is N×R smaller than the N×N size of the regular inverse transform matrix, and is in a transposition relationship with the reduced transform T _R × _N matrix shown in Equation 4.

[159] The matrix T ^t in the reduced inverse transform block shown in FIG. 7(b) may denote the matrix T _R × _N ^T of the inverse RST (superscript T denotes transposition). When the inverse RST matrix T _R × _N ^T is multiplied by the transform coefficients for the target block, as shown in FIG. 7(b), modified transform coefficients for the target block or residual samples for the current block can be output. The T _R × _N ^T matrix of the inverse RST can be expressed as (T _R × _N ^T ) _NxR .

[160] More specifically, when the inverse RST is applied as a secondary inverse transform, modified transform coefficients for the target block can be output when the T _R × _N ^T matrix of the inverse RST is multiplied by the transform coefficients for the target block. Meanwhile, the inverse RST can be applied as an inverse primary transform, in which case the residual samples for the target block can be output when the T _R × _N ^T matrix of the inverse RST is multiplied by the transform coefficients for the target block.

[161] In the example, if the block size to which the inverse transform is applied is 8×8 and R=16 (ie, R/N=16/64=1/4), then RST according to FIG. 7(b) can be expressed as a matrix operation as shown in Equation 7 below.

[162] [Equation 7]

[163] In Equation 7, c ₁ -c ₁₆ may represent the conversion coefficients for the target block. As a result of the calculation of Equation 7, r _i representing modified transform coefficients for the target block or residual samples for the target block can be output, and the derivation process for r _i can be the same as in Equation 8.

[164] [Equation 8]

[165] As a result of the calculation of Equation 8, r ₁ to r _N representing the modified transform coefficients for the target block or the residual samples for the target block can be derived. When considered in terms of the size of the inverse transform matrix, the size of the regular inverse transform matrix is 64×64 (N×N), but the size of the reduced inverse transform matrix is reduced to 64×16 (R×N), so that the memory usage in case of performing the inverse The RST can be reduced in the R/N ratio compared to the case of performing a regular inverse transform. Additionally, compared with the number of N×N multiplication calculations in the case of using a regular inverse transform matrix, using a reduced inverse transform matrix can reduce the number of multiplication calculations in the ratio R/N (N×R).

[166] The transform set configuration shown in Table 2 can be applied to 8x8 RST. That is, 8x8 RST can be applied according to the transform set in Table 2. Since one transform set includes two or three transforms (kernels) according to the intra-prediction mode, it can be configured to select one of up to four transformations, including the case where a secondary transformation is not applied. In a transformation where a secondary transformation is not applied, this can be taken into account to apply the identity matrix. Assuming that indexes 0, 1, 2 and 3 are respectively assigned to the four transforms (for example, index 0 could be allocated for the case where the identity matrix is applied, that is, the case where the secondary transform is not applied), the index transform or index lfnst as syntax element may be signaled for each block of transform coefficients, thereby indicating the transform applied. That is, for a top-left 8x8 block, using the transform index, you can specify 8x8 RST in the RST configuration or specify 8x8 lfnst when LFNST is applied. 8×8 lfnst and 8×8 RST refer to transformations applicable to an 8×8 area included in a transform coefficient block when the W and H of the target block to be transformed are equal to or greater than 8, and the 8×8 area can be top-left 8x8 region in the transform coefficient block. Similarly, 4x4 lfnst and 4x4 RST refer to transformations applicable to a 4x4 region included in a block of transform coefficients when the W and H of the target block are equal to or greater than 4, and the 4x4 region may be top- left 4x4 region in the transform coefficient block.

[167] According to an embodiment of the present disclosure, for transformation in the encoding process, only 48 pieces of data can be selected, and a maximum 16x48 matrix of transformation kernels can be applied to them, instead of applying a 16x64 matrix of transformation kernels to 64 pieces of data , forming an 8x8 area. Here, "maximum" means that m has a maximum value of 16 in the m×48 matrix of the transformation kernel for generating m coefficients. That is, when RST is performed by applying an m×48 matrix of transformation kernels (m≤16) to an 8×8 region, 48 pieces of data are input and m coefficients are generated. When m is 16, 48 pieces of data are input and 16 coefficients are generated. That is, assuming that 48 pieces of data form a 48x1 vector, the 16x48 matrix and the 48x1 vector are sequentially multiplied, thereby generating a 16x1 vector. Here, 48 pieces of data forming an 8x8 area can be properly arranged to form a 48x1 vector. For example, a 48×1 vector may be formed based on the 48 pieces of data forming a region excluding the bottom-right 4×4 region among the 8×8 regions. Here, when the matrix operation is performed by applying a maximum 16×48 matrix of transformation kernels, 16 modified transformation coefficients are generated, and the 16 modified transformation coefficients can be arranged in the upper-left region of 4×4 according to the scanning order, and the upper-right region of 4 The ×4 and bottom-left 4x4 area can be filled with zeros.

[168] For inverse transformation during the decoding process, a transposed matrix of the above-described transformation kernel matrix can be used. That is, when the inverse RST or LFNST is performed in the inverse transform process by the decoding device, the input coefficient data to which the inverse RST is applied is configured into a one-dimensional vector according to a predetermined arrangement order, and a vector of modified coefficients obtained by multiplying the one-dimensional vector and the corresponding inverse matrix The RST to the left of the one-dimensional vector can be arranged into a two-dimensional block according to a predefined arrangement order.

[169] Finally, in the transformation process, when RST or LFNST is applied to the 8×8 area, the matrix operation is applied to the 48 transform coefficients in the upper-left, upper-right and lower-left regions of the 8×8 area, excluding the lower-right area among the transform coefficients in an 8x8 region and a 16x48 matrix of transform kernels. For matrix operation, 48 transformation coefficients are entered in a one-dimensional array. When the matrix operation is performed, 16 modified transform coefficients are output, and the modified transform coefficients can be arranged in the top-left region in the 8×8 region.

[170] In contrast, in the inverse transform process, when the inverse RST or LFNST is applied to the 8×8 region, 16 transform coefficients corresponding to the top-left region in the 8×8 region among the transform coefficients in the 8×8 region can be input in a one-dimensional array according to the scanning order and can undergo matrix operation with a 48x16 matrix transformation kernel. That is, the matrix operation can be expressed as (48×16 matrix)*(16×1 vector of transform coefficients)=(48×1 vector of modified transform coefficients). Here, an n×1 vector can be interpreted to have the same value as an n×1 matrix, and can thus be expressed as an n×1 column vector. Also, * means matrix multiplication. When the matrix operation is performed, 48 modified transform coefficients can be output, and the 48 modified transform coefficients can be arranged in the top-left, top-right, and bottom-left regions in the 8×8 area excluding the bottom-right region.

[171] When the secondary inverse transform is based on RST, the inverse transformer 235 of the encoding device 200 and the inverse transformer 322 of the decoding device 300 may include an inverse shortened secondary transformer for deriving modified transform coefficients based on the inverse RST on the transform coefficients and an inverse primary transformer for deriving residual samples for the target block based on the inverse primary transform on the modified transform coefficients. The inverse of the primary transformation refers to the inverse of the primary transformation applied to the remainder. In the present disclosure, deriving a transformation coefficient based on a transformation may refer to deriving a transformation coefficient by applying a transformation.

[172] The above-described non-partitioned transform, LFNST, will be described in detail as follows. The LFNST may include a forward transform by an encoding device and a reverse transform by a decoding device.

[173] The encoder takes the result (or part of the result) output after applying the primary transform as input, and applies the forward secondary transform (secondary transform).

[174] [Equation 9]

[175] In Equation 9, x and y are the inputs and outputs of the secondary transform, respectively, and G is a matrix representing the secondary transform, and the transform basis vectors are formed by column vectors. In the case of inverse LFNST, where the dimension of the transformation matrix G is expressed as [number of rows × number of columns], in the case of forward LFNST, the transposition of the matrix G becomes the dimension G ^T .

[176] For the inverse LFNST, the dimensions of matrix G are [48×16], [48×8], [16×16], [16×8], and matrix [48×8] and matrix [16×8] are partial matrices obtained by discretizing the 8 basis vectors of the left transformation of the [48×16] matrix and the [16×16] matrix, respectively.

[177] On the other hand, for direct LFNST, the dimensions of matrix G ^T are [16×48], [8×48], [16×16], [8×16], and matrix [8×48] and matrix [ 8×16] are partial matrices obtained by discretizing 8 basis vectors of the transformation on top of the [16×48] matrix and the [16×16] matrix, respectively.

[178] Therefore, in the case of direct LFNST, the [48×1] vector or the [16×1] vector is possible as the x input, and the [16×1] vector or the [8×1] vector is possible as the y output. In video encoding and decoding, the output of the direct primary transform is two-dimensional (2D) data, so to generate a [48×1] vector or a [16×1] vector as input x, a 1D vector must be formed by properly arranging the 2D data, which is direct conversion output.

[179] FIG. 8 is a diagram illustrating a layout sequence of direct primary transform output data into a one-dimensional vector according to an example. Left diagrams (a) and (b) in Fig. 8 shows the sequence for generating the [48×1] vector, and the right diagrams (a) and (b) in FIG. 8 shows the sequence for generating the [16×1] vector. In the case of LFNST, a 1D vector x can be obtained by sequentially arranging the 2D data in the same order as in (a) and (b) in FIG. 8.

[180] The layout direction of the forward primary transform output data can be determined in accordance with the intra-prediction mode of the current block. For example, when the intra prediction mode of the current block is in the horizontal direction with respect to the diagonal direction, the forward primary transform output data can be arranged in the order of FIG. 8(a), and when the intra prediction mode of the current block is in the vertical direction with respect to the diagonal direction, the forward primary transform output data can be arranged in the order of FIG. 8(b).

[181] According to the example, a different arrangement order from the arrangement order in FIG. 8(a) and (b), and to obtain the same result (vector y) as obtained by applying the layout orders in FIG. 8(a) and (b), the column vectors of matrix G can be rearranged according to the layout order. That is, you can rearrange the column vectors of G so that each element composing the vector x is always multiplied by the same basis vector of the transformation.

[182] Since the output y obtained by Equation 9 is a one-dimensional vector, when two-dimensional data is required as input in the process of using the direct secondary transform result as an input, such as in the process of performing quantization or remainder encoding, the output vector y in Equation 9 must be properly reassembled as 2D data.

[183] FIG. 9 is a diagram illustrating a layout sequence of the output of the direct secondary transform into a two-dimensional vector according to an example.

[184] In the case of LFNST, the output values can be arranged into a 2D block according to a predefined scan order. Fig. 9(a) shows that when the output y is a [16×1] vector, the output values are arranged at 16 positions of the 2D block according to the diagonal scan order. Fig. 9(b) shows that when the output y is an [8×1] vector, the output values are arranged in 8 positions of the 2D block according to the diagonal scan order, and the remaining 8 positions are filled with zeros. X in fig. 9(b) indicates that this position is filled with zero.

[185] According to another example, since the order in which the output vector y is processed when performing quantization or remainder encoding may be preset, the output vector y may not be arranged into a 2D block as shown in FIG. 9. However, in the case of remainder encoding, encoding of the data may be done in 2D block units (eg 4x4) such as CG (coefficient group), in which case, the data is arranged according to a specific order, such as diagonal order scanning in Fig. 9.

[186] Meanwhile, the decoding apparatus can configure the one-dimensional input vector y by arranging the two-dimensional data output by a dequantization process or the like according to a predetermined scanning order for inverse transformation. The input vector y can be output as the output vector x through the following equation.

[187] [Equation 10]

x=Gy

[188] In the case of an inverse LFNST, the output vector x may be output by multiplying the input vector y, which is a [16×1] vector or an [8×1] vector, by a matrix G. For an inverse LFNST, the output vector x may be either a [48×1] vector or a [16×1] vector.

[189] The output vector x is arranged into a two-dimensional block according to the order shown in FIG. 8, and is assembled as two-dimensional data, and this two-dimensional data becomes the input data (or part of the input data) of the inverse primary transform.

[190] Accordingly, the inverse secondary transformation is generally the opposite process of the direct secondary transformation, and in the case of the inverse transformation, unlike the forward direction, the inverse secondary transformation is first applied and then the inverse primary transformation is applied.

[191] In the inverse LFNST, one of 8 matrices [48×16] and 8 matrices [16×16] can be selected as the transformation matrix G. Whether to use a [48x16] matrix or a [16x16] matrix depends on the size and shape of the block.

[192] In addition, 8 matrices can be derived from four sets of transformations, as shown in Table 2 above, and each set of transformations can consist of two matrices. Which transform set to use among the 4 transform sets is determined according to the intra-prediction mode, and more specifically, the transform set is determined based on the value of the wide-angle intra-prediction (WAIP) extended intra-prediction mode. Which matrix to choose from the two matrices that make up the selected set of transformations is inferred using index signaling. More specifically, 0, 1, and 2 are possible as the passed index value, 0 may indicate that LFNST is not applied, and 1 and 2 may indicate any one of the two transformation matrices constituting the transformation set selected based on the intra-prediction mode value.

[193] FIG. 10 is a diagram illustrating wide-angle intra-prediction modes in accordance with an embodiment of the present document.

[194] A normal intra-prediction mode value may have values from 0 to 66 and 81 to 83, and an intra-prediction mode value expanded due to WAIP may have a value from -14 to 83, as shown. Values from 81 to 83 indicate the CCLM mode (Cross Component Linear Model), and values from -14 to -1 and values from 67 to 80 indicate the intra-prediction mode extended due to the application of WAIP.

[195] When the width of the current prediction block is greater than the height, the top reference pixels are typically closer to positions within the prediction block. Therefore, it would be more accurate to predict in the lower-left direction than in the upper-right direction. In contrast, when a block's height is greater than its width, the left reference pixels tend to be closer to positions inside the predicted block. Therefore, it may be more accurate to predict in the top-right direction than in the bottom-left direction. Therefore, it may be preferable to apply remapping, i.e. modification of the mode index to the wide-angle intra-prediction mode index.

[196] When wide-angle intra-prediction is applied, information about the existing intra-prediction may be signaled, and after this information is obtained through parsing, this information may be re-mapped to the wide-angle intra-prediction mode index. Therefore, the total number of intra-prediction modes for a particular block (eg, a non-square block of a particular size) may not change, and the total number of intra-prediction modes is 67, and the encoding of the intra-prediction mode for a specific block may not change.

[197] Table 3 below shows the process of deriving the modified intra-mode by re-mapping the intra-prediction mode to the wide-angle intra-prediction mode.

[198] [Table 3]

The inputs to this process are:
- variable predModeIntra, specifying the intra-prediction mode,
- variable nTbW, specifying the width of the conversion block,
- variable nTbH, specifying the height of the transformation block,
- variable cIdx, specifying the color component of the current block.
The output of this process is a modified intra-prediction mode predModeIntra.
The variables nW and nH are output as follows:
- If IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT or cIdx is not equal to 0, then the following applies:
nW=nTbW (8-97)
nH=nTbH (8-98)
- Otherwise (IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT, and cIdx is equal to 0), the following applies:
nW=nCbW (8-99)
nH=nCbH (8-100)
The whRatio variable is set to Abs(Log2(nW/nH)).
For non-square blocks (nW is not equal to nH), the intra-prediction mode predModeIntra is modified as follows:
- If all of the following conditions are true, then predModeIntra is set to (predModeIntra+65),
-- nW is greater than nH
-- predModeIntra is greater than or equal to 2
-- predModeIntra is less than (whRatio > 1)?(8+2* whRatio) : 8
- Otherwise, if all of the following conditions are true, then predModeIntra is set to (predModeIntra - 67),
-- nH is greater than nW
-- predModeIntra is less than or equal to 66
-- predModeIntra is greater than (whRatio > 1)?(60-2* whRatio): 60

[199] In Table 3, the value of the extended intra-prediction mode is finally stored in the variable predModeIntra, and ISP_NO_SPLIT indicates that the CU is not subdivided by the intra-subdivision (ISP) method currently adopted in the VVC standard, and the value is 0. Variables 1 and 2 cIdx indicate the case of brightness components, Cb and Cr, respectively. The Log2 function, shown in Table 3, returns the log value in base 2, and the Abs function returns the absolute value.

[200] The predModeIntra variable indicating the intra prediction mode, and the height and width of the transform block, etc. are used as input values of the wide-angle intra-prediction mode display process, and the output value is the modified intra-prediction mode predModeIntra. The height and width of the transform block or encoding block may be the height and width of the current block to re-display the intra-prediction mode. In this case, the whRatio variable, reflecting the ratio of width to height, can be set to Abs(Log2(nW/nH)).

[201] For a non-square block, the intra-prediction mode can be divided into two cases and modified.

[202] First, if all conditions from (1) to (3) are satisfied, (1) the width of the current block is greater than the height, (2) the intra-prediction mode before modification is equal to or greater than 2, (3) intra prediction mode is less than the value derived from (8+2*whRatio) when whRatio is greater than 1, and less than 8 when whRatio is less than or equal to 1 [predModeIntra is less than (whRatio>1) ?(8+2*whRatio):8], intra prediction mode is set to 65 greater than intra prediction mode [predModeIntra is set to (predModeIntra+65)].

[203] If, contrary to the above, that is, the following conditions from (1) to (3) are satisfied: (1) the height of the current block is greater than the width, (2) the intra-prediction mode before modification is less than or equal to 66, ( 3) intra prediction mode is greater than the value derived from (60-2*whRatio) when whRatio is greater than 1, and greater than 60 when whRatio is less than or equal to 1 [predModeIntra is greater than (whRatio> 1) ? (60-2*whRatio):60], intra prediction mode is set to 67 less than intra prediction mode [predModeIntra is set to (predModeIntra-67)].

[204] Table 2 above shows how a set of transforms is selected based on the intra-prediction mode value extended by WAIP in LFNST. As shown in FIG. 10, modes 14 to 33 and modes 35 to 80 are symmetrical with respect to the prediction direction around mode 34. For example, mode 14 and mode 54 are symmetrical with respect to the direction corresponding to mode 34. Therefore, the same set of transformations applies to modes located in mutually symmetrical directions, and this symmetry is also reflected in Table 2.

[205] Meanwhile, it is assumed that the forward LFNST input data for mode 54 is symmetrical with the forward LFNST input data for mode 14. For example, for mode 14 and mode 54, two-dimensional data is reordered into one-dimensional data according to the arrangement order shown in FIG. . 8(a) and FIG. 8(b), respectively. Moreover, it can be seen that the patterns are in the order shown in FIG. 8(a) and FIG. 8(b) are symmetrical with respect to the direction (diagonal direction) indicated by mode 34.

[206] Meanwhile, as described above, which transformation matrix from the [48×16] matrix and the [16×16] matrix is applied to the LFNST is determined by the size and shape of the transformation target block.

[207] FIG. 11 is a diagram illustrating a block shape to which LFNST is applied. Fig. 11(a) shows 4x4 blocks, (b) shows 4x8 and 8x4 blocks, (c) shows 4xN blocks or Nx4 blocks in which N is 16 or more, (d) shows blocks 8x8, (e) shows MxN blocks where M≥8, N≥8 and N>8 or M>8.

[208] In FIG. 11, boxes with bold borders indicate the regions to which LFNST is applied. For blocks according to Fig. 11(a) and (b), LFNST is applied to the top-left 4x4 region, and for the block according to FIG. 11(c), LFNST is applied individually to two top-left 4x4 regions that are adjacent. In fig. 11(a), (b) and (c), since the LFNST is applied in units of 4x4 areas, this LFNST will be hereinafter referred to as "4x4 LFNST". As the corresponding transformation matrix, [16×16] matrix or [16×8] matrix can be applied based on the dimension of matrix G in Equations 9 and 10.

[209] More specifically, the [16×8] matrix is applied to a 4×4 unit (TU 4×4 or CU 4×4) according to FIG. 11(a), and the [16×16] matrix is applied to the blocks in FIG. 11(b) and (c). This is intended to regulate the worst case computational complexity to 8 multiplications per sample.

[210] Referring to FIG. 11(d) and (e), LFNST is applied to the top-left 8×8 region, and this LFNST is hereinafter referred to as “8×8 LFNST”. The corresponding transformation matrix can be a [48×16] matrix or a [48×8] matrix. In the case of forward LFNST, since the [48×1] vector (vector x in Equation 9) is input as input, the values of all samples of the top-left 8×8 region are not used as input values of forward LFNST. That is, as can be seen in the left layout in FIG. 8(a) or the left layout in FIG. 8(b), the [48x1] vector can be generated based on the samples belonging to the remaining 3 4x4 blocks, leaving the bottom right 4x4 block as is.

[211] The [48×8] matrix may be applied to an 8×8 unit (TU 8×8 or CU 8×8) in FIG. 11(d), and the [48×16] matrix can be applied to the 8×8 block in FIG. 11(e). This is also intended to regulate the worst case computational complexity to 8 multiplications per sample.

[212] Depending on the block shape, when the corresponding forward LFNST (4×4 LFNST or 8×8 LFNST) is applied, 8 or 16 outputs are generated (the y vector in Equation 9, the [8×1] vector or [16×1 ]). In direct LFNST, the number of outputs is equal to or less than the number of inputs due to the characteristics of the matrix G ^T .

[213] FIG. 12 is a diagram illustrating an arrangement of forward LFNST output data according to an example, and shows a block in which forward LFNST output data is arranged according to a block shape.

[214] The shaded area of the top-left block shown in FIG. 12 corresponds to the region where the forward LFNST output data is located, the positions labeled 0 indicate the samples filled with 0 values, and the remaining region represents the regions not modified by the forward LFNST. In a region not modified by LFNST, the forward primary transform output remains unchanged.

[215] As described above, since the dimension of the applied transformation matrix varies according to the block shape, the number of output data also varies. As shown in FIG. 12, the forward LFNST output may not completely fill the top-left 4x4 block. In the case of FIG. 12(a) and (d), the [16×8] matrix and the [48×8] matrix are applied to the block indicated by the thick line or the partial region within the block, respectively, and the [8×1] vector is generated as the output of the forward LFNST. That is, according to the scanning order shown in FIG. 9(b), only 8 output data can be filled, as shown in FIG. 12(a) and (d), and 0 can be filled in the remaining 8 positions. In the case of the applied LFNST block according to FIG. 11(d), as shown in FIG. 12(d), the two 4x4 blocks at the top-right and bottom-left of the top-left 4x4 block are also filled with 0 values.

[216] As described above, basically, by signaling the LFNST index, whether LFNST is to be applied and the transformation matrix to be applied is specified. As shown in FIG. 12, when LFNST is applied, since the number of output data of forward LFNST may be equal to or less than the number of input data, an area filled with zero values occurs as follows.

[217] 1) As shown in FIG. 12(a), samples from the 8th and subsequent positions in the scanning order in the top-left 4x4 block, that is, samples from the 9th to the 16th.

[218] 2) As shown in FIG. 12(d) and (e), when a [16×48] matrix or an [8×48] matrix is applied, two 4×4 blocks adjacent to the top-left 4×4 block, or the second and third 4×4 blocks in scanning order.

[219] Therefore, if non-zero data exists when checking areas 1) and 2), then it is clear that LFNST does not apply, so the corresponding LFNST index signaling can be omitted.

[220] According to an example, for example, in the case of LFNST adopted in the VVC standard, since LFNST index signaling is performed after encoding the remainder, the encoder can know whether there are non-zero data (significant coefficients) for all positions in the TU or CU by coding the remainder. Accordingly, the encoding apparatus may determine whether to perform signaling on the LFNST index based on the existence of non-zero data, and the decoding apparatus may determine whether the LFNST index is analyzed. When non-zero data does not exist in the area specified in 1) and 2 above), LFNST index signaling is performed.

[221] Meanwhile, for the adopted LFNST, the following simplification methods can be applied.

[222] (i) According to the example, the number of outputs for direct LFNST can be limited to a maximum of 16.

[223] In the case of FIG. 11(c), 4×4 LFNST can be applied to two 4×4 regions adjacent top-left, respectively, in which case a maximum of 32 LFNST outputs can be generated. When the number of output data for direct LFNST is limited to a maximum of 16, in case of 4xN/Nx4 (N≥16) blocks (TU or CU), 4x4 LFNST is applied to only one 4x4 area from top-left, LFNST can only be applied once to all blocks in FIG. 11. Due to this, the implementation of image encoding can be simplified.

[224] (ii) According to the example, nulling may additionally be applied to an area to which LFNST does not apply. In this document, nulling may mean filling the values of all positions belonging to a particular region with the value 0. That is, nulling may be applied to a region that is not changed by LFNST and retains the result of the direct primary transformation. As described above, since the LFNST is divided into 4×4 LFNST and 8×8 LFNST, the nulling can be divided into two types ((ii)-(A) and (ii)-(B)) as follows.

[225] (ii)-(A) When 4x4 LFNST is applied, the area to which 4x4 LFNST is not applied can be set to zero. Fig. 13 is a diagram illustrating nulling in a block to which a 4x4 LFNST is applied, according to an example.

[226] As shown in FIG. 13, with respect to the block to which the 4x4 LFNST is applied, that is, for all blocks in FIG. 12(a), (b) and (c), the entire region to which LFNST does not apply can be filled with zeros.

[227] On the other hand, FIG. 13(d) shows that when the maximum value of the number of forward LFNST outputs is limited to 16, zeroing is performed on the remaining blocks to which 4x4 LFNST is not applied, according to the example.

[228] (ii)-(B) When 8×8 LFNST is applied, the area to which 8×8 LFNST is not applied may be set to zero. Fig. 14 is a diagram illustrating nulling in a block to which an 8x8 LFNST is applied, according to an example.

[229] As shown in FIG. 14, with respect to the block to which the 8x8 LFNST is applied, that is, for all blocks in FIG. 12(d) and (e), the entire region to which LFNST does not apply may be filled with zeros.

[230] (iii) Due to the nulling presented in (ii) above, the area filled with zeros may not be the same when LFNST is applied. Accordingly, it is possible to check whether non-zero data exists, in accordance with the nulling proposed in (ii), over a wider area than in the case of LFNST according to FIG. 12.

[231] For example, when (ii)-(B) is applied, after checking whether non-zero data exists, where the region is filled with zero values in FIG. 12(d) and (e) in addition to the area filled with 0 additionally in FIG. 14, signaling for the LFNST index can only be performed if non-zero data does not exist.

[232] Of course, even if the nulling proposed in (ii) is applied, it is possible to check whether non-zero data exists in the same way as the existing LFNST index signaling. That is, after checking whether non-zero data exists in the block filled with zeros in FIG. 12, LFNST index signaling may be applied. In this case, only the encoding device performs zeroing, and the decoding device does not assume zeroing, that is, by only checking whether non-zero data exists only in the region explicitly marked as 0 in FIG. 12 can parse the LFNST index.

[233] Various embodiments can be obtained that employ combinations of simplification methods ((i), (ii)-(A), (ii)-(B), (iii)) for LFNST. Of course, combinations of the above simplification methods are not limited to the following embodiments, and any combination may be applied to LFNST.

[234] Embodiment

[235] - Limit the number of outputs for direct LFNST to a maximum of 16 → (i)

[236] - When 4x4 LFNST is applied, all areas to which 4x4 LFNST is not applied are reset to zero → (ii)-(A)

[237] - When 8x8 LFNST is applied, all areas to which 8x8 LFNST is not applied are reset to zero → (ii)-(B)

[238] - After checking whether there are non-zero data, existing areas filled with zero values, and areas filled with zeros due to additional zeros ((ii)-(A), (ii)-(B)), the LFNST index is signaled, only when non-null data does not exist → (iii)

[239] In the case of an embodiment where LFNST is applied, the region in which non-zero output data can exist is limited to the interior of the top-left 4x4 region. In more detail, in the case of FIG. 13(a) and FIG. 14(a), the 8th position in the scan order is the last position where non-zero data can exist. In the case of FIG. 13(b) and (c) and FIG. 14(b), the 16th position in the scanning order (i.e., the position of the bottom-right edge of the top-left 4x4 block) is the last position where data other than 0 can exist.

[240] Therefore, when LFNST is applied, after checking whether non-zero data exists at a position where the remainder encoding process is not allowed (at a position beyond the last position), it can be determined whether the LFNST index is signaled.

[241] In the case of the nulling method proposed in (ii), since the number of data generated finally when both the primary transformation and LFNST are applied, the amount of calculation required to perform the entire transformation process can be reduced. That is, when LFNST is applied, since nulling is applied to the output of the forward primary transform existing in an area to which LFNST is not applied, there is no need to generate data for the area that becomes null when performing the forward primary transform. Accordingly, it is possible to reduce the amount of computation required to generate the corresponding data. The additional effects of the nulling method proposed in (ii) can be summarized as follows.

[242] First, as described above, the amount of computation required to perform the entire conversion process is reduced.

[243] In particular, when (ii)-(B) is applied, the amount of calculation for the worst case is reduced so that the conversion process can be made easier. In other words, a large amount of computation is usually required to perform a large-scale primary transformation. By applying (ii)-(B), the number of data output as a result of executing direct LFNST can be reduced to 16 or less. In addition, since the size of the entire block (TU or CU) is increased, the effect of reducing the size of the conversion operation is further increased.

[244] Secondly, the amount of computation required for the entire conversion process can be reduced, thereby reducing the power consumption required to perform the conversion.

[245] Third, the delay associated with the conversion process is reduced.

[246] A secondary transform, such as LFNST, adds computational overhead to the existing primary transform, thereby increasing the overall latency involved in performing the transform. Particularly in the case of intra-prediction, since the reconstructed data of neighboring blocks is used in the prediction process during encoding, the increase in latency due to the secondary transformation results in an increase in the latency before reconstruction. This may lead to an increase in the overall latency of intra-prediction encoding.

[247] However, if the nulling proposed in (ii) is applied, the delay time for performing the primary transformation can be significantly reduced; when LFNST is applied, the delay time for the entire transformation is maintained or reduced, so that the encoding device can be implemented more simply.

[248] Meanwhile, in conventional intra-prediction, the target encoding block is considered as one encoding unit, and encoding is performed without dividing it. However, ISP (intra-subdivision) encoding refers to performing intra-prediction encoding with the encoding target block divided in the horizontal direction or vertical direction. In this case, a reconstructed block can be generated by performing encoding/decoding in units of divided blocks, and the reconstructed block can be used as a reference block of the next divided block. According to an example, in ISP encoding, one encoding block may be divided into two or four sub-blocks and encoded, and in ISP, intra-prediction is performed on one sub-block with reference to the reconstructed pixel value of a sub-block located adjacent to the left or above it. Further, the term "encoding" can be used as a concept that includes both encoding performed by the encoding device and decoding performed by the decoding device.

[249] The ISP splits a block predicted as intra-luminance into two or four subdivisions in the vertical direction or horizontal direction according to the block size. For example, the minimum block size that ISP can apply to is 4x8 or 8x4. If the block size is larger than 4x8 or 8x4, the block is split into four subdivisions.

[250] When ISP is applied, sub-blocks are sequentially encoded according to the partition type, such as horizontal or vertical, left-to-right or top-to-bottom, and encoding for the next sub-block can be performed after performing the reconstruction process through inverse transform and intra-prediction for one sub-block. For the leftmost or topmost subblock, as in a conventional intra-prediction method, a reconstructed pixel of an encoding block that has already been encoded is referenced. Additionally, if the previous sub-block is not adjacent to each side of the inner sub-block that follows it, to output reference pixels adjacent to the corresponding side, as in a conventional intra-prediction method, reference is made to the reconstructed pixel of the already encoded adjacent encoding block.

[251] In the ISP encoding method, all subblocks may be encoded with the same intra-prediction mode, and a flag indicating whether or not ISP encoding should be used, and a flag indicating in which direction (horizontal or vertical) should be performed splitting may be signaled. At this time, the number of sub-blocks may be adjusted to 2 or 4 depending on the block shape, and when the size (width × height) of one sub-block is less than 16, partitioning cannot be allowed for the corresponding sub-blocks, and the application of ISP coding itself may be limited .

[252] Meanwhile, in the case of the ISP prediction mode, one coding unit is divided into two or four partitioning blocks, that is, sub-blocks, and predicted, and the same intra-prediction mode is applied to the two or four partitioning blocks thus divided.

[253] As described above, as the horizontal direction (if an M×N encoding unit having a horizontal length and a vertical length of M and N, respectively, is divided in the horizontal direction, it is divided into M×(N/2) blocks when divided by two and into M×(N/4) blocks when divided by four) and the vertical direction (if an M×N coding unit is divided in the vertical direction, it is divided into (M/2)×N blocks when divided by two and divided by blocks (M/4)×N when divided by four) are possible as partitioning directions. When partitioning in the horizontal direction, the partition blocks are encoded in order from top to bottom, and when partitioning in the vertical direction, the partition blocks are encoded in order from left to right. The current split block to be encoded may be predicted with reference to the reconstructed pixel values of the top (left) split block in the case of splitting in the horizontal (vertical) direction.

[254] The transformation may be applied to the residual signal generated by the ISP prediction method in units of partitioning blocks. MTS (Multiple Transform Selection) technology, based on the combination of DST-7/DCT-8 as well as the existing DCT-2, can be applied to primary transform (main transform or primary transform) based on forward direction, and LFNST (low frequency inseparable transform) can be applied to the transform coefficient generated in accordance with the primary transform to generate the final modified transform coefficient.

[255] That is, LFNST can also be applied to partition blocks divided by applying the ISP transform mode, and the same intra-prediction mode is applied to the divided partition blocks as described above. Accordingly, when selecting a set of LFNSTs derived based on the intra-prediction mode, the derived set of LFNSTs can be applied to all partitioning blocks. That is, the same intra-prediction mode is applied to all partition blocks, and the same set of LFNST can be applied to all partition blocks.

[256] Meanwhile, according to the example, LFNST can only be applied to transform blocks having horizontal and vertical lengths of 4 or more. Therefore, when the horizontal or vertical length of the partition block divided according to the ISP prediction method is less than 4, LFNST is not applied and the LFNST index is not signaled. Accordingly, when LFNST is applied to each partition block, the corresponding partition block can be considered as one transform block. Of course, when the ISP prediction method is not applied, LFNST may be applied to the coding block.

[257] The application of LFNST to each partition block is described in detail as follows.

[258] According to the example, after applying direct LFNST to an individual partitioning block and after only 16 coefficients (8 or 16) remain in the top-left 4x4 region according to the scanning order of the transform coefficients, zero padding of all the remaining positions and areas with a value of 0.

[259] Alternatively, according to the example, when the length of one side of the partition block is 4, LFNST is applied only to the top-left 4x4 region, and when the length of all sides of the partition block, that is, the width and height, are 8 or more ,LFNST can be applied to the remaining 48 coefficients except the 4x4 bottom-right region in the 8x8 top-left region.

[260] Alternatively, according to the example, to adjust the worst-case computational complexity to 8 multiplications per sample, when each partition block is 4x4 or 8x8, only 8 transform coefficients can be derived after applying forward LFNST. That is, if the partition block is 4x4, an 8x16 matrix can be used as a transformation matrix, and if the partition block is 8x8, an 8x48 matrix can be used as a transformation matrix.

[261] Meanwhile, in the current VVC standard, LFNST index signaling is performed in coding units. Accordingly, when the ISP prediction mode is used and LFNST is applied to all partition blocks, the same LFNST index value can be applied to the corresponding partition blocks. That is, when a LFNST index value is transmitted once at the coding unit level, the corresponding LFNST index can be applied to all partitioning blocks in the coding unit. As described above, the LFNST index value can have the values 0, 1, and 2; 0 indicates the case in which LFNST is not applied, 1 and 2 indicate two transformation matrices present in the same LFNST set when LFNST is applied.

[262] As described above, the set of LFNST is determined by the intra-prediction mode, and since all partitioning blocks in a coding unit are predicted in the same intra-prediction mode in the case of the ISP prediction mode, the partitioning blocks can refer to the same set of LFNST.

[263] As another example, LFNST index signaling is still performed in coding units, but in the case of ISP prediction mode, without determining whether or not LFNST should be applied uniformly to all partitioning blocks, then whether or not the LFNST index value should be applied signaled at the coding unit level, whether or not to apply LFNST to each partitioning block may be determined by a separate condition. Here, a separate condition may be signaled in the form of a flag for each partitioning block by the bitstream, and when the flag value is 1, the index value LFNST signaled at the coding unit level may be applied, and when the flag value is 0, LFNST may not be applied.

[264] Next, a method for maintaining worst-case computational complexity when LFNST is applied to ISP mode will be described.

[265] In the case of ISP mode, in order to maintain the number of multiplications per sample (or per coefficient or per position) at a certain level or less when LFNST is applied, the application of LFNST may be limited. Depending on the size of the partitioning block, the number of multiplications per sample (or per coefficient or per position) can be kept to 8 or less by applying LFNST as follows.

[266] 1. When both the horizontal length and the vertical length of a partition block are equal to or greater than 4, the same method as the worst-case computational complexity adjustment method for LFNST in the current VVC standard can be applied.

[267] That is, when the partitioning block is a 4x4 block, instead of a 16x16 matrix, in the forward direction, an 8x16 matrix obtained by sampling the top 8 rows from a 16x16 matrix can be used, and in the reverse direction, a matrix can be applied 16x8, obtained by sampling the left 8 columns from a 16x16 matrix. Additionally, when the partitioning block is an 8x8 block, in the forward direction, instead of a 16x48 matrix, an 8x48 matrix obtained by sampling the top 8 rows of a 16x48 matrix can be used, and in the reverse direction, instead of a 48x16 matrix , a 48×8 matrix can be used, obtained by sampling the left 8 columns from a 48×16 matrix.

[268] In the case of a 4×N or N×4 block (N>4), when direct transformation is performed, the 16 coefficients generated after applying the 16×16 matrix to only the top-left 4×4 block are arranged in the top-left region 4x4, and other areas can be filled with zero values. Additionally, when performing the inverse transform, the 16 coefficients located in the top-left 4x4 block can be arranged in scan order to configure the input vector, and then the 16 outputs can be generated by multiplying with the 16x16 matrix. The generated output data can be ordered in the top-left 4x4 region, and the remaining regions except the top-left 4x4 region can be filled with zeros.

[269] In the case of an 8×N or N×8 (N>8) block, when direct transformation is performed, 16 coefficients generated after applying the 16×48 matrix only to the ROI region in the top-left 8×8 block (the remaining regions, excluding the bottom-right 4x4 block from the top-left 8x8 block) can be ordered into the top-left 4x4 region, and other regions can be filled with null values. Additionally, when performing the inverse transform, the 16 coefficients located in the top-left 4x4 block can be arranged in scan order to configure the input vector, and then the 48 outputs can be generated by multiplying with the 48x16 matrix. The generated output may fill the ROI region, and other regions may be filled with null values.

[270] As another example, to maintain the number of multiplications per sample (or per coefficient or per position) at some value or less, the number of multiplications per sample (or per coefficient or per position) is based on the ISP encoding unit size rather than ISP partitioning block size can be supported at 8 or less. If there is only one block among the ISP partitioning blocks that satisfies the condition under which LFNST is applied, the LFNST worst-case complexity calculation may be applied based on the corresponding coding unit size rather than the partitioning block size. For example, when a luminance coding block for a certain coding unit is divided into 4 partition blocks of size 4x4 and encoded by the ISP, and when a non-zero transform coefficient exists for two partition blocks among them, the other two partition blocks can be set accordingly to generate 16 coefficients conversions instead of 8 (encoder based).

[271] Next, a method for signaling the LFNST index in the case of the ISP mode will be described.

[272] As described above, the LFNST index can have the values 0, 1, and 2, where 0 indicates that LFNST is not applied, and 1 and 2, respectively, indicate any one of the two LFNST core matrices included in the selected LFNST set. LFNST is applied based on the LFNST kernel matrix selected by the LFNST index. The transmission method of the LFNST index in the current VVC standard will be described as follows.

[273] 1. The LFNST index may be transmitted once for each coding unit (CU), and in the case of a dual tree, individual LFNST indexes may be signaled for a luma block and a chrominance block, respectively.

[274] 2. When the LFNST index is not signaled, the LFNST index value is assumed to be the default value 0. The case where the LFNST index value is assumed to be 0 is as follows.

[275] A. In case of a mode in which no conversion is applied (for example, skip conversion, BDPCM, lossless encoding, etc.)

[276] B. When the primary transform is not DCT-2 (DST7 or DCT8), that is, when the horizontal transform or vertical transform is not DCT-2

[277] C. When the horizontal length or vertical length for the luminance block of a coding unit exceeds the size of the converted maximum luminance transform, for example, when the size of the converted maximum luminance transform is 64, and when the size for the luminance block of the coding unit is 128×16, LFNST cannot apply.

[278] In the case of a dual tree, it is determined whether or not the size of the maximum luma transform for each of the luma component coding unit and the chrominance component coding unit is exceeded. That is, for a luma block, it is checked whether the size of the maximum luma transform transform is exceeded, and for a chroma block, it is checked whether the horizontal/vertical length of the corresponding luma block for the color format and the size of the transformable maximum luma transform are exceeded. For example, when the color format is 4:2:0, the horizontal/vertical length of the corresponding lumina block is equal to twice the length of the corresponding chroma block, and the transform size of the corresponding lumina block is equal to twice the size of the corresponding chroma block. As another example, when the color format is 4:4:4, the horizontal/vertical length and transform size of the corresponding luminance block are the same as the dimensions of the corresponding chrominance block.

[279] A transform of length 64 or a transform of length 32 may mean a transform applied to a width or height having a length of 64 or 32, respectively, and "transform size" may mean 64 or 32 as the corresponding length.

[280] In the case of a single tree, after checking whether the horizontal length or the vertical length of the luma block exceeds the maximum size of the luma transform block being converted, if it exceeds it, then the LFNST index signaling may be omitted.

[281] D. The LFNST index can be transmitted only when the horizontal length and vertical length of the coding unit are equal to or greater than 4.

[282] In the case of a dual tree, the LFNST index may be signaled only when both the horizontal and vertical lengths for the corresponding component (i.e., luma or chrominance component) are equal to or greater than 4.

[283] In the case of a single tree, the LFNST index may be signaled when both the horizontal and vertical lengths for the luma component are equal to or greater than 4.

[284] E. If the position of the last non-zero coefficient is not a DC position (top-left block position) and if the position of the last non-zero coefficient is not a DC position, in the case of a dual-type tree luma block, then the index LFNST is transmitted. In the case of a dual-type tree chromaticity block, if any one of the last non-zero coefficient position for Cb and the last non-zero coefficient position for Cr is not a DC position, the corresponding LNFST index is transmitted.

[285] In the case of a single type tree, if the position of the last non-zero coefficient of any one of the luminance component, the Cb component and the Cr component is not a DC position, the index LFNST is transmitted.

[286] Here, if the value of the coded block flag (CBF) indicating whether or not a transform coefficient exists for one transform block is 0, the position of the last non-zero coefficient for the corresponding transform block is not checked to determine whether or not LFNST is signaled. That is, when the value of the corresponding CBF is 0, since the transformation is not applied to the corresponding block, the position of the last non-zero coefficient may not be taken into account when checking the condition for signaling the LFNST index.

[287] For example, 1) in the case of a dual type tree and a luma component, if the value of the corresponding CBF is 0, the LFNST index is not signaled, 2) in the case of a dual type tree and a chrominance component, if the CBF value for Cb is 0 and the CBF value for Cr is 1, then only the position of the last non-zero coefficient for Cr is checked, and the corresponding LFNST index is passed. 3) In the case of a single tree type, the position of the last non-zero coefficient is checked only for components having a CBF value of 1 for each of luma, Cb and Cr.

[288] F. When it is confirmed that a transform coefficient exists at a position other than a position where a transform coefficient LFNST may exist, the LFNST index alarm may be omitted. In the case of a 4x4 transform block and an 8x8 transform block, the transform coefficients LFNST may exist at eight positions from the DC position according to the scan order of transform coefficients in the VVC standard, and the remaining positions are filled with zeros. Additionally, when the 4x4 transform block and the 8x8 transform block are absent, the transform coefficients LFNST may exist at sixteen positions from the DC position according to the scan order of transform coefficients in the VVC standard, and the remaining positions are filled with zeros.

[289] Accordingly, if non-zero transform coefficients exist in a region that should be filled with a zero value after the remainder encoding is advanced, the LFNST index signaling may be omitted.

[290] Meanwhile, the ISP mode may also be applied to only the luma block, or may be applied to both the luma block and the chrominance block. As described above, when ISP prediction is applied, the corresponding coding unit may be divided into two or four partition blocks and predicted, and a transform may be applied to each of the partition blocks. Therefore, also when defining the condition for signaling the LFNST index in coding units, it is necessary to take into account the fact that LFNST may be applied to the corresponding partitioning blocks. In addition, when the ISP prediction mode is applied only to a specific component (eg, a luminance block), the LFNST index may be signaled in consideration of the fact that only that component is divided into partitioning blocks. The LFNST index signaling methods in ISP mode can be summarized as follows.

[291] 1. The LFNST index may be transmitted once for each coding unit (CU), and in the case of a dual tree, individual LFNST indexes may be signaled for a luma block and a chrominance block, respectively.

[292] 2. When the LFNST index is not signaled, the LFNST index value is considered to be the default value of 0. The case where the LFNST index value is considered to be 0 is as follows.

[293] A. In the case of a mode in which no transform is applied (for example, skip transform, BDPCM, lossless encoding, etc.)

[294] B. When the horizontal length or vertical length for the luminance block of a coding unit exceeds the size of the converted maximum luminance transform, for example, when the size of the converted maximum luminance transform is 64 and when the size for the luminance block of a coding unit is 128×16, LFNST cannot be applied. .

[295] Whether or not to signal the LFNST index may be determined based on the partitioning block size instead of the coding unit. That is, if the horizontal or vertical length of the partition block for the corresponding luminance block exceeds the size of the converted maximum luminance transform, the LFNST index signaling may be omitted and the LFNST index value may be considered 0.

[296] In the case of a dual tree, it is determined whether or not the maximum luma transform size is exceeded for each coding unit or splitter for the luma component and coding unit or splitter for the chrominance component. That is, if the horizontal or vertical lengths of a coding unit or splitter for a luminance component are compared to the size of the maximum luminance transform, respectively, and at least one of them is larger than the size of the maximum luminance transform, LFNST does not apply, and in the case of a coding unit or the chroma partition block, the horizontal/vertical length of the corresponding luminance block for the color format, and the size of the maximum luminance transform being converted are compared. For example, when the color format is 4:2:0, the horizontal/vertical length of the corresponding lumina block is equal to twice the length of the corresponding chroma block, and the transform size of the corresponding lumina block is equal to twice the size of the corresponding chroma block. As another example, when the color format is 4:4:4, the horizontal/vertical length and size of the corresponding luminance block are the same for the corresponding chrominance block.

[297] In the case of a single tree, after checking whether the horizontal length or the vertical length of the luminance block (coding unit or partitioning block) exceeds the size of the maximum luminance transform transform block, if it exceeds it, then the LFNST index signaling may be omitted.

[298] C. If the LFNST included in the current VVC standard is used, the LFNST index can only be transmitted when both the horizontal length and the vertical length of the splitter are equal to or greater than 4.

[299] If the LFNST for a 2xM (1xM) or Mx2 (Mx1) block is applied in addition to the LFNST included in the current VVC standard, the LFNST index can only be transmitted if the partition block size is equal to or greater than than a 2×M (1×M) or M×2 (M×1) block. Here, the expression “block P×Q is equal to or greater than block R×S” means that P≥R and Q≥S.

[300] As a result, the LFNST index can only be transmitted if the partitioning block is equal to or greater than the minimum size to which the LFNST applies. In the case of a dual tree, the LFNST index may be signaled only if the splitter for the luma or chrominance component is equal to or greater than the minimum size to which LFNST applies. In the case of a single tree, the LFNST index may be signaled only when the split block for the luma component is equal to or greater than the minimum size to which LFNST applies.

[301] In this document, the expression "M×N block is greater than or equal to K×L block" means that M is greater than or equal to K, and N is greater than or equal to L. The expression "M×N block is greater than K×L block" means that M is greater than or equal to K and N is greater than or equal to L, and that M is greater than K or N is greater than L. The expression "M×N block is less than or equal to K×L block" means that M is less than or is equal to K and N is less than or equal to L, while the expression "a block M×N is less than a block K×L" means that M is less than or equal to K and N is less than or equal to L, and that M is less than K, or N is less than L.

[302] D. If the position of the last non-zero coefficient is not a DC position (top-left block position) and if the position of the last non-zero coefficient is not a DC position in any one of all partition blocks. In the case of a dual-type tree luma block, the index LFNST is transmitted. In the case of a dual type tree and a chroma block, if at least one of the position of the last non-zero coefficient of all partitioning blocks for Cb (if the ISP mode is not applied to the chroma component, the number of partitioning blocks is treated as one) and the position of the last non-zero coefficient of all blocks partition for Cr (if the ISP mode is not applied to the chroma component, the number of partition blocks is considered to be one) is not a DC position, the corresponding LNFST index can be transmitted.

[303] In the case of a single type tree, if the position of the last non-zero coefficient of any one of all partitioning blocks for the luma component, the Cb component and the Cr component is not a DC position, the corresponding LFNST index shall be transmitted.

[304] Here, if the value of the coded block flag (CBF) indicating whether a transform coefficient exists for each partition block is 0, the position of the last non-zero coefficient for the corresponding partition block is not checked to determine whether the LFNST index is signaled or not. That is, when the corresponding CBF value is 0 because no transformation is applied to the corresponding block, the position of the last non-zero coefficient for the corresponding partition block is not taken into account when checking the condition for LFNST index signaling.

[305] For example, 1) in the case of a dual type tree and luma component, if the corresponding CBF value for each partitioner is 0, the partitioner is excluded when determining whether or not to signal the LFNST index, 2) in the case of a dual type tree and chrominance component If the CBF value of Cb is 0 and the CBF value of Cr is 1 for each partition block, only the position of the last non-zero coefficient for Cr is checked to determine whether or not to signal the LFNST index. 3) In the case of a single type tree, it is possible to determine whether or not to signal the LFNST index by checking the position of the last non-zero coefficient only for blocks having a CBF value of 1 for all luma component, Cb component, and Cr component partitioning blocks.

[306] In the case of the ISP mode, the image information may also be configured such that the position of the last non-zero coefficient is not checked, and an embodiment thereof is as follows.

[307] i. In the case of ISP mode, LFNST index signaling may be enabled without checking the position of the last non-zero coefficient for both the luma block and the chrominance block. That is, even if the position of the last non-zero coefficient for all partitioning blocks is a DC position or the corresponding CBF value is 0, LFNST index signaling can be enabled.

[308] ii. In the case of ISP mode, checking the position of the last non-zero coefficient only for the luma block can be omitted, and in the case of the chrominance block, checking the position of the last non-zero coefficient can be performed in the manner described above. For example, in the case of a dual type tree and a luminance block, signaling the LFNST index is enabled without checking the position of the last non-zero coefficient, and in the case of a dual type tree and a chroma block, whether or not the corresponding LFNST index is signaled can be determined by checking whether or not DC position for the position of the last non-zero coefficient in the manner described above.

[309] iii. In case of ISP mode and single type tree, method i or ii can be used. That is, in the case of ISP mode and when method i is applied to a single type tree, it is possible to omit the check for the position of the last non-zero coefficient for both the luma and chroma blocks, and enable LFNST index signaling. Alternatively, by applying method ii, for partition blocks for the luma component, checking the position of the last non-zero coefficient is omitted, and for partition blocks for the chrominance component (if ISP is not applied for the chroma component, the number of partition blocks can be treated as 1), the position of the last non-zero coefficient is checked in the manner described above, thereby determining whether or not the LFNST index should be signaled.

[310] E. When it is confirmed that a transform coefficient exists at a position other than a position where a transform coefficient LFNST may exist for even one partition block among all partition blocks, the LFNST index signaling may be omitted.

[311] For example, in the case of a 4x4 partitioner and an 8x8 partitioner, the transform coefficients LFNST may exist at eight positions from the DC position according to the scanning order of the transform coefficients in the VVC standard, and the remaining positions are filled with zeros. Additionally, if it is equal to or greater than a 4x4 partition block, and is neither a 4x4 partition block nor an 8x8 partition block, the LFNST transform coefficients may exist at 16 positions from the DC position according to the scanning order of the coefficients conversions in the VVC standard, and all other positions are filled with zeros.

[312] Accordingly, if non-zero transform coefficients exist in a region that should be filled with a zero value after undergoing remainder encoding, the LFNST index signaling may be omitted.

[313] Meanwhile, in the case of ISP mode, the length condition is independently considered for the horizontal direction and vertical direction, and DST-7 is applied instead of DCT-2 without MTS index signaling. It is determined whether the horizontal or vertical length is greater than or equal to 4 and less than or equal to 16, and the primary transformation kernel is determined according to the determination result. Accordingly, in the case of ISP mode where LFNST can be applied, the following transformation combination configuration is possible.

[314] 1. When the LFNST index is 0 (including the case in which the LFNST index is considered to be 0), the ISP-time primary conversion decision condition included in the current VVC standard can be followed. In other words, it can be checked whether the length condition (equal to or greater than 4, or equal to or less than 16) is satisfied or not independently for the horizontal and vertical directions, respectively, and if it is satisfied, DST-7 can be applied instead of DCT- 2 for the primary conversion, and if not satisfied, then DCT-2 can be used.

[315] 2. For the case where the LFNST index is greater than 0, the following two configurations may be possible as the primary conversion.

[316] A. DCT-2 can be applied to both vertical and horizontal directions.

[317] B. You can follow the ISP-time primary conversion decision condition included in the current VVC standard. In other words, it can be checked whether the length condition (equal to or greater than 4, or equal to or less than 16) is satisfied or not independently for the horizontal and vertical directions, respectively, and if it is satisfied, DST-7 can be applied instead of DCT- 2, and if it is not satisfied, DCT-2 may be applied.

[318] In ISP mode, image information can be configured such that the LFNST index is transmitted for each partitioning block rather than for each coding unit. In this case, in the above-described LFNST index signaling method, it can be considered that only one partitioning block exists in the unit in which the LFNST index is transmitted, and whether or not to signal the LFNST index can be determined.

[319] Next, an embodiment for applying LFNST only to the luma component in a single tree is described.

[320] A coding unit syntax table for LFNST index signaling and MTS index signaling according to an example is shown below.

[321] [Table 4]

[322] The values of the main variables in the table above are as follows.

[323] 1. cbWidth, cbHeight: Width and height of the current encoding block

[324] 2. log2TbWidth, log2TbHeight: The base 2 logarithm values of the width and height of the current transform block. The size of the current transform block can be reduced to the top-left region in which a non-zero coefficient can exist by applying nulling.

[325] 3. sps_lfnst_enabled_flag: A flag indicating whether LFNST is enabled. A flag value of 0 indicates that LFNST is not enabled, and a flag value of 1 indicates that LFNST is enabled. The flag is defined in the Sequence Parameter Set (SPS).

[326] 4. CuPredMode[chType][x0][y0]: Prediction mode corresponding to variable chType and position (x0, y0). chType can have the values 0 and 1, where 0 represents the luma component and 1 represents the chrominance component. Position (x0, y0) indicates the position in the picture, and MODE_INTRA (intra-prediction) and MODE_INTER (inter-prediction) are possible with the value CuPredMode[chType][x0][y0].

[327] 5. IntraSubPartitionsSplitType: Indicates which ISP is applied to the current coding unit, and ISP_NO_SPLIT indicates that the coding unit is not divided into split blocks.

[328] 6. intra_mip_flag[x0][y0]: The position (x0, y0) is described as in 4 above. intra_mip_flag is a flag indicating whether the matrix-based intra-prediction (MIP) mode is applied. A flag value of 0 indicates that MIP is not applicable, and a flag value of 1 indicates that MIP is applied.

[329] 7. cIdx: A value of 0 indicates brightness, and values of 1 and 2 indicate the chrominance components Cb and Cr, respectively.

[330] 8. treeType: Indicates a single tree, dual tree, or the like. (SINGLE_TREE: Single tree, DUAL_TREE_LUMA: Dual tree for luma component, DUAL_TREE_CHROMA: Dual tree for chrominance component).

[331] 9. lfnst_idx[x0][y0]: The LFNST index syntax element to be parsed. If this fails through parsing, then the element is considered to have a value of 0. That is, the default value is set to 0, which indicates that LFNST is not applied.

[332] The description of the preceding syntax elements may apply to the syntax elements shown in the following tables.

[333] In Table 4, transform_skip_flag[x0][y0][0]==0 is a condition for determining whether the index lfnst with respect to the luma component is signaled according to whether the transformation is skipped.

[334] Accordingly, according to the example, the following coding unit syntax table is proposed to eliminate the dependency between transformation skip flag signaling for the luma component and LFNST index signaling for the chrominance component.

[335] [Table 5]

[336] In the embodiment shown in Table 5, the LFNST index signaling for the luma component depends only on the transform skip flag for the luma component for both dual-type tree and single-type tree partitioning modes. In dual tree mode, the LFNST index for a chrominance component may be signaled depending only on the transform skip flag for the chrominance component. In single tree split mode, LFNST is not applied to the chroma component to reduce worst-case latency.

[337] The variable LfnstTransformNotSkipFlag shown in Table 5 is set according to the tree type of the current block and the value of the transform skip flag for the color component, and only when the value of the variable is 1 can the LFNST index be signaled.

[338] When the tree type is a non-dual chroma tree (treeType != DUAL_TREE_CHROMA), that is, when the tree type is a single tree or a dual chroma tree, if the value of the transform skip flag for the luminance component is 0, the LfnstTransformNotSkipFlag variable may be set to 1 (treeType!=DUAL_TREE_CHROMA?transform_skip_flag [x0][y0][0]==0:(transform_skip_flag[x0][y0][1]==0||transform_

skip_flag[x0][y0][2]==0)), and when the tree type is a dual chroma tree, if the value of the transformation skip flag (transform_skip_flag[x0][y0][1]) for the chroma component Cr is 0 or the value transform skip flag (transform_skip_flag[x0][y0][1]) for the chrominance component Cr is 0, the LfnstTransformNotSkipFlag variable can be set to 1 (treeType!=DUAL_TREE_CHROMA?transform_skip_flag [x0][y0][0]==0:( transform_skip_flag[x0][y0][1]==0|| transform_skip_flag[x0][y0][2]==0)).

[339] In the present disclosure, the operator "x?y:z" indicates that x is y if x is TRUE, and x is z otherwise (if x is TRUE, then takes the value y; otherwise takes the value z).

[340] The specification text for the conversion process based on Table 5 is as follows.

[341] [Table 6]

8.7.4 Conversion process for scaled conversion factors
8.7.4.1 General information
The inputs of this process are:
- brightness location (xTbY, yTbY), specifying the top-left sample of the current brightness conversion block relative to the top-left sample of the current picture brightness,
- variable nTbW, which specifies the width of the current transformation block,
- variable nTbH, which specifies the height of the current transformation block
- variable cIdx, which specifies the color component of the current block,
- array (nTbW)×(nTbH) d[x][y] of scaled conversion coefficients with x=0…nTbW-1, y=0…nTbH-1.
The output of this process is an array of (nTbW)×(nTbH) res[x][y] residual samples with x=0…nTbW-1, y=0…nTbH-1.
The applyLfnstFlag variable is output as follows:
- If treeType is SINGLE_TREE and lfnst_idx is not 0 and transform_skip_flag[xTbY][yTbY][cIdx] is 0 and cIdx is 0, and both nTbW and nTbH are greater than or equal to 4, applyLfnstFlag is set to 1.
- Otherwise, if treeType is not equal to SINGLE_TREE and lfnst_idx is not equal to 0 and transform_skip_flag[xTbY][yTbY][cIdx] is equal to 0 and cIdx is equal to 0, and both nTbW and nTbH are greater than or equal to 4, applyLfnstFlag is set to 1.
- otherwise, applyLfnstFlag is set to 0.
When applyLfnstFlag is 1, the following applies:
- The variables predModeIntra, nLfnstOutSize, log2LfnstSize, nLfnstSize and nonZeroSize are output as follows:
predModeIntra=(cIdx==0) ? IntraPredModeY[xTbY][yTbY] : IntraPredModeC[xTbY][yTbY] (1149)
nLfnstOutSize=(nTbW >= 8 && nTbH >= 8) ? 48:16 (1150)
log2LfnstSize=(nTbW >= 8 && nTbH >= 8) ? 3:2 (1151)
nLfnstSize=1 << log2LfnstSize (1152)
nonZeroSize=(nTbW == 4 && nTbH == 4) || (nTbW == 8 && nTbH ==8) ? 8:16 (1153)

[342] The variable LfnstZeroOutSigCoeffFlag in Table 4 is equal to 0 if there is a significant coefficient at the zeroing position when LFNST is applied, and equal to 1 otherwise. The LfnstZeroOutSigCoeffFlag variable can be set according to a variety of conditions, shown below in Table 11.

[343] The variable LfnstZeroOutSigCoeffFlag indicates whether a significant coefficient exists in a second region other than the top-left first region of the current block. The value of the variable is initially set to 1 and can be changed to 0 when a significant coefficient exists in the second region. The LFNST index can be obtained by parsing only when the initially set value of the LfnstZeroOutSigCoeffFlag variable is stored. When determining and outputting whether the value of the variable LfnstZeroOutSigCoeffFlag is equal to 1, LFNST may be applied to the luma component or all chrominance components of the current block, and thus the color index of the current block is not determined.

[344] According to the example, the variable LfnstDcOnly in Table 4 is equal to 1 when all the last significant coefficients for transform blocks for which the corresponding coded block flag (CBF, which is equal to 1 when there is at least one significant coefficient in the corresponding block, and equals 0 otherwise) equals 1 are in the DC positions (upper left positions), and equals 0 otherwise. Specifically, the position of the last significant coefficient is checked with respect to one luma transform block in the dual luma tree, and the position of the last significant coefficient is checked with respect to both the transform block for Cb and the transform block for Cr in the dual chroma tree. In a single tree, the position of the last significant coefficient can be checked against the transform blocks for luminance, Cb and Cr.

[345] A syntax table for a coding unit signaling index LFNST according to another example is as follows.

[346] [Table 7]

[347] In Table 7, the variable transform_skip_flag[x0][y0][cIdx] indicates whether transform skipping is applied to the encoding block for the component specified by cIdx. cIdx can have a value of 0, 1, and 2, where 0 indicates the luminance component, and 1 and 2 indicate the Cb component and the Cr component, respectively. A transform_skip_flag[x0][y0][cIdx] value of 1 indicates that transform skipping is applied, and 0 indicates that transform skipping is not applied.

[348] In Table 7, the variable LfnstNotSkipFlag can be set to 1 only when skip conversion does not apply to all components (all coding blocks) forming the current coding unit, and can be set to 0 otherwise, and the index LFNST (lfnst_idx in Table 7) can only be signaled when LfnstNotSkipFlag is 1.

[349] When the current coding unit is encoded in a single tree structure (when treeType is SINGLE_TREE in Table 7), all of the components include Y, Cb and Cr, when the current coding unit is encoded in a separate tree structure for luma (when treeType represents is DUAL_TREE_LUMA in Table 7), all of the components include only Y, and when the current coding unit is encoded in a separate tree structure for chroma (when treeType is DUAL_TREE_CHROMA in Table 7), all of the components include Cb and Cr.

[350] In other words, when even one of the components consisting of the current encoding unit is encoded by skipping the transform, the LFNST index is not signaled and the LFNST index value is considered to be 0. That is, LFNST is not applied.

[351] In a structure restricting that LFNST is not applied when even one of the components (Y, Cb and Cr) is encoded by skipping transformation, as in Table 7, when multiple components (Y, Cb and Cr) are encoded sequentially in a single tree , and the component subject to skip conversion is determined (eg, when skip conversion is determined for Cb and Cr) during parsing, the corresponding transform coefficients may be configured not to be further buffered with respect to the corresponding components encoded by skip conversion until the index value is LFNST will not be parsed.

[352] For example, when any one component is found to be encoded by skipping the transform, one can be sure that LFNST is not applied, and thus inverse quantization, inverse transform, and the like can be performed immediately thereafter.

[353] Instead of Table 7, the syntax table can be briefly described as in Table 8.

[354] [Table 8]

[355] When determining whether to signal the LFNST index by checking only whether skip conversion applies to the luma component in a single tree, the syntax table for the coding unit can be configured as follows.

[356] [Table 9]

[357] In Table 9, LfnstNotSkipFlag is defined in the same way as in Table 7 or Table 8, in a separate tree (that is, in a separate luma tree, LfnstNotSkipFlag is set to 1 when skip conversion is not applied to the luma component, and is set to 0 otherwise, and in a separate chroma tree, LfnstNotSkipFlag is set to 1 when skip conversion is not applied to either the Cb or Cr component, and is set to 0 otherwise). In a single tree, LfnstNotSkipFlag is set to 1 when skip conversion is not applied to the luma component only, and is set to 0 otherwise. Instead of Table 8, the syntax table shown in Table 9 may be used.

[358] [Table 10]

[359] An embodiment for determining a LFNST index signaling condition when applying LFNST only to a luma component in a single tree is described below.

[360] In the syntax tables of the encoding units according to Table 4 and Table 5, Table 7, Table 8, Table 9 and Table 10, the LfnstDcOnly variable and the LfnstZeroOutSigCoeffFlag variable are used as conditions for signaling the LFNST index. Basically, both the LfnstDcOnly variable and the LfnstZeroOutSigCoeffFlag variable are initialized to 1, as shown in Table 7, and the values of the two variables can be updated to 0 in the syntax table for encoding the remainder, as shown in Table 11. For reference, when a component (which can be Y, Cb or Cr) is encoded with a skip transformation, another syntax table (transform_ts_coding) is imported instead of importing the remainder encoding in Table 11, and thus the LfnstDcOnly variable and the LfnstZeroOutSigCoeffFlag variable are not updated when the LFNST index for the component is parsed.

[361] [Table 11]

[362]

[363] In Table 11, lastSubBlock indicates the position of the subblock (coefficient group (CG)) in which the last significant (non-zero) coefficient is positioned in scan order. 0 indicates a subblock including a DC component, and a value greater than 0 indicates a subblock not including a DC component.

[364] lastScanPos indicates the position of the last significant coefficient in one subblock in scan order. When one subblock includes 16 positions, values from 0 to 15 are possible.

[365] LastSignificantCoeffX and LastSignificantCoeffY indicate the x-coordinate and y-coordinate of the last significant coefficient positioned in the transform block. X-coordinates start at 0 and increase from left to right, and y-coordinates start at 0 and increase from top to bottom. Values of two variables equal to 0 mean that the last significant coefficient is positioned in DC.

[366] Basically, Table 11 is applied to the embodiment of Table 5 to determine the values of the LfnstDcOnly variable and the LfnstZeroOutSigCoeffFlag variable. When Table 11 is applied and a coding block is coded in a single tree, the remainder coding presented in Table 11 can be imported for all components. For example, when Y, Cb, and Cr components are all skip-coded, remainder coding may be performed on each component.

[367] Therefore, when Table 11 is applied, and the current encoding block is encoded by a single tree, if the last non-zero coefficient of even one component is positioned at a position other than the DC position (top-left position of the transform block), the value of the LfnstDcOnly variable can be updated to 0, and when the position of the last non-zero coefficient of even one component is positioned in a region where the transform coefficient in the case where LFNST is applied cannot be positioned (that is, positioned in a region other than the first to eighth positions according to the scanning order direct conversion coefficients in a 4x4 transform block or an 8x8 transform block, or is positioned in a region other than the top-left 4x4 region in a transform block to which LFNST in the modern VVC standard applies), the value of the LfnstZeroOutSigCoeffFlag variable may be updated to 0. As shown in the encoding unit syntax table of Table 5, Table 7, Table 8, Table 9 and Table 10, the LFNST index can be signaled only when the value of the LfnstZeroOutSigCoeffFlag variable is 1, and in mode other than ISP mode, the LFNST index can Signal only when the value of the LfnstDcOnly variable is 0.

[368] However, when LFNST is applied only to the luma component in a single tree, it may not be possible to update the LfnstDcOnly variable and the LfnstZeroOutSigCoeffFlag variable when remainder encoding is performed on components to which LFNST is not applied (Cb or Cr chrominance components). This is because it is counterintuitive for the arrangement or distribution of transform coefficients for a component to which LFNST is not applied to determine whether LFNST is signaled, that is, whether LFNST is applied.

[369] Table 12 limits the LfnstDcOnly variable and the LfnstZeroOutSigCoeffFlag variable to be updated only for the luma component in a single tree. In a case other than a single tree, the LfnstDcOnly variable and the LfnstZeroOutSigCoeffFlag variable can be updated for all components, as in Table 11.

[370] [Table 12]

[371]

[372] Based on the details of Table 5, some of the details may be replaced by embodiments of Table 7 to Table 10, or the details of Table 11 or Table 12 may be used. The following possible combinations may be configured based on Table 7 to Table 10, Table 11, or Table 12.

[373] 1. Table 7 (or Table 8) + Table 11

[374] 2. Table 7 (or Table 8) + Table 12

[375] 3. Table 9 (or Table 10) + Table 11

[376] 4. Table 9 (or Table 10) + Table 12

[377] Next, a method for applying a scaling list to chroma components when LFNST is applied only to the luma component in a single tree is described.

[378] Currently, VVC WD defines a syntax element called scaling_matrix_for_lfnst_disabled_flag. When scaling_matrix_for_lfnst_disabled_flag is 1, the scaling list is not applied when LFNST is applied, and when scaling_matrix_for_lfnst_disabled_flag is 0, the scaling list can be applied when LFNST is applied.

[379] Here, the scaling list is a matrix for specifying the specific weight (weighted value) for each transform coefficient position in the transform block and allows dequantization or quantization by multiplying by the weight for each transform coefficient, thereby allowing differential dequantization or quantization to be applied at according to the importance of the conversion factor.

[380] In a single tree, LFNST can only be applied to the luma component as in the embodiment of Table 5, and when the value of scaling_matrix_for_lfnst_disabled_flag is 1 and LFNST is applied when a coding block is encoded in the single tree, the scaling list is not applied to the luma component. Here, the scaling list can be applied to the chroma component to which LFNST does not apply.

[381] Table 13 shows an example of a dequantization process (scaling process) to implement the above case.

[382] Table 13

8.7.3 Scaling process for converting coefficients.
The inputs to this process are:
- luminance location (xTbY, yTbY), specifying the top-left sample of the current luminance transform block relative to the top-left luminance sample of the current picture,
- variable nTbW, which specifies the width of the conversion block,
- variable nTbH, which specifies the height of the transformation block,
- predMode variable, which specifies the coding unit prediction mode,
- variable cIdx, which specifies the color component of the current block.
- a treeType variable that specifies whether a single tree (SINGLE_TREE) or a dual tree is used to split a coding tree node, and, when a dual tree is used, and whether the luminance (DUAL_TREE_LUMA) or chrominance (DUAL_TREE_CHROMA) components are currently being processed.
The output of this process is an array (nTbW)x(nTbH) d of scaled transform factors with elements d[x][y].
The quantization parameter qP is derived as follows:
- If cIdx is 0, then the following applies:
qP=Qp′ _Y
- Otherwise, if TuCResMode[xTbY][yTbY] is 2, the following applies:
qP=Qp′ _CbCr
- Otherwise, if cIdx is 1, the following applies:
qP=Qp′ _Cb
- Otherwise (cIdx equals 2), the following applies:
qP=Qp′ _Cr
The quantization parameter qP is modified and the variables rectNonTsFlag and bdShift are output as follows:
- If transform_skip_flag[xTbY][yTbY][cIdx] is 0, the following applies:
qP=qP - (cu_act_enabled_flag[xTbY][yTbY] ? 5 : 0)
rectNonTsFlag=( ( (Log2(nTbW) + Log2(nTbH) ) & 1)== 1) ? 1 : 0 (1134)
bdShift=BitDepth+rectNonTsFlag + (1135) ( (Log2(nTbW) + Log2(nTbH) ) / 2) - 5+pic_dep_quant_enabled_flag
- Otherwise (transform_skip_flag[xTbY][yTbY][cIdx] is equal to 1), the following applies:
qP=Max(QpPrimeTsMin, qP) - (cu_act_enabled_flag[xTbY][yTbY] ? 5 : 0) (1136)
rectNonTsFlag=0
bdShift=10

[383]

The bdOffset variable is output as follows:
bdOffset=(1 << bdShift) >> 1
The list levelScale[ ][ ] is given as levelScale[j][k]={{40, 45, 51, 57, 64, 72}, {57, 64, 72, 80, 90, 102}} with j=0. .1, k=0..5.
Array (nTbW)x(nTbH) dz is set equal to array (nTbW)x(nTbH) TransCoeffLevel[xTbY][yTbY][cIdx].
To derive the scaled conversion factors d[x][y] for x=0..nTbW - 1, y=0..nTbH - 1, the following applies:
- The intermediate scaling factor m[x][y] is derived as follows:
- If one or more of the following conditions is true, m[x][y] is set to 16:
-- sps_scaling_list_enabled_flag is 0.
-- pic_scaling_list_present_flag is 0.
-- transform_skip_flag[xTbY][yTbY][cIdx] is equal to 1.
-- scaling_matrix_for_lfnst_disabled_flag is 1 and lfnst_idx[xTbY][yTbY] is not 0 and one of the following conditions is true.
--- treeType is SINGLE_TREE and cIdx is 0.
--- treeType is DUAL_TREE_LUMA or DUAL_TREE_CHROMA.
- Otherwise, the following applies:
-- The id variable is derived based on predMode, cIdx, nTbW and nTbH as defined in Table 36 and the log2MatrixSize variable is derived as follows:
log2MatrixSize=(id < 2) ? 1 : (id < 8) ? 2 : 3
-- The scaling factor m[x][y] is derived as follows:
m[x][y]=ScalingMatrixRec[id][i][j]
with i=(x << log2MatrixSize) >> Log2(nTbW), j=(y << log2MatrixSize) >> Log2( nTbH )
-- If id is greater than 13 and x and y are 0, m[0][0] is further modified as follows:
m[0][0]=ScalingMatrixDCRec[id-14]
……

[384] In Table 13, treeType indicates the tree type of the coding unit to which the currently processed transform block belongs, and SINGLE_TREE, DUAL_TREE_LUMA, and DUAL_TREE_CHROMA indicate single tree, luma single tree, and chrominance single tree (dual chroma tree), respectively.

[385] In this embodiment, since LFNST can only be applied to the luma component in a single tree, when the value of scaling_matrix_for_lfnst_disabled_flag is 1 and LFNST is applied (the value of lfnst_idx[xTbY][yTbY] is greater than 0), the scaling list is not applied to the luma component (when cIdx value is 0).

[386] However, for the chroma component (when the value of cIdx is greater than 0), whether the scaling list should be applied can be determined by additionally checking another condition (eg, checking transform_skip_flag[xTbY][yTbY][cIdx]).

[387] In a single tree, as in the case of the luma component in a single tree, when the scaling_matrix_for_lfnst_disabled_flag value is 1 and LFNST is applied (the value of lfnst_idx[xTbY][yTbY] is greater than 0), the scaling list is not applied to the luma and chrominance components.

[388] Further, in a single tree, as in the case of a chroma component in a single tree, whether a scaling list should be applied can be determined by additionally checking another condition (eg, checking transform_skip_flag[xTbY][yTbY][cIdx]).

[389] Therefore, when the value of scaling_matrix_for_lfnst_disabled_flag is 1 and LFNST can only be applied to the luma component in a single tree, the scaling list may not apply to the luma component and may be applied to the chroma component.

[390] According to the example, a combination of Table 13 and the above-described embodiments (a combination of substituting some parts with embodiments of Table 7 to Table 10 or using parts of Table 11 or Table 12 based on the details of Table 5) may be used.

[391] In this case, as in the specification text "Transform Process for Scaled Transform Factors" in Table 6, LFNST can be configured to apply only to the luma component in a single tree.

[392] The following drawings are provided to describe specific examples of the present disclosure. While specific terms for devices or specific terms for signals/messages/fields illustrated in the drawings are provided for illustration purposes, the technical features of the present disclosure are not limited to the specific terms used in the following drawings.

[393] FIG. 15 is a flowchart illustrating operation of a decoding apparatus in accordance with an embodiment of the present disclosure.

[394] Each process disclosed in FIG. 15 is based on some of the details described with reference to FIG. 5 - fig. 14. Therefore, the description of specific details overlapping with the description with reference to FIG. 3 and fig. 5 to fig. 14 will be omitted or presented schematically.

[395] The decoding apparatus 300 according to an embodiment may receive flag information indicating whether the scaling list is available when LFNST is executed, the LFNST index for the current block, and residual information from the bitstream (S1510).

[396] Specifically, the decoding apparatus 300 can decode quantized transform coefficient information for a current block from the bit stream, and can output quantized transform coefficients for a target block based on the quantized transform coefficient information for the current block. Information about the quantized transform coefficients for the target block may be included in a sequence parameter set (SPS) or slice header and may include at least one of information about whether RST is applied, information about a shortened metric, information about a minimum transform size for applying the RST, information about the maximum transformation size for applying the RST, the size of the inverse RST, and information about a transformation index indicating any one of the transformation kernel matrices included in the set of transformations.

[397] The decoding apparatus may also receive information about the intra-prediction mode for the current block and information about whether ISP is applied to the current block. The decoding apparatus may receive and parse flag information indicating whether ISP encoding or ISP mode should be applied, thereby determining whether the current block is divided into a predetermined number of subdivision transform blocks. Here, the current block may be an encoding block. In addition, the decoding apparatus may output the size and number of divided subdivision blocks through flag information indicating the direction in which the current block is divided.

[398] The LFNST index is the value for specifying the LFNST matrix when LFNST is applied as an inverse secondary non-separable transform, and can have a value ranging from 0 to 2. For example, a LFNST index value of 0 may indicate that LFNST is not applied to the current block, the value LFNST index value 1 may indicate a first LFNST matrix, and LFNST index value 2 may indicate a second LFNST matrix.

[399] ISP and LFNST index information may be received at the coding unit level.

[400] Flag information indicating whether a scaling list is available when LFNST is executed that is received by the decoding device may be represented by scaling_matrix_for_lfnst_disabled_flag or sps_scaling_matrix_for_lfnst_disabled_flag and may be signaled in the sequence parameter set. A value of 1 for this flag indicates that the scaling list is not applied when LFNST is applied, and a flag value of 0 indicates that the scaling list is applicable when LFNST is applied. The scaling list is a matrix for specifying the specific weight (weighted value) for each transform factor position in the transform block allows dequantization or quantization by multiplying by the weight for each transform factor, thereby allowing differential dequantization or quantization to be applied according to the importance of the transform factor.

[401] The decoding apparatus 300 can output whether the scaling list is applied to the current block based on the application of LFNST and the tree type of the current block to dequantize transform coefficients for the current block, and can derive transform coefficients for the current block from residual information based on the output result (S1520).

[402] Whether the scaling list is applied may be determined based on the flag information and the index value of LFNST.

[403] When the tree type of the current block is a single tree, the color components of the current block may include a luma component, a first chroma component indicating the chroma Cb, and a second chroma component indicating the chrominance Cr, and when the tree type of the current block is a dual luma tree , the current block may include a luma component. When the tree type of the current block is a dual chroma tree, the color components of the current block may include a first chroma component and a second chroma component.

[404] Here, the current block may be a transform block, which is a transform unit, and when the tree type of the current block is a single tree, the current block may include a transform block for the luma component, a transform block for the first chrominance component, and a transform block for the second chromaticity component. When the tree type of the current block is a dual luma tree, the current block may include a transform block for a luma component, and when the tree type of the current block is a dual chrominance tree, the current block may include a transform block for a first chrominance component and a transform block for a second chromaticity component.

[405] According to the example, when the type of the current block is a single tree, LFNST can only be applied to the luma component, and when the current block is encoded in a single tree, the value of scaling_matrix_for_lfnst_disabled_flag is 1 and LFNST is applied, the scaling list is not applied to the luma component. However, a scaling list can be applied to a chroma component to which LFNST does not apply.

[406] Finally, in the case where the scaling list flag information indicates that the scaling list is not available and the LFNST index is greater than 0 (that is, LFNST is applied), when the tree type of the current block is a single tree and the current block is a luma component , the scaling list may not be applied, and when the tree type of the current block is single tree and the current block is a chroma component, the scaling list may be applied.

[407] According to an example, in the case where the scaling list flag information indicates that the scaling list is not available and the index LFNST is greater than 0, when the tree type of the current block is a dual chroma tree, LFNST is applicable to the current block, and thus Thus, the scaling list is not applied to the chroma component.

[408] According to the example, in the case where the scaling list flag information indicates that the scaling list is not available and the index LFNST is greater than 0, when the tree type of the current block is a dual luma tree, LFNST is applicable to the current block, and thus Thus, the scaling list is not applied to the luma component.

[409] The output transform coefficients may be arranged in units of a 4×4 block according to a reverse diagonal scan order, and the transform coefficients in a 4×4 block may also be arranged according to a reverse diagonal scan order. That is, the dequantized transform coefficients can be ordered according to the reverse scan order used in a video codec, such as VVC or HEVC.

[410] The decoding apparatus can output modified transform coefficients from the transform coefficients based on the LFNST index and the LFNST matrix of the LFNST, that is, by applying the LFNST (S1530).

[411] LFNST is an inseparable transform in which the transform is applied to the coefficients without separating the coefficients in a particular direction, as opposed to first transforming the vertically or horizontally separable coefficients to be transformed and transforming them. This non-separable transform may be a low-frequency non-separable transform applying direct transform only in the low-frequency range, rather than the entire block region.

[412] The decoder may output various variables to apply LFNST, and may determine whether to apply LFNST based on the tree type and current block size.

[413] The decoding apparatus may output a first variable (LfnstDcOnly variable) indicating whether a significant coefficient exists at a position other than the DC component in the current block, and a second variable (LfnstZeroOutSigCoeffFlag variable) indicating whether a transform coefficient exists in the second region other than the top-left first region of the current block.

[414] The first variable and the second variable are initially set to 1, the first variable may be updated to 0 when a significant coefficient exists at a position other than the DC component in the current block, and the second variable may be updated to 0 when the conversion coefficient exists in the second area.

[415] When the first variable is updated to 0 and the second variable is stored at 1, LFNST can be applied to the current block.

[416] For a luma component to which the intra-subdivision (ISP) mode is applied, the LFNST index can be obtained by parsing without deriving the LfnstDcOnly variable.

[417] Specifically, in the case where ISP mode and transform skip flag is applied, that is, transform_skip_flag[x0][y0][0], the luma component is 0 when the tree type of the current block is single tree or dual luma tree , the LFNST index can be signaled regardless of the value of the LfnstDcOnly variable.

[418] However, for a chroma component to which ISP mode is not applied, the value of the LfnstDcOnly variable may be set to 0 according to the value of the transform skip flag for the Cb chrominance component, transform_skip_flag[x0][y0][1], and the value of the flag skip transform for Cr chroma component, transform_skip_flag[x0][y0][2]. That is, when the value of cIdx in transform_skip_flag[x0][y0][cIdx] is 1, the value of the variable LfnstDcOnly can be set to 0 only when the value of transform_skip_flag[x0][y0][1] is 0, and when the value of cIdx is 2, the value of the LfnstDcOnly variable can be set to 0 only when the value of transform_skip_flag[x0][y0][2] is 0. When the value of the LfnstDcOnly variable is 0, the decoding device can parse with respect to the LFNST index, and otherwise, the LFNST index may be considered 0 without signaling.

[419] The second variable may be a LfnstZeroOutSigCoeffFlag variable, which may indicate that zeroing is performed when LFNST is applied. The second variable may be initially set to 1 and may be changed to 0 when a significant coefficient exists in the second region.

[420] The variable LfnstZeroOutSigCoeffFlag may be output as 0 when the index of the subblock in which the last nonzero coefficient exists is greater than 0 and both the width and height of the transform block are equal to or greater than 4, or when the position of the last nonzero coefficient in the subblock , in which the last non-zero coefficient exists is greater than 7, and the transform block size is 4x4 or 8x8. A sub-block refers to a 4x4 block used as a coding unit in residue coding, and may be referred to as a coefficient group (CG). Subblock index 0 refers to the top-left 4x4 subblock.

[421] That is, when a non-zero coefficient is output in a region other than the top-left region in which a transform coefficient LFNST may exist in a transform block, or a non-zero coefficient exists in a position other than the eighth position in the scanning order for a 4×4 block or an 8x8 block, the variable LfnstZeroOutSigCoeffFlag is set to 0.

[422] The decoding apparatus may determine a LFNST set including LFNST matrices based on the intra-prediction mode derived from the intra-prediction mode information, and may select any one of the LFNST matrices based on the LFNST set and the LFNST index.

[423] Here, the same set of LFNST and the same LFNST index may be applied to the subdivision transform blocks into which the current block is divided. That is, since the same intra-prediction mode is applied to the subdivision transform blocks, the set of LFNST determined based on the intra-prediction mode can also be equally applied to all subdivision transform blocks. Additionally, since the LFNST index is signaled at the coding unit level, the same LFNST matrix can be applied to the subdivision transform blocks into which the current block is divided.

[424] As described above, a transformation set can be determined in accordance with the intra-prediction mode for a transformation block to be transformed, and the inverse LFNST can be performed based on the transformation kernel matrix, that is, any one of the LFNST matrices included in the transformation set , indicated by the index LFNST. The matrix applied to the inverse LFNST may be called an inverse LFNST matrix or an LFNST matrix, and is defined by either term if the matrix is a transpose of the matrix used for the forward LFNST.

[425] In an example, the inverse LFNST matrix may be a non-square matrix in which the number of columns is less than the number of rows.

[426] The decoding apparatus may output residual samples for the current block based on the primary inverse transform for the modified transform coefficients (S1540).

[427] Here, the general separable transform or the above-described MTS can be used as the primary inverse transform.

[428] Next, the decoding apparatus 300 can generate reconstructed samples based on the residual samples for the current block and the prediction samples for the current block (S1550).

[429] The following drawings are provided to describe specific examples of the present disclosure. While specific terms for devices or specific terms for signals/messages/fields illustrated in the drawings are provided for illustration purposes, the technical features of the present disclosure are not limited to the specific terms used in the following drawings.

[430] FIG. 16 is a flowchart illustrating operation of a video encoding apparatus according to an embodiment of the present disclosure.

[431] Each process disclosed in FIG. 16 is based on some of the details described with reference to FIG. 4 - fig. 14. Therefore, the description of specific details overlapping with the description with references to FIGS. 2 and fig. 4 - fig. 14 will be omitted or presented schematically.

[432] The encoder 200 according to an embodiment can output prediction samples for the current block based on the intra-prediction mode applied to the current block (S1610).

[433] When ISP is applied to the current block, the encoder may perform prediction on each subdivision transform block.

[434] The encoding apparatus can determine whether ISP encoding or the ISP mode should be applied to the current block, that is, an encoding block, and can determine the direction in which the current block is divided, and can output the size and number of divided sub-blocks according to the determination result .

[435] The same intra-prediction mode may be applied to the subdivision transform blocks into which the current block is divided, and the encoder may output a prediction sample for each subdivision transform block. That is, the encoding device sequentially performs intra-prediction, for example, horizontally or vertically, or from left to right or from top to bottom, according to the partitioning shape of the subdivision transform blocks. For the leftmost or topmost subblock, the reconstructed pixel of the already encoded encoding block is referenced as in the conventional intra-prediction method. Moreover, for each side of the subsequent intra subdivision transform block that is not adjacent to the previous subdivision transform block, to output reference pixels adjacent to the side, the reconstructed pixel of the already encoded adjacent encoding block is referenced as in the conventional intra-prediction method.

[436] The encoder 200 may output residual samples for the current block based on the prediction samples (S1620).

[437] Encoder 200 may output transform coefficients for the current block by applying at least one of LFNST or MTS to the residual samples, and may order the transform coefficients according to a predetermined scan order.

[438] The encoder may derive transform coefficients for the current block based on a transform process such as a primary transform and/or a secondary transform on residual samples, may apply LFNST when the current block is a single tree and luma component, and may not apply LFNST , when the current block is a single tree and a chroma component.

[439] The encoding device may output transform coefficients for the current block based on the primary transform on the residual samples (S1630), and the primary transform may be performed by a plurality of transform kernels as in MTS, in which case the transform kernel may be selected based on the intra-prediction mode .

[440] The encoding apparatus 200 can determine whether to perform a secondary transform or an inseparable transform, specifically LFNST, on the transform coefficients for the current block, and can output modified transform coefficients by applying LFNST to the transform coefficients (S1640).

[441] LFNST is an inseparable transform in which the transform is applied to the coefficients without separating the coefficients in a particular direction, as opposed to first transforming the vertically or horizontally separable coefficients to be transformed and transforming them. This non-separable transform may be a low-frequency non-separable transform of applying the transform only to the low-frequency region and not to the entire target block to be transformed.

[442] The encoder may output various variables for applying LFNST and may determine whether to apply LFNST based on the tree type and current block size.

[443] The encoder may output a first variable (LfnstDcOnly variable) indicating whether a significant coefficient exists at a position other than the position of the DC component in the current block, and a second variable (LfnstZeroOutSigCoeffFlag variable) indicating whether a transform coefficient exists in the second region other than the top-left first region of the current block.

[444] The first variable and the second variable are initially set to 1, the first variable may be updated to 0 when a significant coefficient exists at a position other than the position of the DC component in the current block, and the second variable may be updated to 0 when the coefficient transformation exists in the second area.

[445] When the first variable is updated to 0 and the second variable stores the value 1, LFNST can be applied to the current block.

[446] For a luma component to which the intra-subdivision (ISP) mode is applicable, LFNST can be applied without emitting the LfnstDcOnly variable.

[447] Specifically, in the case where ISP mode is applied, and the transform skip flag, that is, transform_skip_flag[x0][y0][0], for the luma component is 0 when the tree type of the current block is single tree or dual tree for brightness, LFNST can be applied regardless of the value of the LfnstDcOnly variable.

[448] However, for a chroma component to which the ISP mode is not applied, the value of the LfnstDcOnly variable may be set to 0 according to the value of the transform skip flag for the Cb chrominance component, transform_skip_flag[x0][y0][1], and the value of the flag skip transform for Cr chroma component, transform_skip_flag[x0][y0][2]. That is, when the value of cIdx in transform_skip_flag[x0][y0][cIdx] is 1, the value of the variable LfnstDcOnly can be set to 0 only when the value of transform_skip_flag[x0][y0][1] is 0, and when the value of cIdx is 2, the value of the LfnstDcOnly variable can be set to 0 only when the value of transform_skip_flag[x0][y0][2] is 0. When the value of the LfnstDcOnly variable is 0, the encoder can apply LFNST, and otherwise, the encoder cannot apply LFNST.

[449] The second variable may be a LfnstZeroOutSigCoeffFlag variable, which may indicate that zeroing is performed when LFNST is applied. The second variable may be initially set to 1 and may be changed to 0 when a significant coefficient exists in the second region.

[450] The variable LfnstZeroOutSigCoeffFlag may be output as 0 when the index of the subblock in which the last nonzero coefficient exists is greater than 0 and both the width and height of the transform block are equal to or greater than 4, or when the position of the last nonzero coefficient in the subblock , in which the last non-zero coefficient exists is greater than 7, and the transform block size is 4x4 or 8x8. A sub-block refers to a 4x4 block used as a coding unit in residue coding, and may be referred to as a coefficient group (CG). Subblock index 0 refers to the top-left 4x4 subblock.

[451] That is, when a non-zero coefficient is output in a region other than the top-left region in which a transform coefficient LFNST may exist in a transform block, or a non-zero coefficient exists in a position other than the eighth position in the scanning order for a 4×4 block or an 8x8 block, the variable LfnstZeroOutSigCoeffFlag is set to 0.

[452] The encoding apparatus may determine a set of LFNSTs including LFNST matrices based on an intra-prediction mode derived from the intra-prediction mode information, and may select any one of a plurality of LFNST matrices.

[453] Here, the same set of LFNST and the same LFNST index may be applied to the subdivision transform blocks into which the current block is divided. That is, since the same intra-prediction mode is applied to the subdivision transform blocks, the set of LFNST determined based on the intra-prediction mode can also be equally applied to all subdivision transform blocks. Moreover, since the LFNST index is signaled at the coding unit level, the same LFNST matrix can be applied to the subdivision transform blocks into which the current block is divided.

[454] As described above, a transformation set may be determined in accordance with an intra-prediction mode for a transformation block to be transformed, and LFNST may be performed based on a transformation kernel matrix, that is, any one of the LFNST matrices included in the transformation set LFNST . The matrix used for LFNST may be called the LFNST matrix or defined by any term as long as the matrix is a transpose of the matrix used for the inverse LFNST.

[455] In an example, the matrix LFNST may be a non-square matrix in which the number of rows is less than the number of columns.

[456] The encoding device can output whether a scaling list is applied to the current block based on whether LFNST is performed in the transform process and the tree type of the current block, and can perform quantization based on transform coefficients or modified transform coefficients and the scaling list (S1650 ).

[457] A scaling list is a matrix for specifying a specific weight (weighted value) for each transform coefficient position in a transform block and allows dequantization or quantization by multiplying by a weight for each transform coefficient, thereby allowing differential dequantization or quantization to be applied according to the importance of the conversion factor.

[458] According to an example, when the tree type of the current block is a single tree and the current block is a luma component, the encoding device may not apply the scaling list, and when the tree type of the current block is a single tree and the current block is a chroma component, the encoding device may apply scaling list.

[459] When the tree type of the current block is a single tree, the color components of the current block may include a luma component, a first chroma component indicating a Cb chroma and a second chroma component indicating a Cr chroma, and when the tree type of the current block is a dual luma tree , the current block may include a luma component. When the tree type of the current block is a dual chroma tree, the color components of the current block may include a first chroma component and a second chroma component.

[460] Here, the current block may be a transform block that is a transform unit, and when the tree type of the current block is a single tree, the current block may include a transform block for the luma component, a transform block for the first chrominance component, and a transform block for the second chromaticity component. When the tree type of the current block is a dual luma tree, the current block may include a transform block for a luma component, and when the tree type of the current block is a dual chrominance tree, the current block may include a transform block for a first chrominance component and a transform block for a second chromaticity component.

[461] According to an example, when the current block is a single tree, the encoder may apply LFNST only to the luma component, and when LFNST is applied, the encoder does not apply the scaling list to the luma component. However, the encoder may apply a scaling list to a chroma component to which LFNST does not apply.

[462] Finally, in the case where the LFNST index is greater than 0 (that is, LFNST is applied), when the tree type of the current block is a single tree and the current block is a luma component, the encoder may not apply the scaling list, and when the type The tree of the current block is a single tree and the current block is a chroma component, the encoder can apply a scaling list.

[463] According to an example, in the case where the index LFNST is greater than 0, when the tree type of the current block is a dual chroma tree, LFNST is applied to the current block, and thus the encoder does not apply the scaling list.

[464] According to an example, in the case where the index LFNST is greater than 0, when the tree type of the current block is a dual luma tree, LFNST is applied to the current block, and thus the encoder does not apply the scaling list.

[465] The encoder may quantize the transform coefficients based on determining whether a scaling list is applied to the current block.

[466] That is, the encoder can quantize transform coefficients for a transform block to which LFNST is not applied using a scaling list, and can quantize transform coefficients for a transform block to which LFNST is applied without using a scaling list.

[467] The encoding device may encode and output residual information and flag information indicating whether a scaling list is available when LFNST is executed (S1660).

[468] Flag information indicating whether a scaling list is available when LFNST is executed may be represented by scaling_matrix_for_lfnst_disabled_flag or sps_scaling_matrix_for_lfnst_disabled_flag and may be signaled in the sequence parameter set. A value of this flag of 1 indicates that the scaling list is not applied when LFNST is applied, and a flag value of 0 indicates that the scaling list is applicable when LFNST is applied.

[469] When the LFNST index is greater than 0 and the current block is a single tree, LFNST is applied to the luma component, and thus the encoder can encode the flag value to 1.

[470] However, when the LFNST index is greater than 0 and the current block is a single tree, LFNST is not applied to the chrominance component, and thus the encoding device can construct the image information such that the scaling list can be applied.

[471] When the LFNST index is greater than 0 and the tree type of the current block is a dual chroma tree, LFNST is applied to the current block, and thus the encoder can encode the flag value to 1 so that the scaling list is not applied to the chroma component .

[472] According to the example, when the index LFNST is greater than 0 and the tree type of the current block is a dual luma tree, LFNST is applicable to the current block, and thus the encoder can encode the flag value to 1, so that the scaling list does not apply to the luma component.

[473] The encoder may output quantized transform coefficients by quantizing the modified transform coefficients for the current block and may encode the LFNST index.

[474] The encoder may generate residual information, including information about quantized transform coefficients. The residual information may include the above-described information/syntactic element related to the transformation. The encoding apparatus may encode image/video information including residual information, and may output the image/video information in the form of a bit stream.

[475] Specifically, the encoding device 200 can generate information about the quantized transform coefficients and can encode the information about the quantized transform coefficients.

[476] The LFNST index syntax element according to the present embodiment may indicate any one of whether the (inverse) LFNST and any one of the LFNST matrices included in the LFNST set are applied, and when the LFNST set includes two transformation kernel matrices, The LFNST index syntax element can have three values.

[477] According to an example, when the partition tree structure of the current block is a dual tree, the index LFNST may be encoded for each of the luma block and the chrominance block.

[478] According to an embodiment, values of the transform index syntax element may include 0 indicating that the (reverse) LFNST is not applied to the current block, 1 indicating the first LFNST matrix among the LFNST matrices, and 2 indicating the second LFNST matrix among the LFNST matrices.

[479] In the present disclosure, at least one of quantization/dequantization and/or transform/inverse transform may be omitted. When quantization/dequantization is omitted, the quantized transform coefficient may be referred to as the transform coefficient. When the transform/inverse transform is omitted, the conversion factor may be referred to as the coefficient or residual coefficient, or may still be referred to as the conversion factor for purposes of expression consistency.

[480] In addition, in the present disclosure, the quantized transform coefficient and the transform coefficient may be referred to as the transform coefficient and the scaled transform coefficient, respectively. In this case, the residual information may include information about the transform coefficient(s), and the information about the transform coefficient(s) may be signaled by a residual encoding syntax. Transform coefficients can be derived based on the residual information (or information about the transform coefficient(s)), and scaled transform coefficients can be derived by inversely transforming (scaling) the transform coefficients. Residual samples can be inferred based on the inverse transform (transform) of the scaled transform coefficients. These details may also be applied/expressed in other parts of this disclosure.

[481] In the above-described embodiments, the methods are explained on the basis of flowcharts, using a sequence of steps or blocks, but the present disclosure is not limited to this order of steps or blocks, and a particular step may be performed in an order other than described above, or simultaneously with another stage. Additionally, one skilled in the art will appreciate that the steps shown in the flowchart above are not exclusive, that another step may be included, or that one or more steps in the flowchart may be deleted without impact to the scope of this disclosure.

[482] The above-described methods according to the present disclosure may be implemented in the form of software, and the encoding device and/or decoding device according to the present disclosure may be included in an image processing device such as a television, computer, smartphone, set-top box , display device, etc.

[483] When embodiments of the present disclosure are implemented in software, the above-described methods may be implemented as modules (processes, functions, etc.) to perform the above-described functions. Modules can be stored in memory and can be executed by the processor. The memory may be internal or external to the processor and may be coupled to the processor by various well known means. The processor may include an application specific integrated circuit (ASIC), other chipsets, logic circuitry, and/or data processing apparatus. The memory may include read only memory (ROM), random access memory (RAM), flash memory, memory card, storage medium, and/or other storage device. That is, the embodiments described in the present disclosure may be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional blocks shown in each drawing may be implemented and executed on a computer, processor, microprocessor, controller, or chip.

[484] In addition, the decoding device and encoding device to which the present disclosure applies may be included in a multimedia broadcast transceiver, mobile communication terminal, home theater device, digital cinema device, surveillance camera, video talking device, communication device in such as video communication, mobile streaming device, storage medium, camera recorder, video on demand (VoD) device, over-the-top (OTT) video delivery device, internet streaming device, three-dimensional (3D) device ) video, video teleconferencing device and medical video device, and can be used to process video signal and data signal. For example, an OTT video device may include a game console, Blu-ray player, Internet-enabled television, home theater system, smartphone, tablet PC, digital video recorder (DVR), and the like.

[485] Moreover, the processing method to which the present disclosure applies may be in the form of a computer-executable program and may be stored in a computer-readable recording medium. Multimedia data having a data structure in accordance with the present disclosure may also be stored in computer-readable recording media. The computer-readable recording medium includes all types of storage devices and distributed storage devices in which computer-readable data is stored. Computer-readable recording media may include, for example, Blu-ray disc (BD), universal serial bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical storage device data. Additionally, the computer-readable recording medium includes media implemented in the form of a carrier wave (eg, transmission over the Internet). In addition, the bit stream generated by the encoding method may be stored in a computer-readable recording medium or transmitted over wired or wireless communication networks. Additionally, embodiments of the present disclosure may be implemented as a computer program product through program codes, and the program codes may be executed on a computer by embodiments of the present disclosure. Program codes may be stored on a computer-readable medium.

Clause 1. A method for decoding an image performed by a decoding device, the method comprising:

receiving residual information from the bit stream;

deriving transformation coefficients for the current block by performing dequantization based on the residual information; And

deriving modified transform coefficients by applying LFNST to the transform coefficients,

wherein dequantization is performed based on a predefined scaling list, and

wherein whether the scaling list should be applied is inferred based on whether LFNST is applied and the tree type of the current block.

Statement 2. An image decoding method according to Position 1, in which, when the tree type of the current block is a single tree and the current block is a luma component, the scaling list is not applied, and when the tree type of the current block is a single tree and the current block is a chroma component, the scaling list is applied.

Position 3: A method for decoding an image according to Position 2, which further receives flag information indicating whether a scaling list is available when LFNST is executed.

Position 4. An image decoding method according to Position 3, in which, when the flag information indicates that the scaling list is not available and LFNST has an index greater than 0, the scaling list is not applied to the luma component.

Statement 5. An image decoding method according to Position 3, in which, when the flag information indicates that the scaling list is not available and LFNST has an index greater than 0, if the tree type of the current block is a dual chroma tree, the scaling list is not applied to the chroma component .

Position 6. An image decoding method according to Position 3, in which, when the flag information indicates that the scaling list is not available and LFNST has an index greater than 0, if the tree type of the current block is a dual luminance tree, the scaling list is not applied to the luminance component .

Position 7: A method for decoding an image according to Position 1, in which the current block contains a transform block.

Clause 8: An image encoding method performed by an image encoding device, the method comprising:

outputting discrete prediction samples for the current block;

deriving residual samples for the current block based on the prediction samples;

deriving conversion coefficients for the current block based on the primary conversion of residual discrete samples;

deriving modified conversion factors from conversion factors by applying LFNST; And

quantization of transform coefficients or modified transform coefficients,

wherein the quantization is performed based on a predefined scaling list, and

wherein whether the scaling list should be applied is inferred based on whether LFNST is applied and the tree type of the current block.

Statement 9. An image encoding method according to Position 8, in which, when the tree type of the current block is a single tree and the current block is a luminance component, the scaling list is not applied, and

wherein, when the tree type of the current block is a single tree and the current block is a chroma component, the scaling list is applied.

Position 10: An image encoding method according to Position 9 in which LFNST is not performed on the chrominance component of the current block.

Statement 11. An image encoding method according to Position 8, in which, when the tree type of the current block is a dual chroma tree and LFNST is executed on the current block, the scaling list is not applied to the chroma component.

Statement 12: An image encoding method according to Statement 8, wherein when the tree type of the current block is a dual luma tree and LFNST is executed on the current block, the scaling list is not applied to the luma component.

Regulation 13. An image encoding method according to Regulation 8, further comprising:

encoding and outputting flag information indicating whether the scaling list is available when LFNST is executed.

Statement 14: An image encoding method according to Position 8, in which the current block contains a transform block.

Claim 15: A computer-readable digital storage medium that stores instruction information to enable execution of an image decoding method, wherein the image decoding method comprises:

receiving residual information via a bit stream;

deriving transformation coefficients for the current block by performing dequantization based on the residual information; And

deriving modified transform coefficients by applying LFNST to the transform coefficients,

wherein dequantization is performed based on a predefined scaling list, and

wherein whether the scaling list should be applied is inferred based on whether LFNST is applied and the tree type of the current block.

Claims (48)

1. A method for decoding an image performed by a decoding device, the method comprising: obtaining a low frequency non-separable transform (LFNST) index for the current block, the LFNST index indicating that LFNST is applied to the current block; deriving dequantized transform coefficients for the current block by performing dequantization on the quantized transform coefficients for the current block; deriving residual samples for the current block by performing an inverse transform on the dequantized transform coefficients; wherein dequantization is performed based on a predetermined scaling matrix or a predetermined constant value, wherein whether the scaling matrix should be applied is determined based on the LFNST index and the tree type of the current block, in this case, the tree type of the current block is one of three predefined tree types, wherein predefined tree types include single tree, dual luma tree, dual chroma tree, wherein, in response to the case where the tree type of the current block is a single tree and the current block is a chroma component, it is determined that a scaling matrix is applied to the current block, and wherein, in response to the case where the tree type of the current block is a dual chroma tree, it is determined that the predetermined constant value is applied to the current block. 2. The image decoding method of claim 1, wherein, in response to the case where the tree type of the current block is single tree and the current block is a luminance component, it is determined that said predetermined constant value is applied to the current block. 3. The image decoding method as set forth in claim 2, wherein flag information indicating whether a scaling matrix is available when LFNST is executed is obtained from the bit stream. 4. The image decoding method of claim 3, wherein, in response to a case where the flag information indicates that the scaling matrix is not available and the index LFNST is greater than zero, it is determined that said predetermined constant value is applied to the luminance component. 5. The image decoding method according to claim 3, wherein, in response to the case where the flag information indicates that the scaling matrix is not available and the index LFNST is greater than zero, if the tree type of the current block is a dual luminance tree, it is determined that the aforementioned the specified constant value is applied to the brightness component. 6. The image decoding method according to claim 1, wherein the current block contains a transform block. 7. An image encoding method performed by an image encoding device, the method comprising: outputting discrete prediction samples for the current block; deriving residual samples for the current block based on the prediction samples; deriving conversion coefficients for the current block based on the primary conversion of residual discrete samples; deriving modified transform coefficients from transform coefficients by applying low frequency non-separable transform (LFNST); And quantization of transform coefficients or modified transform coefficients, this encodes the index LFNST indicating that LFNST is applied to the current block, wherein the quantization is performed based on a predetermined scaling matrix or a predetermined constant value, wherein whether the scaling matrix should be applied is determined based on the LFNST index and the tree type of the current block, in this case, the tree type of the current block is one of three predefined tree types, wherein predefined tree types include single tree, dual luma tree, dual chroma tree, wherein, in response to the case where the tree type of the current block is a single tree and the current block is a chroma component, it is determined that a scaling matrix is applied to the current block, and wherein, in response to the case where the tree type of the current block is a dual chroma tree, it is determined that the predetermined constant value is applied to the current block. 8. The image encoding method according to claim 7, wherein, in response to the case where the tree type of the current block is a single tree and the current block is a luminance component, it is determined that said predetermined constant value is applied to the current block. 9. The image encoding method of claim 8, wherein LFNST is not performed on the chrominance component of the current block. 10. The image encoding method of claim 7, wherein, in response to the case where the tree type of the current block is a dual luminance tree and LFNST is performed on the current block, it is determined that said predetermined constant value is applied to the luminance component. 11. The image encoding method of claim 7, further comprising encoding and outputting flag information indicating whether a scaling matrix is available when LFNST is executed. 12. The image encoding method according to claim 7, wherein the current block comprises a transform block. 13. A method for transmitting data for image information, comprising the steps of: outputs discrete prediction samples for the current block; outputting residual samples for the current block based on the prediction samples; output conversion coefficients for the current block based on the primary conversion of the residual discrete samples; deriving modified transform coefficients from the transform coefficients by applying a low frequency non-separable transform (LFNST); performing quantization of the transform coefficients or modified transform coefficients; encoding residual information related to the quantized transform coefficients to generate a bit stream; And transmit data containing a bit stream, this encodes the index LFNST indicating that LFNST is applied to the current block, wherein the quantization is performed based on a predetermined scaling matrix or a predetermined constant value, wherein whether the scaling matrix should be applied is determined based on the LFNST index and the tree type of the current block, in this case, the tree type of the current block is one of three predefined tree types, wherein predefined tree types include single tree, dual luma tree, dual chroma tree, wherein, in response to the case where the tree type of the current block is a single tree and the current block is a chroma component, it is determined that a scaling matrix is applied to the current block, and wherein, in response to the case where the tree type of the current block is a dual chroma tree, it is determined that said predetermined value constant is applied to the current block.