RU2781175C1 - Conversion for matrix internal prediction when encoding images - Google Patents

Abstract

FIELD: imaging technology.

SUBSTANCE: invention relates to means for encoding images. The type of internal forecasting for the current block is determined. Information of the type of internal forecasting for the current block is generated based on the type of internal forecasting. The forecast samples of the current block are extracted based on the type of internal forecasting. Residual samples of the current block are generated based on forecast samples. Conversion coefficients for the current block are extracted based on residual samples. The associated residual information is formed based on the conversion coefficients. Information such as internal forecasting and related residual information is encoded. The internal prediction type information includes a matrix of internal prediction (MIP) flag representing whether or not the MIP is applied to the current block. The associated residual information includes information of the low-frequency inseparable transformation index (LFNST) for the current block based on the MIP flag.

EFFECT: increase in the encoding efficiency.

13 cl, 15 dwg

Description

State of the art

The field of technology to which the invention belongs

[1] The present disclosure relates to image coding technology, and more specifically, to a transformation for matrix intra prediction in image coding.

Description of the Prior Art

[2] Recently, the demand for high-quality high-definition image/video, such as 4K, 8K, or even higher ultra-high definition (UHD) images/video, has been growing in various fields of technology. As the resolution or quality of images/video becomes higher, a relatively larger amount of information or number of bits is transmitted than for conventional image/video data. Therefore, if image/video data is transmitted via a medium such as an existing wired/wireless broadband line, or stored in a legacy storage medium, transmission and storage costs are greatly increased.

[3] In addition, there is growing interest and demand for virtual reality (VR) and artificial reality (AR) content and immersive multimedia such as holograms; and the broadcasting of images/videos showing image/video characteristics different from those of the actual image/video, such as game images/videos, is also on the rise.

[4] Therefore, a highly efficient image/video compression technology is required in order to efficiently compress and transmit, store or reproduce high-quality high-resolution images/videos exhibiting various characteristics as described above.

The essence of the invention

[5] According to an embodiment of the present document, a method and equipment for improving image/video coding efficiency is provided.

[6] According to an embodiment of the present document, a method and equipment for converting a block to which matrix intra prediction (MIP) is applied in image coding is provided.

[7] According to an embodiment of the present document, a method and equipment for signaling a transform index for a block to which MIP is applied is provided.

[8] According to an embodiment of the present document, a method and equipment for signaling a transform index for a block to which no MIP is applied is provided.

[9] According to an embodiment of the present document, a method and equipment for inferring a transformation index for a block to which MIP is applied is provided.

[10] According to an embodiment of the present document, a method and equipment for binarizing or encoding a transform index for a block to which MIP is applied is provided.

[11] According to an embodiment of the present document, a video/image decoding method by decoding equipment is provided.

[12] According to an embodiment of the present document, decoding equipment is provided to perform video/image decoding.

[13] According to an embodiment of the present document, a video/image encoding method is provided by encoding equipment.

[14] According to an embodiment of the present document, encoding equipment for performing video/image encoding is provided.

[15] According to an embodiment of the present document, a computer-readable digital storage medium is provided that stores encoded video/image information generated according to the video/image coding method disclosed in at least one embodiment of this document.

[16] According to an embodiment of the present document, a computer-readable digital storage medium is provided that stores encoded information or encoded video/image information instructing decoding equipment to perform the video/image decoding method disclosed in at least one embodiment of this document.

[17] According to the present document, the overall efficiency of image/video compression can be improved.

[18] According to the present document, a transform index for a block to which matrix intra prediction (MIP) is applied can be effectively signaled.

[19] According to the present document, a transform index for a block to which MIP is applied can be encoded efficiently.

[20] According to the present document, a transform index for a block to which MIP is applied can be inferred without separate signaling of the transform index.

[21] According to the present document, if MIP and a low frequency inseparable transform (LFNST) are applied, interference between them can be minimized, optimal coding efficiency can be maintained, and complexity can be reduced.

[22] The benefits that can be obtained through the detailed example of this document are not limited to the benefits listed above. For example, various technical effects may be envisaged, which may be understood or inferred by those skilled in the art from this document. Accordingly, the detailed benefits of this document are not limited to those expressly stated herein and may include various benefits that may be understood or inferred from the technical features of this document.

Brief description of the drawings

[23] FIG. 1 schematically illustrates an example of a video/image coding system to which this document is applicable.

[24] FIG. 2 is a diagram schematically explaining the configuration of video/image coding equipment to which this document is applicable.

[25] FIG. 3 is a diagram schematically explaining the configuration of video/image decoding equipment to which this document is applicable.

[26] FIG. 4 schematically illustrates a multiple conversion technology according to an embodiment of the present document.

[27] FIG. 5 roughly illustrates the intra-directional modes in 65 prediction directions.

[28] FIG. 6 and 7 are diagrams explaining RST according to an embodiment of the present document.

[29] FIG. 8 exemplifies context-adaptive binary arithmetic coding (CABAC) for encoding syntax elements.

[30] FIG. 9 is a diagram explaining the MIP for an 8×8 block.

[31] FIG. 10 is a flowchart explaining the method to which MIP and LFNST are applied.

[32] FIG. 11 and 12 schematically illustrate a video/image coding method and an example of related components according to the embodiment(s) of the present document.

[33] FIG. 13 and 14 schematically illustrate a video/image decoding method and an example of related components according to the embodiment(s) of the present document.

[34] FIG. 15 illustrates an example of a content streaming system to which the embodiments disclosed herein are applicable.

Detailed description of embodiments

[35] The present disclosure may be modified in various forms, and specific embodiments thereof are described and illustrated in the drawings. However, these embodiments are not intended to limit the disclosure. The terms used in the description below are used to simply describe specific embodiments, but are not intended to limit the disclosure. The singular expression includes the plural expression, as long as they are clearly interpreted differently. Terms such as "include" and "have" are intended to indicate that there are features, numbers, steps, operations, elements, components, or combinations of the foregoing used in the description below, and as such should be understood , which does not exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, elements, components, or combinations of the above.

[36] In addition, each configuration of the drawings described in this document is an independent illustration for explaining functions as features that differ from each other, and does not mean that each configuration is implemented by mutually different hardware or different software. For example, two or more patterns may be combined to form one pattern, and one pattern may also be split into multiple patterns. Without departing from the spirit of this document, embodiments in which configurations are combined and/or separated are included within the scope of the claims.

[37] Hereinafter, examples of the present embodiment are described in detail with reference to the accompanying drawings. In addition, like reference numerals are used to refer to like elements throughout the drawings, and like descriptions of like elements are omitted.

[38] This document relates to video/image coding. For example, the methods/embodiments disclosed in this document may relate to the Universal Video Coding (VVC) (ITU-T (Rec. (H.266)) standard, the next generation video/picture coding standard after VVC, or other related video coding standards (eg, High Efficiency Video Coding (HEVC) standard (ITU-T (Rec. (H.265), Fundamental Video Coding (EVC) standard, AVS2 standard, etc.).

[39] This document proposes various video/image coding embodiments, and the above embodiments may also be performed in combination with each other unless otherwise noted.

[40] In this document, video can mean a sequence of images over time. A frame generally means a unit representing one picture in a particular temporal movie frame, and a slice/tile means a unit constituting a part of a frame from an encoding point of view. A slice/tile may include one or more coding tree units (CTUs). One frame may consist of one or more slices/tiles. One frame may consist of one or more tile groups. One group of tiles may include one or more tiles.

[41] A pixel or pel can mean the smallest unit that makes up one frame (or image). In addition, "sample" can be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel/pixel value of a luma component, or only a pixel/pixel value of a chroma component.

[42] The unit may represent a basic image processing unit. The unit may include at least one of a specific area of the frame and information associated with the area. One unit may include one luminance block and two chrominance blocks (eg, Cb, Cr). The unit can be used interchangeably with terms such as block or zone in some cases. In general, an MxN block may include samples (or arrays of samples) or a set (or array) of transform coefficients of M columns and N rows. Alternatively, the sample may mean a pixel value in the spatial domain, and when such a pixel value is converted to the frequency domain, it can mean a transform factor in the frequency domain.

[43] In this document, the terms "/" and "," are to be interpreted as indicating "and/or". For example, the expression "A/B" can mean "A and/or B". Additionally, "A, B" may mean "A and/or B". Additionally, "A/B/C" may mean "at least one of A, B and/or C". In addition, "A/B/C" may mean "at least one of A, B and/or C".

[44] Additionally, in the document, the term "or" should be interpreted as indicating "and/or". For example, the expression "A or B" may contain 1) only A, 2) only B, and/or 3) both A and B. In other words, the term "or" in this document is to be interpreted as indicating "in addition or alternatively" .

[45] In the present description of the invention, "at least one of A and B" can mean "only A", "only B", or "both A and B". In addition, in the present description of the invention, the expression "at least one of A or B" or "at least one of A and/or B" can be interpreted identically to "at least one of A and B".

[46] Additionally, in the present description of the invention, "at least one of A, B and C" can mean "only A", "only B", "only C", or "any combination of A, B and C". In addition, "at least one of A, B or C" or "at least one of A, B and/or C" can mean "at least one of A, B and C".

[47] Additionally, the parentheses used in the present description of the invention may mean "for example". In particular, in the case where "prediction (internal prediction)" is expressed, it may be indicated that "internal prediction" is proposed as an example of "prediction". In other words, the term "prediction" in the present description of the invention is not limited to "intra prediction", and it may be indicated that "intra prediction" is offered as an example of "prediction". Additionally, even if "prediction (ie, intra prediction)" is expressed, it may be indicated that "intra prediction" is offered as an example of "prediction".

[48] In the present description of the invention, the technical features, separately explained in one drawing, may be implemented separately or may be implemented simultaneously.

[49] FIG. 1 illustrates an example of a video/image coding system to which the disclosure of this document may apply.

[50] Referring to FIG. 1, a video/image coding system may include a source device and a destination device. The source device may transmit the encoded information or video/image data to the receiving device via a digital storage medium or a network in the form of a file or streaming.

[51] The source device may include a video source, encoding equipment, and a transmitter. The receiver may include a receiver, decoding hardware, and a renderer. The encoding equipment may be referred to as "video/image coding equipment", and the decoding equipment may be referred to as "video/image decoding equipment". The transmitter may be included in the encoding equipment. The receiver may be included in the decoding equipment. The renderer may include a display, and the display may be configured as a separate device or an external component.

[52] A video source may acquire a video/image through a process of capturing, synthesizing, or generating a video/image. The video source may include a video/image capture device and/or a video/image generation device. The video/image capture device may include, for example, one or more cameras, video/image archives including previously captured videos/images, and the like. The video/image generation apparatus may include, for example, computers, tablet computers, and smartphones, and may (electronically) generate video/images. For example, the virtual video/image may be generated via a computer or the like. In this case, the video/image capturing process may be replaced by a linked data generation process.

[53] The encoding equipment may encode the input video/image. The encoding equipment may perform a series of procedures such as prediction, transformation, and quantization for compression and encoding efficiency. Encoded data (encoded video/picture information) may be output in the form of a bit stream.

[54] The transmitter may transmit information or coded image/image data output in the form of a bit stream to the receiver of the receiving device via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. The transmitter may include an element for generating a media file via a predetermined file format and may include an element for transmission via a broadcast/communication network. The receiver may receive/retrieve the bitstream and transmit the received bitstream to the decoding equipment.

[55] The decoding equipment can decode the video/image by performing a sequence of procedures such as dequantization, inverse transformation, and prediction corresponding to the operation of the encoding equipment.

[56] The rendering module may prepare a decoded video/image by rendering. The rendered video/image can be displayed through the display.

[57] FIG. 2 is a diagram principally illustrating the configuration of video/image coding equipment to which the disclosure of this document may apply. Hereinafter in this document, what is referred to as "video encoding equipment" may include image encoding equipment.

[58] Referring to FIG. 2, the encoding equipment 200 may include and be configured with an image segmentation module 210, a prediction module 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The prediction module 220 may include a mutual prediction and module 222 internal prediction. Residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. Residual processor 230 may further include a subtractor 231. Adder 250 may be referred to as a "rebuilder" or "recovered block generator". The image segmentation module 210, prediction module 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 as described above may be configured by one or more hardware components (e.g., chipsets or encoder processors) according to an embodiment. In addition, the storage device 270 may include a decoded frame buffer (DPB) and may also be configured by a digital storage medium. The hardware component may further include a storage device 270 as an internal/external component.

[59] The image segmentation module 210 may split the input image (or frame, movie frame) input to the encoding equipment 200 into one or more processing units. As an example, a processing unit may be referred to as a "coding unit (CU)". In this case, a coding unit may be recursively split according to a quadtree, binary tree, and ternary tree (QTBTTT) structure from a coding tree unit (CTU) or a largest coding unit (LCU). For example, one coding unit may be split into multiple deeper coding units based on a quadtree structure, a binary tree structure, and/or a ternary tree structure. In this case, for example, a quadtree structure is applied first, and a binary tree structure and/or a ternary tree structure may subsequently be applied. Alternatively, a binary tree structure may also be applied first. An encoding procedure according to the present disclosure may be performed based on a final coding unit that is no longer split. In this case, based on the coding efficiency according to the characteristics of images and the like, the maximum coding unit may be directly used as the final coding unit, or, if necessary, the coding unit may be recursively split into coding units of greater depth, so that a coding unit having optimal size, can be used as the final coding unit. Here, the encoding procedure may include a procedure such as prediction, transformation, and recovery, which is described below. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, each of the prediction unit and the transformation unit may be split or segmented from the above final coding unit. The prediction unit may be a selective prediction unit, and the transform unit may be a unit for inferring a transform coefficient and/or a unit for inferring a residual signal from a transform coefficient.

[60] A unit can be used interchangeably with a term such as block or zone in some cases. Typically, an MxN block may represent samples consisting of M columns and N rows, or a group of transform coefficients. A sample can generally represent a pixel or a pixel value, and can also represent only a pixel/pixel value of the luminance signal component, as well as represent only a pixel/pixel value of the chrominance signal component. Sample can be used as a term corresponding to a pixel or pel configuring one frame (or image).

[61] The subtractor 231 may generate a residual signal (residual block, residual samples, or residual sample array) by subtracting the prediction signal (predicted block, predictive samples, or predictive sample array), output from prediction unit 220 from the input image signal (original block, original samples or an array of original samples) and the generated residual signal is provided to the converter 232. Prediction module 220 may perform prediction on a target processing block (hereinafter referred to as "current block") and generate a prediction block including prediction samples for the current block. Prediction module 220 may determine whether or not intra prediction or inter prediction is applied, for the current block or in units of CUs. As described below in the description of each prediction mode, the prediction module may generate various kinds of prediction-related information, such as prediction mode information, and pass the generated information to the entropy encoder 240. The prediction information may be encoded in the entropy encoder 240 and output in the form bit stream.

[62] Intra prediction module 222 may predict the current block with reference to samples within the current frame. The referenced samples may be located as adjacent to the current block, or may also be located at a large distance from the current block according to the prediction mode. Prediction modes in intra prediction may include a plurality of non-directional modes and a plurality of directional modes. The non-directional mode may include, for example, a DC mode or a planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the fine prediction direction degree. However, this is illustrative, and directional prediction modes that are greater than or less than the above number may be used according to the setting. The intra prediction unit 222 may also determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[63] The inter-prediction unit 221 may infer a predicted block of the current block based on a reference block (reference sample array) indicated by a motion vector for the reference frame. At this time, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the adjacent block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0 prediction, L1 prediction, bi-prediction, and the like). In the case of inter prediction, the neighbor block may include a spatial neighbor block existing in the current frame and a temporal neighbor block existing in the reference frame. The reference frame including the reference block and the reference frame including the temporal neighbor block may be identical to each other or different from each other. The temporal neighbor block may be referred to as a "collocated reference block", "collocated CU (colCU)" or the like, and a reference frame including the temporal neighbor block may also be referred to as a "collocated frame ( colpic)". For example, the inter prediction unit 221 may configure a motion information candidate list based on neighboring blocks, and generate information indicating which candidate is used to derive the motion vector and/or the reference frame index of the current block. Inter prediction may be performed based on various prediction modes, and, for example, in the case of the skip mode and the join mode, the inter prediction unit 221 may use the neighboring block motion information as the current block motion information. In the case of the skip mode, the residual signal may not be transmitted, in contrast to the combine mode. The motion vector prediction (MVP) mode may indicate the motion vector of the current block by using the motion vector of an adjacent block as a motion vector predictor and signaling the motion vector difference.

[64] The prediction module 220 may generate a prediction signal based on various prediction methods described below. For example, the prediction module may not only apply intra prediction or inter prediction to predict one block, but also apply both intra prediction and inter prediction at the same time. This may be referred to as "combined mutual and intra prediction (CIIP)". In addition, the prediction module may perform intra-block copy (IBC) to predict a block. Intra-block copying can be used for encoding content images/moving images of a game and the like, such as screen content coding (SCC). The IBC essentially performs prediction in the current frame, but it can be performed similar to inter-prediction in which the reference block is retrieved in the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described herein.

[65] The predictive signal generated via the inter prediction module 221 and/or the intra prediction module 222 may be used to generate a recovered signal or generate a residual signal. Converter 232 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of discrete cosine transform (DCT), discrete sine transform (DST), graph-based transform (GBT), or conditionally non-linear transform (CNT). Here, GBT means a transformation obtained from a graph when relationship information between pixels is represented by a graph. CNT means the transformation obtained based on the predictive signal generated using all previously reconstructed pixels. In addition, the transformation process may be applied to square pixel blocks having the same size, or may be applied to blocks having a variable size rather than square.

[66] The quantizer 233 may quantize the transform coefficients and pass them to the entropy encoder 240, and the entropy encoder 240 may encode the quantized signal (information regarding the quantized transform coefficients) and output a bitstream. The information regarding the quantized transform coefficients may be referred to as "residual information". The quantizer 233 may repackage the block quantized transform coefficients into one-dimensional vector form based on the scan order of the coefficients, and generate information about the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form. Entropy encoder 240 may implement various coding methods such as exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like, for example. Entropy encoder 240 may encode information needed to reconstruct the video/images together with or separately from the quantized transform coefficients (eg, syntax element values, etc.). Encoded information (eg, encoded video/image information) may be transmitted or stored in units of a network abstraction layer (NAL) in the form of a bitstream. The video/image information may further include information regarding various parameter sets such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. Herein, signaling/transmitted information and/or syntax elements, which are described below, may be encoded via the above-described encoding procedure and included in a bitstream. The bitstream may be transmitted over a network or may be stored on a digital storage medium. Here, the network may include a broadcast network and/or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. . A transmitter (not illustrated), a transmitting signal output from the entropy encoder 240, and/or a storage module (not illustrated) storing the signal may be configured as an internal/external element of the encoding equipment 200, and alternatively, the transmitter may be included in the entropy encoder. 240.

[67] The quantized transform coefficients output from the quantizer 233 may be used to generate a predictive signal. For example, the residual signal (residual block or residual samples) may be recovered by applying dequantization and inverse transform to the quantized transform coefficients via dequantizer 234 and inverse transform 235. Summer 250 adds the recovered residual signal to the predictive signal output from predictor 220 to generate a reconstructed signal (recovered frame, recovered samples, or array of recovered samples). If there is no remainder for the target block to be processed, such as when the skip mode is applied, the predicted block may be used as the reconstructed block. The generated recovered signal may be used to intra-predict the next target block for processing in the current frame, and may be used to inter-predict the next frame through filtering as described below.

[68] Meanwhile, luminance chrominance scaling (LMCS) transform may be applied during the encoding and/or frame reconstruction process.

[69] The filter 260 can improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, filter 260 may generate a modified reconstructed frame by applying various filtering methods to the reconstructed frame and store the modified reconstructed frame in memory 270, namely the DPB of memory 270. Various filtering methods may include, for example, filtering to deblock , sampled adaptive offset (SAO), adaptive loop filter, bilateral filter, etc. The filter 260 may generate various kinds of information related to the filtering and pass the generated information to the entropy encoder 290 as described below in the description of each filtering method. Filtering related information may be encoded by an entropy encoder 290 and output in the form of a bit stream.

[70] The modified reconstructed frame transmitted to the memory 270 may be used as a reference frame in the inter prediction module 221 . When inter prediction is applied by the encoding equipment, prediction mismatch between the encoding equipment 200 and the decoding equipment can be eliminated, and encoding efficiency can be improved.

[71] The DPB of the memory 270 may store the modified reconstructed frame for use as a reference frame in the inter prediction module 221 . The memory 270 may store block motion information from which motion information in the current frame is extracted (or encoded) and/or motion information of blocks in the frame already recovered. The stored motion information may be transmitted to the inter-prediction unit 221 for use as spatial neighbor block motion information or temporal neighbor block motion information. The memory 270 may store the recovered samples of the recovered blocks in the current frame and may pass the recovered samples to the intra prediction module 222 .

[72] FIG. 3 is a diagram for schematically explaining the configuration of video/image decoding equipment to which the disclosure of the present document may apply.

[73] Referring to FIG. 3, the decoding equipment 300 may include and be configured with an entropy decoder 310, a residual processor 320, a prediction unit 330, an adder 340, a filter 350, and a memory 360. The prediction unit 330 may include an inter prediction unit 331 and an internal prediction unit 332. forecasting. Residual processor 320 may include a dequantizer 321 and inverse converter 322. Entropy decoder 310, residual processor 320, predictor 330, summer 340, and filter 350, as described above, may be configured by one or more hardware components (e.g., chipsets or decoder processors) according to the embodiment. Additionally, memory 360 may include a decoded frame buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include a storage device 360 as an internal/external component.

[74] When a bitstream including video/image information is input, the decoding equipment 300 may reconstruct an image in response to a process in which the video/image information is processed in the encoding equipment illustrated in FIG. 2. For example, decoding equipment 300 may extract units/blocks based on blocking-related information obtained from the bitstream. The decoding equipment 300 may perform decoding using the processing unit used for the encoding equipment. Therefore, the processing unit for decoding, for example, may be a coding unit, and the coding unit may be partitioned according to a quadtree structure, a binary tree structure, and/or a ternary tree structure from a coding tree unit or a maximum coding unit. One or more transformation units may be derived from a coding unit. In addition, the reconstructed image signal decoded and output by the decoding equipment 300 can be reproduced by the playback equipment.

[75] The decoding equipment 300 may receive a signal output from the encoding equipment of FIG. 2 in the form of a bitstream, and the received signal may be decoded via entropy decoder 310. For example, entropy decoder 310 may parse the bitstream to extract information (eg, video/image information) needed for image recovery (or frame recovery). The video/image information may further include information regarding various parameter sets such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The decoding equipment may further decode the frame based on the parameter set information and/or general constraint information. Signaling/receiving information and/or syntax elements described later in this document may be decoded, may be decoded by a decoding procedure, and obtained from a bit stream. For example, entropy decoder 310 decodes information in the bitstream based on an encoding method such as Exponential Golomb, CAVLC, or CABAC coding and output syntax elements required for image reconstruction and quantized transform coefficient values for the remainder. More specifically, the CABAC entropy decoding method can receive a bin corresponding to each syntax element in the bit stream, determine a context model by using the decoding target syntax element information, the decoding information of the decoding target block, or the symbol/bin information decoded in the previous step, and perform arithmetic decoding on the permission element by predicting the occurrence probability of the permission element according to the determined context model, and generating a character corresponding to the value of each syntax element. In this case, the CABAC entropy decoding method can update the context model by using the decoded symbol/bin information for the context model of the next symbol/bin after the context model is determined. Prediction-related information from the information decoded by the entropy decoder 310 may be provided to the prediction unit 330, and information regarding the residue for which entropy decoding is performed in the entropy decoder 310, i. quantized transform coefficients and related parameter information may be input to the dequantizer 321. In addition, information regarding filtering from the information decoded by the entropy decoder 310 may be provided to the filter 350. Meanwhile, a receiver (not illustrated) for receiving a signal output from the equipment encoding equipment may be further configured as an internal/external member of the decoding equipment 300, or a receiver may be a constituent element of the entropy decoder 310. Meanwhile, the decoding equipment according to the present document may be called "video/image/frame decoding equipment", and the decoding equipment can be classified into an information decoder (video/picture/frame information decoder) and a sampling decoder (video/picture/frame samples decoder). The information decoder may include an entropy decoder 310, and the selective decoder may include at least one of a dequantizer 321, an inverse transform 322, a predictor 330, an adder 340, a filter 350, and a memory 360.

[76] The dequantizer 321 may dequantize the quantized transform coefficients so as to output the transform coefficients. The dequantizer 321 may recompose the quantized transform coefficients in a two-dimensional block form. In this case, the reassembly may be performed based on the scan order of the coefficients performed by the encoding equipment. A dequantizer 321 may perform dequantization on the quantized transform coefficients using a quantization parameter (eg, quantization step size information) and obtain transform coefficients.

[77] The inverse transform 322 inversely transforms the transform coefficients to obtain a residual signal (residual block, array of residual samples).

[78] Prediction module 330 may perform prediction on the current block and generate a predicted block including the prediction samples of the current block. The prediction module may determine whether intra prediction is applied or inter prediction is applied to the current block based on the prediction information output from the entropy decoder 310 and determine a specific intra/inter prediction mode.

[79] The prediction module may generate a prediction signal based on various prediction methods described below. For example, the prediction module may not only apply intra prediction or inter prediction in order to predict one block, but also apply intra prediction and inter prediction at the same time. This may be referred to as "combined mutual and intra prediction (CIIP)". In addition, the prediction module may perform intra-block copy (IBC) to predict a block. Intra-block copying can be used for encoding content images/moving images of a game and the like, such as screen content coding (SCC). The IBC essentially performs prediction in the current frame, but it can be performed similar to inter-prediction in which the reference block is retrieved in the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described herein.

[80] The intra prediction module 332 may predict the current block by referring to the samples in the current frame. The reference samples may be located in the surroundings of the current block, or may be spaced apart from the current block according to the prediction mode. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit 332 may determine the prediction mode to be applied to the current block by using the prediction mode applied to the adjacent block.

[81] The inter-prediction unit 331 may derive a predicted block for the current block based on a reference block (reference sample array) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the adjacent block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include information regarding the direction of inter-prediction (L0 prediction, L1 prediction, bi-prediction, and the like). In the case of inter prediction, the neighbor block may include a spatial neighbor block existing in the current frame and a temporal neighbor block existing in the reference frame. For example, inter-prediction unit 331 may construct a list of motion information candidates based on adjacent blocks, and extract the current block's motion vector and/or reference frame index based on the received candidate selection information. Inter prediction may be performed based on various prediction modes, and the prediction information may include information indicative of the inter prediction mode for the current block.

[82] The adder 340 may generate a reconstructed signal (recovered frame, reconstructed block, or array of reconstructed samples) by summing the obtained residual signal with the predictive signal (predicted block or array of predicted samples) output from prediction module 330. If there is no remainder for the target processing block, such as when the skip mode is applied, the predicted block may be used as the reconstructed block.

[83] The adder 340 may be referred to as a "recovery module" or "recovered block generator". The generated recovered signal may be used for intra-prediction of the next block to be processed in the current frame, and as described below, may also be filtered out, or may also be used for inter-prediction of the next frame.

[84] Meanwhile, luma signal scaling (LMCS) transformation can also be applied in the frame decoding process.

[85] The filter 350 can improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, filter 350 may generate a modified reconstructed frame by applying various filtering methods to the reconstructed frame and store the modified reconstructed frame in memory 360, namely the DPB of memory 360. Various filtering methods may include, for example, filtering to deblock , sampled adaptive bias, adaptive loop filter, bilateral filter, etc.

[86] The (modified) reconstructed frame stored in the DPB of the memory 360 may be used as a reference frame in the inter prediction unit 331 . The memory 360 may store block motion information from which motion information in the current frame is retrieved (or decoded) and/or motion information of already reconstructed blocks in the frame. The stored motion information may be transmitted to the inter-prediction unit 331 so that it is used as spatial neighbor block motion information or temporal neighbor block motion information. The memory 360 may store the recovered samples of the recovered blocks in the current frame and pass the recovered samples to the intra prediction module 332 .

[87] In the present specification, the embodiments described in the prediction module 330, the dequantizer 321, the inverse transform 322, and the filter 350 of the decoding equipment 300 can also be applied in the same way or respectively to the predictor 220, the dequantizer 234, the inverse transform 235, and the filter 260 equipment 200 encoding.

[88] Meanwhile, as described above, when video encoding is performed, prediction is performed in order to improve the compression efficiency. Through this, a predicted block including prediction samples for the current block can be generated as a block to be encoded (ie, an encoding target block). Here, the prediction block includes prediction samples in the spatial domain (or pixel domain). The prediction block is retrieved identically in the encoding equipment and the decoding equipment, and the encoding equipment can signal information (residual information) about the residual between the original block and the predicted block, rather than the original sample value of the original block, to the decoding equipment, thereby improving efficiency. image encoding. The decoding equipment may extract a residual block including residual samples based on the residual information, sum the residual block and the predicted block to generate reconstructed blocks including the reconstructed samples, and generate a reconstructed frame including the reconstructed blocks.

[89] The residual information may be generated through a transformation and quantization procedure. For example, the encoding equipment may extract the residual block between the original block and the predicted block, perform a transform procedure on the residual samples (array of residual samples) included in the residual block to extract transform coefficients, perform a quantization procedure on the transform coefficients to extract quantized transform coefficients, and signal-related residual information to the decoding equipment (via the bitstream). Here, the residual information may include quantized transform coefficient value information, location information, transform technology, transform kernel, quantization parameter, and the like. The decoding equipment may perform a dequantization/inverse transform procedure based on the residual information and extract residual samples (or residual blocks). The decoding equipment may generate a reconstructed frame based on the predicted block and the residual block. In addition, for reference for inter-picture prediction later, the encoding equipment may also dequantize/inversely transform the quantized transform coefficients to extract a residual block and generate a reconstructed frame based on it.

[90] FIG. 4 schematically illustrates the multiple transform technology of the present document.

[91] Referring to FIG. 4, the converter may correspond to the converter in the encoding equipment of FIG. 2 as described above, and the inverse converter may correspond to the inverse converter in the encoding equipment of FIG. 2 or the inverse converter in the decoding equipment of FIG. 3 as described above.

[92] The converter can extract (primary) transform coefficients by performing primary transform based on the residual sample (residual sample array) in the residual block (S410). Such a primary transformation may be referred to as a "basic transformation". Here, the primary transform may be based on multiple transform selection (MTS), and in case the multiple transform is applied as the primary transform, it may be referred to as "multiple basic transform".

[93] For example, the multiple base transform may represent a transform method by further using discrete cosine transform (DCT) type 2 (DCT-II), discrete sine transform (DST) type 7 (DST-VII), DCT type 8 (DCT- VIII) and/or DST-type 1 (DST-I). Thus, the multiple base transform may represent a transform method for converting a spatial domain residual signal (or residual block) into frequency domain transform coefficients (or primary transform coefficients) based on a plurality of transform kernels selected from DCT type 2, DST type 7, DCT type 8 and DST type 1. Here, the primary transform coefficients may be referred to as "temporal transducer-side transform coefficients".

[94] In other words, in case the existing transform method is applied, the spatial domain transform for the residual signal (or residual block) into the frequency domain can be applied based on DCT type 2, and transform coefficients can be generated. However, in contrast, in case a multiple base transform is applied, the spatial domain transform for the residual signal (or residual block) to the frequency domain may be applied based on DCT type 2, DST type 7, DCT type 8, and /or DST type 1, and transform coefficients (or primary transform coefficients) may be generated. Here, DCT type 2, DST type 7, DCT type 8, and DST type 1 may be referred to as "transformation type", "transformation core", or "transformation base". Types of DCT/DST transformations can be specified based on basis functions.

[95] In case multiple base transform is performed, the vertical transform kernel and/or the horizontal transform kernel for the target block may be selected from among the transform kernels, the vertical transform for the target block may be performed based on the vertical transform kernel, and the horizontal transform for the target block can be executed on the basis of the horizontal transformation kernel. Here, the horizontal transform may represent a transform for the horizontal components of the target box, and the vertical transform may represent the transform for the vertical components of the target box. The vertical transform kernel/horizontal transform kernel may be adaptively determined based on the prediction mode and/or transform index of the target block (CU or subblock) including the residual block.

[96] Further, for example, in the case of performing primary transformation by applying the MTS, specific basis functions can be configured as the specified values, and in the case of vertical transformation or horizontal transformation, the transformation relationship for the transformation kernel can be configured by combining which basis functions apply. For example, in the case where the horizontal direction transformation kernel is represented by trTypeHor and the vertical direction transformation kernel is represented by trTypeVer, trTypeHor or trTypeVer having a value of 0 can be configured as DCT2, and trTypeHor or trTypeVer having a value of 1 can be configured as DCT7 ; trTypeHor or trTypeVer having a value of 2 may be configured as DCT8.

[97] Additionally, for example, to indicate any of several sets of transform kernels, the MTS index may be encoded, and the MTS index information may be signaled to the decoding equipment. Here, the MTS index may be represented as a tu_mts_idx syntax element or an mts_idx syntax element. For example, if the MTS index is 0, it may represent that the trTypeHor and trTypeVer values are 0, and if the MTS index is 1, it may represent that the trTypeHor and trTypeVer values are 1. If the MTS index is 2, it may represent that the trTypeHor value is 2 and the trTypeVer value is 1, and if the MTS index is 3, it may represent that the trTypeHor value is 1 and the trTypeVer value is 2. If the MTS index is 4, it may represent that the values of trTypeHor and trTypeVer are 2. For example, the set of transformation kernels according to the MTS index may be represented as indicated in the following table.

[98] Tab. one

[99] The converter can extract modified (secondary) transform coefficients by performing secondary transform based on the (primary) transform coefficients (S420). The primary transform may be a spatial-to-frequency domain transform, and the secondary transform may be a transform to a more compressive expression by using the correlation existing between the (primary) transform coefficients.

[100] For example, the secondary transform may include an inseparable transform. In this case, the secondary transform may be referred to as "non-separable secondary transform (NSST)" or "mode-dependent non-separable secondary transform (MDNSST)". The non-separable second transform may represent a transform for generating modified transform coefficients (or secondary transform coefficients) for the residual signal by second transforming the (primary) transform coefficients derived through the primary transform based on the non-separable transform matrix. Here, the vertical transformation and the horizontal transformation may not be applied separately (or independently) with respect to the (primary) transformation coefficients based on the non-separable transformation matrix, but may be applied simultaneously.

[101] In other words, an inseparable secondary transform may be a transform method for recomposing, for example, two-dimensional signals (transform coefficients) into a one-dimensional signal through a specifically defined direction (for example, a row-first direction or a column-first direction), without separating the coefficients (primary ) transforming into vertical components and horizontal components, and then generating modified transform coefficients (or secondary transform coefficients) based on an inseparable transform matrix.

[102] For example, the direction (or order) row first may represent the layout of the MxN block in line in the order "first row - Nth row", and the direction (or order) column first may represent the layout of the MxN block in line in the order "first column - Mth column". Here, M and N may represent the width (W) and height (H) of the box, and may be positive integers.

[103] For example, an inseparable secondary transform may be applied to the top left region of a block consisting of (primary) transform coefficients (hereinafter, the transform coefficient block). For example, if the width (W) and height (H) of the transform coefficient block are equal to or greater than 8, an 8×8 inseparable secondary transform may be applied to the upper left 8×8 region of the transform coefficient block. Additionally, if the width (W) and height (H) of the transform coefficient block are equal to or greater than 4 and less than 8, a 4x4 inseparable secondary transform may be applied to the upper left zone min(8, W) x min(8, H) of the coefficient block transformations. However, the embodiments are not limited to this, and for example, even if such a condition that the width (W) and height (H) of the transform coefficient block is equal to or greater than 4 is satisfied, a 4×4 inseparable secondary transform can be applied to the left upper zone min(8, W) x min(8, H) of the transform coefficient block.

[104] Specifically, for example, in the case where a 4×4 input block is used, the non-separable secondary transform may be performed as follows.

[105] Input block X 4×4 can be represented as follows.

[106] Equation 1

[107] For example, a vector shape X can be represented as follows.

[108] Equation 2

может представлять вектор X, и двумерный блок X в уравнении 1 может перекомпоновываться и представляться в качестве одномерного вектора в соответствии с порядком сначала по строкам.[109] Referring to Equation 2,

may represent a vector X, and the two-dimensional block X in Equation 1 may be rearranged and represented as a one-dimensional vector according to row first order.

[110] In this case, the secondary inseparable transformation can be calculated as follows.

[111] Equation 3

может представлять вектор коэффициентов преобразования, и T может представлять матрицу (неразделимого) преобразования 16×16.[112] Here,

may represent a vector of transform coefficients, and T may represent a 16×16 (inseparable) transform matrix.

, имеющий размер 16×1, может извлекаться, и

может реорганизовываться в качестве блока 4×4 через порядок сканирования (горизонтальный, вертикальный или диагональный). Тем не менее, вышеописанное вычисление является примерным, и чтобы уменьшать вычислительную сложность неразделимого вторичного преобразования, гиперкубическое преобразование Гивенса (HyGT) и т.п. может использоваться для того, чтобы вычислять неразделимое вторичное преобразование.[113] Based on Equation 3,

, having a size of 16×1, can be extracted, and

can be reorganized as a 4×4 block through the scan order (horizontal, vertical or diagonal). However, the above calculation is exemplary, and in order to reduce the computational complexity of the inseparable secondary transform, the Givens Hypercubic Transform (HyGT) and the like. can be used for computing an inseparable secondary transformation.

[114] Meanwhile, in an inseparable secondary transform, a transform kernel (either a transform base or a transform type) may be selected in a mode-dependent manner. Here, the mode may include an intra prediction mode and/or an inter prediction mode.

[115] For example, as described above, the NSST may be performed based on an 8×8 transform or a 4×4 transform determined based on the width (W) and height (H) of the transform coefficient block. For example, if W and H are equal to or greater than 8, the 8×8 transform may represent a transform that can be applied to the 8×8 zone included in the corresponding transform coefficient block, and the 8×8 zone may represent the top left 8×8 zone in corresponding block of transformation coefficients. Further, similarly, if W and H are equal to or greater than 4, the 4×4 transform may represent a transform that can be applied to the 4×4 zone included in the corresponding transform coefficient block, and the 4×4 zone may represent the upper left 4× zone. 4 in the corresponding transform coefficient block. For example, an 8x8 transform kernel matrix may be a 64x64/16x64 matrix, and a 4x4 transform kernel matrix may be a 16x16/8x16 matrix.

[116] In this case, for mode-based transform core selection, two non-separable sub-transform cores per transform set for non-separable secondary transform can be configured with respect to all of 8×8 transform and 4×4 transform, and four can be provided. conversion set. Thus, four transform sets can be configured with respect to the 8×8 transform, and four transform sets can be configured with respect to the 4×4 transform. In this case, each of the four transform sets for 8×8 transform may include two 8×8 transform kernels, and each of the four transform sets for 4×4 transform may include two 4×4 transform kernels.

[117] However, the size of the transform sub-block, the number of sets, and the number of transform cores in the set are exemplary, and a size other than 8×8 or 4×4 may be used, or n sets may be configured and k transform cores may be included in each set. Here, n and k can be positive integers.

[118] For example, a transform set may be referred to as an "NSST set", and a transform core in an NSST set may be referred to as an "NSSAT core". For example, selection of a particular set of sets for transformation may be performed based on the intra prediction mode of the target block (CU or sub-block).

[119] For example, the intra prediction mode may include two non-directional or non-angular intra prediction modes and 65 directional or angular intra prediction modes. The non-directional intra prediction modes may include a planar intra prediction mode number 0 and a DC intra prediction mode number 1, and the directional intra prediction modes may include 65 intra prediction modes (numbers 2-66). However, this is exemplary, and the embodiment according to the present document can be applied even to the case in which a different number of intra prediction modes are provided. Meanwhile, in some cases, the intra prediction mode number 67 may additionally be used, and the intra prediction mode number 67 may represent a linear model (LM) mode.

[120] FIG. 5 roughly illustrates the intra-directional modes in 65 prediction directions.

[121] Referring to FIG. 5, the modes can be divided into intra prediction modes having a horizontal directionality and intra prediction modes having a vertical directionality with respect to the intra prediction mode number 34 having a left upper diagonal prediction direction. In FIG. 5, H and V may denote horizontal orientation and vertical orientation, respectively, and the numbers -32 to 32 may represent offsets in units of 1/32 for a position on the sample grid. It may represent an offset for the mode index value.

[122] For example, intra prediction modes 2-33 may be horizontal, and intra prediction modes 34-66 may be vertical. Meanwhile, from a technical point of view, the intra prediction mode number 34 can be considered to have neither horizontal directionality nor vertical directionality, but can be classified as belonging to horizontal directionality from the point of view of defining a transformation set for secondary transformation. This is because the input data is transposed and used with respect to the vertical directional modes symmetrical with respect to the intra prediction mode number 34, and the layout method of the input data for the horizontal directional mode is used with respect to the intra prediction mode number 34. Here, the transposition of the input data may mean the NxM data configuration in such a way that rows become columns and columns become rows with respect to MxN two-dimensional block data.

[123] Additionally, the intra prediction mode number 18 and the intra prediction mode number 50 may represent the horizontal intra prediction mode and the vertical intra prediction mode, respectively, and the intra prediction mode number 2 may be referred to as the "right upper diagonal intra prediction mode" because the prediction is performed in the direction up and to the right with the left reference pixel. In the same context, intra prediction mode number 34 may be referred to as "right lower diagonal intra prediction mode", and intra prediction mode number 66 may be referred to as "left lower diagonal intra prediction mode".

[124] Meanwhile, if it is determined that a specific set is used for a non-separable transform, one of the k transform kernels in the specific set may be selected through the non-separable secondary transform index. For example, the encoding equipment may derive a non-separable second transform index representing a particular transform kernel based on a rate-distortion (RD) test, and may signal the non-separable second transform index to the decoding equipment. For example, the decoding hardware may select one of the k transform cores in a particular set based on the non-separable secondary transform index. For example, an NSST index having a value of 0 may represent a first non-separable subtransform kernel, an NSST index of 1 may represent a second non-separable subtransform kernel, and an NSST index of 2 may represent a third non-separable subtransform kernel. . Alternatively, an NSST index having a value of 0 may represent that the first non-separable secondary transform is not applied to the target block, and an NSST index having a value of 1-3 may indicate three transformation kernels as described above.

[125] The transformer may perform an inseparable secondary transform based on the selected transform kernels and may obtain modified (secondary) transform coefficients. The modified transform coefficients may be extracted as quantized transform coefficients via the above-described quantizer, and may be signal-coded to the decoding equipment, and may be transmitted to a dequantizer/inverse converter in the encoding equipment.

[126] Meanwhile, if the secondary transform is omitted as described above, the (primary) transform coefficients, which are the outputs of the primary (separable) transform, can be extracted as quantized transform coefficients via a quantizer as described above, and can be encoded with the possibility be signaled to the decoding equipment and may be sent to a dequantizer/inverse converter in the encoding equipment.

[127] Referring again to FIG. 4, the reverse converter may perform the sequence of procedures in reverse order of the procedures performed by the above-described converter. The inverse transform can receive (dequantized) transform coefficients, extract (primary) transform coefficients by performing a secondary (inverse) transform (S450), and obtain a residual block (residual samples) by performing a primary (inverse) transform with respect to the (primary) transform coefficients (S460) . Here, the primary transform coefficients may be referred to as "modified transform coefficients on the inverter side". As described above, the encoding equipment and/or the decoding equipment may generate a reconstructed block based on the residual block and the predicted block, and may generate a reconstructed frame based on them.

[128] Meanwhile, the decoding equipment may further include a secondary inverse transform application/non-application determination unit (or an element for determining whether or not to apply the secondary inverse transform), and a secondary inverse transformation determination unit (or an element for determining the secondary inverse transform). transformations). For example, the secondary inverse transform application/non-application determination module may determine whether or not the secondary inverse transformation should be applied. For example, the secondary inverse transform may be NSST or RST, and the secondary inverse transform apply/non-apply determination module may determine whether or not the secondary inverse transform should be applied based on the secondary transform flag parsed or obtained from the bitstream. Alternatively, for example, the second inverse transform application/non-application determination module may determine whether or not the secondary inverse transform should be applied based on the transform coefficient of the residual block.

[129] The secondary inverse transform determination module may determine the secondary inverse transform. In this case, the secondary inverse transform determination module may determine the secondary inverse transform applied to the current block based on the set for the NSST (or RST) transform specified according to the intra prediction mode. Alternatively, the method for determining the secondary transform may be determined depending on the method for determining the primary transform. Alternatively, different combinations of the primary transform and the secondary transform may be determined according to the intra prediction mode. For example, the secondary inverse transform determination module may determine the area to which the secondary inverse transform is applied based on the size of the current block.

[130] Meanwhile, if the secondary (inverse) transform is omitted as described above, the residual block (residual samples) can be obtained by receiving the (dequantized) transform coefficients and performing the primary (separable) inverse transform. As described above, the encoding equipment and/or the decoding equipment may generate a reconstructed block based on the residual block and the predicted block, and may generate a reconstructed frame based on them.

[131] Meanwhile, herein, in order to reduce the computational amount and the required storage space caused by the inseparable secondary transform, a reduced secondary transform (RST) having a reduced transform matrix (kernel) size can be applied to the NSST concept.

[132] As used herein, RST may mean a (simplified) transformation performed on the residual samples for a target block based on a transformation matrix whose size is reduced in accordance with the simplification factor. If this is done, the amount of computation required during the transformation may be reduced due to the reduction in the size of the transformation matrix. Thus, RST can be used to solve the problem of computational complexity arising during the transformation of a block having a large size or non-separable transformation.

[133] For example, RST may be referred to by various terms such as "reduced transformation", "reduced secondary transformation", "reduced transformation", "simplified transformation", or "simple transformation", and the names by which the RST is called are not limited to the examples listed. Further, RST is mainly performed in the low frequency region including coefficients that are not 0 in the transform block, and thus may be referred to as "low frequency inseparable transform (LFNST)".

[134] Meanwhile, in the case where the secondary inverse transform is performed based on the RST, the inverse transformer 235 of the encoding equipment 200 and the inverse transformer 322 of the decoding equipment 300 may include an inverse RST module extracting modified transform coefficients based on the inverse RST of the transform coefficients. , and an inverse primer extracting residual samples for the target block based on the inverse primer for the modified transform coefficients. Reverse primary means the reverse of the primary transform applied to the remainder. As used herein, extracting transform coefficients based on a transform may mean extracting transform coefficients by applying an appropriate transform.

[135] FIG. 6 and 7 are diagrams explaining RST according to an embodiment of the present document.

[136] For example, FIG. 6 may be a drawing explaining that a direct shorthand transformation is applied, and FIG. 7 may be a drawing explaining that an inverse shorthand transformation is applied. Here, a target block may represent a current block, a residual block, or a transform block, depending on what is being encoded.

[137] For example, in RST, an N-dimensional vector may be transformed into an R-dimensional vector located in another space, and a shorthand transformation matrix may be determined. Here, N and R may be positive integers, and R may be less than N. N may mean the square of the length of one side of the block to which the transform is applied, or the total number of transform coefficients corresponding to the block to which the transform is applied, and the simplification factor may mean R/N value. The simplification factor may be referred to by various terms such as "reduced factor", "reduction factor", "simplified factor", or "simple factor". Meanwhile, R may be referred to as a "reduced factor", and in some cases, the simplification factor may mean R. Additionally, in some cases, the simplification factor may mean an N/R value.

[138] For example, the simplification factor or reduced factor may be signaled via the bitstream, but is not limited to this. For example, preset values for the simplifying factor or the reduced factor may be stored in the encoding equipment 200 and the decoding equipment 300, in which case, the simplifying factor or the reduced factor may not be signaled separately.

[139] For example, the size (RxN) of the simplified transformation matrix may be smaller than the size (NxN) of the regular transformation matrix, and may be given in the following equation.

[140] Equation 4

[141] For example, the matrix T in the reduced transform block illustrated in FIG. 6 may represent the TRxN matrix of Equation 4. As shown in FIG. 6, in case the residual samples for the target block are multiplied by the simplified transform matrix TRxN, the transform coefficients for the target block can be extracted.

[142] For example, in the case where the block size to which the transform is applied is 8×8 and R is 16 (i.e., R/N=16/64=1/4), the RST of FIG. 6 may be expressed by a matrix operation as indicated in Equation 5 below. In this case, the memory and the multiplication operation may be reduced to about 1/4 by the simplification factor.

[143] As used herein, a matrix operation can be understood as an operation to obtain a column vector by placing a matrix to the left of the column vector and multiplying the matrix and the column vector.

[144] Equation 5

[145] In Equation 5, r1-r64 may represent the residual samples for the target block. Alternatively, for example, they may be transform coefficients generated by applying a primary transform. Based on the operation result of Equation 5, the conversion coefficients ci for the target block can be retrieved.

[146] For example, if R is 16, the transform coefficients c1-c16 for the target block may be extracted. If a transformation matrix having a size of 64×64 (NxN) is multiplied by residual samples having a size of 64×1 (Nx1) through applying a regular transformation rather than RST, 64 (N) transformation coefficients for the target block can be extracted, but since RST is applied, only 16 (N) transform coefficients for the target block can be extracted. Since the total number of transform coefficients for a target block is reduced from N to R, the amount of data that the encoding equipment 200 transmits to the decoding equipment 300 can be reduced, and therefore the transmission efficiency between the encoding equipment 200 and the decoding equipment 300 can be improved.

[147] Considering the size of the transformation matrix, since the size of the regular transformation matrix is 64×64 (NxN), and the size of the simplified transformation matrix is reduced to 16×64 (RxN), the memory usage when performing RST can be reduced in terms of R/N by compared to the case in which a regular transformation is performed. Additionally, compared to the number (NxN) of multiplications when using a regular transformation matrix, the use of a simplified transformation matrix can reduce the number of multiplications (RxN) with respect to R/N.

[148] In an embodiment, the transformer 232 of the encoding equipment 200 may derive transform coefficients for a target block by performing a primary transform and a secondary transform based on RST on the residual samples for the target block. The transform coefficients may be passed to an inverse transform of the decoding equipment 300, and the inverse transformer 322 of the decoding equipment 300 may derive modified transform coefficients based on an inverse reduced secondary transform (RST) for the transform coefficients, and may derive residual samples for a target block based on an inverse primary transform of the modified coefficients. transformations.

[149] The size of the TNxR inverse RST matrix according to an embodiment may be NxR, which is smaller than the NxN size of the regular inverse transform matrix, and may have a transposed relationship with the simplified transform matrix TRxN illustrated in Equation 4.

[150] The matrix Tt in the reduced inverse transform block illustrated in FIG. 7 may represent an inverse RST matrix of TRxNT. Here, the superscript T may represent a transposition. As shown in FIG. 7, in case the transform coefficients for the target block are multiplied by the inverse RST matrix TRxNT, modified transform coefficients for the target block or residual samples for the target block can be extracted. The inverse RST matrix TRxNT can be expressed as (TRxN)TNxR.

[151] More specifically, in the case where an inverse RST is applied as a secondary inverse transform, the modified transform coefficients for the target block may be derived by multiplying the transform coefficients for the target block by the inverse RST matrix TRxNT. Meanwhile, the inverse RST may be applied as an inverse primary transform, in which case, the residual samples for the target block may be extracted when the inverse RST matrix TRxNT is multiplied by the transform coefficients for the target block.

[152] In an embodiment, in the case where the block size to which the inverse transform is applied is 8x8 and R is 16 (i.e., R/N=16/64=1/4), the RST of FIG. 7 can be expressed by a matrix operation as indicated in Equation 6 below.

[153] Equation 6

[154] In Equation 6, c1-c16 may represent transform coefficients for the target block; rj representing the modified transform coefficients for the target block or residual samples for the target block may be derived based on the result of the operation of Equation 6. Thus, r1-rN representing the modified transform coefficients for the target block or residual samples for the target block may be derived.

[155] Considering the size of the inverse transform matrix, since the size of the regular inverse transform matrix is 64×64 (NxN) and the size of the simplified inverse transform matrix is reduced to 64×16 (NxR), the memory usage when performing the inverse RST may be reduced in relation to R/N compared to the case in which a regular inverse transformation is performed. Additionally, compared to the number (NxN) of multiplications when using a regular inverse transform matrix, using a simplified inverse transform matrix can reduce the number (NxR) of multiplications in terms of R/N.

[156] Meanwhile, conversion sets can be configured and applied even with respect to RST 8×8. Thus, the corresponding RST 8x8 can be applied according to the conversion set. Since one transform set is composed of two or three transform kernels according to the intra prediction mode, it can be configured to select one of four transforms, which at most includes even the case in which the secondary transform is not applied. In a transformation where no secondary transformation is applied, the identity matrix can be considered to be applied. If it is assumed that index 0, 1, 2, or 3 is given for four transformations (for example, index number 0 may be allocated to the case in which the identity matrix, i.e., the secondary transformation, is not applied), the transformation to be applied may be denoted by signaling a syntax element that is an NSST index to each block of transform coefficients. Thus, through the NSST index, NSST 8x8 can be denoted for the top left 8x8 block, and in the RST configuration, RST 8x8 can be denoted. NSST 8×8 and RST 8×8 may represent transforms capable of being applied to the 8×8 zone included in the corresponding transform coefficient block in case the W and H of the target block that becomes the transform target is equal to or greater than 8, and the zone is 8 x8 may represent the top left 8x8 zone in the corresponding transform coefficient block. Similarly, NSST 4x4 and RST 4x4 may represent transforms capable of being applied to a 4x4 zone included in the corresponding transform coefficient block in case W and H of the target block are equal to or greater than 4, and the 4x4 zone may represent is the top left 4×4 zone in the corresponding transform coefficient block.

[157] Meanwhile, for example, the encoding equipment can extract a bit stream by encoding the value of a syntax element or quantized transform coefficient values for the remainder based on various encoding methods such as exponential Golomb code, context-adaptive variable length coding (CAVLC), context-sensitive adaptive binary arithmetic coding (CABAC), and the like. Additionally, the decoding equipment can extract the value of the syntax element or the quantized transform coefficient values for the residue based on various encoding methods such as Exponential Golomb encoding, CAVLC, CABAC, and the like.

[158] For example, the above-described encoding methods can be performed as content, which is described below.

[159] FIG. 8 exemplifies context-adaptive binary arithmetic coding (CABAC) for encoding a syntax element.

[160] For example, in the CABAC encoding process, if the input signal is a syntax element that is not a binary value, the value of the input signal can be converted to a binary value through binarization. Additionally, if the input signal is already a binary value (ie, if the value of the input signal is a binary value), binarization may not be performed and the input signal may be used as is. Here, each binary number 0 or 1 constituting a binary value may be referred to as a "permission element". For example, if the binary string after binarization is 110, each of 1, 1, and 0 can be represented as one permission element. The permission element(s) for one syntax element may represent the value of the syntax element. The binarization may be based on a different binarization method such as a truncated Rice code binarization process or a fixed length code binarization process, and the binarization method for the target syntax element may be preset. The binarization procedure may be performed by a binarization module in the entropy encoder.

[161] Thereafter, the binarized syntax element sample elements may be input to a regular encoding engine or a bypass encoding engine. The regular coding engine of the encoding equipment may extract a context model that reflects a probability value with respect to the corresponding bin, and encode the corresponding bin based on the extracted context model. The encoding hardware's regular encoding mechanism may update the context model for the corresponding permission element after encoding is performed on the corresponding permission elements. Permission elements encoded as the content described above may be represented as context-encoded permission elements.

[162] Meanwhile, in case the binarized syntax element sample elements are input to the bypass encoding mechanism, they may be encoded as follows. For example, the bypass encoding mechanism of the encoding equipment may omit a procedure for estimating a probability with respect to an input bin and a procedure for updating a probability model applied to a bin after encoding. In the case that bypass coding is applied, the encoding equipment may encode the input bin by applying a regular probability distribution instead of extracting a context model, and thereby the encoding rate may be increased. The permission element encoded as the content described above may be presented as a bypass permission element.

[163] Entropy decoding may represent a process for performing a process identical to the above-described entropy encoding in reverse order.

[164] The decoding equipment (entropy decoder) can decode the encoded image/video information. The image/video information may include segmentation-related information, prediction-related information (e.g., inter-/intra prediction split information, intra-prediction mode information, inter-prediction mode information, and the like), residual information, or in-loop filtering related information. information or may include various syntactic elements from it. Entropy encoding may be performed in units of a syntax element.

[165] The decoding hardware may binarize the target syntax elements. Here, the binarization may be based on various binarization methods such as a truncated Rice code binarization process or a fixed length code binarization process, and the binarization method for the target syntax element may be previously set. The decoding hardware can extract the available bin strings (possible bin strings) for the available target syntax element values via a binarization procedure. The binarization procedure may be performed by a binarization module in the entropy decoder.

[166] The decoding hardware may compare the extracted bin string with the available bin strings for the corresponding syntax elements when sequentially decoding or parsing the corresponding bins for target syntax elements from the input bit(s) in the bitstream. If the extracted permission element string is equal to one of the available permission element strings, the value corresponding to the corresponding permission element string is retrieved as the value of the corresponding syntax element. If not, the decoding equipment may re-execute the above procedure after further parsing the next bit in the bitstream. Through such a process, it is possible to signal specific information (or a specific syntax element) in a bit stream using a variable length bit even without using a start bit or end bit of the corresponding information. Through this, a relatively smaller bit can be allocated with respect to a smaller value, and thereby the overall coding efficiency can be improved.

[167] The decoding equipment may perform decoding based on context models or bypass decoding the corresponding bins in a bin string from a bit stream based on an entropy coding technology such as CABAC or CAVLC.

[168] In case the syntax element is decoded based on the context model, the decoding equipment may receive a permission element corresponding to the syntax element via a bit stream, may determine the context model by using the syntax element, and decoding the decoding target block or neighbor block information or character information / bins decoded in the previous step, and can extract the value of the syntax element by performing arithmetic decoding of the bin by predicting the occurrence probability of the received bin according to the determined context model. Thereafter, the context model of the next decodable permission element may be updated based on the determined context model.

[169] The context model may be allocated and updated by context-encoded (regularly encoded) permission elements, and the context model may be specified based on a context index (ctxIdx) or a context index increment (ctxInc). CtxIdx can be derived based on ctxInc. In particular, for example, a ctxIdx representing the context model for each of the regularly encoded permission elements can be extracted by the sum of ctxInc and the context index offset (ctxIdxOffset). For example, ctxInc may be retrieved differently via permission elements. ctxIdxOffset can be represented as the smallest value of ctxIdx. Typically, ctxIdxOffset may be a value used to distinguish it from context models for other syntax elements, and the context model for one syntax element may be separated or derived based on ctxInc.

[170] In the entropy encoding procedure, it may be determined whether to perform encoding through a regular encoding mechanism or to perform encoding through a bypass encoding mechanism, and accordingly, the encoding path may be switched. Entropy decoding may perform the same process as entropy encoding in reverse order.

[171] Meanwhile, for example, in the case where the bypass decoding of the syntax element is performed, the decoding equipment may receive a bin corresponding to the syntax element via the bitstream, and may decode the input bin by applying a regular probability distribution. In this case, the decoding hardware may omit the syntax element context model extraction procedure and the context model update procedure applied to the permission element after decoding.

[172] As described above, the residual sample may be extracted as quantized transform coefficients through transform and quantization processes. The quantized transform coefficients may be referred to as "transform coefficients". In this case, the transform coefficients in the block may be signaled in the form of residual information. The residual information may include a syntax or syntax element regarding the residual encoding. For example, the encoding equipment may encode the residual information and may output it in the form of a bitstream, and the decoding equipment may decode the residual information from the bitstream and may extract the residual (quantized) transform coefficients. As described below, the residual information may include syntax elements representing whether or not a transform is applied to the corresponding block, where the location of the last effective transform coefficient in the block is, whether or not an effective transform coefficient exists in the sub-block, or what is the size/sign of the effective conversion factor.

[173] Meanwhile, for example, the prediction unit in the encoding equipment of FIG. 2 or the predictor in the decoding equipment of FIG. 3 can perform intra prediction. In the following, the internal prediction is described in more detail as follows.

[174] Intra prediction may represent prediction for generating predictive samples for the current block based on reference samples in a frame (hereinafter, "current frame") to which the current block belongs. In case intra prediction is applied to the current block, adjacent reference samples to be used for intra prediction of the current block may be extracted. Adjacent reference samples of the current block may include the sample adjacent to the left boundary of the current block having size nWxnH and the sum of 2xnH samples adjacent to the lower left, the samples adjacent to the upper boundary of the current block, and the sum of 2xnW samples adjacent to the right the top part, and one selection bordering the left top part relative to the current block. Alternatively, the adjacent reference samples of the current block may include a top multiple column neighbor sample and a left multiple row neighbor sample. Alternatively, the adjacent reference samples of the current block may include the sum of the nH samples adjacent to the right boundary of the current block having size nWxnH, the sum of the nH samples adjacent to the right boundary of the current block, the sum of the nW samples adjacent to the lower boundary of the current block, and one the selection bordering the bottom right of the current block.

[175] However, some adjacent reference samples of the current block may not yet be decoded or may not be available. In this case, the decoder may configure adjacent reference samples to be used for prediction by substituting available samples for unavailable samples. Alternatively, adjacent reference samples to be used for prediction may be configured through interpolation of the available samples.

[176] In case neighboring reference samples are extracted, (i) a predictive sample may be inferred based on an average or interpolation of neighboring reference samples of the current block, and (ii) a predictive sample may be inferred based on a reference sample existing in a particular direction (prediction ) relative to the predictive sample from neighboring reference samples of the current block. Case (i) may be referred to as "non-directional mode" or "non-angular mode", and case (ii) may be referred to as "directional mode" or "angular mode".

[177] Additionally, a prediction sample may be generated through interpolation between a first neighbor sample and a second neighbor sample located in the opposite direction from the intra prediction mode prediction direction of the current block, based on the prediction sample of the current block from the neighbor reference samples. The above case may be referred to as "linear interpolation intra prediction (LIP)". Additionally, chroma prediction samples may be generated based on the luma samples using a linear model. This case may be referred to as "linear model (LM) mode". Alternatively, the current block prediction sample may be derived by extracting the current block temporal prediction sample based on the filtered neighboring reference samples and performing a weighted summation of the temporal prediction sample and at least one reference sample derived according to the intra prediction mode from the unfiltered neighboring reference samples. The above case may be referred to as "position dependent intra prediction (PDPC)". Alternatively, intra prediction coding may be performed in a method for extracting a prediction sample using a reference sample located in the prediction direction in the reference sample line having the highest prediction accuracy, by selecting the corresponding line from the adjacent multiple reference sample lines of the current block, and for indicating (transmitting in overhead) of the reference sample line in use at the time to the decoding equipment. The above case may be referred to as "multiple reference line (MRL) intra prediction" or "MRL intra prediction". Additionally, when performing intra prediction based on identical intra prediction modes by dividing the current block into vertical or horizontal sub-segments, adjacent reference samples may be extracted and used in sub-segment units. Thus, the intra prediction mode for the current block can equally apply to sub-segments, and in this case, since adjacent reference samples are extracted and used in sub-segment units, intra prediction performance can be improved in some cases. This prediction method may be referred to as "intra-sub-segment prediction (ISP) or ISP-based".

[178] The above-described intra prediction methods may be referred to as "intra prediction type" in contrast to the intra prediction mode. The type of intra prediction may be referred to by various terms such as "intra prediction technology" or "additional intra prediction mode". For example, the intra prediction type (or additional intra prediction mode) may include at least one of LIP, PDPC, MRL, and ISP as described above. The general intra prediction method except for a specific type of intra prediction such as LIP, PDPC, MRL, or ISP may be referred to as "normal intra prediction type". In case the above-described specific type of intra prediction is not applied, the normal intra prediction type may generally be applied, and prediction may be performed based on the above-described intra prediction mode. Meanwhile, as needed, post-filtering for the extracted predictive sample can be performed.

[179] In other words, the intra prediction procedure may include determining an intra prediction mode/type, extracting neighboring reference samples, and extracting predictive samples based on the intra prediction mode/type. Additionally, as needed, post-filtering on the extracted predictive sample may be performed.

[180] Meanwhile, of the above-described types of intra prediction, the ISP may divide the current block in the horizontal direction or the vertical direction, and may perform intra prediction in units of divided blocks. Thus, the ISP can extract sub-blocks by dividing the current block in the horizontal direction or the vertical direction, and can perform intra prediction for each of the sub-blocks. In this case, a reconstructed block may be generated by performing encoding/decoding in units of divided sub-blocks, and the reconstructed block may be used as a reference block of the next divided sub-block. Here, a sub-unit may be referred to as an "inner sub-segment".

[181] For example, in the case where ISP is applied, the current block may be divided into two or four sub-blocks in the vertical or horizontal direction based on the size of the current block.

[182] For example, to apply the ISP, a flag representing whether or not to apply the ISP may be transmitted in units of blocks, and in case the ISP is applied to the current block, the flag representing whether the segmentation type is horizontal or vertical, i.e. .e. whether the segmentation direction is the horizontal direction or the vertical direction may be encoded/decoded. A flag representing whether or not an ISP should be applied may be referred to as an "ISP flag", and the ISP flag may be represented as an intra_subpartitions_mode_flag syntax element. Additionally, a flag representing a segmentation type may be referred to as an "ISP segmentation flag", and an ISP segmentation flag may be represented as an intra_subpartitions_split_flag syntax element.

[183] For example, by means of an ISP flag or an ISP segmentation flag, information representing that ISP is not applied to the current block (IntraSubPartitionsSplitType==ISP_NO_SPLIT), information representing segmentation in the horizontal direction (IntraSubPartitionsSplitType==ISP_HOR_SPLIT), information representing segmentation in the vertical direction (IntraSubPartitionsSplitType==ISP_VER_SPLIT). For example, an ISP flag or an ISP segmentation flag may be referred to as "ISP related information" regarding block subsegmentation.

[184] Meanwhile, in addition to the above-described types of intra prediction, affine linear weighted intra prediction (ALWIP) can be used. ALWIP may be referred to as "linear weighted intra prediction (LWIP)", "matrix weighted intra prediction (MWIP)", or "matrix intra prediction (MIP)". In case ALWIP is applied to the current block, i) using adjacent reference samples for which the averaging procedure is performed, ii) the matrix-vector multiplication procedure can be performed, and iii) as necessary, the prediction samples for the current block can be extracted by additional execution horizontal/vertical interpolation procedures.

[185] The intra prediction modes used for ALWIP may be the intra LIP, PDPC, MRL or ISP prediction described above, but may be configured differently compared to the intra prediction modes used in normal intra prediction. The intra prediction mode for ALWIP may be referred to as "ALWIP mode". For example, according to the intra prediction mode for ALWIP, the matrix and bias used in matrix vector multiplication may be configured differently. Here, the matrix may be referred to as a "(affine) weight matrix", and the bias may be referred to as a "(affine) bias vector" or "(affine) shift vector". Herein, the intra prediction mode for ALWIP may be referred to as "ALWIP mode", "ALWIP intra prediction mode", "LWIP mode", "LWIP intra prediction mode", "MWIP mode", "MWIP intra prediction mode", "MWIP mode". prediction", "MIP mode", or "MIP mode of intra prediction". The detailed ALWIP method is described below.

[186] FIG. 9 is a diagram explaining the MIP for an 8x8 block.

[187] To predict the samples of a rectangular block having a width W and a height H, MIP may use the samples bordering the left border of the block and the samples bordering the top border. Here, the samples adjacent to the left border may represent samples located in the same line adjacent to the left border of the block, and may represent recovered samples. The samples adjacent to the upper bound may represent the samples located in the same line adjacent to the upper bound of the block, and may represent reconstructed samples.

[188] For example, if reconstructed samples are not available, reconstructed samples, similar to prior art intra prediction, may be generated or retrieved and may be used.

[189] The predictive signal (or predictive samples) may be generated based on an averaging process, a matrix vector multiplication process, and a (linear) interpolation process.

[190] For example, the averaging process may be a process for extracting samples from a boundary through averaging. For example, if the width W and the height H of the samples are 4, the extracted samples may be four samples, and may be 8 samples otherwise. For example, in FIG. 9, bdryleft and bdrytop may represent the extracted left samples and top samples, respectively.

[191] For example, the process of matrix-vector multiplication may be a process of performing matrix-vector multiplication with averaged samples as inputs. Additionally, an offset may be added. For example, in FIG. 9, Ak may represent a matrix, bk may represent an offset, and bdryred may represent a reduced signal for the samples extracted through the averaging process. Additionally, bdryred may be shorthand for bdryleft and bdrytop. The result may be a reduced predictive signal predred for the set of downsampled samples in the original block.

[192] For example, the (linear) interpolation process may be a process in which the predictive signal is generated at the remaining locations from the predictive signal for the set downsampled by linear interpolation. Here, the linear interpolation may represent one linear interpolation in the respective directions. For example, linear interpolation may be performed based on the reduced predictive signal predred marked in gray in the block in FIG. 9 and neighboring boundary samples, and thereby all predictive samples in the block can be extracted.

[193] For example, matrices (Ak in FIG. 9) and bias vectors (bk in FIG. 9) required to generate a predictive signal (either prediction block samples or predictive samples) may be obtained from the three sets S0, S1 and S2. For example, the set S0 may consist of 18 matrices (A ₀ ⁱ , i=0, 1, ...,1, ..., 17) and 18 bias vectors (b ₀ ⁱ , i=0, 1, ... , 17). Here, each of the 18 matrices can have 16 rows and 4 columns, and each of the 18 displacement vectors can have 16 sizes. The offset matrices and vectors of the set S0 may be used for a block having a size of 4x4. For example, the set S1 may consist of 10 matrices (A ₁ ⁱ , i=0, 1, ..., 9) and 10 bias vectors (b ₁ ⁱ , i=0, 1, ..., 9). Here, each of the 10 matrices can have 16 rows and 8 columns, and each of the 10 displacement vectors can have 16 sizes. The offset matrices and vectors of set S1 may be used for a block having a size of 4x8, 8x4, or 8x8. For example, the set S2 may consist of 6 matrices (A ₂ ⁱ , i=0, 1, ..., 5) and 6 bias vectors (b ₂ ⁱ , i=0, 1, ..., 5). Here, each of the 6 matrices can have 64 rows and 8 columns, and each of the 6 displacement vectors can have 64 sizes. The offset matrices and vectors of set S2 can be used for all remaining blocks.

[194] Meanwhile, in an embodiment of the present document, LFNST index information may be signaled with respect to a block to which MIP is applied. Alternatively, the encoding equipment may generate a bitstream by encoding the LFNST index information to transform the block to which the MIP is applied, and the decoding equipment may obtain LFNST index information to transform the block to which the MIP is applied by parsing or decoding the bitstream.

[195] For example, the LFNST index information may be information for distinguishing the LFNST transform set according to the number of transforms constituting the transform set. For example, the optimal LFNST core may be selected with respect to the block in which the intra prediction to which MIP is applied based on the LFNST index information. For example, the LFNST index information may be represented as the st_idx syntax element or the lfnst_idx syntax element.

[196] For example, LFNST index information (or the st_idx syntax element) may be included in the syntax as indicated in the following tables.

[197] Tab. 2

[198] Tab. 3

[199] Tab. four

[200] Tab. 5

[201] Tables 2-5, as described above, may sequentially represent one syntax or information.

[202] For example, in Tables 2-5, the information or semantics represented by the intra_mip_flag syntax element, the intra_mip_mpm_flag syntax element, the intra_mip_mpm_idx syntax element, the intra_mip_mpm_remainder syntax element, or the st_idx syntax element may be as indicated in the following table.

[203] Tab. 6

- intra_mip_flag[x0][y0] equal to 1 indicates that the type of intra prediction for luminance samples is matrix intra prediction; intra_mip_flag[x0][y0] equal to 0 indicates that the type of intra prediction for luma samples is not matrix intra prediction.

When intra_mip_flag[x0][y0] is not present, it may be logically inferred to be 0.

The syntax elements intra_mip_mpm_flag[x0][y0], intra_mip_mpm_idx[x0][y0] and intra_mip_mpm_remainder[x0][y0] indicate the matrix intra prediction mode for luminance samples. The array indexes x0, y0 indicate the location (x0, y0) of the top left luma sample of the coding block in question, relative to the top left luma sample of the movie frame.

When intra_mip_mpm_flag[x0][y0] is equal to 1, the matrix intra prediction mode is inferred from an adjacent intra predicted coding unit.

When intra_mip_mpm_flag[x0][y0] is not present, it may be logically inferred to be 1.

- st_idx[x0][y0] indicates which transform kernels (LFNST kernels) are applied to the LFNST for the current block, st_idx may indicate one of the transform kernels in the LFNST transform set, which may be determined based on intra/inter prediction and/ or the block size of the current block.

[204] For example, the intra_mip_flag syntax element may represent information regarding whether or not MIP is applied to luma samples or the current block. Additionally, for example, the intra_mip_mpm_flag syntax element, the intra_mip_mpm_idx syntax element, or the intra_mip_mpm_remainder syntax element may represent information regarding the intra prediction mode to be applied to the current block in case MIP is applied. Additionally, for example, the st_idx syntax element may represent information regarding a transformation kernel (LFNST kernel) to be applied to the LFNST for the current block. Thus, the st_idx syntax element may be information representing one of the transformation kernels in the LFNST transformation set. Here, the st_idx syntax element may be represented as the lfnst_idx syntax element or LFNST index information.

[205] FIG. 10 is a flowchart explaining the method to which MIP and LFNST are applied.

[206] Meanwhile, another embodiment of the present document may not signal the LFNST index information regarding the block to which the MIP is applied. Further, the encoding equipment may generate a bitstream by encoding image information except for the LFNST index information for transforming a block to which MIP is applied, and the decoding equipment may parse or decode the bitstream and may perform a block transform process without LFNST index information for the transformation of the block to which the MIP is applied.

[207] For example, if the LFNST index information is not signaled, the LFNST index information may be inferred as a default value. For example, the LFNST index information inferred as a default value may be the value 0. For example, the LFNST index information having the value 0 may represent that the LFNST is not applied to the corresponding block. In this case, since the LFNST index information is not transmitted, the bit size for encoding the LFNST index information may be reduced. Additionally, the complexity can be reduced by preventing the simultaneous application of MIP and LFNST, and thus the delay time can also be reduced.

[208] Referring to FIG. 10, it may first be determined whether or not the MIP is applied to the corresponding block. Thus, it can be determined whether the value of the syntax element intra_mip_flag is 1 or 0 or not (S1000). For example, if the value of the intra_mip_flag syntax element is 1, it may be considered to represent "true" or "Yes", and this may represent that the MIP is applied to the corresponding block. Accordingly, MIP prediction may be performed for the corresponding block (S1010). Thus, a prediction block for the corresponding block can be retrieved by performing MIP prediction. Thereafter, the inverse primary conversion procedure may be performed (S1020) and the internal recovery procedure may be performed (S1030). In other words, the residual block may be extracted by performing an inverse primary transform on the transform coefficients obtained from the bitstream, and the reconstructed block may be generated based on the prediction block according to the MIP prediction and the residual block. Thus, LFNST index information for a block to which MIP is applied may not be included. Additionally, LFNST may not be applied to the block to which the MIP is applied.

[209] Additionally, for example, if the value of the intra_mip_flag syntax element is 0, it may be considered to represent "false" or "No", and this may represent that the MIP is not applied to the corresponding block. Thus, conventional intra prediction can be applied to the corresponding block (S1040). Thus, a prediction block for the corresponding block can be retrieved by performing conventional intra prediction. Thereafter, it may be determined whether or not the LFNST is applied to the corresponding block based on the LFNST index information. In other words, it may be determined whether or not the value of the syntax element st_idex is greater than 0 (S1050). For example, if the value of the st_idex syntax element is greater than 0, the inverse LFNST transformation procedure may be performed using the transformation kernel represented by the st_idex syntax element (S1060). Additionally, if the value of the st_idex syntax element is not greater than 0, it may represent that LFNST is not applied to the corresponding block, and the LFNST inverse transform procedure may not be performed. Thereafter, the inverse primary conversion procedure may be performed (S1020) and the internal recovery procedure may be performed (S1030). In other words, the residual block may be extracted by performing an inverse primary transform on the transform coefficients obtained from the bitstream, and the reconstructed block may be generated based on the prediction block according to conventional intra prediction and the residual block.

[210] In general, if MIP is applied, a MIP prediction block may be generated without decoding the LFNST index information, and the final intra recovery signal may be generated by applying an inverse primary transform to the received coefficient.

[211] Conversely, if MIP is not applied, the LFNST index information may be decoded, and if the value of the flag (either the LFNST index information or the st_idx syntax element) is greater than 0, the final internal recovery signal may be generated by applying an inverse LFNST transform and an inverse primary conversion relative to the received coefficient.

[212] For example, for the above procedure, LFNST index information (or the st_idx syntax element) may be included in the syntax or image information based on information (or the intra_mip_flag syntax element) as to whether MIP is applied or not, and may be signaled. Additionally, the LFNST index information (or the st_idx syntax element) can be selectively configured/parsed/signaled/transmitted/received with reference to information (or the intra_mip_flag syntax element) regarding whether or not MIP is applied. For example, the LFNST index information may be represented as the st_idx syntax element or the lfnst_idex syntax element.

[213] For example, LFNST index information (or st_idx syntax element) may be included as indicated in Table 7 below.

[214] Tab. 7

[215] For example, referring to Table 7, the st_idx syntax element may be included based on the intra_mip_flag syntax element. In other words, if the value of the intra_mip_flag syntax element is 0 (!intra_mip_flag), the st_idx syntax element may be included.

[216] Additionally, for example, LFNST index information (or the lfnst_idx syntax element) may be included as indicated in Table 8 below.

[217] Tab. eight

[218] For example, referring to Table 8, the lfnst_idx syntax element may be included based on the intra_mip_flag syntax element. In other words, if the value of the intra_mip_flag syntax element is 0 (!intra_mip_flag), the lfnst_idx syntax element may be included.

[219] For example, referring to Table 7 or Table 8, the st_idx syntax element or the lfnst_idx syntax element may be included based on intra-subsegment-based (ISP) mode-related information regarding block sub-segmentation. For example, the ISP-related information may include an ISP flag or an ISP segmentation flag, and thereby information regarding whether or not sub-segmentation is performed with respect to a block may be presented. For example, information regarding whether or not sub-segmentation is performed may be represented as IntraSubPartitionsSplitType, ISP_NO_SPLIT may represent that sub-segmentation is not performed, ISP_HOR_SPLIT may represent that sub-segmentation is performed in the horizontal direction, and ISP_VER_SPLIT may represent that sub-segmentation is performed in the vertical direction. .

[220] The associated residual information may include LFNST index information based on the MIP flag and ISP related information.

[221] Meanwhile, in another embodiment of the present document, LFNST index information can be inferred with respect to the block to which MIP is applied without separate signaling. Further, the encoding equipment may generate a bitstream by encoding image information except for the LFNST index information to transform the block to which the MIP is applied, and the decoding equipment may parse or decode the bitstream, infer, and obtain the LFNST index information to transform the block to which the MIP is applied, and perform the block transformation process based on it.

[222] Thus, the LFNST index information may not be decoded with respect to the corresponding block, but through an inference process, an index for segmenting the transforms constituting the LFNST transform set may be determined. Additionally, through the inference process, it may be determined that a separate optimized transformation kernel is used for the block to which the MIP is applied. In this case, the optimal LFNST core may be selected relative to the block to which the MIP is applied, and the bit size for encoding it may be reduced.

[223] For example, LFNST index information may be inferred based on at least one of intra prediction reference line index information, intra prediction mode information, block size information, or MIP application/non-application information.

[224] Meanwhile, in another embodiment of the present document, LFNST index information for a block to which MIP is applied may be binarized to be signaled. For example, the number of applicable LFNST transformations may differ depending on whether or not MIP is applied to the current block, and to this end, the binarization method for obtaining LFNST index information may be selectively switched.

[225] For example, one LFNST core may be used relative to a block to which MIP is applied, and this core may be one of the LFNST cores applied to a block to which no MIP is applied. Additionally, an existing LFNST core used may not be used for the block to which the MIP is applied, but a separate kernel optimized for the block to which the MIP is applied may be defined and used.

[226] In this case, since a reduced number of LFNST cores is used with respect to a block to which MIP is applied compared to the number of LFNST cores for a block to which no MIP is applied, overhead due to signaling of LFNST index information may decrease, and the complexity may decrease.

[227] For example, the LFNST index information may use a binarization method as indicated in the following table.

[228] Tab. 9

Syntactic structure syntax element Binary conversion Process Input parameters st_idx[][] intra_mip_flag[][]="false" TR cMax=2, cRiceParam=0 st_idx[][] intra_mip_flag[][]="true" FL cMax=1

[229] Referring to Table 9, for example, the st_idx syntax element may be binarized to a truncated Rice (TR) code in case MIP is not applied to the corresponding block, in case of intra_mip_flag[][]=="false", or in case the value of the intra_mip_flag syntax element is 0. In this case, for example, cMax, which is an input parameter, may have a value of 2, and cRiceParam may have a value of 0.

[230] Additionally, for example, the st_idx syntax element may be converted to fixed length (FL) binary form in case the MIP is applied to the corresponding block, in the case of intra_mip_flag[][]=="true", or in case the value of the syntax element intra_mip_flag is 1. In this case, for example, cMax, which is an input parameter, may have the value 1.

[231] Here, the st_idx syntax element may represent LFNST index information and may be represented as the lfnst_idx syntax element.

[232] Meanwhile, in another embodiment of the present document, LFNST related information may be signaled with respect to the block to which the MIP is applied.

[233] For example, the LFNST index information may include one syntax element, and may represent information regarding whether or not LFNST is applied based on one syntax element, and information regarding the kind of transformation kernel used for the LFNST. In this case, the LFNST index information may be represented as st_idx syntax element or lfnst_idx syntax element, for example.

[234] Additionally, for example, the LFNST index information may include one or more syntax elements and may represent information regarding whether or not LFNST is applied based on one or more syntax elements, and information regarding the kind of transformation kernel used for the LFNST . For example, the LFNST index information may include two syntax elements. In this case, the LFNST index information may include a syntax element representing information on whether or not the LFNST is applied, and a syntax element representing information on the kind of transformation kernel used for the LFNST. For example, information regarding whether or not LFNST is applied may be represented as an LFNST flag, and may be represented as an st_flag syntax element or an lfnst_flag syntax element. Additionally, for example, information regarding the kind of transformation kernel used for the LFNST may be represented as a transformation kernel index flag, and may be represented as a st_idx_flag syntax element, an st_kernel_flag syntax element, an lfnst_idx_flag syntax element, or an lfnst_kernel_flag syntax element. For example, in case the LFNST index information includes one or more syntax elements as described above, the LFNST index information may be referred to as "LFNST related information".

[235] For example, LFNST related information (eg, st_flag syntax element or st_idx_flag syntax element) may be included as indicated in Table 10 below.

[236] Tab. ten

[237] Meanwhile, the block to which the MIP is applied may use a different number of LFNST transforms (kernels) from that of the block to which the MIP is not applied. For example, a block to which MIP is applied can only use one LFNST transform kernel. For example, one LFNST transform kernel may be one of the LFNST kernels applied to a block to which no MIP is applied. Additionally, instead of using an existing LFNST kernel used relative to the block to which the MIP is applied, a separate kernel optimized for the block to which the MIP is applied can be defined and used.

[238] In this case, information (e.g., a transform kernel index flag) regarding the kind of transform kernel used for the LFNST from the LFNST-related information may be selectively signaled depending on whether or not MIP is applied, and related to LFNST information at this time may be included, for example, as indicated in Table 11 below.

[239] Tab. eleven

[240] In other words, referring to Table 11, information (or syntax element st_idx_flag) regarding the kind of transformation kernel used for LFNST may be included based on information (or syntax element intra_mip_flag) regarding whether or not MIP is applied to the corresponding block. Additionally, for example, the st_idx_flag syntax element may be signaled in !intra_mip_flag in case the MIP is not applied to the corresponding block.

[241] For example, in Table 10 or Table 11, the information or semantics represented by the st_flag syntax element or the st_idx_flag syntax element may be as indicated in the following table.

[242] Tab. 12

- st_flag[x0][y0] indicates whether the secondary transformation is applied or not; st_flag[x0][y0] equal to 0 indicates that no secondary transformation is applied. The indexes x0, y0 of the arrays indicate the location (x0, y0) of the top left sample of the transform block in question, relative to the top left sample of the frame.

When st_flag[x0][y0] is not present, st_idx[x0][y0] is logically inferred to be 0.

- st_idx_flag[x0][y0] indicates which secondary transform kernel is applied between the two candidate kernels in the selected transform set. The indexes x0, y0 of the arrays indicate the location (x0, y0) of the top left sample of the transform block in question, relative to the top left sample of the frame.

When st_idx_flag[x0][y0] is not present, st_idx[x0][y0] is inferred to be 0.

[243] For example, the st_flag syntax element may represent information as to whether or not a secondary transformation is applied. For example, if the value of the st_flag syntax element is 0, it may represent that a secondary transformation is not applied, while if 1, it may represent that a secondary transformation is applied. For example, the st_idx_flag syntax element may represent information regarding the applied secondary transform kernel from the two candidate kernels in the selected transform set.

[244] For example, for LFNST related information, a binarization method may be used as indicated in the following table.

[245] Tab. 13

Syntactic structure syntax element Binary conversion Process Input parameters ....... ....... ....... st_flag[][] FL cMax=1 st_idx_flag[][] FL cMax=1

[246] Referring to Table 13, for example, the st_flag syntax element can be binarized based on FL. For example, in this case, cMax, which is an input parameter, may have a value of 1. Additionally, for example, the st_idx_flag syntax element may be binarized based on FL. For example, in this case, cMax, which is an input parameter, could be 1.

[247] For example, referring to Table 10 or Table 11, the descriptor of the st_flag syntax element or the st_idx_flag syntax element may be ae(v). Here, ae(v) may represent context-adaptive arithmetic entropy coding. Additionally, the syntax element for which the descriptor is ae(v) may be a context-adaptive arithmetically entropy encoded syntax element. Thus, context-adaptive arithmetic entropy coding can be applied to LFNST related information (eg, st_flag syntax element or st_idx_flag syntax element). Additionally, the LFNST related information (eg, st_flag syntax element or st_idx_flag syntax element) may be information or syntax element to which context adaptive arithmetic entropy coding is applied. Additionally, LFNST-related information (eg, permission elements from the permission element string of the st_flag syntax element or the st_idx_flag syntax element) may be encoded/decoded based on the above-described CABAC, and the like. Here, context-adaptive arithmetic entropy coding may be represented as context model-based coding, context coding, or regular coding.

[248] For example, incrementing the context index (ctxInc) of LFNST related information (e.g., st_flag syntax element or st_idx_flag syntax element) or ctxInc according to the location of the resolution element of the st_flag syntax element or the st_idx_flag syntax element may be allocated or determined as specified in Table 14. Additionally, as indicated in Table 14, the context model may be selected based on ctxInc in accordance with the location of the permission element of the st_flag syntax element or the st_idx_flag syntax element that is allocated or defined as indicated in Table 14.

[249] Tab. fourteen

syntax element binIdx 0 one 2 3 four >=5 ...... ...... ...... ...... ...... ...... ...... st_flag[][] 0.1 (section 9.5.4.2.8) n/a n/a n/a n/a n/a st_idx_flag[][] bypass n/a n/a n/a n/a n/a

[250] Referring to Table 14, for example (permission element from a string of permission elements or first permission element), the st_flag syntax element can use two context models (or ctxIdx), and the context model can be selected based on ctxInc having a value of 0 or 1. Additionally, for example, the bypass encoding may be applied to the (permission element of a string of permission elements or the first permission element) st_idx_flag syntax element. Additionally, encoding may be performed by applying a regular probability distribution.

[251] For example, ctxInc (of the permission element from the string of permission elements or the first permission element) of the st_flag syntax element may be determined based on Table 15 below.

[252] Tab. fifteen

9.5.4.2.8. The process of extracting ctxInc for the st_flag syntax element

The inputs to this process are the index cIdx of the color component, the location (x0, y0) of the luma or chrominance signal indicating the top left sample of the current luminance or chrominance coding block relative to the top left sample of the current frame depending on cIdx, the treeType of the tree, and index tu_mts_idx[x0][y0] of the multiple choice transform. The output of this process is ctxInc.

The destination of ctxInc is specified as follows:

- ctxInc=(tu_mts_idx[x0][y0]=0 andand treeType!=SINGLE_TREE)?1:0

[253] Referring to Table 15, for example, the ctxInc (of the permission element from the string of permission elements or the first permission element) of the st_flag syntax element may be determined based on the MTS index (or the tu_mts_idx syntax element) or tree type information (treeType). For example, ctxInc can be retrieved as 1 if the MTS index value is 0 and the tree type is not a single tree. Additionally, ctxInc may be retrieved as 0 if the MTS index value is not 0 or the tree type is a single tree.

[254] In this case, since a reduced number of LFNST cores is used with respect to a block to which MIP is applied compared to the number of LFNST cores to which no MIP is applied, the overhead due to signaling of the LFNST index information may decrease, and complexity can be reduced.

[255] Meanwhile, in another embodiment of the present document, the LFNST core may be inferred and used relative to the block to which the MIP is applied. Thus, the LFNST core can be inferred without separate overhead information regarding the LFNST core. Further, the encoding equipment may generate a bitstream by encoding image information except for the LFNST index information to transform the block to which the MIP is applied and information regarding the (kind of) transformation kernel suitable for the LFNST, and the decoding equipment may parse or decode a bit stream, infer and obtain LFNST index information to transform the block to which the MIP is applied and information regarding the transformation kernel used for the LFNST, and perform a block transformation process based on it.

[256] Thus, the LFNST index information for the corresponding block or information regarding the transform kernel used for the LFNST may not be decoded, but through an inference process, an index for segmenting the transforms constituting the LFNST transform set may be determined. Additionally, through the inference process, it may be determined that a separate optimized transformation kernel is used for the block to which the MIP is applied. In this case, the optimal LFNST core may be selected relative to the block to which the MIP is applied, and the bit size for encoding it may be reduced.

[257] For example, LFNST index information or information regarding a transform kernel used for LFNST may be inferred based on at least one of intra prediction baseline index information, intra prediction mode information, block size information, or MIP application information. / non-use.

[258] In the above embodiments of the present document, fixed length (FL) binarization may represent a fixed length binarization method, e.g., with a specific number of bits, and the specific number of bits may be predetermined or may be represented based on cMax. Truncated Binary (TU) Binary Conversion can represent a variable length binarization method without appending 0 in case the number of characters to be expressed using 1, the number of which is not less than the number of characters and one 0, is equal to the maximum length, and the maximum length may be represented based on cMax. Truncated Rice (TR) binarization may represent a binarization method in a form in which a prefix and a suffix are concatenated, such as TU+FL, using maximum length and offset information, and in case the offset information is 0, it may be TU. Here, the maximum length may be represented based on cMax, and shift information may be represented based on cRiceParam.

[259] FIG. 11 and 12 schematically illustrate a video/image coding method and an example of related components according to the embodiment(s) of the present document.

[260] The method disclosed in FIG. 11 may be performed by the encoding equipment disclosed in FIG. 2 or fig. 12. In particular, for example, S1100-S1120 of FIG. 11 may be performed by the coding equipment predictor 220 of FIG. 12 and S1130-S1150 of FIG. 11 may be performed by the encoding equipment residual processor 230 of FIG. 12 and S1160 of FIG. 11 may be performed by the entropy encoder 240 of the encoding equipment of FIG. 12. Additionally, although not illustrated in FIG. 11, encoding equipment predictor 220 in FIG. 12 may extract the prediction samples or prediction-related information, the encoding equipment residual processor 230 may extract the residual information from the original samples or prediction samples, and the encoding equipment entropy encoder 240 may generate a bit stream from the residual information or prediction-related information. The method disclosed in FIG. 11 may include the embodiments of the present document described above.

[261] Referring to FIG. 11, the encoding equipment may determine the intra prediction type of the current block (S1100), and may generate intra prediction type information for the current block based on the intra prediction type (S1110). For example, the encoding equipment may determine the type of intra-prediction of the current block given the rate-of-rate (RD) distortion cost function. The intra prediction type information may represent information as to whether to apply a normal intra prediction type using a reference line adjacent to the current block, a multiple reference line (MRL) based mode using a reference line that is not adjacent to the current block, a mode on internal sub-segment prediction (ISP) that performs sub-segmentation for the current block, or matrix intra prediction (MIP) using a matrix.

[262] For example, the intra prediction type information may include a MIP flag representing whether or not MIP is applied to the current block. Further, for example, the intra prediction type information may include intra-sub-segment (ISP) mode-related information regarding the ISP sub-segmentation for the current block. For example, the ISP related information may include an ISP flag representing whether or not an ISP is applied to the current block, or an ISP segmentation flag representing a segmentation direction. Additionally, for example, the intra prediction type information may include a MIP flag and ISP related information. For example, a MIP flag may represent the intra_mip_flag syntax element. Additionally, for example, an ISP flag may represent an intra_subpartitions_mode_flag syntax element, and an ISP segmentation flag may represent an intra_subpartitions_split_flag syntax element.

[263] Additionally, although not illustrated in FIG. 11, for example, the encoding equipment may determine the intra prediction mode for the current block, and may generate intra prediction information for the current block based on the intra prediction mode. For example, the encoding equipment may determine the intra prediction mode considering the RD cost. The intra prediction mode information may represent the intra prediction mode to be applied to the current block from the intra prediction modes. For example, intra prediction modes may include intra prediction modes number 0-66. For example, intra prediction mode number 0 may represent a planar mode, and intra prediction mode number 1 may represent a DC mode. Additionally, intra prediction modes number 2-66 may be represented as directional or angle intra prediction modes, and may represent directions for reference. Additionally, intra prediction modes number 0 and number 1 may be represented as non-directional or non-angular intra prediction modes. A detailed explanation of the above is given with reference to FIG. 5.

[264] For example, the encoding equipment may generate prediction related information for the current block, and the prediction related information may include intra prediction mode information and/or intra prediction type information.

[265] The encoding equipment may derive prediction samples of the current block based on the type of intra prediction (S1120). Additionally, for example, the encoding equipment may generate predictive samples based on the intra prediction mode and/or type of intra prediction. Additionally, the encoding equipment may generate prediction samples based on the prediction-related information.

[266] The encoding equipment may generate residual samples of the current block based on the predictive samples (S1130). For example, the encoding equipment may generate residual samples based on the original samples (eg, the input image signal) and the predictive samples. Additionally, for example, the encoding equipment may generate residual samples based on the difference between the original samples and the predictive samples.

[267] The encoding equipment may derive transform coefficients for the current block based on the residual samples (S1140). For example, the encoding equipment may derive transform coefficients by performing a primary transform based on the residual samples. Additionally, for example, the encoding equipment may derive temporal transform coefficients by performing a primary transform based on the residual samples, and may derive transform coefficients by applying LFNST to the temporal transform coefficients. For example, in case LFNST is applied, the encoding equipment may generate LFNST index information. Thus, the encoding equipment can generate LFNST index information based on a transform kernel used to derive transform coefficients.

[268] The encoding equipment may generate associated residual information based on the transform coefficients (S1150). For example, the encoding equipment may extract the quantized transform coefficients by performing quantization based on the transform coefficients. Additionally, the encoding equipment may generate information regarding the quantized transform coefficients based on the quantized transform coefficients. Additionally, the associated residual information may include information regarding quantized transform coefficients.

[269] The encoding equipment may encode intra prediction type information and associated residual information (S1160). For example, the associated residual information may include information regarding quantized transform coefficients, as described above. Additionally, for example, the associated residual information may include LFNST index information. Additionally, for example, the associated residual information may not include LFNST index information.

[270] For example, the associated residual information may include LFNST index information representing information regarding the inseparable transform for the low-frequency transform coefficients of the current block. Additionally, for example, the associated residual information may include LFNST index information based on a MIP flag or a current block size. Additionally, for example, the associated residual information may include LFNST index information based on a MIP flag or information regarding the current block. Here, the information regarding the current block may include at least one of the size of the current block, tree structure information representing a single tree or a double tree, an LFNST activation flag, or ISP related information. For example, the MIP flag may be one of a plurality of conditions for determining whether or not the associated residual information includes LFNST index information, and through other conditions such as the size of the current block, in addition to the MIP flag, the associated residual information may include LFNST index information. However, later in this document, an explanation is provided for the MIP flag. Here, the LFNST index information may be represented as transformation index information. Additionally, the LFNST index information may be represented as a st_idx syntax element or an lfnst_idx syntax element.

[271] For example, the associated residual information may include LFNST index information based on a MIP flag representing that MIP is not applied. Additionally, for example, the associated residual information may not include LFNST index information based on a MIP flag representing that MIP is applied. Thus, in the case where the MIP flag represents that the MIP is applied to the current block (for example, in the case where the value of the intra_mip_flag syntax element is 1), the associated residual information may not include LFNST index information, and in the case where The MIP flag represents that the MIP is not applied to the current block (eg, in case the value of the intra_mip_flag syntax element is 0), the associated residual information may include LFNST index information.

[272] Additionally, for example, the associated residual information may include LFNST index information based on the MIP flag and ISP related information. For example, in case the MIP flag represents that the MIP is not applied to the current block (for example, in case the value of the intra_mip_flag syntax element is 0), the associated residual information may include LFNST index information with reference to the ISP related information (IntraSubPartitionsSplitType). Here, IntraSubPartitionsSplitType may represent that ISP is not applied (ISP_NO_SPLIT), ISP is applied in horizontal direction (ISP_HOR_SPLIT), or ISP is applied in vertical direction (ISP_VER_SPLIT), and it may be derived based on ISP flag or ISP segmentation flag.

[273] For example, since the MIP flag represents that the MIP is applied to the current block, the LFNST index information may be inferred or retrieved for use, in which case, the associated residual information may not include the LFNST index information. Thus, the encoding equipment may not signal the LFNST index information. For example, the LFNST index information may be inferred or retrieved for use based on at least one of the reference line index information for the current block, the intra prediction mode information of the current block, the current block size information, and the MIP flag.

[274] Additionally, for example, the LFNST index information may include a LFNST flag representing whether or not an inseparable transform is applied to the low-frequency transform coefficients of the current block, and/or a transform kernel index flag representing a transform kernel applied to the current block. , from possible variants of transformation kernels. Thus, while the LFNST index information may represent inseparable transform information for the low frequency transform coefficients of the current block based on one syntax element or one piece of information, it may also represent information based on two syntax elements or two pieces of information. For example, an LFNST flag may be represented as a st_flag syntax element or an lfnst_flag syntax element, and a transformation kernel index flag may be represented as a st_idx_flag syntax element, a st_kernel_flag syntax element, an lfnst_idx_flag syntax element, or an lfnst_kernel_flag syntax element. Here, a transform kernel index flag may be included in the LFNST index information based on the LFNST flag representing that a nonseparable transform is applied and the MIP flag representing that MIP is not applied. Thus, in case the LFNST flag represents that an inseparable transform is applied and the MIP flag represents that a MIP is applied, the LFNST index information may include a transform kernel index flag.

[275] For example, since the MIP flag represents that the MIP is applied to the current block, the LFNST flag and the transform kernel index flag may be inferred or retrieved for use, in which case, the associated residual information may not include LFNST- flag and index flag of the conversion kernel. Thus, the encoding equipment may not signal the LFNST flag and the transform kernel index flag. For example, the LFNST flag and the transform kernel index flag may be inferred or derived for use based on at least one of the reference line index information for the current block, the current block intra prediction mode information, the current block size information, and the MIP flag.

[276] For example, in the case where the associated residual information includes LFNST index information, the LFNST index information may be represented via binarization. For example, LFNST index information (e.g., st_idx syntax element or lfnst_idx syntax element) may be represented via truncated Rice (TR) binarization based on a MIP flag representing that MIP is not applied, and LFNST index information ( for example, the st_idx syntax element or the lfnst_idx syntax element) may be represented via binarization with a fixed length (FL) code based on a MIP flag representing that the MIP is applied. Thus, in case the MIP flag represents that the MIP does not apply to the current block (e.g., in the event that the intra_mip_flag syntax element is 0 or false), the LFNST index information (e.g., the st_idx syntax element or the syntax element element lfnst_idx) can be represented via TR-based binarization, and in case the MIP flag represents that the MIP is applied to the current block (for example, in case the intra_mip_flag syntax element is equal to 1 or is "true"), LFNST index information (eg, st_idx syntax element or lfnst_idx syntax element) may be represented via FL-based binarization.

[277] Additionally, for example, if the associated residual information includes LFNST index information, and the LFNST index information includes a LFNST flag and a transform core index flag, the LFNST flag and the transform core index flag may be represented via a transform. to binary form with a fixed length (FL) code.

[278] For example, the LFNST index information may be represented as a (bin) string of bins through the above-described binarization, and by encoding it, a bit, a bit string, or a bit stream may be generated.

[279] For example, the (first) bin of the LFNST flag bin string may be encoded based on the context encoding, and the context encoding may be performed based on the context index increment value for the LFNST flag. Here, context encoding is encoding performed based on a context model and may be referred to as "regular encoding". Additionally, the context model may be represented by the context index ctsIdx, and the context index may be represented based on the increment ctxInc of the context index and the offset ctxIdxOffset of the context index. For example, the context index increment value may be represented as one of the options including 0 and 1. For example, the context index increment value may be determined based on an MTS index (e.g., the mts_idx syntax element or the tu_mts_idx syntax element) representing a set of transformation kernels A to be used for the current block, from sets of transform kernels, and tree type information representing the segmentation structure of the current block. Here, the tree type information may represent a single tree representing that the segmentation structures of the luminance signal component and the chrominance signal component of the current block are equal to each other, or a dual tree representing that the segmentation structures of the luminance signal component and the chrominance signal component of the current block are different from each other. from friend.

[280] For example, the (first) ROI from the ROI string of the transformation kernel index flag may be encoded based on bypass coding. Here, bypass coding may represent that context coding is performed based on a regular probability distribution, and coding efficiency may be improved by omitting the context coding update procedure.

[281] Additionally, although not illustrated in FIG. 11, for example, the encoding equipment may generate reconstructed samples based on residual samples and predictive samples. Additionally, the reconstructed block and the reconstructed frame may be derived based on the reconstructed samples.

[282] For example, the encoding equipment may generate bitstream information or encoded information by encoding image information including all or parts of the above-described pieces of information (or syntax elements). Additionally, the encoding equipment may output the information in the form of a bit stream. Additionally, the bitstream or encoded information may be transmitted to the decoding equipment via a network or a storage medium. Further, the bitstream or encoded information may be stored in a computer-readable storage medium, and the bitstream or encoded information may be generated by the above-described image encoding method.

[283] FIG. 13 and 14 schematically illustrate a video/image decoding method and an example of related components according to the embodiment(s) of the present document.

[284] The method disclosed in FIG. 13 may be performed by the decoding equipment disclosed in FIG. 3 or fig. 14. In particular, for example, the S1300 of FIG. 14 may be performed by the entropy decoder 310 of the decoding equipment of FIG. 14 and S1310 and S1320 of FIG. 13 may be performed by the decoding hardware residual processor 320 of FIG. 14. Additionally, although not illustrated in FIG. 13, the entropy decoder 310 of the decoding equipment of FIG. 14 can extract prediction related information or residual information from the bit stream, the decoding equipment residual processor 320 can extract residual samples from the residual information, the decoding equipment predictor 330 can extract predictive samples from the prediction related information, and the decoding equipment adder 340 can extract the reconstructed block or reconstructed frame from residual samples or predictive samples. The method disclosed in FIG. 13 may include the embodiments of the present document described above.

[285] Referring to FIG. 13, the decoding equipment can obtain intra prediction type information for the current block and associated residual information from the bit stream (S1300). For example, the decoding equipment may obtain intra prediction type information or associated residual information by parsing or decoding the bitstream. Here, the bit stream may be referred to as "encoded information (pictures)".

[286] For example, the decoding equipment may obtain prediction related information from the bitstream, and the prediction related information may include intra prediction mode information and/or intra prediction type information. For example, the decoding hardware may generate predictive samples for the current block.

[287] The intra prediction mode information may represent the intra prediction mode to be applied to the current block from the intra prediction modes. For example, intra prediction modes may include intra prediction modes number 0-66. For example, intra prediction mode number 0 may represent a planar mode, and intra prediction mode number 1 may represent a DC mode. Additionally, intra prediction modes number 2-66 may be represented as directional or angle intra prediction modes, and may represent directions for reference. Additionally, intra prediction modes number 0 and number 1 may be represented as non-directional or non-angular intra prediction modes. A detailed explanation of the above is given with reference to FIG. 5.

[288] Additionally, the intra prediction type information may represent information as to whether to apply a normal intra prediction type using a reference line adjacent to the current block, a multiple reference line (MRL) based mode using a reference line that is not adjacent to the current block, an intra-subsegment based (ISP) mode performing sub-segmentation for the current block, or matrix intra prediction (MIP) using a matrix.

[289] For example, the decoding equipment may obtain the associated residual information from the bitstream. Here, the associated residual information may represent information used to extract the residual samples, and may include information on the residual samples, (inverse) transform-related information, and/or (inverse) quantization-related information. For example, the associated residual information may include information regarding quantized transform coefficients.

[290] For example, the intra prediction type information may include a MIP flag representing whether or not MIP is applied to the current block. Further, for example, the intra prediction type information may include intra-sub-segment (ISP) mode-related information regarding the ISP sub-segmentation for the current block. For example, the ISP related information may include an ISP flag representing whether or not an ISP is applied to the current block, or an ISP segmentation flag representing a segmentation direction. Additionally, for example, the intra prediction type information may include a MIP flag and ISP related information. For example, a MIP flag may represent the intra_mip_flag syntax element. Additionally, for example, an ISP flag may represent an intra_subpartitions_mode_flag syntax element, and an ISP segmentation flag may represent an intra_subpartitions_split_flag syntax element.

[291] For example, the associated residual information may include low frequency inseparable transform (LFNST) index information representing information regarding the inseparable transform for the low frequency transform coefficients of the current block. Additionally, for example, the associated residual information may include LFNST index information based on a MIP flag or a current block size. Additionally, for example, the associated residual information may include LFNST index information based on a MIP flag or information regarding the current block. Here, the information regarding the current block may include at least one of the size of the current block, tree structure information representing a single tree or a double tree, an LFNST activation flag, or ISP related information. For example, the MIP flag may be one of a plurality of conditions for determining whether or not the associated residual information includes LFNST index information, and through other conditions such as the size of the current block, in addition to the MIP flag, the associated residual information may include LFNST index information. However, later in this document, an explanation is provided for the MIP flag. Here, the LFNST index information may be represented as transformation index information. Additionally, the LFNST index information may be represented as a st_idx syntax element or an lfnst_idx syntax element.

[292] For example, the associated residual information may include LFNST index information based on a MIP flag representing that MIP is not applied. Additionally, for example, the associated residual information may not include LFNST index information based on a MIP flag representing that MIP is applied. Thus, in the case where the MIP flag represents that the MIP is applied to the current block (for example, in the case where the value of the intra_mip_flag syntax element is 1), the associated residual information may not include LFNST index information, and in the case where The MIP flag represents that the MIP is not applied to the current block (eg, in case the value of the intra_mip_flag syntax element is 0), the associated residual information may include LFNST index information.

[293] Additionally, for example, the associated residual information may include LFNST index information based on the MIP flag and ISP related information. For example, in case the MIP flag represents that the MIP is not applied to the current block (for example, in case the value of the intra_mip_flag syntax element is 0), the associated residual information may include LFNST index information with reference to the ISP related information (IntraSubPartitionsSplitType). Here, IntraSubPartitionsSplitType may represent that ISP is not applied (ISP_NO_SPLIT), ISP is applied in horizontal direction (ISP_HOR_SPLIT), or ISP is applied in vertical direction (ISP_VER_SPLIT), and it may be derived based on ISP flag or ISP segmentation flag.

[294] For example, since the MIP flag represents that the MIP is applied to the current block, the LFNST index information may be inferred or retrieved for use in case the associated residual information does not include LFNST index information, i. in case the LFNST index information is not signaled. For example, the LFNST index information may be derived based on at least one of the reference line index information for the current block, the intra prediction mode information of the current block, the current block size information, and the MIP flag.

[295] Additionally, for example, the LFNST index information may include an LFNST flag representing whether or not an inseparable transform is applied to the low-frequency transform coefficients of the current block, and/or a transform kernel index flag representing a transform kernel applied to the current block. , from possible variants of transformation kernels. Thus, while the LFNST index information may represent inseparable transform information for the low frequency transform coefficients of the current block based on one syntax element or one piece of information, it may also represent information based on two syntax elements or two pieces of information. For example, an LFNST flag may be represented as a st_flag syntax element or an lfnst_flag syntax element, and a transformation kernel index flag may be represented as a st_idx_flag syntax element, a st_kernel_flag syntax element, an lfnst_idx_flag syntax element, or an lfnst_kernel_flag syntax element. Here, a transform kernel index flag may be included in the LFNST index information based on the LFNST flag representing that a nonseparable transform is applied and the MIP flag representing that MIP is not applied. Thus, in case the LFNST flag represents that an inseparable transform is applied and the MIP flag represents that a MIP is applied, the LFNST index information may include a transform kernel index flag.

[296] For example, since the MIP flag represents that the MIP is applied to the current block, the LFNST flag and the transform kernel index flag can be inferred or extracted in case the associated residual information does not include the LFNST flag and the kernel index flag. transformations, i.e. The LFNST flag and the transform kernel index flag are not signaled. For example, the LFNST flag and the transform kernel index flag may be derived based on at least one of the reference line index information for the current block, the intra prediction mode information of the current block, the current block size information, and the MIP flag.

[297] For example, in the case where the associated residual information includes LFNST index information, the LFNST index information may be represented via binarization. For example, LFNST index information (e.g., st_idx syntax element or lfnst_idx syntax element) can be retrieved through truncated Rice (TR) binarization based on a MIP flag representing that MIP is not applied, and LFNST index information ( for example, the st_idx syntax element or the lfnst_idx syntax element) can be retrieved via fixed length (FL) code binarization based on a MIP flag representing that the MIP is applied. Thus, in case the MIP flag represents that the MIP does not apply to the current block (e.g., in the event that the intra_mip_flag syntax element is 0 or false), the LFNST index information (e.g., the st_idx syntax element or the syntax element element lfnst_idx) can be retrieved via TR-based binarization, and in case the MIP flag represents that the MIP is applied to the current block (for example, in case the intra_mip_flag syntax element is equal to 1 or is "true"), LFNST index information (eg, st_idx syntax element or lfnst_idx syntax element) can be retrieved via FL-based binarization.

[298] Additionally, for example, if the associated residual information includes LFNST index information, and the LFNST index information includes a LFNST flag and a transform core index flag, the LFNST flag and the transform core index flag can be retrieved via the transform. to binary form with a fixed length (FL) code.

[299] For example, the LFNST index information can extract the options through the above-described binarization, can compare the bins represented by parsing or decoding the bitstream with the options, and thereby the LFNST index information can be obtained.

[300] For example, the (first) permission element from the string of permission elements of the LFNST flag may be derived based on the context encoding, and the context encoding may be performed based on the increment value of the context index for the LFNST flag. Here, context encoding is encoding performed based on a context model and may be referred to as "regular encoding". Additionally, the context model may be represented by the context index ctsIdx, and the context index may be derived based on the increment ctxInc of the context index and the offset ctxIdxOffset of the context index. For example, the context index increment value may be retrieved as one of the options including 0 and 1. For example, the context index increment value may be derived based on an MTS index (e.g., the mts_idx syntax element or the tu_mts_idx syntax element) representing a set of cores. a transformation to be used for the current block, from sets of transformation kernels, and tree type information representing the segmentation structure of the current block. Here, the tree type information may represent a single tree representing that the segmentation structures of the luminance signal component and the chrominance signal component of the current block are equal to each other, or a dual tree representing that the segmentation structures of the luminance signal component and the chrominance signal component of the current block are different from each other. from friend.

[301] For example, the (first) ROI from the ROI string of the transform kernel index flag can be extracted based on the bypass encoding. Here, bypass coding may represent that context coding is performed based on a regular probability distribution, and coding efficiency may be improved by omitting the context coding update procedure.

[302] The decoding equipment may derive transform coefficients for the current block based on the associated residual information (S1310). For example, the associated residual information may include information regarding the quantized transform coefficients, and the decoding equipment may derive quantized transform coefficients for the current block based on the information regarding the quantized transform coefficients. For example, the decoding equipment may derive transform coefficients for the current block by performing dequantization of the quantized transform coefficients.

[303] The decoding equipment may generate residual samples of the current block based on the transform coefficients (S1320). For example, the decoding equipment may generate residual samples from the transform coefficients based on the LFNST index information. For example, in case LFNST index information is included in the associated residual information, or LFNST index information is inferred or extracted, LFNST may be performed on transform coefficients according to the LFNST index information, and modified transform coefficients may be extracted. Thereafter, the decoding equipment may generate residual samples based on the modified transform coefficients. Additionally, for example, in the event that LFNST index information is not included in the associated residual information, or LFNST appears not to be performed, the decoding equipment may not perform LFNST on the transform coefficients, but may generate residual samples based on the transform coefficients.

[304] Although not illustrated in FIG. 13, for example, the decoding hardware may generate reconstructed samples based on the predictive samples and the residual samples. Additionally, for example, the reconstructed block and the reconstructed frame may be derived based on the reconstructed samples.

[305] For example, the decoding equipment can obtain image information including all or parts of the above-described pieces of information (or syntax elements) by decoding the bit stream or encoded information. Additionally, the bitstream or encoded information may be stored in a computer-readable storage medium and may cause the decoding method described above to be performed.

[306] Although the methods are described based on a flowchart in which steps or blocks are listed in sequence in the above-described embodiments, the steps of this document are not limited to a certain order, and a certain step may be performed in a different step or in a different order, or simultaneously. regarding what is described above. Additionally, those skilled in the art will appreciate that the steps in the flowchart are not the only ones possible, and another step may be included, or one or more steps in the flowchart may be removed without affecting the scope of the present disclosure.

[307] The above method according to the present disclosure may be implemented in the form of software, and the encoding equipment and/or decoding equipment according to the present disclosure may be included in an apparatus for performing image processing, such as a television, computer, smartphone, set-top box, display device, and etc.

[308] When embodiments of the present disclosure are implemented by software, the above method may be implemented by a module (process or function) that performs the above function. The module may be stored in a storage device and executed by a processor. The storage device may be installed inside or outside the processor and may be connected to the processor through various known means. The processor may include an application specific integrated circuit (ASIC), other chipsets, logic, and/or a data processing device. The storage device may include Read Only Memory (ROM), Random Access Memory (RAM), flash memory, a memory card, a storage medium, and/or other storage device. In other words, embodiments according to the present disclosure may be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional modules illustrated in the respective drawings may be implemented and executed on a computer, processor, microprocessor, controller, or chip. In this case, implementation information (eg, information regarding instructions) or algorithms may be stored on a digital storage medium.

[309] In addition, the decoding equipment and encoding equipment to which the embodiment(s) of the present document applies may be included in a broadcast multimedia transceiver, a mobile communication terminal, a home theater system video device, a digital cinema system video device, a surveillance camera, video chatting device and real-time communication device, such as video communication, mobile streaming device, storage media, video recording camera, video-on-demand (VoD) service provider, video device over networks (OTT), service provider Internet streaming device, 3D video device, virtual reality (VR) style device, augmented reality (AR) style device, telephone video device, in-vehicle terminal (e.g., in-vehicle terminal (including autonomous vehicle), air terminal udne or terminal in a sea vessel) and a medical video device; and may be used to process the image signal or data. For example, an OTT video device may include a game console, a Blu-ray player, an Internet TV, a home theater system, a smartphone, a tablet PC, and a digital video recorder (DVR).

[310] In addition, the processing method to which the embodiment(s) of the present document is applied may be formed in the form of a program executable by a computer and may be stored in a computer-readable recording medium. The multimedia data having the data structure according to the embodiment(s) of the present document may also be stored in a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices that store computer-readable data. The computer-readable recording medium may include, for example, a Blu-ray Disc (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical device. data storage. The computer-readable recording medium also includes media implemented in the form of a carrier wave (eg, transmission over the Internet). In addition, the bit stream generated by the encoding method may be stored in a computer-readable recording medium or transmitted via a wired or wireless communication network.

[311] In addition, the embodiment(s) of the present document may be implemented as a computer program product based on the program code, and the program code may be executed on the computer according to the embodiment(s) of the present document. The program code may be stored on a computer-readable medium.

[312] FIG. 15 shows an example of a content streaming system to which an embodiment of the present document may be applied.

[313] Referring to FIG. 15, the content streaming system to which the embodiments of the present document apply may generally include an encoding server, a streaming server, a web server, a media storage, a user device, and a media input device.

[314] The encoding server functions to compress content input from media input devices such as a smartphone, camera, camcorder, and the like into digital data to generate a bitstream and transmit it to the streaming server. As another example, in the case where a media input device such as a smartphone, a camera, a camcorder or the like directly generates a bitstream, the encoding server may be omitted.

[315] The bit stream may be generated by an encoding method or a bit stream generation method to which the embodiments of the present document apply. In addition, the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[316] The streaming server transmits media data to the user equipment based on a user request via a web server that functions as a tool that informs the user as to which service is provided. When the user requests a service that the user wants, the web server passes the request to the streaming server, and the streaming server transmits the media data to the user. In this regard, the content streaming system may include a separate control server, in which case, the control server functions to manage commands/responses between respective equipments in the content streaming system.

[317] The streaming server may receive content from a media store and/or an encoding server. For example, in case the content is received from the encoding server, the content may be received in real time. In this case, the streaming server may keep the bitstream for a predetermined period of time to smoothly provide the streaming service.

[318] For example, the subscriber equipment may include a mobile phone, a smartphone, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable media player (PMP), a navigation device, a stylus tablet PC, a tablet PC, an ultrabook, a wearable device (e.g., watch terminal (smart watch), glasses terminal (smart glasses), head-mounted display (HMD)), digital TV, desktop computer, digital signage system, etc.

[319] Each of the servers in the content streaming system can operate as a distributed server, in which case, data received by each server can be processed in a distributed manner.

[320] the Claims in the present description can be combined in various ways. For example, the technical features in the method claims of the present disclosure can be combined with the ability to be implemented or performed in equipment, and the technical features in the equipment claims can be combined with the ability to be implemented or performed in the method. Additionally, the technical features in the method claim(s) and the equipment claim(s) can be combined with the ability to be implemented or performed in the equipment. Additionally, the technical features in the method claim(s) and the equipment claim(s) can be combined with the ability to be implemented or performed in the method.

Claims (60)

1. An image decoding method carried out by means of decoding equipment, the method comprising the steps of: - receive information type intra prediction and associated residual information for the current block from the bit stream; extracting transform coefficients for the current block based on information regarding quantized transform coefficients included in the associated residual information; and - form the residual samples of the current block based on the transformation coefficients, wherein the intra prediction type information includes a matrix intra prediction (MIP) flag representing whether or not MIP is applied to the current block, wherein the associated residual information includes low-frequency inseparable transform (LFNST) index information for the current block based on the MIP flag, while the residual samples are formed from the transformation coefficients based on the information of the LFNST index, wherein the parsing order of the MIP flag precedes the parsing order of the LFNST index information, wherein, based on the MIP flag value other than 1, the associated residual information includes LFNST index information and wherein the LFNST index information is configured in a coding unit syntax based on the MIP flag. 2. The image decoding method of claim 1, wherein the intra prediction type information further includes intra-sub-segment (ISP) mode-related information regarding the sub-segmentation of the current block, and wherein the associated residual information includes LFNST index information based on the MIP flag and ISP related information. 3. The image decoding method according to claim 1, wherein the LFNST index information is extracted through Truncated Rice (TR) binarization. 4. The image decoding method according to claim 1, wherein the LFNST index information comprises a LFNST flag representing whether or not an inseparable transform is applied to the low-frequency transform coefficients of the current block, and a transform kernel index flag representing a transform kernel applied to the current block , from possible variants of transformation kernels. 5. The image decoding method of claim 4, wherein a transformation kernel index flag is included in the LFNST index information based on an LFNST flag representing that a nonseparable transformation is applied and a MIP flag representing that MIP is not applied. 6. The image decoding method according to claim 4, wherein the LFNST flag and the transform kernel index flag are extracted via fixed length (FL) code binarization. 7. The image decoding method according to claim 4, wherein the first bin from the string of LFNST flag bins is extracted based on the context encoding, the context encoding is performed based on the context index increment value of the LFNST flag, and the context index increment value is extracted as one of the possible options, including 0 and 1, and wherein the first permission element from the string of permission elements of the transformation kernel index flag is extracted based on the bypass encoding. 8. The image decoding method of claim 7, wherein the context index increment value is derived based on a multiple transform selection (MTS) index representing a set of transform kernels to be used for the current block, from the sets of transform kernels, and tree type information representing segmentation structure of the current block. 9. An image encoding method carried out by encoding equipment, the method comprising the steps of: - determine the type of internal prediction for the current block; generating intra prediction type information for the current block based on the intra prediction type; extracting prediction samples of the current block based on the type of intra prediction; - form the residual samples of the current block based on the predictive samples; extracting transform coefficients for the current block based on the residual samples; - form the associated residual information based on the transformation coefficients and - encode the intra prediction type information and the associated residual information, wherein the intra prediction type information includes a matrix intra prediction (MIP) flag representing whether or not MIP is applied to the current block, wherein the associated residual information includes low-frequency inseparable transform (LFNST) index information representing information regarding the inseparable transform of the low-frequency transform coefficients of the current block based on the MIP flag, wherein the LFNST index information is generated based on the transform kernel used to extract the transform coefficients, wherein the parsing order of the MIP flag precedes the parsing order of the LFNST index information, wherein, based on the MIP flag value other than 1, the associated residual information includes LFNST index information and wherein the LFNST index information is configured in a coding unit syntax based on the MIP flag. 10. The image decoding method of claim 1, wherein the associated residual information includes LFNST index information further based on an LFNST activation flag representing that the LFNST is being activated, and wherein the LFNST index information is configured in the coding unit syntax further based on the LFNST activation flag. 11. The image decoding method of claim 1, wherein the associated residual information includes LFNST index information further based on a minimum value of a current block width and height equal to or greater than 4, and wherein the LFNST index information is configured in the coding unit syntax further based on the minimum value. 12. A non-volatile computer-readable storage medium that stores a bit stream generated by the encoding method, the method comprising: - determining the type of internal prediction for the current block; - generating information of the type of intra prediction for the current block based on the type of intra prediction; - extracting prediction samples of the current block based on the type of internal prediction; - formation of residual samples of the current block based on predictive samples; - extracting the transformation coefficients for the current block based on the residual samples; - the formation of related residual information based on the transformation coefficients and - encoding information such as internal prediction and associated residual information, wherein the intra prediction type information includes a matrix intra prediction (MIP) flag representing whether or not MIP is applied to the current block, wherein the associated residual information includes low-frequency inseparable transform (LFNST) index information representing information regarding the inseparable transform of the low-frequency transform coefficients of the current block based on the MIP flag, wherein the LFNST index information is generated based on the transform kernel used to extract the transform coefficients, wherein the parsing order of the MIP flag precedes the parsing order of the LFNST index information, wherein, based on the MIP flag value other than 1, the associated residual information includes LFNST index information and wherein the LFNST index information is configured in a coding unit syntax based on the MIP flag. 13. A method for transmitting data for an image, the method comprising the steps of: - a bit stream for the image is obtained, the bit stream is generated by performing intra prediction type determination for the current block, generating intra prediction type information for the current block based on the intra prediction type, extracting prediction samples of the current block based on the intra prediction type, generating residual samples of the current block based on the prediction samples, extracting transform coefficients for the current block based on the residual samples, generating associated residual information based on the transform coefficients, and encoding intra prediction type information and associated residual information to generate a bit stream; and - transmit data containing a stream of bits, wherein the intra prediction type information includes a matrix intra prediction (MIP) flag representing whether or not MIP is applied to the current block, wherein the associated residual information includes low-frequency inseparable transform (LFNST) index information representing information regarding the inseparable transform of the low-frequency transform coefficients of the current block based on the MIP flag, wherein the LFNST index information is generated based on the transform kernel used to extract the transform coefficients, wherein the parsing order of the MIP flag precedes the parsing order of the LFNST index information, wherein, based on the MIP flag value other than 1, the associated residual information includes LFNST index information and wherein the LFNST index information is configured in a coding unit syntax based on the MIP flag.

Images